Evaluation of 3-year tuberculosis external quality assessment results of public health laboratories

INTRODUCTION

Tuberculosis (TB) maintains its importance as a public health threat. Since the definitive diagnosis is made bacteriologically, it is necessary to assess the accuracy and reliability of the test results provided by laboratories. External quality assessment (EQA) is a major tool that ensures the reliability of laboratory studies and are done by two different approaches that complement each other, internal and external quality control. Internal quality control (IQC) is used for daily monitoring of accuracy and repeatability. IQC studies are performed before patient samples are processed and, if necessary, at regular intervals throughout the day. Patient samples should be studied after control results are within acceptable limits.^1–4

On the other hand, EQA is the process of verifying laboratory test results by an impartial, external organization. In this process, a laboratory compares its results with similar laboratories and has information about its analytical performance. In addition, false results reported as a result of quality control studies provide benefits in terms of identifying and correcting the incorrect operations.^5,6

In Turkey, the sixteenth article of the Directive on the Service Units and Duties of the General Directorate of Public Health (GDPH) defines the job descriptions of the Department of Microbiology Reference Laboratories and Biological Products (DMRLBP). According to article e of this directive, ‘To lead the preparation of laboratory and field guides at the national level, to prepare national microbiology standards for laboratory diagnostic standardization in the field of microbiology, and to carry out external quality control studies’ are some of the duties of the National Tuberculosis Reference Laboratory (NTRL).⁷

We evaluated the 3-year participation status of TB laboratories within the public health laboratories (PHLs) in the nationally organized EQA and the EQA results.

METHODS

There are 84 PHLs in 81 provinces across Turkey, 3 in Istanbul, 2 in Antalya, and 1 each in other provinces. These laboratories are healthcare facilities providing clinical and non-clinical services. TB laboratory services are classified as Level-1, -2 and -3 according to the provinces’ capacity of device, biosafety conditions, equipment, personnel and microbiology specialists.^8,9 Level-1 laboratories use only microscopic techniques, Level-2 laboratories do microscopy and culture, and Level-3 laboratories also do drug susceptibility tests (DST). The strains grown in culture in Level-2 and -3 TB laboratories in the TB network within the scope of Tuberculosis Laboratories Surveillance Network (TULSA) are sent to the NTRL and identified by advanced techniques.

The NTRL yearly (in the second half) organizes the EQA programme according to the ISO 17043 standard. Almost all TB laboratories participate in the EQA programme. PHLs participate in the EQA programme organized by the NTRL every year on three parameters: microscopy, culture and phenotypic first-line drug susceptibility tests.

The samples to use in the EQA programme are prepared from patient samples and standard strains known susceptibility results (ATCC) and EQA samples. The sample set number to be prepared is determined by considering the number of participants in the EQA programme in the previous year and the applications after the announcement. The prepared EQA samples are sent to participating laboratories. Five smear samples (acid-fast bacilli [AFB] positive [with grade] and AFB negative) are prepared for microscopy. In the preparation of negative smears, the samples with negative test results (for microscopy, culture and polymerase chain reaction [PCR]) are pooled, and in the preparation of positive smear, the patient samples with positive test results (AFB, culture and PCR) are pooled in our laboratory. During preparation of each smear, the sample mixtures are thoroughly vortexed to ensure homogeneity, and the prepared smears are dried and fixed under appropriate conditions.

For culture, the distribution of positive and negative samples consisting of Mycobacterium tuberculosis complex (MTBC) and non-tuberculosis mycobacteria (NTM) are provided. For first-line DST, five samples are prepared from MTBC isolates consisting of susceptible and resistant strains. These samples are obtained from fresh MTBC cultures (up to 4 weeks old). First, 50 ml falcon tubes containing glass beads are added to 10 ml sterile distilled water or sterile saline. Then, a large number of colonies are picked from the fresh cultures and placed in the falcon tubes and the bacteria are vortexed homogeneously via glass beads and vortex. After waiting for 30 minutes, 1.5 ml of this mixture is distributed into 2 ml plastic tubes; the tubes are tightly capped and stored in the deep freezer until packaging.

After the EQA samples are prepared, 10% of the smears are randomly selected for microscopy. After that, homogeneity is checked in terms of preparation technique and accuracy of the results. At least two sets are taken randomly from the samples for culture and DST. Tests are carried out by both the methods used in the NTRL and the same/different methods used in other laboratories and the results are evaluated. In addition, to control the shipping conditions, two sets of each parameter are kept in room conditions for 5 days, and at the end of this period, the samples are evaluated in terms of stability controls by performing tests according to the protocol. The samples controlled for homogeneity and stability are sent to the laboratories in triple packaging systems following biosafety rules. Laboratories are requested to test the EQA samples together with routine patient samples and use the routinely used method. Laboratories send the results of the EQA samples to the NTRL via the TULSA web system according to the determined result reporting dates. The NTRL collects results from participants and performs an analysis involving all laboratories. The results obtained from laboratories are analysed according to the determined reference results. The laboratory results compared with the expected values are scored, and if the score is 80% and above, the laboratory is considered to have passed on that parameter. As a result of the analysis, evaluation reports are prepared. Evaluation reports are also sent to the laboratories via the TULSA web system.^6,10–12 Results obtained from the NTRL, which has ISO 15189 accreditation certificate for microscopy, culture and DST were used as a gold standard. Ehrlich–Ziehl–Neelsen (EZN) staining method, a type of acid-fast stain, culture using solid (LJ slant) and liquid medium (BACTEC MGIT 960 system) and also DSTs were performed by the BACTEC MGIT 960 system. All parameters used in the NTRL were validated by an external quality control programme. Microsoft Office Excel was used in the calculation of number and percentage distribution.

RESULTS

The 19 Level-2 laboratories are located in the following provinces: Ankara, Aydin, Denizli, Diyarbakir, Edirne, Eskisehir, Izmir, Karabük, Kirikkale, Kocaeli, Mugla, Manisa, Ordu, Sivas, Tekirdag, Trabzon, Usak, Yozgat and Van. The 5 Level-3 TB laboratories serving within the PHLs are located in the following provinces: Antalya, Erzurum, Bursa, Samsun and Istanbul.

All these laboratories are adequate in terms of physical infrastructure, biosafety conditions and equipment, and tests are carried out under the supervision of a microbiologist. Technical personnel and microbiology specialists working in TB laboratories are constantly included in the in-service TB training given within TULSA’s scope and carry out their work in coordination with the NTRL.

A total of 24 TB laboratories in 2018, 30 in 2019 and 23 in 2020 participated in the EQA (Fig. 1). The test results reported by the laboratories were evaluated by scoring based on the analyses done by the NTRL.

Close

FIG 1.

Public health laboratories participation status in the tuberculosis external quality assessment programme

Export to PPT

In 2018, 3 Level-1, 16 Level-2, 5 Level-3; in 2019, 6 Level-1, 19 Level-2, 5 Level-3; in 2020, 3 Level-1, 15 Level-2 and 5 Level-3 laboratories participated in the EQA. These laboratories’ participation and EQA results in 2018, 2019 and 2020 are shown in Figs 2–4, respectively.

Close

FIG 2.

Qualification rate of participating public health laboratories in 2018

Export to PPT

Close

FIG 3.

Qualification rate of participating public health laboratories in 2019

Export to PPT

Close

FIG 4.

Qualification rate of participating public health laboratories in 2020

Export to PPT

In 2018, 20 of the laboratories participating in the EQA achieved 100% and four of them 80% success in terms of sensitivity, specificity, accuracy and reproducibility in microscopy. On the other hand, in culture, 16 laboratories achieved 100% success and 2 laboratories achieved 80% success. Therefore, there is no laboratory with a success rate below 80% for all three parameters (Fig. 2).

In 2019, 28 of the participating laboratories in the EQA showed 100% success and 2 had 80% success in terms of sensitivity, specificity, accuracy and reproducibility for microscopy. For culture, 11 laboratories had 100% success, 6 laboratories had 80% success, and 1 laboratory had 60% success.

In 2020, 20 laboratories achieved a success rate of 100% and 3 of 80% in terms of sensitivity, specificity, accuracy and reproducibility for microscopy. Thirteen laboratories achieved 100% success and 3 laboratories achieved 80% success for culture.

When a 3-year evaluation for DSTs was done, 3 of 5 laboratories achieved 100% success in 2018 in terms of sensitivity, specificity, accuracy and reproducibility for streptomycin (SM), isoniazid (INH), rifampicin (RIF) and ethambutol (EMB). For 2 laboratories, the sensitivity for SM was 100%, while the accuracy was 80%. In 2019 and 2020, the success rate of all 5 laboratories for SM, INH, RIF and EMB was 100%.

DISCUSSION

Accurate, reliable and timely diagnosis of TB can be made by TB laboratories established in accordance with national and international standards. In Turkey, the Ministry of Health Tuberculosis Diagnosis and Treatment Guide was published in 2011. The ‘Communiqué on Working Procedures and Principles of the National Tuberculosis Diagnostic Laboratories Network’ was published in the Official Gazette dated 22 February 2012, and came into force. This Communiqué was updated and republished in the Official Gazette dated 25 October 2015.¹³ TB laboratories, which are authorized according to this Communiqué, have developed and are still developing their current capacity by entering into structuring according to the determined standards. In addition, the national laboratory standards were published in 2014 as the ‘National Microbiology Standards National Tuberculosis Diagnostic Guide’ (NTDG)¹⁴ as a ‘Project for the Surveillance and Control of Infectious Diseases (TR0802).¹⁵ This guide has been an advisor for TB laboratories. In addition, it has been the aim to teach these standards and spread proper practices through the training given to expert and technical personnel at the national level.

TB laboratories in PHLs in Turkey constantly improve themselves in terms of personnel, infrastructure, equipment and training in coordination with central units. The structuring of TB diagnostic laboratories in PHLs in 2014 accelerated these improvement efforts. PHLs try to ensure that practices are carried out following quality standards. Participation in the EQA programmes is also an essential part of this effort. Participation in EQAs provides valuable information for laboratories and helps to compare their performance with different laboratories.^15,16 In addition, it provides added value in areas such as early warning in the detection of systematic problems affecting the test result, providing objective evidence for test quality, identifying deficient areas or areas that need improvement, if any, and identifying training needs.

When we look at the EQA studies conducted worldwide, in 2009, Jou et al. reported an EQA study in Taiwan that included 9 participating laboratories in the first round and 30 participating laboratories in the second round. The average accuracy of detection of resistance was 91.6% for INH, 96.1% for RIF, 90.5% for EMB and 93.9% for SM in the first round. In the second round, it was 95.7% for INH, 97.2% for RIF, 82% for EMB and 86.8% for SM. The majority of participating laboratories showed improvement in the second EQA. Therefore, continuity is essential to improve the test quality of EQA.¹⁷

In an EQA reported by Van Deun et al. in 2011, 27 laboratories in the supranational TB reference laboratory network participated in the EQA programme. Within this 9-cycle EQA study, both sensitivity and specificity were >95% for INH, RMP and SM, and >80% for EMB. It has been reported that 16 participating laboratories consistently performed successfully in all tours.¹⁵

According to the results obtained from an EQA review conducted in China between 2014 and 2019, participation increased over the years in the six EQA cycles, and all participating laboratories performed successfully. In addition, it was emphasized that EQA studies should be combined with training and on-site evaluation visits.¹⁸

Our study reports the EQA participation of PHLs in Turkey, and the success rate in EQA is generally above 80%. In 2019, only 1 laboratory with a success rate of <80% was identified and this laboratory’s employees were included in the training course given within the TULSA scope. All test processes were reviewed by making an on-site evaluation visit, quality indicators were examined and the EQA success status in the following years continued to be monitored. EQA is an indispensable element in maintaining the accuracy and reliability of analytical methods used in laboratories in the long term. It provides important contributions to the development of the laboratory with further studies.