Performance evaluation of laboratory professionals on tuberculosis microscopy at Hawassa Town, Southern Ethiopia

Microscopic diagnosis of Ziehl-Neelsen stained sputum by microscopists has remained the best routine laboratory method for the diagnosis of tuberculosis (TB). However, detection and identification of TB require skilled laboratory personnel. The aim of the study was to assess the performance of laboratory professionals in detecting TB bacilli at Hawassa town health institutions. A cross-sectional study design was employed among a total of 81 laboratory professionals working in public and private health facilities. A standardized pre-validated slide panel and questionnaires were distributed to laboratory professionals. Agreement in detection of TB bacilli sensitivity, specificity and predictive values of readings were assessed using SPSS version 16.0. Among the 81 participant, 11(13.6%) correctly reported all panel slides, 70 (86.4%) missed at least one slides. A total of 29.75% (241/810) error was reported that include major errors of 2.22% (13 HFN; 5 HFP) and minor errors of 27.5% (25 LFN; 60 LFP and138 QE). The sensitivity and specificity of participants in detecting TB bacilli as compared to the reference reading were 91.97, 80.00, 87.30 and 86.92%, respectively. Overall agreement of participants with the reference reading on TB detection was 95.18% (Kappa = 0.73). Agreement of the participants with reference reading in the detection of TB bacilli was good. Even though the study revealed only 2.22% major error, the laboratory professionals need continuous supervision and remedial actions on time for successful TB control programs.


INTRODUCTION
According to WHO tuberculosis (TB) report 2011, Ethiopia ranks 7 th in the list of the world"s 22 high burden countries for TB with incidence estimated at 379/100,000 for all forms of TB and 168/100,000 for smear positive tuberculosis (Boulahbal et al., 1976).Direct sputum smear microscopy remains the most cost effective tool for diagnosing patients with infectious tuberculosis and for monitoring progress of treatment although the limited diagnostic capacity for TB in the country remains a challenge to improving case detection rate (World Health Organization, 2011;Ethiopian Health and Nutrition Research Institute, 2008).
Case detection through quality assured laboratories is an essential element of the WHO STOP TB Strategy (Ethiopian Health and Nutrition Research Institute, 2008;World Health Organization, 2006).The WHO for tuberculosis control (DOTS) relies on a network of laboratories that provide acid fast bacilli (AFB) sputum smear microscopy.But, if the laboratory diagnosis is unreliable, all other activities will be affected.It is known that microscopy errors are likely to result in failure to detect persons with infectious TB who will then continue to spread infection in the community, or unnecessary treatment for non-cases.On the other hand, errors in reading follow up smears can result in patients being placed on prolonged treatment, or in treatment discontinued prematurely.Therefore, quality assurance of AFB sputum smear microscopy is essential to reduce such type of problems (Ethiopian Health and Nutrition Research Institute, 2009;World Health Organization, 2011).
Quality of AFB smear microscopy is dependent on national programs that monitor and perform proficiency testing of individual laboratories (Jakes and Joyce, 2001).Proficiency testing is a system in which "reference material" of known but undisclosed content are introduced into the laboratory and examined by the staff using the same procedures as would normally be used to examine patients" specimen of the same type (Ethiopian Health and Nutrition Research Institute, 2008;Jakes and Joyce, 2001).
This study intends to assess the proficiency level in detecting Mycobacterium tuberculosis from sputum smears by the microscopists at Hawassa health institution laboratories.

MATERIALS AND METHODS
A prospective cross-sectional study was conducted to evaluate the quality of TB smear microscopic examination from April 23 to June 26 2012.The study was conducted at Hawassa City: two governmental hospital, two private hospital, two governmental health center and one non-governmental health institution laboratories.The study involves 81 laboratory professionals, all laboratories were provided TB microscopy on daily basis.The professionals were selected based on convenient availability at time of data collection.

Panel slide preparation and distribution
Experts who have been qualified and certified on TB microscopy at the Hawassa regional laboratory had prepared and validated the panel slides.Both TB positive and TB negative sputum were used.Further concentration as well as dilution of bacilli was done after bleach concentration techniques.Experts interpreted the prepared AFB smears using investigative criteria for the presence or absence of TB bacilli and also quantification of bacilli number.
For this study, ten slides per set, covering the full range from negative to strongly positive were used according to WHO manual for preparation of proficiency test slides.The composition of test panels standardized according to WHO manual and Ethiopian Health & Nutrition Research Institutes guide line (National Tuberculosis and Leprosy Control Programme, 1999;Ethiopian Health and Nutrition Research Institute, 2009).Each panel of slides include four with negative slide and six with different bacterial density (3 with 1-9 AFB/100 fields (trace), one with 1 + , one with 2 + and one with 3 + ).The total number of slides per panel was ten.Groups of standardized panels with respect to the characteristics of the positive (mycobacterium and level of bacteria) as well as negative slides were used so that the results of the assessment by different laboratories could be compared.Next to preparation and validation, the slides were arranged in ten sets and then packed for distribution to the participant.The reporting formats and orders of how to perform the tests were packed separately.
A structured questionnaire including information on the participating facilities and professionals was distributed.The questionnaire includes the socio-demographic characteristics, educational background and service of the professionals.
The result of TB diagnosis reported by the participants was evaluated using different parameter.Sensitivity was determined as the ability of participants to diagnose positive TB slides whereas; specificity was calculated for their ability to diagnose negative TB slides.Inter-rater agreement is the degree of agreement between participants and expertise reader.It is calculated by computing the sum of true positives and true negatives and then divided by the total.
"Major error" was defined as incorrect diagnosis of TB, that is, reporting "negative" in the case of a sample with TB bacilli (HFN); and falsely reporting "positive" in the absence of TB bacilli in the sample (HFP)."Minor error" was defined as quantification error(QE) and reporting negative with the correct report was 1-9AFB/100 fields and vice versa (Table 1).The distinction between a minor and a major error was based on the effect the error could potentially have on the patient's diagnosis and clinical management (2, 6 Ethiopian Health andNutrition Research Institute, 2008 and2009).
Regarding the score, in a set of 10 panel testing slides, each slide carries 10 points, total possible score of 100.Committing major error (HFP and HFN) result in a score of 0 where as minor error (LFP, LFN and QE (QE = 2 grades difference)) result in scores of 5 points.In general, assessment of performance was based on Table 1.Passing score was 80 points and poor performance was <80%.Interpretation of results was done based on International Union against Tuberculosis and Lung Disease (IUATLD) WHO recommended grading of sputum smear microscopy results using World Health Organization (2002).
The laboratory personals in the selected laboratory reported the results of examined panel slides along with the principal investigator as compared to the results against the expertise reading.Data was entered and analyzed using SPSS statistical soft ware version 16.0 at a statistical significance of p < 0.05 and 95% confidence intervals.Error rates of p ar t ic ip a n t s were statistic ally tested using logistic regression analysis.The level of agreement among various diagnostic levels was measured using Kappa.
The study was presented to the research committee of the department of Medical Laboratory Science and endorsed by the department commission then ethically cleared by the College Research and publication committee.Official letter was written to the participating facilities.In addition, participants had the right not to participate or to withdraw from the study at any time.

RESULTS
A total of 81 laboratory professionals responded to the questionnaires with a response rate of 100%.Forty eight  2).There were no problems with the functionality of microscopes and accessibility of reagents in any of the laboratories.
There was no statistically significant association between the proportion of errors made by the participants in the detection of TB bacilli and their sex, experience, number of TB case examined per day, inservice training and EQA involvement (Table 3).
Of 81 participants, 11 (13.6%)correctly interpreted all panel slides, 19 (23.5%) made two incorrect reading out of the ten panel slides and 70 (86.4%)committed at least one error among 10 slides.Fifty one (63.0%)participants correctly reported all four negative slides, 20 (24.7%) had at least one error among 4 negative slides and 10 (12.3%) misread all of the 4 negative slides.Eighteen (22.2%) of the participants reported all positive slides correctly, 22 (27.2%) had two error among 6 positive slides and 3 (3.7%) reported incorrectly five of the 6 positive slides (Figure 1).
Overall agreement of participants with the reference reading on TB detection was 95.18% (Kappa = 0.73).Comparison across institutions showed that agreement in detection was higher in NGO health institutions (95%) (Kappa = 0.89).The lowest agreement on detection was found among working in private hospital with an agreement of 80.68% with reference reading (Kappa = 0.59) (Table 4).

DISCUSSION
Sputum smear examination is the most important test to diagnose a person infected with pulmonary TB.
Although new diagnostic technologies were available, still microscopic examination of sputum smear was used in Ethiopia.Therefore, the skill of laboratory personals on AFB examination, seriously affect the case management of TB.Consequently, proficiency testing in sputum smear microscopy was essential for a successful TB control program (Ethiopian Health and Nutrition Research Institute, 2009).
Quality of AFB microscopy relies on national programs that support, train and monitor the testing performance of individual laboratories.It is well known that grave deficiencies can occur in the laboratory when insufficient attention is given to the quality of the work product.Many countries including Ethiopia, however, have no complete laboratory EQA program or do not provide sufficient administrative support and attention (John, 2002).
In this study, the overall sensitivity, specificity, PPV and NPV of participants in detecting TB bacilli were 91.97, 80.00, 87.30 and 86.92%, respectively.These findings were in agreement with a study conducted in northern Ethiopia, and elsewhere in the world (Mekete et al., 2011;Mundy et al., 2002;Rieder et al., 1997;Nguyen et al., 1999;Boulahbal et al., 1976;Selvakumar et al., 2005;Fadizilah et al., 2010).Low in detection of TB bacilli indicates that there were many false positive results, that is, false diagnoses of uninfected individual.This can lead to exposure and wastage of treatment, unnecessary care as well as financial lose.
Our finding of an overall agreement on detection of TB bacilli with expertise reading was 95.18% (kappa = 0.73) which is defined as good agreement based on the Kappa index interpretation (Landis and Koch, 1977).It is known that sputum microscopy never reach 100% agreement in reading smear even among expertise (World Health Organization, 1998).The overall agreement in the current study was almost similar with findings with external quality assessment of national public health laboratories in Africa 2002-2009(Mundy et al., 2002) also with study in Ethiopia (Mekete et al., 2011;Estifanos et al., 2005), 98% (Kappa = 0.85) and 88.2% in Malaysia (Fadzilah et al., 2013).
In the present study, major error were reported by 18 (2.22%) of participants, among these, 5 (0.62 %) were HFP.Even though the reading is lower than the national Hailemariam et al. 1137 critical cut-off point of 2% set by the NTLCP (National Tuberculosis and Leprosy Control Programme,1999), the possible reason could be due to lack of knowledge in identifying AFB, work overload and carelessness of technicians in reading of smears.Similar finding was also reported from different study (Estifanos et al., 2005;Nguyen et al., 1999).On the other hand, HFN 13 (1.6%)result was also below the national threshold for corrective action (1.6 vs. 5%) (National Tuberculosis and Leprosy Control Programme, 1999).HFN indicate misclassifications that fundamentally change the disease classifycation and management of a patient.The main drive of an EQA programme is to identify HFN which suggests that patients with TB case were not treated on time because of diagnosis error.HFN result in suffering due to the disease; further spread of TB and elevated transmitted rate in general population, in addition, patients may lose confidence in the health services or a particular laboratory.This finding was similar with Malaysia (Fadzilah et al., 2013).Even though the major error in relative term was little, but it does not mean the finding was insignificant; as the problem was major public health important in the area, further attention need to be paid (World Health Organization, 2002).Especially, HFN results were serious because positive persons went to the community without treatment so that it will increase the transmission of the disease drastically.
In this study, 27.5% minor error (LFP 7.4%, LFN 3.08% and QE 17.03%) rates were reported.This suggested that microscopist fail to detect slides with low AFB count (Trace and 1+ slide).Because of this, the dominant errors committed among technicians were QE and LFP.Even though quantification errors are minor importance, it does not influence case management.But this type of error can distinguish performances of laboratory personals.The possible reason for miss might be the laboratory personals not performing reading in all parts of the fields.On the other hand, as one of the minor error, QE of 17% in this study indicates possibility of microscopists not following the standard procedure for reading the smears.In the case of LFN, under-reading of number of AFB can give indication of problem areas in the diagnostic process.As a result, patients with bacillary disease may perhaps be misdiagnosed as negative results in AFB microscopy.In additions such type of error led to failures of WHOs strategy, so patients will not receive treatment on time, resulting in further community spread and failure in diagnosis of pulmonary TB.The consequence of LFP results in the beginning of treatment is unnecessary, also anti-tuberculosis drugs are wasted (John et al., 2012).
The limitation of this study is that we only used proficiency testing of slide reading using known panels to evaluate the skill of laboratory professionals under optimal conditions, rather than routine or day to day performance in the diagnosis of TB.Besides, we did not evaluate the performance of the laboratory personnel with regards to smearing, staining qualities and the post analytical prospects like documentation of data.Findings and interpretations from this study were only applicable to the microscopists in the study area.

Conclusion
Even though the study revealed only 2.22% major error, the overall 29.75% error made due to minor errors are a great concern for countries like Ethiopia where most of the suspected TB cases may have miss diagnosed at the onset of disease.The country now have improved health facilities and better awareness for patients with suspected TB to look for treatment early, however it might not be identified by sputum smear microscopy if such errors are not solved on time.

Table 1 .
Evaluation and interpretation of errors between expertise and participant technicians.
than 10 TB case per day.Fifty two (64.2%) of participant were BSc.degree holder and 37 (45.7%) were serving for less than two years.The males were 45 (55.6%), mean age of the participants was 28 (SD = 2.4) years (Table
respectively.Agreement with reference was 95.18% (Kappa = 0.73) on detection of TB bacilli.Assessment

Table 4 .
Overall sensitivity, specificity, predictive value and agreement of participants in detecting TB bacilli by health institution.

Table 5 .
Type of errors of participant in detecting TB bacilli by health institution, (N=81).