CDST_LT: External quality assurance

Ensuring accurate sputum microscopy results is paramount to maintain optimal sensitivity and specificity of sputum microscopy.

In this regard, External Quality Assurance (EQA) is an important aspect of the National TB Elimination Programme (NTEP) Laboratory Network. Different levels of EQA activities are undertaken to ensure that the system follows a standard protocol and remains efficient. The timely completion of EQA activities is important.

The EQA activities are interlinked and together (from National Reference Laboratory (NRL) to Tuberculosis Units (TUs)) give a proper output adequately ensuring accurate microscopic examination of sputum at the Designated Microscopy Centres (DMCs).

1. The EQA related activities of NRLs: This EQA ensures that the Intermediate Reference Laboratory (IRL) team maintains proficiency to carry on EQA of districts.
   1. On-site evaluation of State TB Training and Demonstration Center (STDC)/ Intermediate Reference Laboratory (IRL)
   2. Manufacture of panel testing slides and panel testing of STDC/ IRL lab staff
   3. Training of STDC supervisory staff in:
      • On-site evaluation of Senior TB Laboratory Supervisors (STLS)
      • Manufacture of panel slides
      • Assessment of blinded re-checking of DMC slides at District TB Center (DTC)
      • Facilitating the training of STLS for EQA
   4. Re-training of STDC/ IRL supervisory staff, if required
   5. Prompt reporting of the results of activities and feedback to State TB Officer/ Deputy Director General TB
2. The EQA related activities of STDC/ IRL labs: This is an ongoing process to train/retrain and maintain proficiencies of STLSs, District TB Officers (DTO) and Medical Officer - TB Control (MO-TC) to carry on EQA of DMCs.
   • Training of EQA for STLS, DTO, MO-TC
   • On-site evaluation of DTC labs and a sample DMCs
   • Manufacture of Panel testing slides and panel testing of DTC lab supervisors, including all STLS of the district
   • Re-training of DTC LT/ STLS, if required
   • Prompt reporting the results of activities by Director STDC/ IRL to STO, Central TB Division (CTD) & NRL
3. The EQA related activities of the DTC/ TB Unit: This is an ongoing monthly activity, to ensure proper proficiency of Laboratory Technicians (LTs) at DMCs to carry out sputum microscopy.
   • On-site evaluation of DMC labs
   • Unblinded re-checking of DMC slides at DMC
   • Random Blinded Re-checking (RBRC) of DMC slides at DTC
   • Prompt reporting of the results of activities to LT and Medical Officer of DMC as well as STDC/ IRL
   • Panel testing and re-training of DMC LTs, if required

Data collection, analysis and problem-solving are key components of this process.

Figure: Structure and Functions of EQA

Resources

• Training Module (1-4) for Program Managers and Medical Officer, NTEP, MoHFW, 2020.

Kindly provide your valuable feedback on the page to the link provided HERE

The External Quality Assessment (EQA) for Fluorescent Microscopy (FM) follows the same basic principles as for sputum smear microscopy using Ziehl-Neelsen (ZN) staining:

1. On-site Evaluation (OSE)
2. Panel Testing
3. Random Blinded Re-checking

Differences between EQA for FM and sputum smear stained by ZN method:

• The stained slides for FM are stored in the dark as fluorescence is prone to fading over time.
• The slides for FM are viewed under 40X without use of immersion oil.
• Acid-fast bacilli appear bright yellow against the dark background in FM.

During the OSE visit, the Senior Laboratory Supervisor (STLS) will cross check all positive smears and selected negative smears.

Resources

• Manual for Sputum Smear Fluorescence Microscopy.
• Module for Laboratory Technicians.

Kindly provide your valuable feedback on the page to the link provided HERE

One of the methods for internal monitoring of the quality of the microscopy process is the use of Quality Control (QC) slides.

QC is a systematic internal monitoring of working practices, technical procedures, equipment and materials, including quality of stains.

Quality Control Positive (QCP) and Quality Control Negative (QCN) slides are used for sputum smear microscopy quality monitoring.

Process for quality control using QCP and QCN is as follows:

• Senior Tuberculosis Laboratory Supervisor (STLS) prepares QCP & QCN slides.
• Preparation of QCP - QCP slides have to be prepared by pooling 3+ grade sputum samples.
• Preparation of QCN - QCN slides have to be prepared by pooling Negative sputum samples with an adequate number of pus cells (approximately 20 cells/field).
• While preparing each new batch of staining reagent, STLS has to use one set of QCP & QCN and its entry needs to be made in the batch register (an Internal Quality Control (IQC) document).
• STLS will supply one set of QCP & QCN to the Designated Microscopy Centre (DMC) Laboratory Technician (LT) along with each batch of reagent. The DMC LT has to stain and examine the QCP & QCN slides and enter the results in the IQC document.
• All QC slides needs to be stored for three months.

QC slides are used in the following ways​:

1. A set of one QCP and one QCN is used for monitoring the quality of a new batch of reagents prepared by the STLS​.
2. A set of one QCP and one QCN is used for a new batch of reagents prepared, by the LT at the Culture and Drug Sensitivity Testing (C&DST) laboratory​​.
3. A set of one QCP and one QCN is also supplied to DMCs along with the new batch of reagents for QC testing at microscopic centres.

​

The results of QCP and QCN examination are to be documented in separate registers meant for QC.

Any discordant results require discarding the reagents prepared/ supplied.

​

Resources

• Training Module (1-4) for Program Managers and Medical Officer, NTEP, MoHFW, 2020.   
• WHO: Regional Guidelines for Countries in the Western Pacific: Quality Assurance of Sputum Microscopy in DOTS Programs.  

Kindly provide your valuable feedback on the page to the link provided HERE

Panel testing is a method of External Quality Assurance (EQA) that is used to determine the adequacy of a laboratory technician to perform Acid-fast Bacillus (AFB) smear microscopy. This method evaluates individual performance in staining and reading and not the other laboratory activities.

• Panel testing under National TB Elimination Programme (NTEP) is used for Intermediate Reference Laboratories (IRLs) and District TB Centres (DTCs) during on-site evaluation, because these institutions do not have routine slides for blinded rechecking.
• A panel consists of a batch of stained and/or unstained smears that are sent out by the higher-level reference laboratory to the peripheral laboratories for processing, reading, and reporting of results
• Panel testing is not performed as a routine in DMCs, as they will have regular on-site evaluation and blinded rechecking.

Uses of Panel Testing

• Supplement re-checking programmes
• Provide information on the capabilities of the peripheral laboratories prior to
  implementing a re-checking program
• Assess status level of performance or to quickly detect problems associated with very poor performance
• Evaluate proficiency of laboratory technicians following training
• Monitor performance of individuals when adequate resources are not available to implement a re-checking program.

Resources

• RNTCP Laboratory Network Guidelines

Assessment

A field visit is an ideal way to obtain a realistic assessment of the conditions and skills practiced in the laboratory. Under National TB Elimination Programme (NTEP), On-site Evaluation (OSE) of Intermediate Reference Laboratories (IRLs) and District TB Centres (DTCs)/ Designated Microscopy Centres (DMCs) is therefore an essential component of a meaningful Quality Assurance (QA) programme.

OSE should be performed a tleast once a year by personnel from a higher-level laboratory (IRL/ National Reference Laboratory (NRL)) in order to evaluate the overall operational conditions in the microscopy centers.

Laboratory Supervisors must be knowledgeable in all operational and technical elements of AFB smear microscopy and have sufficient expertise to observe technicians performing routine tasks.

Importance of OSE

• Ensures that Standard Operating Procedures (SOPs) are in place and are displayed in all DMCs, internal Quality Check (QC) as per RNTCP is performed, and a functional binocular microscope is available
• Provides an opportunity for immediate problem-solving corrective action and on-site retraining.

Frequency of OSE (Figure)

• At least once a month visit by Senior TB Lab Supervisor (STLS) to the DMC, is required with re-checking of five positive and five negative slides.
• Regular on-site evaluation by District TB Officer (DTO) is important to assure recording and reporting of results; assessing operational conditions, safety, supplies, equipment and total workload.
• At least once a year visit by laboratory supervisors is recommended for IRLs by NRLs and for DTCs by IRLs.
• When poor performance has been identified through OSE, blinded rechecking or panel testing, additional visits by trained laboratory personnel from a higher-level laboratory (the IRL or NRL laboratory) supervisor are mandatory to perform a comprehensive evaluation of all laboratory procedures, implement corrective action, and provide training.

Components of OSE (Figure)

The visit includes:

• Comprehensive assessment of laboratory safety including infection control measures; conditions of equipment, adequacy of supplies as well as the technical components of Acid-fast Bacillus (AFB) smear microscopy. Sufficient time must be allotted for the visit to include observation of all the work associated with AFB smear microscopy, including preparing smears, staining and reading of smears.
• Examining a few stained positive and negative smears to observe the quality of smearing and staining as well as condition of the microscope
• Facilitating quality improvement through on the spot problem solving and suggestions for corrective action, wherever needed.

The NRLs provide training to all IRL personnel responsible for OSE. Additionally, non-laboratory personnel (e.g., DTOs) should acquire working knowledge of routine laboratory operations, including proper NTEP procedures, appropriate supplies, laboratory safety, basic microscope operations, and requirements of panel testing or rechecking programmes.

Figure: Frequency and Components of On-site Evaluation; Source: RNTCP Laboratory Network Guidelines

Checklist for IRL OSE and NRL OSE

On-site quarterly evaluation report of DTCs visited by IRL team to be given to concerned State TB Officer (STO), IRL and NRL using an IRL OSE Checklist while OSE checklist for NRL Laboratory Personnel to IRL is prepared using a NRL OSE Checklist.

Resources

RNTCP Laboratory Network Guidelines.

Assessment

Documentation of any significant problems (technical, operational, others) and feedback for corrective action during On-site Evaluation (OSE) is necessary to formulate plans with the District TB Officer (DTO), Senior TB Laboratory Supervisor (STLS), Medical Officer (MO), Intermediate Reference Laboratory (IRL), National Reference Laboratory (NRL) and Laboratory Technician (LT) to improve the quality of smear microscopy.

Checklists for OSE

• OSE is done through comprehensive checklists.
• The checklist must be used by NRL, IRL and District TB Center (DTC) STLS and a shorter version by DTO/ Medical Officer TB Center (MOTC) during each OSE visit to document observations and corrective action.
• The checklists contain open, non-leading questions and recommended observations along with objective criteria for acceptable practices so that the supervisor can assess how well the technician understands proper procedures.

Feedback

Feedback is the process of communicating results of External Quality Assurance (EQA) to the original laboratory, including suggestions for possible causes of errors and remedies.

TB Unit On-site Evaluation (TU-OSE) Checklist and Feedback

The observations in TU-OSE Checklist, including observation of five positive and negative slides must be documented (Figure 1). Corrective actions must be discussed by the STLS with LT and checklist should be signed by MO of DMC.

1: Write smear and grade
2: Tick appropriate column
3: Tick if good; write ‘U’ if under-decolourized, ‘O’ if over-decolourized
4: Tick if good; write ‘B’ if too big, ‘S’ if too small
5: Tick if good; write ‘K’ if too thick, ‘N’ if too thin
6: Tick appropriate column

Figure 1: Review of five positive and five negative slides by STLS and instructions to fill observations in TU-OSE Checklist; Source: RNTCP Laboratory Network Guidelines

Action Required based on TU-OSE Checklist

The STLS should:

• Enter the summary of ‘action required’ in the Supervision Register before leaving the DMC
• Submit the summary report of DMCs under him to DTO on a monthly basis
• Maintain separate files for checklists including summary reports of each DMC in respective TU and submitted for review by higher level supervisors including on-site evaluation visits by the IRL to the district.

The DTO should:

• Give monthly feedback to the MO in-charge of the respective DMC
• Send summary of results of checklists for each DMC in the district once every quarter for analysis by the respective CMO/ DMO and IRL.

Checklist IRL OSE and NRL OSE and Feedback

• On-site quarterly evaluation report of DTCs visited by IRL team should be given to concerned State TB Officer (STO), IRL and NRL using an IRL OSE Checklist.
• OSE Checklist for NRL Laboratory Personnel to IRL is prepared using a NRL OSE Checklist.
• The IRLs will submit a consolidated quarterly report to the concerned STO and NRL. In turn, the NRL will submit a consolidated quarterly report to the Central TB Division.
• A comprehensive list of all operational elements to be observed will help to ensure consistency in laboratory evaluations and provide immediate feedback to the technicians to facilitate rapid corrective action, as well as serve as documentation of the visit and record of current conditions and actions needed.
• All potential sources of error should be investigated, including quality of stains and staining procedure, quality of microscopes, and administrative procedures that may contribute to recording errors and corrective actions provided (Figure 2).
• All problems contributing to errors must be resolved.
• Possible causes of errors, and suggested evaluation steps are provided in Annexure K.
• Remedial training must be provided for technicians unable to properly identify AFB in smears.

Figure 2: Documentation of corrective actions in IRL-OSE Checklist )RNTCP Laboratory Network Guidelines)

Action Required for IRL/NRL-OSE Checklist

A state level consolidated summary will be prepared by the respective IRL every quarter from the district summaries for submission to STO and NRL. The DTO/ CMO of the district is to submit an action taken report on the team’s recommendations to the STO within a month of the IRL visit.

Resources

RNTCP Laboratory Network Guidelines.

Assessment

Random Blinded Rechecking (RBRC) is an External Quality Assurance (EQA) method that provides reliable assurance that a district has an efficient Acid-fast Bacillus (AFB) microscopy laboratory network to support National TB Elimination Programme (NTEP) (Figure).

Blinded rechecking is a process of re-reading a statistically valid sample of slides from a laboratory to assess whether that laboratory has an acceptable level of performance.

Figure: Overview of Random Blinded Rechecking (RBRC) under NTEP;
Source: RNTCP Laboratory Network Guidelines

Key Components of RBRC

• The sample contains a sufficient number of randomly selected slides to be representative.
• A system utilizing Lot Quality Assurance Sampling (LQAS) method is in use for RBRC.
• The supervisor of the laboratory (controller) must be unaware of the original result of peripheral laboratory technician to prevent bias, i.e. is “blinded”.
• Discrepant results are resolved by a second controller.
• There must be a system to provide timely feedback and improvements to the laboratories that are supervised.

Resources

RNTCP Laboratory Network Guidelines.

Assessment

Annual slide sample and the monthly sample for Random Blinded Rechecking (RBRC) is determined by the District TB Officer (DTO), assisted by the Intermediate Reference Laboratory (IRL), Statistical Assistant (SA) or Data Entry Operator (DEO) based on Lot Quality Assurance Sampling (LQAS) method.

Under the LQAS method, sample size depends on 3 components:

1. Annual Negative Slide Volume (ANSV)
2. Slide Positivity Rate (SPR)
3. Sensitivity of picking up lower bacili count in microscopy

Process of RBRC (Figure 1)

• The LT at each DMC stores slides for RBRC in slide boxes.
  • The laboratory must store slides in a way that allows for easy retrieval of every slide identified for the rechecking sample. Therefore, all slides are to be stored in the provided slide boxes in the same order as they are listed in the laboratory register. Slides are marked as ‘a’ and ‘b’ along with the lab serial number for first spot and early morning specimen.
  • The result of the smear examination must not be written on the slide.
  • Removal of immersion oil is to be done using tissue paper before storing slides in a slide boxSenior TB Lab Supervisors (STLS) of the district, informing them of the total number of slides to be collected every month from each DMC.
• The STLS then select the required number of slides from the Laboratory Register and the LT records the results as per Annexure B (Figure 2).

Collection of slides for RBRC explained with an example

• • If the sample size is calculated to be 180 smears per year, 15 slides are to be collected during each monthly visit.
  • If the STLS observes that the laboratory processed 82 slides since the last monthly visit, they could collect for example every fifth (82/15 = 5.4 or 5th) slide randomly to obtain the required 15 slides and may begin with any number between 1 to 5, say 3.
  • The first random number may be selected by choosing last digit on any available currency note.
  • The remaining slides are chosen by adding serially 5 till 15 slides are selected.
  • In this example, the 3rd, 8th, 13th, 18th, 23rd, 28th, 33rd, 38th, 43rd, 48th, 53rd, 58th, 63rd, 68th and 73rd slides are selected to obtain 15 slides required for that month.

• Annexure B (Figure 2) is then put into an envelope and sealed. The number of slides packed is written on the top of the envelope.
• Both the slide box and the envelope are marked with the name of the respective DMC, the name of the TB Unit and district, and the month and year.
• The slide box and the sealed envelope are then taken by the STLS for handing over to the DTO.

Figure 1: Process of random blinded rechecking (RBRC) performed by (1) STLS during on-site visit to DMC and (2) at DTC (RNTCP Laboratory Network Guidelines)

Figure 2: Annexure B to document Smear Results Sheet for Blinded Rechecking (RNTCP Laboratory Network Guidelines)

The DTO is responsible for ensuring that the blinding process is strictly followed.

Resources

RNTCP Laboratory Network Guidelines.

Assessment

Random Blinded Rechecking (RBRC) is the process of re-reading a statistically valid sample of routine slides from a designated microscopy centre (DMC) based on lot quality assurance strategy (LQAS) in a blinded manner to assess the laboratory performance.

RBRC Process

• The district TB officer (DTO) sends information to all senior TB lab supervisors (STLS) of the district on the number of slides to be collected every month from each DMC
• The STLS then selects the required number of slides from the TB Laboratory Register and marks the RBRC selected slides in the laboratory register with a circle
• The laboratory technician (LT) fills out Annexure B (Figure 1) for the selected slides
• LT seals the filled Annexure B in an envelope and marks the slide box with the Serial No. of slides, name of the DMC and TB unit (TU), month and year
• The STLS hands over sealed envelopes and the slide box to the DTO

Figure 1: Annexure B for Blinded Rechecking of DMC Slides (RNTCP Laboratory Network Guidelines)

• The DTO with the help of the statistical assistant (SA) removes all the identifying attributes of the selected slides (including the test results). This process is called Blinding.
• Blinded re-examination of selected slides is done by the STLS of another TB Unit (TU) within the respective district. The STLS (controllers) must have demonstrated proficiency with the Ziehl-Neelsen (ZN) staining and reading method (as seen by panel testing done by Intermediate Reference Laboratory (IRL)).
• Smears may be evaluated for specimen quality (sputum versus saliva), appropriate size and thickness, and quality of staining (as per Annexure C, Figure 2).

Figure 2: Worksheet for Blinded Rechecking of DMC Slides as per Annexure C (RNTCP Laboratory Network Guidelines)

• Problems detected by the controller are noted on the form, as this information may be very useful to supervisors responsible for providing feedback to the peripheral technicians, assessing possible reasons for high false positive or false negative results, and implementing plans for retraining and corrective action. 
• The DTO with the help of the statistical assistant compares the results provided by the STLS against the original results provided by the LT. This process is called Unblinding.
• The discrepant slides are sent for umpire (second controller) reading.
• All discrepant slides are re-stained and re-examined by the second controller, as there is likelihood of fading of carbol fuchsin. This rechecking of discordant slides by a second controller also acts to evaluate the performance of the first controllers.

Feedback after RBRC

• Regular and timely feedback to the DMC is essential to improve performance
• Feedback and remedial actions are provided at the end of each sampling period i.e., completion of rechecking of the annual sample
• Also, feedback is given on a monthly basis to the respective DMCs using the form in Annexure D (Figure 3) during the monthly on-site evaluation visit by the STLS responsible for the respective DMC
• Feedback includes the return of slides with discordant results to be re-read by the original LT of the respective DMC
• Potential sources of errors are investigated during the on-site evaluation visit
• Appropriate corrective actions and/or remedial training are provided within one month

Figure 3: Quality Assurance Report on Sputum Microscopy as per Annexure D (RNTCP Laboratory Network Guidelines)

Resources

RNTCP Laboratory Network Guidelines

Assessment