VIROLOGY QUALITY ASSURANCE PROGRAM STATISTICAL CENTER

Size: px

Start display at page:

Download "VIROLOGY QUALITY ASSURANCE PROGRAM STATISTICAL CENTER"

Jeffrey Shelton
5 years ago
Views:

1 TO: CC: Members of the VQA Advisory Board (VQAAB) Bill Meyer Bob Coombs/Ming Chang Nicole Tobin Belinda Yen-Lieberman Joan Dragavon Urvi Parikh Jessica Fogel James Bremer Cheryl Jennings Carolyn Yanavich/Diane Costello Anita Gillis/Jenny Nguyen Heather Sprenger Ron Bosch Joe Fitzgibbon, Keith Crawford FROM: Don Brambilla, Dan Zaccaro, Amy Couzens and Barbara Do DATE: SUBJECT: Samples of RNA035, Revised The following is a revision of the first report on the analysis of VQA HIV RNA proficiency testing data through samples of RNA035. In this revision, we have corrected the description of the results for dataset 93, and removed one dataset (dataset 37) from the analysis. The removed dataset has been re-analyzed and included in the secondary analysis of samples of RNA035. Ninety-three datasets from 70 laboratories were included in the analysis. Datasets that were generated by different kits in the same laboratory were analyzed separately. Eighty-seven of the 93 datasets consisted of results from the most recent four 5-member panels (6-25 of RNA035). Datasets 3 and 50 consisted of the results of a 20- member qualifying panel that was recently tested plus member panels that were tested after the 20-member panel. As usual, the composition of the panels depended upon the kit that was used in a given laboratory (Tables 1-4). Proficiency panels are scored on the total assay standard deviation of log10 recovery, accuracy, counts of negative results near the limit of detection, false positives and invalid results. The total assay standard deviation (SD) is related to the intra-assay and inter-assay standard deviations as ST=(SA 2 +SE 2 ) 1/2 where ST is the total assay SD, SA is the intra-assay SD and SE is the inter-assay SD. The intra-assay, inter-assay and total assay SD are calculated for each data set (Table 5) and are used to identify which data sets exhibit problems with precision and which component of variation is contributing to the elevated precision statistic. Sensitivity is assessed by determining the number of negative results obtained from replicates at the limit of detection for the assay in question. The limit of detection is defined as the nominal concentration at which 95% of assay results are expected to be positive (Roche COBAS AmpliPrep / COBAS TaqMan HIV-1, version 2 assay (Roche TaqMan), Abbott RealTime HIV-1 assay (Abbott RealTime), and internally developed taqman (IH) assays: 50 copies/ml; and biomerieux NucliSens EasyQ HIV RNA (EasyQ): 200 copies/ml). Specificity is assessed by monitoring the results obtained on samples that do not contain HIV. It is also expected that all results should meet established criteria for assay validity. Accuracy is assessed by comparing average(mean) log10 recovery in a laboratory with expected log10 recovery. The average is calculated using the subset of samples that is used to assess the total assay standard deviation. Median log10 recovery across all laboratories in which the same kit was used on the panels in question is treated as the expected value. Estimated kit-specific log10 values are calculated for each analysis (Table 6). Per the VQAAB s recommendation, an analysis investigating the impact of standard errors on the accuracy statistics on previous rounds of testing was conducted in April As a result of this investigation, a modified approach for the scoring procedure is implemented when the SE on a round of testing is low enough to be of concern. Under this modified approach, the observed standard 1

2 error on a round of testing is replaced by the historical median of when the observed SE is below the median; otherwise, the observed value would continue to be used (Table 7). A minimum of 7 data sets must be available in order to generate a reliable median. If fewer than 7 data sets are available for any given assay, then accuracy is assessed using modified criteria. In this case, median log10 recovery is compared to 1) the nominal concentration; 2) an overall real-time cross-platform derived median; and 3) an overall cross-platform derived median (Table 8). The overall real-time cross-platform-derived median was determined using data only from all the commercially available real-time PCR kits (specifically, Abbott RealTime and Roche TaqMan). The overall crossplatform-derived median is determined using data from all the commercially available PCR kits. All three medians are used for determining a final accuracy score. If the same penalty is assessed for 2 of the 3 medians, then that penalty is assessed (this could be a no-penalty scenario in which 2 out of the 3 p-values are > 0.05). If the results indicated three different penalties (i.e. >0.05, between 0.05 and 0.01, or <0.01) then a penalty of between 0.05 and 0.01 is assessed, which is equivalent to a PC. In this analysis, there were fewer than 7 data sets available for the IT and EasyQ assays. The modified criteria for assessing accuracy were used to evaluate accuracy in these data sets. The VQA will now be providing official scores for non-technical criteria including the timeliness of query responsiveness and the submission of data containing patient identifiable information or protected health information (PII/PHI). The inclusion of these non-technical criteria were unofficial for two rounds of testing starting with the scoring of RNA032-1 (July 2016). The official scoring of these non-technical criteria will begin with the scoring of samples including RNA032-3 (November 2016). The non-technical scoring nomenclature will help to differentiate between problems with late data submissions (T), unresponsiveness to data queries (Q) and submission of data containing PII/PHI (P). A number of 1 or 2 will be used to indicate if one (1) or more (2) of a given problem was noted. Multiple letters will indicate if multiple problems were noted in the last four analyses (C1 T,P or C2 Q ). Technical scores will be indicated as done previously (C, PC, P) and the nontechnical problems will be indicated as superscripts to the technical score. The numeric score for each round that is used for determining the cumulative rating will be based on the combined technical and non-technical criteria. Beginning in September 2017, the VQA provided both an official and an unofficial score based on reported Abbott false negative results derived from the 50cp/mL controls. The official score did not incorporate sensitivity results based on 50cp/mL samples; unofficial scores reflected false negative results as per our historical criteria. Sensitivity results were excluded from official scores due to an investigation into the increased rate of false negative results (from Abbott kits) for VQA HIV RNA proficiency testing samples with a nominal value of 50 cp/ml. As of the primary analysis for RNA panel 035, samples 6-10, the VQA has resumed official scoring for sensitivity for Abbott data starting with samples 6-10 of RNA035. Any false negatives reported for panels RNA032 or RNA033 will not be included in sensitivity scoring, and have been removed from Table 9; only data from RNA035 testing or later will be used for sensitivity scoring. Beginning in March 2018, the VQA will be providing both an official and an unofficial score based on accuracy results for the Roche cobas HIV-1 quantitative nucleic acid test. The official score will not incorporate accuracy statistics; unofficial scores will reflect accuracy statistics per the historical criteria. Accuracy results will be excluded from official scores until sufficient numbers of data sets (>7) are available to permit the estimation of within-assay median values for use in accuracy scoring. It is anticipated that this report will be discussed during our next conference call. We can be reached for questions or comments via (dzaccaro@rti.org), telephone ( ) or FAX ( ). 2

3 RESULTS: ROCHE COBAS ASSAY No technical problems were identified in the 8 datasets generated with the Roche cobas HIV-1 assay (Table 9). Scores of C are recommended for the datasets generated with this assay. Trends in log10 recovery with Roche Cobas kit data are illustrated in Table 12 and Figure 3. RESULTS: ROCHE TAQMAN ASSAY Technical problems were identified in 5 of the 30 datasets generated with the Roche TaqMan assay (Table 9). Accuracy issues were noted in datasets 14, 16 and 19. Scores of PC are recommended for datasets 16 and 19 (accuracy statistics of and were observed), and a score of P is recommended for dataset 14 (an accuracy statistic of was observed). A failed VQA 200 copy control (VQA200 reported as 708 copies/ml) was observed in dataset 23. This invalidates all 5 samples. A score of P is recommended for dataset 23. Scores of C are recommended for the other 25 datasets generated with this assay. We continue to monitor log10 recovery for laboratories using the Roche TaqMan Assay (Table 10; Figure 1) and will continue to share data with Roche Molecular Systems (RMS) on the performance of their assay; new analyses looking specifically at log10 recovery between data sets generated with AmpliPrep vs. High Pure manual extractions have been added to this monitoring. RESULTS: IN-HOUSE ASSAY No technical problems were identified in the 1 dataset generated with the In-house (IH) assay (Table 9). A score of C is recommended for the dataset generated with this assay. RESULTS: EASYQ ASSAY No technical problems were identified in the 2 datasets generated with the EASYQ assay (Table 9). Scores of C are recommended for the datasets generated with this assay. RESULTS: GENEXPERT ASSAY No technical problems were identified in the 8 datasets generated with the GeneXpert assay (Table 9). Scores of C are recommended for the datasets generated with this assay. Trends in log10 recovery with GeneXpert kit data are illustrated in Table 13 and Figure 3. RESULTS: ABBOTT REALTIME ASSAY Technical problems were identified in 2 of the 44 datasets generated with the Abbott RealTime assay (Table 9). One false positive result was noted in dataset 61 which was a carryover from the last round (35-4), but the laboratory did not submit a repeat panel. A score of PC is recommended for this dataset. Two false negative results and an elevated total assay standard deviation due to increased inter-assay standard deviation were noted in dataset 93. Both of these false negatives are from RNA proficiency panel 035 (one from round 35-3 and one from 35-4, but the laboratory did not submit a repeat panel). A score of P is recommended for dataset 93. Scores of C are recommended for the other 42 datasets generated with this assay. Trends in log10 recovery with Abbott kit data are illustrated in Table 11 and Figure 2. The Abbott mean log10 recovery continues to be relatively flat. No problems with query unresponsiveness or the submission of data that included patient identifiable information (PII) or protected health information (PHI) were noted in any of the data sets included in this analysis. It is anticipated that this report will be discussed during our next conference call. We can be reached for questions or comments via (dzaccaro@rti.org), telephone ( ) or FAX ( ). 3

4 TABLE 1. COMPOSITION OF SAMPLES 6-10 OF RNA PROFICIENCY PANEL RNA035*. NOMINAL CONCENTRATION (COPIES/ML) RT , , ,000 1 TABLE 2. COMPOSITION OF SAMPLES OF RNA PROFICIENCY PANEL RNA035*. NOMINAL CONCENTRATION (COPIES/ML) RT ,250 34, ,000 1 TABLE 3. COMPOSITION OF SAMPLES OF RNA PROFICIENCY PANEL RNA035*. NOMINAL CONCENTRATION (COPIES/ML) RT , , ,000 TABLE 4. COMPOSITION OF SAMPLES OF RNA PROFICIENCY PANEL RNA035*. NOMINAL CONCENTRATION (COPIES/ML) RT ,250 34, ,000 1 *Prior to panel RNA035, the OT configuration was used by labs using the EasyQ assay; the RT configuration was used by labs testing the Roche Cobas, TaqMan, In-House, GeneXpert or Abbott RealTime assays. Starting in panel RNA035, all labs used the RT configuration. 4

5 TABLE 5. THE INTRA-ASSAY, INTER-ASSAY AND TOTAL ASSAY STANDARD DEVIATIONS FOR RNA PROFICIENCY PANELS RNA035 (SAMPLES 6-25). DATASETS WITH HIGH INTRA-ASSAY OR INTER-ASSAY SDs ARE LISTED IN BOLD. KIT DATASET INTRA-ASSAY SD INTER-ASSAY SD TOTAL ASSAY SD RC RT

6 KIT DATASET INTRA-ASSAY SD INTER-ASSAY SD TOTAL ASSAY SD IH ME GX AR

7 KIT DATASET INTRA-ASSAY SD INTER-ASSAY SD TOTAL ASSAY SD AR RC: ROCHE COBAS RT: ROCHE TAQMAN ASSAY; IH: IN-HOUSE ASSAY; GX: GENEXPERT ASSAY; ME: EASYQ STANDARD PREP; AR: ABBOTT REALTIME ASSAY TABLE 6: MEDIAN LOG10 RECOVERY FOR EACH KIT OVER PAST FOUR ROUNDS, RNA035 (SAMPLES 6-25). Excludes samples <100 copies/ml for RT, ME, GX and AR. Assay Number of Labs* Number of Samples Tested* Median Log Recovery ALL AR AR-RT RT * The ALL median log 10 recovery was generated from the data from all laboratories that participated in the four most recent 5-member RNA panels with a commercially available PCR kit; AR: ABBOTT REALTIME; the AR-RT median log 10 recovery was generated from the data from all laboratories that participated in the four most recent 5-member RNA panels with a commercially available real-time PCR kit (specifically, the Abbott RealTime and Roche TaqMan assays); RT: ROCHE TAQMAN. ** Number of labs/samples included in the calculation of the median log recovery value. TABLE 7: THE STANDARD ERROR OF LOG10 RECOVERY OVER PAST FOUR ROUNDS, RNA035 (SAMPLES 6-25), USED IN THE ACCURACY STATISTIC. Excludes samples <100 copies/ml for RT, ME, GX and AR. OBSERVED STANDARD ERROR THRESHOLD STANDARD ERROR STANDARD ERROR USED IN ANALYSIS

8 TABLE 8. ACCURACY SCORING FOR DATASETS GENERATED WITH AN ASSAY THAT DOES NOT HAVE AN ASSAY-SPECIFIC MEDIAN. MEDIAN LOG10 RECOVERY COMPARED TO NOMINAL, A REALTIME MEDIAN AND AN ALL-ASSAY MEDIAN. KIT DATASET Accuracy (Nominal) P-Value Accuracy (RealTime Median) P-Value Accuracy (All-Assay Median) RC IH ME GX P-Value RC: ROCHE COBAS ASSAY; GX: GENEXPERT ASSAY; IH: IN-HOUSE ASSAY; ME: EASYQ STANDARD PREP 8

9 TABLE 9. PERFORMANCE STATISTICS FOR RNA PROFICIENCY PANELS RNA035 (SAMPLES 6-25) FOR ASSESSMENTS THAT INCLUDE ACCURACY AND THE INTRA-ASSAY STANDARD DEVIATION. (Entries of zero are excluded for clarity) DS: DATASET; VER: ROCHE TAQMAN ASSAY USED (48 OR 96; AmpliPrep or High Pure [HP]) AND VERSION [v2]; INHOUSE ASSAY USED [IH];ABBOTT RT ASSAY USED (0.6ML PROTOCOL [06], AUTOMATED [AUTO] OR MANUAL [MAN] EXTRACTION); MOLECULAR BEACONS EASYQ ASSAY VERSION 2.0 [ME]. NI: THE NUMBER OF INVALID SAMPLES ON THE PANEL; FP: NUMBER OF NEG SAMPLES ABOVE DETECTION LIMIT; FN: NUMBER OF POS SAMPLES BELOW DETECTION LIMIT; NDL: NUMBER OF SAMPLES AT THE LIMIT OF DETECTION; FN50/FN200: NUMBER OF POS SAMPLES AT 50 COPIES/ML BELOW DETECTION LIMIT ON THE ABBOTT RT ASSAY AND THE ROCHE TAQMAN ASSAY/ NUMBER OF POS SAMPLES AT 200 COPIES/ML BELOW DETECTION LIMIT ON THE ME ASSAY; N: THE NUMBER OF SAMPLES INCLUDED IN THE STANDARD DEVIATION; EST: THE TOTAL-ASSAY STANDARD DEVIATION; DIFF: AVERAGE DIFFERENCE BETWEEN THE LOG10 RECOVERY IN A LABORATORY WITH THE EXPECTED LOG10 RECOVERY EXPONENTIATED BACK INTO THE RAW SCALE; ACC: ACCURACY STATISTIC; P: THE P-VALUE FOR EVALUATING FN50/FN200, SD, OR ACCURACY; UNOFF SC: UNOFFICIAL SCORE INCLUDING SCORING FOR SENSITIVITY FOR ABBOTT KITS AND SCORING FOR ACCURACY FOR ROCHE COBAS KITS OFF SC: OFFICIAL SCORE BASED ON PERFORMANCE INCLUDING NON-TECHNICAL SCORING CRITERIA (QUERY RESPONSIVENESS AND INCLUSION OF PHI/PII). THIS EXCLUDES SCORING FOR SENSITIVITY FOR ABBOTT KITS KIT DATASET VER NI NIC FP FN NDL FN50, FN200 TOTAL ASSAY SD ACCURACY UNOFF OFF ACC P SC SC N P N EST P AVE RC 1 ROCH DEV C C 2 ROCH C C 3 ROCH C C 4 ROCH C C 5 ROCH C C 6 ROCH C C 7 ROCH C C 8 ROCH C C RT 9 RT48v C 10 RT48v C 11 RT48v C 12 RT48v C 13 RT48v C 14 RT48v2HP P 15 RT48v C 16 RT48v2HP PC 17 RT48v C 18 RT48v C 19 RT48v2HP PC 20 RT96v C 21 RT96v C 22 RT96v C 9

10 KIT DATASET VER NI NIC FP FN NDL FN50, FN200 TOTAL ASSAY SD ACCURACY UNOFF OFF ACC P SC SC N P N EST P AVE RT 23 RT96v DEV P 24 RT96v C 25 RT96v C 26 RT96v C 27 RT96v C 28 RT96v C 29 RT96v C 30 RT96v C 31 RT96v C 32 RT96v C 33 RT96v C 34 RT96v C 35 RT96v C 36 RT96v C 38 RT96v C 39 RT96v C IH 40 InHouse C ME 41 STDv C 42 STDv C GX 43 GeneXper C 44 GeneXper C 45 GeneXper C 46 GeneXper C 47 GeneXper C 48 GeneXper C 49 GeneXper C 50 GeneXper C AR 51 06AUTO C 52 06AUTO C 53 06AUTO C 54 06AUTO C 55 06AUTO C 56 06AUTO C 57 06MAN C 58 06AUTO C 59 06AUTO C 60 06AUTO C 61 06AUTO PC 10

11 KIT DATASET VER NI NIC FP FN NDL FN50, FN200 TOTAL ASSAY SD ACCURACY UNOFF OFF ACC P SC SC N P N EST P AVE AR 62 06AUTO DEV C AR 63 06MAN C AR 64 06AUTO C AR 65 06AUTO C AR 66 06AUTO C AR 67 06MAN C AR 68 06MAN C AR 69 06AUTO C AR 70 06AUTO C AR 71 06MAN C AR 72 06AUTO C AR 73 06AUTO C AR 74 06AUTO C AR 75 06AUTO C AR 76 06AUTO C AR 77 06AUTO C AR 78 06AUTO C AR 79 06AUTO C AR 80 06AUTO C AR 81 06MAN C AR 82 06AUTO C AR 83 06AUTO C AR 84 06AUTO C AR 85 06MAN C AR 86 06MAN C AR 87 06AUTO C AR 88 06MAN C AR 89 06AUTO C AR 90 06AUTO C AR 91 06AUTO C AR 92 06AUTO C AR 93 06AUTO P AR 94 06MAN C RC: ROCHE COBAS ASSAY; RT: ROCHE TAQMAN ASSAY; IH: IN-HOUSE ASSAY; ME: EASYQ STANDARD PREP; AR: ABBOTT ASSAY DATA SETS WITH PROBLEMS ARE HIGHLIGHTED WITH SHADING. 11

12 TABLE 10. COMPARISON OF KIT LOT NUMBER, MEAN LOG10 RECOVERY, TOTAL ASSAY SD, AND SCORE FOR DATA GENERATED WITH THE ROCHE TAQMAN ASSAY IN RNA035 (SAMPLES 6-25). SHADED ROWS REPRESENT DATASETS WITH TECHNICAL PROBLEMS IN THIS LATEST ROUND OF TESTING. RED FONT INDICATES LABORATORIES THAT ARE BEING MONITORED BUT DID NOT SUBMIT DATA AND GET SCORED FOR THIS ROUND. VER DATASET KIT LOT MEAN LOG 10 TOTAL ASSAY SD SCORE RT48v2 9 Y15616 Y24080 E00164 E C C C C RT48v2 10 Y24080 E00164 E00164 E C C C C RT48v2 - Y15616 E00164 EXE OH EXE OH EXE OH C C EXE OH RT48v2 - Y16242 EXE EXE EXE EXE EXE EXE 0.23 EXE EXE EXE PC EXE EXE EXE RT48v2 11 Y15616 Y24080 E00164 E C C C C RT48v2 12 Y11397 Y18458 Y14818 Y C PC C C RT48v2 - Y24080 Y24080 OH OH OH OH OH OH C C OH OH RT48v2 13 Y14818 Y14818 Y21188 Y C C C C RT48v2HP P P P P RT48v2 15 Y15616 Y24080 Y24080 Y C C C C RT96v2 - Y10651 OH OH OH 0.08 OH OH OH 0.13 OH OH OH C OH OH OH RT48v2 - X09013 X09013 Y17461 EXT EXT EXT C C C EXT RT48v2HP 16 EXE EXE EXE EXE P P PC RT48v2 17 Y15616 Y15616 E00164 E C C C C RT48v2 18 Y24080 Y24080 E00164 E C C C C RT48v2HP PC PC PC PC RT48v2 20 Y15616 Y24080 Y24080 E C C C C RT96v2 - EXE EXE OH OH EXE EXE OH OH EXE EXE OH OH EXE EXE OH OH RT96v2 21 Y14632 Y18458 Y21188 Y C C C C RT96v2 22 X14181 X14181 Y16242 Y C C C C 12

13 VER DATASET KIT LOT MEAN LOG 10 TOTAL ASSAY SD SCORE RT96v2 23 Y16242 Y10651 Y10651 Y C C C P RT96v2 24 Y11397 Y17461 Y21188 Y C C C C RT96v2 25 X13007 Y18458 Y18458 Y C C C C RT96v2 26 Y16242 Y10651 Y10651 Y C C C C RT96v2 27 Y11397 Y11397 Y11397 Y C C P C RT96v2 28 Y14632 Y28460 Y23998 E C C C C RT96v2 29 Y14818 Y14818 Y21188 E C C C C RT96v2 - Y03434 Y03434 Y17121 EXT EXT EXT C C C EXT RT96v2 30 Y16436 Y16436 Y17461 Y C C PC C RT96v2 - X09013 X09013 Y EXT EXT EXT C C C EXT RT96v2 31 Y17461 Y18458 Y22924 E C C C C RT96v2 32 Y17121 Y20212 E00588 Y C C C C RT48v2 33 Y10651 Y16436 Y16436 Y C C C C RT96v2 34 X05514 Y17121 Y20212 Y C C C C RT96v2 35 Y10651 Y10651 Y08928 Y C C C C RT96v2 36 Y06193 Y14632 Y21188 Y C PC C C RT96v2 - X X13007 Y C C C - RT96v2 38 Y16436 Y17461 Y22924 E C C C C RT96v2 39 X06831 E00164 E00164 E C C C C 13

14 TABLE 11. COMPARISON OF KIT LOT NUMBER, MEAN LOG10 RECOVERY, TOTAL ASSAY SD, AND SCORE FOR DATA GENERATED WITH THE ABBOTT REALTIME ASSAY IN RNA035 (SAMPLES 6-25). SHADED ROWS REPRESENT DATASETS WITH TECHNICAL PROBLEMS IN THIS LATEST ROUND OF TESTING. RED FONT INDICATES LABORATORIES THAT ARE BEING MONITORED BUT DID NOT SUBMIT DATA AND GET SCORED FOR THIS ROUND. VER DATASET KIT LOT MEAN LOG 10 TOTAL ASSAY SD SCORE AR0.6AUT C C C C AR0.6AUT C C C C AR0.6AUT C C C C AR0.6AUT C C C C AR0.6AUT C C C C AR0.6AUT C C C C AR0.6MAN C C C C AR0.6AUT C C C C AR0.6AUT C C C C AR0.6AUT C C C C AR0.6AUT PC C PC PC AR0.6AUT C C C C AR0.6AUT C C C C AR0.6MAN C C C C AR0.6AUT C C C C AR0.6AUT C C C C AR0.6MAN C C C C AR0.6MAN C C C C AR0.6AUT C C C C AR0.6AUT PC C C C AR0.6MAN PC C C C AR0.6AUT OH OH OH C C C OH 14

15 VER DATASET KIT LOT MEAN LOG 10 TOTAL ASSAY SD SCORE AR0.6AUT C C C C AR0.6AUT C C C C AR0.6AUT C C C C AR0.6AUT C C C C AR0.6AUT C C C C AR0.6AUT C C C C AR0.6AUT C C C C AR0.6AUT C C C C AR0.6AUT EXT EXT EXT C C C EXT AR0.6AUT EXE EXE EXE EXE EXE EXE 0.09 C EXE EXE C AR0.6MAN EXE EXE EXE EXE EXE EXE 0.11 C EXE EXE C AR0.6AUT C C C C AR0.6AUT PC C C C AR0.6AUT C C C C AR0.6MAN C C C C AR0.6MAN C C C C AR0.6AUT 87 EXE EXE EXE EXE C C C AR0.6MAN M M M C C C C AR0.6AUT C C C C AR0.6AUT C C C C AR0.6AUT EXT EXT EXT C C C EXT AR0.6AUT C C C C AR0.6AUT EXT EXT EXT C C C EXT AR0.6AUT C PC C C AR0.6AUT C C PC P AR0.6MAN C C C C 15

16 TABLE 12. COMPARISON OF KIT LOT NUMBER, MEAN LOG10 RECOVERY, TOTAL ASSAY SD, AND SCORE FOR DATA GENERATED WITH THE ROCHE COBAS ASSAY IN RNA035 (SAMPLES 6-25). SHADED ROWS REPRESENT DATASETS WITH TECHNICAL PROBLEMS IN THIS LATEST ROUND OF TESTING. RED FONT INDICATES LABORATORIES THAT ARE BEING MONITORED BUT DID NOT SUBMIT DATA AND GET SCORED FOR THIS ROUND. VER DATASET KIT LOT MEAN LOG 10 TOTAL ASSAY SD SCORE ROCH C P C C ROCH C C C C ROCH C C ROCH C C C C ROCH C C C C ROCH C C C ROCH C C C ROCH C C C C 16

17 TABLE 13. COMPARISON OF KIT LOT NUMBER, MEAN LOG10 RECOVERY, TOTAL ASSAY SD, AND SCORE FOR DATA GENERATED WITH THE GENEXPERT ASSAY IN RNA035 (SAMPLES 6-25). SHADED ROWS REPRESENT DATASETS WITH TECHNICAL PROBLEMS IN THIS LATEST ROUND OF TESTING. RED FONT INDICATES LABORATORIES THAT ARE BEING MONITORED BUT DID NOT SUBMIT DATA AND GET SCORED FOR THIS ROUND. VER DATASET KIT LOT MEAN LOG 10 TOTAL ASSAY SD SCORE GeneXpert C C C C GeneXpert C C C C GeneXpert C C C C GeneXpert C C C C GeneXpert C C GeneXpert C C C C GeneXpert C C C C GeneXpert C C C C GeneXpert C GeneXpert EXT EXT C C C EXT GeneXpert C C 17

18 Mean LOG 10 RNA Recovery Mean LOG 10 RNA Recovery VIROLOGY QUALITY ASSURANCE PROGRAM FIGURE 1. LINE PLOT OF MEAN LOG10 RECOVERY FOR DATA GENERATED WITH THE ROCHE TAQMAN ASSAY IN RNA028 SAMPLES, RNA029 SAMPLES, RNA030 SAMPLES, RNA031 SAMPLES, RNA032 SAMPLES, RNA033 SAMPLES AND RNA035 SAMPLES LABORATORIES WITHOUT ( left panel ) and WITH (right panel) SCORING PROBLEMS. Taqman: Labs with all C scores Taqman: Labs with at least one PC or P score 0.4 Kits S02574 and higher High Pure Kits Adjusted Means Other Kits 0.4 Kits S02574 and higher Adjusted Means value= High Pure Kits Other Kits RNA 028 RNA 029 RNA 030 RNA 031 RNA 032 RNA 033 RNA RNA 028 RNA 029 RNA 030 RNA 031 RNA 032 RNA 033 RNA

19 Mean LOG 10 RNA Recovery Mean LOG 10 RNA Recovery VIROLOGY QUALITY ASSURANCE PROGRAM FIGURE 2. LINE PLOT OF MEAN LOG10 RECOVERY FOR DATA GENERATED WITH THE ABBOTT ASSAY IN RNA028 SAMPLES, RNA029 SAMPLES, RNA030 SAMPLES, RNA031 SAMPLES, RNA032 SAMPLES, RNA033 SAMPLES AND RNA035 SAMPLES LABORATORIES WITHOUT (left panel) and WITH (right panel) SCORING PROBLEMS. 0.4 Abbott: Labs with all C scores Adjusted Means 0.4 Abbott: Labs with at least one PC or P score value= Adjusted Means value= value= value= value= value= RNA 028 RNA 029 RNA 030 RNA 031 RNA 032 RNA 033 RNA RNA 028 RNA 029 RNA 030 RNA 031 RNA 032 RNA 033 RNA

20 Mean LOG 10 RNA Recovery Mean LOG 10 RNA Recovery VIROLOGY QUALITY ASSURANCE PROGRAM FIGURE 3. LINE PLOT OF MEAN LOG10 RECOVERY FOR DATA GENERATED WITH THE GENEXPERT (left panel) and ROCHE COBAS (right panel) ASSAYs IN RNA029 SAMPLES, RNA030 SAMPLES, RNA031 SAMPLES, RNA032 SAMPLES, RNA033 SAMPLES AND RNA035 SAMPLES ALL LABORATORIES INCLUDED. GeneXpert: All Labs Roche Cobas: All Labs 0.4 Adjusted Means 0.4 Adjusted Means RNA 028 RNA 029 RNA 030 RNA 031 RNA 032 RNA 033 RNA RNA 028 RNA 029 RNA 030 RNA 031 RNA 032 RNA 033 RNA

VIROLOGY QUALITY ASSURANCE PROGRAM STATISTICAL CENTER

VIROLOGY QUALITY ASSURANCE PROGRAM STATISTICAL CENTER TO: CC: Members of the VQA Advisory Board (VQAAB) Bill Meyer Bob Coombs/Ming Chang Nicole Tobin Belinda Yen-Lieberman Joan Dragavon Urvi Parikh Jessica Fogel James Bremer Cheryl Jennings Carolyn Yanavich/Diane