Journal of Clinical Virology 48 (2010) 234–238

Contents lists available at ScienceDirect

Journal of Clinical Virology journal homepage: www.elsevier.com/locate/jcv

Validation of the Cepheid Xpert Flu A Real Time RT-PCR detection panel for Emergency Use Authorization Anthony R. Sambol a,∗ , Peter C. Iwen a , Maura Pieretti b , Samik Basu c , Michael H. Levi c , Kimberly D. Gilonske d , Kimberly D. Moses d , Jamie L. Marola e , Preveen Ramamoorthy e a

Department of Pathology & Microbiology, University of Nebraska Medical Center, 42nd and Emile Str., Omaha, NE 68198-5900, United States BayCare Health System, Clearwater, FL 33756, United States Albert Einstein College of Medicine, Montefiore Medical Center, Bronx, NY 10467, United States d Mid America Clinical Laboratories, Indianapolis, IN 46219, United States e National Jewish Health, Denver, CO 80216, United States b c

a r t i c l e

i n f o

Article history: Received 15 March 2010 Received in revised form 26 May 2010 Accepted 2 June 2010 Keywords: Cepheid Xpert Emergency Use Authorization Clinical Laboratory Improvement Amendment 2009 Influenza A/H1N1 RT-PCR

a b s t r a c t Background: In April 2009, the United States Secretary of the Department of Health and Human Services declared a public health emergency concerning the 2009 influenza H1N1 outbreak. This declaration allowed the FDA to issue Emergency Use Authorization (EUA) of approved in vitro diagnostics to detect the 2009 influenza H1N1 in clinical specimens. Objectives: This report outlines the validation testing of the Cepheid Xpert Flu A Panel for the qualitative detection of 2009 H1N1 viral RNA. Study design: This study was a multi-site, dual-method clinical evaluation comparing the results of testing between the Xpert Panel assay to the FDA-cleared Luminex Molecular Diagnostics xTAGTM Respiratory Viral Panel (Luminex RVP) assay and the EUA-granted Focus Diagnostics Influenza A/H1N1 (2009) Real Time RT-PCR (Focus H1N1) assay. Results: When compared to Luminex RVP (n = 300) for influenza A detection, the Xpert Panel had a sensitivity of 91.2% (95% CI: 85.1–95.4), specificity of 99.4% (95% CI: 96.7–100), positive predictive value (PPV) of 99.2% (95% CI: 95.6–100), and a negative predictive value (NPV) of 93.1% (95% CI: 88.3–96.4). When compared to the Focus H1N1 (n = 258) for detection of H1N1, the Xpert Panel had a sensitivity of 92.1% (95% CI: 82.4–97.4), specificity of 100% (95% CI: 98.5–100), PPV of 100% (95% CI: 95.0–100), and a NPV of 97.5% (95% CI: 94.3–99.2). Conclusions: The results show the Cepheid Xpert Flu A Panel to be comparable to both the Luminex RVP and the Focus H1N1 assays. The Cepheid Xpert Panel was granted an EUA on 24 Dec 2009. Published by Elsevier B.V.

1. Background Rapid influenza diagnostic tests (RIDTs) are used routinely in clinical laboratories, physician offices, and emergency rooms to screen clinical specimens for the influenza A and B viruses. Unfortunately, performances of these immunoassays have been shown to vary widely in both sensitivity and specificity. These variations are due, in part, to the prevalence of the disease in the population at

Abbreviations: UNMC, University of Nebraska Medical Center; U.S. FDA, United States Food and Drug Administration; CDC, Centers for Disease Control and Prevention; CLIA, Clinical Laboratory Improvement Amendment; EUA, Emergency Use Authorization. ∗ Corresponding author at: Department of Pathology & Microbiology, University of Nebraska Medical Center, Durham Research Center 2, Room 8030, Omaha, NE 68198-5900, United States. Tel.: +1 402 559 3032; fax: +1 402 559 7799. E-mail address: [email protected] (A.R. Sambol). 1386-6532/$ – see front matter. Published by Elsevier B.V. doi:10.1016/j.jcv.2010.06.001

the time, specimen collection type, and adequate training of testing personnel.1,2 To address these issues, additional testing of patient specimens is often necessary to confirm screening assay results. However, sensitive and specific confirmatory tests, such as direct fluorescent antibody and viral culture, are labor intensive and require skilled and highly trained personnel. Viral cultures can take up to 5 days to receive final results, although positive culture results are often available within 2 days.3 By contrast, real time reverse transcriptase polymerase chain reaction assays (RT-PCR) may be used as a rapid initial or confirmatory test for influenza. In preparation for an outbreak of a novel strain of influenza virus, the United States Food and Drug Administration (FDA) published a draft guidance document in 2008 for the detection and differentiation of influenza viruses.4 Shortly after the appearance of novel H1N1 virus in North America in the spring of 2009,5 an emergency was declared by the Secretary of the Department of Health and

A.R. Sambol et al. / Journal of Clinical Virology 48 (2010) 234–238

Human Services. This allowed the FDA to rapidly grant Emergency Use Authorization (EUA) to manufacturers of 2009 H1N1 RT-PCR tests upon submission and review of accelerated clinical validation studies that were designed in compliance with the 2008 draft guidance.6 Following the emergency declaration, a number of H1N1 RT-PCR assays received EUA status.7,8 Cepheid (Sunnyvale, CA) was granted an EUA from the FDA for its Xpert(R) Flu A Panel test, which was the only 2009 H1N1 test to have received the Clinical Laboratory Improvement Amendments (CLIA) classification of “moderate complexity”. This RT-PCR assay is performed using the GeneXpert(R) System (Cepheid) and allows the identification of the 2009 H1N1 influenza virus in less than 1 h. 2. Objectives This paper presents the EUA validation process for the performance of the Cepheid Xpert Flu A Panel for the qualitative detection of 2009 H1N1 viral RNA. 3. Study design The study was designed to evaluate the results of the Xpert Panel assay compared to the FDA-cleared Luminex Molecular Diagnostics xTAGTM Respiratory Viral Panel (Luminex RVP; Luminex, Austin, TX) and the EUA-granted Focus Diagnostics Influenza A/H1N1 (2009) Real Time RT-PCR (Focus H1N1; Focus Diagnostics, Cypress, CA) for the detection of influenza A and, specifically, H1N1. The detection of influenza B was not addressed in this study.

235

3.3. Reproducibility and assay complexity To demonstrate the intra-site reproducibility and the assay complexity, a panel of 4 samples comprised of 3 different influenza A strains, prepared as a low-positive, high-positive and negative samples. These samples were tested on 3 different days by 6 different operators (3 with prior PCR experience and 3 without). One lot of the Xpert Panel was used and the assays were performed according to the Xpert Panel procedure. 3.4. Specimen collection, storage, and transport Specimens were collected as a part of standard care for patients with influenza-like illness. Specimens were either a nasal aspirate/wash (NA/W) or a nasopharyngeal swab (NP). The samples were placed into viral culture medium and following routine testing were stored at refrigerator temperatures for no longer than 24 h prior to aliquoting. A minimum of 1 ml for each specimen was required to provide sufficient sample volume for inclusion in the study. The samples were subsequently aliquoted into 500 ␮L samples and stored at −70 ◦ C prior to additional testing. The selection of transport media, specimen volumes, shipping conditions, and interval from specimen collection time to testing were followed per each manufacturer’s guidelines. Specimens were tested by both the Xpert Panel and the comparative assays within 24 h of thawing. Samples were stored at 2–8 ◦ C between assays. 3.5. Comparator assay testing and discrepant results

The three clinical study sites chosen for this study were BayCare Health System, Clearwater, FL; Albert Einstein College of Medicine, Montefiore Medical Center, Bronx, NY; and the University of Nebraska Medical Center, Omaha, NE. All study sites were assessed by Cepheid Clinical Affairs staff prior to the start of the study to ensure that each site had Institutional Review Board (IRB) approval or waiver prior to initiating study procedures and providing specimen samples. The other participating institutions provided testing service only and were enrolled by the sponsor.

It should be noted that only 258 of the 300 samples had sufficient volume for all 3 required assays. Duplicate samples from any one patient were excluded from the analyses. Comparator assay testing was conducted in accordance with the corresponding package labeling by all operators. For the purpose of the EUA submission, the test results of the Xpert Panel were compared independently to either the Luminex RVP or the Focus H1N1 assays. Each of these comparator tests was designated as the gold standard by which to compare the Xpert Panel results. No discrepant testing was performed when the results obtained with the Xpert Flu A Panel did not correlate with the results of the corresponding comparator test.

3.2. Method comparison

3.6. Data and statistical analysis

To determine the clinical performance of the Xpert Panel, influenza-positive and influenza-negative specimens were tested using the Xpert Panel and each comparator assay. In the Luminex RVP evaluation, 300 specimens were tested and included 47 “seasonal” influenza A/H1, 23 influenza A/H3, 65 influenza A “untypeable” (not “seasonal” H1 or H3), 164 influenza-negative and 1 specimen that was reported as indeterminate. In the Focus H1N1 comparison, 258 specimens were tested and included 26 “seasonal” influenza A/H1, 7 were influenza A/H3, 62 were influenza A “untypeable”, 162 were negative and 1 specimen that was reported as indeterminate. Performance of the Xpert Panel was compared individually to the results of testing with Luminex RVP or Focus H1N1 assays. In the event of an indeterminate finding by the Xpert Panel, a single retest of the sample was performed and the results reported accordingly. The Luminex RVP detects the presence of multiple influenza strains; however, for purposes of this part of the study, only influenza A was considered in the assessment of the Xpert Panel performance versus the Luminex RVP. The results from the Focus H1N1 were used to assess the Xpert Panel performance relative to 2009 H1N1 detection.

Data was tabulated for the Xpert Panel versus the corresponding comparator methods for sensitivity (percent positive agreement), specificity (percent negative agreement), positive predictive value (PPV), and negative predictive value (NPV). Confidence intervals were calculated using Minitab v15 (Minitab, Inc., State College, PA). The Fisher’s exact test for poolability was assessed across sample types for both the influenza A and 2009 H1N1 viruses.

3.1. Site selection

4. Results 4.1. Specimen accountability In the first part of the clinical studies, a total of 346 patient specimens were evaluated for inclusion. Of these specimens, 312 were eligible for inclusion and 34 were excluded (21 were of unspecified specimen type, 11 were an incorrect specimen type, and 2 were “extra” specimens that were not tested by any method). Of the 312 eligible specimens, 12 (3.8%) were excluded for indeterminate results (“invalid” or “error” based on internal assay controls), leaving 300 included in the final dataset used for the analysis of performance compared to the Luminex RVP assay. Forty-four of

236

A.R. Sambol et al. / Journal of Clinical Virology 48 (2010) 234–238

Fig. 1. Specimen selection and accountability.

the specimens included in the final dataset for the analyses of performance of Xpert Panel versus the Luminex RVP were not sent for Focus H1N1 testing due to insufficient sample volumes. Therefore, in the second part of the study, a total of 258 specimens were available for analysis of performance of Xpert Panel versus Focus H1N1. Specimen accountability is summarized in Fig. 1.

4.2. Assay reproducibility and laboratory operator experience An evaluation was conducted to ascertain whether there was a difference in assay performance between Xpert Panel operators in this study with and without prior laboratory experience. The influenza strains and concentrations for the reproducibility and assay complexity study are shown in Table 1. At each of the 3 participating sites, 1 of the 2 operators had previous laboratory experience and 1 did not. The results demonstrated that the inexperienced operator performed as well as the experienced operators in that 162 of 162 (100%) tests conducted were in complete agreement (data not shown).

Table 1 Influenza A virus strains used to prepare analytical precision study samples. a

Influenza A virus strains

High-positive

Low-positive

Solomon/Islands/2/2006 H1b Brisbane/10/07 H3N2b W1/629-D00015/2009 H1N1c

5.64 × 102 2.45 × 103 1.12 × 103

5.64 × 100 2.45 × 100 1.12 × 100

a

Units are in tissue culture infective doses (TCID)50/mL. Influenza strains supplied by Zeptomextrix, Buffalo, NY. Influenza strain supplied by Dr. Nathan Ledeboer, Dynacare Laboratories, Medical College of Wisconsin. b

c

Table 2 Cepheid Xpert Flu A Panel performance compared with the Luminex RVP assay. Xpert Flu A Panel

Luminex RVP Positive

Negative

Total

Positive Negative

124 12

1 163

125 175

Total

136

164

300

% Sensitivity: 91.2% (95% CI: 85.1; 95.4); % specificity: 99.4% (95% CI: 96.7; 100); positive predictive value: 99.2% (95% CI: 95.6; 100); negative predictive value: 93.1% (95% CI: 88.3; 96.4).

4.3. Comparative results The overall performance of the Xpert Panel relative to the Luminex RVP for detection of influenza A is shown in Table 2. Sensitivity and specificity were 91.2% and 99.4%, respectively, while the PPV was 99.2% and NPV was 93.1%. The overall performance of Table 3 Cepheid Xpert Flu A Panel performance compared to Focus 2009 H1N1 assay. Xpert Flu A Panel

Focus 2009 H1N1 Positive

Negative

Total

Positive Negative

58 5a

0 195

58 200

Total

63

195

258

% Sensitivity: 92.1% (95% CI: 82.4; 97.4); % specificity: 100% (95% CI: 98.5; 100); positive predictive value: 100% (95% CI: 95.0; 100); negative predictive value: 97.5% (95% CI: 94.3; 99.2). a None of the 5 discordant results tested positive for the influenza A virus matrix sequence by the Xpert Flu A Panel.

A.R. Sambol et al. / Journal of Clinical Virology 48 (2010) 234–238

237

Table 4 Cepheid Xpert Flu A Panel performance compared with comparator assays by specimen type. Comparator

% Diagnostic accuracy (95% confidence interval) Sensitivity

Specificity

PPV

NPV

Luminex (influenza A) (n = 300) NA/W (n = 90) NP (n = 210) Combineda

94.6(81.8; 99.3) 89.9(82.2; 95.1) 91.2(85.1; 95.4)

98.1(89.9; 100) 100(97.3; 100) 99.4(96.7; 100)

97.2(85.5; 99.9) 100(96.7; 100) 99.2(95.6; 100)

96.3(87.3; 99.6) 91.7(85.3; 96.0) 93.1(88.3; 96.4)

Focus (2009 H1N1) (n = 258) NA/W (n = 81) NP (n = 177) Combinedb

100 (85.4; 100) 88.6(75.4; 96.2) 92.1(82.4; 97.4)

100 (95.3; 100) 100(97.8; 100) 100(98.5; 100)

100(85.4; 100) 100(92.6; 100) 100(95.0; 100)

100(95.3; 100) 96.4(91.8; 98.8) 97.5(94.3; 99.2)

Abbreviations: PPV = positive predictive value; NPV = negative predictive value; Focus = Focus Diagnostics Influenza A/H1N1 (2009) Real Time RT-PCR test; Luminex = Luminex Molecular Diagnostics xTAGTM Respiratory Viral Panel; NA/W = nasal aspirate/wash specimens; NP = nasopharyngeal swab specimens. a The Luminex poolability results were: sensitivity p = 0.512; specificity p = 0.323; overall agreement p = 0.761. b Focus poolability results were: sensitivity p = 0.311; specificity p = 1.000; overall agreement p = 0.329.

the Xpert Panel relative to the Focus H1N1 for detection of 2009 H1N1 is shown in Table 3. Sensitivity and specificity were greater than 91.2% and 100%, respectively, while the PPV was 100% and NPV was 97.5%. Table 4 shows the performance values of the Xpert Panel versus either the Luminex RVP or the Focus H1N1 when specimen types were differentiated between NA/W and NP. Poolability was assessed across sample types for both influenza A and 2009 H1N1. The results of the analysis show no significant difference in sensitivity or specificity by sample type, thus supporting the combining of results. 5. Discussion The capability of the laboratory to provide timely and accurate results when testing clinical specimens for influenza is paramount for proper treatment and care of afflicted individuals. In the US, human testing for influenza virus may only be conducted using FDA-approved assays and any assay used without this approval must undergo an extensive in-house validation process. Unfortunately, these criteria do not allow for the rapid development of assays during a pandemic situation. In 2008, the FDA developed new guidelines that established the minimum performance characteristics for the emergency approval of in vitro diagnostic devices for detection and differentiation of influenza viruses. One criterion for EUA approval is the demonstration of analytical sensitivity as well as operator reproducibility of the new assay. In this study, both experienced and inexperienced laboratory operators of the test were able to reproduce the same test results; thus, the Xpert Flu A assay was performed successfully by personnel who were inexperienced with PCR testing, which supports its designation as a “moderately complex” test. EUA approval also requires that specimen collection and acceptance criteria, cross-reactivity, sensitivity, specificity, PPV, and NPV of a new assay be compared to assays already marketed. In this study, we compared assay results for the Cepheid Xpert Flu A Panel to the results of the FDA-cleared Luminex Molecular Diagnostics xTAGTM Respiratory Viral Panel (Luminex RVP) assay for the detection of influenza A, and to the results of the EUA-granted Focus Diagnostics Influenza A/H1N1 (2009) Real Time RT-PCR (Focus H1N1) assay for detection of 2009 H1N1. The overall performance of the Xpert Panel relative to the approved comparative assays was ≥91.2% for sensitivity and ≥99.4% for specificity. For these analyses, the number of positive samples selected for study was enriched to meet the FDA requirements for EUA approval; thus, the prevalence rate of 45% was greater than that experienced by testing sites during the pandemic. As a result, the PPV and NPVs obtained for the Xpert Panel relative to the Luminex RVP and relative to the Focus H1N1 should not be consid-

ered to be representative of the assay’s performance in an infected population. At the beginning of the pandemic influenza outbreak, there were no FDA-cleared 510(k) assays that would definitely detect and identify the 2009 H1N1 strain of influenza A. The availability of an EUA process allowed for new assays to be developed, approved, and commercialized rapidly. This helped to alleviate the problem of limited rapid and reliable means for determination of infection in patients. Cepheid cartridges are designed to be run on-demand. The encouraging sensitivity and specificity profile for the Xpert Flu A panel combined with the reproducibility of assay results and ease of use may allow this assay to fill an important gap in performance between RIDTs and the new gold standard of PCR testing. Funding Cepheid, Inc. provided Xpert Flu A reagents and financial support for this study. Ethical approval All study sites were granted waivers of informed consent by their IRBs for this study. Conflict of interest The authors declare no conflicts of interest. Acknowledgements Dana El-Hajjar and David Moran of the University of Nebraska Medical Center, David Hinkle of Mid America Clinical Laboratories, Kathleen Schulte of National Jewish Health and Carolyn Dowell and Jelena Mulin of BayCare Laboratories for their work in the comparative testing of the specimens used in this study. References 1. Ginocchio CC, Zhang F, Manji R, Arora S, Bornfreund M, Falk L, et al. Evaluation of multiple test methods for the detection of the novel 2009 influenza A (H1N1) during the New York City outbreak. J Clin Virol 2009;45:191–5. 2. Sambol AR, Abdalhamid B, Lyden ER, Aden TA, Noel RK, Hinrichs SH. Use of rapid influenza diagnostic tests under field conditions as a screening tool during an outbreak of the 2009 novel influenza virus: practical considerations. J Clin Virol 2010;47:229–33. 3. Hurt AC, Alexander R, Hibbert J, Deed N, Barr IG. Performance of six influenza rapid tests in detection human influenza in clinical specimens. J Clin Virol 2007;39:132–5. 4. United States Food and Drug Administration. Establishing performance characteristics of in vitro diagnostic devices for detection and differentiation of influenza viruses. Draft guidance; 2008. February 15.

238

A.R. Sambol et al. / Journal of Clinical Virology 48 (2010) 234–238

5. Centers for Disease Control and Prevention. Outbreak of swine-like origin influenza A (H1N1) virus infection—Mexico, March–April 2009. Morb Mortal Wkly Rep 2009;58:467–70. 6. Memorandum. Determination Pursuant to § 564 of the Federal Food, Drug, and Cosmetic Act; 26 April 2009.

7. Centers for Disease Control and Prevention. CDC protocol of real time RTPCR for Influenza A (H1N1). April 2009. http://www.who.int/csr/resources/ publications/swineflu/CDCrealtimeRTPCRprotocol 20090428.pdf [accessed 29 September 2009]. 8. Food and Drug Administration Website: http://www.cdc.gov/h1n1flu/guidance/ diagnostic tests.htm.

ID 3369800

Title Validation of the Cepheid Xpert Flu A Real Time RT-PCR detection panel for Emergency Use Authorization

http://fulltext.study/journal/3060

http://FullText.Study

Pages 5