Xpert® MTB/RIF assay for pulmonary tuberculosis and rifampicin resistance in adults

Tuberculosis

TB is caused by the bacterium Mycobacterium tuberculosis and is spread from person to person through the air. TB most commonly affects the lungs (pulmonary TB), but may affect any organ or tissue, such as the brain or bones, outside of the lungs (extrapulmonary TB). Signs and symptoms of pulmonary TB include cough for at least two weeks, fever, chills, night sweats, weight loss, haemoptysis (coughing up blood), and fatigue. Signs and symptoms of extrapulmonary TB depend on the site of disease. TB treatment regimens must contain multiple drugs to which the organisms are sensitive to be effective. The treatment of MDR-TB is complex, usually requiring two years or more of therapy and drugs that are less potent and more toxic than the drugs used to treat drug-susceptible TB. The WHO issues international guidelines for TB treatment which are regularly updated.

Rifampicin resistance

Rifampicin inhibits bacterial DNA-dependent RNA polymerase, encoded by the RNA polymerase gene (rpoB) (Hartmann 1967). Resistance to this drug has mainly been associated with mutations in a limited region of the rpoB gene (Telenti 1993). Rifampicin resistance may occur alone or in association with resistance to isoniazid and other drugs. In high MDR-TB settings, the presence of rifampicin resistance alone may serve as a proxy for MDR-TB (WHO Rapid Implementation 2011). Patients with drug-resistant TB can transmit the infection to others.

Settings of interest

We defined the settings of interest as intermediate-level and peripheral-level laboratories. The latter may be associated with primary health care facilities. We acknowledge that not all peripheral-level laboratories will be able to satisfy the operational requirements recommended for Xpert MTB/RIF, namely an uninterrupted and stable electrical power supply, temperature control, and yearly calibration of the instrument modules. However, Xpert MTB/RIF is most likely to have an impact on patient health when it is used in a setting, such as a primary health care facility, where treatment can be started as soon as possible. The level of laboratory services is not to be confused with the setting where the patient received treatment. The Global Laboratory Initiative Roadmap presents a tiered system to describe laboratory service levels: peripheral, intermediate, and central, each level with its own set of responsibilities (Global Laboratory Initiative 2010). Although three levels are described, the Roadmap recognizes that responsibilities at a given level may vary, depending on the needs of countries and diagnostic strategies. Intermediate-level laboratories typically perform tests such as microscopy, rapid molecular tests, culture, and DST. Peripheral-level laboratories typically perform only smear-microscopy and refer samples or patients in need of further tests, such as rapid molecular testing, culture, or DST, to a higher level laboratory (Global Laboratory Initiative 2010).

It should be noted that in the original Cochrane Review, we described the setting of interest as peripheral-level laboratories based on a different classification system previously in use (WHO Policy Framework 2010).

Role of index test(s)

We were interested in the following purposes for testing:

I. Xpert MTB/RIF for TB detection

A. Xpert MTB/RIF used as an initial test replacing smear microscopy in a population unselected by smear status

B. Xpert MTB/RIF used as an add-on test following a negative smear microscopy result

II. Xpert MTB/RIF for rifampicin resistance detection

A. Xpert MTB/RIF used as an initial test for rifampicin resistance replacing conventional phenotypic DST as the initial test

As mentioned, Xpert MTB/RIF does not eliminate the need for subsequent culture and phenotypic DST, which are required to monitor treatment progress and to detect resistance to drugs other than rifampicin.

Alternative test(s)

In this section, we describe selected alternative tests for detection of TB and rifampicin resistance. For a comprehensive review of these tests, we refer the reader to several excellent resources (Drobniewski 2012; Nahid 2012; UNITAID 2013).

Smear microscopy, which involves the direct examination of sputum smears with Ziehl-Neelsen staining for acid-fast bacilli (M. tuberculosis bacteria), is the most commonly used test for TB detection in resource-limited settings (International Standards 2009). Advantages of smear microscopy include its simplicity, low cost, speed, and high specificity in high TB burden areas. In addition, smear microscopy identifies the most infectious TB patients. Smear microscopy can be performed in basic laboratories. Drawbacks of smear microscopy include the need for specialized training and its relatively low sensitivity, 50% to 60% on average for a direct smear (Steingart 2006a). Although, the sensitivity of microscopy can be improved by approximately 10% with fluorescence (Steingart 2006), a large number of TB cases still go undiagnosed. Smear-negative TB is disproportionately higher in HIV-positive than HIV-negative individuals, accounting for 24% to 61% of all pulmonary cases in people living with HIV (Getahun 2007; Perkins 2007). Microscopy cannot distinguish between drug-susceptible TB and drug-resistant TB.

Nucleic acid amplification tests (NAATs) are molecular systems that can detect small quantities of genetic material (DNA or RNA) from microorganisms, such as M. tuberculosis. A variety of molecular amplification methods are available, of which PCR is the most common. NAATs are available as commercial kits and in-house tests (based on a protocol developed in a non-commercial laboratory) and are used routinely in high-income countries for TB detection. In-house PCR is widely used in developing countries because these tests are less expensive than commercial kits. However, in-house PCR is known to produce highly inconsistent results (Flores 2005). The use of NAATs has recently been recommended as standard practice in the United States (CDC 2009). The main advantage of NAATs is that they can provide results several weeks earlier than culture (CDC 2009). Drawbacks are that these tests are often too expensive and complex for routine use by TB programmes in resource-limited settings. In addition, although the specificity of NAATs is high, some NAATs have shown variable and low sensitivity, especially in sputum smear-negative patients (Flores 2005; Greco 2006; Ling 2008a).

Alternative molecular methods for DST include the commercial line probe assays, INNO-LiPA Rif.TB (Innogenetics, Ghent, Belgium) and GenoType® MTBDRplus assay (Hain LifeScience GmbH, Nehren, Germany). The INNO-LiPA Rif.TB assay targets common mutations in the rpoB gene associated with rifampicin resistance, while the GenoType® MTBDRplus assay also targets the common mutations in katG and inhA genes associated with isoniazid resistance in addition to the mutations in the rpoB gene (UNITAID 2013). Advantages of line probe assays are that they can provide a result for detection of TB and drug resistance in one to two days. Also, they have both high sensitivity (greater than 97%) and high specificity (greater than 99%) for the detection of rifampicin resistance alone, or in combination with isoniazid (sensitivity greater than 90%; specificity greater than 99%), on TB isolates and smear-positive sputum specimens (Ling 2008). Drawbacks are that line probe assays are expensive and must be used in reference laboratories (Nahid 2012). These tests have been endorsed by WHO (WHO Policy Line Probe Assays 2008).

Types of studies

We included primary studies that assessed the diagnostic accuracy of Xpert MTB/RIF for both pulmonary TB and rifampicin resistance, or pulmonary TB alone. Diagnostic accuracy studies are typically cross-sectional in design. However, we also searched for randomized controlled trials (RCTs) and cohort studies. We only included studies that reported data comparing Xpert MTB/RIF to an acceptable reference standard from which we could extract true positive (TP), true negative (TN), false positive (FP), and false negative (FN) values. Xpert MTB/RIF could be assessed alone or together with other tests.

We excluded studies with a case-control design because these types of studies are prone to bias, in particular, studies enrolling patients with severe disease and healthy participants without disease. We also excluded studies reported only in abstracts.

Participants

We included studies that recruited adult or predominantly adult patients, aged 15 years or older, presumed to have pulmonary TB or MDR-TB, with or without HIV infection. Also, we included studies that assessed the diagnostic accuracy of Xpert MTB/RIF using sputum and other respiratory specimens (such as fluid obtained from bronchial alveolar lavage and tracheal aspiration) consistent with the intended use of the manufacturer (Cepheid 2009), and studies from all types of health facilities and all laboratory levels (peripheral, intermediate, and central) from all countries. The majority of included studies provided data on the age of study participants. We considered it highly likely that studies that did not report age data involved all or mostly adults for the following reasons: the vast majority of specimens evaluated with Xpert MTB/RIF were sputum specimens and children have difficulty producing sputum; we excluded data on specimens obtained by gastric aspiration, as this specimen collection method is used mostly for investigating TB in children; we excluded studies that specifically evaluated the use of Xpert MTB/RIF in children; and we performed a sensitivity analysis by dropping studies that did not report age data to check whether the accuracy results changed Sensitivity analyses.

Index tests

Xpert MTB/RIF was the index test under evaluation.

We also compared Xpert MTB/RIF with smear microscopy, either Ziehl-Neelsen microscopy, fluorescence microscopy, or both microscopy methods.

Target conditions

The target conditions were active pulmonary TB and rifampicin resistance.

Reference standards

For TB, acceptable reference standards used solid media (Löwenstein-Jensen, Middlebrook 7H10 or 7H11, or Ogawa media) or a commercial liquid culture system, (such as BACTEC™ 460TB System or BACTEC™ MGIT™ 960 Mycobacterial Detection System, BD, USA; BacT/ALERT® System, bioMérieux, France; or VersaTREK® Mycobacteria Detection & Susceptibility, Thermo Fisher Scientific, USA).

For rifampicin resistance, the reference standards were phenotypic culture-based DST methods recommended by WHO (WHO Policy DST 2008). Acceptable methods were the proportion method performed on solid media (such as Löwenstein-Jensen, Middlebrook 7H10 or 7H11, or Ogawa media), use of a commercial liquid culture system (such as BACTEC™ 460TB System or BACTEC™ MGIT™ 960 Mycobacterial Detection System, BD, USA; BacT/ALERT® System, bioMérieux, France; or VersaTREK® Mycobacteria Detection & Susceptibility, Thermo Fisher Scientific, USA), or both.

Electronic searches

Vittoria Lutje, (VL) the Information Specialist for the Cochrane Infectious Diseases Group, performed searches on three occasions, 25 September 2011, 15 December 2011, and 7 February 2013. Using the strategy described in Appendix 1, she searched the following databases: Cochrane Infectious Diseases Group Specialized Register; MEDLINE; EMBASE; ISI Web of Knowledge; MEDION; LILACS; BIOSIS; and SCOPUS. She also searched the metaRegister of Controlled Trials (mRCT) and the search portal of the WHO International Clinical Trials Registry Platform, to identify ongoing trials. We limited all searches to 2007 onward because the development of Xpert MTB/RIF was completed in 2009 and the first paper describing its clinical use was published electronically in 2009 (Helb 2010). VL performed the searches without language restriction.

Searching other resources

To identify additional published, unpublished, and ongoing studies, we performed the following tasks:

• reviewed reference lists of included articles and review articles identified through the above methods;

• contacted Cepheid, the test manufacturer;

• handsearched WHO reports on Xpert MTB/RIF;

• contacted researchers at the Foundation for Innovative New Diagnostics (FIND), members of the Stop TB Partnership's New Diagnostics Working Group, and other experts in the field of TB diagnosis.

Selection of studies

Two review authors (KRS and DJH) independently scrutinized titles and abstracts identified from electronic literature searches to identify potentially eligible studies. We retrieved the article of any citation identified by either review author for full-text review. KRS and DJH independently assessed articles for inclusion using predefined inclusion and exclusion criteria, and resolved any discrepancies by discussion between the review authors. We listed the excluded studies and the reasons for their exclusion.

We named studies according to the surname of the first author and year of publication. For multicentre studies, the study-naming scheme uniquely identified multiple study centres from within each study (for example, Boehme 2010a; Boehme 2010b), each of which reported data separately for a distinct population at a given study site. Hence, the number of study centres exceeds the number of studies.

Data extraction and management

We extracted data on the following characteristics: 

• author, publication year, study design, case country of residence, country income status classified by the World Bank List of Economies (World Bank 2012), level of laboratory services, type of setting for running Xpert MTB/RIF (clinical or laboratory);

• population, age, gender, HIV status, smear status, and follow-up;

• reference standard;

• Xpert MTB/RIF assay version;

• specimen collection (such as expectorated sputum, induced sputum);

• condition of the specimen (fresh or frozen);

• preparation of the specimen (processed or unprocessed);

• QUADAS-2 items (Whiting 2011);

• data for two-by-two tables for Xpert MTB/RIF, including results reported as uninterpretable (results reported as indeterminate, invalid, error, or no result);

• time to diagnosis (time from specimen collection until there is an available TB result in laboratory or clinic);

• time to treatment initiation (time from specimen collection until time patient starts treatment).

Whenever possible, we extracted TP, FP, FN, and TN values based on one Xpert MTB/RIF result for one specimen provided by one patient. However, in some of the studies, the number of specimens (and Xpert MTB/RIF results) exceeded the number of patients, suggesting that a single patient may have provided multiple specimens. We therefore compared pooled sensitivity and specificity for TB detection in all studies with pooled sensitivity and specificity in the subset of studies that provided one Xpert MTB/RIF result based on one specimen provided by one patient (see Sensitivity analyses).

Concerning the condition of the specimen, although the manufacturer recommends use of fresh specimens, we were aware that studies had been conducted using frozen specimens so we extracted this information as well.

Concerning the definition of smear positivity, as the vast majority of included studies performed Xpert MTB/RIF in intermediate-level or central-level laboratories, we assumed these studies adhered to the revised definition of a new sputum smear-positive pulmonary TB case based on the presence of at least one acid-fast bacillus in at least one sputum sample in countries with a well-functioning external quality assurance system (WHO Policy Smear-positive TB Case 2007). 

We developed a standardized data extraction form and piloted the form with four studies. Based upon the pilot, we finalized the form. Two review authors (KRS and DH) independently extracted data from each study using the final form. We contacted study authors for missing data and clarifications and entered all data into Microsoft® Excel. The final data extraction form is in Appendix 2.

Assessment of methodological quality

We appraised the quality of included studies with the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool (Whiting 2011). QUADAS-2 consists of four domains: patient selection, index test, reference standard, and flow and timing. We assessed all domains for the potential for risk of bias and the first three domains for concerns regarding applicability. We used questions, called signalling questions, for each domain to form judgments about the risk of bias. As recommended, we first developed guidance on how to appraise each signalling question and interpret this information tailored to this review. Then, one review author (KRS) piloted the tool with four of the included studies. Based on experience gained from the pilot, we finalized the tool. Two review authors (KRS and DH) independently assessed the methodological quality of the included studies with the finalized tool. We presented results in the text, graphs, and a table. We did not generate a summary "quality score" because of problems associated with such numeric scores (Juni 1999; Whiting 2005). We explained definitions for using QUADAS-2 in Appendix 3.

Statistical analysis and data synthesis

We performed descriptive analyses for the results of the included studies using Stata 12 (Stata) and presented key study characteristics in Characteristics of included studies. We used data reported in the two-by-two tables to calculate sensitivity and specificity estimates and 95% confidence intervals (CI) for individual studies and to generate forest plots using Review Manager 5. Whenever possible, we included NTM as non-TB for specificity determinations. We chose to use data that were not subject to discrepant analyses (unresolved data), since resolved data after discrepant analyses are a potential for risk of bias (Hadgu 2005).

We carried out meta-analyses to estimate the pooled sensitivity and specificity of Xpert MTB/RIF separately for TB detection (I. A. and I. B.) and rifampicin resistance detection (II. A.). When possible, we determined pooled estimates using an adaptation of the bivariate random-effects model (Reitsma 2005) to allow for a hierarchical structure for the two multicentre studies (Boehme 2010; Boehme 2011). The bivariate random-effects approach allowed us to calculate the pooled estimates of sensitivity and specificity while dealing with potential sources of variation caused by (1) imprecision of sensitivity and specificity estimates within individual studies; (2) correlation between sensitivity and specificity across studies; and (3) variation in sensitivity and specificity between studies. In a few cases, namely TB detection among smear-positive individuals and rifampicin resistance detection (described below), where data were insufficient for bivariate analyses, we performed univariate analyses.

To compare the relative value of Xpert MTB/RIF and smear microscopy, we estimated the difference between their pooled sensitivities and pooled specificities. For this analysis, the specificity of smear was assumed to be 100% (Toman 2004a; Steingart 2006a). We also presented the data in a descriptive plot showing the estimates of sensitivity and specificity of Xpert MTB/RIF compared with those of smear microscopy in studies that reported on both tests.

To determine the value of Xpert MTB/RIF as a replacement test for smear (I. A.), we included studies with unselected individuals presumed to have TB to estimate pooled sensitivity and specificity. To determine the value of Xpert MTB/RIF as an add-on test (I. B.), we estimated its sensitivity and specificity among smear-negative individuals presumed to have TB. We did this by including individual studies that enrolled individuals preselected to be predominantly smear negative as well as studies providing results for unselected smear-negative individuals.

For rifampicin resistance detection, we performed univariate meta-analyses (using all available data) to determine sensitivity and specificity estimates separately. We did this because, in several studies, all patients were rifampicin susceptible (rifampicin resistance negatives), thus contributing data for specificity but not for sensitivity. We also performed a sensitivity analysis using the bivariate random-effects model for the subset of studies that provided data for both sensitivity and specificity.

We estimated all models using a Bayesian approach with non-subjective prior distributions and implemented using WinBUGS (Version 1.4.3) (Lunn 2000). Under the Bayesian approach, all unknown parameters must be provided a prior distribution that defines the range of possible values of the parameter and the likelihood of each of those values based on information external to the data. In order to let the observed data determine the final results, we chose to use low-information prior distributions over the pooled sensitivity and specificity parameters and their between-study standard deviation parameters. The model we used is summarized in the Statistical Appendix together with the WinBUGS program used to implement it (Appendix 4). Information from the prior distribution is combined with the likelihood of the observed data in accordance with Bayes theorem to obtain a posterior distribution for each unknown parameter.

Using a sample from the posterior distribution, we can obtain various descriptive statistics of interest. We estimated the median pooled sensitivity and specificity and their 95% credible intervals (CrI).The median or the 50% quantile is the value below which 50% of the posterior sample lies. We reported the median because the posterior distributions of some parameters may be skewed and the median would be considered a better point estimate of the unknown parameter than the mean in such cases. The 95% CrI is the Bayesian equivalent of the classical (frequentist) 95% CI. (We have indicated 95% CI for individual study estimates and 95% CrI for pooled study estimates as appropriate). The 95% CrI may be interpreted as an interval that has a 95% probability of capturing the true value of the unknown parameter given the observed data and the prior information.

We also estimated the 'predicted' sensitivity and specificity in a future study together with their 95% CrIs. The predicted estimate is our best guess for the estimate in a future study and is the same as the pooled estimate. The CrIs, however, may be different. These values are derived from the predicted region typically reported in a bivariate meta-analysis plot. If there is no heterogeneity at all between studies, the CI (or CrI) around the predicted estimate will be the same as the CI around the pooled estimate. On the other hand, if there is considerable heterogeneity between studies, the CI around the predicted estimate will be much wider than the CI around the pooled estimate. We generated the plots using R (version 2.15.1) (R 2008).

Investigations of heterogeneity

Sensitivity analyses

We performed sensitivity analyses by limiting inclusion in the meta-analysis to: 1) studies that provided data by age that explicitly met the age criterion for participants; 2) studies where consecutive patients were selected; 3) studies where a single specimen yielded a single Xpert MTB/RIF result for a given patient; and 4) studies that explicitly represented the use of Xpert MTB/RIF for the diagnosis of individuals thought to have TB. In order to assess the influence of two large multicentre manufacturer-supported studies on the summary estimates, we performed an analysis excluding these studies (Boehme 2010; Boehme 2011).

Assessment of reporting bias

We chose not to carry out formal assessment of publication bias using methods such as funnel plots or regression tests because such techniques have not been helpful for diagnostic test accuracy studies (Macaskill 2010). However, Xpert MTB/RIF is produced by only one manufacturer and, as a new test for which there has been considerable attention and scrutiny, we believe reporting bias was minimal.

I. Xpert MTB/RIF for TB detection

A. Xpert MTB/RIF used as an initial test replacing smear microscopy in a population unselected by smear status

We have presented forest plots of Xpert MTB/RIF sensitivity and specificity for TB detection for the 27 studies (36 study centres) in Figure 5. Sensitivity estimates ranged from 58% to 100% and specificity estimates ranged from 86% to 100%.

We included 22 of the total 27 studies (8998 participants) in this meta-analysis. We excluded five studies that enrolled primarily only smear-positive or smear-negative patients (Friedrich 2011; Ioannidis 2011; Moure 2011; Van Rie 2013; Williamson 2012). For TB detection, Xpert MTB/RIF pooled sensitivity and specificity were 89% (95% CrI 85% to 92%) and 99% (95% CrI 98% to 99%), respectively (Table 1). The predicted sensitivity and specificity of Xpert MTB/RIF for TB detection were 89% (95% CrI 63% to 97%) and 99% (95% CrI 90% to 100%), respectively. In relation to the pooled values, the wider 95% CrIs around the predicted values suggested some variability between studies, particularly in sensitivity (Table 1). Figure 6 presents the pooled and predicted sensitivity and specificity estimates together with the credible and prediction regions for Xpert MTB/RIF for TB detection. The summary point appears close to the upper left-hand corner of the plot, suggesting high accuracy of Xpert MTB/RIF for TB detection. The 95% credible region around the summary (pooled) value of sensitivity and specificity, the region that contains likely combinations of the pooled sensitivity and specificity, is relatively narrow. The 95% prediction region is wider, displaying more uncertainty as to where the likely values of sensitivity and specificity might occur in a future study.

TB detection, Xpert MTB/RIF compared with smear microscopy

Twenty-one studies (8880 participants) provided data from which to compare the sensitivity of Xpert MTB/RIF and smear microscopy. Figure 7 displays results of smear microscopy versus Xpert MTB/RIF for the individual studies. In the meta-analysis, the sensitivity estimate for Xpert MTB/RIF was the same as the estimate in the meta-analysis in I. A., the difference in the number of studies and participants being due to use of the subset of studies that also reported results by smear status. For smear microscopy, the pooled sensitivity was 65% (95% CrI 57% to 72%). For Xpert MTB/RIF, the pooled sensitivity was 88% (95% CrI 84% to 92%). Therefore, in comparison with smear microscopy, Xpert MTB/RIF increased TB detection among culture-confirmed cases by 23% (95% CrI 15% to 32%).

B. Xpert MTB/RIF used as an add-on test following a negative smear microscopy result

Three studies performed microscopy and, for those patients found to be smear-negative, subsequently ran Xpert MTB/RIF (Ioannidis 2011; Moure 2011; Van Rie 2013). Two of these studies were laboratory-based assessments performed in high-income countries (Ioannidis 2011; Moure 2011). One study performed Xpert MTB/RIF at a primary care clinic in a low-income country (Van Rie 2013). For the three studies, sensitivities ranged from 64% to 83% and specificities from 94% to 100% (Figure 8).

In the meta-analysis, the pooled sensitivity was 67% (95% CrI 60% to 74%) and the pooled specificity was 99% (95% CrI 98% to 99%; 21 studies, 6950 participants; Table 1). Therefore, 67% of smear-negative culture-confirmed TB cases were detected using Xpert MTB/RIF following smear microscopy, increasing case detection by 67% (95% CrI, 60% to 74%) in this group.

Figure 9 presents the pooled and predicted sensitivity and specificity estimates together with the credible and prediction regions for this analysis. The summary point is relatively far from the upper left-hand corner of the plot, suggesting lower accuracy of Xpert MTB/RIF when used as an add-on test than as a replacement test. The 95% credible region around the summary value of sensitivity and specificity is relatively wide.

Investigations of heterogeneity, TB detection

It is possible that the accuracy of Xpert MTB/RIF in clinical subgroups of patients could differ causing heterogeneity in Xpert MTB/RIF performance. We therefore determined sensitivity and specificity estimates for patients grouped by smear or HIV status.

TB detection in smear-positive and smear-negative individuals presumed to have TB

Smear-positive TB

Figure 10 displays the forest plots for studies reporting data for smear-positive patients (24 studies, 33 study centres, 2020 participants). There was little heterogeneity in the sensitivity estimates (range 95% to 100%). In the meta-analysis, the pooled sensitivity for smear-positive, culture-positive TB was 98% (95% CrI 97% to 99%; 21 studies, 1936 participants; Table 2). We did not include Van Rie 2013 in the meta-analysis as this study preselected smear-negative patients, though it did report a sensitivity estimate for Xpert MTB/RIF of 75% among four smear-positive patients. We did not estimate Xpert MTB/RIF pooled specificity in the studies in the smear-positive subgroup because almost all participants were considered to be true TB positive.

Table 2. Impact of covariates on heterogeneity of Xpert MTB/RIF sensitivity and specificity, TB detection

* We selected 30% as a cut-off based on the median proportion of TB cases in the included studies **Results are from a univariate analysis ***Values could not be determined #Results are from a univariate analysis based on three studies
Covariate (Number of studies)		Median pooled sensitivity (95% credible interval)	Median pooled specificity (95% credible interval)
Smear status
Smear + (21)		98% (97, 99)	***
Smear - (21)		67% (60, 74)	99% (98, 99)
Difference (Smear+ minus Smear-)		31% (24, 38)	***
P (Smear+ > Smear-)		1.00	***
HIV status
HIV- (7)		86% (76, 92)	99% (98, 100)
HIV+ (7)		79% (70, 86)	98% (96, 99)
Difference (HIV- minus HIV+)		7% (-5, 18)	1% (-1, 3)
P (HIV- > HIV+)		0.90	0.85
Covariate (number of studies)	Within smear positive	Within smear negative
	Median pooled sensitivity (95% credible interval)	Median pooled sensitivity (95% credible interval)	Median pooled specificity (95% credible interval)
HIV status
HIV-	***	***	***
HIV+ (4)	97% (90, 99)**	61% (40, 81)**	99% (97, 100)#
Difference (HIV- minus HIV+)	***	***	***
P (HIV- > HIV+)	***	***	***
Condition of specimen
Fresh (12)	99% (98, 100)	67% (58, 76)	99% (98, 100)
Frozen (6)	97% (95, 99)	61% (48, 73)	98% (95, 99)
Difference (Fresh minus Frozen)	1% (-0.4, 4)	6% (-9, 22)	1% (-0.4, 4)
P (Fresh > Frozen)	0.92	0.79	0.92
Specimen preparation
Unprocessed (10)	98% (97, 99)	69% (60, 78)	98% (97, 99)
Processed (11)	99% (97, 99)	64% (54, 75)	99% (98, 100)
Difference (Unprocessed minus Processed)	-0.1% (-2, 2)	5% (-9, 18)	-1% (-2, 1)
P (Unprocessed > Processed)	0.45	0.76	0.20
Proportion TB cases in the study
> 30% (12)*	99% (97, 99)	70% (62, 78)	98% (96, 99)
≤ 30% (9)*	98% (96, 99)	61% (50, 73)	99% (98, 100)
Difference (> 30% minus ≤ 30%)	0.5% (-1, 2)	9% (-5, 22)	-1% (-3, 0.2)
P (> 30% minus ≤ 30%)	0.74	0.90	0.05
Country income level
High-income (8)	99% (98, 100)	73% (62, 83)	99% (97, 100)
Low-income and middle-income (13)	98% (97, 99)	64% (56, 73)	99% (97, 99)
Difference (High-income minus Low- and middle-income)	1% (-1, 2)	9% (-5, 22)	0.3% (-1, 2)
P (High-income > Low- and middle-income)	0.88	0.90	0.69

Smear-negative TB

Figure 8 displays the forest plots for studies reporting data for smear-negative patients (24 studies, 33 study centres, 7264 participants). There was considerable variability in sensitivity estimates (range 43% to 100%). Specificity estimates showed less variability (range 86% to 100%). Lawn 2011 yielded the lowest sensitivity. This study used Xpert MTB/RIF as a TB screening test in HIV-infected patients with advanced immunodeficiency enrolling in antiretroviral therapy services. The meta-analysis included 21 studies. The pooled sensitivity estimate for smear-negative, culture-positive TB was 67% (95% CrI 60% to 74%), considerably lower than the pooled sensitivity estimate for smear-positive, culture-positive TB which was 98% (95% CrI 97% to 99%; Table 2).

TB detection in HIV-negative and HIV-positive individuals presumed to have TB

Figure 11 displays the forest plots for studies reporting data for HIV-negative individuals (nine studies, 18 study centres, 2555 participants) and HIV-positive individuals (10 studies, 16 study centres, 2474 participants). Sensitivity was variable in both the HIV-negative subgroup (56% to 100%) and HIV-positive subgroup (0% to 100%).  The small number of participants in several studies may have contributed to some of this variability. Specificity varied less than sensitivity in both subgroups: 96% to 100% in the HIV-negative subgroup and 92% to 100% in the HIV-positive subgroup.

The meta-analysis included seven studies that provided data for both HIV-negative (1470 participants) and HIV-positive (1789 participants) individuals (Boehme 2010; Boehme 2011; Hanrahan 2013; Rachow 2011; Scott 2011; Theron 2011; Van Rie 2013). The pooled sensitivity was 86% (95% CrI 76% to 92%) in the HIV-negative subgroup and 79% (95% CrI 70% to 86%) in the HIV-positive subgroup (Table 2). Corresponding pooled specificities were similar: 99% (95% CrI 98% to 100%) and 98% (95% CrI 96% to 99%), respectively (Table 2). When adjusting for the percentage of smear-positive patients in each study, the impact of HIV decreased suggesting that some of the differences between the HIV-positive and HIV-negative subgroups could be attributed to differences in smear status (Table 2).

TB detection among HIV-positive individuals by smear status

Four studies reported data from which to assess the accuracy of Xpert MTB/RIF in HIV-positive individuals by smear status (Balcells 2012; Carriquiry 2012; Lawn 2011). Among people with HIV, Xpert MTB/RIF pooled sensitivity was 61% (95% CrI 40% to 81%) for smear-negative, culture-positive TB compared with 97% (95% CrI 90% to 99%) for smear-positive, culture-positive TB, a statistically significant result (Table 2). Hence, among people with HIV-TB coinfection, people with HIV infection and smear-positive disease were more likely to be diagnosed with TB using Xpert MTB/RIF than those with HIV infection and smear-negative disease.

Effect of condition of the specimen

As mentioned above, although the manufacturer recommends use of fresh specimens, we were aware that studies had been conducted using frozen specimens; therefore we explored the effect of the condition of the specimen on Xpert MTB/RIF performance. The pooled sensitivities and specificities were slightly higher for fresh specimens compared with frozen specimens within both the smear-positive and smear-negative groups, however there was considerable overlap in the CrIs around these estimates (Table 2).

Effect of specimen preparation

The pooled sensitivity was higher for unprocessed specimens compared with processed specimens in smear-negative patients, though there was considerable overlap in the CrIs around these estimates (Table 2).

Effect of the proportion of culture-confirmed TB cases in the study

For this analysis, we used a cutoff of 30% TB cases because 30% was around the median proportion of TB cases in the included studies. Within smear-negatives, the pooled sensitivity was higher for studies with a higher proportion of TB cases; however there was considerable overlap in the CrIs around these estimates (Table 2).

Effect of country income status

There did not appear to be an important difference in the pooled sensitivity of Xpert MTB/RIF according to country income status after adjusting for smear status (Table 2).

II. Rifampicin resistance detection

A. Xpert MTB/RIF used as an initial test replacing conventional DST

The 24 studies (33 study centres) in this analysis included 555 rifampicin-resistant specimens, median two specimens (range 1 to 250). Two studies accounted for the majority (82%, 455/555) of the rifampicin-resistant specimens (Boehme 2010; Boehme 2011). Seven studies contributed only specificity data (presence of rifampicin-susceptible TB) (Al-Ateah 2012; Ciftci 2011; Hanif 2011; Marlowe 2011; Rachow 2011; Safianowska 2012; Van Rie 2013) but not sensitivity data (presence of rifampicin-resistant TB). Figure 12 shows the forest plots of sensitivity and specificity for this analysis. The individual study centres in the plots are ordered by decreasing sensitivity and decreasing number of true positive results. Although, there was heterogeneity in sensitivity estimates (ranging from 33% to 100%), in general there was less variability among study centres with a higher number of rifampicin-resistant specimens. Specificity showed less variability than sensitivity, ranging from 83% to 100%. The pooled sensitivity and specificity by univariate analysis were 95% (95% CrI 90% to 97%) and 98% (95% CrI 97% to 99%), respectively (Table 1).

Investigations of heterogeneity, rifampicin resistance detection

Sensitivity analyses

For TB detection, we undertook sensitivity analyses by limiting inclusion in the meta-analysis to: 1) studies that provided age data that met our inclusion criterion for adults; 2) studies that used consecutive sampling; 3) studies where a single specimen yielded a single Xpert MTB/RIF result for a given individual; and 4) studies that explicitly tested individuals with presumed TB. We also performed a sensitivity analysis by excluding from the meta-analysis the two large multicentre studies (Boehme 2010; Boehme 2011). These sensitivity analyses made no difference to any of the findings (Table 4).

Other analyses

Application of the meta-analysis to a hypothetical cohort

The Summary of findings tables summarize the findings of the review by applying the results to a hypothetical cohort of 1000 individuals thought to have TB or MDR-TB. We present several different scenarios: for Xpert MTB/RIF used as an initial test for TB detection or as an add-on test following microscopy, the prevalence of TB in the setting or patient subgroup varies from 2.5% to 5% to 10%; and for Xpert MTB/RIF for rifampicin resistance detection, the prevalence of rifampicin resistance in the setting varies from 5% to 15% (5% is estimated to be equivalent to the upper limit for rifampicin resistance prevalence in new cases; 15% is estimated to be the lower limit for rifampicin resistance prevalence among previously treated cases). The consequences of false positive results are likely patient anxiety, morbidity from additional testing and unnecessary treatment, and possible delay in further diagnostic evaluation. The consequences of false negative results are an increased risk of patient morbidity and mortality, and continued risk of community transmission of TB.

Completeness of evidence

This data set involved comprehensive searching and correspondence with experts in the field and the test manufacturer to identify additional studies, as well as repeated correspondence with study authors to obtain additional data and information that was missing from the papers. The search strategy included studies published in all languages. However, we acknowledge that we may have missed some studies despite the comprehensive search. Lastly, the evidence in this review is mostly derived from high TB incidence countries and should be carefully extrapolated to low incidence settings.

Accuracy of the reference standards used

Culture is regarded as the best available reference standard for active TB disease and was the reference standard for TB in this review. Phenotypic culture-based DST methods using WHO-recommended critical concentrations were the reference standards for rifampicin resistance (WHO Policy DST 2008). Concerning the latter, several studies have raised concerns about rapid DST methods, in particular automated MGIT 960, for rifampicin using the recommended critical concentrations. Van Deun 2009 reported that BACTEC 460 and MGIT 960 missed certain strains associated with low-level rifampicin resistance. Furthermore, using Xpert MTB/RIF and gene sequencing, Williamson 2012a identified four patients (three with clinical information available) whose TB isolates contained mutations to the rpoB gene but appeared to be rifampicin susceptible using MGIT 960 (36). In this study, 2/49 (4.1%) patients whose isolates did not have apparent rpoB gene mutations, experienced treatment failure compared with 3/3 (100%) patients whose isolates did have rpoB gene mutations and were deemed rifampicin susceptible with phenotypic methods (Williamson 2012a). Recently, in a study involving retreatment patients, Van Deun and colleagues found that disputed rpoB mutations conferring low-grade resistance were often missed by rapid phenotypic DST, particularly with the MGIT 960 system, but to a minor extent also by conventional slow DST. The authors suggested this may be the reason for the perceived insufficient specificity of molecular DST for rifampicin (Van Deun 2013). In light of these findings, it is unclear whether and to what extent Xpert MTB/RIF might out-perform phenotypic DST methods for rifampicin resistance. Specifically, the determination of the specificity of a molecular DST method based on phenotypic DST alone may underestimate the specificity of a molecular DST.

Quality and quality of reporting of the included studies

The majority of studies used consecutive selection of participants and interpreted the reference standard results without knowledge of Xpert MTB/RIF results. Xpert MTB/RIF results are generated automatically, without requiring subjective interpretation. In general, studies were fairly well reported, though we corresponded with almost all authors for additional data and missing information. We encourage authors of future studies to follow the recommendations in the STARD statement to improve the quality of reporting (Bossuyt 2003).

Completeness and relevance of the review

We noted that most studies performed Xpert MTB/RIF in intermediate-level or peripheral-level laboratories, which are settings that matched the review question. This review included studies using all four generations of Xpert MTB/RIF (G1, G2, G3, G4 cartridges). G4, which is used with software version 4.0 or higher, is now included in all Xpert MTB/RIF kits. This review did not address the use of Xpert MTB/RIF in children or in non-respiratory specimens for the diagnosis of extrapulmonary TB.

TABLES

Domain 1: Patient selection

Risk of bias: Could the selection of patients have introduced bias?

Signalling question 1: Was a consecutive or random sample of patients enrolled? We scored 'yes' if the study enrolled a consecutive or random sample of eligible patients; 'no' if the study selected patients by convenience; and 'unclear' if the study did not report the manner of patient selection or we could not tell.

Signalling question 2: Was a case-control design avoided? Studies using a case-control design were not included in the review because this study design, especially when used to compare results in severely ill patients with those in relatively healthy individuals, may lead to overestimation of accuracy in diagnostic studies. We scored 'yes' for all studies.

Signalling question 3: Did the study avoid inappropriate exclusions? We scored 'yes' if the study included both smear-positive and smear-negative individuals or only smear-negative individuals; 'no' if the study included only smear-positive individuals; and 'unclear' if we could not tell.

Applicability: Are there concerns that the included patients and setting do not match the review question?

We were interested in how Xpert MTB/RIF performed in patients presumed to have pulmonary TB or MDR-TB whose specimens (predominantly sputum specimens) were evaluated as they would be in routine practice, in intermediate-level laboratories or primary health care facilities. We judged 'unclear' concern if Xpert MTB/RIF was run in a central-level laboratory. We assumed a central-level laboratory used highly trained staff. However, we acknowledge, that for some studies, the reason Xpert MTB/RIF was performed in the central-level laboratory was the requirement for a sophisticated laboratory infrastructure to perform culture (the reference standard) not to perform Xpert MTB/RIF. We judged 'high concern' if the majority of specimens were respiratory specimens other than sputum.

Domain 2: Index test

Risk of bias: Could the conduct or interpretation of the index test have introduced bias?

Signallingquestion 1: Were the index test results interpreted without knowledge of the results of the reference standard? We answered this question 'yes' for all studies because Xpert test results were automatically generated and the user was provided with printable test results. Thus, there is no room for subjective interpretation of test results.

Signallingquestion 2: If a threshold was used, was it prespecified? The threshold was prespecified in all versions of Xpert. We answered this question 'yes' for all studies.

For risk of bias, we scored 'low concern' for all studies.

Applicability: Are there concerns that the index test, its conduct, or its interpretation differ from the review question? Variations in test technology, execution, or interpretation may affect estimates of the diagnostic accuracy of a test. However, we judged these issues to be of ‘low concern’ for all studies in this review.

Domain 3: Reference standard

Risk of bias: Could the reference standard, its conduct, or its interpretation have introduced bias?

We considered this domain separately for the reference standard for TB detection and the reference standard for rifampicin resistance.

Signallingquestion 1: Is the reference standard likely to correctly classify the target condition? For pulmonary TB: although culture is not 100% accurate, it is considered to be the gold standard for TB diagnosis. For rifampicin resistance: similarly, although DST by conventional phenotypic methods is not 100% accurate, it is considered to be the gold standard. We answered this question ‘yes’ for all studies.

Signallingquestion 2: (TB) Were the reference standard results interpreted without knowledge of the results of the index test? We scored 'yes' if the reference test provided an automated result (for example, MGIT 960), blinding was explicitly stated, or it was clear that the reference standard was performed at a separate laboratory and/or performed by different people. We scored 'no' if the study stated that the reference standard result was interpreted with knowledge of the Xpert test result. We scored 'unclear' if we could not tell.

Signallingquestion 3: (Rifampicin resistance) We added a signalling question for rifampicin resistance because judgments might differ for TB and for rifampicin resistance, the two target conditions. Were the reference standard results interpreted without knowledge of the results of the index test? We scored 'yes' if the reference test provided an automated result (for example, MGIT 960), blinding was explicitly stated, or it was clear that the reference standard was performed at a separate laboratory and/or performed by different people. We scored 'no' if the study stated that the reference standard result was interpreted with knowledge of the Xpert test result. We scored 'unclear' if we could not tell.

Applicability: Are there concerns that the target condition as defined by the reference standard does not match the question? We judged applicability to be of 'low concern' for all studies for both pulmonary TB and rifampicin resistance.

Domain 4: Flow and timing

Risk of bias: Could the patient flow have introduced bias?

Signallingquestion 1: Was there an appropriate interval between the index test and reference standard? In the majority of included studies, we expected specimens for Xpert and culture to be obtained at the same time when patients were presumed to have TB. However, even if there were a delay of several days or weeks between index test and reference standard, TB is a chronic disease and we considered misclassification of disease status to be unlikely. We answered this question 'yes' for all studies.

Signallingquestion 2: Did all patients receive the same reference standard? We answered this question 'yes' for all studies as an acceptable reference standard (either solid or liquid culture) was specified as a criterion for inclusion in the review. However, we acknowledge that it is possible that some specimens could undergo solid culture and others liquid culture. This could potentially result in variations in accuracy, but we thought the variation would be minimal.

Signallingquestion 3: Were all patients included in the analysis? We determined the answer to this question by comparing the number of patients enrolled with the number of patients included in the two-by-two tables.

Bayesian bivariate hierarchical model

The Bayesian bivariate hierarchical model used for the meta-analyses is summarized below. The hierarchical framework took into account heterogeneity between studies and also between centres within two of the largest studies. The model was derived as an extension of previously described models (Chu 2009; Reitsma 2005). A WinBUGS program to fit this model is provided below. Three independent, dispersed sets of starting values were used to run separate chains. The Gelman-Rubin statistic within the WinBUGS program was used to assess convergence. No convergence problems were observed. The first 3000 iterations were treated as burn-in iterations and dropped. Summary statistics were obtained based on a total of 15,000 iterations resulting from the three separate chains.

Notation: From the j^th centre in the i^th study we extracted the cross-tabulation between the index and reference tests TP_ij, FP_ij, TN_ij, FN_ij. The sensitivity in ij^th study is denoted by S_ij and the specificity by SP_ij. We denote the Binomial probability distribution with sample size N and probability p as Binomial(p,N), the Bivariate Normal probability distribution with mean vector μ and variance-covariance matrix Σ as BVN(μ, Σ), the univariate Normal distribution with mean m and variance s by N(m, s) and the Uniform probability distribution between a and b by Uniform(a,b).

Likelihood Figure 13:

The pooled sensitivity is given by 1/1+exp (-μ₁) and pooled specificity as 1/1+exp (μ₂).

Prior distributions Figure 14:

Meta-regression models:

To examine the impact of a dichotomous covariate (Z) on the pooled sensitivity and specificity parameters, we expressed the logit(sensitivity) and logit(specificity) as linear functions of Z as follows:

μ₁ = a₁+ b₁Z and μ₂ = a₂+ b₂Z

Prior distributions were placed over the coefficients in the linear function: a₁ and a₂˜ N(0,4) and b₁ and b₂˜ N(0,1.39) (Buzoianu 2008). 

 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

# WinBUGS PROGRAM FOR ESTIMATING A BIVARIATE HIERARCHICAL META-ANALYSIS MODEL

# FOR SENSITIVITY AND SPECIFICITY ALLOWING FOR HETEROGENEITY BETWEEN STUDIES

# AND HETEROGENEITY BETWEEN CENTRES WITHIN TWO OF THE STUDIES (BOEHME 2010 and 2011)

 model {

 #############################    BOEHME 2010    #############################

                 for(j in 1:5) {

                                logit(se.q[j])<-q1[j,1]

                                logit(sp.q[j])<-q1[j,2]

                                q1[j,1:2]˜ dmnorm(l[1,1:2], T1[1:2,1:2])

                                pos1[j]<-TP1[j]+FN1[j]

                                neg1[j]<-TN1[j]+FP1[j]

                                TP1[j] ˜ dbin(se.q[j],pos1[j])

                                FP1[j] ˜ dbin(sp.q[j],neg1[j])                               

                }

                 T1[1:2,1:2]<-inverse(SIGMA1[1:2,1:2])

                # Between-centre variance-covariance matrix for Boehme 2010               

                SIGMA1[1,1] <- sigma1[1]*sigma1[1]

                SIGMA1[2,2] <- sigma1[2]*sigma1[2]

                SIGMA1[1,2] <- k1*sigma1[1]*sigma1[2]             

                SIGMA1[2,1] <- k1*sigma1[1]*sigma1[2]             

                prec1[1] ˜ dgamma(2,0.5)

                prec1[2] ˜ dgamma(2,0.5)

                k1 ˜ dunif(-1,1)

                sigma1[1]<-pow(prec1[1],-0.5)

                sigma1[2]<-pow(prec1[2],-0.5)

                # Overall sens/spec across centres in Boehme 2010

                se[1]<-1/(1+exp(-l[1,1]))

                sp[1]<-1/(1+exp(l[1,2]))

                l[1,1:2] ˜ dmnorm(mu[1:2], T[1:2,1:2])

#############################    BOEHME 2011    #############################

                 for(j in 1:6) {

                                logit(se.r[j])<- r1[j,1]

                                logit(sp.r[j])<- r1[j,2]

                                r1[j,1:2]˜ dmnorm(l[2,1:2], T2[1:2,1:2])

                                pos2[j]<-TP2[j]+FN2[j]

                                neg2[j]<-TN2[j]+FP2[j]

                                TP2[j] ˜ dbin(se.r[j],pos2[j])

                                FP2[j] ˜ dbin(sp.r[j],neg2[j])

                }

                 T2[1:2,1:2]<-inverse(SIGMA2[1:2,1:2])

                # Between-centre variance-covariance matrix for Boehme 2011               

                SIGMA2[1,1] <- sigma2[1]*sigma2[1]

                SIGMA2[2,2] <- sigma2[2]*sigma2[2]

                SIGMA2[1,2] <- k2*sigma2[1]*sigma2[2]             

                SIGMA2[2,1] <- k2*sigma2[1]*sigma2[2]             

                prec2[1] ˜ dgamma(2,0.5)

                prec2[2] ˜ dgamma(2,0.5)

                k2 ˜ dunif(-1,1)

                sigma2[1]<-pow(prec2[1],-0.5)

                sigma2[2]<-pow(prec2[2],-0.5)

                # Overall sens/spec across centres in Boheme 2011

                se[2]<-1/(1+exp(-l[2,1]))

                sp[2]<-1/(1+exp(l[2,2]))

                l[2,1:2] ˜ dmnorm(mu[1:2], T[1:2,1:2])

#############################    SINGLE CENTRE STUDIES    #############################

                 for(i in 3:22) {                    

                                logit(se[i]) <- l[i,1]

                                logit(sp[i]) <- l[i,2]

                                pos[i]<-TP[i]+FN[i]

                                neg[i]<-TN[i]+FP[i]

                                TP[i] ˜ dbin(se[i],pos[i])

                                FP[i] ˜ dbin(sp[i],neg[i])

                                l[i,1:2] ˜ dmnorm(mu[1:2], T[1:2,1:2])

                }

 #############################    HYPER PRIOR DISTRIBUTIONS    #############################

                mu[1] ˜ dnorm(0,0.25)

                mu[2] ˜ dnorm(0,0.25)

                T[1:2,1:2]<-inverse(TAU[1:2,1:2])

                # Between-study variance-covariance matrix    

                TAU[1,1] <- tau[1]*tau[1]

                TAU[2,2] <- tau[2]*tau[2]

                TAU[1,2] <- rho*tau[1]*tau[2]  

                TAU[2,1] <- rho*tau[1]*tau[2]  

                tau[1]<-pow(prec[1],-0.5)

                tau[2]<-pow(prec[2],-0.5)

                # prec is the between-study precision in the logit(sensitivity) and logit(specificity)

                prec[1] ˜ dgamma(2,0.5)

                prec[2] ˜ dgamma(2,0.5)

                rho ˜ dunif(-1,1)

                # Pooled sensitivity and specificity          

                Pooled_S<-1/(1+exp(-mu[1]))

                Pooled_C<-1/(1+exp(mu[2]))

                # Predicted sensitivity and specificity in a new study

                l.new[1:2] ˜ dmnorm(mu[],T[,])

                sens.new <- 1/(1+exp(-l.new[1]))

                spec.new <- 1/(1+exp(l.new[2]))

}

 ############################## DATA #####################################

 # DATA WAS READ FROM THREE SEPARATE FILES

# DATA 1 - BOEHME 2010

TP1[]     FP1[]     FN1[]     TN1[]

123       1          24         68

201       0          8          101

136       1          10         185

36         3          7          215

179       0          8          35

END

#row 1 : Azerbaijan

#row 2 : Peru

#row 3 : South Africa, Cape Town

#row 4 : South Africa, Durban

#row 5 : India

 ############################################################################

 # DATA 2 - FROM BOEHME 2011

TP2[]     FP2[]     FN2[]     TN2[]

203       4          26         303

171       3          6          825

201       2          32         669

121       0          24         144

101       16         0          671

136       5          12         234

END

#Boheme 2011

#row 1 : Azerbaijan

#row 2 : Peru

#row 3 : South Africa

#row 4 : Uganda

#row 5 : India

#row 6 : The Philippines

 ############################################################################

# DATA 3 - FROM BOEHME 2011

TP[]        FP[]        FN[]       TN[]

NA          NA          NA          NA

NA          NA          NA          NA

42         0          2          128

11         1          1          147

37         0          15         16

60         1          4          29

44         2          1          84

24         1          1          59

54         0          6          146

41         2          22         479

67         0          15         25

102       17         5          104

42         2          30         320

12         0          0          46

116       4          14         82

27         2          2          58

61         1          8          102

15         1          2          127

58         3          9          107

56         2          6          55

111       19         30         320

31         0          4          68

END

#row 1 : Boheme 2010

#row 2 : Boheme 2011

#row 3 : Al-Ateah 2012

#row 4 : Balcells 2012

#row 5 : Barnard 2012

#row 6 : Bowles 2011

#row 7 : Carriquiry 2012

#row 8 : Cifci 2011

#row 9 : Hanif 2011

#row 10 : Hanrahan 2013

#row 11 : Helb 2010

#row 12 : Kurbatova 2012

#row 13 : Lawn 2011

#row 14 : Malbruny 2011

#row 15 : Marlowe 2011

#row 16 : Miller 2011

#row 17 : Rachow 2011

#row 18 : Safianowska 2012

#row 19 : Scott 2011

#row 20 : Teo 2011

#row 21 : Theron 2011

#row 22 : Zeka 2011

 ############################################################################

Summary

Name: Tom Boyles

Affiliation: University of Cape Town

I certify that I have no affiliations with or involvement in any organization or entity with a financial interest in the subject matter of my feedback.

In the initial version of Steingart et al's systematic review of the Xpert® MTB/RIF assay for pulmonary tuberculosis and rifampicin resistance in adults (Steingart 2013) includes 15 studies where Xpert was used as an initial test replacing smear microscopy, with the majority of patients being drawn from two major studies (Boehme 2010a, Boehme 2011a). My comment relates to the appropriate reference standard for tuberculosis is these studies. The systematic review appraised the quality of included studies with the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) (Whiting 2011) tool which states that estimates of test accuracy are based on the assumption that the reference standard is 100% sensitive and that specific disagreements between the reference standard and index test result from incorrect classification by the index test.

For each of the studies in question the reference standard for tuberculosis is listed as “Löwenstein-Jensen culture and MGIT 960” and the review considers that the reference standard is likely to correctly classify the target condition. There is considered to be low risk of bias or applicability concerns relating to the reference test.

However, in Boehme et al 2010 there were 105 patients with ‘clinical tuberculosis’ who were excluded from the analysis. These patients were negative by the reference standard of Löwenstein-Jensen culture and MGIT 960 and should have been included in the ‘no tuberculosis’ group. In Boehme et al 2011 there were 153 similar patients who were excluded from the analysis.

Neither paper gives justification for the exclusion of these patients who according to QUADAS-2 were negative by the reference standard and should be included in the ‘no tuberculosis’ group. Ideally the systematic review should be amended to include these patients but if the data is unavailable the risk of bias should be acknowledged.

Note from the Editors: In addition to the above feedback, Boyles et al. published a case study in The International Journal of Tuberculosis and Lung Disease which outlined the above arguments, and illustrates this with a case study ( Boyles 2014 ); which the Cochrane authors respond to, in the same journal (see below).

Reply

The review authors thank Boyles et al. for this comment. They raise important points about the selective exclusion of culture negative clinical TB cases in the Boehme studies.

We considered the published case study (Boyles 2014) in detail, and in response we carried out additional analyses to determine whether the Boehme studies unduly influenced the overall findings of this Cochrane review. One way we did this was by repeating the meta-analysis with studies for which we could extract data for all enrolled participants, including patients classified as ‘clinical tuberculosis’ with negative sputum culture. We considered these participants as not having TB. In the new analysis, we found pooled sensitivity and specificity estimates to be similar to those we previously reported.

We published our findings as a response to Boyles et al. in The International Journal of Tuberculosis and Lung Disease (Steingart 2015).