To systematically review evidence of the accuracy of the procalcitonin test for diagnosis of bacterial infection in patients with autoimmune disease.
To systematically review evidence of the accuracy of the procalcitonin test for diagnosis of bacterial infection in patients with autoimmune disease.
The major databases Medline, EMBase, and the Cochrane Library were searched for studies published between January 1966 and October 2011 that evaluated procalcitonin, alone or in comparison with other laboratory markers such as C-reactive protein (CRP), as a diagnostic marker for bacterial infection in patients with autoimmune disease and provided sufficient data to permit construction of 2 × 2 tables.
Nine studies were included in the final meta-analysis. The area under the summary receiver operating characteristic curve values were 0.91 (95% confidence interval [95% CI] 0.88–0.93) for procalcitonin and 0.81 (95% CI 0.78–0.84) for CRP. In general, testing for procalcitonin was highly specific for identifying infectious complications, although it was not as sensitive as testing for CRP. Pooled sensitivity was 0.75 (95% CI 0.63–0.84) for procalcitonin tests and 0.77 (95% CI 0.67–0.85) for CRP tests. Pooled specificity was 0.90 (95% CI 0.85–0.93) for procalcitonin tests and 0.56 (95% CI 0.25–0.83) for CRP tests. The positive likelihood ratio for procalcitonin (7.28 [95% CI 5.10–10.38]) was sufficiently high to qualify procalcitonin testing as a rule-in diagnostic tool, while the negative likelihood ratio (0.28 [95% CI 0.18–0.40]) was not sufficiently low to qualify procalcitonin testing as a reliable rule-out diagnostic tool.
Procalcitonin has higher diagnostic value than CRP for the detection of bacterial sepsis in patients with autoimmune disease, and the test for procalcitonin is more specific than sensitive. A procalcitonin test is not recommended to be used in isolation as a rule-out tool.
It is critically important but can be challenging to distinguish infection from disease flare in febrile patients with autoimmune diseases. Administration of additional or higher dosages of immunosuppressive agents can ameliorate disease flare but can exacerbate infection. Disease flare and infection have similar clinical manifestations and are identified by similar laboratory markers, such as leukocytosis with or without a left shift and elevation of the serum C-reactive protein (CRP) level (1, 2). Moreover, the leukocyte count and the serum CRP level can also be affected by systemic corticosteroid and immunosuppressive therapy (1, 2). Thus, there is an urgent need for a reliable biomarker that provides high sensitivity and specificity for the early discrimination of infection from disease flare in febrile patients with autoimmune diseases.
Procalcitonin is a precursor of calcitonin that is primarily produced by parafollicular or calcitonin cells of the thyroid gland. Serum procalcitonin is normally undetectable (<0.05 ng/ml) in healthy individuals, but its level increases rapidly following bacterial infection (3). In contrast to CRP, expression of procalcitonin is not elevated following noninfectious inflammation or nonbacterial infections, so it may be useful as a marker to distinguish bacterial infection from disease flare in patients with autoimmune diseases (4–12). However, previous studies have demonstrated elevated serum levels of procalcitonin in patients who have certain autoimmune diseases (granulomatosis with polyangiitis [Wegener's] , adult-onset Still's disease [AOSD] , and Kawasaki disease during acute exacerbation ) but no evidence of bacterial infection.
We conducted a systematic review and meta- analysis to quantitatively summarize current evidence regarding the use of serum procalcitonin as a marker to distinguish bacterial infection from disease flare in patients with systemic autoimmune diseases.
We followed standard guidelines and used standard methods for systematic reviews and meta-analyses of diagnostic tests (15).
We performed a comprehensive literature search to identify studies that examined the diagnostic accuracy of procalcitonin in the evaluation of patients with autoimmune diseases, by searching the Medline, EMBase, and Cochrane Library databases for the period of time from 1966 to October 2011. “Procalcitonin” is not listed as a MeSH term, so we initially searched for “procalcitonin” as a text word with no language restriction and then combined “procalcitonin” with keywords for different autoimmune diseases. Thus, the search phrase was as follows: “Procalcitonin AND (systemic lupus erythematosus OR rheumatoid arthritis OR juvenile rheumatoid arthritis OR Felty's syndrome OR autoimmune disease OR lupus vasculitis OR multiple sclerosis OR Sjögren's syndrome OR mixed connective tissue disease OR ankylosing spondylitis OR reactive arthritis OR psoriatic arthritis OR Wegener's granulomatosis OR Churg-Strauss syndrome OR polyarteritis nodosa OR microscopic polyangitis OR microscopic polyangitis OR giant cell arteritis OR Takayasu arteritis OR Henoch-Schönlein purpura OR Behçet's disease OR relapsing polychondritis OR autoimmune thyroiditis OR Kawasaki disease OR amyloidosis OR sarcoidosis OR hereditary periodic fever syndromes OR myasthenia gravis OR Lambert-Eaton myasthenic syndrome OR primary biliary cirrhosis OR bullous pemphigoid OR autoimmune hepatitis OR primary sclerosing cholangitis OR Crohn's disease OR diabetes OR autoimmune hemolytic anemia OR idiopathic thrombopenic purpura OR Goodpasture's syndrome OR pernicious anemia OR Hashimoto thyroiditis). We identified additional references by cross-checking the bibliographies of retrieved full-text articles.
We included studies that evaluated procalcitonin alone or in comparison with other laboratory markers (such as CRP) to diagnose bacterial infection in patients with autoimmune diseases and that provided sufficient data for construction of a 2 × 2 contingency table. Two of the authors (JYW and PHY) independently assessed all titles and abstracts to assure that the inclusion criteria were satisfied. Full-text articles were retrieved if any of the authors considered the abstract suitable. The 2 authors then independently examined the full text of the retrieved studies and assessed the suitability for inclusion. Discrepancies between the 2 authors were resolved by ascertaining assessments from additional authors (SSC and CCL), and the decision for inclusion was made by consensus.
The 2 original authors independently extracted data from each selected study. These included overall characteristics of the study (first author, country, language, and year of publication), characteristics of the patients (age range, clinical setting, population parameters, disease type), availability of data required for reconstruction of a 2 × 2 table (number of participants, sensitivity, specificity, number of cases), and characteristics of the procalcitonin test (cutoff level), study settings, definitions of outcomes, and method used to document infection (microbiologic or clinical). The quality of selected studies was assessed using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) criteria (16). We used data with the highest Youden index ([sensitivity + specificity] − 1) for meta-analysis and sensitivity analysis (17). Data with the highest Youden index, rather than the highest sensitivity, were used.
We used the bivariate model for diagnostic meta-analysis to obtain weighted overall estimates of the sensitivity and specificity of procalcitonin for discriminating fever related to bacterial infection from other causes of fever in patients with autoimmune disease (18). The bivariate approach assumes a bivariate distribution for the logit-transformed sensitivity and specificity. This approach accounts for study size and adjusts for the negative correlation of sensitivity and specificity of the index test that may arise from the use of different thresholds in different studies. A summary receiver operating characteristic (ROC) curve was constructed to summarize the true-positive and false-positive rates of different diagnostic studies (19). Overall sensitivity, specificity, and 95% confidence intervals (95% CIs) were calculated based on the binominal distributions of the true-positive and true-negative rates.
We also calculated I2 statistics to quantify the extent of variation among studies (i.e., heterogeneity) (20). An I2 value of >50% was considered an indication of significant heterogeneity. We also performed a conventional diagnostic odds ratio (OR) meta-analysis. Unadjusted data were used exclusively in all meta-analyses. We used a linear regression of log odds ratio on the inverse root of effective sample size as a test for funnel plot asymmetry in diagnostic meta-analyses. A non-zero slope is suggestive of a significantly small study bias (P < 0.05). We defined cutoff value and underlying disease as a priori clinical and design study characteristics that were potentially relevant covariates. All statistical analyses were conducted using Stata version 11.0 software, notably with the midas and metandi commands. All statistical tests were 2-sided. P values less than 0.05 were considered significant.
Our initial search yielded 151 citations. After exclusion of certain publications based on review of the title and abstract, we retrieved 30 potentially relevant articles for full-text review. Twenty-one of these articles were excluded because related diagnostic tests or outcomes were not reported. A total of 9 citations were thus included in our final analysis (Figure 1). These studies contained data on 975 patients in whom the procalcitonin level was measured and 330 patients in whom the CRP level was measured. The prevalence of bacterial infection was 18.6% among patients in whom procalcitonin was measured (181 of 975 patients) and 25.5% among patients in whom CRP was measured (84 of 330 patients).
We evaluated the quality of the 9 included studies, using the QUADAS criteria (Figure 2). All included studies were prospective, clearly described the selection criteria, and used the same reference criteria for all patients. The 9 included studies enrolled heterogeneous patient populations and may not be representative of the full spectrum of patients with autoimmune diseases. Some studies used clinically documented infection as an outcome definition but did not clearly describe whether clinical diagnosis was independent of the biomarker tests. None of the included studies described whether the physicians were blinded with regard to the index tests when they made the diagnosis of bacterial infection. In addition, none of the included studies explicitly provided explanations for withdrawal, and none showed uninterpretable results.
Table 1 shows the characteristics of the 9 included studies. Six studies evaluated patients with various systemic autoimmune diseases, and 3 studies evaluated patients with a specific autoimmune disease. The outcome definitions were classified as microbiologically documented infection or clinically documented infection. Most of the studies used a LumiTest (Brahms Diagnostica) for procalcitonin measurement. Table 1 also shows the sensitivity and specificity of procalcitonin and CRP as biomarkers. When multiple sensitivity or specificity values at different cutoff values were reported in a study, we used the pair of values that maximized the Youden index.
|Author, year, country (ref)||Disease(s)||Prevalence (no.)||Biomarker||PCT testing system||Cutoff value(s)†||Outcome(s)||Severity of infection||PCT||CRP|
|Eberhard et al, 1997, Germany (6)||Systemic autoimmune diseases||0.04 (397)||PCT||LumiTest‡||PCT ≥0.5||Microbiologically documented infection||Systemic bacterial infection||100||84.0||NA|
|Brunkhorst et al, 2000, Germany (4)||Systemic autoimmune diseases||0.30 (53)||PCT||LumiTest‡||PCT ≥0.5||Clinically documented infection, systemic||Systemic bacterial infection||100||84.0||NA|
|Martinot et al, 2005, France (7)||Acute arthritis||0.26 (42)||PCT, CRP||LumiTest‡||PCT ≥0.3, CRP ≥50||Microbiologically documented infectious arthritis||Generalized and local bacterial arthritis||72.7||93.5||100||40.0|
|Scire et al, 2006, Italy (8)||Systemic autoimmune diseases||0.45 (44)||PCT, CRP||PCT LIA (LumiTest)‡||PCT ≥0.5, CRP ≥60||Microbiologically documented infection||Systemic and local infection||75.0||75.0||95.0||8.30|
|Tamaki et al, 2008, Japan (10)||Systemic autoimmune diseases||0.29 (99)||PCT||SphereLight§||PCT ≥0.53||Clinically and microbiologically documented infection||SIRS and non-SIRS||51.7||97.1||NA|
|Chen et al, 2009, Taiwan (5)||Adult-onset Still's disease||0.32 (38)||PCT, CRP||VIDAS PCT assay¶||PCT ≥1.4, CRP ≥101||Microbiologically documented infection||Systemic and local infection||100||100||73.1||83.3|
|Suzuki et al, 2009, Japan (9)||Neuro-Behçet's disease||0.36 (11)||PCT||NA||PCT ≥0.5||Microbiologically documented bacterial CNS infection||Bacterial meningitis and brain abscess||90.0||90.0||NA|
|Sleglova et al, 2010, Czech Republic (12)||Systemic autoimmune diseases||0.20 (212)||PCT, CRP||LumiTest‡||PCT ≥0.5, CRP ≥20||Clinically and microbiologically documented infection||Systemic and local infection||52.4||94.0||76.2||67.9|
|Joo et al, 2011, Korea (11)||Systemic autoimmune diseases||0.41 (79)||PCT, CRP||VIDAS PCT assay¶||PCT ≥0.09, CRP ≥71.8||Clinically documented infection||SIRS and non-SIRS||81.3||78.7||71.9||68.1|
Table 2 shows the pooled sensitivity and specificity analysis for procalcitonin and CRP. Analysis of discrimination indicated that procalcitonin had a greater area under the ROC curve value (0.91 [95% CI 0.88–0.93]) compared with CRP (0.81 [95% CI 0.78–0.84]) (Figure 3). Procalcitonin had greater specificity than CRP, but CRP had greater sensitivity than procalcitonin. Procalcitonin had a high positive likelihood ratio (7.28 [95% CI 5.10–10.38]), making it an excellent rule-in test for the diagnosis of infectious complications in patients with autoimmune diseases. In contrast, CRP had a low positive likelihood ratio (1.76 [95% CI 0.88–3.49]) and was unsuitable as a rule-in test. Neither marker had a negative likelihood ratio of <0.2, so neither was suitable as a rule-out test. The diagnostic ORs for procalcitonin and CRP were 25.2 (95% CI 15.4–41.5) and 5.06 (95% CI 1.7–14.7), respectively (Table 2 and Figure 4). There was substantial heterogeneity among the studies for CRP (I2 = 68.3% [95% CI 18.2–87.7]) but not for procalcitonin (I2 = 0.0% [95% CI 0.0–64.8]).
|Variable (refs.)||No. of studies||Sensitivity (95% CI)||Specificity (95% CI)||Positive LR (95% CI)||Negative LR (95% CI)||AUROC (95% CI)||Diagnostic OR (95% CI)||I2 (95% CI)||Publication bias, Egger's test P value|
|Overall analysis (4–12)||9||0.75 (0.63–0.84)||0.90 (0.85–0.93)||7.28 (5.10–10.38)||0.28 (0.18–0.40)||0.91 (0.88–0.93)||25.2 (15.4–41.5)||0.0 (0.0–64.8)||0.021|
|Systemic autoimmune disease population (4, 6, 8, 10–12)||6||0.73 (0.58–0.84)||0.89 (0.82–0.94)||6.76 (4.47–10.20)||0.31 (0.20–0.48)||0.91 (0.88–0.93)||22.6 (13.3–38.3)||0.0 (0.0–74.6)||0.049|
|AOSD excluded (4, 6, 7, 9)||4||0.81 (0.49–0.95)||0.86 (0.78–0.91)||5.83 (3.57–9.53)||0.22 (0.06–0.73)||0.88 (0.85–0.90)||27.0 (6.1–119.2)||65.9 (0.0–88.4)||0.082|
|Cutoff 0.5 ng/ml (4–9, 12)||7||0.76 (0.56–0.89)||0.88 (0.82–0.92)||6.19 (4.43–8.39)||0.27 (0.14–0.53)||0.90 (0.87–0.92)||25.6 (15.0–43.5)||56.2 (7.74–79.2)||0.401|
|Higher cutoff value (>0.5 ng/ml) (5, 7, 10)||3||0.42 (0.31–0.54)||0.95 (0.89–0.98)||4.62 (0.58–37.11)||0.52 (0.38–0.72)||0.58 (0.47–1.00)||8.85 (0.98–79.66)||71.2 (2.1–91.5)||0.044|
|Overall analysis (5, 7, 8, 11, 12)||5||0.77 (0.67–0.85)||0.56 (0.25–0.83)||1.76 (0.88–3.49)||0.41 (0.24–0.67)||0.81 (0.78–0.84)||5.06 (1.7–14.7)||68.3 (18.2–87.7)||0.788|
We performed several subgroup analyses to examine the source of heterogeneity, and Table 2 shows specific test accuracy indices that may be of interest to clinicians. After restriction of analysis to studies with a cutoff value of 0.5 ng/ml, the negative likelihood ratio improved (0.27 [95% CI 0.14–0.53]), indicating that an infectious complication is unlikely when the serum level of procalcitonin is <0.5 ng/ml. Because the use of one standard cutoff point may lead to loss of diagnostic information, we sought to perform subgroup analysis with 2 sets of cutoff points, one with high sensitivity and one with high specificity. Three studies provided diagnostic accuracy indices on cutoff points of >0.5 ng/ml. Pooled results from these studies showed greatly improved specificity (0.95 [95% CI 0.89–0.98]). Only 2 studies provided results on cutoff points of <0.5 ng/ml; therefore, we could not perform a subgroup analysis for low cutoff points. The diagnostic accuracy indices of procalcitonin in the 6 studies with mixed autoimmune disease populations did not differ significantly from the results of all 9 studies. Considering that procalcitonin expression may be elevated in some vasculitis syndromes and AOSD, we performed an additional analysis by excluding 5 studies containing patients with AOSD. The sensitivity of procalcitonin did improve (0.81 [95% CI 0.49–0.95]) among patients with autoimmune diseases other than AOSD.
We evaluated the use of procalcitonin and CRP as biomarkers to identify bacterial infection as the cause of fever in patients with autoimmune diseases, by performing a meta-analysis of 9 prospective studies. Our results indicate that procalcitonin had higher diagnostic value than CRP for differentiating between bacterial infection and disease flare in patients with autoimmune diseases. In particular, the area under the ROC curve values were 0.91 for procalcitonin and 0.81 for CRP. In general, procalcitonin was more specific than CRP as a marker of bacterial infection, and CRP was more sensitive than procalcitonin as a marker of bacterial infection. Procalcitonin had a high positive likelihood ratio, making it sufficient to serve as a rule-in biomarker. In contrast, because CRP had a low positive likelihood ratio, CRP testing may lead to a high false-positive rate if used in isolation. Both markers had a suboptimal negative likelihood ratio and thus are not suitable for the exclusion of bacterial infection in febrile patients with autoimmune diseases.
This study provides evidence that measurement of procalcitonin is a clinically useful procedure for identification of the cause of fever in patients with autoimmune diseases. A previous meta-analysis that compared procalcitonin and CRP in the diagnosis of bacterial infections in hospitalized patients demonstrated that procalcitonin testing was more sensitive (88% versus 75%) and more specific (81% versus 67%) than CRP testing (21). A more recent meta-analysis that evaluated the expression of procalcitonin and CRP in patients after surgery or trauma showed that the area under the summary ROC curve for procalcitonin was better than that for CRP (22).
Our finding of a suboptimal sensitivity of procalcitonin (75%) in identifying bacterial infections in patients with autoimmune disease may be attributable to several factors. For instance, the studies we included used heterogeneous groups of patients as controls; some studies used patients with autoimmune disease flares, and others used healthy individuals. Previous observational studies have shown that procalcitonin expression may be elevated in the presence of several systemic vasculitis syndromes (granulomatosis with polyangiitis [Wegener's] , Kawasaki disease , AOSD , or Goodpasture's syndrome ), even in the absence of bacterial infection.
Different cutoff levels are needed to optimize the discriminative capability of procalcitonin for different autoimmune diseases. However, this requires a large study or studies of patients with the same autoimmune disease. Until the results of such studies are available, we recommend using a standard procalcitonin cutoff value of 0.5 ng/ml, which has reasonable sensitivity (76%) and specificity (88%).
Our finding of suboptimal sensitivity of procalcitonin in identifying bacterial infections in febrile patients with autoimmune disease may also be attributable to the imperfect ascertainment of outcomes of infectious complications. In particular, some of the analyzed studies used only clinical criteria to document infection. There is no gold standard test for bacterial sepsis, so an interventional trial that compares procalcitonin-guided treatment and conventional treatment, as previously reported for respiratory tract infection (25), will provide stronger evidence than that provided by the current observational studies. Another factor that can explain the suboptimal sensitivity of procalcitonin may be the procalcitonin testing system used in the included studies. Five of the 9 included studies used a procalcitonin line immunoassay (LumiTest) with a reported functional sensitivity of 0.5 ng/ml (coefficient of variation 20%). Values of <0.5 ng/ml (a cutoff point that is still of clinical importance) may lack precision. Only 2 studies used VIDAS assays (bioMérieux), which have higher precision (functional sensitivity of 0.09 ng/ml), and none used the most advanced Kryptor procalcitonin assay (Brahms Diagnostica), which has a reported functional sensitivity of 0.06 ng/ml. With use of these highly sensitive assays, the performance of procalcitonin testing in differentiating bacterial infection from autoimmune disease flare may be further improved.
The current study has strengths and limitations. A major strength is that we reported likelihood ratios in addition to sensitivity, specificity, and the area under the ROC curve. In contrast to sensitivity and specificity, likelihood ratios are less likely to change with the prevalence of the disorder, can be compared among different studies, and can be used to calculate posttest probability for a target disorder (26). One of the major limitations of our study is that we included studies that examined patients with a single autoimmune disease, such as AOSD (5), neuro-Behçet's disease (9), and acute arthritis (7). This may have led to between-study heterogeneity. We decided to include these studies because they may have increased the precision of our meta- analysis results, and because our meta-regression indicated that the results of the overall analysis did not significantly differ when those subgroups were included.
In conclusion, the results of our meta-analysis indicate that procalcitonin has a higher diagnostic value than CRP for differentiating bacterial infection from disease flare in febrile patients with autoimmune disease. Analysis of the pooled data suggests that procalcitonin is a more specific indicator of bacterial infection than CRP, but that CRP is a more sensitive indicator of bacterial infection than procalcitonin. Thus, procalcitonin testing should not be used in isolation as a rule-out tool. Future studies should investigate whether a procalcitonin-guided treatment algorithm provides better outcome in patients with autoimmune disease who present with fever.
All authors were involved in drafting the article or revising it critically for important intellectual content, and all authors approved the final version to be published. Dr. Chang had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study conception and design. Wu, S.-H. Lee, Shen, Yo, Cheng.
Acquisition of data. Hsieh.
Analysis and interpretation of data. Chan, C.-C. Lee, Chang.
We thank Pei-Shan Hsieh (Medical Wisdom, Inc.) for assistance with the statistical analysis.