SEARCH

SEARCH BY CITATION

Abstract

  1. Top of page
  2. Abstract
  3. PATIENTS AND METHODS
  4. RESULTS
  5. DISCUSSION
  6. AUTHOR CONTRIBUTIONS
  7. REFERENCES

Objective

To establish a set of clinical and paraclinical criteria potentially useful as a diagnostic screening tool for polyarteritis nodosa (PAN).

Methods

The abilities of individual descriptive items to predict a diagnosis of PAN were evaluated by screening available data from 949 patients from the French Vasculitis Study Group database, including 262 with PAN and 687 with control vasculitides. Selected items were tested in a logistic regression model to establish a minimal set of nonredundant PAN-predictive criteria. The discriminative accuracy of these items and of the American College of Rheumatology (ACR) 1990 criteria were assessed by reapplying them to the initial patient sample and a subgroup restricted to PAN and microscopic polyangiitis (MPA) patients. A computer simulation procedure was conducted on artificially generated patient data to evaluate the usefulness of these criteria in predicting a diagnosis of PAN.

Results

The analysis resulted in the retention of 3 positive predictive parameters (hepatitis B virus antigen and/or DNA in serum, arteriographic anomalies, and mononeuropathy or polyneuropathy) and 5 negative predictive parameters (indirect immunofluorescence detection of antineutrophil cytoplasmic antibody; asthma; ear, nose, and throat signs; glomerulopathy; and cryoglobulinemia) for the criteria set. These criteria yielded 70.6% sensitivity for all control vasculitides and 89.7% for MPA controls, with 92.3% specificity for all controls and 83.1% for MPA controls. The discriminant abilities of this set of items outperformed the ACR 1990 criteria in all analytical situations, showing better robustness to variations in the prevalence of individual vasculitides.

Conclusion

The use of positive and negative discriminant criteria could constitute a sound basis for developing a diagnostic tool for PAN to be used by clinicians. Further prospective analyses and validations in different populations are needed to confirm these items as satisfactory diagnostic criteria.

The systemic vasculitides are a heterogeneous group of diseases that have blood vessel inflammation as a common trait. In 1990, relying on many previous efforts, the American College of Rheumatology (ACR) proposed a set of classification criteria that were selected by a panel of experts based on an analysis of multicenter patient data (1). Seven distinct entities were thus characterized, including polyarteritis nodosa (PAN) (2). Although the primary purpose of the ACR 1990 criteria was to standardize the classification of vasculitis patients to facilitate communication among researchers (3), their good discriminative accuracy indicated by the initial assessments, with sensitivities of 71–94% and specificities of 87–92% for different types of vasculitides, suggested a potential usefulness for diagnostic prediction (4).

Subsequent evaluations of the ACR 1990 criteria sets used for diagnostic screening under routine clinical conditions yielded inconsistent and unsatisfactory results, with positive predictive values ranging from 17% to 75% for different types of vasculitides and high percentages of false-positive diagnoses (4). Further assessments, which extended to other classification systems, such as the Chapel Hill Nomenclature (5, 6), were equally disappointing, highlighting the necessity of developing separate criteria for classification and diagnostic purposes (7).

We conducted a prospective analysis of the French Vasculitis Study Group (FVSG) patient database with the intention of evaluating the feasibility of establishing a minimal set of predictive clinical and paraclinical features that could serve as a screening tool for the diagnosis of PAN in situations in which the clinical picture is suggestive of systemic vasculitis. Because the ACR 1990 criteria do not distinguish between PAN and microscopic polyangiitis (MPA), which was formally defined by the Chapel Hill Consensus Conference Nomenclature of systemic vasculitides in 1994 (8), a secondary objective of our analysis was to achieve better discrimination between PAN and MPA.

PATIENTS AND METHODS

  1. Top of page
  2. Abstract
  3. PATIENTS AND METHODS
  4. RESULTS
  5. DISCUSSION
  6. AUTHOR CONTRIBUTIONS
  7. REFERENCES

Analytical approach.

The analytical design comprised 2 distinct stages. The first stage was to select a minimum set of low-redundant items of positive and/or negative predictive value for PAN from among those exhibiting the highest individual accuracy in distinguishing PAN from other systemic vasculitides. The selection of this set relied on clinical judgment supported by a combination of univariate and multivariate statistical analyses of clinical and paraclinical items used to describe characteristics of patients in the FVSG database. During the second stage, the PAN-predictive abilities of the selected set of criteria were evaluated through an unsupervised computer simulation procedure designed to reproduce the case-based aspect of clinical diagnostic reasoning. During both analytical stages, the selected items were compared with the ACR 1990 criteria for the classification of PAN (2), which is considered to be the most reliable reference criteria to date.

Analysis of the capabilities of the available clinical and paraclinical items to predict PAN.

The FVSG patient database contains a large set of clinical and paraclinical items designed to describe the characteristics of the systemic vasculitides with a good level of detail. Our analysis relied on patient information extracted from the FVSG database, after filtering for missing items and secondary vasculitides associated with other systemic diseases, such as rheumatoid arthritis and systemic lupus erythematosus. This procedure selected 949 patients with primary systemic vasculitides for which definitive diagnostic evidence was available. In all cases, the diagnoses were based on compatible clinical manifestations, biochemical parameters, including results of antineutrophil cytoplasm antibody (ANCA) testing, histologic analysis, and when available, angiography. Histologic confirmation of the clinical diagnosis relied on the same elements as used in the ACR 1990 analysis (1, 2) and was a mandatory selection criterion. This criterion was used only as diagnostic reference to ensure a reliable assessment of the available clinical and paraclinical items.

The selected sample of 949 patients had the following distribution of vasculitis types: 262 (27.6%) had PAN, among whom 108 had hepatitis B virus (HBV)–related PAN (41.2% of all PAN), 256 (27%) had Wegener's granulomatosis (WG), 207 (21.8%) had MPA, 150 (15.8%) had Churg-Strauss syndrome (CSS), 18 (1.9%) had cryoglobulin-associated vasculitis, and 56 (5.9%) had other primary systemic vasculitides.

Analysis of these data in our search for a minimal set of low-redundant PAN-predictive items, was conducted in 2 steps. During the first step, the entire list of over 100 clinical and paraclinical items used to describe the characteristics of the patients in the FVSG database, including all of the ACR 1990 criteria, was subjected to univariate analysis to assess the individual discriminative value of each available feature. Among these items, the presence of HBV surface antigen, one of the ACR 1990 criteria, was replaced by markers reflecting active HBV replication, such as the detection of hepatitis B envelope antigen and/or DNA of >105 copies/ml in serum (9). An indirect immunofluorescence assay was used to test for ANCA according to the recommendations of the European Vasculitis Study Group (10). Because the ANCA specificity (myeloperoxidase or proteinase 3) was not systematically determined, it was ignored.

The univariate analysis assessed the strength of individual associations between the diagnosis of PAN and the available clinical or paraclinical features by relying on a normalized, pairwise, mutual information measure, which is a well-established entropic approach for quantifying mutual dependence (e.g., positive or negative) between variables (11). A short presentation of the mutual information measure, together with other formal aspects pertaining to the analytical strategy of the study, is provided in Supplementary materials posted online at http://corneliu.henegar.info/projects/PAN/arthritis_rheumatism_2008.htm. The univariate analysis enabled quantification of the usefulness of the information provided by each clinical and paraclinical item for diagnosing PAN and allowed the ranking of available items by the decreasing order of their PAN-predictive value.

To establish a minimal set of low-redundant PAN-predictive criteria, we conducted an exploratory multivariate analysis that relied on available parameters to build a logistic regression model through a forward inclusion approach based on the R2 criterion. The inclusion procedure was directed by clinical judgment and by the PAN-predictive value of individual items, as determined by the univariate analysis, and was reiterated until the logistic regression model could no longer be significantly improved by further inclusion of additional items. This analysis was conducted in 2 distinct situations, one in which all non-PAN vasculitides were considered as controls and the other in which controls were restricted to MPA, in order to favor the selection of discriminant items capable of distinguishing between PAN and MPA, which was not differentiated by the ACR 1990 analysis. Also, taking into consideration the current trend toward a reduction in the incidence of HBV infection as a result of systematic vaccination programs as well as its diminishing association with PAN, the discriminant performance of these criteria was evaluated separately in a subgroup of vasculitides that included only HBV-negative PAN.

The resulting FVSG set of criteria, which were selected by multivariate analysis, was further used to derive a set of relevant positive and negative association rules based on the implementation (12) of a decision-tree inference algorithm (13) designed to optimize the ratio between the accuracy of diagnostic prediction and its cost (e.g., the number of required items). This analysis was undertaken to devise a tool for future clinical use of this set of criteria by exposing the most relevant association rules between selected items. Finally, the PAN-predictive accuracy of the FVSG set of criteria and the ACR 1990 classification criteria were comparatively assessed in terms of sensitivity and specificity by receiver operating characteristic (ROC) curve analysis (14) in each of the above-mentioned analytical situations.

Computer simulation of the PAN-predictive abilities of the FVSG criteria and the ACR 1990 criteria.

During the second analytical stage, computer simulations were run to evaluate the PAN-predictive abilities of the 2 sets of criteria under various conditions, which were simulated through artificially generated vasculitis patient data. The computer simulation procedure was designed to reproduce the case-based aspect of medical diagnostic reasoning, which attempts to ascribe unambiguous labels (e.g., corresponding to distinct pathologic entities) to clusters of cases with similar clinical and paraclinical features. The aim of this simulation was to test the dependence of the PAN-predictive performances of these criteria sets on the prevalence of individual vasculitides in the patient samples analyzed. The Boolean aspect of the presence or absence of clinical and paraclinical features in vasculitis patients suggested the possibility of relying on a model of aggregated dependent Bernoulli trials to represent the real joint distributions of the clinical and paraclinical parameters specific for each form of vasculitis. To ensure good reproducibility of the computer-simulation results, we considered 2 distinct approaches to quantifying and expressing marginal distributions and dependencies between individual parameters.

The first approach relied on the Bahadur-Lazarsfeld theoretical framework, which computes a complete representation of the joint distribution of a set of n correlated Bernoulli trials (e.g., corresponding to n clinical or paraclinical items) through an expansion of a binomial law (15). A short formal presentation of the Bahadur-Lazarsfeld framework is provided in Supplementary materials posted online at http://corneliu.henegar.info/projects/PAN/arthritis_rheumatism_2008.htm. Despite its good theoretical accuracy, a major drawback of the Bahadur-Lazarsfeld expansion is its requirement of a high number of dependency parameters (e.g., correlation parameters expressing dependencies between Bernoulli trials from the second order to the nth order), which challenges the computational tractability of the model, even for a moderate number of trials. These considerations suggested the usefulness of a recently proposed theoretical solution, which relies on the maximum entropy principle to optimize the inference of missing parameters of a truncated Bahadur-Lazarsfeld expansion (e.g., which considers as input parameters only marginal probabilities and second-order correlations) (16), thereby allowing for a highly precise reproduction of the clinical and paraclinical characteristics of real vasculitis patients in artificially generated data. The required marginal probabilities and the second-order correlations between clinical and paraclinical items were computed from the patient sample extracted from the FVSG database.

The second approach used to generate artificial patient data relied on a maximum-spanning tree (MST) dependence-modeling technique, which approximates dependencies between individual parameters by arbitrarily limiting them to those expected to have the most impact on the results (17, 18). The principle of the MST dependence-modeling technique resides in estimating the joint distribution of item presence by relying on an MST representation of their strongest interdependencies. Further details are provided in Supplementary materials posted online at http://corneliu.henegar.info/projects/PAN/arthritis_rheumatism_2008.htm. The marginal probabilities of clinical and paraclinical items and the pairwise mutual information coefficients, which are required by the MST approach, were computed from the FVSG patient sample.

The generation of artificial vasculitis patient data aimed to reflect 2 types of situations. The first was a reference situation, in which patient data were artificially generated by using either the maximum entropy correction of a truncated Bahadur-Lazarsfeld expansion or the MST method to simulate intricate interdependencies between various clinical and paraclinical parameters seen in the FVSG patient database. In this situation, the relative frequencies of the 4 main types of vasculitides represented within artificially generated patient samples were chosen to reflect the prevalence reported in the French population: 34.03% PAN, 27.83% MPA, 26.27% WG, and 11.86% CSS (19). In the second situation, the relative frequencies of PAN cases were modified to test the effect of variations in vasculitis prevalence on the predictive performances of the 2 sets of criteria. To this end, the relative frequency of PAN patients was arbitrarily reduced to 10% of all generated cases, while the relative frequencies of the other 3 vasculitides were increased to 30% each.

The artificial patient data thus generated were further used in a computer simulation to evaluate the usefulness of the 2 sets of criteria in screening potential vasculitis patients for a positive diagnosis of PAN. To achieve this goal, we relied on a combination of an unsupervised hierarchical clustering approach, used to group artificially generated cases based on the similarities of their clinical and paraclinical profiles, and a supervised labeling procedure, which assigns to each resulting cluster of similar cases the true label of the cases that form the majority of its content.

The clustering approach starts by grouping the two most similar cases together to form a first cluster and then reiterating the agglomerative procedure until all cases are collected in a single cluster, thereby generating a new partition of cases into clusters at each iterative step. The choice of the optimal partition of clusters (e.g., reflecting the actual distribution of distinct pathologic entities in the analyzed patient data) is a fundamental issue in unsupervised learning. A popular solution to this problem is to simplify it by finding the partition that provides the best tradeoff between the homogeneity of the clusters and their isolation on the partition (20). Although there is no best approach to fit all situations, the computation of the Silhouette index, a well-understood partition quality indicator, was shown to be a simple yet robust strategy for predicting optimal clustering partitions (ref.21, and Supplementary materials posted online at http://corneliu.henegar.info/projects/PAN/arthritis_rheumatism_2008.htm).

After identifying the optimal partition and labeling its clusters, the predictive abilities of the 2 sets of criteria were evaluated by computing estimations of sensitivity, specificity, positive predictive value, and negative predictive value from the contingency table, reflecting the attribution of cases to each type of vasculitis. The differences between the estimated performances of the ACR and the FVSG criteria sets, which were computed from 30 independent iterations of the simulation procedure, were assessed for statistical significance by chi-square test, comparing the distributions of discrete variables, and by Student's paired t-test, evaluating mean values from continuous distributions. All statistical analyses and simulations were conducted with the use of SPSS software version 13.0 (SPSS, Chicago, IL) and the R software environment for statistical computing (22).

RESULTS

  1. Top of page
  2. Abstract
  3. PATIENTS AND METHODS
  4. RESULTS
  5. DISCUSSION
  6. AUTHOR CONTRIBUTIONS
  7. REFERENCES

The logistic regression analysis, which considered all non-PAN vasculitides as controls, retained a set of 8 minimally redundant PAN-predictive items (Table 1), including 3 parameters that were positively associated and 5 parameters that were negatively associated with PAN (Table 2). When restricted to the subgroup of HBV-negative PAN patients, this analysis confirmed the PAN-predictive abilities of 1 positively associated parameter and 5 negatively associated parameters (Table 2). Combining these items yielded sensitivities of 70.6% for all PAN cases and 76.6% for HBV-negative PAN, with specificities of 92.3% and 88.9%, respectively. When controls were restricted to MPA alone, the nonredundant PAN-predictive abilities were confirmed for only 4 of the previously identified items, 2 were positively associated with a diagnosis of PAN, and 2 were more frequent in MPA patients (Table 2). One positively associated parameter and 2 negatively associated parameters (Table 2) showed significant nonredundant PAN-predictive abilities when considering only the subgroup of HBV-negative PAN with controls restricted to MPA alone. The combined sensitivity of these items was 89.7% for all PAN cases and 83.1% for HBV-negative PAN, with specificities of 83.1% and 83.6%, respectively.

Table 1. The FVSG minimal set of low-redundant PAN-predictive criteria derived from analysis of the FVSG patient database*
CriterionPAN associationDefinition
  • *

    HBV = hepatitis B virus; HBeAg = hepatitis B envelope antigen; ANCA = antineutrophil cytoplasmic antibody; ENT = ear, nose, and throat.

  • Significant positive or negative association with a diagnosis of polyarteritis nodosa (PAN) among patients in the French Vasculitis Study Group (FVSG) database.

1. HBV infectionPositiveMarkers reflecting active HBV replication, such as the presence of HBeAg in serum and/or the detection of HBV DNA at >105 copies/ml
2. ANCA positivityNegativePresence of ANCA in serum, as determined by indirect immunofluorescence
3. AsthmaNegativePersonal antecedents of asthma
4. ENT signsNegativeSigns of maxillary sinusitis or otitis media
5. Cryoglobulin positivityNegativeDetection of cryoglobulins in serum
6. GlomerulopathyNegativeSigns of glomerulopathy, such as proteinuria and/or hematuria, with or without renal insufficiency, not due to urinary tract infection, urolithiasis, or hematologic or other nonglomerular causes
7. Arteriographic anomaliesPositiveArteriogram showing aneurysms or occlusions of the visceral arteries, not due to arteriosclerosis, fibromuscular dysplasia, or other noninflammatory causes
8. Mono-/polyneuropathyPositiveDevelopment of mononeuropathy, multiple mononeuropathies, or polyneuropathy
Table 2. Discriminant performance of a minimal set of low-redundant predictive items of the FVSG criteria for distinguishing PAN from other systemic vasculitides or MPA among patients in the FVSG database, considering all PAN cases and considering only HBV-negative PAN cases*
FVSG discriminant itemOdds ratio (95% CI)R2
  • *

    Each discriminant item showed a significant positive (+) or negative (−) association with polyarteritis nodosa (PAN). FVSG = French Vasculitis Study Group; MPA = microscopic polyangiitis; HBV = hepatitis B virus; 95% CI = 95% confidence interval; ANCA = antineutrophil cytoplasmic antibody; ENT = ear, nose, and throat.

  • Stepwise estimation of the odds ratio (OR).

  • Incremental R2 was computed at each step of the logistic regression model.

  • §

    Signs such as maxillary sinusitis or otitis media.

All PAN cases  
 PAN versus other vasculitides  
  1. (+) HBV infection (active replication)16.41 (7.34–36.68)0.323
  2. (−) ANCA positivity0.04 (0.02–0.08)0.539
  3. (−) Asthma0.10 (0.04–0.21)0.598
  4. (−) ENT signs§0.09 (0.03–0.25)0.629
  5. (−) Cryoglobulin positivity0.13 (0.04–0.40)0.644
  6. (−) Glomerulopathy0.36 (0.22–0.58)0.656
  7. (+) Arteriographic anomalies3.48 (1.68–7.22)0.668
  8. (+) Mono-/polyneuropathy1.87 (1.19–2.94)0.674
 PAN versus MPA  
  1. (−) ANCA positivity0.03 (0.01–0.08)0.508
  2. (+) HBV infection (active replication)18.40 (6.10–55.51)0.607
  3. (−) Glomerulopathy0.19 (0.10–0.33)0.648
  4. (+) Arteriographic anomalies6.13 (2.13–17.65)0.669
Only HBV-negative PAN cases  
 HBV-negative PAN versus other vasculitides  
  1. (−) ANCA positivity0.02 (0.01–0.07)0.336
  2. (−) Asthma0.10 (0.04–0.23)0.427
  3. (−) ENT signs§0.10 (0.03–0.29)0.468
  4. (−) Glomerulopathy0.37 (0.23–0.61)0.485
  5. (−) Cryoglobulin positivity0.19 (0.06–0.56)0.503
  6. (+) Arteriographic anomalies3.18 (1.49–6.77)0.515
 HBV-negative PAN versus MPA  
  1. (−) ANCA positivity0.02 (0.01–0.07)0.495
  2. (−) Glomerulopathy0.18 (0.10–0.33)0.556
  3. (+) Arteriographic anomalies5.61 (1.86–16.89)0.581

The logistic regression analysis confirmed the abilities of 7 items from the ACR 1990 classification criteria to identify PAN (Table 3). However, the positive association with a diagnosis of PAN indicated by the analysis of the ACR 1990 criteria could not be confirmed for 1 item (renal insufficiency), which occurred more frequently in non-PAN vasculitides. These 7 criteria yielded a combined sensitivity of 48.9%, with 95.6% specificity when all other vasculitides served as controls. When considering only the subgroup of HBV-negative PAN cases, with all other non-PAN vasculitides as controls, the nonredundant PAN-predictive abilities of the ACR 1990 criteria were confirmed for only 4 positively associated parameters and 1 negatively associated parameter (Table 3), resulting in a major decrease in sensitivity to 8.4%, with 98% specificity in this situation. When controls were restricted to MPA patients, the discriminant abilities of nonredundant PAN items were confirmed for only 3 of them (Table 3), 2 of which were positively associated with a diagnosis of PAN, while 1 (renal insufficiency) occurred more frequently in MPA. In the latter settings, the 3 discriminant items yielded 50.8% sensitivity for all forms of PAN, with 96.1% specificity. One positively associated parameter and 1 negatively associated parameter (Table 3) showed significant nonredundant PAN-predictive abilities when considering only the subgroup of HBV-negative PAN, yielding 12.3% sensitivity, with 99% specificity.

Table 3. Discriminant performance of significant and nonredundant items of the ACR 1990 criteria for distinguishing PAN from other systemic vasculitides or MPA among patients in the FVSG database, considering all PAN cases and considering only HBV-negative PAN cases*
ACR 1990 criteriaOdds ratio (95% CI)R2
  • *

    Each discriminant item showed a significant positive (+) or negative (−) association with polyarteritis nodosa (PAN) or showed no significant usefulness (0) for distinguishing PAN. ACR = American College of Rheumatology; MPA = microscopic polyangiitis; FVSG = French Vasculitis Study Group; HBV = hepatitis B virus; 95% CI = 95% confidence interval; BP = blood pressure.

  • Stepwise estimation of the odds ratio (OR).

  • Incremental R2 was computed at each step of the logistic regression model.

All PAN cases  
 PAN versus other vasculitides  
  1. (+) HBV infection25.53 (13.67–47.67)0.323
  2. (+) Arteriographic anomalies4.40 (2.43–7.95)0.356
  3. (−) Renal insufficiency0.31 (0.18–0.56)0.377
  4. (+) Livedo reticularis2.11 (1.24–3.58)0.387
  5. (+) Mono-/polyneuropathy1.69 (1.16–2.46)0.397
  6. (+) Testicular pain or tenderness3.12 (1.19–8.20)0.402
  7. (+) Diastolic BP >90 mm Hg1.66 (1.06–2.61)0.408
  8. (0) Weight loss ≥4 kg0.75 (0.52–1.08)
  9. (0) Myalgias1.40 (0.98–2.00)
 PAN versus MPA  
  1. (+) HBV infection21.21 (8.17–55.09)0.293
  2. (−) Renal insufficiency0.13 (0.07–0.25)0.380
  3. (+) Arteriographic anomalies7.45 (2.94–18.86)0.427
Only HBV-negative PAN cases  
 HBV-negative PAN versus other vasculitides  
  1. (+) Arteriographic anomalies4.42 (2.38–8.21)0.043
  2. (−) Renal insufficiency0.27 (0.15–0.51)0.080
  3. (+) Livedo reticularis2.07 (1.21–3.55)0.095
  4. (+) Myalgias1.55 (1.07–2.25)0.108
  5. (+) Diastolic BP >90 mm Hg1.69 (1.06–2.71)0.116
  6. (0) Testicular pain or tenderness2.69 (0.98–7.41)
  7. (0) Weight loss ≥4 kg0.72 (0.50–1.05)
  8. (0) Mono-/polyneuropathy1.45 (0.99–2.15)
 HBV-negative PAN versus MPA  
  1. (−) Renal insufficiency0.13 (0.07–0.26)0.147
  2. (+) Arteriographic anomalies6.99 (2.68–18.20)0.209

The 2 sets of association rules established from the selected FVSG criteria are presented in Table 4. When all other vasculitides served as controls, 4 positive and 5 negative association rules could be established, based on 7 of the 8 previously selected features (excluding asthma). This model had a combined sensitivity of 70.2%, with 88.2% specificity. When the controls were restricted to MPA, 3 positive and 2 negative association rules were established, which included all 4 of the features identified by the logistic regression analysis (Table 4).

Table 4. Two sets of positive-association and negative-association rules illustrating the potential use of the FVSG set of discriminant items to distinguish PAN from other systemic vasculitides or from MPA under clinical conditions*
RuleConfidence, %No. of cases§
  • *

    FVSG = French Vasculitis Study Group; PAN = polyarteritis nodosa; MPA = microscopic polyangiitis; HBV = hepatitis B virus; ANCA = antineutrophil cytoplasmic antibody; ENT = ear, nose, and throat.

  • 1 indicates the presence of the item; 0 indicates its absence.

  • Precision of the rule, as evaluated using FVSG data.

  • §

    Number of cases in which the rule was applicable.

PAN versus other vasculitides  
 Positive PAN association  
  1. If HBV = 1 and arteriographic anomalies = 1 then PAN = 110031
  2. If HBV = 1 and ANCA = 0 and neuropathy = 1 then PAN = 19796
  3. If HBV = 1 and cryoglobulins = 0 and glomerulopathy = 0 and ENT signs = 0 then PAN = 19680
  4. If ANCA = 0 and arteriographic anomalies = 1 and ENT signs = 0 then PAN = 17966
 Negative PAN association  
  1. If HBV = 0 and ANCA = 1 then PAN = 099401
  2. If ANCA = 1 and glomerulopathy = 1 and arteriographic anomalies = 0 then PAN = 099270
  3. If ENT signs = 1 then PAN = 098253
  4. If neuropathy = 0 and arteriographic anomalies = 0 then PAN = 086377
  5. If HBV = 0 and arteriographic anomalies = 0 then PAN = 083779
PAN versus MPA  
 Positive PAN association  
  1. If HBV = 1 and ANCA = 0 then PAN = 196108
  2. If ANCA = 0 and arteriographic anomalies = 1 then PAN = 19357
  3. If ANCA = 0 and glomerulopathy = 0 then PAN = 186221
 Negative PAN association  
  1. If ANCA = 1 then PAN = 094138
  2. If HBV = 0 and glomerulopathy = 1 and arteriographic anomalies = 0 then PAN = 085161

ROC curve analyses were performed on the selected FVSG items and the ACR 1990 criteria, considering all other vasculitides as controls (Figure 1A). This yielded areas under the curves of 0.916 (95% confidence interval [95% CI] 0.898–0.934) for the FVSG selected items and 0.711 (95% CI 0.671–0.750) for the ACR 1990 criteria, confirming the significantly better discriminant accuracy of the FVSG criteria (P < 0.05). The asymptotic significance for these curves was acceptable for both sets of criteria (P < 0.001). When controls were restricted to MPA (Figure 1B), ROC curve analyses yielded areas under the curves of 0.906 (95% CI 0.878–0.934) for the selected FVSG criteria and 0.588 (95% CI 0.537–0.639) for the ACR 1990 criteria, thereby confirming the significantly better predictive accuracy of the FVSG criteria set in this situation as well (P < 0.05). Again, the asymptotic significance for these ROC curves was acceptable for both sets of criteria (P < 0.001).

thumbnail image

Figure 1. Receiver operating characteristic curve analyses comparing the discriminant performance of the French Vasculitis Study Group (FVSG) discriminant set of criteria for polyarteritis nodosa (PAN) with the American College of Rheumatology (ACR) 1990 classification criteria for PAN in 2 situations: A, considering all cases of systemic vasculitides as controls and B, restricting controls to patients with microscopic polyangiitis.

Download figure to PowerPoint

The results of the computer simulation are summarized in Figure 2. No significant differences were observed between the data sets obtained with the 2 distinct approaches used to generate artificial patient data, thus indicating the good robustness and reproducibility of our simulation procedures (data not shown). Application of the 2 sets of criteria to the artificially generated data suggested a significant relationship between the sparseness and the quality (i.e., in terms of intracluster homogeneity) of the resulting cluster partitions and the positive predictive value of the criteria sets used to generate these partitions. Indeed, case partitions obtained with selected FVSG items displayed significantly lower sparseness and higher intracluster homogeneity than did those generated with the ACR 1990 criteria (Figures 2A and B), particularly when used to distinguish PAN from MPA (P < 0.05). Most notably, the FVSG set of items yielded higher PAN-predictive performances in terms of positive predictive value and sensitivity (Figures 2C and E) than did the ACR 1990 criteria (P < 0.05). This better performance was conserved even when the prevalence of PAN cases in generated patient populations was artificially lowered. The gain in PAN-predictive ability achieved with the FVSG criteria was not accompanied by any significant decrease in the specificity or the negative predictive value (Figures 2D and F) as compared with the ACR 1990 criteria.

thumbnail image

Figure 2. Results of computer simulations evaluating the potential usefulness of the French Vasculitis Study Group (FVSG) discriminant set of criteria (blue boxes) compared with the American College of Rheumatology (ACR) 1990 criteria (red boxes) for a diagnosis of polyarteritis nodosa (PAN). Artificial patient data were generated by using the maximum entropy correction of a truncated Bahadur-Lazarsfeld expansion, either respecting the reported French prevalence of the 4 main types of vasculitides (PAN, Wegener's granulomatosis, Churg-Strauss syndrome, and microscopic polyangiitis [MPA]) (hatched boxes) or after arbitrarily modifying them to low PAN prevalence (10%) (open boxes). Data were computed from 30 consecutive iterations of the simulation procedure (see Patients and Methods for details). Shown are the optimal case partitions, represented as both the number of clusters (A) and the silhouette width (B), the sensitivity (C), the specificity (D), the positive predictive value (E), and the negative predictive value (F) of the 2 criteria sets. Data are shown as box plots. Each box represents the first and third quartiles (upper and lower limits, respectively). Whiskers represent the minimum and maximum range. Lines inside the boxes represent the median.

Download figure to PowerPoint

DISCUSSION

  1. Top of page
  2. Abstract
  3. PATIENTS AND METHODS
  4. RESULTS
  5. DISCUSSION
  6. AUTHOR CONTRIBUTIONS
  7. REFERENCES

Validated diagnostic criteria for systemic diseases could be a useful tool for clinicians who cannot obtain a histologic diagnosis or who want to avoid performing biopsies because of the risk of side effects. Despite their inconsistent performance as reported in the literature (4, 7), many clinicians still inappropriately use the ACR 1990 classification criteria for diagnostic purposes. As some analysts suggested, a potential explanation for the inconsistent results obtained with those criteria when used for diagnostic predictions could be the dependence of their discriminant abilities on the prevalence of individual vasculitides within the populations examined (23, 24). Moreover, similar studies related to other vasculitis classification systems, such as the Chapel Hill Nomenclature (8), showed similar unsatisfactory performance when used for diagnostic purposes (5, 6) and highlighted the need to develop separate criteria for classification and diagnosis of the various systemic vasculitides (7, 25).

Herein, we report our analysis of patient data accumulated in the FVSG database in which we targeted 2 main objectives. The first was to establish a minimal set of nonredundant criteria that were positively or negatively predictive of PAN and were potentially useful not only for classification, but also for diagnostic purposes. The discriminant accuracy of the selected set of FVSG items (Table 1) was compared with that of the ACR 1990 criteria for PAN in a sample of patients with histologically proven systemic vasculitides selected from the FVSG database. The second objective was to evaluate the PAN-predictive abilities of each of these sets of criteria in a computer simulation procedure that relied on an unsupervised hierarchical classification algorithm applied to artificially generated patient data.

The results of our analysis showed that when the ACR 1990 criteria were applied to the FVSG database patients, their initially reported PAN discriminant abilities were not confirmed (2). In contrast, the selected set of FVSG items established by our analysis significantly outperformed the ACR 1990 criteria, regardless of the analytical situation being considered (Tables 2 and 3 and Figure 1). The difference between the discriminant performance of the 2 sets of criteria was even more significant when the analysis was restricted to the subgroup of HBV-negative PAN patients, which the ACR 1990 criteria could not distinguish from the group with non-PAN vasculitides, thus demonstrating the extremely low sensitivity of these criteria in this particular situation.

Furthermore, although 7 of the 9 original items of the ACR 1990 criteria (except for histology) were found to have significantly nonredundant usefulness for distinguishing PAN from other systemic vasculitides in the FVSG database patients, a significant positive association with a diagnosis of PAN could not be confirmed for 3 of the items (Table 3). Moreover, renal insufficiency, which yielded significant discriminant ability to distinguish PAN in our analysis, showed a lower frequency in PAN patients than in other systemic vasculitides or in MPA (Table 3). These findings are consistent with those in previous studies (4) that indicated a low positive predictive value of the ACR 1990 criteria for PAN.

As previously suggested, in addition to some epidemiologic differences between patient samples, another potential explanation for this phenomenon could be the lack of distinction between PAN and MPA that is inherent in the ACR 1990 analysis. Indeed, a strong argument in support of this hypothesis is the poorly discriminant performance of the ACR 1990 criteria in our analyses when used to distinguish between histologically confirmed PAN and MPA (Table 3), which was already shown to be a major drawback of these criteria (24, 26, 27).

A third possible explanation of the observed differences in discriminant abilities of the ACR 1990 criteria and the FVSG item set could lie in the methodologic differences between the 2 analyses. While the ACR 1990 analysis focused on the selection of positive discriminant criteria for classification purposes, our analysis sought to maximize the combined predictive relevance of the selected items by considering both positively associated and negatively associated parameters. Indeed, this strategic difference could potentially explain why significant negatively discriminant items, for example, ANCA positivity by indirect immunofluorescence, could have been missed by the ACR 1990 analysis. The importance of ANCA positivity in the diagnosis of vasculitis was first recognized by the Chapel Hill Nomenclature group (8). In our analysis, ANCA positivity yielded the strongest discriminant ability, covering 22% of the variance in the logistic regression model when used to distinguish PAN from other vasculitides and 50% of the variance when used to distinguish PAN from MPA (Table 2). It is widely acknowledged that the detection of ANCA by indirect immunofluorescence provides a lower specificity than detection by antigen-specific enzyme-linked immunosorbent assay techniques (10), and it should therefore be expected that the combination of these 2 detection methods might further improve the discriminant performance of the FVSG item set.

Our assessment of the relevance of the 2 criteria sets in the diagnosis of PAN was complemented by a computer simulation procedure that relied on both sets of items to perform an unsupervised classification task on artificially generated patient data. Although the numbers of case clusters contained by the optimal partitions were not the same as the real number of vasculitis entities (e.g., 4 in our case) represented in the artificial patient samples under either of the analytical conditions considered, the FVSG item set yielded 2–5 times fewer clusters (i.e., closer to the real number of vasculitis entities) than did the ACR 1990 criteria. This lower sparseness of the partitions obtained with the FVSG items was associated with significantly better intracluster homogeneity of the resulting case clusters than that obtained with the ACR 1990 criteria, thereby confirming the superior PAN-predictive abilities of the FVSG item set.

The FVSG item set also exhibited significantly stronger robustness when the prevalence of vasculitides was artificially varied in generated patient samples, thus suggesting better adaptability of this set of criteria to various epidemiologic conditions. In addition, the low sensitivities and positive predictive values obtained with the ACR 1990 criteria during the computer simulation procedure are consistent with the high percentages of false-positive diagnoses of PAN previously reported with this criteria set (4), indicating good reliability of computer simulation analyses.

In conclusion, the results of the analyses described herein suggest that the combined use of positive and negative criteria could significantly improve discriminant performance while providing a more appropriate support for analytical medical reasoning as it examines, with equal importance, both positive and negative rationales for considering a diagnosis. Indeed, as the results of our computer simulation suggest, the combination of positively and negatively discriminant criteria may prove beneficial for the establishment of a diagnostic screening tool for vasculitis patients. Further prospective validation of this set of criteria in a multicenter international study of different populations and different epidemiologic settings is needed in order to confirm that these items together provide a satisfactory diagnostic tool for PAN.

AUTHOR CONTRIBUTIONS

  1. Top of page
  2. Abstract
  3. PATIENTS AND METHODS
  4. RESULTS
  5. DISCUSSION
  6. AUTHOR CONTRIBUTIONS
  7. REFERENCES

Drs. Henegar and Guillevin had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study design. Henegar, Pagnoux, Puéchal, Guillevin.

Acquisition of data. Pagnoux, Saba, Bagnères, Meyer, Guillevin.

Analysis and interpretation of data. Henegar, Pagnoux, Puéchal, Saba, Guillevin.

Manuscript preparation. Henegar, Pagnoux, Zucker, Le Guern, Guillevin.

Statistical analysis. Henegar, Zucker, Bar-Hen.

REFERENCES

  1. Top of page
  2. Abstract
  3. PATIENTS AND METHODS
  4. RESULTS
  5. DISCUSSION
  6. AUTHOR CONTRIBUTIONS
  7. REFERENCES
  • 1
    Bloch DA, Michel BA, Hunder GG, McShane DJ, Arend WP, Calabrese LH, et al. The American College of Rheumatology 1990 criteria for the classification of vasculitis: patients and methods. Arthritis Rheum 1990; 33: 106873.
  • 2
    Lightfoot RW Jr, Michel BA, Bloch DA, Hunder GG, Zvaifler NJ, McShane DJ, et al. The American College of Rheumatology 1990 criteria for the classification of polyarteritis nodosa. Arthritis Rheum 1990; 33: 108893.
  • 3
    Fries JF, Hochberg MC, Medsger TA Jr, Hunder GG, Bombardier C, and the American College of Rheumatology Diagnostic and Therapeutic Criteria Committee. Criteria for rheumatic disease: different types and different functions. Arthritis Rheum 1994; 37: 45462.
  • 4
    Rao JK, Allen NB, Pincus T. Limitations of the 1990 American College of Rheumatology classification criteria in the diagnosis of vasculitis. Ann Intern Med 1998; 129: 34552.
  • 5
    Lane SE, Watts RA, Barker TH, Scott DG. Evaluation of the Sorensen diagnostic criteria in the classification of systemic vasculitis. Rheumatology (Oxford) 2002; 41: 113841.
  • 6
    Sorensen SF, Slot O, Tvede N, Petersen J. A prospective study of vasculitis patients collected in a five year period: evaluation of the Chapel Hill nomenclature. Ann Rheum Dis 2000; 59: 47882.
  • 7
    Hunder GG. The use and misuse of classification and diagnostic criteria for complex diseases. Ann Intern Med 1998; 129: 4178.
  • 8
    Jennette JC, Falk RJ, Andrassy K, Bacon PA, Churg J, Gross WL, et al. Nomenclature of systemic vasculitides: proposal of an international consensus conference. Arthritis Rheum 1994; 37: 18792.
  • 9
    Servoss JC, Friedman LS. Serologic and molecular diagnosis of hepatitis B virus. Infect Dis Clin North Am 2006; 20: 4761.
  • 10
    Hagen EC, Daha MR, Hermans J, Andrassy K, Csernok E, Gaskin G, et al, for the EC/BCR Project for ANCA Assay Standardization. Diagnostic value of standardized assays for anti-neutrophil cytoplasmic antibodies in idiopathic systemic vasculitis. Kidney Int 1998; 53: 74353.
  • 11
    Yao YY. Information-theoretic measures for knowledge discovery and data mining. In: Karmeshu, editor. Entropy measures, maximum entropy principle and emerging applications. 1st ed. Berlin: Springer; 2003. p. 11536.
  • 12
    RipleyBD, editor. Pattern recognition and neural networks. 1st ed. Cambridge (UK): Cambridge University Press; 1996.
  • 13
    BreimanL, FriedmanJH, OlshenRA, StoneCJ, editors. Classification and regression trees. 1st ed. Belmont (CA): Wadsworth; 1984.
  • 14
    Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983; 148: 83943.
  • 15
    Bahadur RR. A representation of the joint distribution of responses to n dichotomous items. In: SolomonH, editor. Studies in item analysis and prediction. 1st ed. Stanford: Stanford University Press; 1961. p. 15868.
  • 16
    Van Der Geest PA. The binomial distribution with dependent Bernoulli trials. J Stat Comput Simulat 2005; 75: 14154.
  • 17
    Yu CT, Buckley C, Lam K, Salton G. A generalized term dependence model in information retrieval. Inf Technol Res Dev 1983; 2: 12954.
  • 18
    Chow CK, Liu CN. Approximating discrete probability distributions with dependence trees. IEEE Trans Inf Theory 1968; 14: 4627.
  • 19
    Mahr A, Guillevin L, Poissonnet M, Ayme S. Prevalences of polyarteritis nodosa, microscopic polyangiitis, Wegener's granulomatosis, and Churg-Strauss syndrome in a French urban multiethnic population in 2000: a capture–recapture estimate. Arthritis Rheum 2004; 51: 929.
  • 20
    Xu R, Wunsch DN. Survey of clustering algorithms. IEEE Trans Neural Netw 2005; 16: 64578.
  • 21
    Rousseeuw PJ. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J Computat Appl Math 1987; 20: 5365.
  • 22
    R Development Core Team. R: a language and environment for statistical computing, version: 2.4.0. Vienna: R Foundation for Statistical Computing; 2006. Online at: http://www.R-project.org.
  • 23
    Heller I, Isakov A, Topilsky M. American College of Rheumatology Criteria for the diagnosis of vasculitis [letter]. Ann Intern Med 1999; 130: 861.
  • 24
    Watts RA, Jolliffe VA, Carruthers DM, Lockwood M, Scott DG. Effect of classification on the incidence of polyarteritis nodosa and microscopic polyangiitis. Arthritis Rheum 1996; 39: 120812.
  • 25
    Watts R, Lane S, Hanslik T, Hauser T, Hellmich B, Koldingsnes W, et al. Development and validation of a consensus methodology for the classification of the ANCA-associated vasculitides and polyarteritis nodosa for epidemiological studies. Ann Rheum Dis 2007; 66: 2227.
  • 26
    Bruce IN, Bell AL. Effect of classification on the incidence of polyarteritis nodosa and microscopic polyangiitis: comment on the article by Watts et al [letter]. Arthritis Rheum 1997; 40: 1183.
  • 27
    Bruce IN, Bell AL. A comparison of two nomenclature systems for primary systemic vasculitis. Br J Rheumatol 1997; 36: 4538.