A further evaluation of the clinical use of specific IgE antibody testing in allergic diseases

Authors


Lars Söderström
Pharmacia Diagnostics
SE-751 82 Uppsala
Sweden

Abstract

Background: The evaluation and interpretation of the results from blood tests measuring specific immunoglobulin E (IgE) antibody concentration is currently made using the dichotomized result from the test despite a quantitative result is obtained. It has been shown that different levels of IgE antibodies, assessed by blood test and skin prick test, may have a relation to presence of symptoms, implying that there is more information in a quantitative result than in the dichotomous – positive or negative.

Objective: To investigate the clinical utility of quantification of IgE antibodies in the diagnosis of allergic patients and whether such procedure has any advantage to the presently dichotomously used sensitivity and specificity at a fixed cut-off.

Methods: Data from a previously published study (R. Paganelli, I.J. Ansoteugi, J. Sastre, C.-E. Lange, M.H.W.M. Roovers, H. de Groot, N.B. Lindholm, P.W. Ewan, Allergy, 1998; 53) analysing diagnosis of allergic patients in four different clinics were re-evaluated. In the original study consecutive patients with suspected IgE-mediated allergy had been examined and evaluated according to the clinical routine at each clinic, using case history, physical examination, skin tests and laboratory tests, except the test to be evaluated, and given an ‘doctors’ allergen-specific diagnosis’ as positive or negative. In the present study the relation between ‘doctors’ allergen-specific diagnosis’, expressed as pos/neg, and the quantitative levels of specific IgE antibody concentration was analysed using a logistic regression model. This presentation of results was also compared with the more common characteristics of sensitivity and specificity, and also with Receiver–operator characteristics (ROC) curves.

Results: The used logistic model described the relationship between allergen-specific diagnosis in each study and the levels of IgE antibodies. The shape of the curve illustrated the physicians’ disposition for a positive diagnose in the study, in relation to the specific IgE antibody level. Differences in the shape of the curve was found both between allergens within clinics and between clinics for the same allergen. No association could be demonstrated between prevalence and shape of the curve.

Conclusions: Conventional sensitivity/specificity figures or ROC concepts only use the qualitative statement of whether IgE is present or not. A risk assessment using the quantitative level of IgE antibody to an allergen increases the utility of the information in clinical context compared with a qualitative statement of whether IgE is present or not. The quantification demonstrated the link between specific IgE antibodies and allergic reactions. The use of objective, well performing quantitative tests should help improve diagnostic accuracy and might provide a way for the patient to understand and manage his or her daily situation and risk for reactions.

Testing for the presence of immunoglobulin E (IgE) antibodies has been used for years as a tool in the diagnosis of allergy diseases and identification of the relevant allergen (1–4). Patients with IgE mediated allergy are in most cases sensitized to several allergens. This may cause difficulties for the clinician in identifying the most relevant allergen(s) that is essential to avoid optimal treatment of the patient. The present communication test the hypothesis whether or not quantification of the specific IgE antibody levels to inhalant allergens may give an added value to the diagnosis of allergic diseases.

Since the first development of tests for measurement of specific IgE in the early seventies, there has been continued technical improvements and some of the tests can now be regarded as true quantitative tests (3). Prerequisites for such a quantitative test include the use of a well-defined calibration, excess of well standardized extracts of allergens on the solid phase, linearity of the test for different concentrations and good precision (1, 4, 5). An accurate test result is obtained with good technical performance characteristics but the question of interpretation is left to the user, i.e. to decide whether the quantitative result is an indication of the allergic disease.

The traditional way of describing the performance of specific IgE determinations is in terms of performance characteristics like sensitivity and specificity, i.e. the dichotomized result from the test is compared with the result of a well-defined reference, a so called gold standard (6). This is a measure of the test performance in a defined situation, e.g. a clinical trial, although of limited use in the daily clinical routine where the aim is to distinguish patients with symptoms from their IgE mediated allergy to one or several allergens from patients with other clinical conditions with similar symptoms. For a more complete description of the test behaviour, other measures might be more useful for the physician. These include the likelihood ratio, odds ratio and especially the tests positive predictive value (PPV) i.e. the tests ability to detect truly diseased persons (7, 8).

In the diagnosis of allergic patients and separating them from patients suffering from conditions with similar symptoms, information from several sources has to be considered, e.g. physical examination, heredity, case history and different test results. These factors make the interpretation of diagnostic test results complex and sometimes difficult. This is further emphasized by the fact that the specific IgE test indicates only whether or not an individual is sensitized, i.e. produce specific IgE antibodies. The normal procedure for dealing with tests measuring specific IgE antibodies, has been to dichotomize the result into positive or negative with the use of a cut-off (6). Such a cut-off gives a simple classification, but may be difficult to interpret in the clinical context. This is most evident in situations where an individual has low IgE antibody levels and vague clinical symptoms, sometimes requiring a challenge test for a clear interpretation. Using a dichotomization of this result will give rise to the suspicion of a false positive test result. The technique using Receiver–operator characteristics (ROC) plots gives an increased information regarding the ability to discriminate between positive and negative, and is also very useful in comparing different methods but is for nonexperienced users difficult to fully understand and appreciate.

The use of a semi-quantitative scale like classes, improves the possibility to interpret results. The class ranges are, however, often wide and may therefore hide information as the classes presently used are not equidistant, differ between different test systems and uses different calibrations and therefore different classes (5).

It has been shown that the concentrations of IgE antibodies, assessed by blood test and skin prick test (SPT), has a relation to presence of symptoms (2, 9), implying that there is more information in a quantitative result then in the dichotomous result positive or negative. This can be interpreted as a higher risk for an allergic reaction with higher levels of IgE antibodies.

The objective of this study was to analyse the clinical utility of quantification of IgE antibodies in the diagnosis of allergic patients.

Material and methods

Patient materials

Data from four different clinics, previously presented (10), were re-evaluated using a logistic regression model. The original data presented were from six independent studies where two of the studies were performed in 1987 and followed up in 1995 and the other four studies were performed in 1995 using consecutive patients with suspected IgE-mediated allergy. In this re-evaluation only the data from the four clinical studies performed in 1995 were used, in total 681 patients, 6–81 years of age. In the original study the physician saw each patient on one occasion. Case history was taken, and the patient was examined and evaluated according to the routine at each clinic. In addition blood samples were also collected for analysis with UniCAP® specific IgE (Pharmacia Diagnostics, Uppsala, Sweden) but the results of these tests were not known by the physicians at the time of diagnosis. For each patient a ‘doctors’ allergen-specific diagnosis’ based on established procedures, including optional SPT and or in vitro tests used in ordinary clinical routine at the different clinics were made (10). The physicians’ diagnosis was used as the true status. Relevant allergens for testing were selected by the physicians to cover the majority of the local specific sensitivities represented among the patients, using 14 allergens, as summarized in Table 1.

Table 1.  Allergens included in four clinical studies by Pagenelli et al. (10)
AllergenClinic
1234
Dermatophagoides pteronyssinus, d1xxxx
Dermatophagoides farinae, d2xxx 
Cat dander, e1xx x
Horse dander, e3   x
Dog dander, e5 x x
Rye-grass, g5xx  
Timothy, g6x x 
Grass mix, gx1   x
Alternaria alternata, m6xx x
Common silver birch, t3  xx
Olive, t9xx  
Parietaria judaica, w21x   
Mugwort, w6  xx
Ribwort (Plantain), w9x   

Statistical methods

Using a technical re-formulation of the classical definitions (6) a universal form was used to establish a more thorough evaluation. Thus a variable Y for the result of the Gold Standard and a variable X for the result of the test was defined. Further I(H) was defined as an indicator function, where I(H) = 1, if H is true and I(H) = 0, if H is false. A cut-off value, discriminating between positive and negative test results, was then defined as any real number z in the measuring range of the test. A positive test result was then obtained when Xi > z, Table 2.

Table 2.  Classical presentation of the evaluation of a test using an optional cut-off
 Test status 
PositiveNegative
Positive∑/(Xi > z | Y = 1)∑/(Xi < z | Y = 1)∑/(Y = 1)
Gold stand
Negative∑/(Xi > z | Y = 0)∑/(Xi < z | Y = 0)∑/(Y = 0)
 ∑/(Xi > z)∑/(Xi < z)∑/(Y = 1) + ∑/(Y = 0)

Using this reformulation sensitivity and specificity were evaluated using a cut-off of 0.35 kU/l as well as at an optional cut-off value z.

Empirical ROC plots were constructed by varying z over the measuring range and plotting sensitivity(z) against [1 − specificty(z)]. The area under the ROC curve was estimated using the Mann–Whitney two-sample statistic. In addition, an optimization of the cut-off was made by plotting [sensitivity(z) and specificity(z)] vs concentration, and choosing the cut-off as the point z where both measures gave the best results.

For the quantitative evaluation a logistic regression model was formulated as the probability of receiving a positive clinical diagnosis as a function of the specific IgE concentration, or the logarithm of the concentration (11).

image

The quantitative model would then describe the relationship between sensitization and clinical diagnosis of allergy, i.e. the level of IgE antibodies in relation to predominance for an allergic reaction or clinical allergy diagnosis. Based on the model a theoretical curve for the probability was constructed (Fig. 1).

Figure 1.

Theoretical curve of a logistic relationship between immunoglobulin E antibody concentration and the probability to react or show symptoms.

The curve demonstrated the probability of a clinical allergy diagnosis, made by the physician used as the gold standard, as a function of the specific IgE concentration. A low level of IgE antibodies represents a low probability for a clinical allergy diagnosis. Consequently, a high IgE antibody level represents a high probability for a clinical allergy diagnosis.

Results

Application of the logistic model in inhalant allergy

The classical performance characteristics using a cut-off of 0.35 kU/l were calculated for the different allergens and clinics, based on results from patients that were tested for a specific allergen and where there was a doctors’ diagnosis. Using this approach the performance characteristics varied between allergens and also between clinics. The sensitivities for all allergens and clinics ranged between 0.57 and 1.00, the specificities between 0.77 and 1.00, and the prevalence of patients suffering from allergic problems varied between 0.04 and 0.75.

To further explore a dichotomized evaluation using a fixed cut-off point, five different optimization criteria were applied using various cut-off points over the measuring range. The optimal cut-off varied between allergens and between clinics for the same allergen, so also the shape of the ROC-plots. The area under curve (AUC) varied between 0.76 and 1.0. Number of patients, performance characteristics and AUC for each allergen and clinic are presented in Table 3.

Table 3.  Peformance characteristics for 16 allergens in four clinics from the multicenter study published by Paganelli et al. (10)
AllergenClinicNo. of patitentsResults using 0.35 kU/l as cut-offROC AUC
SensitivitySpecificityPrevalencePPV
  1. ROC, Receiver–operator characteristics; AUC, area under curve.

Dermatophagoides pteronyssinus, d111400.910.900.310.800.93
D. pteronyssinus, d121440.830.880.130.500.87
D. pteronyssinus, d131000.970.910.350.850.96
D. pteronyssinus, d141930.930.950.500.950.96
Dermatophagoides farinae, d211240.860.890.300.760.91
D. farinae, d221410.810.890.110.480.86
D. farinae, d2340110.7511
Cat dander, e111090.860.890.130.550.89
Cat dander, e121450.790.980.130.880.89
Cat dander, e141920.810.970.390.950.90
Horse dander, e341880.750.890.110.450.85
Dog dander, e521430.860.880.150.550.90
Dog dander, e541860.770.840.310.680.84
Rye-grass, g511340.920.940.370.900.95
Rye-grass, g521490.910.950.440.940.95
Timothy, g61990.890.940.360.890.94
Timothy, g631620.970.960.400.940.98
Grass mix, gx141910.950.950.350.910.97
Alternaria alternata, m611090.860.980.060.750.93
A. alternata, m621470.750.980.080.750.87
A. alternata, m641860.750.970.040.550.87
Common silver birch, t331510.920.980.340.960.96
Common silver birch, t341930.890.950.320.900.94
Olive, t911120.800.850.220.610.85
Olive, t921490.820.880.340.780.88
Parietaria judaica, w2111340.910.940.260.840.94
Mugwort, w631540.900.890.140.580.93
Mugwort, w641950.920.850.120.470.93
Ribwort (Plantain), w919710.770.050.191

The logistic model was then applied for each allergen at each clinic (Fig. 2). The used logistic model described the relationship between allergen-specific diagnosis in each study, based on established procedures at the different clinics, and the level of sensitization. The shape of the curve illustrated the physicians’ disposition for a positive clinical allergy diagnosis in the study, in relation to the specific IgE antibody level. For house dust mite Dermatophagoides farinae, ImmunoCAP d2 at clinic 3 with only 40 patients, a perfect discrimination between positive and negative patients by the test, gave a curve showing a probability of 1.00 for a positive clinical allergy diagnosis above 0.35 kUA/l, i.e. a sensitivity and specificity of 100% using a cut-off of 0.35 kU/l. For all the other allergens at the different clinics the curves revealed a less steep slope and an overlap between positive and negative clinical allergy diagnosis in relation to the specific IgE antibody level.

Figure 2.

Probability of receiving a positive diagnosis at a given immunoglobulin E value for different allergens at four different clinics.

Comparisons between allergens within clinics

At each clinic the probability curves for the allergens appeared in groups indicating a similar judgement behaviour for each cluster of allergens at that clinic. For clinic 1 the results for Ribwort (Plantago lanceolata), ImmunoCAP w9, gave a significant different intercept (P < 0.001) compared with all other allergens. The results for Alternaria alternata, ImmunoCAP m6, showed the steepest slope, which was significantly different from the results for Dermatophagoides pteronyssinus, D. farinae, Cat dander and Olive (ImmunoCAP d1, d2, e1 and t9 respectively). At clinic 2 no significant differences between the different intercepts or slopes were found. At clinic 3 the Mugwort (Artemisia vulgaris), ImmunoCAP w6, and Silver Birch (Betula verrucosa), ImmunoCAP t3, both were significantly different from the other allergens (P < 0.05). At clinic 4 the results for D. pteronyssinus, ImmunoCAP d1, and Cat dander, ImmunoCAP e1, were similar to each other and significantly different from Horse dander, ImmunoCAP e3, and Dog dander, ImmunoCAP e5, both regarding intercept and slope (P < 0.01).

Comparison between clinics

As the allergens used for testing were selected by the investigators to cover the majority of the specific sensitivities represented among their patients, not all allergens were present at all clinics, and thus relevant comparisons could not be made for all allergens. A comparison between the clinics demonstrated that only for D. pteronyssinus, ImmunoCAP d1, and Cat dander, ImmunoCAP e1, a statistically significant difference was found. For D. pteronyssinus, ImmunoCAP d1, there was a significant difference between clinic 4 and the other three clinics, both for the intercept and slope (P < 0.05). For Cat dander, ImmunoCAP e1, there was a significant difference between clinic 1 and the other two clinics, both for intercept and slope (P < 0.05) (Fig. 3A). For all other allergens that were analysed in several clinics, no statistically significant differences between the clinics could be found in this study.

Figure 3.

(A) Probability of receiving a positive diagnosis at a given immunoglobulin E value for the same allergen at different clinics for Dermatophagoides pteronyssinus d1 and Cat dander e1. (B) ROC curves for the same allergen at different clinics for D. pteronyssinus d1 and Cat dander e1.

A ROC curve analysis for the same allergens gave values for the AUC between 0.87 and 0.96, where the lowest area for D. pteronyssinus, ImmunoCAP d1, was obtained for clinic 2. For Cat dander, ImmunoCAP e1, the values for the area under the curve were rather similar for the two clinics, 0.89–0.90, Fig. 3B.

Influence of prevalence

Although having similar prevalence's a comparison between Cat dander, ImmunoCAP e1, and Dog dander, ImmunoCAP e5, at clinic 4 gave very different probability curves between the two allergens (Fig. 4). The two curves gave significant different intercepts and slopes (P < 0.001).

Figure 4.

(A) Probability of receiving a positive diagnosis at a given immunoglobulin E (IgE) value for two allergens having the same prevalence; Cat dander e1 and Dog dander e5 at clinic 4. (B) Probability of receiving a positive diagnosis at a given IgE value for two allergens having the same prevalence; Cat dander e1 and Dog dander e5 at clinic 4.

At clinic 2 the prevalence for allergy to Cat dander (e1) was 0.13 and to Rye-grass (g5) 0.44, but the two allergens displayed very similar probability curves. Similar probability curves for similar prevalence's were found for instance at clinic 4 for Common silver birch (t3) and the Grass mix (gx1). Furthermore different curves for different prevalence's were also seen, for instance at clinic 3 for Mugwort (w6) and Timothy (g6).

Discussion

Allergic diseases represent an increasing problem in the western world, with symptoms that may not be easily distinguished from other disorders (12–15). Furthermore, there are many different allergens, which may trigger the clinical symptoms and it is often not easy to distinguish which allergen that is the most offending one (1). A given test should identify the IgE related allergic condition among other similar conditions and ideally also identify the clinically most offensive allergen. However, allergic symptoms related to IgE antibodies are not only dependent on the presence of IgE antibodies but also on a number of related and unrelated confounding factors. These factors involve inflammation, organ function, presence of infection, physical and psychological stress, hormonal influences, exposure to irritants and other environmental factors. Therefore there is no real good gold standard for clinical diagnosis of an allergic reaction (16, 17). In order to refine the diagnosis different contributing tools are used, e.g. challenge test, SPT and test for IgE antibody determination in blood (3, 16). The specialists are experienced in taking case history and making appropriate physical examination but are still often helped by a test demonstrating the presence of specific IgE antibodies to support their decision. The primary care being exposed to allergic patients less frequently and having less practice in their management would need a tool for judgement whether allergy is present or not and when to refer to a specialist (3).

When evaluating the quality of a test, i.e. the ability to discriminate between health and disease, measures like sensitivity and specificity can be seen as properties inherent to a test. The ROC curves increases the amount of information but has the limitation that the actual decision-thresholds are not displayed and that they for small samples become jagged and bumpy. Predictive values are properties of the application once the context is established. In that sense predictive values can be seen as an aid in interpreting test results more than a measure of performance (18). The logistic model for a quantitative IgE antibody test, not using a fixed cut-off, gives a better presentation of the agreement between IgE antibody level and the presence of clinical disease than only conventional sensitivity and specificity figures.

Based on a Bayesian approach Sampson (19, 20) described the use of clinical decision points in order to eliminate the need for double-blind, placebo-controlled food challenges in his clinic. These authors concluded that it was possible to identify decision points, i.e. some level of sensitization, to discriminate between where no challenge was necessary, where there were need for challenge at the physicians clinic and where challenge could be performed at home. In the same study Sampson also presented similar probability curves, as in the present communication, showing that these curves provided similar information as the proposed decision points (20). The curves, also clearly displayed the important information that even a low concentration of food-specific IgE antibodies might be associated with a risk of clinical reactivity.

The probability curves obtained in the present communication demonstrated that also for inhalant allergens there was a relation between an allergen-specific clinical allergy diagnosis based on established procedures at different clinics and the level of IgE sensitization. The used logistic regression model that was applied could visualize this relationship. Similar to what was found for food allergens by Sampson (20) there were differences also between the inhalant allergens and between clinics.

Different slopes and intercepts may indicate different identification patterns of symptoms; a steep curve would indicate symptoms easily linked to an allergen even with low levels of IgE antibodies. A more flat curve in contrast, would indicate a difficulty to link even high levels of IgE antibodies as the trigger of the symptoms.

In the present data no connection between shape of the curves and different prevalence's was found. As the logistic model describes the relationship between the quantitative measure of IgE sensitization and the dichotomous diagnosis, it is not in the same manner depending on the prevalence as the dichotomized calculation of the PPV, where the PPV is a function of prevalence. Predictive values obtained in one clinical study should therefore not be applied in a different clinical setting.

The present study illustrates that in a clinical situation, in order to determine proper treatment for the patient, the quantification of IgE antibodies and the concept of probability for an allergic reaction or positive clinical allergy conclusion would be a better diagnostic tool compared with the traditional measures.

Conclusion

The extension from a qualitative statement of whether IgE is present or not, to a risk assessment for a quantitative level increases the amount of useful clinical information from measurements of IgE antibodies. In the present study the quantification not only showed the link between specific IgE and allergy related reactions, it also provided a more meaningful way of interpreting the result. They may therefore also be used as an aid in discussing the results with the patient. By expanding the evaluating from classical performance characteristics to the application and also presenting how the diagnose is related to the level of sensitization an increased amount of information was obtained. The results could and should never be taken as a black or white statement of clinical allergy, but it might provide a way for the patient to understand and manage his or her situation and risk for reactions.

Evaluation and interpretation of clinical studies should make use of the whole quantitative scale by expressing the probability for a positive clinical allergy diagnosis, not only for the dichotomized result, but also as a function of the measured quantitative result.

Ancillary