Placebo reactions in double-blind, placebo-controlled food challenges in children

Authors


B. J. Vlieg-Boerstra
Division of Paediatric Pulmonology and Paediatric Allergy
Beatrix Children’s Hospital
University Medical Centre Groningen
PO Box 30.001
9700RB Groningen
the Netherlands

Abstract

Background:  A cardinal feature of the double-blind, placebo-controlled food challenge (DBPCFC) is that placebo administration is included as a control. To date, the occurrence and diagnostic significance of placebo events have not extensively been documented.

Objective:  To analyse the occurrence and features of placebo events in DBPCFCs and to assess their contribution to the diagnostic accuracy of the DBPCFC in children.

Methods:  The study population consisted of 132 challenges in 105 sensitized children (age range 0.7–16.6 years, median 5.3 years), who underwent DBPCFCs with cow’s milk, egg, peanut, hazelnut and soy. Placebo and active food challenges were performed on different days.

Results:  A total number of 17 (12.9%) positive placebo events occurred, which could be classified as immediate (9/17), late-onset (8/17), objective (11/17) or subjective (6/17). Four of 74 (5.4%) positive active food challenges were revealed to be false positive by administration of a placebo challenge. This is 3% (4/132) of all challenges. When computed by a statistical model, the false positive rate was 0.129 (12.9% of all challenges).

Conclusion:  Placebo events with diverse clinical characteristics occur in DBPCFCs in a significant number of children. The diagnostic significance of the administration of a placebo challenge is first, to identify false positive diagnoses in DBPCFCs by refuting false positive tests in individual patients. Secondly, to allow for blinding of the active food challenge. Thirdly, applying a statistical model demonstrates that some positive challenges may be false positive and that the test may need to be repeated in selected cases.

A cardinal feature of the double-blind, placebo-controlled food challenge (DBPCFC) is that the placebo administration included as a control. Despite the importance of placebos, very little has been published on the occurrence and features of placebo ‘reactions’ or events following placebo administration, which we will here refer to as ‘placebo events’. In many studies on DBPCFCs, the occurrence of placebo events are not described (1–8). In other studies, placebo events are not described in detail (9–13), and it thus remains unclear how DBPCFCs were assessed when placebo events occurred.

To date, attempts to standardize the DBPCFC have not resulted in universally accepted procedures for this test (14–17). Assumptions have been made in designing test procedures derived mainly from clinical practice, including the timing of the administration of active and placebo challenges, criteria to terminate the test (subjective or objective symptoms) and interpretation of test results (only immediate or also late onset symptoms). All these features may influence the number of reactions seen to active challenges as well as the number of placebo events and hence the outcome of DBPCFCs.

The purpose of this study was to document the prevalence and features of placebo events in a large population of children suspected of IgE-mediated food allergy. Placebo events were assessed according to a standardized algorithm for the assessment of each challenge session and according to a protocol for the assessment of the results of the complete DBPCFC. We also estimated the diagnostic significance of placebo administration in DBPCFCs by comparing outcomes of the entire test to outcomes of challenge sessions with the active food only, as well as by analysis, applying a statistical model.

Methods

Study population

The study population consisted of consecutive sensitized children in whom DBPCFCs with cow’s milk, egg, peanut, hazelnut or soy were performed in our centre between January 2004 and September 2005. Nonsensitized children and children suspected of having nonIgE mediated allergic disorders were excluded. This study was exempt from medical ethical approval, as DBPCFCs in children were performed as a routine diagnostic test. Information on gender, age, allergic symptoms at the time of challenge, dietary history with regard to the challenged food and sensitization was obtained. Medical assessment of allergic symptoms was performed just before the DBPCFC was performed. Clinical symptoms and overall condition had to be stable, and children were instructed to discontinue antihistamines 72 h prior to DBPCFC if possible.

Sensitization to the allergenic food in question was determined by ImmunoCap RAST (Phadia AB, Uppsala, Sweden) and skin prick test (SPT) with commercially available extracts (ALK-Abelló, Horsholm, Denmark) within 6 months prior to the DBPCFC. RAST results of ≥0.35 kU/l and SPTs (HEP) of ≥0.3 were considered positive. Children showing either or both positive SPT or specific IgE to the food tested were considered as sensitized to the food in question.

Challenge procedure

Prior to the DBPCFC, elimination of the food in question for at least 6 weeks was confirmed by a dietician. Placebo and active challenges were administered in a random order and were administered on separate days with at least 2 weeks interval in between. Randomization was determined by computer. Recipes for the test foods were prepared for each challenge session individually and recipe and randomization code were verified by a second individual. For the active challenge, the suspected allergenic food was disguised in a food matrix to which the patient was tolerant. Unequivocal tolerance to the food matrix was ascertained by dietary history by the dietician. Placebo and active foods were as similar as possible in sensory properties. Validation of adequate blinding of the test materials was achieved by sensory testing in a dedicated food laboratory (18).

Total challenge dose and incremental scale

The challenge procedure included a 4- to 6-step incremental design in which progressively greater quantities of the same allergenic food were administered. Pasteurized cow’s or soy milk, baked egg, roasted peanuts and unroasted hazelnuts were used. The incremental scale and total challenge dose used are shown in Table 1. The incremental scale was achieved by varying the volume of the test food. The time interval between two challenge doses was 30 min in most cases. The total amount of allergenic food administered was limited by: (i) the total amount of allergenic food that could be masked in the food matrices as the highest challenge dose in an acceptable volume, (ii) the starting dose and (iii) an acceptable duration of the challenge session (4–6 h), taking into consideration that the challenge had to be performed in an out-patient clinical setting.

Table 1.   Incremental scale and challenge doses used in double-blind, placebo-controlled food challenges in children
 Cow’s milk (ml)Soy milk (ml)Egg (mg)Protein equivalent (mg)Peanut (mg)Hazelnut (mg)Protein equivalent (mg)
Dose 10.050.05131.756121.75
Dose 20.10.1273.5012253.5
Dose 30.40.4108144810014
Dose 42.02.05387024150070
Dose 510.010.02690350480860130
Dose 650.050.013460175012062500350
Total63.063.016830 (∼1/3 egg) 21902000 (∼5–7 peanut kernels)4000 (∼ 4 small hazelnuts)570

Assessment protocol of challenge sessions and total DBPCFCs

Challenge sessions in which children consumed <75% of the maximum challenge dose in absence of symptoms, were considered invalid. The challenge was discontinued when objective allergic symptoms occurred, or subjective allergic symptoms occurred twice on two successive administrations of the challenge material. Objective symptoms and signs were defined as (angio)oedema, urticaria, exacerbation of atopic eczema, rash, vomiting, diarrhoea, lip or tongue swelling, rhinoconjunctivitis, stridor, coughing, wheezing, hoarseness, collapse, tachycardia and hypotension. Subjective symptoms were defined as exacerbation of generalized itch (in case of atopic eczema), abdominal pain, nausea and/or cramp, oral allergy symptoms, itchy throat or sensation of throat swelling, difficulty in swallowing and ‘other’ symptoms such as drowsiness and irritability. Immediate symptoms were defined as symptoms occurring during the challenge or within 2 h after the last challenge dose. Two days after each challenge session late onset reactions were recorded by telephone questionnaire. Late onset symptoms were defined as symptoms occurring between 2 and 48 h after the last challenge dose.

For the optimal consistency of assessment of challenge results, we devised a standardized algorithm to assess immediate and late onset events following each challenge session (Fig. 1). Events following test food administration were classified as (strongly) positive or negative.

Figure 1.

 Algorithm for assessment of allergic symptoms following a challenge session in double-blind, placebo-controlled food challenges in children (with the exception of nonIgE mediated allergic disorders).

Forty-eight hours after the second challenge session, the code was broken and the outcome of the DBPCFC was assessed according to a protocol, shown in Table 2. Negative DBPCFCs were followed by introduction of the challenged food into the diet. Patients received written instructions explaining how to introduce the food at home, using incrementing amounts of allergenic food ranging from the maximum challenge dose to normal daily food servings. Results of introduction were evaluated by telephone 1 month after the DBPCFC.

Table 2.   Assessment protocol for the outcome of DBPCFCs
Active food challengePlacebo challengeAssessment of DBPCFC
  1. In exceptional cases, a challenge session or total DBPCFC may be assessed as questionable. Questionable DBPCFCs are repeated.

  2. DBPCFC, double-blind, placebo-controlled food challenges in children.

PositiveNegativePositive
Positive (clearly more positive than placebo)PositivePositive
NegativeNegativeNegative
Negative (or positive, but clearly less positive than placebo)PositiveNegative

Documentation of placebo events and statistics

The prevalence and features of placebo events were recorded in the whole study population, and were classified according to whether symptoms were immediate or late onset symptoms, and according to whether symptoms were objective or subjective.

The clinical relevance of placebo administration in DBPCFCs was estimated by comparing outcomes of the entire test to outcomes of challenge sessions with the active food only. Furthermore, according to a statistical model of Brigs et al. (19) and Hansen et al. (20), the false positive rate was calculated as the number of subjects who responded with a positive reaction to the placebo challenge, divided by the total number of challenges.

The chi-square test (spss Software, 12th edition) was used to analyse differences between immediate and late onset placebo events with regard to type of symptoms following challenges, type of food and challenge order. For statistical analysis, symptoms following challenges were categorized to a nominal scale as 1. dermal symptoms, 2. gastro-intestinal symptoms, 3. local and upper airway symptoms, 4. lower airway symptoms, 5. anaphylaxis and 6. other symptoms. (Fig. 2).

Figure 2.

 Comparison of type of symptoms which occurred in immediate and late onset placebo events.

Differences in age between immediate and late onset placebo events were assessed by Mann–Whitney test for not normally distributed values (two sided).

Results

Study population

A total number 105 children (median 5.3 years, range 0.7–16.6 years; 68 males, 37 females), were included in the study. Three of these children had their first known exposure to these foods by these food challenges. At the time of challenge, 93 children reported symptoms of atopic eczema (89%), 39 rhinitis (37%) and 58 asthma (55%). The median SPT (HEP) to the food in question was 0.90 (range 0–2.9) (124 cases), the median RAST score (kU/l) was 3.54 (<0.35 to >100) (131 cases).

Outcome of DBPCFCs

One hundred and thirty-two challenges were included in this study. These DBPCFCs were performed with cow’s milk (n = 43), hen’s egg (n = 31), peanut (n = 35), hazelnut (n = 17) and soy (n = 6). Seventy DBPCFCs (53%) were assessed as positive and 62 (47%) DBPCFCs were negative. No reactions were reported following a negative DBPCFC when introducing the challenged food at home, except in three cases (one each of egg, milk and peanut) (4.8%). These patients reported mild recurrent symptoms of skin rash, oral allergy symptoms or abdominal pain when consuming a normal food serving. They were offered a repeated DBPCFC, but they declined. For the purpose of this analysis, these DBPCFCs were considered as negative.

Placebo events

A total of 132 placebo challenges was analysed and 17 (12.9%) positive placebo events occurred in 17 different children (Table 3). All categories of symptoms occurred (dermal, gastro-intestinal, local and upper airway, lower airway and ‘other’ symptoms), with the exception of anaphylaxis.

Table 3.   Characteristics of placebo events
Patient No.Food on active challenge daySymptoms on placebo dayTime of onsetObjective/ subjective symptoms
 1EggDiarrhoea
Generalized itching
ImmediateObjective (and subjective)
 2PeanutDiarrhoea
Cramp
Nausea
ImmediateObjective (and subjective)
 3EggLip swellingImmediateObjective
 4SoyCoughing
Stridor
Hoarseness
ImmediateObjective
 5PeanutRash urticariaImmediateObjective
 6Cow’s milkTight and itchy throat
Generalized itching
ImmediateSubjective
 7HazelnutTight and itchy throat
Itchy tongue
ImmediateSubjective
 8PeanutAbdominal pain
Cramp
ImmediateSubjective
 9HazelnutItchy mouth and tongueImmediateSubjective
10EggGeneralized itching
Exacerbation of atopic eczema
Late onsetObjective (and subjective)
11Cow’s milkGeneralized itching
Exacerbation of atopic eczema
Late onsetObjective (and subjective)
12Cow’s milkGeneralized itching
Exacerbation of atopic eczema
Irritability
Late onsetObjective (and subjective)
13HazelnutExacerbation of atopic eczema
Nausea
Vomiting
Drowsiness
Late onsetObjective (and subjective)
14Cow’s milkDiarrhoeaLate onsetObjective
15Cow’s milkDiarrhoeaLate onsetObjective
16EggGeneralized itching
Irritability
Late onsetSubjective
17EggNausea
Cramp
Late onsetSubjective

Further 9/17 (53%) placebo events were classified as immediate events and 8/17 (47%) as late onset events. In 11/17 (65%) placebo events objective symptoms were observed, were as in 6/17 (35%) placebo events only subjective symptoms were reported (Table 3 and Fig. 3).

Figure 3.

 Occurrence of placebo events classified according to time of onset and objective or subjective symptoms.

Immediate symptoms consisted of generalized itching, rash and urticaria in three challenges; diarrhoea, cramp, nausea and abdominal pain in three challenges; lip swelling, stridor, tight and itchy throat, mouth or tongue in five challenges; and coughing and hoarseness in one challenge.

Late-onset symptoms consisted of generalized itching and exacerbation of atopic dermatitis in five challenges; nausea, vomiting, diarrhoea and cramp in four challenges; and irritability or drowsiness in three patients.

Objective symptoms consisted of rash, urticaria and exacerbation of atopic eczema in five challenges; diarrhoea and vomiting in five challenges; lip swelling and stridor in two challenges and lower airway symptoms in one challenge.

Subjective symptoms consisted of generalized itching in six challenges; cramp, nausea and abdominal pain in four challenges; tight and itchy throat, mouth or tongue in three challenges and irritability and drowsiness in three challenges.

There were no significant differences between the frequency of placebo events during the first and the second challenge session.

Comparison between immediate and late onset placebo events

A comparison of the type of symptoms of immediate and late onset placebo symptoms is shown in Fig. 2. ‘Local and upper airway symptoms’ occurred significantly more often in immediate than late placebo events (P = 0.029). Late onset placebo events tended to consist of (mild) exacerbations of atopic dermatitis/generalized itching and/or mild gastro-intestinal symptoms, such as abdominal pain or diarrhoea, but these differences were not statistically significant. A comparison of foods involved in immediate and late onset placebo events shows, that late onset placebo events tended to be observed more often for cow’s milk and hen’s egg than for other foods, but these differences were not statistically significant. There were no significant differences in age between children showing placebo events during DBPCFCs with cow’s milk or hen’s egg, and those showing placebo events during DBPCFCs to other foods.

There were no significant differences with regard to the age of the patients and challenge order of the placebo between immediate and late onset placebo events.

Diagnostic significance of placebo events

A total of 70 DBPCFCs (53%) were assessed as positive. When considering only the active food challenge sessions, 74/132 of such challenge sessions were assessed as positive. When comparing the results of the positive DBPCFCs to the results of the positive active food challenge sessions alone, four of 74 (5.4%) positive active food challenges were thus revealed to be false positive by administration of a placebo challenge (Table 3, patients no 1, 3, 12 and 17). This is 3% (4/132) of all challenges. In these four DBPCFCs, both active food and placebo challenges were positive. Two of these placebo events occurred immediately, two occurred late-onset. One (of four) events was objective only, one subjective only and the remaining two events were classified as both objective and subjective. These four children introduced the challenged food successfully according to the protocol for negative DBPCFCs. The other 13 DBPCFCs in which placebo events were observed were assessed as negative (10 cases), because the active food challenges were negative. In the remaining three DBPCFCs in which a placebo event was observed, the active food challenge sessions were clearly more positive than the placebo challenges. One of these DBPCFCs was repeated and was assessed as positive. The other two children declined from repeated challenge. For the purpose of this analysis, these two challenges were assessed as positive.

According to the statistical model (19, 20), the false positive rate was 0.129 (17/132), which is 12.9% of all challenges.

Discussion

In this study, we found a rate of placebo events of 17/132 (12.9%), when all symptoms (objective, subjective, immediate and late onset) were considered valid. Approximately 50% of all placebo events consisted of either immediate or late-onset events. Approximately 65% of all placebo events consisted of objective symptoms and 35% of the symptoms were subjective. Thus, clinicians should be aware that all these types of placebo events may occur in a significant number of sensitized children. Furthermore, placebo events may present with a variety of symptoms, such as dermal, gastro-intestinal, local- and upper airway symptoms, lower airway symptoms, but we observed no anaphylaxis (cardio-vascular symptoms).

Rates of placebo events similar to ours were reported in other studies on food allergy. Rates of 0.2–3.6% were reported (9–13, 21, 22), but also somewhat higher prevalences were reported by Ballmer Weber in studies of carrot allergy (10%), and of celery allergy (6%) respectively (23, 24) and by Ortolani in hazelnut allergy (10%) (25). In threshold studies, rates of 4 to 7% placebo events were found (26–28). These events were all immediate placebo events, and these rates are comparable to the number of immediate placebo events in our study.

In some studies placebo events were not observed (29–32). In some cases, this may be due to short observation periods following placebo administration, such as when active food and placebo are administered on the same day (1, 6, 10, 29), or interspersed with each other (12, 13, 16, 23, 32, 33). Because we performed active and placebo challenges on separate days at an interval of at least 2 weeks, we were able to clearly distinguish between immediate and late onset reactions to active challenges and events following placebo challenges.

Monitoring and assessment of symptoms during challenge tests represent a key problem in the assessment of the outcome of DBPCFCs (16). For this reason, we standardized the assessment procedures of each challenge session and the complete DBPCFC, which facilitates comparison to similar studies and allows for a consistent assessment of each challenge (session).

Depending on the criteria used for the assessment of symptoms and termination of a challenge session, the rate of placebo events differs considerably. To date, validation and clinical relevance of immediate vs late onset, and objective vs subjective symptoms have not been established, and there is no universal consensus on which symptoms are necessary and sufficient to terminate the challenge. If subjective symptoms are observed, repeated challenges are generally thought to be required (34). In our protocol, the challenge session was considered positive when objective or repeated subjective allergic symptoms occurred.

For reliable results, it is important not to exclude placebo events from statistical analysis (14, 15, 19, 20). Only a few studies report on the assessment of placebo events in the context of the total challenge assessment (20, 21, 26–28). In many other studies no details are provided. To date, little has been published on the interpretation and clinical significance of placebo events (14, 15, 19, 20). We calculated a false positive rate of 4/74 (5.4%) positive food challenges for all positive test results, because in four DBPCFCs, the active food challenge was refuted as ‘positive’ because of administration of a placebo challenge. This is 3% (4/132) of all test results. However, when applying the statistical model of Briggs et al. (19) and Hansen et al. (20), the false positive rate calculated for all test results was higher (17/132 = 12.9%). In this model, all subjects with the tendency to give false positive responses (all placebo events) are estimated and incorporated in the calculation. Thus, clinicians should be aware that, statistically there is a chance that some double-blind, placebo-controlled positive test results will be false positive, and that some tests may need to be repeated in selected cases.

DBPCFC with a negative active food challenge session and a positive placebo challenge session were assessed as being negative. DBPCFCs in which both placebo and active food challenge sessions were positive could theoretically be assessed as either negative or as inconclusive. In our protocol, the latter DBPCFCs were assessed as negative (four DBPCFCs). These four children introduced the challenged foods successfully, according to the protocol for negative DBPCFCs, suggesting that such results are indeed negative. This protocol provides for the active and careful monitoring of successful introduction of the food in question at home and thus excludes the possibility of false negative test results. DBPCFCs in which active food challenges are clearly more positive than placebo challenges could either be assessed as inconclusive or positive. In our protocol, these DBPCFCs were assessed as positive, but may be repeated (three DBPCFCs).

It is generally accepted that the DBPCFC is the gold standard for the diagnosis of food allergy, whereas open food challenges (OFCs) may render false positive results because of lack of blinding and a lack of administration of placebos (34). However, little data have been published comparing the results of these two types of challenges (35, 36). Brouwer et al. (35) found that in 14 infants with atopic eczema with a positive OFC who were recruited from a primary care setting the diagnosis cow’s milk allergy could be confirmed in only four infants by DBPCFC, resulting in a false positive rate of 71% (10/14) of all positive test results. In a prevalence study by Venter et al. (36), a false positive rate of 20.5% (8/39) for OFCs in 12-months-old children was found. These differences in positive test results suggest that the diagnostic contribution of the administration of a placebo challenge is not only to identify placebo events, but quantitatively more importantly, to allow for blinding of the active food challenge. Blinded administration of placebos is thus important to minimize false positive food challenges.

In conclusion, placebo events with diverse clinical characteristics occur in DBPCFCs in a significant number of children. Placebo events may be immediate or late-onset and objective of subjective. The diagnostic significance of the administration of a placebo challenge is first, to identify false positive diagnoses in DBPCFCs by refuting false positive test results in individual patients. Secondly, to allow for blinding of the active food challenges. Thirdly, clinicians should also be aware that single challenges tests may be false positive, and that the test may need to be repeated in selected cases.

Funding

This study was funded by University Medical Centre Groningen, University of Groningen, the Netherlands.

Ancillary