Consensus statements, evidence-based medicine and guidelines in allergic diseases

Authors

  • J. Bousquet,

    1. University Hospital and INSERM, Montpellier, France
    Search for more papers by this author
    • *

      Member of GA2LEN (Global Allergy and Asthma European Network), supported by the EU Framework programme for research, contract no. FOOD-CT-2004-506378

  • T. Bieber,

    1. Department of Dermatology and Allergy, University of Bonn, Bonn, Germany
    Search for more papers by this author
  • W. Fokkens,

    1. Department of Otorhinolanyngology, University of Amsterdam, Amsterdam, the Netherlands
    Search for more papers by this author
    • *

      Member of GA2LEN (Global Allergy and Asthma European Network), supported by the EU Framework programme for research, contract no. FOOD-CT-2004-506378

  • M. Humbert,

    1. Service de Pneumologie, Hôpital Antoine-Béclère, Université Paris-Sud, Clamart, France
    Search for more papers by this author
    • *

      Member of GA2LEN (Global Allergy and Asthma European Network), supported by the EU Framework programme for research, contract no. FOOD-CT-2004-506378

  • M. L. Kowalski,

    1. Department of Immunology, Rheumatology and Allergy, Medical University of Łódź, Łódź, Poland
    Search for more papers by this author
    • *

      Member of GA2LEN (Global Allergy and Asthma European Network), supported by the EU Framework programme for research, contract no. FOOD-CT-2004-506378

  • B. Niggemann,

    1. German Red Cross Hospital, Berlin, Germany
    Search for more papers by this author
    • *

      Member of GA2LEN (Global Allergy and Asthma European Network), supported by the EU Framework programme for research, contract no. FOOD-CT-2004-506378

  • H.-U. Simon,

    1. Institute of Pharmacology, University of Bern, Bern, Switzerland
    Search for more papers by this author
  • H. Schünemann

    1. Department of Epidemiology, Italian National Cancer Institute ’Regina Elena’, Rome, Italy
    Search for more papers by this author

Jean Bousquet
Clinique des Maladies Respiratoires
Inserm U454
Hôpital Arnaud de Villeneuve
University Hospital
34295 Montpellier Cedex 5
France

Abbreviations:
AAAAI

American Academy of Allergy, Asthma and Immunology

ACCP

American College of Chest Physicians

ADR

adverse drug reaction

ARIA

Allergic Rhinitis and its Impact on Asthma

EAACI

European Academy of Allergy and Clinical Immunology

EPOS

European Position Paper on Rhinosinusitis and Nasal Polyps

EBM

Evidence-based medicine

EDF

European Dermatology Forum

GA2LEN

Global Allergy and Asthma European Network

GINA

Global Initiative for Asthma

GOLD

Global Initiative for Obstructive Pulmonary Diseases

GRADE

Grading of Recommendations Assessment, Development and Evaluation

PRACTALL

Practical Allergology (AAAAI-EAACI)

RCT

randomized controlled trial

The American College of Chest Physicians (1) defines a consensus statement as ‘a document that represents the collective opinion of a convened expert panel. The opinions expressed in the statement do not reflect a formal evidence review and were not developed in accordance with the process outlined for evidence-based clinical practice guidelines’. Therefore, certain caveats may apply when developing such consensus statements (Table 1).

Table 1.   Points to consider for a consensus
  1. Modified from the American College of Chest Physicians (1).

The terms ‘recommendation’, ‘evidence-based’ and ‘guideline’ should not be used in the context of consensus statements. Findings of a consensus panel should be stated as ‘opinions’ or ‘suggestions’.
Consensus panels should be encouraged to include a methods section on how the consensus was achieved, either formally or informally.
Whether included in the methods section or (if no methods section) in the introduction, explicit details should be provided to the reader explaining that the information contained in the document is based on consensus and expert opinion.
Information should be included to explicitly state that the opinions expressed in the document should not be used for performance measurement or for competency purposes since they are not based on systematically reviewed evidence.
Conflict of interest for all members of the consensus and financial support for the production of the consensus should be clearly stated.
Expert opinion represents the interpretation of available evidence which may be of various quality. Even high-quality evidence, such as that coming from randomized controlled trials, requires interpretation and this interpretation can be expressed as an opinion.

The development of clinical guidelines should follow the strict process which has been established by several organizations: various systems have been used to grade the quality of evidence (sometimes called levels of evidence) (2, 3).

Early guidelines were predominantly derived from unsystematically compiled opinions of experts based on clinical trials and mechanistic approaches (opinion-based medicine) (4). ‘Evidence-based medicine’ (EBM) has become an essential component in the preparation of guidelines. It is the ability to track down, critically appraise (for its validity and usefulness) and incorporate the information obtained from the best available evidence [ideally randomized controlled trial (RCTs)] in order to establish the clinical bases for diagnosis, prognosis and therapeutics (5, 6). Evidence-based medicine attempts to provide a logical and convenient framework from which the quality and relevance of clinical studies may be assessed in an unbiased manner (7). Systematic reviews contribute to resolving uncertainty when original research, reviews, and editorials disagree (8). The Cochrane Collaboration has led the way in setting new standards for the preparation of systematic reviews (9) despite existing criticism (10, 11).

The Institute of Medicines (IoM; http://www.iom.edu; accessed 13 November 2007) defines clinical practice guidelines as ‘systematically developed statements to assist practitioner and patient decision about appropriate health care for specific clinical circumstances’. Evidence and recommendations in these guidelines should be graded following a defined and validated procedure for the quality of evidence and strength of recommendations. The evidence-based clinical practice guidelines then undergo peer review. Conflict of interest for all members participating to the guideline and financial support for the production in the guideline should be clearly stated.

Whilst there is increasing agreement regarding the components of correct clinical practice guidelines and what constitutes high quality evidence, it is also clear that the highest quality evidence from RCTs is often based on selected patients. Therefore, they may fall short of representing the entire population. For instance, the criteria for selecting asthma patients for a clinical trial are usually: absence of co-morbidity, FEV1 50–85% of predicted values, present or historical reversibility 12% last year, nonsmoker or, if an exsmoker, a smoke burden of less than 10 pack years. In a study carried out in general and specialist practices in Norway, only 5.4% of the study asthma patients met with these criteria (12). Moreover, compliance to treatment, a major problem of allergy and asthma management, is far better in RCTs than in real life (13). Thus, ‘real life’ or pragmatic trials are needed (14) but are rarely available for allergic diseases (15).

Another example of the shortcomings of currently available RCTs is demonstrated by the great progress that has been made in obtaining reliable evidence on the beneficial effects of interventions, but the challenges have not been overcome in the identification, interpretation and reporting of harmful effects (16, 17). In fact, RCTs are insufficient in the assessment of the rare side-effects of treatments. Thus, there is an urgent need to get better evidence about side-effects (risks) (18). The safety of any treatment is fundamental. Evidence is required throughout the entire spectrum of the treatment life cycle, from the premarketing to the postmarketing phase. Drug safety and effectiveness need to be assessed in the real world, where outcomes may differ from those of controlled clinical trials that provide premarket test results. Drug regulatory initiatives include data mining, active adverse drug reaction (ADR) surveillance, independent, multidisciplinary evaluation of suspected ADRs, and formal pharmaco-epidemiology studies (19).

However, RCTs offer the most methodological and rigorous approach in establishing the cause and effect, thereby, providing the highest quality evidence available. A number of approaches have been used to grade the quality of evidence and the resulting strength of recommendations (26, 27). The large number of systems for measuring the quality of evidence and recommendations is confusing (18) and all previously used approaches for grading levels of evidence and the strength of recommendations have important shortcomings (26). Global Initiative for Asthma (22), Global Initiative for Obstructive Pulmonary Diseases (23), and the third expert panel report of the US asthma guidelines (24) used one guide for level of evidence, whereas Allergic Rhinitis an its Impact on Asthma (ARIA) (25) and European Position Paper on Rhinosinusitis and Nasal Polyps (EPOS) (34) used the Shekelle approach (21) to develop guidelines. Thus, the many international organizations inconsistently applied grading systems, many of which are of limited usefulness.

The recommendations follow criteria which may differ from country to country. Furthermore, there are considerable differences between guidelines which are difficult to compare.

More recently, the ‘Guidelines for WHO guidelines’ recommended using a specific, uniform grading system (28). The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach is one of the recommended systems (27) and is being used increasingly by a number of organizations. The GRADE working group has published the results of its work (29). It grades recommendations into two levels – strong and weak – and quality evidence into four categories – high, moderate, low, and very low (18, 27). However, some organizations have combined the low and very low category (30). Five factors may decrease and three factors increase the quality of evidence (18) (Table 2).

Table 2.   Factors in deciding on confidence in estimates of benefits, risks, burden, and costs
  1. From Guyatt et al. (18)

Factors that may decrease the quality of evidencePoor quality of planning and implementation of the available RCTs suggesting high likelihood of bias
Inconsistency of results
Indirectness of evidence
Sparse evidence
Reporting bias (including publication bias)
Factors that may increase the quality of evidence based on observational studiesLarge magnitude of effect
All plausible confounding factors would reduce a demonstrated effect
Dose-dependent gradient

Based on the GRADE approach, recommendations established on efficacy only are insufficient for the classification of clinical practice guidelines, and panels should consider several factors (Table 3). When desirable consequences of an intervention clearly show a preferance over the undesirable consequences (including harms and burden), or clearly do not, then strong recommendations are warranted. If the benefits do not clearly outweigh the downsides or vice versa, then weaker recommendations should follow.

Table 3.   Factors that panels should consider when deciding upon a recommendation
  1. From Guyatt et al. (30)

Methodological quality of the evidence supporting estimates of the likely benefits and likely risk, inconvenience and costs for each outcome
Importance of the outcome prevented by treatment
Magnitude of the effect of treatment
Risks associated with therapy
Burdens of therapy
Risks of target event
Costs

Thus, the GRADE approach incorporates the previously described aspects in an explicit way (Fig. 1).

Figure 1.

 Development of guidelines from Bousquet et al. (4, 20).

In this issue of the Journal, a Consensus on Pediatric Asthma is published by Practical Allergology (AAAAI-EAACI) (PRACTALL) and suggests management strategies that are specifically tailored for children (31). Children with asthma are fundamentally different to adults with asthma. Their lungs are still developing, their immune systems are immature and they have smaller airways that become more easily obstructed. The new PRACTALL consensus was developed to address the lack of up-to-date international guidelines that focus exclusively on pediatric asthma. It was developed by the PRACTALL Pediatric Asthma Group, which consists of approximately 40 international experts on pediatric allergy and asthma. The consensus is part of the PRACTALL initiative, which is endorsed by the European Academy of Allergy and Clinical Immunology (EAACI) and the American Academy of Allergy, Asthma and Immunology (AAAAI). The new PRACTALL consensus addresses these issues and proposes strategies for the diagnosis, management and monitoring of asthma in children. By definition, this is not a guideline.

In this same issue, the methodology for the update of the ARIA guidelines is also published (32). In this paper, the GRADE approach has been used. However, due to the number of clinical questions raised, we used a pragmatic approach that is similar to the rapid advice approach recently applied by WHO in a guideline project (33). The first meeting of this panel was held at the Agenzia Italiana del Farmaco. The recommendations will be finalized in 2008 and will lead to the ARIA revision, the first international guideline in respiratory and allergic disease based on GRADE.

We are also announcing that Allergy will devote a Series to explaining the GRADE approach. This Series will be directed by Dr Holger Schünemann. Articles will summarize how judgments regarding the quality of evidence supporting therapeutic and diagnostic interventions are made, and how the strength of a recommendation is determined by an explicit and transparent way using the GRADE system for both therapeutic and diagnostic interventions. Examples specific to the field of allergy and asthma will be used.

Finally, not only the ARIA revision will be based on GRADE, but, in collaboration with Global Allergy and Asthma European Network (GA2LEN), other guidelines will also be produced. These may include rhinosinusitis based on EPOS (34) and urticaria based on the EAACI/GA2LEN/EDF guidelines on urticaria (35, 36). We believe that our increasing agreement on common approaches to assess our confidence in the evidence and estimates of effects for both benefits and downsides will ultimately help clinicians to make evidence-based decisions. These decisions and the clinicians’ expertise will help integrate the best evidence with the clinical state and circumstances of their patients’ values and preferences.

Ancillary