The American College of Chest Physicians (1) defines a consensus statement as ‘a document that represents the collective opinion of a convened expert panel. The opinions expressed in the statement do not reflect a formal evidence review and were not developed in accordance with the process outlined for evidence-based clinical practice guidelines’. Therefore, certain caveats may apply when developing such consensus statements (Table 1).
|The terms ‘recommendation’, ‘evidence-based’ and ‘guideline’ should not be used in the context of consensus statements. Findings of a consensus panel should be stated as ‘opinions’ or ‘suggestions’.|
|Consensus panels should be encouraged to include a methods section on how the consensus was achieved, either formally or informally.|
|Whether included in the methods section or (if no methods section) in the introduction, explicit details should be provided to the reader explaining that the information contained in the document is based on consensus and expert opinion.|
|Information should be included to explicitly state that the opinions expressed in the document should not be used for performance measurement or for competency purposes since they are not based on systematically reviewed evidence.|
|Conflict of interest for all members of the consensus and financial support for the production of the consensus should be clearly stated.|
|Expert opinion represents the interpretation of available evidence which may be of various quality. Even high-quality evidence, such as that coming from randomized controlled trials, requires interpretation and this interpretation can be expressed as an opinion.|
The development of clinical guidelines should follow the strict process which has been established by several organizations: various systems have been used to grade the quality of evidence (sometimes called levels of evidence) (2, 3).
Early guidelines were predominantly derived from unsystematically compiled opinions of experts based on clinical trials and mechanistic approaches (opinion-based medicine) (4). ‘Evidence-based medicine’ (EBM) has become an essential component in the preparation of guidelines. It is the ability to track down, critically appraise (for its validity and usefulness) and incorporate the information obtained from the best available evidence [ideally randomized controlled trial (RCTs)] in order to establish the clinical bases for diagnosis, prognosis and therapeutics (5, 6). Evidence-based medicine attempts to provide a logical and convenient framework from which the quality and relevance of clinical studies may be assessed in an unbiased manner (7). Systematic reviews contribute to resolving uncertainty when original research, reviews, and editorials disagree (8). The Cochrane Collaboration has led the way in setting new standards for the preparation of systematic reviews (9) despite existing criticism (10, 11).
The Institute of Medicines (IoM; http://www.iom.edu; accessed 13 November 2007) defines clinical practice guidelines as ‘systematically developed statements to assist practitioner and patient decision about appropriate health care for specific clinical circumstances’. Evidence and recommendations in these guidelines should be graded following a defined and validated procedure for the quality of evidence and strength of recommendations. The evidence-based clinical practice guidelines then undergo peer review. Conflict of interest for all members participating to the guideline and financial support for the production in the guideline should be clearly stated.
Whilst there is increasing agreement regarding the components of correct clinical practice guidelines and what constitutes high quality evidence, it is also clear that the highest quality evidence from RCTs is often based on selected patients. Therefore, they may fall short of representing the entire population. For instance, the criteria for selecting asthma patients for a clinical trial are usually: absence of co-morbidity, FEV1 50–85% of predicted values, present or historical reversibility 12% last year, nonsmoker or, if an exsmoker, a smoke burden of less than 10 pack years. In a study carried out in general and specialist practices in Norway, only 5.4% of the study asthma patients met with these criteria (12). Moreover, compliance to treatment, a major problem of allergy and asthma management, is far better in RCTs than in real life (13). Thus, ‘real life’ or pragmatic trials are needed (14) but are rarely available for allergic diseases (15).
Another example of the shortcomings of currently available RCTs is demonstrated by the great progress that has been made in obtaining reliable evidence on the beneficial effects of interventions, but the challenges have not been overcome in the identification, interpretation and reporting of harmful effects (16, 17). In fact, RCTs are insufficient in the assessment of the rare side-effects of treatments. Thus, there is an urgent need to get better evidence about side-effects (risks) (18). The safety of any treatment is fundamental. Evidence is required throughout the entire spectrum of the treatment life cycle, from the premarketing to the postmarketing phase. Drug safety and effectiveness need to be assessed in the real world, where outcomes may differ from those of controlled clinical trials that provide premarket test results. Drug regulatory initiatives include data mining, active adverse drug reaction (ADR) surveillance, independent, multidisciplinary evaluation of suspected ADRs, and formal pharmaco-epidemiology studies (19).
However, RCTs offer the most methodological and rigorous approach in establishing the cause and effect, thereby, providing the highest quality evidence available. A number of approaches have been used to grade the quality of evidence and the resulting strength of recommendations (26, 27). The large number of systems for measuring the quality of evidence and recommendations is confusing (18) and all previously used approaches for grading levels of evidence and the strength of recommendations have important shortcomings (26). Global Initiative for Asthma (22), Global Initiative for Obstructive Pulmonary Diseases (23), and the third expert panel report of the US asthma guidelines (24) used one guide for level of evidence, whereas Allergic Rhinitis an its Impact on Asthma (ARIA) (25) and European Position Paper on Rhinosinusitis and Nasal Polyps (EPOS) (34) used the Shekelle approach (21) to develop guidelines. Thus, the many international organizations inconsistently applied grading systems, many of which are of limited usefulness.
The recommendations follow criteria which may differ from country to country. Furthermore, there are considerable differences between guidelines which are difficult to compare.
More recently, the ‘Guidelines for WHO guidelines’ recommended using a specific, uniform grading system (28). The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach is one of the recommended systems (27) and is being used increasingly by a number of organizations. The GRADE working group has published the results of its work (29). It grades recommendations into two levels – strong and weak – and quality evidence into four categories – high, moderate, low, and very low (18, 27). However, some organizations have combined the low and very low category (30). Five factors may decrease and three factors increase the quality of evidence (18) (Table 2).
|Factors that may decrease the quality of evidence||Poor quality of planning and implementation of the available RCTs suggesting high likelihood of bias|
|Inconsistency of results|
|Indirectness of evidence|
|Reporting bias (including publication bias)|
|Factors that may increase the quality of evidence based on observational studies||Large magnitude of effect|
|All plausible confounding factors would reduce a demonstrated effect|
Based on the GRADE approach, recommendations established on efficacy only are insufficient for the classification of clinical practice guidelines, and panels should consider several factors (Table 3). When desirable consequences of an intervention clearly show a preferance over the undesirable consequences (including harms and burden), or clearly do not, then strong recommendations are warranted. If the benefits do not clearly outweigh the downsides or vice versa, then weaker recommendations should follow.
|Methodological quality of the evidence supporting estimates of the likely benefits and likely risk, inconvenience and costs for each outcome|
|Importance of the outcome prevented by treatment|
|Magnitude of the effect of treatment|
|Risks associated with therapy|
|Burdens of therapy|
|Risks of target event|
Thus, the GRADE approach incorporates the previously described aspects in an explicit way (Fig. 1).
In this issue of the Journal, a Consensus on Pediatric Asthma is published by Practical Allergology (AAAAI-EAACI) (PRACTALL) and suggests management strategies that are specifically tailored for children (31). Children with asthma are fundamentally different to adults with asthma. Their lungs are still developing, their immune systems are immature and they have smaller airways that become more easily obstructed. The new PRACTALL consensus was developed to address the lack of up-to-date international guidelines that focus exclusively on pediatric asthma. It was developed by the PRACTALL Pediatric Asthma Group, which consists of approximately 40 international experts on pediatric allergy and asthma. The consensus is part of the PRACTALL initiative, which is endorsed by the European Academy of Allergy and Clinical Immunology (EAACI) and the American Academy of Allergy, Asthma and Immunology (AAAAI). The new PRACTALL consensus addresses these issues and proposes strategies for the diagnosis, management and monitoring of asthma in children. By definition, this is not a guideline.
In this same issue, the methodology for the update of the ARIA guidelines is also published (32). In this paper, the GRADE approach has been used. However, due to the number of clinical questions raised, we used a pragmatic approach that is similar to the rapid advice approach recently applied by WHO in a guideline project (33). The first meeting of this panel was held at the Agenzia Italiana del Farmaco. The recommendations will be finalized in 2008 and will lead to the ARIA revision, the first international guideline in respiratory and allergic disease based on GRADE.
We are also announcing that Allergy will devote a Series to explaining the GRADE approach. This Series will be directed by Dr Holger Schünemann. Articles will summarize how judgments regarding the quality of evidence supporting therapeutic and diagnostic interventions are made, and how the strength of a recommendation is determined by an explicit and transparent way using the GRADE system for both therapeutic and diagnostic interventions. Examples specific to the field of allergy and asthma will be used.
Finally, not only the ARIA revision will be based on GRADE, but, in collaboration with Global Allergy and Asthma European Network (GA2LEN), other guidelines will also be produced. These may include rhinosinusitis based on EPOS (34) and urticaria based on the EAACI/GA2LEN/EDF guidelines on urticaria (35, 36). We believe that our increasing agreement on common approaches to assess our confidence in the evidence and estimates of effects for both benefits and downsides will ultimately help clinicians to make evidence-based decisions. These decisions and the clinicians’ expertise will help integrate the best evidence with the clinical state and circumstances of their patients’ values and preferences.