Guidelines in infectious diseases: how reliable are they?
Article first published online: 21 NOV 2013
© 2013 The Authors Clinical Microbiology and Infection © 2013 European Society of Clinical Microbiology and Infectious Diseases
Clinical Microbiology and Infection
Volume 19, Issue 12, pages 1093–1095, December 2013
How to Cite
Paul, M., Roilides, E. and Tassios, P. T. (2013), Guidelines in infectious diseases: how reliable are they?. Clinical Microbiology and Infection, 19: 1093–1095. doi: 10.1111/1469-0691.12424
- Issue published online: 21 NOV 2013
- Article first published online: 21 NOV 2013
- Accepted manuscript online: 9 OCT 2013 05:01AM EST
In the current theme issue, we have asked three expert teams to address the question of the reliability of guidelines in the prevention and treatment of infectious diseases. Reliable guidelines should provide unbiased recommendations based on the best available evidence, allowing local application in different settings. Recommendations go a step further than a simple review of the evidence, as they may incorporate preferences, considerations of cost-effectiveness, applicability in special populations, and temporal and local particularities. Local and temporal considerations are particularly important in infectious diseases, as the epidemiology of pathogens and their resistance to antibiotics change with time and from place to place.
The three reviews have adopted three distinct viewpoints in assessing guidelines. The first is based on a survey conducted expressly for this review in 2013, and reports on National Immunization Technical Advisory Groups (NITAGs) in different European states: their composition, the procedures followed, and the criteria considered for decision-making regarding immunization policies . A second review undertakes the systematic grading of the methodology of 20 published guidelines on the treatment of pneumonia and urinary tract infections (UTIs), according to the Appraisal of Guidelines for Research & Evaluation II (AGREE-II) criteria . The third review compares recommendations on the management of invasive fungal infections of the European Conference on Infection in Leukaemia, the European Society of Clinical Microbiology and Infectious Diseases, and the Infectious Diseases Society of America, examining their grading systems, the actual recommendations, and their grading . These three reviews, from their respective angles, have identified several weaknesses and areas in need of development and improvement.
There is a clear advantage in harmonizing efforts to review the evidence on which guidelines are based, as noted in the review on immunization practices in Europe . Multiple efforts across the globe to summarize evidence on vaccine efficacy or any other topic are redundant. Systematic reviews of the literature, obtaining published and unpublished clinical trials , critically appraising primary trials and grading the evidence should be the standard. Guideline panels need to define the scope of their guideline, the conditions addressed, relevant outcomes, and relevant patient subgroups of interest. Guideline developers might then need to commission a review when one does not exist, to ask a review group to update an existing review, or address outcomes, interventions or subgroups that have not been included in the original review. Frameworks for such collaborations should be encouraged. The Cochrane Infectious Diseases Group interacts with various guideline development panels in the WHO in this way, thus improving the efficiency of both systematic review and guideline development processes [5, 6]. This leaves guideline panels the complex process of reviewing the evidence with respect to local and contemporary epidemiology and resistance patterns, addressing local applicability, and accounting for stakeholders’ preferences or values, the availability of vaccines and drugs, and the evaluation of costs and cost-effectiveness. Thus, different national guideline committees might use the same evidence summary differently. Examples include: guidelines on the use of pneumococcal vaccine or rotavirus vaccine in one country but not another, based on differences in epidemiology; a national policy to vaccinate adolescent girls against human papilloma virus, based on local preferences or values ; guidelines for the treatment of UTIs that recommend mecillinam in Denmark, but quinolones in the USA, based on local resistance patterns and drug availability; and guidelines for the treatment of fungal infections that recommend conventional amphotericin B in some settings but liposomal amphotericin B in others, based on economic considerations. Recommendations should be made even when evidence is lacking, but in this case we expect to know when a recommendation is based on expert opinion.
The process of devising guidelines for prevention or treatment should start by defining the interventions, outcomes and populations of relevance for the condition addressed. Data on these should then be sought from the existing evidence: systematic reviews, clinical trials, and, when these are lacking, observational studies. Guideline developers should be careful not to reverse this logical process by summarizing interventions and outcomes reported in clinical trials before, or instead of, defining clinically and policy-relevant outcome parameters. For example, in the review on management of fungal infections, a difference was observed with regard to the recommendation for the use of amphotericin B in the empirical treatment of candidaemia in non-neutropenic patients. In the Infectious Diseases Society of America guidelines, conventional amphotericin B received a class A recommendation (i.e. ‘good evidence to support a recommendation for use’) , whereas, in the European Society of Clinical Microbiology and Infectious Diseases guidelines, it was classified as D (‘supports a recommendation against use’) . The explanation for the class D recommendation is ‘Substantial renal and infusion-related toxicity’. If guideline developers were to define in advance the goal of empirical treatment (e.g. reducing mortality among patients with candidaemia), it is likely that both guidelines would result in a similar recommendation (based on the results for mortality in the clinical trials).
For both antifungal and antibacterial agents, there are two levels of ‘benefit’ to be considered when a recommendation is devised: in vitro coverage and efficacy. It is easy enough to consider coverage in decision-making, but more difficult to incorporate efficacy data. In the review of the guidelines for the treatment of bacterial infections, the authors found very little discussion on the effectiveness of recommended antibiotics with regard to the relevant outcomes . Rather, the more evidence-based guidelines compiled interventions that were tested and proved to be non-inferior or superior to a comparator, and based their recommendations mainly on coverage. Clinicians expect guideline developers to consider comparative efficacy data when selecting between classes of antibiotics or specific antibiotics; unfortunately, this is rarely presented in current guidelines.
Recommendations in evidence-based guidelines are classified with a level of evidence and a level of recommendation. There is a misunderstanding in some guidelines about the difference between the two, with the grading of recommendation appearing completely parallel with the evidence grading. Furthermore, there is a lack of uniformity in scoring systems. This is confusing to end-users, and makes the comparison between guidelines difficult. Finally, as noted in the review examining the methodological quality of guidelines on the treatment of bacterial infections, evidence appraisal by guideline panels is currently mostly limited to the assessment of study design. The assessment of the internal validity of study reports is very limited, and is not incorporated in the recommendation grading. In this respect, the GRADE working group is an important initiative to harmonize grading of the quality of evidence and strength of recommendations . The GRADE system addresses study design, but also the risk of bias in randomized controlled trials, inconsistency, indirectness, imprecision, publication bias, effect size, confounding, and dose–response relationship, and proposes a uniform grading system. Summary-of-findings tables based on the GRADE classification are currently included in Cochrane systematic reviews. They have yet to be adopted in current guidelines.
Public and patient consultation was a weak point identified in the reviews on immunization policies and treatment of bacterial infections [1, 2]. This was especially surprising when guidelines on the treatment of UTIs were examined, as the treatment of uncomplicated UTIs is aimed mainly at symptom improvement. It would therefore seem self-evident that women should be consulted when outcomes are prioritized. In fact, none of the current guidelines on the treatment of UTIs described a process of public or patient consultation. Similarly, when the activities of NITAGs in Europe were examined, lay members were represented in a single NITAG among the 22 surveyed. As an example, the current need for and degree of public involvement in national vaccination programmes was demonstrated in Israel, where the recent re-emergence of wild-type poliovirus triggered national recommendations to vaccinate all children who were not previously vaccinated with the live oral polio vaccine . This resulted in a huge public debate about the safety and effectiveness of the polio vaccine, subsequently evolving to a debate about childhood vaccination in general. Barring the public from decision-making regarding national vaccination programmes is likely to thwart attempts at the eradication of certain infectious diseases.
Last, but not least, there is the issue of managing the unavoidable problem of conflict of interest. There is always a concern that undue (e.g. corporate) influences may affect decision-making in guideline development groups. Although strict application of evidence-based methods in guideline development minimizes these influences, there will always remain questions where the evidence is insufficient or requires interpretation that allows personal opinion to influence decisions. We have not found sufficient attention to this problem in current guidelines and NITAGs. Conflict of interest statements are not universally provided. When they were, we did not find a description of how these were dealt with. Both as lay persons and as physicians, we would like persons with commercial conflicts of interests to be excluded from decision-making on topics related to their conflict, at the very least.
Clinicians appreciate guidelines, as they form the basis for their own decision-making, and standardize and simplify clinical practice. Guidelines published on the worldwide web are currently available at the touch of a finger. With increasing use and accessibility, guideline developers need to consider their great impact. Equally, journal editors and societies need to consider carefully the quality of published guidelines. Recently, the Dutch Q fever Consensus Group proposed guidelines for the diagnosis of chronic Q-fever . At the time of writing, these had already been cited 15 times, much more frequently than Raoult's commentary pointing at conceptual and actual inaccuracies in this consensus statement . In recent years, guidelines in infectious diseases have come a long way towards evidence-based medicine, as well as clear and uniform presentation. There remains, however, a considerable way to go, in the direction of improving their quality and transparency.
The author declares no conflicts of interest.
- 2Guidelines for the treatment of pneumonia and urinary tract infections: evaluation of methodological quality using the Appraisal of Guidelines, Research and Evaluation II instrument. Clin Microbiol Infect 2013; 19: 1106–1114., , , .
- 4Transparency interrupted: the curtailment of the European Medicines Agency's policy on access to documents. JAMA Intern Med 2013. doi:10.1001/jamainternmed.2013.9989..
- 5Oral iron supplements for children in malaria-endemic areas. Cochrane Database Syst Rev 2011; CD006589. doi:10.1002/14651858.CD006589.pub3., , , .
- 6World Health Organization. Iron supplementation for children in malaria-endemic regions. Available at: http://www.who.int/elena/titles/iron_infants_malaria/en/ (last accessed 15 October 2013).
- 11Insidious reintroduction of wild poliovirus into Israel, 2013. Euro Surveill 2013; 18: Article ID 20586., , et al.