Guidelines in disrepute: a case study of influenza vaccination of healthcare workers

What's the problem?

Despite the emphasis on the importance of high-quality evidence to underpin policy and practice, the evidence used to support guidelines may be of inadequate quality or may be misused to underpin particular advocacy positions. We discuss some of the ways in which the use or misuse of evidence in the development of guidelines can be problematic (summary provided in Table 1).

Examination of the guidelines for healthcare worker vaccination

There are clear guidelines, both in Australia and the other four countries reviewed, for HCWs to vaccinate against influenza. (Table 2) Vaccination for influenza is considered to be an efficacious public health intervention that reduces the burden of influenza illness. It can also be framed as an issue of occupational health and safety, and respect for patient safety. Vaccination programs have been one of the greatest success stories of public health. One consequence of this is that there are strongly held beliefs among individuals working in infectious disease and many working in public health, that vaccination is beneficial and advantageous under any circumstances. There are recorded instances of transmission of influenza from HCWs to patients^17,18 and it is reasonable to believe that transmission rates are higher than documented. However, there is also considerable resistance to influenza vaccine uptake: without significant efforts to encourage vaccination, staff vaccination rates are low.^19,20 Even with significant encouragement and support, in the absence of mandatory requirements, staff uptake is usually well below the universal coverage required to best protect patients.^19,20 In this paper we examine the gaps between evidence, guidelines and practice for HCW influenza vaccination rates.

We begin with the evidence. Seasonal influenza vaccines are around 80% effective in healthy adults, with the effectiveness being even higher when there is a close match between the vaccine and the circulating viral strain.²¹ There is also some evidence that the HCW vaccination for influenza decreases staff illness and absenteeism^22–24 and is cost-effective with respect to both direct costs of healthcare-acquired infection and indirect costs of staff absenteeism.^25–28

Direct evidence that HCW vaccination reduces transmission of influenza virus in healthcare settings and reduces patient mortality is more difficult to find. Individual studies and a Cochrane review of influenza vaccination for aged-care workers (which draws on two randomised controlled trials (RCTs) and one cohort study) suggest there is some evidence that this intervention is effective in protecting the elderly in care settings.²⁹ A later Cochrane review, which pooled data from three cluster RCTs,^30–32 found no effect on influenza, pneumonia and death from pneumonia, although there was lower resident all-cause mortality and reduced influenza-like illness.³³ The Cochrane review³³ highlights this inconsistency and the authors suggest that it is the result of biased selection of subjects, and/or performance bias in that the studies were underpowered to detect the outcomes of interest because of low levels of vaccine uptake in the intervention arms of the studies. Either way, currently, there is a paucity of evidence to support substantial investment in HCW vaccination programs in order to protect patients from nosocomial influenza infection in hospital settings.

Beyond this, we suggest that it is problematic to extend evidence found in aged-care settings to hospitals where visitors are more common and staff numbers larger and turning over more frequently. A single longitudinal observational hospital study found significant declines in nosocomial influenza infection among patients and staff when HCW influenza vaccination increased from 4% to 67%³⁴ but the level of evidence is low. None of the studies described above provide high-quality evidence that vaccinating HCWs would result in statistically significant reductions in nosocomial seasonal influenza infection in hospital or clinic settings. Given the economic investment in influenza vaccination programs for HCWs, the absence of high-quality evidence is striking.

However it can also be argued that such evidence is difficult to collect.³⁵ RCTs would be impossible to conduct since these would require random selection of hospitals for mandatory HCW influenza vaccination or non-vaccination. Such an approach would be unethical for a number of reasons, including infringement of personal autonomy and possible harms to HCWs from vaccination or non-vaccination. Such RCTs would also be extremely difficult to execute and prohibitively expensive. However, a natural experiment is under way, with mandatory vaccination for influenza recently instituted at some US health institutions³⁶ and it could be feasible to compare these institutions with matched control institutions.

Despite the lack of evidence, guidelines in the countries examined are often underpinned by claims that such evidence exists. For example, the Australian Immunisation Handbook 9^thEdition, in recommending the vaccination of HCWs, states “it has been shown that vaccinating [HCWs] protects those at high-risk”³⁷ and cites a single paper which relates to the use of vaccination of aged-care workers to protect the elderly in their care.³² This paper and similar papers,^30,31,38, describing work in aged care settings, are cited repeatedly in support of guidelines for universal HCW vaccination, an example of citation bias (see Table 1). Selective reporting of findings may also introduce bias. For example, the 2010 Recommendations of the Advisory Committee on Immunization Practices (ACIP) which form the basis of the US's CDC recommendations states that “A review concluded that vaccination of HCP in settings in which patients also were vaccinated provided significant reductions in deaths among elderly patients from all causes and deaths from pneumonia.”³⁹ What ACIP recommendations do not state is the other finding of the cited review: “If patients were not vaccinated, staff immunisation had no effect”.⁴⁰ The authors of the ACIP recommendations have selectively used the finding from the review which supports their recommendations for HCW vaccination while failing to include the finding from the same review which does not support their case. In a similar example, the Canadian Statement on Seasonal Trivalent Inactivated Influenza Vaccine (TIV) for 2010–11,⁴¹ reports both the findings of the papers described above and the 2010 Cochrane review,³³ while glossing over the review's findings as to the need for patient vaccination and potential bias in the included studies. It is highly probable that, in healthcare settings, most patients are neither vaccinated nor located in cloistered environs characteristic of aged care facilities. Therefore, it could be argued that there is, at present, no direct evidence to support universal HCW influenza vaccination.

Citation amplification is apparent in the numerous peer reviewed publications which advocate mandates for universal HCW influenza vaccination and in the immunisation recommendations listed in Table 2. For example, a Scopus database search for papers advocating compulsory universal HCW influenza vaccination published during 2010, which included reference to supportive evidence, identified 16 papers, all of which relied on one or more of four papers describing work in aged or long-term care settings^30–32,38 or a single longitudinal observational study.³⁴ The literature advocating universal HCW influenza vaccination resembles an inverted pyramid – a large volume of review and commentary articles supported by a very small number of empirical studies carried out in long-term care settings. In addition, the guidelines listed in Table 1 are often used to lend legitimacy to the arguments for universal and compulsory influenza HCW vaccination.

Dead-end citation is evident in a discussion paper from the Australian Influenza Specialist Group, an influential group with respect to Australian vaccination policy, which recommends that HCWs vaccinate against influenza thereby “indirectly protecting those most vulnerable to the virus”.⁴² To support this recommendation, the authors cite studies examining: HCW attitudes to vaccination,⁴³ ethical issues associated with HCW vaccination^44,45 and risk of acquisition of influenza in non-vaccinated HCWs.^46,47 None of these studies could be considered to provide direct evidence for the value of HCW influenza vaccination in the protection of patients. However, if the value of vaccination as a public health tool is considered beyond question, reflecting a culturally shared belief, then the cited papers could be seen to provide background arguments for why HCWs should be targeted as a special case and for special measures.

Finally, health messages may be over-simplified. In a culture where vaccination is seen as universally ‘good’ and non-compliance as irrational, nuanced messages about risks posed by disease and discussions about uncertainty in evidence may be seen as unnecessary and potentially confusing. Over-simplification may also result in downplaying of the potential for adverse post-vaccination events. The Australian Influenza Specialist group position document⁴² plays down adverse events, describing them as “negative attitudes and misconceptions”. In doing so, it does not acknowledge the burden of the occasional risks with seasonal influenza vaccines described later in the document of up to 10% of vaccinated individuals experiencing “symptoms mimicking a light influenza infection, such as fever” for up to 48 hours post vaccination.⁴² A figure of 10% is high but may be seen with certain influenza vaccines although systemic reactions with placebo injections may also be as high.⁴⁶ Recent RCTs in healthy adults over 2005–06 and 2006–07 showed that fever occurred post-vaccination in 3% of participants given active vaccines and 1% of participants in the placebo control arm. That is, there is evidence of a mild but real reaction in approximately 2% of vaccinated participants in what were mild influenza seasons.⁴⁸ A Canadian qualitative research study⁴⁹ showed that unvaccinated HCWs valued the protective effects of vaccination but did not believe vaccines were effective. They also believed that the vaccine made them sick and/or that the focus of vaccination was to protect patients to the potential detriment of the HCWs.⁴⁹

In the case study

In summary, we contend that there are two principal factors characterising the evidence guidelines-practice gap in HCW influenza vaccination: (i) obfuscation of uncertainty, in the evidence base and intervention efficacy; and (ii) oversimplification of the vaccination message. These factors reduce transparency and may increase mistrust between HCWs, institutional administrators and policy implementers. This has ramifications for other areas where staff goodwill and compliance are required. This may be particularly so if adverse events occur with other vaccines, such as the paediatric FluVax vaccine withdrawn from the Australian market in 2010.

In some sectors, the risk to patients of this guidelines-practice gap may be low and the principal adverse outcome may be increased staff absenteeism. However, in sectors such as aged care, intensive care and oncology, gaps in vaccine coverage of HCWs may place patients at considerable risk and compulsory vaccination of HCWs can be readily justified.⁵⁰ The provision of more complex and nuanced health information for HCWs may be beneficial, not only in terms of greater transparency but also in building trust between HCWs and administrators and policy makers. Balanced discussion of the uncertainty in the evidence and better recognition of the need for reciprocal responsibility of institutions towards HCWs, who place themselves at risk of adverse vaccination events, may increase both the level of trust and vaccine uptake.

Given the poor evidence base and the uncertainty about the degree of risk to patients from unvaccinated HCWs in many settings, it may be difficult to justify the degree of emphasis currently placed on mandating universal HCW vaccination. There may be well-founded ethical reasons why we might expect HCWs to vaccinate.^44,50 However, we suggest that, as a society, we should take particular care in making demands for strong actions and impositions on individual autonomy where the evidence base is poor.

Even with transparently drafted guidelines, many HCWs may choose not to vaccinate. This may be difficult for many working in infectious disease control to accept. There are three courses of action which might be taken. First, greater efforts should be undertaken to collect more definitive evidence that universal HCW vaccination would make a difference to patient well-being. Modelling may provide some additional information.⁵¹ Second, should it be decided that universal HCW vaccination can be justified on grounds other than evidence, then, in consultation with HCWs, systems should be adopted which acknowledge and compensate HCWs for harms experienced. Finally, it may be more useful to reinforce HCW vaccination programs where the evidence is stronger and target individuals working in high risk, relatively cloistered clinical areas. Compulsory vaccination programs in these areas could be allied with a strong emphasis on enhanced hygiene practice and staff support.

For guidelines generally

There is a growing body of literature documenting and critiquing the influence of shared scientific or practitioner community belief, in concert with professional power and corporate interests, in shaping health policy and clinical practice. In 2009, questions were raised about conflicts of interest among scientists advising the WHO on pandemic planning,^52,53 in particular, why these advisers had “declarable financial and research ties with pharmaceutical companies producing antivirals and influenza vaccines”.⁵³ Additionally, there was criticism of the lack of communication about uncertainty both in the time projections and the evidence for benefits and harms associated with pandemic vaccines.⁵³ Similarly, papers^54,55 examining the efficacy of antiviral drugs detailed concerns about drug company influence and called for independent data review. These cases, and similar concerns raised about the nature of evidence presentation in the climate change debate,⁵² provide key lessons for guideline producers. First, transparency of process is essential; the public and, in this case, HCWs, must be convinced that the guidelines are unbiased and evidence-based. Second, it is essential to acknowledge uncertainty and document the strength of evidence underpinning the guidelines. Evidence from deliberative democratic processes and risk communication theory and practice suggests that acknowledging uncertainty, in a context of respect for persons, can reduce the divisions between experts and lay publics, improve dialogue and increase policy legitimacy.⁵⁴ Disseminating half-truths and exaggerated claims are likely to increase mistrust, which spills into other areas of vaccine health promotion but, as Sandman and Lanard indicate, we do not have much evidence to support that claim.⁵⁶ Nevertheless, in Australia, there are clear guidelines on how to develop guidelines.⁵⁷ These require that the transition from evidence to guidelines be transparent and documented. It is no longer sufficient to simply provide a recommendation: the strength of the recommendation must be indicated and there are processes for documenting this.^58–60 Adhering to these processes would prevent many of the issues discussed in this paper.