Evidence-based medicine (EBM) has emerged as a dominant paradigm in healthcare, strongly influencing clinical decision-making, access to and funding for interventions. However, EBM has a number of limitations, which appear to have been under-represented in the literature. We explore the development and shortcomings of EBM, and consider a complementary role for practice-based evidence in guiding clinical decision-making. EBM is a valuable and important part of the medical landscape. However, a range of significant limitations makes over-reliance on this paradigm problematic. Appropriate recognition of practice-based evidence helps to bridge the gap between evidence and clinical practices.
Given the pivotal role of evidence-based medicine in health care and resource allocation, there is an ongoing need to critically examine its validity, and to consider constructive ways of managing its Achilles heel, the evidence gap. In this article, we explore the development and limitations of evidence-based medicine, and consider a complementary role for practice-based evidence in guiding clinical decision-making.
Uncertainty about treatment decisions is a universal source of anxiety for patients and clinicians alike (1,2). Evidence-based medicine (EBM) assists in managing clinical uncertainty, but its applicability to individual cases is rarely absolute (3). The resulting ‘evidence gap’ is typically bridged through extrapolation of available evidence to the clinical situation at hand. This approach increases the assumptions required of clinicians who naturally attempt to stretch evidence to fit the clinical scenario. This can result in a pragmatic but optimistic and sometimes misleading use of EBM at the coalface. Although rational acceptance of the evidence gap is ideal, frustrated clinicians often respond with defensive practices including excessive investigation (4) and distortion of evidence [‘sweeping uncertainty under the carpet’ (5)]. There must be a more balanced approach to utilising EBM in practice.
We conducted a focused literature review to examine the historical origins and current status of EBM, critically appraising its usefulness and limitations. We used search terms, ‘evidence-based medicine’ and ‘practice-based evidence’, to identify articles analysing these concepts. We considered factors leading to the promotion of EBM, as well as problems with the concept, including confused terminology, uncritical acceptance and other barriers to its effective implementation. Revaluation of other forms of evidence allows us to propose a synthesis of EBM with PBE as a refined model for evidence-based clinical practice.
Evidence-based medicine is now a dominant paradigm in medicine (6). In one sense, EBM is not a new approach, as medical evidence has been collected and variably applied for centuries (7). During the latter half of the 20th century, EBM gained traction with the rise of consumerism which, inter alia, called for increased accountability in medical practice (8,9). The resulting demand for evidence coincided with developments in information technology, making clinical trial results and treatment guidelines increasingly accessible (10). Methods to appraise intervention effectiveness became standardised, culminating in the establishment of the Cochrane Collaboration. In the following decades, EBM gained momentum, the actual term being coined in 1992 by Guyatt and associates (11).
Since the 1990s, EBM has become a pervasive concept across all branches of clinical medicine. It underpins management guidelines worldwide, and is taught widely in medical schools, post-graduate specialist training and continuing medical education. Paralleling its increasing popularity is an emerging medico-legal imperative that clinicians ignore EBM-sanctioned practice ‘at their peril’ (12). This has caused unease among many clinicians who prefer an arguably less rigid approach to the use of medical evidence.
Present concept of EBM
Evidence-based medicine has been usefully defined as ‘the conscientious and judicious use of current best evidence from clinical care research in the management of individual patients’ (9). Sackett emphasised that research-derived evidence needed combining with clinical expertise to achieve evidence-based practice (EBP). EBP defined in this way appears to be a sensible way of combining experimental and experiential evidence. Unfortunately, ‘EBM’ and ‘EBP’ are often used interchangeably, leading to confusion. EBM, as defined, focuses only on research evidence and, in contrast to EBP, does not extend to other factors in clinical decision-making. Replacing ‘EBM’ with ‘research evidence’ would help to clarify these concepts and simplify the debate for all concerned.
Clinical expertise is broadly defined and includes individual clinician experience together with consideration of patient rights and preferences. However, clinical expertise has clearly been undervalued in most treatment guidelines and evidence hierarchies.
Practice-based evidence (PBE) has received comparatively little attention in medical literature. Although there is no agreed definition, Girard’s description from the nursing literature is noteworthy: ‘the systematic collection of data about client progress generated during treatment to enhance the quality and outcome of care’ (13). PBE can be thought of as an extension of ordinary clinical practice whereby clinician and patient collect evidence about the latter’s treatment history and apply this to clinical decisions. This approach creates unique treatment algorithms for the patient, ideally subject to continuing revision and improvement (14). As PBE should arguably be closely linked to both patient rights/preferences and to clinician experience, a broader definition of PBE could usefully supplant ‘clinical expertise’ in the combination with EBM that Sackett envisioned (Figure 1).
EBM (research evidence) hierarchies
Several hierarchies have been developed to grade the quality of medical evidence (15). Despite subtle differences, these hierarchies share a common structure and are stratified according to risk of bias. The resulting spectrum of evidence ranges from ‘strongest’ (derived from meta-analyses and systematic reviews of randomised controlled trials), through ‘medium strength’ (observational studies, non-randomised and uncontrolled trials) to ‘weaker’ evidence (descriptive/case studies and expert opinion). The latter category thus occupies an uneasy position in conventional EBM hierarchies, dominated as they are by more ‘scientific’/objective research methodology. More subjective forms of evidence could, however, fit readily within an expanded definition of PBE.
Limitations of EBM
There are many reasons why EBM is in fact less ‘objective’ or reliable than clinicians might expect. A major limitation relates to a set of overlapping factors outside clinician control, including the determinants of research, its publication and dissemination.
Power and generalisability
There is a dearth of reliable and relevant scientific evidence in many areas of therapeutics, often because of limitations in sample size and generalisability. Flawed studies appear to be the rule rather than the exception. Systematic reviews and meta-analyses are partial solutions, but depend on the variable quality of included studies, and may in themselves be conducted in a less than objective manner. Indeed, conclusions drawn from meta-analyses are, at times, inconsistent with those drawn from subsequent large scale RCTs (16).
Researchers and meta-analysts face the dilemma of where to set inclusion criteria. More inclusive studies are typically more generalisable, but tend to have higher random variation and may also be more subject to bias (see below). Random variation in these studies is sometimes because of a failure to identify relevant sub-groups (17). Even relatively inclusive studies still tend to exclude minority groups (e.g. pregnant women, those at the extremes of age, patients with comorbidity) (18). Less inclusive studies aim to reduce random variation, but are less generalisable, and may lack power because of small sample size.
Consequently, clinicians are often presented with limited or tentative conclusions on which to base treatment decisions.
Bias affects conclusions from studies in a number of ways (19,20). Technical bias is the preferential study of topics that are more amenable to investigation by methods privileged in EBM hierarchies. Publication bias (differential publication of positive results) is still rife despite requirements by many journals and statutory authorities that trials be registered at the outset. Funding/sponsorship bias is widespread and skews published data in favour of commercially valuable interventions (21,22). Unfortunately, the ‘conflict of interest’ declarations required by many journals constitute a well intentioned but inadequate safeguard against funding bias and in some circumstances may even paradoxically worsen bias (23). Journal editors can themselves experience conflicts of interest, for example when considering publication of clinical trials with an anticipated high demand for reprints (24). Commercially unbiased sources of evidence (e.g. publications certified by the International Society of Drug Bulletins, http://www.isdbweb.org) thus provide an important complement to conventional publication of drug trials and treatment guidelines.
Context and models
Evidence in medicine is context dependent; research conducted in one setting or time period is not necessarily valid for another. Summerfield, for example, criticises the use of Western paradigms to study mental disorders in other cultures (25). Contextual differences between the origin of research-derived evidence and the setting in which it is applied pose a major obstacle to EBM; in a general sense, culture and geography can markedly limit the applicability of research findings.
Over-reliance on Western scientific paradigms has led to a myopic view of ‘alternative’ treatments, with considerable scepticism persisting in cases lacking a conventional model to explain observed benefit (26). Acupuncture analgesia is a case in point: scepticism remained until a link with endorphin release was discovered (27), legitimising gradual acceptance by the scientific community and health funders in some countries. Whether acceptance of efficacy should depend on a validated ‘scientific’ model is also controversial as some models of drug action appear more relevant to commercial promotion than to science (28).
A further determinant of treatment outcome arises from systems of classification. For example, the efficacy of antidepressant medication varies according to how depressive disorders are sub-typed (29). As classificatory systems and their diagnostic criteria evolve, it is difficult for results from future trials to be reconciled, let alone usefully combined, with results from trials based on earlier classifications. This problem is further complicated by changes in thresholds for clinical intervention. For example ‘disease mongering’ is a recognised strategy of pharmaceutical companies and other commercial entities to expand markets by promoting treatment for indications at the margins of recognised disease entities or laboratory abnormalities (30).
Ultimately, interventions should be justified based on clinical outcome (18). There is a danger in assuming that relying upon and modifying surrogate end-points (e.g. serum cholesterol) will change clinical outcomes (e.g. cardiovascular events), unless these putative links are specifically studied and verified. There is often merit in investigating such end-points as indicators of potential benefit, but clinicians are warned that corroborative evidence is required to show causality and changes in clinical outcome. Without such caution, reliance on surrogate end-points can influence therapeutic decisions in dangerous and expensive ways (31,32).
Together, these issues limit the validity of trial results and narrow the total body of data and analyses from which reliable conclusions can be drawn. Critical appraisal (see below) comprises an important, but as yet incomplete set of strategies to assess these issues and their impact.
Another limitation of EBM arises from factors primarily under clinician control.
Skills in critical appraisal are now taught widely in undergraduate and postgraduate medical settings in many countries. The task of evaluating clinical trials is often delegated to trainees and medical students, with the intention that these practices will foster routine critical appraisal. Access to evidence for critical appraisal is facilitated by the increased availability of research and decision support tools on the Internet, notably including open access journals and the Cochrane Library. Notwithstanding a growing awareness of statistical concepts used in clinical research, most clinicians have neither the time nor the expertise to apply them routinely. Many are understandably reluctant to invest time in a process that often fails to deliver robust conclusions and leaves clinical questions unanswered.
Most clinicians are interested in ‘bottom lines’ from good quality research and appropriate integration by meta-analysis. This may account for the popularity of editorials that review and critically appraise research findings published in the same issue of major journals such as BMJ and JAMA.
Access to evidence base
The evidence ‘base’ in medicine is always provisional and exists to be superseded by better evidence. However, because of the time it takes for large trials and meta-analyses to produce quality evidence, clinicians typically become aware of evidence for specific clinical issues in bursts rather than continuously. The process of incorporating best evidence into clinical guidelines is unfortunately even more tentative, despite the intent of professional colleges to continually improve guidelines (33). The episodic incorporation of evidence is often out of phase with the continuous emergence of new data, and reduces the certainty clinicians can expect of treatment guidelines at any given time. This leaves clinicians in a precarious position when faced with complex cases in which typical intervention options have been exhausted. Should clinicians adhere to the boundaries of professionally endorsed treatment guidelines that, although limited in many circumstances, afford some measure of collegial support and medico-legal protection? Or should a novel intervention with ‘weaker’ evidence be trialled? Peer review (which can be considered as part of PBE) is a valuable exercise in these situations to garner support for, and share the results of, novel treatment approaches.
Limitations of EBM in practice
The limitations of EBM constrain the amount and quality of evidence upon which clinicians can base decisions. However, even the application of robust, quality evidence is not without its problems. One of the main criticisms levelled at EBM relates to the problem of applying global evidence to individual patients. Clinical trials give good estimates of average effects, but these, by definition, do not apply to everyone (34). Average effects obscure response heterogeneity between sub-groups of patients (17). Mechanisms producing response heterogeneity are complex, and only a subset can be presently understood through variations in pathophysiology, enzyme and other genetic polymorphisms. Heterogeneity in target populations naturally limits the degree of generalisability of research evidence, thereby compromising the external validity of trials. Sub-group analyses offer a partial solution to this problem, but require careful, preferably a priori, hypotheses and definition of categories. Unfortunately, sub-group analyses are limited by the twin problems of reduced statistical power (because of reduced sample size), and sometimes also the Bonferroni effect of spurious associations arising from multiple analyses, or ‘data dredging’ (35).
Another issue concerns how clinicians utilise research evidence. Gabbay and May found that clinicians rarely access and apply explicit evidence (36). Instead, clinicians develop tacit, internalised and collectively reinforced guidelines (‘mindlines’), informed by brief reading, but mainly by networking with other clinicians, ‘opinion leaders’, and patients. Their findings support the promotion of networking as an important source of locally informed healthcare knowledge, and of targeting opinion leaders to ensure that knowledge with a sound evidence base is being filtered down to local levels. Mindlines also fit well within the concept of PBE.
Communicating evidence to patients in a meaningful way requires considerable skill. An important development has been the use of Numbers Needed to Treat or Harm (NNT/NNH), which conveys relative benefit/harm rates to patients in an understandable way (37). This can be enhanced by the use of visual treatment analogue diagrams and natural frequencies (18). Although the calculations required to arrive at NNT/NNH are theoretically simple, the enormous amount of data supplied by some trials makes this task daunting for many clinicians. NNT/NNH measures are yet to be consistently provided by authors of large trials, although their increasing visibility in systematic reviews and meta-analyses is encouraging.
Taken together, the limitations of EBM weaken its contribution to EBP and widen the evidence gap perhaps more than clinicians generally appreciate or accept. Although it is useful to address these limitations, PBE may provide a complementary approach to bridging the evidence gap.
Evidence-based practice re-discovered
Considering the factors described above, it is timely to review our understanding of EBP, and to propose a refinement of the model (Figure 2). As discussed, research-derived (scientific/objective) evidence can be more usefully described as research evidence (in preference to ‘EBM’), allowing various forms of more subjective evidence to be considered within a broader concept of PBE. With a clearer distinction between predominantly objective and subjective evidence, research evidence and PBE can sit in partnership, rather than at opposite ends of a fragmented hierarchy of strength.
Disclosure and informed consent
With a grasp of both the benefits and limitations of research evidence, what then should be communicated to patients about evidence? Although there may be an ethical and medico-legal imperative to provide patients with a balanced view, disclosure of the limits of research evidence is a double-edged sword. On the one hand, disclosure is consonant with contemporary medical values, creating a more equal distribution of knowledge (and therefore power) in the clinician-patient dyad. Such a stance is also in keeping with the principles of informed consent. On the other hand, patients may be required to accept more uncertainty (and therefore anxiety) depending on the quality of the evidence presented. This may translate into distress for the patient, and a likely reduction in the magnitude of useful placebo effects.
Practice-based evidence offers a solution to the disclosure dilemma. When approached collaboratively, PBE can alleviate illness-anxiety related to the uncertainty inherent in research evidence. Statements such as ‘…the research regarding which medication will work best for your illness is not conclusive, but based on current evidence, my experience and your treatment history, x is a reasonable choice…’ are reassuring as they provide a sense of control in an uncertain situation, without attempting to unreasonably stretch the evidence to fit the clinical scenario.
Implications for clinical practice
Research evidence generated using the EBM paradigm has profoundly influenced clinical practice at the individual level and in public policy. Clinicians are strongly encouraged to use interventions which appear to have a strong evidence base. Research evidence-sanctioned interventions are also more likely to be funded by governments and insurers (38). Sole reliance on research evidence, however, may lead to the often erroneous conclusion that interventions arising out of PBE but yet to be endorsed by formal RCTs are necessarily inferior, and consequently not offered to patients who may benefit. There is little ethical basis to deny patient access to such interventions, particularly in the absence of applicable research evidence for alternatives. PBE should be formally recognised as an essential part of EBP. Furthermore, standardising PBE in medical records (using objective outcome measures) may lead to integration with research, making the everyday ‘subjective’ more ‘objective’ and contribute to research outcomes (14). Bayesian statistical approaches may further assist in combining clinical particulars and subjective evidence with objective evidence, particularly where there is paucity of the latter (39).
Evidence-based medicine (research evidence) has come to dominate clinical decision support in recent decades, heavily influencing what clinicians believe about interventions, and significantly determining the availability of treatments. Clinicians and funders require awareness of both the benefits and limitations of research evidence, and its corresponding uncertainty. Recognising an expanded concept of PBE as a valid and important partner to research evidence offers a means of bridging the evidence gap. Clinicians should be encouraged to balance the tensions between research evidence and PBE to optimise the use of EBP in individual patients.
Both authors developed the concept of this work and contributed equally to the literature review and analysis. RD drafted the first version of the manuscript.
Both authors declare that they have no competing interests with regard to this work.
The authors received no specific funding for this study. We thank Professor Graham Mellsop, University of Auckland, for discussions.