The case for, and against, evidence-based psychiatry
Evidence-based medicine (EBM) in psychiatry is under attack: either in the name of biological (as opposed to clinical or statistical) methods (1, 2) or as a post-modernist manipulation of the pharmaceutical industry (3) or as ignoring the humanities (4) or as still reflecting the disguised persistence of authority (5). Defenders of EBM tend to allude, somewhat dryly, to the benefits of objective measurement (6). Rarely are papers both defences and critiques of EBM simultaneously (7).
I believe both the attacks and the defences of EBM have been misplaced.
Galenic vs. Hippocratic medicine
The key rationale for EBM, in my view, is historical: we need clinical observation, with statistical evaluation, because otherwise we tend to practice medicine poorly: we believe in false theories for immense periods of time, and we make faulty observations. Let me defend these theses.
There are, and always have been, two basic philosophies of medicine. One is Galenic: there is a theory, and it is right. For our purposes, the content of such theories do not matter [they can be about humours, or serotonin and dopamine neurotransmitters (8) or ECT (9) or even psychoanalysis]: what matters is that hardly any scientific theory (especially in medicine) is absolutely right (10). The error is not so much in the content, but in the method of this way of thinking: the focus is on theory, not reality; on beliefs, not facts; on concepts, not clinical observations.
There is, and has always been, a second approach, much more humble and simple – the idea that clinical observation, first and foremost, should precede any theory; that theories should be sacrificed to observations, and not vice versa; that clinical realities are more basic than any other theory; and that treatments should also be based on observational bases, not ideas. This second approach was first promulgated clearly by Hippocrates and his school in the fifth century BC, but Galen demolished Hippocratic medicine (while claiming its mantle) and it lay dormant until revived 1000 years later in the Renaissance (11, 12).
From the fifth century AD until the past century, Galen’s theory about the four humours ruled medicine. Its corollary was that the treatment of disease involved getting the humours back in order; releasing them through bloodletting was the most common procedure, often augmented by other means of freeing bodily fluids (e.g. purgatives and laxatives). For 14 centuries, physicians subscribed to this wondrous biological theory of disease: we bled our patients until they lost their entire blood supply; we forced them to puke and defecate and urinate; we alternated extremely hot showers with extremely frigid ones – all in the name of normalizing those humours (13). Yet, it all proved to be wrong.
How could we be so wrong for so long?
Perhaps the core problem is confounding bias (14). Behind the menacingly statistical phrase lays a deep and very basic clinical problem: we, clinicians, cannot believe our eyes. Confounding bias means that in the course of your clinical experience, there are many other factors of which you are not aware that can impact what you observe. Thus, it can appear that something is the case, when it is not; that some treatment is improving matters, when it is not. And these confounding factors are present not just some of the time, but most of the time.
Now perhaps most clinicians would admit this basic fact, but it is important to draw both the clinical and scientific implications.
Clinically, the reality of confounding bias teaches us the deep need for a Hippocratic humility, as opposed to a Galenic arrogance (Galen once said, ‘My treatment only fails in incurable cases’) – a recognition that we might be wrong, indeed we often are, even in our most definitive clinical experiences (11). Everybody thought Galen was right for 14 centuries; the end of Galenic treatments came about in the 19th century because of EBM –‘the numerical method’ of Pierre Louis, who showed, by counting about 70 patients rather than relying on single cases or clinical experience, that bleeding accelerated death in, rather than cured, pneumonia (13). Counting patients, the numerical method, EBM – that has been the source of the greatest medical advances, not the exquisite case study, nor the brilliance of any person (be he Freud or Kraepelin or even our most prominent professors today), nor decades of clinical experience.
Truths of theory are transient. Not only is Galen out of date, but so is the much vaunted catecholamine theory of depression; today’s most sophisticated neurobiology will be passé by the end of the decade. Clinical observation and research, by contrast, is more steady: that same melancholia that Hippocrates described can be discerned in today’s major depression; that same mania that Arateus of Cappadocia explained in the second century AD is visible in current mania [obviously social and cultural factors come into play, and such presentations vary somewhat in different epochs, as social constructionists will point out (15)]. Clinical research is the solid ground of medicine; biological theory is a necessary but changing superstructure. If these relations are reversed, then mere speculation takes over, and the more solid ground of science is lost.
Scientifically, confounding bias leads to the conclusion that any observation, even the most repeated and detailed, can be – indeed often is – wrong; thus valid clinical judgments can only be made after removing confounding factors (14, 16). Randomization, developed by the biologist Ronald Fisher in the 1920s (17), is the most effective way to remove confounding bias. Hormone replacement therapy was the cure for all kinds of feminine ills: decades of experience with millions of patients, huge observational studies with thousands of subjects and the almost unanimous consensus of experts, all came to nought when randomized studies proved the futility of the belief in that treatment, not to mention its carcinogenic harm (18).
If we accept, then, that clinical observation is the core of medicine (rather than theory), and that confounding bias afflicts it, and that randomization is the best solution, then we have accepted EBM. That is the core of EBM, and the rationale for the levels of evidence where randomized data are more valid than observational data (7). These are new methods (the first randomized clinical trial, RCT, in medicine occurred in 1948 with streptomycin, directed by A. Bradford Hill), and major advances in medical treatment in the past 50 years are unimaginable without RCTs in specific, and EBM in general. Indeed, perhaps the greatest public health advance of our era – the linking of cigarette smoking and cancer (led by Hill) – was both source and consequence of EBM methods (although not, interestingly by RCTs, but by attention to better statistical analysis of clinical observations, i.e. epidemiology).
Cochranian Oxford and Galenic Rome
I think there is a legitimate critique to be made of EBM, but it is not in medical traditionalism (2) nor in post-modernist history (3). None get at the core rationale for EBM: the reality of confounding bias.
The main problem with EBM, in my view, is that, when conducted on an industrial scale, and rarefied in the ivory tower, it returns us to Galenic arrogance. The Cochrane Collaboration is perhaps the greatest manifestation of what might be called ivory-tower EBM – the idea that unless there are double-blind randomized placebo-controlled data, then there is no ‘evidence’ (7). The EBM concept of levels of evidence is useful, and the Cochrane group deserves credit for upholding it; the fetishization of RCTs is harmful, for there always is evidence; indeed the whole point of EBM is to give us a method to weigh that evidence. Even non-randomized evidence may be correct and useful in the absence of randomized data or given certain constraints; for instance, the link between cigarettes and smoking is completely based on non-randomized evidence but with a great deal of careful statistical analysis to assess confounding factors.
If the Cochrane Collaboration set the standard, then we should avoid penicillin, and not counsel against tobacco.
This ivory-tower EBM exhibits a rarefied positivism that reflects a lack of understanding of the nature of evidence (and science) (7). In my experience, it is not uncommon to hear academic leaders (and journal peer reviewers) disparage important observational data as mere ‘chart reviews’, as if they are thereby useless. This kind of fetishization of RCTs reflects a misunderstanding of science. We need informed critiques of EBM – because it can be misunderstood, and even abused – not to destroy, but rather to improve it.
Otherwise, as Alvan Feinstein aptly put it, we would only be replacing the tyranny of Galenic Rome for that of Cochranian Oxford (19).
In sum, those who think EBM cannot be applied to psychiatry should think about the implications given the history of medicine. Without the application of scientific principles to clinical research, we will have nothing but opinion – a post-modern relativist world where all is ideology. Without scientific, evidence-based clinical research, in the Hippocratic tradition of careful attention to clinical observation – and its statistical correlates in the need for combating confounding bias – psychiatry, and all of medicine, would be but a mere shadow of what is, and a pale reflection of what it can be. On the other hand, industrial scale meta-analyses of our limited RCT database will only get us so far, and will overlook or misinterpret many important truths. Medical traditionalism and post-modernist relativism are not the solutions, but neither is ivory-tower EBM. Recovering the Hippocratic heritage of scientifically sound clinical observation – above and beyond biological theory, post-modernist critique, and industrial number-crunching – should be our goal.