Under-recruitment of patients for clinical trials: an illustrative example of a failed study
Article first published online: 5 APR 2007
Acta Psychiatrica Scandinavica
Volume 115, Issue 5, pages 337–339, May 2007
How to Cite
Rendell, J. M. and Licht, R. W. (2007), Under-recruitment of patients for clinical trials: an illustrative example of a failed study. Acta Psychiatrica Scandinavica, 115: 337–339. doi: 10.1111/j.1600-0447.2007.01019.x
- Issue published online: 5 APR 2007
- Article first published online: 5 APR 2007
There are few trials comparing active treatments for acute depressive episodes in bipolar disorder. Accordingly, the non-industry sponsored trial reported on in this issue by Nolen et al. (1) addressed an important clinical question by comparing tranylcypromine and lamotrigine, i.e. two treatments with quite different pharmacological profiles, as add-on to a mood-stabilizer in the treatment of patients with bipolar depression not responding to conventional antidepressants. It is disappointing therefore that only 20 of the planned 70 participants were recruited, making it a failed study. This occurrence, however, is far from unusual: many randomized clinical trials (RCTs) fail to recruit with negative effects on the usefulness of the results (2, 3). Within psychiatry, there are several other recent examples of RCTs that have experienced serious recruitment problems. In the UK, the CUtLASS trial which compared the first and the second generation antipsychotics for treatment-resistant schizophrenia achieved only 40% of the target sample size (4) and the AD2000 trial of cholinesterase inhibitors for mild to moderate Alzheimer's disease achieved only 19% of the planned sample (5). Reports from both trials and from the ongoing BALANCE trial of long-term treatment for bipolar disorder (6) highlight the disparity between the number of psychiatrists who agree to take part in a trial and the number who actually enroll patients. In addition to reducing the robustness of the results of individual trials, non-recruitment by investigators greatly increases the cost per participant in trials thereby reducing both the number of trials that can be undertaken and the willingness of funding agencies to fund trials in psychiatry.
Trials with inadequate sample sizes may fail to detect clinically worthwhile differences between treatments, and surveys of trials of antidepressants and of treatments for schizophrenia have found that many trials are underpowered (7, 8). It is particularly confusing when small underpowered trials come out with numerically large but statistically non-significant differences between treatments. This is well demonstrated in the trial by Nolen et al. (1). Numerically, the response rate was higher for tranylcypromine than for lamotrigine but the difference was not statistically significant. Indeed, the small sample size resulted in wide confidence intervals that do not exclude that the response rate for lamotrigine could be over twice the size of that for tranylycypromine.
So, why does such dramatic under-recruitment occur? In a recent study, one of the reasons identified by psychiatrists with expertise in conducting clinical trials was the fact that investigators have treatment preferences and do not think that there is equipoise among active treatment arms (9). There is mounting evidence that equipoise can be obscured when new publications appear during the recruitment phase, even when they do not provide an answer to the clinical question (10). In CUtLASS (4), the publication of NICE treatment guidelines (11) during the recruitment phase recommending the use of the second rather than the first generation antipsychotics as first-line treatment for schizophrenia may have reduced the recruitment rate. A similar effect was noted in AD2000. The recommendation in 2000 by NICE guidelines that cholinesterase inhibitors should be used to treat mild Alzheimer's Disease (11) not only reduced the recruitment rate but also led to randomized participants being withdrawn from the trial (5).
Nolen et al. (1) also attribute the low rate of recruitment and clear preference for lamotrigine rather than tranylcypromine at the USA site to new publications. First, new data on the efficacy of lamotrigine became available and the drug was given FDA approval for relapse prevention in bipolar disorder. Secondly, the revised version of the APA practice guidelines (12) listed lamotrigine among the preferred drugs for bipolar depression. However, neither factor reduced the level of uncertainty about treatment for treatment-resistant bipolar depression and in particular about the relative pros and cons of lamotrigine and tranylcypromine.
The fact that the trial by Nolen et al. (1) recruited patients who were already participating in the Stanley Foundation Bipolar Network offers further valuable insights into the size of the possible impact of investigators’ belief of lack of equipoise on recruitment. Based on information given in their paper we estimated that, in the Netherlands, approximately 30% of patients potentially eligible for the trial (here defined as all patients treated with lamotrigine or tranylcypromine) were randomized whereas, in the US, less than 1% of these potentially eligible patients were randomized. This difference can most likely be explained by the investigators’ preferred use of lamotrigine at the US site, as 226 out of 240 (94%) non-randomized (but potentially eligible as defined above) patients received this drug. In contrast, only 24 out of 41 (58%) non-randomized (potentially eligible) patients at the Dutch site were treated with lamotrigine. This happened despite the commitment of the investigators to the Network (as stated in the paper). Another point of general concern is that investigators’ preferences to treat eligible patients without randomization may lead to a highly selected final sample, i.e. a sample selected beyond the formal selection criteria, potentially creating limitations in generalisability of study results.
Nolen et al. (1) argue that funding was not a factor in their under-recruitment. However, had this been an industry-sponsored trial given high priority, additional centres might have been established, solving the power problem, but not the selection problem.
Among other reasons for under-recruitment to trials in psychiatry, is the fact that many patients are unwilling to give their informed consent or are too ill to be approached for the trial (13). There is no specific information related to this issue in the report, but as all the patients at least had consented to participate in the Network, these factors seem not to have influenced the under-recruitment to any larger extent.
A recent systematic review did not find a consistent impact for any factor on under-recruitment for any factor across different trials (14), although it is clear that individual factors may contribute to the problem in single trials. One of these factors, as indicated above, is the extent to which investigators believe that there is non-equipoise and the effect of new information on that belief. One potential strategy to improve recruitment would be to work with the commitment of the investigators, explicitly addressing beliefs of non-equipoise and discussing both the implications of any new data that arise and the limitations of guidelines when they are based on inadequate evidence. This may be facilitated by setting-up of frameworks ensuring a systematic evaluation of all potentially eligible patients, including documentation (used in consensus meetings) on each eligible patient that was not randomized. In a recent completed investigator-initiated multi-centre trial, such a strategy was successfully implemented at the main centre where more than 80% of the total study sample were recruited, despite the fact that the five other participating centres had comparable catchment areas (15).
- 4Randomized controlled trial of the effect on Quality of Life of second- vs first-generation antipsychotic drugs in schizophrenia: Cost Utility of the Latest Antipsychotic Drugs in Schizophrenia Study (CUtLASS 1). Arch Gen Psychiatry 2006;63:1079–1087., , et al.
- 5AD 2000 Collaborative Group. Long-term donepezil treatment in 565 patients with Alzheimer's disease (AD2000): randomised double-blind trial. Lancet 2004;383:2105–2115.
- 9Barriers to recruitment to RCTs: a Delphi study. Fifth European Stanley Conference on Bipolar Disorder. October 5th–7th 2006 Abstract 1–050 in abstract book (Oral presentation and poster)., .
- 11The National Institute of Clinical Excellence (NICE). Accessed online http://www.nice.org.uk
- 12American Psychiatric Association. Practice guideline for the treatment of patients with bipolar disorder, revision. Am J Psychiatry 2002;159(suppl.):1–50.
- 14Incentives and disincentives to participation by clinicians in randomised controlled trials. The Cochrane Database of Systematic Reviews 2006, Issue 4. Art. No.: MR000021., , .