The need for an evidence base for practice underpinning the management of those with a chronic condition is clear. The increasing numbers of individuals with a chronic condition mean that effective and efficient care is crucial for the running of health services. In addition, those living with a chronic condition need to be assured that the most appropriate treatment and the best possible management advice are being offered by their healthcare professionals. It is generally recognised that the best way to provide evidence is through the use of randomised controlled trials (RCTs). However, when investigating the management of chronic illness, there are factors that present serious limitations to the appropriateness of RCTs in terms of quality of design and external validity of the results produced.
Randomised controlled trials are widely cited as providing the best form of evidence for clinical practice and for many questions they probably are the best. According to Pocock (1983), the evaluation of new treatments was a fairly haphazard process until the 1980s when properly conducted clinical trials were recognised as offering ‘the only reliable basis for evaluating the efficacy and safety of new treatments’ (Pocock 1983, p. 1). Since then procedures and guidelines have been developed and refined to enhance the clarity and rigour of each stage of an RCT (Altman et al. 2001).
The strengths of RCTs are not in question. For example, when testing out a new medication in which one group takes the new product whilst the other group takes the placebo. What is challenged is the appropriateness of this methodology when investigating the impact of innovations for the management of chronic illness in which there is a significant behavioural component.
This is not a new dilemma.
The RCT is a very beautiful technique, of wide applicability, but as with everything else there are snags. When humans have to make observations there is always the possibility of bias (Cochrane 1972, p. 2).
In recognition of the problems faced when designing experiments involving behaviour to improve health, the Medical Research Council developed a framework to guide research development (Campbell et al. 2000). This framework illustrates that preliminary stages are required before an RCT with a complex intervention can be realistically conducted.
In a clinical trial, great care is taken to control all variables that might impact the research topic and to manipulate only those being tested in the experiment. Yet when researching people who have chronic conditions, the effects of personality, social environment and the enduring nature of the condition on each individual may all have an uncontrollable impact on the rigor of the trial. According to Pocock (1983) the first randomised experiments were conducted in agriculture in which the experiments were based on plots of land to which different crops or fertilisers were assigned. He points out that this enabled the experimenter to have all units available at one time and to tightly control the situation, unlike research in which people are the unit of experiment. There are now several papers in which the appropriateness of RCTs, in the context of chronic illness management, is challenged.
Lindsay (2004) drew attention to the problems of applying RCT methodology to socially complex interventions. Such interventions are those ‘characterised by actions that are difficult to define and by varied, and difficult to control, contextual factors’ (p. 85). Most interventions designed to improve or assist in the management of chronic conditions are complex because living with chronic illness will always be dominated by contextual factors. For example, Glasgow (1999) presents a pyramid of social-ecological perspectives that may both influence and impact upon outcomes relevant to diabetes self-management education. The pyramid includes what he terms formal spheres of influence such as healthcare systems, work sites and policy actions as well as informal ones including social, physical and environmental factors. All these factors may influence the behaviour of an individual involved in a clinical trial and conversely a trial would need to take into account an array of outcomes, not only the physiological ones that tend to be most readily measured.
Rolfe (2002), on the other hand, points out that there may be philosophical limitations to using RCT methodology. Clinical trials are, out of necessity, protocol driven; thus, they may deny individuals the opportunity to use their own initiative and expertise while following a pre-ordained protocol. This approach is at odds with current health policy espousing the importance of empowered patients or indeed expert patients.
Patients and healthcare professionals will always have a right to some degree of free will and judgement in a healthcare situation; therefore, interventions that are socially or behaviourally complex will always be impossible to control fully. Heller (2002) points out that clinicians must learn to accept patients’ choices even when they challenge professional opinion and cites the stance of Műlhauser and Berger (2000) who remind us that ‘failure to respect a truly informed choice is probably unethical’ (p. 263). However, when patients or healthcare professionals exercise their right to make choices, in the context of an RCT, they may also undermine its rigour.
The opportunity for participants to make choices may occur at various stages in the process of an RCT and a few aspects that are notoriously difficult to control are mentioned below. With respect to patient recruitment, all patients must be fully informed about the study before agreeing to participate yet gaining informed consent is known to be difficult (Newton et al. 1998; Stead et al. 2005).
Once patients have agreed to participate they are randomised to either an experimental group or to usual care. Randomisation can be a difficult concept for potential participants to understand (Snowdon et al. 1997). Some patients will only take part in an experiment if they are in the intervention arm, while some prefer to have usual care; this may lead them to withdrawal from the study if they are not randomised to their preferred group. Thus, the allocation may not be truly random. Furthermore, randomisation can be fraught with bias created by those running the trial. According to Schulz (1995, p. 783):
Researchers need to realise that, given the opportunity, trial implementers will frequently subvert the intended aims of random assignment.
Thus, steps need to be taken to ensure that those with an interest in the outcomes of the clinical trial cannot influence the allocation of participants to the experimental or control group.
Sample size is of crucial importance to a trial (Warlow 2002) yet gaining informed participation and randomising can be problematic; so, it usually takes longer than expected to achieve the required sample size. If money or time is running short, the trial may need to be concluded with an inadequate sample size; this in turn will mean that the evidence generated by the research is less robust.
In behavioural research, lack of concordance with the intervention can prove to be a major limitation as it is known that many patients do not follow prescribed treatment regimens (Donnan et al. 2002). Imagine a behavioural intervention that involves dietary change or an increase in exercise over time. The extent to which those in the intervention group actually do make these changes will have a crucially important influence upon the results. If the outcome measures between the intervention and control groups are non-significant, it could be difficult to determine whether the intervention was ineffective per se or whether it was not followed. The degree of individual concordance with the intervention needs to be measured and ideally allowed for in the analysis.
The above issues are mentioned as examples RCT limitations that can have a detrimental effect upon the quality of the subsequent evidence produced. However, another issue of relevance to RCTs in chronic illness management is that even when well designed, the amount of selectivity and control demanded by the trial actually detracts from the extent to which the results may be widely applicable.
A study has high external validity if the results can be generalised to other people. There are several threats to external validity, but sampling and intervention support will be mentioned as examples. Most RCTs have strict inclusion and exclusion criteria. These serve a useful purpose as the criteria are set to exclude other confounding variables. However, there may be tension in excluding many variables yet still having a sample that can be generalised to the study population in question (Yusuf et al. 1990). The criteria may lead to a final sample that does not represent the population from which it is drawn and the intervention, if found to be effective, may not be of benefit to the wider population.
The nature of the intervention and the amount of support it requires is also very important. It may be possible to apply an intervention that requires intensive support in a research setting while staff or resources have been funded but such interventions are less likely to be translated to general health care. Thus, an RCT may appear to provide the best evidence but may lack external validity.
In addition, interventions need to be clearly described and this can be difficult when using socially complex social interventions or involving behaviour change in which individual opinion usually plays a part. It may be hard to capture the essence of what aspect of the intervention is actually making a difference.
An investigation needs to be both robustly conducted and also have a high degree of external validity before it will have the potential to be widely applicable to health care.
This is not a new dilemma
Over the years, there has been a growing voice calling for other approaches to research to improve care for those with chronic illness to be given priority. Glasgow et al. (2001) recommend that research needs to move from efficacy studies to those that will make a difference to care in practice. They ask for research to involve a wider range of patients to be representative of communities and to include the currently under-served and minority populations. In addition, research needs to be broad enough to effect health delivery systems rather than very narrow and controlled elements of care.
Heller (2002, p. 264) also argued that it was time for a change in research priorities:
The final decades of the 20th century were the era of the large-scale trial and molecular genetics. The early years of the new century need to be the era of research into delivering effective health care.
However, generalisable research needs substantial funding and, while clear arguments can be made to support the need for research designs other than RCTs, the funding focus also needs to change (Glasgow et al. 2001).
The tipping point
If RCTs are to be applied to interventions relating to the management of chronic conditions, then the design must minimise potential limitations. However, for some research questions, it may be decided that an RCT is not an appropriate design and that other approaches have more to offer. The points raised above are not new, yet the reliance placed on RCTs has not diminished. So, the question raised is where is the tipping point? What needs to happen in order that other robust means of evaluating complex interventions are given a higher priority in the research hierarchy. This means that there are opportunities for such investigations to be funded and also that the results once obtained are respected. According to Gladwell (2000), great strides can be made if those few special people who hold social power can be persuaded of the need for change. He suggests that tipping points are a reaffirmation of the potential for change.
Look at the world around you. It may seem like an immovable, implacable place. It is not. With the slightest push – in just the right place – it can be tipped (Gladwell 2000, p. 259).
We need to locate the tipping points.