Partially randomised patient preference trials

This edition of the Journal contains a report of a comparison between a conventional randomised controlled trial and a partially randomised patient preference (PRPP) trial. This pioneering piece of research has come from the Health Services Research Unit and the Department of Obstetrics and Gynaecology at Aberdeen University. Although the trial report concisely justifies the reason for comparing a conventional randomised controlled trial with that of a PRPP trial it may be worth considering in some more detail what a PRPP trial is, why this design has been used, its potential advantages and disadvantages, the evidence that exists to support its use and the future of this form of trial design.

What is a PRPP trial?

Let us consider a randomised controlled trial which seeks to compare two treatment options, treatment A and treatment B. These may be drugs, surgical procedures or packages of care or as in the case of the trial reported in this issue of the Journal (pages 1367–1373)¹ they may be medical versus surgical management. In a standard conventional randomised controlled trial patients who were eligible for the trial would be randomised to receive either treatment A or treatment B. Depending on the nature of the treatments in the trial, patients who decline to enter may be offered one, or either of the two treatments, as judged necessary by the attending clinician in consultation with the patient. If one of the treatments is a new drug this is unlikely to be available as an option outside the context of the trial. If the intervention is a new technique, such as was the case with the Neonatal ECMO (extracorporeal membrane oxygenation) Trial, then this option may also not be available outside the context of the trial. If, however, the trial compares two existing therapies it is likely that both of the two options in the trial are available to the clinician and the patient.

In a PRPP trial eligible patients are offered three options. Either they can be randomised (to treatment A or treatment B) or they can choose to have treatment A, or they can choose to have treatment B^2–4. In agreeing to choose their treatment they consent to enter a study which will seek to follow them up in the same way as patients participating in the conventional randomised controlled trial.

The rationale for randomisation is the control of bias and in particular the elimination of confounding by random allocation to treatment so that the two groups being compared are as similar as possible for all known and unknown factors which may influence the outcome under investigation. This is the basis for all conventional randomised controlled trials. This unbiased comparison between two or more groups also remains essential to the design and undertaking of a PRPP trial in that the sample size required for the randomised component of a PRPP trial will be the same as that needed for the conventional randomised controlled trial⁵. Therefore the total population under investigation in a PRPP trial will always be larger than a conventional randomised controlled trial.

Why use a PRPP trial?

One of the criticisms levelled at conventional randomised controlled trials rests on the assertion that patients who are involved in the decision making process concerning their care and treatment are likely to do better than those who have treatments or aspects of care imposed upon them⁶. The process of random allocation of treatment takes all elements of choice away from the patient and although all patients who agree to enter randomised controlled trials should be aware of the process before they consent to enter, it is likely that many patients are unhappy with their allocated treatment and would have preferred the alternative⁷. This may be particularly so where the ‘newer’ treatment is not available outside the context of the trial. For instance, in the Neonatal ECMO Trial, some of the parents of the babies who were randomly allocated to conventional treatment felt that they had missed out on the opportunity of the new treatment which many of them assumed was superior⁸. Dissatisfaction with the allocation may lead to differential

compliance and follow up of the two trial arms resulting in two groups which cannot then be assumed to be similar in every way apart from the treatment under investigation. In addition the outcome measures could be affected by how satisfied the patients are with their allocated treatment. The effect that patient preference has on outcome would appear to depend on the particular outcomes being measured. If the principal outcome of a trial is death then the effect that patient preferences are likely to have would probably be small, whereas if the principal outcome is satisfaction with care, the effect that patient preference may have could be large⁹. If patient preference has a major effect on the relationship between the intervention and the outcome then conventional randomised controlled trials will tend to underestimate the strength of this relationship i.e. the results of the randomised controlled trial will represent the minimum effect size of the intervention. A comparison between two groups of patients who have chosen their treatment, thereby optimising their satisfaction with the choice of treatment has been considered by some to represent the maximum effect size of the intervention. It is then assumed that such an intervention when adopted into routine clinical practice will have an effect size somewhere between the minimum and maximum seen from the conventional randomised part of the trial and the preference part of the trial⁵.

The justification therefore for partially randomised PRPP trials is one of generalisability. How are the results of trials interpreted and what impact will they have if interventions are adopted into routine clinical practice where the choice of one intervention over another is a complex process which involves an element of decision making by the patient? The extent to which the PRPP trial can achieve this has been much discussed but little investigated.

Potential disadvantages

Some of the advantages of PRPP trials have been touched on in the section above and relate to the interpretation of the results of PRPP trials. There are, however, potentially major practical disadvantages with this design of study. Firstly there will be effects on the trial size. Although this type of trial design will not influence the total size of the population that participants are recruited from, the size of the total PRPP trial cohort will need to be much larger than that for a conventional randomised controlled trial. As already mentioned the randomised element of a PRPP trial needs to be the same size as a conventional randomised controlled trial as the same hypothesis is being tested. In addition the number of patients participating in the preference arm of the trial, i.e. those who choose their treatment, will need to be of a similar size to the randomised element of the trial if similar differences are to be investigated. This means that a PRPP trial needs to be approximately twice the size of a conventional randomised controlled trial. This will increase the cost and/or duration of the study. With increasing competition for funds to undertake comparisons of new or existing interventions the increase in the cost of undertaking these trials has to be justified. The extent to which the extra cost of a PRPP trial can be justified will depend on the extent to which the investigators feel that the results of a conventional randomised controlled trial will be uninterpretable without the addition of the preference component. This will obviously depend very much on the individual circumstances of the patient population, the interventions being tested and the outcomes being measured.

Another potential disadvantage of PRPP trials is the organisation of the study. If the majority of patients offered the choice between being randomised or choosing either of the two options opt for one of the two options¹⁰ this will have major implications for the duration and cost of the trial. The trial will not be complete until the necessary number of participants have been recruited into the randomised and preference elements of the trial. This cannot be predicted before recruitment starts without adequate pilot studies. When planning any randomised controlled trial a certain proportion of eligible patients are expected to decline to participate. If the number of eligible patients who participate is much less than assumed then the trial will be in major difficulties if recruitment is to be achieved within the time proposed. The feasibility of trials, particularly large trials, is a matter of much discussion between investigators and funding bodies. With a PRPP trial this condition is made even more unpredictable. The likelihood that a PRPP trial can be completed within the specified time period and for the specified budget will be even less than for a conventional randomised controlled trial.

Another potential practical concern about PRPP trials is that those patients who do not consent to be randomised, but who choose their treatment, may all choose one treatment group, thereby making a comparison of the two groups in the preference arm impossible or extremely difficult. Once again this cannot necessarily be predicted before the trial starts.

The theoretical advantages of PRPP trials and generalisability have been discussed. Are there any potential disadvantages concerning the generalisability of the results of PRPP trials? The reason for undertaking trials which allocate treatments by randomisation is to reduce selection bias. This is one of the most compelling reasons why clinicians should not allocate treatments in a comparison of effectiveness as they are likely to recruit different types of women to the two different treatments, whether consciously or subconsciously, depending on their belief about the effectiveness of the two interventions. Thus the two groups are not comparable with respect to other factors which may influence their outcome and any differences in their outcomes cannot be assumed to be solely the result of the difference in their interventions^5,11.

If the choice of intervention is made by the patient rather than the clinician will this make a difference? Firstly it is unlikely that patients make decisions solely on their own. Clinicians and the discussions that take place within the consultation are likely to play a large part in the final decision which is reached. The use of the term ‘Patient preference’ is likely to be misleading. As a consequence a comparison of treatment effectiveness between the preference arms of the study has the same problems of interpretation as any observational study. How might this affect generalisability of the findings? If the randomised controlled trial component of such a study demonstrates no effect and the preference arm demonstrates a beneficial effect the situation is impossible to interpret. Either patient preference makes a substantial difference to the relationship between the treatment and the outcome or the relationship is confounded and the treatment is not effective. Either way, little advance will have been made in deciding on subsequent practice.

What is the evidence to support the use of PRPP trials compared with conventional randomised controlled trials?

Until now there has been little evidence to suggest what effect PRPP trials have on recruitment and other organisational difficulties. What evidence there is relates to comparing PRPP trials with the similar interventions tested in conventional randomised controlled trials. This, however, is akin to comparing a case series of one treatment with a case series of another. There is the potential for bias in the interpretation of these results which makes them unusable. The only methodologically rigorous way of comparing a randomised controlled trial with a PRPP trial is to undertake a randomised comparison of the two approaches. In this issue of the Journal the first report of such a comparison is presented and the findings are relatively reassuring. First, offering patients random allocation or either of the two treatments did not affect the recruitment to the randomised element of the comparison. Second, when patients were given the option of choosing between one or other treatment similar numbers chose either option. The total size of the trial and the cost were, as predicted, increased. There was a consistent improvement in symptoms and general health scores with surgical versusmedical management for heavy menstrual bleeding regardless of whether the treatment had been randomly allocated or chosen. These results suggest that the extra cost and complexity were not justified by the comparison of the results of the randomised element and the preference element in this situation. The conclusions therefore are that, in this situation, the choice of a PRPP trial did not affect the overall duration of the study although it did increase the number of women necessary and this extra effort was not rewarded by an increase in the generalisability of the findings.

The future for PRPP trials

To what extent is the conclusion reached in the above report generalisable to other interventions in other population groups? At present this question is difficult to answer. It is possible that with other interventions and other outcomes, particularly those which may be more susceptible to influence by patient preference, the results may have been very different and the inclusion of a preference element to the trial would greatly help the interpretation and the generalisability of the results. This however may not be the case. The only way to resolve this conflict is to continue to perform further randomised comparisons between conventional randomised controlled trials and PRPP trials in as many different clinical settings as possible. Consistency of results would support the viewpoint that PRPP trials are unnecessary. Such information would be important not only for clinicians and policy makers, who could be reassured that the results of randomised controlled trials, particularly pragmatic randomised controlled trials, are generalisable, but would also be extremely useful for trialists and funding bodies.

Are there alternative trial designs which could address the same question?

At the end of their paper, the authors suggest that a conventional randomised controlled trial could address the effect that patient preference has on the outcome by recording this information before allocation. This would allow resources to be concentrated on getting as many patients as possible into the randomised comparison but would allow stratification of the result by initial preference and whether those who had a preference were or were not allocated that preference. The power that any trial would have to explore this possibility is however, difficult to estimate. The assumption that the effect may be different in subgroups of the trial population (i.e. there is an interaction) would require very large numbers of patients and unless the interaction was large it is unlikely that the trial would detect such a difference⁵. Whether a conventional randomised controlled trial with stratification for motivational factors is better than a PRPP trial for assessing this is unknown. What is certain, however, is that the only way to assess it reliably is to undertake a randomised controlled trial of the two approaches.