• Open Access

Random Treatment Assignment Using Mathematical Equipoise for Comparative Effectiveness Trials

Authors

  • Harry P. Selker M.D., M.S.P.H.,

    1. Institute for Clinical Research and Health Policy Studies, Tufts Medical Center and Tufts University School of Medicine, Tufts Clinical and Translational Science Institute, Tufts University, Boston, Massachusetts, USA.
    Search for more papers by this author
  • Robin Ruthazer M.P.H.,

    1. Institute for Clinical Research and Health Policy Studies, Tufts Medical Center and Tufts University School of Medicine, Tufts Clinical and Translational Science Institute, Tufts University, Boston, Massachusetts, USA.
    Search for more papers by this author
  • Norma Terrin Ph.D.,

    1. Institute for Clinical Research and Health Policy Studies, Tufts Medical Center and Tufts University School of Medicine, Tufts Clinical and Translational Science Institute, Tufts University, Boston, Massachusetts, USA.
    Search for more papers by this author
  • John L. Griffith Ph.D.,

    1. Institute for Clinical Research and Health Policy Studies, Tufts Medical Center and Tufts University School of Medicine, Tufts Clinical and Translational Science Institute, Tufts University, Boston, Massachusetts, USA.
    Search for more papers by this author
  • Thomas Concannon M.A., Ph.D.,

    1. Institute for Clinical Research and Health Policy Studies, Tufts Medical Center and Tufts University School of Medicine, Tufts Clinical and Translational Science Institute, Tufts University, Boston, Massachusetts, USA.
    Search for more papers by this author
  • David M. Kent M.D., M.S.

    1. Institute for Clinical Research and Health Policy Studies, Tufts Medical Center and Tufts University School of Medicine, Tufts Clinical and Translational Science Institute, Tufts University, Boston, Massachusetts, USA.
    Search for more papers by this author

HP Selker (hselker@tuftsmedicalcenter.org)

Abstract

In controlled clinical trials, random assignment of treatment is appropriate only when there is equipoise, that is, no clear preference among treatment options. However, even when equipoise appears absent because prior trials show, on average, one treatment yields superior outcomes, random assignment still may be appropriate for some patients and circumstances. In such cases, enrollment into trials may be assisted by real-time patient-specific predictions of treatment outcomes, to determine whether there is equipoise to justify randomization. The percutaneous coronary intervention thrombolytic predictive instrument (PCI–TPI) computes probabilities of 30-day mortality for patients having ST elevation myocardial infarction (STEMI), if treated with thrombolytic therapy (TT), and if treated with PCI. We estimated uncertainty around differences in their respective predicted benefits using the estimated uncertainty of the model coefficients. Using the 2,781-patient PCI–TPI development dataset, we evaluated the distribution of predicted benefits for each patient. For three typical clinical situations, randomization was potentially warranted for 70%, 93%, and 80% of patients. Predictive models may allow real-time patient-specific determination of whether there is equipoise that justifies trial enrollment for a given patient. This approach may have utility for comparative effectiveness trials and for application of trial results to clinical practice. Clin Trans Sci 2011; Volume 4: 10–16

Introduction

Comparative effectiveness research seeks to provide information about which interventions are most effective for which patients under which circumstances. The gold standard research design is the randomized clinical trial. However, the overall (average) result of a randomized trial may not apply to all patients, even those within a trial, since patients differ in characteristics that affect the likelihood of the effects of therapy and clinical outcomes.1–4 Even after a randomized clinical trial establishes overall effectiveness of a treatment, there may remain uncertainty about which patients are most likely to benefit, particularly from therapies with substantial costs or treatment-related harms.3–8 More randomized trial data still may be needed to understand the balance of risks and benefits for individual patients.

The ethical basis for research that randomly assigns different treatments to patients is clinical equipoise, the circumstance of uncertainty among medical experts about the relative effectiveness of alternative treatments.9 When equipoise is dislodged by a pivotal clinical trial, randomization is no longer ethical; all subjects must be offered the most effective therapy. Therefore, once a trial shows overall superior outcomes from a treatment, randomization between it and alternatives becomes problematic, even when there are specific patients and circumstances for which there are reasons that overall average results might not apply. To obtain clinical trial evidence for better individualized decision making, the concept of equipoise may need further specification to take into account that heterogeneity of treatment effects may be obscured by overall results of usual trials.

Derived from patient-specific data from trials and clinical registries, multivariable mathematical models that predict treatment outcomes use variables that capture interactions between individuals’ characteristics and risks for outcomes for a given treatment. Thereby, for a given patient, predictions of likely outcomes of competing treatments can be compared. We hypothesized that these might be used for clinical trial enrollment to determine when treatments’ predicted outcomes are not predicted to be importantly different, suggesting equipoise and permissibility for random treatment assignment. Alternatively, when predicted differences between candidate treatments’ outcomes are outside of the “zone of equipoise,” then the patient should be offered the preferred therapy, as in a cutoff-based trial design.10,11 Accordingly, treatment options would be tested only in patients for whom there still is a lack of evidence for preference. We investigated how mathematical models that predict patient-specific treatment outcomes as real-time clinical decision support12–16 might be used to identify patients for whom, based on the models’ predictions, there appears to be “mathematical equipoise.” This study explored whether patients for whom treatment options’ outcomes are within a zone of equipoise could be included in a trial even when the average superiority or inferiority of a therapy has been established.

To illustrate this approach, we used previously developed multivariable logistic regression models of treatment outcomes for acute ST elevation myocardial infarction (STEMI), for which coronary reperfusion by either thrombolytic therapy (TT) or percutaneous coronary intervention (PCI) is potentially life saving. The thrombolytic predictive instrument (TPI) generates 0–100% probability predictions for 30-day mortality and other outcomes, for if a given patient is treated with TT as coronary reperfusion, and if not so treated, which are automatically printed on the top of the electrocardiogram (ECG) in clinical care.12,13 To the TPI, previously added was a model that predicts outcomes for the alternative treatment, PCI, creating the PCI–TPI.14,15 Interaction variables representing the crucial influence of time delay to treatment and of mortality risk on treatment effect for the outcome mortality are in the predictive models for both reperfusion methods. Thereby the PCI–TPI provides clinicians with mortality predictions comparing, for a given patient with STEMI: (1) no coronary reperfusion treatment; (2) reperfusion by typically rapidly available TT; and (3) reperfusion by PCI across a range of projected time delays. We investigated how these predictions might be used as the basis for determining whether there is equipoise between these two reperfusion methods for a given patient under given circumstances, to allow enrollment into a comparative effectiveness trial.

Methods

The PCI–TPI input variables include patient age, systolic blood pressure, sex, time since symptom onset, the alternative types of treatment (TT or PCI), and the projected time duration from the performance of the ECG until treatment.15 To determine equipoise between TT at time tTT and PCI at time tPCI, uncertainty must be estimated around the difference between their predicted mortality outcomes. To explore the uncertainty in these mortality estimates, we sampled coefficients from the joint normal distribution of the original PCI–TPI model coefficient estimates. We drew 1,000 coefficient vectors, each defining a new PCI–TPI model. From each new PCI–TPI model, 30-day mortality predictions were generated for each treatment, with specific time delays. Defining predicted benefit as the difference between the mortality predictions for TT and PCI, we evaluated the distribution of 1,000 predicted benefits for each patient, and calculated “percentile intervals” to quantify uncertainty in the median predicted benefit. For example, the 95th-percentile interval contains 950 of the 1,000 predicted benefits and is centered at the median predicted benefit. This approach does not account for between-subject variation that is due to unmeasured factors.

This method was applied to generate percentile intervals for each patient in the original PCI–TPI development dataset.14,15 For each patient, we calculated predicted mortality benefits for three circumstances: Circumstance 1 assumed that, following the ECG that allowed diagnosis of STEMI, TT could be given 30 minutes later, and PCI could be performed at 90 minutes, a commonly referenced scenario in which PCI is generally considered favorable. Circumstance 2 assumed TT given at 30 minutes, but PCI could not until 120 minutes, a common delay in getting to the PCI laboratory. Circumstance 3 assumed TT given at 15 minutes and PCI at 180 minutes.

Distributions of predicted benefit for these three circumstances were computed for two types of patients: High risk, reflected by a predicted 30-day mortality of 26% if treated with TT at 30 minutes, and medium risk, reflected by a predicted mortality of 10% with TT. A low-risk patient was not used, as the PCI–TPI truncates mortality comparisons to zero benefit for cases with ≤2% mortality with TT at 30 minutes, due to limited precision of predictions in this low incidence range. For these cases, mortality predictions for PCI at any time were made equivalent to TT at 30 minutes. These predictions were aggregated over groups of patients to explore how random assignment of typical patients in a trial of reperfusion therapy for STEMI would work. We explored two such options for demonstration purposes, outlined below.

As the first option, we looked at the middle 50-percentile interval (between the 25th and 75th percentiles) of the predicted benefit of PCI for each patient (Table 1). The 50-percentile interval was selected as the threshold for demonstration purposes; the method is not dependent on the exact level. If the entire interval was below zero benefit, corresponding to ≥75% chance the patient will have lower mortality with TT, which would be the recommended treatment; equipoise needed for randomization would not be present. If the entire interval was above zero benefit, corresponding to ≥75% chance of better outcome with PCI, the recommended treatment would be PCI; again, equipoise allowing randomization would not be present. However, if zero benefit was within the middle 50-percentile interval, reflecting that neither treatment has ≥75% chance of being more favorable than the other, this was considered to reflect equipoise, and eligibility for randomization.

Table 1.  Predicted benefit for three individual patients.
 Summary of distribution of benefit delta (d)*Circumstance 1: PCI* at 90 minutes vs. TT** at 30 minutesCircumstance 2: PCI* at 120 minutes vs. TT** at 30 minutesCircumstance 3: PCI* at 180 minutes vs. TT** at 15 minutes
  1. *δ, delta; predicted (PCI) survival benefit = p(live) with PCI minus p(live) with TT.

  2. **PCI, percutaneous coronary intervention (angioplasty); TT, thrombolytic therapy.

  3. $This is the relative risk (RR) of death with treatment of thrombolytic therapy versus PCI. This estimate is based on the PCI–TPI model predictions with the published model alone. It does not use results of the 1,000 simulations. A RR > 1.2 or <0.83 indicates predicted mortality at least 20% higher one treatment versus the other.

Patient A: High mortality riskMean δ*9.9%3.0%−20.5%
Median δ (50th percentile)10.0%3.0%−20.5%
95% PI for δ (2.5th to 97.5th percentile)1.5–18.0%−7.8% to 13.7%−40.4% to 0.2%
50% PI for d (Q1 to Q3)7.1–12.9%−0.9% to 6.9%−28.1% to −12.9%
% of δ where PCI better than TT99.1%69.8%2.8%
% of δ in range of (−1%) to (+1%)1.3%11.9%1.4%
Estimated RR from PCI–TPI model≥1.71.20.5
Patient B: Medium mortality riskMean δ*2.8%1.4%−3.4%
Median δ (50th percentile)2.8%1.5%−3.0%
95% PI for δ (2.5th to 97.5th percentile)−0.6% to 5.8%−2.9% to 5.2%−12.4% to 3.5%
50% PI for δ (Q1 to Q3)1.7%–3.9%0.0–2.8%−5.8% to −0.4%
% of δ where PCI better than TT95.5%75.2%21.8%
% of δ in range of (−1%) to (+1%)11.7%27.5%17.6%
Estimated RR from PCI–TPI model$1.41.20.7

The second option was to use relative risks (RR). For this example, we used the threshold of a RR for 30-day mortality of either treatment of ≥1.2 as indicating the other as the preferred treatment. All cases with less than 1.2 RR advantage for one treatment were considered as representing equipoise, and thus eligibility for random assignment between the two treatments. As before, the 1.2 RR threshold is for demonstration only.

Results

Predictions for individual patients

Figure 1 shows the predicted distribution of benefit in each of these three circumstances for the high-risk and medium-risk patients. Distributions of predicted benefit for these patients are in Table 2 and summarized below.

Figure 1.

Predicted (PCI) survival benefit: p(live) with PCI minus p(live) with TT; * percutaneous coronary intervention (angioplasty), ** thrombolytic therapy.

Table 2.  Distribution of randomization assignments among samples of patients with STEMI.
StrategyDecision ruleCircumstance 1: PCI* at 90 minutes vs. TT** at 30 minutesCircumstance 2: PCI* at 120 minutes vs. TT** at 30 minutesCircumstance 3: PCI* at 180 minutes vs. TT** at 15 minutes
  1. *50% prediction interval (PI) for delta (d), where delta = predicted (PCI) survival benefit = p(live) with PCI minus p(live) with TT.

  2. **PCI, percutaneous coronary intervention (angioplasty); TT, thrombolytic therapy.This is the relative risk (RR) of death with treatment of TT versus PCI. This estimate is based on the PCI–TPI model predictions with the published model alone. It does not use results of the 1,000 simulations. A RR > 1.2 (for PCI vs. TT or TT vs. PCI) indicates that the predicted mortality at least 20% higher for one treatment versus the other.

Full sample (n = 2,781)
Use 50% PI for δ for patient*50% PI < 0, treat w/TT**0%0%22%
50% PI contains 0%, randomize60%93%78%
50% PI > 0, treat w/PCI**40%7%0%
 Total 100%Total 100%Total 100%
RR > 1.2 in favor of either treatment$TT predicted to be at least 20% better, treat with TT0%0%25%
Neither has ≥20% predicted benefit, randomize66%100%74%
PCI predicted to be at least 20% better, treat with PCI34%0%<1%
 Total 100%Total 100%Total 100%
Subset of patients from TPI trial (n = 1,037)
Use 50% PI for δ for patient50% PI < 0, treat w/TT0%0%18%
50% PI contains 0%, randomize62%92%82%
50% PI > 0, treat w/PCI38%8%0%
 Total 100%Total 100%Total 100%
RR > 1.2 in favor of either treatmentTT predicted to be at least 20% better, treat with TT0%0%20%
Neither has ≥20% predicted benefit, randomize70%100%79%
PCI predicted to be at least 20% better, treat with PCI30%0%<1%
 Total 100%Total 100%Total 100%

For the high-risk patient, there was substantial variability in estimated benefit in all three circumstances, reflected by wide percentile intervals. For Circumstance 1, PCI yielded a median 30-day mortality benefit of 10.0% (95% PI, 1.5–18.0%), for Circumstance 2, the PCI median benefit was 3.0% (95% PI, −7.8% to +13.7%), whereas for Circumstance 3, PCI was inferior, with a median “benefit” of −20.5% (95% PI, −40.4% to +0.2%). (Narrower 50% percentile intervals are shown in Table 2.) Based on the median predictions, PCI would be superior to TT 99.1% of the time in Circumstance 1, 69.8% in Circumstance 2, and 2.8% in Circumstance 3.

For the medium-risk patient, variability in predicted benefits reflected by the width of the percentile intervals was lower than that for the high-risk patient. The median benefit of PCI in Circumstance 1 was 2.8% (95% PI, −0.6% to +5.8%), in Circumstance 2, 1.5% (95% PI, −2.9% to +5.2%), whereas in Circumstance 3, −3.0% (95% PI, −12.4% to +3.5%). In Circumstance 1, PCI was superior to TT 95.5% of the time, in Circumstance 2, 75.2% of the time, and in Circumstance 3, 21.8% of the time.

Also in Table 2 are predicted RRs, these different risk patients in different treatment circumstances, which follow a similar pattern.

Aggregating predictions from a group of patients

We applied the two options described above to identifying patients for whom equipoise would allow random assignment of treatments, using the middle 50-percentile interval of predicted benefit and the RRs with a threshold of ≥1.2. These two approaches were applied to the 2,781 patients that included the cohort on which the PCI–TPI was developed14,15 and 1,037 patients in the TPI effectiveness trial who had STEMI,15 this latter being representative of all patients with STEMI presenting to emergency departments (EDs). Figure 2 displays the results under all three circumstances using the decision rule based on the RR threshold. Separate plots for the TPI trial patients alone were similar (not shown).

Figure 2.

Distribution of predicted mortality for ST elevation myocardial infarction by percutaneous coronary intervention (PCI) and thrombolytic therapy (TT) mortality for n = 2,781. Subjects in PCI–TPI development sample, together with line of equivalence and zone of equipoise (<20% benefit of either treatment compared to each other for given circumstance).

Both analytic options yielded similar results. In Circumstance 1, with TT at 30 and PCI at 90 minutes, about 30–40% of the patients were identified as having too likely a benefit with PCI to be randomized. In Circumstance 2, with TT at 30 minutes and PCI at 120 minutes, with both approaches, nearly all patients were classified as being eligible for random assignment. In Circumstance 3, with TT at 15 minutes versus PCI delayed to 180 minutes, about 20% of patients were likely to clearly benefit from TT, and the remainder appropriate for random assignment. As shown in Figure 2, in Circumstance 3, it was the higher risk patients that showed expected benefit with TT.

Based on both options examined, for a trial with randomization based on equipoise between predicted outcomes, the high-risk patient would probably not be a candidate for randomization in either Circumstances 1 or 3. The middle-risk patient would be a candidate under almost any scenario. While the choice of decision rules and thresholds for determining benefit deserve additional investigation, these results show how these decisions have an impact on which types of patients would be treated, and under what circumstances.

Discussion

This investigation addressed the need to be able to determine, in clinical practice, whether there is equipoise between treatments that would justify random assignment in a clinical trial, considering an individual patient’s characteristics and circumstances. The approach proposed is based on the use of multivariable predictive models that provide clinicians with 0–100% predictions of the outcomes of alternative treatments and estimates of the uncertainty of these predictions. For patients for whom the competing therapies’ outcomes are predicted to be equivalent, that is, within a zone of “mathematical equipoise” with respect to each other, treatment could be assigned randomly. For patients predicted to have a better outcome from one of the candidate treatments, it would be given. This could facilitate practical and ethical incorporation of comparative effectiveness trials into general clinical care.

We applied this approach to a sample of 2,781 patients treated for STEMI from clinical trials and registries for whom, given overall results of extant trials, inclusion in a randomized trial comparing TT versus PCI would not be feasible because of apparent lack of equipoise. Yet despite that, overall, PCI has been shown more effective in reducing mortality, according to the predictive models, for many individual patients, equipoise remains, at least when one focuses on the outcome of mortality. Consistent with others’ findings that patients at higher risk are likely to benefit more from PCI,17 it was among lower risk patients for whom equipoise was most common (Figure 1).

Another important determinant of whether there was equipoise was the circumstances of care, specifically, how long delays would be for a treatment, illustrated by the three circumstances in Figure 2. When TT was available at a typical 30 minutes after ECG diagnosis of STEMI and PCI was available at 90 minutes, the PCI–TPI models suggested that about 30–40% of patients would be too likely to benefit more from PCI to justify randomization; the remaining 60–70% could be randomized. However, with PCI delayed an additional 30 minutes, nearly all patients would be eligible for random assignment. In the third example, with very prompt TT at 15 minutes, as by emergency medical service (EMS) before hospital arrival, compared to PCI delayed until 180 minutes, as in a rural setting with a PCI center at some distance, about 20% of patients would more likely benefit from TT than PCI such that they should receive thrombolyis and treatment for the remaining 80% could be randomized. Again, in each circumstance, higher risk patients are less likely to be candidates for randomization. This illustrates how this approach could determine for whom and under what circumstance randomization is warranted, and alternatively, when one of the candidate treatments is preferred and should be given.

As an example, for a patient with STEMI seen in the community by EMS, were a cardiac referral center with around-the-clock PCI and a community hospital with only TT both 30–60 minutes away, the PCI–TPI very likely would predict more benefit from PCI, consistent with overall results of randomized clinical trials. For clinical care, it would not seem justified to take the patient to the community hospital for TT, and for a clinical trial, it would not be appropriate to randomly assign treatment. However, had the patient presented much further from the cardiac center than from the community hospital, because the advantage of PCI diminishes with disproportionate delay, at some longer time–distance from the PCI center, the two treatments will have equivalent predicted outcomes. In that case, there will be equipoise, and random treatment assignment would be permissible. Indeed, at yet more disproportionately extended delays to PCI, randomization might not be appropriate because of greater predicted benefit of rapidly available nearby TT. Patients’ respective time–distance thresholds for equipoise will depend on their clinical characteristics. When closer to the community hospital than the PCI center, it may still be inappropriate to randomize a patient with a high-risk large anterior wall STEMI with greater comparative benefit from PCI, whereas it might be appropriate for a patient with a low-risk small inferior wall AMI, for whom predicted mortality from TT and PCI would be very close. Also, although not reflected by the PCI–TPI, the quality of PCI performance, potentially related to annual number done at the cardiac center, might further alter the balance between the options.

To be practical, clinical trial inclusion by mathematical equipoise will require that the outcome predictions be available at the initiation of the clinical encounter. The PCI–TPI can be used in this way in the EMS (and hospital) setting. When the electrocardiograph detects STEMI, the paramedic enters the required PCI–TPI clinical and demographic variables (age, sex, blood pressure, diabetes status), and estimated times to TT and PCI, and then the electrocardiograph automatically acquires the PCI–TPI ECG waveform variables and computes the respective mortality predictions for TT and PCI. This, in consultation with EMS medical control, can allow assessment of whether equipoise exists. Using the middle 50-percentile rule illustrated in this investigation, if no (0%) benefit falls between the 25th and 75th percentiles of likely difference between the treatments, equipoise is suggested and randomization could be considered. Alternatively, if 0% benefit does not lie within this interval, the recommendation would be to go to the hospital that offers the treatment that would most benefit the patient. In this way, patients can be practically and ethically randomized, and yet when randomization is not appropriate, the preferred treatment is facilitated.

More broadly, this approach appears suited for when there is (1) need to compare the effectiveness of treatments for which currently available data leave uncertainty about some patients and circumstances; (2) available data to create a multivariable predictive model that incorporates characteristics of the patient and circumstances that could modify outcome of therapies; and (3) ways outcome predictions and determinations of equipoise could be readily provided to clinicians, such as embedded in electronic health records (EHRs). Given the need for large numbers of patients to be included in clinical trials to inform care over a wide range of patients and practice situations, incorporation into a national EHR system as decision support has several attractive features. First, its incorporation into commonly used EHRs would promote inclusion of “real world” patients and improve generalizability of clinical trials (compared to the inclusion biases intrinsic to typical clinical trials that enroll on the order of 10% of eligible patients). Moreover, this will allow better understanding of the impact of “real world” features (such as availability and distance to PCI) on outcomes. Second, it should greatly speed the conduct of trials, facilitating enrollment of a greater proportion of eligible patients, incorporated into the infrastructure of routine care, and facilitating participation of more clinical sites via ultimately widespread EHRs. The resulting shorter times for trials would be attractive to those who develop and test treatments and would provide earlier public benefit. Third, science would be advanced by adding to understanding of treatments for certain patients and settings, even after overall (average) trial results would otherwise make additional clinical trials hard to justify. Science also would be advanced by the ability to enroll many patients across large systems using EHR decision support, providing statistical power to test treatments for patients and/or outcomes that are infrequent and therefore would be prohibitively expensive to address with usual clinical trial approaches. Fourth, the use and updating of predictive models of treatment outcomes as clinical decision support will facilitate translation of knowledge into practice, including increasing use of treatments already known to be effective but not currently uniformly used.13,18

Limitations and challenges

As this method is appropriate to apply for conditions for which sufficient data (preferably randomized) exist to support multivariate modeling of individual patient benefit, it is likely that the “zones of equipoise” will typically include patient subgroups in whom the true outcome differences between the different treatment strategies is relatively small, and that large sample sizes will be needed to disturb equipoise in these patients. However, acquisition of large numbers of subjects should be facilitated by eligibility decision support in EHRs. Also, as in traditional trials for which direct measurement of important infrequent outcomes, such as mortality, is impractical, a smaller trial using surrogate outcomes (e.g., STEMI size measured by perfusion scan and by left ventricular ejection fraction) or longer term follow-up may yield sufficient information about the superiority of treatments in given circumstances. Additionally, direct, indirect, and induced costs related to different treatments, could supplement choices of treatments.

This approach requires having predictive models for outcomes of candidate treatments that accurately take into account features of the patient and circumstances that determine outcome. However, currently few such models are widely available. Crucial to generating such models is availability of data on which they can be based. For treatments of wide interest, such data often are available from clinical trials and registries. (When treatment is not allocated randomly, as in registries, confounding biases are possible; these challenges are described elsewhere.19) Hopefully growing availability of clinical trial data,20 registry data, and other sources, and increasing expertise in mathematical modeling approaches, will support generation of such models. Moreover, if outcome predictive models are included in clinical trials and analyses to promote optimal use of treatments in practice,21 and to plan follow-on clinical trials, such models will become more common.

Additionally, for mathematical equipoise to be the basis of randomization, rigorous standards for the predictive models that support this decision need to be developed, based on similar efforts informing other evidentiary standards.22 Predictive models should be externally validated, as was the PCI–TPI7,15 and periodically updated to include new relevant data, before being used to determine equipoise. Validation of models should not only include overall predictive performance (such as discrimination and calibration), but stability of the individual coefficients should additionally be tested. Models based on incomplete data will yield imprecise estimates of treatment benefit, which might justify randomization for patients for whom equipoise would be found to be already disturbed, were more data included in the model. Ideally, all relevant clinical trial data would be incorporated into the predictive model so that the model represents an accurate description of the mathematical certainty around individual patient predictions. Seen in this way, the resultant trial would then be used to update the model, narrowing the zone of uncertainty, rather than providing a single answer for all patients who were included in the trail.

Another important question related to standards for predictive models used to support randomization is whether and when treatment-effect interaction terms are included in these models. Since these variables are uniquely influential in determining which patients are eligible for randomization, and since many significant treatment-effect interactions are subsequently found to be spurious,23 we would argue that only the most credible interactions be included in these models—that is, those interactions that have been replicated across multiple datasets or supported by both empirical and strong pathophysiological evidence.24

Finally, especially early in testing a treatment, when outcome data are sparse, the resulting statistical models may generate wider prediction intervals and zones of equipoise than would expert clinical opinion. This will be especially so for infrequent outcomes, such as mortality among low-risk STEMIs, for which the PCI–TPI, and probably most such models, cannot differentiate between treatments’ outcomes with precision. As this approach is put into practice, example cases such as explored in this investigation should be used to compare alternative ways of determining equipoise. When the breadth of the mathematically derived zone of equipoise would seem to allow randomization with differences between predicted outcomes that would seem important clinically, at this stage it would be prudent to ensure that randomization not extend beyond what seems consistent with good clinical care. Nonetheless, even when the equipoise interval is determined by a different approach, such as by expert opinion or by a specified difference between predicted outcomes (e.g., a 1% absolute mortality difference), outcome predictions to support on-site determination of eligibility for a randomized trial still should provide advantages. Enrollment would be assisted by use of multivariable expression of the best available data, a reasonable starting place when precise models are not yet available.

Conclusions

In sum, we have demonstrated how multivariable models that provide patient-specific outcome predictions might be used to determine whether, for a given patient and circumstance, there is equipoise between treatments. This approach could allow ethical and practical inclusion in a comparative clinical effectiveness trial. It could advance scientific understanding by allowing selective testing of treatments for subsets of patients for whom equipoise still applies but otherwise would not benefit from another trial, while not including patients for whom further testing is not warranted. Moreover, when computerized outcome predictions can be provided, such as by the PCI–TPI, this approach may facilitate broader study inclusion, enhancing trials’ efficiency and generalizability. Refinement of the concept of mathematical equipoise and its application is needed, and to better understand its practicality and advantages compared to conventional and other approaches appear warranted.

Conflict of Interest

Funders had no input into the development of the method or the writing of this manuscript.

Acknowledgments

The authors thank Joni Beshansky, RN, MPH, for comments on the manuscript and Muriel Powers for manuscript editing and preparation.

This work was supported by grants R01 HS 10280 from Agency for Healthcare Research and Quality (AHRQ); UL1 RR025752/KL2 RR025751-01A1 from National Center for Research Resources (NCRR), National Institutes of Health (NIH); U01 HL077821 from National Health Lung and Blood Institute (NHLBI) of the NIH; and PG280737 from Pfizer Pharmaceuticals.

Ancillary