Are component endpoints equal? A preference study into the practice of composite endpoints in clinical trials

Abstract Objectives To examine patients’ perspectives regarding composite endpoints and the utility patients put on possible adverse outcomes of revascularization procedures. Design In the PRECORE study, a stated preference elicitation method Best‐Worst Scaling (BWS) was used to determine patient preference for 8 component endpoints (CEs): need for redo percutaneous coronary intervention (PCI) within 1 year, minor stroke with symptoms <24 hours, minor myocardial infarction (MI) with symptoms <3 months, recurrent angina pectoris, need for redo coronary artery bypass grafting (CABG) within 1 year, major MI causing permanent disability, major stroke causing permanent disability and death within 24 hours. Setting A tertiary PCI/CABG centre. Participants One hundred and sixty patients with coronary artery disease who underwent PCI or CABG. Main outcome measures Importance weights (IWs). Results Patients considered need for redo PCI within 1 year (IW: 0.008), minor stroke with symptoms <24 hours (IW: 0.017), minor MI with symptoms <3 months (IW: 0.027), need for redo CABG within 1 year (IW: 0.119), recurrent angina pectoris (IW: 0.300) and major MI causing permanent disability (IW: 0.726) less severe than death within 24 hours (IW: 1.000). Major stroke causing permanent disability was considered worse than death within 24 hours (IW: 1.209). Ranking of CEs and the relative values attributed to the CEs differed among subgroups based on gender, age and educational level. Conclusion Patients attribute different weight to individual CEs. This has significant implications for the interpretation of clinical trial data.


| INTRODUC TI ON
Over the past 40 years, many randomized clinical trials (RCTs) have used composite endpoints when comparing medical interventions. [1][2][3][4] These composite endpoints combine 2 or more clinically relevant endpoints, also known as the component endpoints (CEs), within a single outcome variable to measure clinical benefit of a treatment.
The conclusions of RCTs rely on their primary endpoints, and thus, it is important to choose the most appropriate endpoints when designing clinical research. 5 In recent years, medical care has significantly progressed for patients experiencing cardiovascular events, resulting in low mortality rates. Although death is still considered the primary outcome, it is often difficult for clinical researchers to identify differences in survival rates between the different treatment options. 6,7 To investigate the occurrence of an infrequent event, large sample sizes, as well as prolonged follow-up, are needed, and costs go up. 8 When several adverse events are combined in a composite endpoint, the occurrence of events will increase, thereby expanding the overall treatment effect, and reducing the required sample size and overall costs of cardiovascular trials. 9,10 Analytic approaches to composite endpoints generally assume that all underlying adverse events are of equal value. In practice, this assumption is rarely met, for instance: in some situations, the overall positive treatment effect may be related to "soft events" such as recurrent angina or redo revascularization as opposed to the clinically more relevant "hard" events such as major stroke or death. 11,12 This heterogeneity of effect among CE can result in too optimistic conclusions about the treatment effect and serious misinterpretations. [13][14][15] One can account for these different effects by adjusting trial outcomes using "importance weights (values assigned to CE that reflect the relative importance of these CEs to patients)".
These "importance weights" are almost always derived through evaluations by an expert panel 11,20 ; however, previous research has shown that patient and expert preferences towards CE are different 21 and thus cannot be considered equivalent. Therefore, the aim of this study was to examine patients' perspectives regarding the use of composite endpoints in clinical trials and the importance they attach to possible unfavourable outcomes of treatment. In addition, we examined whether the obtained "importance weights" differed between subgroups based on clinical and demographic characteristics of our study population. Patients who had a CABG procedure completed the survey on day 3 to 4 post-intervention. Patients who underwent CABG plus a surgical intervention to correct cardiac valve disease were not included in this study. In addition, patients who were unable to perform the study task correctly due to the cognitive burden the study posed or due to a language barrier were excluded. The study protocol was submitted to the regional medical-ethics committee (METC Twente, no. K16-45), but was deemed exempt from formal medical ethical evaluation, because the study does not fall within the remit of the Medical Research Involving Human Subjects Act (WMO). 16 All patients provided written informed consent, and all data were anonymized before analysis.

| Patient population
The study complied with the Declaration of Helsinki. The literature provides no guidance to determine minimal required sample sizes for Best-Worst Scaling (BWS) experiments. To determine the minimum sample size needed, we used a rule of thumb for conjoint analysis which states that estimate precision increases rapidly at sample sizes over 150 and flattens out at around 300 observations. 17,18 Taking into account the average number of patients undergoing a revascularization procedure throughout the 2-month study period, we aimed at including at least 150 patients.

| Patient preference survey
The original survey consisted of 4 different parts (Appendix S1). The PRECORE study started by asking patients to read the descriptions of the 8 CE examined in this study (Table 1). After patients read the descriptions, they were asked to answer 4 statements about whether they thought it was equally important to prevent 2 complications (death vs disabling stroke; death vs disabling myocardial infarct (MI); death vs redo CABG; and disabling stroke vs disabling MI). The statements examined whether or not patients weigh the CE equally.
If patients answered at least one of these 4 statements with "yes, the avoidance of one of these 2 complications is more important to me, or they answered at least one of these 4 statements with "do not know," the relative importance of each complication (CE) was examined by means of 6 Best-Worst Scaling questions (BWS); the paragraph below explains this methodology. In addition to the preference elicitation questions, patients were asked for socio-demographics and clinical profile and one final question was asked to directly examine their view on the use of composite endpoints (Appendix S3).
The Web-based survey was programmed using LimeSurvey 19 and was intended for self-completion on a tablet. However, if patients indicated that they needed more explanation or assistance in completing the survey, assistance was given. On average, it took patients 30 minutes to answer the complete survey.

| Best-worst scaling: A method for determining the relative importance of CE to patients
The attributes included in this study were determined in a stepwise manner, which subsequently included a literature review, expert review and individual interviews with patients. First, a list of attributes that describe possible unfavourable outcomes of revascularization was compiled based on previously published literature. [21][22][23][24][25][26] Second, this list of unfavourable outcomes was discussed within the steering committee of the research team (including 2 cardiologists with expertise in these interventions, and 2 senior health scientists for specialist methodological input). This expert review was conducted in order to (i) shorten the list of potential attributes and (ii) to ensure that the attributes were expected to be relevant for all patients who underwent a revascularization procedure. As a third step, 6 individual interviews were conducted with patients who underwent revascularization in order to ensure that (i) the most important attributes to patients were included and (ii) attribute descriptions were clear to patients. This process led to the inclusion of 8 attributes (Table 1).
The BWS method was used to determine the relative importance patients with CAD assign to the CE associated with coronary revascularization procedures. 27,28 BWS is based on the random utility theory, which assumes that a patient's relative preference for characteristic A over characteristic B is a function of the relative frequency with which A is chosen as better than, or preferred to, B. 29,30 This methodology was used, because it avoids and overcomes some of the limitations of rating-and ranking-based measurement methods. 27,28 In BWS (case 1), respondents are asked to choose the best (eg least unfavourable) and worst items (eg most unfavourable) from a set of objects (ie adverse outcomes) 29 (Appendix S2). By presenting several of these set of objects to multiple patients, and studying the probability of patients choosing one objects over the other, the relative desirability of treatment outcomes from the patients' point of view (as a group) can be determined. The number of scenarios per patient was determined using the experimental design software Sawtooth 6.4.6. (Sequim, WA, USA). The most optimal design was a partial-profile BWS case 1 design with 4 versions, 6 scenario-questions per version, and 4 attributes per scenario. 17

| Statistical analysis
By use of IBM SPSS Statistics 23 (SPSS Inc., Chicago, IL, USA) and Stata version 14 (StataCorp, College Station, TX, USA), descriptive statistics were applied to get insights into the patient demographics and their perspectives regarding the use of CE in clinical trials (statement data). All analyses were 2-tailed and applied on the aggregated sample level, as we were interested in overall group preferences.
Best-Minus-Worst counts were calculated to study the distribution of scores. Best and worst counts represent the number of times an attribute level was chosen as best or as worst across all choice-sets and respondents. 19 By subtracting the total number of times it was chosen as worst from the total number of times an outcome was chosen as best, an initial ranking of all 8 attributes from best (ie least unfavourable) to worst (ie most unfavourable) can be determined. To account for the number of times the attribute was available in the BWS design, normalized scores were calculated; that is, the Best-Minus-Worst counts (B-W counts) were divided by the sample size and the frequency that each attribute appeared in the design of the choice set.
As it was chosen to use data on the aggregated sample level, no statistical analysis can be performed to analyse the potential significance of these B-W counts. To explore potential heterogeneity in preferences between certain patient subgroups (gender, age, educational level, current revascularization procedure, previous revascularization experience and previous MI), several count analyses were performed.
P-values <.05 were considered statistically significant.

Treatment outcomes (Attributes) Description to patients
Minor MI You will experience a mild myocardial infarction of which the symptoms disappear within 3 mo after the initial myocardial infarction Major MI You will experience a large myocardial infarction causing permanent disability (ie tire more quickly, less physical capacity) Minor stroke/TIA You will experience a mild stroke of which the symptoms disappear within 24 h Major stroke You will experience a large stroke causing permanent disability (ie loss of function of an arm and/or leg) Angina Pectoris You will experience recurrent angina (ie sensation of chest pain, pressure or squeezing) Redo CABG You need to undergo a bypass surgery within 1 yr following your initial revascularization because of restenosis Redo PCI You need to undergo a PCI within 1 yr following your initial revascularization because of restenosis Death You will die within 24 h post-intervention TA B L E 1 Attributes for the Best-Worst Scaling case 1 choice-questions

| Patient inclusion
Of 176 patients contacted, 9 (5%) were excluded as they did not meet the inclusion criteria, 2 (1%) refused participation, and 5 (3%) were discharged too early to participate. A total of 160 patients met the eligibility criteria, agreed to be surveyed and were included in the PRECORE study. Some patients received hands-on assistance by filling in the survey (n = 31, 19%) as they experienced physical constraint while filling in the survey. Another 6 (4%) patients received additional oral information and instructions after indicating a need for further assistance, and 9 (6%) patients received both hands-on assistance and additional oral information and instructions. The surveys of 13 (8.1%) patients were returned with incomplete BWSdata and could be not used for the analysis of the BWS-data.

| Patient characteristics
The patients' sociodemographic and treatment-related characteristics are presented in Table 2. Of the 160 patients included in this study, a total of 97 (61%) underwent PCI and 63 (39%) were treated by CABG.

| Statement data: Patients' perspective regarding CE differ
A vast majority of patients (n = 129, 80.6%) stated that the common practice of weighing all CE equally is invalid, and more than half of patients (n = 94, 58.8%) indicated that it is more important to prevent a major stroke causing permanent disability than death within 24 hours postintervention (Table 3). Moreover, two-thirds of the patients (n = 126, 78.8%) reported that it is more important to prevent death within 24 hours post-intention than redo CABG. When patient's preferences of the CE were further analysed according to the patient's age, gender, previous MI or current type of revascularization procedure-no statistically significant differences were found (data not shown).

| BWS-data: Patients did not consider all CE equal
Systematic assessment by use of BWS showed that patients did not assign equal weights to all CE (Table 4). Figure 1 shows that patients considered the need to undergo a redo PCI within one year post-intervention the least unfavourable (importance weight: 0.008   20 In that methodology, the authors allocated weights that reflect the relative severity of individual CEs to patients; and the weights were determined through a clinician-investigator Delphi panel. 20 However, instead of experts assigning weights to CE, we suggest incorporating patient preferences in the evaluation of CEs. In addition, it is important to reach agreement on which method is most appropriate to measure patient preferences for adverse outcomes of treatment, such that normalized "importance weights" can be determined, and applied to raw trial data. Meanwhile, existing clinical trial data should be carefully interpreted, as these "non-weighted" data could be misleading.

| CON CLUS IONS
The majority of patients in the PRECORE study indicated that they do not agree with the common practice of weighing clinical endpoints equally. Patients considered "hard" cardiovascular events significantly more unfavourable than "soft" events. One of 2 patients stated to be more worried about permanent stroke causing disability than death, suggesting that many patients fear a loss of mobility patients' preference and encourage a shift in thinking that may lead to importance weight-adjusted composite endpoints for clinical trials.

CO N FLI C T O F I NTE R E S T
All authors have completed the ICMJE uniform disclosure form at www.
icmje.org/coi_disclosure.pdf and declare: CvB reported institutional research grants provided by Biotronik, Boston Scientific and Medtronic.
All other authors declared that they have no conflict of interest.

AUTH O R CO NTR I B UTI O N S
CvB, MK, JvT and MW conceptualized the original study proposal.
All authors contributed substantially to the design of the work; the acquisition, analysis and interpretation of data for the work; and the drafting the work or revising the manuscript critically for important intellectual content. MV searched the literature, collected the data, analysed the data and wrote the first draft of the manuscript. MW provided statistical expertise. As principal investigator, MV had full access to all of the data in the study, and takes responsibility for the integrity of the data and the accuracy of the data analysis. She is the guarantor.

E TH I C A L A PPROVA L
The PRECORE study received approval from a medical ethical committee, and participants gave written informed consent.

DATA S H A R I N G S TAT E M E N T
No additional data available.

TR A N S PA R EN C Y S TATEM ENT
The lead author affirms that the manuscript is an honest, accurate and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any dis- F I G U R E 1 Standardized "best-worst" scores for the 8 potential outcomes of revascularization. A total of 147 patients participated, each of whom chose best and worst attributes from 6 sets of 4 attributes each (4248 total choices). Standardized scores range from −1.0 to 1.0, where higher (positive) values indicate that a given attribute was chosen more often as best than worst, and were more likely to be preferred relative to the other attributes. A score of "0" means that an attribute was selected as best or worst an equal number of times. 15 MI, myocardial infarction; re-CABG, redo coronary artery bypass grafting within a year post-intervention; Re-PCI, redo percutaneous coronary intervention within a year post-intervention

APPENDIX 1
Presents the experimental set up of this study.

APPENDIX 2
Presents an example of a BWS case 1 choice-question used in this study.