Clinical outcome assessment trends in clinical trials—Contrasting oncology and non‐oncology trials

Abstract Background Clinical outcome assessments (COAs) are key to patient‐centered evaluation of novel interventions and supportive care. COAs are particularly informative in oncology where a focus on how patients feel and function is paramount, but their incorporation in trial outcomes have lagged that of traditional survival and tumor responses. To understand the trends of COA use in oncology and the impact of landmark efforts to promote COA use, we computationally surveyed oncology clinical trials in ClinicalTrials.gov comparing them to the rest of the clinical research landscape. Methods Oncology trials were identified using medical subject heading neoplasm terms. Trials were searched for COA instrument names obtained from PROQOLID. Regression analyses assessed chronological and design‐related trends. Results Eighteen percent of oncology interventional trials initiated 1985–2020 (N = 35,415) reported using one or more of 655 COA instruments. Eighty‐four percent of the COA‐using trials utilized patient‐reported outcomes, with other COA categories used in 4–27% of these trials. Likelihood of COA use increased with progressing trial phase (OR = 1.30, p < 0.001), randomization (OR = 2.32, p < 0.001), use of data monitoring committees (OR = 1.26, p < 0.001), study of non‐FDA‐regulated interventions (OR = 1.23, p = 0.001), and in supportive care versus treatment‐focused trials (OR = 2.94, p < 0.001). Twenty‐six percent of non‐oncology trials initiated 1985–2020 (N = 244,440) reported COA use; they had similar COA‐use predictive factors as oncology trials. COA use increased linearly over time (R = 0.98, p < 0.001), with significant increases following several individual regulatory events. Conclusion While COA use across clinical research has increased over time, there remains a need to further promote COA use particularly in early phase and treatment‐focused oncology trials.


| INTRODUCTION
Clinical outcome assessments (COAs) are measures of how patients feel, function, and survive. 1 COAs play a crucial role in holistic evaluation of patient care and can be particularly informative in oncology where the multitude of symptoms, treatment-related toxicities, and often poor prognosis of patients necessitate a focus on symptom management and quality of life.Landmark clinical trials have demonstrated that incorporation of COAs in trial outcomes can inform whether the intervention benefit is meaningful, 2 that COAs can capture within-patient effects, 3 and that patient reporting of symptoms can even lead to prolonged survival. 4Therefore, designing oncology trials to incorporate COAs in conjunction with survival, tumor response, and safety outcomes is key to patientcentered assessment of net clinical benefit of novel interventions and supportive care.
COAs include four categories: patient-reported outcomes (PRO), observer-reported outcomes (ObsRO), clinician-reported outcomes (ClinRO), and performance outcomes (PerfO) measures, which serve complementary roles in clinical trials. 1 PROs and PerfOs enable multifaceted assessments of treatment toxicity and efficacy whereas ClinROs serve in determining trial eligibility and treatment course, and as correlates to objective outcomes and disease progression. 5COAs presently encompass over 2500 individual instruments and 300 composite measures which range in scope and applicability, with indications covering from specific conditions and neoplasms to all areas of clinical care. 6This diversity has invariably presented its own challenges, including the difficulty in selecting valid, reliable, and clinically useful instruments, since quality of life and other constructs of interest are often multidimensional, dynamic, and unique to an individual or situation. 5][11][12] PRO use has been evaluated in general oncology trials and focused cancer trial subsets [13][14][15][16][17][18] but the comprehensive use of COAs in oncology trials across time and how it compares to the use of COAs in non-oncology clinical trials has not been evaluated.Moreover, trends in relation to landmark regulatory and advocacy events have not been assessed, and with few exceptions 19,20 previous evaluations relied on manual curation of the literature which is not feasible for periodic assessments.In this study, we extensively evaluated COA use across oncology trials and how it compares to the COA use in the rest of clinical trials in the Clini calTr ials.govregistry.We leveraged the registry data through programmatic semi-automated means and assessed the impact of key regulatory and advocacy events.

| Selecting and obtaining study data
Clini calTr ials.govregistry was accessed programmatically on August 1, 2021 using R (RPostgreSQL library) through the Aggregate Analysis of Clini calTr ials.govdatabase. 21,22Interventional trials with start date 1985-2020 were retrieved from the registry.Oncology trials were determined by searching for Medical Subject Headings 23 terms under "Neoplasms by Site" (Table S1) in study title, description, or design.The European Union (EU) or The United States (US)-based trials were determined by site country location (Table S2).Trial data obtained included initiation year, phase, primary purpose, intervention model, randomization, blinding, data monitoring committee (DMC) use, regulatory status of drug or device, and primary sponsors.

| Obtaining clinical outcome assessment information
COA instrument details including name, acronyms, COA type (PRO, ClinRO, PerfO, ObsRO, or composite), and therapeutic indications, were extracted from PROQOLID 6 using Python's BeautifulSoup library on October 26, 2020.Several COAs with ambiguous acronyms (such as QLQ or QOL) or ambiguous meanings in trials (e.g., PFS referring either to the Parkinson Fatigue Scale or Progression Free Survival) were excluded from search terms to improve specificity.COA use in trials was identified by searching for COA instrument names and acronyms among trial entries relevant to outcome measures.COA use was further categorized into primary, secondary, or other outcome by matching COA instrument names with trial outcome design descriptions.Similarly, trials assessing overall survival, progression free survival, and tumor response endpoints were identified by searching for relevant terms among trial entries relevant to outcome measures.

| Trends of COA use over time
Trends in COA use over time were assessed using Pearson's correlation and linear regression analysis between trial initiation year and the proportion of trials using COAs.When stratified by trial primary purpose or COA category, the proportion was calculated with respect to the baseline count of all (oncology or nononcology) trials and all (oncology or non-oncology) trials using COAs, respectively.Years with statistically outlying (beyond three times the interquartile range) proportion of trials using COAs were excluded from regression.We removed outlier years with regression standardized residuals larger than three and representing years with fewer than 1% of trials to avoid skew in COA use rates from years with low trial counts and confirmed that regression residuals were normally distributed (Shapiro-Wilk deviation from normality: W = 0.96, p = 0.204) and not dependent on year.

| COA use and landmark events
Chi-square tests with continuity correction were used to compare COA use in trials initiated before versus after five landmark events driven by regulatory bodies and initiatives including the European Medicines Agency (EMA), Food and Drug Administration (FDA), Consolidated Standards of Reporting Trials (CONSORT), International Society for Quality of Life Research (ISOQOL), and Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT).Logistic regression was used to assess linear trends between COA use with binary indicator variables for events (whether the trial took place before or after the event) and trial initiation year as a continuous variable and an interaction term between the two covariates.False discovery rate multiple hypothesis correction was performed across models and covariates.

COA use
Logistic regression was performed to investigate associations between COA use and trial characteristics: phase (as ordinal variable), intervention FDA regulatory status, lead sponsor and collaborator class (National Institutes of Health (NIH) or other US federal agencies, industry, or other, which includes trial networks, academic institutions, and individual investigators), primary purpose (treatment, supportive care, diagnostic, or other, which includes basic science, health services research, prevention, screening, device feasibility, educational/counseling/training), blinding, number of trial arms (single-or multi-arm), data monitoring committee (DMC) use, and randomization.
Analyses were conducted using R version 3.6.0. 24

| Trial characteristics and associations with COA use
A total of 279,855 interventional trials with start dates between 1985 and 2020 (a date range chosen to encompass 95% of all interventional trials in the registry) were retrieved from the registry: 35,415 were determined to be oncology trials studying one or more neoplasms, and the remaining 244,440 were considered non-oncology trials.The design features and the number of trials reporting them are summarized in Figure 1A and Table S3.Briefly, the oncology trials reporting relevant information most commonly had a primary purpose of treatment (77%), studied a non-FDA-regulated intervention (59%), were completed (44%), phase 2 (48%), non-blinded (84%), multi-arm (54%), and non-randomized (57%).Similarly, non-oncology trials predominantly had a primary purpose of treatment (64%) and studied a non-FDA-regulated intervention (72%), but they had more balanced phase distribution (with phase 1, phase 2, and phase 3 at 22%, 25%, and 21%, respectively), and a higher degree of blinding (52% non-blinded), multi-arm design (77%), and randomization (70%) (Figure S1; Table S3).
Eighteen percent of oncology trials reported COA use as determined by the search for individual instruments among trial design entries related to outcomes.Early phase trials had relatively lower rates of COA use (less than 15% each) compared to phase 2/phase 3 (25%), phase 3 (30%), and phase 4 (19%) trials.Nineteen percent of trials studying an FDA-regulated intervention used a COA, versus 27% among trials studying non-FDA-regulated interventions.Eighteen percent of industry-sponsored trials used COAs, versus 13% of NIH and US federal government-sponsored, and 18% of all other sponsors.Trials focused on supportive care had the greatest rate of COA use (49%), followed by trials focused on treatment (17%), diagnostic (6%), with cumulative other trial categories having 15% COA use.Trials with blinding had greater rates of COA use (29%) than open-label trials (16%).Multi-arm trials had greater rates of COA use (25%) compared to single-arm trials (12%).Twenty-seven percent of randomized trials used COAs compared to 12% in non-randomized trials (Table 1).
We next evaluated the interplay of the trial design elements in the implementation of COAs using multi-variate logistic regression across trials reporting sufficient information (N = 7339; Table 1).The likelihood of COA use increased significantly with progressing clinical trial phases (OR = 1.30, p < 0.001).Trials studying FDA-regulated interventions were significantly less likely to use COAs than those studying non-FDA-regulated interventions (OR = 0.81, p = 0.001) whereas trials using DMCs were more    Non-oncology trials in the registry had higher rates of COA use overall (26%) and similar trends in COA use by phase, DMC use, and randomization, whereas unlike in oncology trials, treatment-focused trials had similar rates of COA use compared to supportive care studies (OR = 1.16, p = 0.100).Additionally, compared to industrysponsored trials, trials with other non-classified sponsors were less likely to use COAs than industry-sponsored trials (OR = 0.89, p < 0.001), while trials with sponsorship from NIH or US federal government (versus industry; OR = 1.39, p < 0.001), blinding schemes (OR = 1.82, p < 0.001), and single-arm designs (OR = 1.26, p < 0.001) were significantly more likely to use COAs.The FDA regulatory status of the trial intervention was not significantly associated with COA use among non-oncology trials (Table 1).

| Characteristics of COA instruments used
We identified 655 unique instruments used across the oncology trials, representing a diverse set of COAs and comprising of 24% of all instruments obtained from PROQOLID (N = 2695).The top instruments in order of frequency were the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire-Core Questionnaire (EORTC QLQ-C30) used by 29%, Eastern Cooperative Oncology Group (ECOG) Performance Status used by 14%, Euroqol EQ-5D used by 10%, Hospital Anxiety and Depression Scale (HADS) used by 5%, and Brief Pain Inventory (BPI) used by 4% of trials using COAs (Table S4).Fifty-six percent of trials using COAs reported using one instrument (range: 1-13 instruments per trial).Of trials reporting details of trial outcomes (N = 5795), COAs were used predominantly as a secondary trial outcome (84%), followed by primary trial outcome (23%), and other outcome (7%).Nononcology trials reported a broader range of instruments encompassing 1709 unique COAs; most frequently used instruments were the SF-36 Health Survey (8%), EQ-5D (6%), Patient Health Questionnaire (4%), and HADS (3%) (Table S5).Similarly to oncology, 50% of non-oncology trials using COAs reported using one instrument (range: 1-19).Compared to oncology, a greater proportion of non-oncology trials reported COAs as a primary outcome (49%), with 75% and 7% of trials reporting COAs as secondary and other outcome.
Stratifying oncology trials by primary purpose, treatment-focused trials (N = 4648 reporting COAs) most commonly used oncology-specific quality of life measures (EORTC QLQ-C30 in 33%), measures of overall health status (ECOG Performance Status in 18%), and other quality of life measures (EQ-5D in 11%); all other instruments were reported in fewer than 5% of trials (Table S4).Among trials with a primary purpose of supportive care (N = 946 reporting COAs), the EORTC QLQ-C30 (20%) and EQ-5D (6%) were also among the most commonly used; additionally, a broader range of COAs were more prevalent, including HADS (16%), Functional Assessment of Chronic Illness Therapy (8%), Functional Assessment of Cancer Therapy-General (8%), Short Form Health Survey (8%), BPI (7%), Functional Assessment of Cancer Therapy-Breast Cancer (6%), Center for Epidemiologic Studies Depression Scale (5%), and Pittsburgh Sleep Quality Index (5%) (see Tables S4 and S5 for top COAs used for trials of other purpose and non-oncology trials).
The proportion of non-oncology trials using COAs also increased significantly over time (R = 0.98, p < 0.001), at a rate of 0.94% of trials per year (95% CI: 0.87%-1.02%/year).Stratification by trial purpose and COA type revealed similar significant increases in COA use compared to baseline across all levels (Figures S3A and S4B; Table S6).

| COA use across landmark events
We focused on five regulatory guidelines and interest group recommendations relevant to the use of COAs in clinical trials (Figure 3A) and assessed changes in COA use before versus after each event for the subset

Instrument
COA Category Indication of applicable trials.We noted a significant increase in COA use around every event considered (Table 3).To decouple the contribution of the individual events from the linear increase in COA use over the years, we considered regression models that included separate covariates for a given event, the initiation year, and the interaction between the two (Table 3; Figure S5A).In all these models the association between year and increase in likelihood of COA use across the relevant oncology study subset remained significant.Moreover, there were additional significant increases in the likelihood of PRO use among trials with a site in the EU before versus after 2005 (OR > 10.00, p = 0.007) corresponding to the EMA guidance for HRQoL.A similar change  was noted for industry-sponsored trials with a site in the US before versus after 2009 (OR > 10.00, p = 0.003) corresponding to the FDA Guidance for Industry PRO Use, and all interventional oncology trials before versus after 2013 (OR > 10.00, p < 0.001) corresponding to the ISOQOL PRO Recommendations, when accounting for the underlying increase in COA use over time and its interaction with the respective event years.As in oncology trials, the increase in COA use rates among nononcology trials across each event considered, as well as the association between year and COA use among every trial subset considered were significant.Compared to trends among oncology trials, PRO use rates among non-oncology trials in the EU did not increase after 2005 (OR > 10.00, p = 0.187).However, PRO use as a primary or secondary outcome increased significantly after 2018 (OR > 10.00, p < 0.001) corresponding to SPIRIT-PRO extension (Table 3; Figure S5B).

| DISCUSSION
We assessed the landscape of clinical trials with a focus on understanding differences in COA incorporation distinct from traditional survival and response endpoints in oncology versus non-oncology trials and the impact of regulatory and advocacy efforts to promote COA use in clinical research.COAs were used by 18% of oncology trials, lagging non-oncology trials where use was reported in 26% of studies, albeit the latter rate represents use across diverse disease areas with previously reported heterogeneous landscape of COA use. 25 Use of COAs by sponsor class was found to be similar for oncology trials, whereas for non-oncology trials sponsorship of trials by NIH or other US federal agencies was associated with higher rates of COA use.Similar trends were found in associations between progressing phases, DMC use, randomization, and higher rates of COA use in both categories of trials.However, in contrast to non-oncology trials, oncology trials focused on treatment had significantly lower rates of COA use compared to those focused on supportive care.The overall use of COAs increased over time across both oncology and non-oncology trials, likely reflecting the cumulative impact of regulatory and advocacy efforts.Our assessment also found significant increases in COA use coinciding with focused guidelines, including the EMA guidance for HRQoL, the FDA Guidance for Industry PRO Use, and the ISOQOL PRO Recommendations but the magnitude of the impact could not be determined reliably, pointing to the need to assess additional factors underlying COA use trends.Lower COA use rates in early phase clinical trials reiterate previously reported concerns about phase 1 cancer trials, 15,26 and likely reflect challenges of incorporating COA instruments into small, non-randomized trials 11 often assessing heterogeneous patient populations or multiple tumor types. 27,28The importance of COAs in patient-focused toxicity assessment 29 and the drive to include medical benefit outcomes in early phase cancer trials 30 call for incorporation of meaningful COA outcomes T A B L E 2 Correlation analysis and linear regression between number and proportion of trials using clinical outcomes assessments (COAs) each year, stratified by COA type and primary purpose of trials.to supplement or strengthen early phase studies as advocated by groups including the NIH Office of Patient-Centered Outcomes Research. 31,32Guidelines targeting early-phase COA use or adaptation of existing strategies for randomized trials may be beneficial. 9,11,12he growing role of COAs to establish net clinical benefit of novel therapies 33 and impact regulatory decision making 25,34,35 underscores the need to address the low COA use rates in treatment-focused oncology trials.However, an important caveat when interpreting these rates is that they exclude the use of traditional survival and tumor response endpoints in oncology trials, therefore underestimating the use of ClinROs as they pertain to those endpoints.In fact, we estimate that 54% of oncology trials use these traditional endpoints (66% of treatmentfocused trials and 9% of supportive care trials), versus only 8% of non-oncology trials (Table S7).The need to augment these traditional assessments for treatment-focused trials is driven by the challenges of objectively assessing tumor response, resulting, for example, from inter-tumor heterogeniety, 36 clinical consequences of immune and targeted therapies that are distinct from traditional chemotherapy on which traditional radiographic criteria were developed, 37,38 and the need to establish that the assessed response translates into future benefit. 39Even among supportive care oncology trials, 51% did not use COAs despite emphasis on using COAs to justify the clinical benefit of non-curative therapies. 33Moreover, as a recent assessment of the use of COAs in regulatory decision-making discovered, patient experience as captured by COAs played a central role in FDA decisions when COAs were used as primary endpoints with uses in other endpoints primarily providing supporting information to contextualize rare or poorly-characterized conditions. 35Our assessment found that among trials that reported COA use, only 23% of oncology trials versus 49% of non-oncology trials used COAs as primary or co-primary endpoints.Our results and prior reports 16,40 highlight the unmet recommendations for COA use in trials, which may contribute to the underrealized impact of COAs in drug development.

N trials over
PROs were found to be the most frequently used category of COAs in both categories of trials (84% for oncology and 78% non-oncology trials reporting COA use) tracking Eastern Research Group's recent assessment that 84% of FDA reviews for approved applications (submitted between 2017 and 2020) that mention patient experience data, mention PROs. 35The most frequently reported COA instruments in oncology trials reflected recommended instruments including the EORTC QLQ-C30, reported by nearly one third of all oncology trials using COAs. 41However, our analysis also revealed a broad, heterogeneous group of COA instruments used in oncology compared to non-oncology, with a third of the trials using generic instruments not specific to a disease group.Generic measures of quality of life that are more translatable across disease groups may be paired with measures of cancer site-and therapy-specific efficacy and toxicity profiles, given the growing development of cancer-specific therapies.Standardization of these efforts should reflect the FDA Patient-Focused Drug Development guidelines  and may be further facilitated by core outcome sets with specific COA recommendations within cancer types, 43 cancer-site-specific extensions to validated COAs as in the EORTC QLQ, and instrument libraries such as the Patient-Reported Outcomes Measurement Information System from which targeted items can be selected from a validated pool of questionnaires. 44ur study was limited to the search and analysis of data from the Clini calTr ials.govregistry, which was chosen for being the largest and among the more userfriendly clinical trial registries. 45However, Clini calTr ials.gov,similarly to other registries, is marked by low quality of data reporting, 46,47 often lacking structured or standardized information, and often missing information on exploratory endpoints whose reporting in the registry remains optional.Similarly, use of the free-access version of PROQOLID limited our search to a large but non-exhaustive subset of curated COAs and which, for example, excludes global assessment scales and omits custom-made instruments from industry or individual investigators.While these limitations have likely led to COA use estimates that are too conservative, it is reasonable to posit that oncology and non-oncology trials are similarly affected, and that the differences between the two classes of trials are not artifacts of the search methodology.Among the more complex limitations, we recognize that while the cataloged instruments were curated to be valid and reliable, we did not have information on how reliably and appropriately they were used in the trials or whether they captured the experience of diverse participants.
Event-oriented analyses were limited by subjective selection of events, challenges in disambiguating the effect of individual events, and potential bias from increasing reporting quality over time in the registry. 16However, our programmatic approach enabled a comprehensive, replicable, and efficient review of the data from the registry and utilization of information from additional databases.Future analyses of this kind may benefit from a greater push for COA reporting standardization not only at the existing level of protocol-building and publications, 10,12 but also for trial registries.Recommendations may include reporting of outcome placement of COAs, reporting specific instrument names, use of common terminologies within Clini calTr ials.gov,which, to the best of our knowledge, do not exist for COAs yet, and reporting of post-hoc analyses where COA data from trials are published, if at all. 40The resulting data transparency may also incentivize more intentional, a priori planning of COA outcomes in trial design, and guide future iterations of similar evaluations of COA use trends to gauge the field's progress.These steps may be necessary in the ongoing efforts to establish COAs as integral parts of clinical trial outcomes and advance the state of the practice.

F I G U R E 1
Characteristics of interventional oncology trials and associations with clinical outcomes assessment (COA) use.(A) Baseline characteristics of interventional oncology trials.(B) Proportion of COA use by trial characteristics.N indicates the number of trials using a COA in each category.

1 P h a s e 1 P h a s e 1 / P h a s e 2 P
y P h a s e D ia g n o s ti c S u p p o r t iv e C a r eT r e a t m e n to use COAs than those without DMCs (OR = 1.26, p < 0.001).Compared to the baseline rate of COA use among supportive care trials, trials focused on treatment (OR = 0.34, p < 0.001), diagnostic (OR = 0.12, p < 0.001), and all other purposes (OR = 0.26, p < 0.001) were significantly less likely to use COAs.Randomized trials were significantly more likely to use COAs than non-randomized studies (OR = 2.32, p < 0.001; Figure1B).Primary sponsor class, blinding, and single-versus multi-arm design were not significantly associated with COA use.

F I G U R E 2
Top clinical outcomes assessments used by trials and their indication.

F I G U R E 3
Trends of clinical outcomes assessment (COA) use over time among interventional oncology trials.(A) Linear trend in the number and proportion of trials using COAs across the timeline of notable events for COAs in clinical trials.(B) Trends in use of top COAs over time.
Rate of clinical outcomes assessment (COA) use and associations with trial characteristics via logistic regression.
T A B L E 1 a Number of trials reporting trial characteristic. 42 Clinical outcomes assessment (COA) use across time among oncology trials.COA use across relevant dates.
T A B L E 3 Note: Bolded values indicate statistical significance at α = 0.05.Abbreviation: OR, Odds Ratio.a Out of all interventional oncology trials.