Association between cancer‐specific adverse event triggers and mortality: A validation study

Abstract Background As there are few validated measures of patient safety in clinical oncology, creating an efficient measurement instrument would create significant value. Accordingly, we sought to assess the validity of a novel patient safety measure by examining the association of oncology‐specific triggers and mortality using administrative claims data. Methods We examined a retrospective cohort of 322 887 adult cancer patients enrolled in commercial or Medicare Advantage products for one year after an initial diagnosis of breast, colorectal, lung, or prostate cancer in 2008‐2014. We used diagnosis and procedure codes to calculate the prevalence of 16 cancer‐specific "triggers"–events that signify a potential adverse event. We compared one‐year mortality rates among patients with and without triggers by cancer type and metastatic status using logistic regression models. Results Trigger events affected 19% of patients and were most common among patients with metastatic colorectal (41%) and lung (50%) cancers. There was increased one‐year mortality among patients with triggers compared to patients without triggers across all cancer types in unadjusted and multivariate analyses. The increased mortality rate among patients with trigger events was particularly striking for nonmetastatic prostate cancer (1.3% vs 7.5%, adjusted odds ratio 1.96 [95% CI 1.49‐2.57]) and nonmetastatic colorectal cancer (4.1% vs 11.7%, 1.44 [1.19‐1.75]). Conclusions The association between adverse event triggers and poor survival among a cohort of cancer patients supports the validity of a cancer‐specific, administrative claims‐based trigger tool.


| INTRODUCTION
Despite the complexity and potential toxicity of cancer-specific therapy, there are few high-quality studies characterizing the nature and extent of treatment-related errors and injuries in clinical oncology. [1][2][3] While toxicity assessment is deeply engrained in the cancer clinical trials tradition, much of this work seeks to identify the type and severity of adverse drug reactions inherent in novel therapies rather than injuries due to medical care that may follow a medical error or occur in a vulnerable host. A recent literature review concluded that no consistent methodology or large-scale study allowed for an epidemiologically robust understanding of the extent of oncology treatment-related errors and injuries by facility, location, or across the continuum of cancer care. 4 A well-established approach to identify treatment-related complications uses "trigger" events (such as abnormal laboratory results, unplanned transfer to an intensive care unit, return to the operating room, or administration of antidote medications) to flag candidate adverse events for further review. [5][6][7][8][9][10] Unfortunately, due to the high expected symptom burden among cancer patients undergoing multi-modal therapy, trigger tools evinced poor interrater reliability and low positive predictive values (PPVs) in multiple European studies. [11][12][13][14][15][16][17][18] However, researchers recently reported the successful development of 49 oncology-specific triggers using clinical data from patients at Memorial Sloan Kettering Cancer Center (MSK) undergoing an initial course of cancer-directed therapy. The PPV of the MSK triggers, using physician chart review as the gold standard, was satisfactory at 0.48 for adverse events and 0.18 for preventable events. 19,20 To extend this approach, we developed a set of oncology-specific triggers that used International Classification of Diseases (ICD) and Current Procedural Terminology (CPT) codes to flag eligible cases in a large administrative database of commercial claims. We found high rates of oncology-specific trigger events among a cohort of over 300,000 patients undergoing an initial course of cancer-directed therapy for breast, colorectal, lung, or prostate cancer. 21 The triggers affected 12% of patients with nonmetastatic disease and 39% of those with metastatic cancer. The burden was highest among patients with lung and colorectal cancer, and triggers were overrepresented among non-whites, those with lower family incomes, and patients with lower educational attainment.
The goal of this study was to assess the construct validity of oncology-specific triggers as a quality of care measure by examining the association between oncology triggers and mortality using administrative claims data. We hypothesized that patients who experienced trigger events would have higher mortality rates (controlling for relevant covariates) during an initial year of cancer-directed therapy, and that the trigger-mortality association would vary by trigger, cancer type, and metastatic status.

| Data
We accessed the OptumLabs ® Data Warehouse (OLDW), which includes deidentified administrative claims and electronic health record (EHR) data on over 200 million patients, including billing data for inpatient and ambulatory care for commercial and Medicare Advantage enrollees. 22 It includes limited patient demographic information drawn from enrollment records. Socioeconomic status information in OLDW, including race/ethnicity, household income, and educational attainment, are imputed variables sourced from a national supplier of consumer marketing data. Mortality status is ascertained in OLDW through multiple sources including the Social Security Death Index, inpatient discharge status, and electronic medical records.

| Subjects
We selected a cohort of patients undergoing an initial course of cancer-directed therapy for breast, lung, colorectal, and prostate cancer, using ICD and CPT codes to identify patients who received surgery, radiation therapy, or chemotherapy (including oncolytic, hormonal, or targeted therapies) for a new diagnosis of breast, lung, colorectal, or prostate cancer from 1 January 2008 through 31 December 2014. Patients with cancer diagnoses or treatments in 2005-2007 and those with cancer recurrence codes were excluded. [23][24][25] We abstracted sociodemographic variables included in the OLDW including age, gender, race/ethnicity, insurance (commercial, Medicare Advantage), household income, and educational attainment, excluding subjects with male breast cancer and those under age 18. We also abstracted cancer diagnosis, cancer-specific therapies, and associated diagnosis and treatment dates. We calculated a modified Charlson comorbidity index using an algorithm that excluded cancer as a comorbidity, 26 and an algorithm developed and tested in the OLDW to identify metastatic cancer cases. 27

| Measurements
We defined a set of oncology-specific triggers using ICD and CPT codes corresponding to 16 of the 23 highest (≥50%) PPV triggers from the MSK study. We constrained triggers to events that occurred within specified intervals following relevant exposures, and further assumed that triggered events had a discrete, limited duration. For example, blood transfusion could result from chemotherapy-associated myelosuppression or intraoperative blood loss, but is an unlikely direct result of radiation therapy.
Chemotherapy-associated anemia requiring transfusion would be expected to develop over and persist for several weeks, while peri-operative anemia would present within hours or days and be addressed within a similar interval. Treatment-trigger causal and temporal relationships and trigger event durations were developed in collaboration with clinical oncologists.

| Analyses
We characterized the cohort by sociodemographic and clinical attributes and cancer-specific treatments, stratifying analyses by cancer type (breast, colorectal, lung, prostate) and metastatic status. We tabulated the number of patients who experienced at least one of each trigger type during the first year of cancer therapy by cancer type and metastatic status.
To assess the association of event triggers with mortality, we examined the number and percent of patients with each adverse event trigger who were alive or dead at the end of a one-year period beginning with the date of the initial cancer-directed therapy, performing separate analyses by cancer type and metastatic status.
We estimated adjusted odds ratios and 95% confidence intervals with logistic regression models stratified by cancer type and metastatic status to examine the relationship between the mortality outcome and types of triggers. Independent variables in the regression models controlled for sociodemographic and clinical attributes and the type of cancer-directed therapy-factors that could affect the risk of death. Sociodemographic characteristics included age, gender, race/ethnicity, household income, and educational attainment. Clinical attributes included the Charlson index and treatment type (chemotherapy only, surgery only, radiation only, or multi-modality therapy). Analyses also adjusted for the presence of other triggers.
In preliminary analyses, we noted slightly different trigger prevalence rates among patients with and without missing race/ethnicity, income, or educational attainment data for selected triggers. There was no consistent pattern, however. Concerned that our results could potentially be biased if trigger events were associated with incomplete collection or reporting of sociodemographic information, we used multiple imputation by chained equations, with predictive mean matching for numeric variables and logistic regression for binary variables, to impute multivariate missing data elements for patients with incomplete information. The model to impute missing information included the outcome variable, patient age, gender, race/ethnicity, household income, educational attainment, Charlson index, treatment type, and trigger indicators. 28,29 Average estimates and total variance from regression models fit to each of the imputed data sets were pooled following Rubin's rules. 30 Multiple imputation imputes values for missing observations of a variable using other available information in the dataset. We imputed missing values under the assumption that data are missing at random (MAR), in other words, independent of the value of the variable and conditional on observed covariates. Ignoring the mechanism of "missing-ness" by excluding records with any missing values could lead to biased estimates. While the MAR assumption is not fully testable, advantages of reduced risk in bias compared to a complete case analysis make MAR imputation the most reasonable approach. We understand that missing race/ethnicity, income, and education data reflects administrative practices at insurers that do not incorporate or prioritize collection of this information for enrollment. We are aware of no systematic biases introduced in this process.
In sensitivity analyses designed to assess the reliability of the model under different assumptions about patient enrollment, we examined the impact of excluding patients without a full year of continuous enrollment (and alive at last encounter) on the results. We knew from preliminary analyses that "no trigger" patients were more often lost to follow-up than patients with triggers across all disease types. For example, 12.0% of "no trigger" patients with nonmetastatic breast cancer were lost to follow-up compared to 10.4% among those with at least one trigger. Since there were more patients with incomplete follow-up in the "no trigger" group, excluding these patients could increase the odds of finding higher mortality among "trigger" patients. In an alternate scenario, we assumed that lost-to-follow-up patients had all died. This worst-case scenario would bias the analysis toward the null hypothesis. The "exclusion" and "worst-case" scenarios were compared to the primary analysis that included all lost-tofollow-up patients and imputed missing one-year mortality status.
Analyses were performed using SAS 9.4 for Windows (SAS Institute) and R 3.4.3 (The R Foundation). The Tufts Health Sciences Institutional Review Board (IRB) determined the project to be exempt from human subjects review.

| Cohort characteristics
The final cohort included 322 887 patients ( Table 1). The average age was 64. There was a higher percentage of men than women among patients with colorectal and lung cancer. Nonwhites accounted for at least 13% of the cohort, but race/ethnicity, household income, and education data were missing for about one-third of the cohort. Twenty-seven percent of patients had metastatic disease, with variation by cancer type.

| Trigger prevalence
As we have shown previously, trigger events were common among cohort members, affecting 19% of patients (Appendix A). Patients with colorectal and lung cancers were most likely to experience at least one trigger event. Patients with metastatic disease were particularly vulnerable to cancerspecific triggers, affecting 41.4% of those with metastatic colorectal and 50.2% of those with metastatic lung cancers. The most commonly flagged triggers were for abnormal serum bicarbonate, blood transfusion, noncontrast chest CT following radiation therapy, hypoxemia, contact precautions, neutropenic fever, and abnormal serum potassium.

| Overall mortality
Compared to patients with no oncology trigger, mortality among cancer patients with a treatment-related trigger was substantial. As shown in Table 2, the overall one-year mortality rate among patients with nonmetastatic cancer and at least one trigger event was higher than patients without a trigger across all cancers. For example, among patients with nonmetastatic breast cancer, 0.8% of no-trigger patients died within one year, while 3.1% of those with at least one trigger died. The absolute increase in mortality was particularly striking for nonmetastatic colorectal cancer (4.1% vs 11.7%) and nonmetastatic lung cancer (18.1% vs 33.7%). Oncology-specific triggers were associated with increased one-year mortality among patients with metastatic disease, albeit on a higher baseline mortality risk. As expected, the one-year mortality rate for patients with metastases exceeded the rate of those with nonmetastatic disease for each cancer type. However, patients with metastatic disease and at least one trigger had poorer survival than those with metastatic disease of the same cancer type with no trigger. For example, among patients with metastatic prostate cancer, 8.8% of no-trigger patients died within one year while 20.1% of those with at least one trigger died. In multivariate analyses involving the full cohort and controlling for potential confounders, experiencing at least one cancer-specific trigger conferred increased odds of death within one year across all cancer types and for those with metastatic and nonmetastatic disease.

| Trigger-specific mortality
We replicated the "any trigger" analysis for each individual trigger, examining the relationship between specific triggers and one-year mortality. The one-year mortality rate varied by trigger and cancer type. Among patients with nonmetastatic cancer (Table 3), the mortality rate was higher among patients with 13 of the 14 cancer-specific triggers where there were sufficient cases to calculate a death rate. (Cell frequencies <11 cases are not reported to avoid identifying unique patients.) Unexpectedly, lower one-year mortality was associated with contact precautions among patients with nonmetastatic breast, colorectal, and lung cancers, suggesting that trigger events may be protective under certain circumstances. Institution of contact precautions may, for example, signal an appropriate response to early detection of an infectious complication. Table 4 displays mortality rates, by trigger, among patients with metastatic disease. Mortality rates were higher among patients with a cancer-specific trigger than among patients with no trigger with several exceptions (eg, anticoagulation and return to the operating room triggers among colorectal cancer patients, and elevated creatinine among patients with lung cancer). Like the nonmetastatic cancer cases, contact precautions among patients with metastatic disease were associated with lower mortality rates across all cancer types.
We found a similar pattern in multivariate analyses, modeling the independent relationship of each trigger to mortality by cancer type and metastatic status. As shown in Table 5, bacteremia, blood transfusion, hypoxemia, and nephrology consultation were triggers most commonly associated with increased odds of death. In contrast, contact precautions were associated with reduced odds of death across most cancer types.

| Sensitivity analyses
As there was a higher percent of patients lost to follow up among those with no trigger compared patients with at least one, it was possible that exclusion of lost-to-follow-up patients might increase the apparent mortality risk in the cohort. We performed sensitivity testing to examine the durability of results based on the exclusion, inclusion, or imputation of mortality outcomes of patients lost to follow-up during the initial year of cancer-directed therapy. We found, for example, that the adjusted odds of death in the first treatment year for metastatic breast cancer patients with at least one trigger event was 1.57 using multiple imputation. This value was between 1.68 (lost cases excluded) and 1.21 (lost cases presumed dead), consistent with our hypothesis. This pattern was consistent in sensitivity analysis with each disease type and by metastatic status, as shown in Appendix B. Appendix B also shows that lost-to-follow-up patients comprised 4%-12% of the sample. Reassured about the consistency of the analyses, we presorted adjusted odds ratios and confidence intervals in Tables 2 and 5 using imputed mortality outcomes that include patients lost to follow-up.

| DISCUSSION
In this retrospective cohort study of 322 887 patients with breast, colorectal, lung and prostate cancer treated for an initial course of cancer-directed therapy, patients with at least one cancer-specific trigger had a substantially higher mortality risk than patients with no trigger. This relationship was robust across cancer types and metastatic status in logistic regression models controlling for multiple potential confounders. Patients who experienced treatmentrelated triggers were at substantially higher risk than those with none. Individual triggers, including bacteremia, blood transfusion, hypoxemia, nephrology consultation, neutropenic fever, pressure ulcers, and return to the operating room or interventional radiology suite, were strongly associated with poor one-year survival.
To assess the validity of cancer-specific adverse event triggers, we hypothesized that patients with any adverse event trigger would be more likely to die than those with no trigger, other things equal. This effect should be evident among a cancer cohort as these are patients with serious illnesses undergoing toxic therapies and may have relatively  Observed mortality rate (n died/n exposed); excludes patients lost to follow-up.  Observed mortality rate (n died/n exposed); excludes patients lost to follow-up.  little physiologic reserve to overcome an iatrogenic injury. The convergence of triggers, which denotes treatment-related adverse events, and mortality would support the construct validity of event triggers as quality of care measures. Previous validation studies of adverse event trigger tools relied on concurrent or retrospective medical record review. [7][8][9]31 Expert clinicians judged whether a trigger indicated that the patient suffered harm. Validation rates varied by trigger, with PPVs of 17%-45% for groups of triggers comprising a trigger tool using physician chart review as the gold standard. Lipitz-Snyderman and colleagues used this approach to derive the triggers employed in the current study, selecting from among 49 candidate triggers with an aggregate PPV of 44.8% those 16 items with individual PPVs exceeding 50%. 19 We are aware of no previous research applying trigger tools to administrative data. However, Iezzoni, Romano, and others have used diagnosis and procedure codes to flag potential surgical and procedural complications in what have become the US Agency for Healthcare Research and Quality's Patient Safety Indicators (PSIs). [32][33][34] Validation of the PSIs relied initially on demonstration of higher-than-expected mortality rates among patients with PSI flags compared to patients without, correlation with other quality metrics, and retrospective chart review of flagged cases. [35][36][37][38] While many PSI indicators have been criticized for lack of or poor validation based on medical record review, advocates argue that rate-based metrics (such as PSI-3 "failure to rescue") can be validated based on correlation with mortality and other outcome metrics. 39 Medical record review is generally the gold standard for evaluating the validity of a proposed quality measure. Our adverse event triggers were derived from medical record review and then encoded using diagnosis and procedure codes for use with billing data. In future research, claims-based triggers will need to be "reverse" validated to ensure triggers appropriately identify problematic cases. Several groups have described the use of triggers, embedded in electronic medical record systems, to flag cases with potential adverse events for investigation and mitigation. [40][41][42][43][44] That said, there is significant potential benefit from trigger tools that use administrative data, as these instruments can be used to examine the performance within and across health care organizations, networks, and regions to characterize patient safety outcomes. This is especially important in cancer care, where institution and population-level quality metrics are sparse. 45 A measure set that draws from administrative data and spans the continuum of cancer care over an initial course of therapy may afford clinical leaders, researchers, and policy makers a useful tool for assessing disease-and treatment-specific harm among extraordinarily vulnerable patients.

|
This research is subject to several limitations. The OLDW includes information about commercial and Medicare Advantage patients and our findings therefore may not be generalizable to a Medicare Fee for Service or Medicaid cohort. Claims-based algorithms may fail to accurately distinguish patients with late recurrences and to characterize those with metastatic disease, although we sought to minimize this problem by drawing on well-validated coding algorithms. Given the burden of disease-related morbidity in cancer care, there is expected confounding of adverse events attributed to disease and to treatment. We attempted to address this inherent challenge by linking treatment exposure to triggered event by type of exposure, timing of event relative to exposure, and duration of event. While this approach improved the likelihood that a given trigger was caused by treatment, perfect attribution of trigger to treatment would require expert chart review. Certain important clinical variables, such as performance status (ie, the type of assistance required to perform normal activities), were unavailable to the research team and could affect mortality risk. Although we may have underestimated mortality rates given the wellknown limitations of the Social Security Death Master File, use of mortality status in OLDW mitigates some concern by drawing on multiple data sources.
In sum, this commercial claims-based study found a consistent association between cancer-specific adverse event triggers and poor survival among a large cohort of patients with breast, colorectal, lung, and prostate cancer. This finding supports the validity of a cancer-specific trigger tool for measuring quality of care in oncology, although further research is needed to replicate these findings.