A novel data mining application to detect safety signals for newly approved medications in routine care of patients with diabetes

ABSTRACT Background Clinical trials are often underpowered to detect serious but rare adverse events of a new medication. We applied a novel data mining tool to detect potential adverse events of canagliflozin, the first sodium glucose co‐transporter 2 (SGLT2 inhibitor) in the United States, using real‐world data from shortly after its market entry and before public awareness of its potential safety concerns. Methods In a U. S. commercial claims dataset (29 March 2013–30 Sept 2015), two pairwise cohorts of patients over 18 years of age with type 2 diabetes (T2D) who were newly dispensed canagliflozin or an active comparator, that is a dipeptidyl peptidase 4 inhibitor (DPP4) or a glucagon‐like peptide 1 receptor agonist (GLP1), were identified and propensity score‐matched. We used variable ratio matching with up to four people receiving a DPP4 or GLP1 for each person receiving canagliflozin. We identified potential safety signals using a hierarchical tree‐based scan statistic data mining method with the hierarchical outcome tree constructed based on international classification of disease coding. We screened for incident adverse events where there were more outcomes observed among canagliflozin vs. comparator initiators than expected by chance, after adjusting for multiple testing. Results We identified two pairwise propensity score variable ratio matched cohorts of 44,733 canagliflozin vs. 99,458 DPP4 initiators, and 55,974 canagliflozin vs. 74,727 GLP1 initiators. When we screened inpatient and emergency room diagnoses, diabetic ketoacidosis was the only severe adverse event associated with canagliflozin initiation with p < .05 in both cohorts. When outpatient diagnoses were also considered, signals for female and male genital infections emerged in both cohorts (p < .05). Conclusions and relevance In a large population‐based study, we identified known but no other adverse events associated with canagliflozin, providing reassurance on its safety among adult patients with T2D and suggesting the tree‐based scan statistic method is a useful post‐marketing safety monitoring tool for newly approved medications.


| INTRODUC TI ON
Identifying adverse events of a newly approved medication is initially based on the results of clinical trials. [1][2][3][4] This can be problematic since medications are typically approved based on 1 or 2 pivotal clinical trials that may enrol less than 1000 patients per drug and often select healthier patients than in usual care. 5,6 While the preapproval trials may provide information on common adverse events, rare serious adverse events may go undetected. 7,8 Until additional safety data are actively reported (eg voluntary reports regulators) or published in the scientific literature (eg observational studies), clinicians rely on relatively scarce data to evaluate a medication's safety.
Detecting drug-related adverse events, in particular rare events, generally requires a large sample size and prior knowledge of the potential association with a specific adverse event. 7,9 However, prior knowledge is often limited when a drug first enters the market. To detect unsuspected adverse reactions, data mining tools are advantageous as they are hypothesis-free 10,11 and can leverage information for millions of patients and thousands of potential outcomes when used in the context of longitudinal data sources such as healthcare claims data. 12,13 Tree-based scan statistics are a data mining approach implemented by the free TreeScan™ software (www.trees can.org), which can evaluate a wide range of health outcomes, arranged in a hierarchical tree, while adjusting for multiple testing. 10,11,[14][15][16] In pharmacovigilance, TreeScan was initially used to evaluate vaccine safety and was recently implemented by the Food and Drug Administration (FDA) to monitor the short-term safety of the human papillomavirus vaccine. 10,14,15 TreeScan has also been used to determine if it can identify well-established side effects of widely used medications, including diabetes medications and antifungal medications that have been in use for decades. 10 However, whether a more recently proposed method that combines TreeScan with propensity score-matched analysis in the context of a new-user active comparator study design can be used to reliably identify drug-related adverse events among patients with diabetes using newly approved medications remains unknown. 17 Thus, we sought to evaluate whether TreeScan combined with propensity score matching and a new-user active comparator design could help identify incident adverse events of a newly approved diabetes medication shortly after its market entry. This was implemented in a cohort study of adult patients with type 2 diabetes (T2D) initiating canagliflozin, the first marketed sodium glucose co-transporter 2 (SGLT2) inhibitors in the United States, compared to two active comparators between March 2013 and September 2015.

| Study population
We conducted a population-based, new-user, cohort study using data from the IBM MarketScan database. 12 This database includes patient demographics and longitudinal, patient-level data on healthcare utilization, inpatient and outpatient diagnostic tests and procedures, and pharmacy dispensing of drugs to over 50 million patients in the United States. 12 We compared adults with T2D who were newly prescribed canagliflozin or one of two comparators: a dipeptidyl peptidase 4 (DPP4) inhibitor (ie sitagliptin, saxagliptin, linagliptin, alogliptin) or a glucagon-like peptide 1 (GLP1) receptor agonist (ie exenatide, liraglutide, albiglutide, dulaglutide) in two pairwise comparisons between 29 March 2013 (date of approval of canagliflozin in the United States) and 30 September 2015 (last available data) (Figure 1). We focused on canagliflozin because it made up more than 90% of SGLT2 prescribing during this time period.
Patients with diabetes mellitus type 2 were identified using the International Classification of Diseases, Ninth Revision (ICD-9) codes similar to previous studies. 18 New users of canagliflozin or a DPP4 inhibitor were defined as those without a prior prescription for an SGLT2 inhibitor or a DPP4 inhibitor in the preceding 180 days.
Similarly, new users of canagliflozin or a GLP1 agonist were defined as those without a prior prescription for an SGLT2 inhibitor or a GLP1 agonist in the preceding 180 days. Cohort entry date was the date of first prescription. DPP4 inhibitors and GLP1 agonists were chosen as the comparator medications because during the study period they were considered as a second-line treatment for diabetes, similar to SGLT2 inhibitors. 7 Patients receiving both canagliflozin and a comparator on the cohort entry date were excluded. Patients with any of the following characteristics in the 180 days prior to cohort entry were also excluded: insufficient enrolment (ie less than 180 days of baseline data), end-stage renal disease or cancer. The latter two were identified using ICD9 codes similar to prior studies. 18 The Brigham and Women's Hospital Institutional Review Board provided ethics approval and a valid data use agreement for the IBM MarketScan ('MarketScan') database was in place.

| Cohort follow-up
Follow-up began on the day after cohort entry and continued until the first occurrence of the end of the study period (ie the first of: 30 September 2015, 180 days after the index date, end of continuous health plan enrolment, discontinuation of the initial medication or switching to or adding one of the comparator medications, or death). The follow-up period was truncated at 180 days since we were interested in acute rather than long-term adverse reactions. 10 A medication was considered discontinued if 60 days elapsed after the expiration of the last prescription's supply. 7,18

| Baseline covariates
Patient demographics and characteristics were assessed during the 180 days before cohort entry. The characteristics were selected based on diagnoses and procedures covered: chronic medical conditions, markers of diabetes severity, healthcare utilization, diabetes medications and non-diabetes-related medications.

| Hierarchical tree of potential outcomes
The potential outcomes to be included in our hierarchical classification system ('tree') were developed using ICD-9 diagnosis codes.
We removed outcomes that were unlikely to represent an acute

| Incident outcomes
We defined an incident outcome as the first inpatient or emergency department diagnosis code that occurred during a patient's available follow-up time for which there was not another inpatient, emergency department or outpatient diagnosis with the same ICD-9 code during the 180 day period. 10 Specifically, in the tree looking at second level outcomes, if the exact second level outcome occurred in the preceding 180 days (in addition to the 180 days before the index date), then this event would not be counted. This step was purposeful to increase the likelihood of identifying real incident events, rather than pre-existing chronic medical conditions. If there were more than one potential incident outcome on the same day, we selected the one that was less common based on the frequency of the code in our dataset. 10 This approach is in line with prior studies applying TreeScan because a key aim of this approach is to detect rare adverse events and has the goal to reduce the likelihood of false signals.
Incident outcomes were assessed at the second, third, fourth and fifth level of the ICD9 hierarchical tree. Potential outcomes at level 1 of the tree were not considered because of the broad nature of these categories. In secondary analyses, we also explored incident outcomes based on outpatient diagnoses in addition to inpatient and emergency visits. This was to assess signals for potential adverse events that may be managed in an outpatient setting without requiring a hospitalization or an emergency department visit.

| Statistical analysis
Propensity score (PS) matching methodology was used to adjust for confounding using a nearest neighbour matching within a caliper of 0.05. 17 The probability of initiating canagliflozin versus a DPP4 inhibitor or a GLP1 agonist was calculated through a multivariable logistic regression model which contained all of the potential confounders at baseline. The estimated PS was used to match initiators of canagliflozin with initiators of a comparator, using variable ratio matching with up to 4 comparators to each canagliflozin initiator. Covariate balance between the matched cohorts was assessed using standardized differences. 19 A standardized difference of 0.1 or less indicates negligible differences between groups. 19 The standardized differences were calculated for each of the two pairwise comparisons.
The TreeScan method tests the null hypothesis of no difference in risk of adverse events in any outcome node in the tree against a one-sided alternative that there is at least one outcome node where the risk of adverse events is higher in the exposed group than in the comparator group. When screening potential multiple outcomes for signal identification, it is critical to control the rate of false positives.
TreeScan generates multiplicity-adjusted p-values that accurately reflect the type I error rate in the absence of confounding. 10,16,17,[20][21][22][23] That is, if there is not a single outcome with an excess risk, we have a 95% probability of finding zero signals.
We used the unconditional Bernoulli tree-based scan statistic.
To meet the assumptions of this statistic, all patients within each matched set were censored at the end of follow-up of the canagliflozin initiator or at the end of follow-up of the uncensored comparator initiator with the longest follow-up, whichever came first.
Failing to do so would result in differential follow-up time making it challenging to know if an observed signal is related to a true adverse event or is instead related to a longer follow-up period for detection. The log-likelihood ratio for each node was calculated based on the number of cases in the exposed (ie canagliflozin) or comparator group (ie DPP4 inhibitors or GLP1 agonists) as well as the probability of being in the exposed group (Appendix Figure A1).
For our matched cohort, this probability was set to the proportion of patients receiving canagliflozin or a comparator. Since the distribution of the tree-based scan statistic method is unknown, we derived multiple testing adjusted p-values non-parametrically using Monte Carlo hypothesis testing where permutations of the data are generated under the null hypothesis. 10 The multiple testing adjusted p-value was determined by ranking the test statistics from 9,999 datasets simulated under the null and the observed dataset from largest to smallest. The p-value was calculated as the rank of the observed dataset test statistic divided by 10,000 (9,999 simulated datasets +1 observed dataset). The multiplicityadjusted p-values were interpreted as the probability of seeing an association of the observed magnitude or one more extreme if the null hypothesis was true. Together with the relative risk estimates, these p-values were used as a means to prioritize alerts for further investigation (Appendix Figure A2). Specifically, we rank ordered the signals by their p-value from lowest to highest p-value. As a surveillance method to detect potential problems, the alerts should not determine whether there is an association without such a follow-up investigation. Rate ratios and rate differences per 1,000 person years were calculated nominally.
The cohort was generated using R version 3.

in the validated
Aetion platform. 24 The hierarchical tree was built using SAS Version 9.4 and scanned using the free TreeScan v9.4 software available at: www.trees can.org.

| Study population
After the application of the study selection criteria (Figure 1 inhibitor (Appendix Table A1). All differences in patient characteristics were well balanced, as assessed by standardized differences.
Across the two pairwise cohorts, study participants had average age of 55 years, 9% had history of ischaemic heart disease, and 4% had a recent hospitalization. Patients included in the canagliflozin vs. DPP4 inhibitor pairwise cohort were more frequently males compared with patients included in the canagliflozin vs. GLP1 agonists pairwise cohort (53% vs. 50%), and they were more frequently treated with metformin (63% vs. 56%), less frequently treated with insulin (23% vs. 26%) and had less frequent visits with an endocrinologist (11% vs. 15%). (Table 1). The average duration of follow-up was approximately 19 weeks.

| TreeScan-detected signals for potential adverse events
When assessing potential serious incident adverse events based on inpatient or emergency room diagnoses, TreeScan identified signals for a potential increased risk of diabetes ketoacidosis associated with canagliflozin initiation compared with the initiation of a comparator medication in both pairwise cohorts (Table 2). Specifically, signals emerged at the fourth level of the ICD9 hierarchical tree in the canagliflozin vs. DPP4 inhibitor cohort (p = .043) and at the fourth and fifth level in the canagliflozin vs. GLP1 agonist cohort (p = .0006 and p = .032, respectively). A complete list of potential signals is provided in the appendix.
When we considered potential adverse events based on any di- infections associated with the use of canagliflozin compared with the use of a comparator medication in both cohorts (Table 3). Signals emerged at all investigated levels of the ICD9 hierarchical tree and included specific clinical conditions (eg candidiasis of vulva and vagina, balanoposthitis, vaginitis and vulvovaginitis), as well as aspects pertaining to symptoms, laboratory findings or aspects of care related to genital infections (eg pruritus of genital organs, glycosuria, gynaecological examination).
No other clinical entities generated signals that were deemed to require further investigation (see Appendix Table A2-A5 for all TreeScan generated results).

| DISCUSS ION
In this large population-based study of adult patients with T2D, TreeScan consistently identified diabetic ketoacidosis and genital infection as potential adverse events associated with the initiation of canagliflozin compared with the initiation of other T2D medications.
These represent known adverse events associated with canagliflozin and provide a proof of principle that TreeScan may help monitor the safety of new medications.
The most common adverse event with canagliflozin, and other SGLT2 inhibitors, is yeast infections of the genitalia. This is based on data from both observational studies and a recent meta-analysis of clinical trial data. 25  .0001 Gynaecological examination (V72.3x) implementation of TreeScan and other data mining tools. 34 Our study provides a framework for how TreeScan might be applied to identify potential adverse events of newly marketed medications.
We have identified four important methodologic aspects to using TreeScan which we will discuss individually.
First, an appropriate comparator should be selected.
Identifying an appropriate comparator requires expertise in the clinical domain being studied. We identified DPP4 inhibitors and GLP1 agonists as two potential active comparators since both were second-line medications for T2D at the time of this investigation. 35 Using two separate active comparators allowed us evaluate the robustness of our results, but there are many clinical scenarios where only one active comparator exists. In our study, regardless of the active comparator used, diabetic ketoacidosis and genital infection were consistently observed associated adverse events of canagliflozin.
Second, a tree of diagnoses is required. Using ICD9 codes is one approach because the data to construct the hierarchical tree are publicly available. How the tree should then be pruned depends on the clinical context. We excluded groups of diagnoses that were unlikely to represent acute drug reactions (eg congenital diagnoses) to limit false signals. Another approach is to include an unpruned tree that includes all diagnoses, but this slightly decreases power to detect effects of exposure. One potential approach is to include an unpruned tree as a sensitivity analysis, but this can lead to spurious findings.

| CON CLUS ION
In a large population-based study, we identified known but no other adverse events associated with canagliflozin, providing reassurance on its safety among adult patients with T2D. The results of our study demonstrate that TreeScan may aid in the process of studying the safety of newly approved medications soon after approval, providing information on signals for potential adverse events in near real time. Additional studies will be necessary to understand the settings where this approach works well and settings where it may not.       Abbreviations: GLP-1RA: glucagon-like peptide-1 receptor agonists; LLR: log-likelihood ratio; p: p-value. a Based on inpatient or emergency department diagnoses (any position). Abbreviations: GLP-1RA: glucagon-like peptide-1 receptor agonists; LLR: log-likelihood ratio; p: p-value. a Based on inpatient, emergency department, or outpatient diagnoses (any position).