• acute myocardial infarction;
  • diabetes mellitus;
  • drug safety;
  • drug surveillance;
  • saxagliptin


  1. Top of page


To describe a protocol for active surveillance of acute myocardial infarction (AMI) in users of a recently approved oral antidiabetic medication, saxagliptin, and to provide the rationale for decisions made in drafting the protocol.


A new-user cohort design is planned for evaluating data from at least four Mini-Sentinel data partners from 1 August 2009 (following US Food and Drug Administration's approval of saxagliptin) through mid-2013. New users of saxagliptin will be compared in separate analyses with new users of sitagliptin, pioglitazone, long-acting insulins, and second-generation sulfonylureas. Two approaches to controlling for confounding will be evaluated: matching by exposure propensity score and stratification by AMI risk score. The primary analyses will use Cox regression models specified in a way that does not require pooling of patient-level data from the data partners. The Cox models are fit to summarized data on risk sets composed of saxagliptin users and similar comparator users at the time of an AMI. Secondary analyses will use alternative methods including Poisson regression and will explore whether further adjustment for covariates available only at some data partners (e.g., blood pressure) modifies results.


The results of this study are pending.


The proposed protocol describes a design for surveillance to evaluate the safety of a newly marketed agent as postmarket experience accrues. It uses data from multiple partner organizations without requiring sharing of patient-level data and compares alternative approaches to controlling for confounding. It is hoped that this initial active surveillance project of the Mini-Sentinel will provide insights that inform future population-based surveillance of medical product safety. Copyright © 2012 John Wiley & Sons, Ltd.


  1. Top of page

One of the goals of the Sentinel Initiative of the US Food and Drug Administration (FDA) is to develop a capability for conducting active surveillance to evaluate the potential risks (or adverse outcomes) associated with medical products when in use in the general population. Acute myocardial infarction (AMI) is a good example of such an outcome and one that has emerged repeatedly in the past decade as an area of concern.[1-3] Thus, in developing the Year 1 workplan for the Mini-Sentinel pilot—a component of the Sentinel Initiative created to inform and facilitate the development of a fully operational Sentinel System—a high priority was to create a protocol for active surveillance for drug-induced AMI.

Since 2008, FDA has requested that new medications for treatment of Type 2 diabetes mellitus (T2DM) be systematically evaluated for risk of cardiovascular disease (CVD).[4] The FDA approved a new dipeptidyl-peptidase-4 inhibitor, saxagliptin, for treatment of T2DM on 31 July 2009. Preapproval trials provided no signal of increased CVD risk with saxagliptin. In fact, pooled data suggest that it could be modestly protective.[5] However, these trials were not prospectively designed to assess CVD risk, had low reported CVD event rates, and excluded patients at high risk for CVD events from the trial. With help from an advisory committee, FDA determined that there was adequate evidence of CVD safety to support approval of saxagliptin with a need for additional assessment after approval. The FDA mandated a randomized, double-blind, controlled postmarket trial evaluating saxagliptin's effect on the incidence of major CVD events.[6] The primary objective of the trial is to establish that the upper bound of the 2-sided 95% confidence interval for the estimated risk ratio comparing CVD incidence in saxagliptin users versus the control arm is less than 1.3.

When FDA reviewed potential new molecular entities for the first active surveillance activity in Mini-Sentinel, saxagliptin provided a good option, in particular because of the planned postmarket randomized trial. Comparing results of active population-based surveillance with results of the trial could yield important insights.

This article describes the protocol developed by a group of Mini-Sentinel collaborators working together with the FDA in September 2010.[7] Given that this is the first active surveillance project conducted within the Mini-Sentinel pilot,[8] we describe the deliberations underlying the protocol in hopes that they will be useful in planning future surveillance projects within the Sentinel Initiative and elsewhere. In instances where the best methodological approach to active surveillance was not apparent, the protocol incorporated two distinct approaches, which we plan to compare for feasibility and efficiency during the project. The full protocol, including a more detailed presentation of workgroup deliberations, is available online.[7]


  1. Top of page

Surveillance of a parallel cohort is proposed in which new users of saxagliptin are identified, beginning in August of 2009, along with new users of four comparator antidiabetic agents and followed longitudinally for incidence of AMI. In separate analyses, the experience of new saxagliptin users is compared with that of each comparator after removing prior users of that comparator from the saxagliptin cohort.

The new-user design[9, 10] is chosen over a mixed-user design (i.e., prevalent as well as new users) for several reasons. Essentially all saxagliptin users would be new users because identification of users begins on the day saxagliptin became available in the USA. This design therefore equalizes duration of use at the beginning of follow-up, removing risk of a “healthy user” or “depletion of susceptibles” bias, wherein patients who have already survived early exposure to therapy may be more prevalent in comparator groups. This design also allows detection of early and delayed effects of the drug of interest, ensures that all persons included in the analyses were considered eligible and appropriate for starting a new antidiabetic therapy when follow-up begins, and allows measurement of key patient characteristics before therapy begins.

Data sources

New users of saxagliptin and comparator agents, along with enrollment, covariate, and endpoint information, will be extracted from at least four data partners of the Mini-Sentinel pilot: HealthCore, Inc.; Humana; The HMO Research Network; and Kaiser Permanente's Center for Effectiveness and Safety Research. The Mini-Sentinel data partners have each organized and formatted their data into the Mini-Sentinel Common Data Model, which is a standardized, distributed data structure that enables data partners to run a single search or analysis program against data stored locally, thus creating the Mini-Sentinel Distributed Database (MSDD). [11] Aggregated follow-up time and events, as will be described, will be submitted to the coordinating center in 2011. Additional transfers will occur as frequently as quarterly, depending on the volume of new users and frequency of data updates through mid-2013.

In 2010, the Office for Human Research Protections, Department of Health and Human Services, determined that the Common Rule does not apply to activities conducted under the Sentinel Initiative. FDA has determined that the Mini-Sentinel pilot is part of the Sentinel Initiative. Therefore, institutions participating in Mini-Sentinel do not need to obtain review by their institutional review boards to participate or provide data for Mini-Sentinel activities.

Choice of comparators

New users of four comparator antidiabetic agents (sitagliptin, long-acting insulins, pioglitazone, and second-generation sulfonylureas (glimepiride, glipizide, and glyburide)) will be identified, followed, and compared in separate analyses to new users of saxagliptin. Multiple comparators were specified because there are many treatment alternatives to saxagliptin for patients with T2DM, whether used as a second- or third-line therapy. These agents may have differing CVD risk profiles, although actual risks and risk differences are currently not well defined. Thus, the comparative safety of saxagliptin versus each comparator is of interest, and no rationale for choosing one primary comparator was apparent. An additional advantage of multiple comparators is the opportunity provided to compare risks among these agents.

Identification of the new-user cohort

The cohort will include all eligible patients aged 18 years and older during the surveillance period. A previous diagnosis of diabetes or a previous prescription for another antidiabetic medication will be required to ensure that each patient truly has diabetes. If new use is for insulin, further evidence will be required of a prior prescription for an oral antidiabetic agent to reduce chances of including persons with Type 1 diabetes.

New dispensings for saxagliptin or a comparator will be sought beginning 1 August 2009. A 1-year period of continuous health plan enrollment and prescription drug coverage (i.e., no more than a 45-day gap in either) is required before the first new antidiabetic agent dispensing. Relatively high rates of enrollee turnover within certain health plans preclude requiring longer prior enrollment. This 1-year follow-back period will allow inclusion of some apparent new users who are not truly new users because of past use before the follow-back period. This may not be rare among apparent new users of comparators that have been commonly used for many years. We will attempt to quantify the extent of this potential concern by looking for earlier use in persons who meet criteria for new use but have more than 12 months of prior enrollment (i.e., up to 10 years of data are available in the MSDD).

There are few other reasons for exclusion from the new-user cohort. In analyses of each comparator, any prior users of saxagliptin would also be excluded. However, prior use of one comparator does not preclude inclusion as a new-user of another comparator; over time, patients can become new users of more than one comparator. In comparisons of saxagliptin with pioglitazone, patients with any history of chronic heart failure will be excluded from both groups because the condition is a relative contraindication to pioglitazone and may also predict AMI.

The FDA-mandated postmarket trial of saxagliptin is excluding patients who had been discharged within the prior 60 days for an AMI.[6] We will do the same because persons with such a recent history of AMI are at high risk for having a subsequent event during the aftermath of the first event, a period when the likelihood of getting treatment intensification is much greater and the choice of agents different, with a greater tendency to initiate insulin. Thus, confounding may be quantitatively greater shortly after an AMI.

Patients with other CVD diagnoses in the 12-month baseline period also raise concerns about confounding. A broad range of CVD-related comorbidities are considered to be potential complications of T2DM. Their presence suggests greater severity of disease and can lead to confounding for the reasons mentioned earlier. There was additional concern that the relationships of other confounders to both treatment choices and AMI risk may differ in the presence of CVD in ways that make standard approaches to confounder adjustment inadequate. For these reasons, all analyses will first be done separately in subgroups defined by presence versus absence of CVD. Subsequent combined analyses of the two subgroups will be stratified.

Cohort follow-up

Follow-up begins on the date of the first filled prescription for saxagliptin or comparator and continues until the first of the following: occurrence of an AMI, fatal or nonfatal; death from any cause; disenrollment, that is, a gap of greater than 45 days in continuous enrollment or pharmacy benefit coverage; a gap in medication possession for the drug of interest that is greater than one third of the days' supply at the most recent dispensing (and at least 10 days); or the end of the surveillance. For patients followed as saxagliptin users, adding a comparator ends follow-up for that comparison. For patients followed as new users of a comparator, adding saxagliptin ends follow-up, although such patients may then qualify as new users of saxagliptin if there are remaining comparators to which they have not been exposed.

Receipt and review of death certificates in a timely manner will not be possible in the context of active surveillance, but any deaths identified from hospitalization summaries or membership/enrollment data will end follow-up. Data partners were unanimous that short enrollment gaps were typically due to administrative errors rather than genuine loss of coverage but that longer gaps often reflected true cessation of coverage such that events or prescriptions claims could be missed. Allowing a gap in medication possession recognizes that individuals on chronic medications occasionally fail to take the medication due to forgetfulness, acute illnesses, or lost medication. The gap is expressed as a proportion of previous days' supply rather than a fixed length because typical prescription sizes varied widely (from 30 to 100 days' supply) across data partners. Primary analyses will not allow for “former use” exposure categories or for resumption of follow-up when medication use resumes after a gap. Concerns that such users may differ from continuous users for unmeasured reasons as well as the challenges of aggregating such complex exposure data led to this decision. In secondary analyses planned at the end of the surveillance (see below), we will examine the effects of allowing longer gaps in possession, categories for former use, and resumption of medication use.

Outcome events

The primary outcome event is an AMI occurring during eligible follow-up time. AMI is identified as a hospital discharge with a primary or principal discharge International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) code of 410.x0 or 410.x1 or, alternatively, as any record of death occurring within 24 h of an emergency department visit for ischemic heart disease (ICD-9-CM codes 410.x0, 410.x1, 411.1, 411.8, and 413.x). No length of stay requirement will be applied for otherwise eligible hospitalizations. This hospitalization-based definition is being validated within Mini-Sentinel using standardized criteria.[12] The emergency-department-based definition is intended primarily to capture deaths from AMI occurring in an emergency department that may be missed by hospital discharge claims. It is similar to that employed in a recent study of CVD endpoints in users of rosiglitazone versus pioglitazone.[13]

For two data partners, discharge diagnoses are not classified as principal versus secondary. However, codes are conventionally listed in order of importance, and prior research efforts from these data partners have treated the first-listed code as primary. Discharge codes of 410.x2 are not included as outcome events because these specifically refer to prior rather than acute events. One validation study suggested that including these codes lowers the positive predictive value for AMI.[14]

Controlling for confounding

New users of saxagliptin may differ from those who initiate other antidiabetic agents on demographic or clinical characteristics that are also predictive of AMI and therefore confound observed associations of saxagliptin use and AMI. Many such covariates (Table 1), including age, sex, comorbid diagnoses, and use of other medications, can be found in the MSDD. These covariates were selected a priori on the basis of their availability in the MSDD and the plausibility of having an association with risk for AMI. Most are comorbidities or concomitant drugs likely to be prevalent in at least 1% of the diabetes population. Prevalent CVD diagnoses and procedures will identify patients for stratification as having CVD.

Table 1. Baseline covariates to be used to adjust surveillance analyses for possible confounding*
  • *

    Only those diagnostic codes associated with encounters (inpatient or outpatient) will be used.

  • These diagnoses will be coded distinctly for inpatient stays within prior 30 days versus earlier or as outpatient.

  • Cancer (any cancer code other than non-melanoma skin cancer)

  • §

    Presence of any of these diagnoses or procedures during baseline period will serve to place patient in stratum with CVD.

Age at first cohort entry
Residence or stay in nursing home, skilled nursing facility, or rehabilitation facility during prior year
Utilization measures during baseline year
Any hospitalization within prior 30 days
Any hospitalization during prior 31–365 days
Any emergency department visit within prior 30 days
Any emergency department visit during prior 31–365 days
Number of outpatient visits in prior year
Number of unique medications in prior year
Comorbid diagnoses during baseline year
Obesity (or weight gain)
Hyperlipidemia or lipid disorder
Cigarette smoking—based on diagnostic code
Renal insufficiency
End-stage renal disease
Asthma or chronic obstructive pulmonary disease
Peripheral neuropathy
Concurrent medications
Other antidiabetic medications (current at baseline, use during the baseline year)
Antihypertensive agents (current at baseline, use during the baseline year)
Lipid-lowering agents (current at baseline, use during the baseline year)
Prior cardiovascular disease diagnoses or procedures§
Prior acute myocardial infarction (i.e., >60 days prior)
Other ischemic heart disease diagnosis
Coronary revascularization procedures: coronary artery bypass graft or percutaneous coronary intervention
Other heart disease
Carotid revascularization procedures: endarterectomy or carotid bypass
Lower extremity amputation
Lower extremity revascularization: endarterectomy, lower extremity bypass, or amputation

The number of saxagliptin users and the number of outcomes will be small during the early phases of surveillance, making it difficult to adjust for such a large number of covariates. Exposure propensity scores (PS)[15-17] and disease risk scores (DRS)[18-20] have been used to reduce numerous covariates into single summary scores that can be used for matching or stratification to overcome this challenge. Such scores also facilitate building and analyzing study-wide datasets from data reduced at the source so that patients are no longer identifiable through their covariate vector, avoiding the need for transmission of potentially identifiable individual-level data.[21]

Both PS and DRS offer potential advantages (Table 2). It will be interesting to compare their performance in the context of active surveillance in a distributed data system. PS have the intuitive advantage of balancing the populations under assessment in a way that mimics randomized trials. They are easier to use when evaluating multiple outcomes in relation to a binary exposure and are advantageous when there is more data or externally derived knowledge for modeling the exposures of interest. The DRS is easier to use when comparing multiple types or levels of exposure with respect to a single outcome and when there are more data or externally derived knowledge on outcomes of interest. In the present context, predicting the propensity to use saxagliptin will require waiting at each data partner until sufficient saxagliptin users (i.e., more than 300) accumulate at a data partner to calculate a robust PS. By contrast, the DRS can be calculated using predictors of AMI during a baseline period before licensure of the target drug or the start of surveillance. The predictors of AMI are better understood and are expected to be more consistent across data partners (and practice settings within data partners) and over time. Nevertheless, it is possible that the availability and quality of data on some risk factors (e.g., smoking, hypertension, and hyperlipidemia) could change over time. Given these considerations, we will use and evaluate both PS and DRS approaches, and neither will be designated as primary.

Table 2. Comparative advantages of the exposure PS and the DRS for active surveillance of medical products
Advantages of the PS
  1. PS, propensity score; DRS, disease risk score.

(1) A single PS readily accommodates analyses of multiple outcomes, whereas separate DRS would be needed unless the two outcomes share predictors completely.
(2) Fixed-ratio PS matching facilitates examination of the achieved balance in covariates between comparison groups in the actual population under assessment.
(3) PS matching effectively “trims” away patients in areas of nonoverlap, which has been shown to reduce confounding.[29]
(4) If the outcome events are rare, and the target and comparator drugs are commonly used, then the PS strategy is more feasible than is the DRS strategy; it is more feasible to model—and adjust for—the confounders' associations with the exposure than their associations with the outcome.
(5) PS matching (especially if 1:1) permits transparent intuitive analyses as are frequently done in randomized controlled trials.
Advantages of the DRS
(1) A single DRS readily allows comparisons of event rates between any two or more exposures, whereas PS is typically calculated as pairwise scores for various comparators.
(2) A stable DRS can in theory be calculated before surveillance of a new agent begins, assuming that there is a sufficient period of data availability before the surveillance period and assuming further that key predictors of the outcome do not change much over time in the data, which would require monitoring. By contrast, the PS will require accumulation of sufficient numbers of users at a source after use of the new agent begins before stable models can be estimated.
(3) Predictors of drug choice are more likely than predictors of the outcome to change over time, requiring periodic recalculation of PS, although associations of covariates with outcomes will also require monitoring during the surveillance period.
(4) The major predictors of certain outcomes (e.g., acute myocardial infarction) are well established and should also be relatively similar across data sources, increasing comfort with pooling data across sources, whereas PS models are likely to be very different across data sources.
(5) Because the meaning of the DRS is clearer than that of the PS, comparisons of DRS distributions between drugs of interest, across data sources, or across time may be more readily understood than similar comparisons of PS distributions.
(6) Because the DRS reflects risk status for the outcome, subgroup analyses of drug safety by DRS level are also of interest and readily interpretable.

We considered two distinct ways to use the PS and DRS: matching and stratification. Matching (1:1) minimizes confounding by restricting comparisons to the single best match for each saxagliptin user and facilitates simple transparent bivariate analyses such as is often done with data from a randomized trial. However, 1:1 matching reduces power by discarding information from users not matched. When there are more comparators than saxagliptin users, the matching ratio could in theory be designed to be 1:N (or M:N) to increase power. However, the advantage of simplicity would be lost unless N remains fixed across matched sets. This would likely mean that N would remain small (1 or 2) because at least some saxagliptin users would have only have one or two acceptable matches and we would be reluctant to discard many saxagliptin users. Stratification is less burdensome to implement in multisite, sequential surveillance. These considerations led the working group to propose using both matching (for the PS) and stratification (by deciles) for the DRS.

Propensity score estimation and matching

Programming for summarizing the data, deriving the four PS, matching, and evaluating the balance achieved by PS matching will be written centrally and distributed to data partners. Each site will estimate PS locally and separately within subgroups defined by the presence or absence of prior CVD. All covariates shown in Table 1 will be included in each model at each site, regardless of the statistical significance of associations with AMI. Variables listed under the Prior CVD section of Table 1 will be relevant only within strata with a history of CVD. In a secondary analysis, data partners that have additional covariate information (e.g., blood pressures, body mass index, laboratory values, and race or ethnicity) will be asked to calculate additional PS that may be examined to determine whether further adjustment for covariates modifies effect estimates. Matching will use a “greedy” algorithm[22] and a maximum tolerable distance for an acceptable match of ±0.25 standard deviation of the PS on the scale of the log odds of saxagliptin use. Once a match (or matches) is made, it is not revised later in surveillance, and no comparator user is matched to more than one saxagliptin user. If any saxagliptin users cannot be matched, they will be retained for later examination but not included in PS-matched analyses.

The PS is recalculated quarterly for newly identified eligible patients. To improve reliability, each recalculation will include data on all new users identified to date at that data partner site; however, only the newly added users will be matched on the basis of the calculated PS. Previously matched users will remain linked with their original match. If covariate relationships with saxagliptin use appear to be changing substantially as new data accumulate, interaction terms of time (i.e., quarter) with covariates will be included in later PS models.

DRS calculation and stratification

The DRS reduces the dimensionality of the covariates to a single multivariable prediction score for AMI. To calculate the DRS, a cohort will be built by each data partner using a centrally written program applied to MSDD data from 2007–2008 to identify all members who meet eligibility criteria for the surveillance by having T2DM, including use of at least one antidiabetes medication. Follow-up for AMI will begin as soon as 12 months of baseline data are available for collection of covariate data, in subgroups defined by the presence or absence of a prior history of CVD, and will continue through the end of 2009. Programs for fitting and evaluating Cox proportional hazards models will also be distributed to generate DRS at each data partner. Models will again include all covariates shown in Table 1. Follow-up rules and endpoint definitions described earlier for surveillance analyses will be used. Coefficients for each covariate will be combined to estimate the relative hazard for AMI for each member of the new-user cohort. Predicted hazards (relative to a reference covariate profile) will then be ordered from lowest to highest and divided into deciles. These coefficients and decile boundaries will subsequently be used to calculate DRS and stratify all new users identified once active surveillance begins.

Primary statistical analyses

During surveillance, we will monitor rates at which new users of each of the drugs appear, accumulate follow-up, and are censored for various reasons across data partners. We will examine differences across data partners in patterns of days' supply and adherence and in numbers and crude incidence of AMI. Baseline covariate patterns will be examined by index drug and over time. Results of PS and DRS models will be compared across data partners and time.

Both PS-matched and DRS-stratified analyses will employ stratified Cox proportional hazards models in primary analyses that estimate adjusted hazard ratios and 95% confidence intervals. Each stratified Cox model specifies the hazard ratio, summarized over all of the strata and times of outcome events, as a function of a binary indicator of saxagliptin use, an unspecified baseline hazard, and no additional covariates. Covariate adjustment is accomplished either by the PS matching or the DRS stratification. Time is measured in days relative to the first prescription until an AMI or censoring occurs. With PS matching, each risk set includes everybody in a given data partner who is uncensored and under observation on a day when there was at least one AMI in the population under assessment; the risk sets are stratified by data partner, quarter of cohort entry, quarter of follow-up, and by presence or absence of CVD (at baseline). With DRS stratification, the risk sets are additionally stratified by DRS level (decile).

Cox models are powerful, flexible, and widely used in cohort studies when the outcome is binary and there is varying length of follow-up and rates of censoring. Both PS matching and DRS stratification permit use of stratified Cox models that can be easily fit to risk set-level data restricted to the informative risk sets (i.e., those with one or more outcome event and one or more new users of each of the drugs). Only one record of data per risk set is required—this record summarizes the defining characteristics of the risk set and the numbers of its saxagliptin users and comparator users with and without an outcome event. There is no need for individual-level data.[23]

Surveillance is planned to include up to 10 sets of sequential analyses as the MSDD is updated periodically. The first interim analysis will use all data available as of early 2011. Thereafter, up to nine additional sets of analyses will be done sequentially after incorporating an additional quarter's data from all participating data partners. To limit chances of a Type I error, we developed an alpha spending plan following the approach of Lan and DeMets.[24] We specify a nominal p-value threshold at each analysis that keeps the overall chances of a Type I error to 5% across 10 sequential analyses. (This threshold can be updated—consistent with our alpha spending plan—if we need to revise the number of planned interim analyses.) To estimate the magnitude of relative risk for AMI that is detectable by the planned sequential surveillance and to determine the appropriate p-value threshold for up to 10 sequential analyses, we used simulations based on observed AMI event rates, which averaged 9 per thousand person years in preliminary data for patients with diabetes from the four data partners and the following assumptions: (i) 0.25% overall prevalence of new use of saxagliptin by the end of the first surveillance period; (ii) in the final quarter of the initial period, accrual as new users of saxagliptin would be equivalent to 0.1% of the diabetes population; (iii) total numbers of new users of saxagliptin would thereafter increase by approximately 15% per quarter; and (iv) the average follow-up time on saxagliptin and the comparators will be 6 months per new user. These assumptions predict 46 000 new users of saxagliptin, 23 000 person-years of follow-up, and 20–60 new AMI events per quarter for each pairwise comparison. Given these assumptions, simulations indicated that a nominal p-value threshold of 0.0144 at each analysis would be required. Our simulations were similar to that proposed by Li and Kulldorff for a conditional sequential sampling procedure.[25]

This alpha spending plan, like other “flat-boundary” sequential designs (see Kulldorff et al[26] and also Pocock[27]), keeps the nominal threshold for signaling constant at all times when sequential tests are done. In effect, it “spends” more alpha early in surveillance than do approaches commonly used in randomized trials (e.g., O'Brien-Fleming[28]) that “conserve” more alpha for a final analysis. This approach was felt to be more transparent and more appropriate for surveillance, where there is a priority for timely signaling of possible safety concerns. The overall power of the proposed design to detect relative risks of 1.25, 1.33, 1.4, and 1.5 by the final analysis is 0.61, 0.81, 0.91, and 0.98, respectively. If monitoring suggests that saxagliptin uptake, average length of follow-up, or event rates are falling substantially short of assumptions, the FDA will be notified immediately of the consequences for power. Some lengthening of the surveillance period, the interval or number of sequential analyses, or other surveillance design adaptations may be deemed acceptable if it appears that adequate numbers can eventually be reached.

Secondary statistical analyses

In secondary analyses, Poisson regression models will be fit to aggregated data summarizing the experience of the new-user cohorts. Outcome events and follow-up time will be aggregated in strata defined by data partner, time since starting drug, calendar period, and—for DRS stratified cohorts—the decile of the DRS. Poisson regression is more intuitive than are Cox models to some researchers and yields explicit estimates of AMI incidence as well as incidence rate ratios associated with saxagliptin use. Thus, Mini-Sentinel can gain experience conducting surveillance with Poisson regression and Cox regression in the context of sequential analyses and the distributed database (with constraints on pooling patient-level data). We expect Cox regression and Poisson regression to yield similar results. Results would be virtually identical if ratios of follow-up time in the sites, subgroups, and time periods specified by the Poisson model are the same as the ratios of saxagliptin users to comparator users in the risk sets of the Cox model. If Cox regression and Poisson regression yield estimates that differ nontrivially, then the Cox model is less vulnerable to bias because its risk sets are anchored to specific time points, which should make them more homogeneous in level of risk than when the risk sets are anchored to the stratified time periods used in Poisson regression.

The protocol proposes a number of additional secondary analyses to be conducted at the end of the surveillance. These secondary analyses will compare the final risk estimates obtained using the analytic approaches described earlier for risk-set-level data with those obtained using standard multivariable methods in individual-level data. These individual-level analyses will be done separately at each data partner, and then results will be combined using meta-analysis. Individual-level analyses will also allow exploration of the influence of specific covariates on exposure–outcome associations in terms of both confounding and possible effect modification. We will also examine other approaches to modeling exposure, including lengthening the allowable gap in medication possession, inclusion of follow-up if exposure is resumed, and long-term follow-up for AMI after an initial period of exposure that ceases (i.e., intent-to-treat analyses). At present, there are no plans to pool deidentified individual-level data across data partners. Finally, the results of our primary and secondary analyses will be compared with results—if available—from the ongoing postmarket randomized trial conducted by saxagliptin's manufacturer.


  1. Top of page

We describe an analytic approach to active surveillance for the safety of a newly approved pharmaceutical agent that will use data on the populations of four or more data partners within the Mini-Sentinel pilot. This project introduces challenges not faced in most observational studies but are likely to be typical of future surveillance projects of the Sentinel Initiative. A desire to keep most or all individual-level data within the original data partner environment creates a need for multivariable analytic approaches based primarily on aggregated data. This and the probability that medication prescribing patterns and possibly AMI risk will differ across populations require that each data partner calculate its own summary confounder scores and perform matching or stratification. Central generation of programs and oversight will be employed. The gradual accumulation of data and endpoints coupled with a need to detect a safety signal as soon as possible necessitate both data reduction, using either a PS or DRS approach, and a statistical plan for sequential hypothesis testing. Uncertainty regarding the rate of uptake of saxagliptin use in these health plan populations makes it difficult to definitively specify the duration and power of surveillance. An important limitation of the planned surveillance is that a number of potential confounders—including smoking, obesity, race or ethnicity, duration of diabetes, and assessments of blood pressure, lipids, and hemoglobin A1c—are not yet consistently measured and available throughout the MSDD, although data on these risk factors may become available in the MSDD for future surveillance.

Novel aspects of the analytic work led us to propose simultaneous assessment of alternative approaches to both data reduction and analysis. We will compare the ease of implementation and precision of PS and DRS approaches for summarizing covariate data. We will compare 1:1 matching with stratification by the summary scores. We will evaluate the relative utility of Cox proportional hazards regression and Poisson regression approaches for estimating the relative risk with adjustment for potential confounders. Ultimately, we hope to compare each of these approaches with multivariable analyses conducted with individual-level data.

We believe that this initial active surveillance project of the Mini-Sentinel will provide important insights that will inform the design and conduct of future similar assessments of medical product safety.


  1. Top of page

The authors have no conflict of interest.


  • A new-user cohort design is planned for quarterly surveillance of AMI in users of saxagliptin versus users of four comparator anti-diabetes medications.
  • Several methods of adjustment for confounding will be compared, including matching by propensity score and stratification by AMI risk score.
  • Stratified Cox regression models will be fit to data on risk sets—patients similarly at risk on the day of an AMI; there is no need for pooling patient-level data which remains distributed across Mini-Sentinel's partner organizations.


  1. Top of page

The authors thank Sebastian Schneeweiss for contributions to the design of the proposed surveillance and Nicolas Beaulieu for overseeing and programming the collection of preliminary data from participating Mini-Sentinel data partners. Mini-Sentinel is funded by the FDA through the Department of Health and Human Services Contract Number HHSF223200910006I.


  1. Top of page