Study designs used to assess the effectiveness of risk minimization interventions
Study designs used to assess each component of risk minimization (knowledge, attitudes, and behavior) are shown in Table 1. It should be noted that the effectiveness of a given RMI could be evaluated through more than one study. For example, the effectiveness of safety warnings on antidepressants issued by the FDA was evaluated in six studies published in the literature.[17, 20-24] Also, six studies evaluating label changes and Dear Health Care Professional (DHCP) letters on cisapride were found.[25-30]
Table 1. Distribution of study designs used to assess the various components of risk minimization interventions
| || ||Knowledge||Attitudes||Behavior||Clinical outcome (adverse event)|
|Randomized controlled trials|| ||0||1||1||4|
|Interrupted time series analysis||With comparison||0||0||2||1|
|Pre–post intervention||With comparison||0||0||3||4|
|Cohort studies/registry||With comparison||0||0||1||1|
|Cross-sectional studies|| ||1||0||4||1|
|Qualitative assessment|| ||3||1||1||0|
The great majority of studies that evaluated behavior used interrupted time series analyses or pre–post intervention designs without a comparison group, often using data from administrative claims databases. Given the availability of historical data (i.e., pre-intervention), most of the RMIs involving safety warnings or DHCP letters were evaluated using time trends in prescribing or laboratory test monitoring. We have identified only six studies that used a parallel comparison group. Such comparison groups were either randomized or consisted of other geographical locations (states) or medical centers where the RMI was not implemented,[33, 34] another drug not targeted by the intervention, or young adults when children or adolescents were the target of the intervention. In another study in the absence of a reference population, all external factors, such as publicity and media coverage, which could also influence prescribing trends in addition to the DHCP letter, were qualitatively assessed. Restricted distribution programs, generally implemented at the time of product approval, tended to be evaluated through cohort studies.[35, 37, 38]
Knowledge and attitudes were most often evaluated using surveys or qualitative assessments,[17, 42, 75, 76] whereas a randomized trial or pre–post intervention without a comparison group was rarely used. As shown in Table 1, among the 16 studies that evaluated the impact of the RMI on the occurrence of AEs, robust designs such as RCTs[33, 34, 40, 41] or comparative designs (n = 6) were often used.
Measures of effectiveness, or success, of RMIs included prescription or dispensing rate (n = 18, 27.6%), frequency of AEs (n = 16, 24.6%), including pregnancy rate during isotretinoin or thalidomide treatment, contraindicated co-prescription rate (n = 10, 15.4%), laboratory testing rate (n = 7, 10.8%), knowledge retention and attitude toward the RMI (n = 7, 10.8%), changes in medical practice (n = 6, 9.2%), rate of treatment initiation (n = 4, 6.2%), awareness of RMI (n = 4, 6.2%), rate of use of treatment alternatives (n = 3, 4.6%). Several studies evaluated the effectiveness of the RMI with more than one outcome.[14, 15, 17, 18, 36, 42-44]
Of the 41 RMIs consisting of education, 31 used prescription rate or contraindicated co-prescription rate as a measure of effectiveness. Using the information matrix included in the Supporting Information, we have ascertained the aim of the intervention and the measure of effectiveness, from which we have assessed concordance. In nearly one third (n = 18) of the studies, the effectiveness measure did not correspond to the aim of the intervention. Inconsistencies originated from the following: prescription rates were used as measures of effectiveness while the RMI aimed at preventing the occurrence of AEs (n = 10),[20-24, 45-49] or the RMI sought to enhance laboratory testing (n = 2).[50, 51] In other instances, occurrence of AEs was used as the effectiveness measure while the intervention aimed at avoiding concomitant use of contraindicated medications (n = 2),[52, 53] at increasing laboratory testing (n = 2),[54, 55] or at enhancing information disclosure by healthcare professionals (n = 1).
For studies that involved surveys, the definition of RMI effectiveness was not explicit, consisting mainly of a change in practice (change in medication dosage or drug, referral of patients to a specialist, performance of baseline assessment before initiation of a medical treatment, etc.) (n = 5),[14, 15, 17, 18, 42] knowledge retention (n = 5),[11, 19, 42, 43, 52] awareness of RMIs (n = 5),[14, 15, 17, 18, 57] and increase of disclosure of information (n = 1). In cases where effectiveness was defined as knowledge retention, there was no specification on what represented adequate knowledge retention or what information should the target population have retained.[19, 43, 52]
Main data sources used to ascertain effectiveness outcomes included claims databases (n = 34, 52.3%), questionnaires (n = 19, 29.2%), medical charts or electronic health records (n = 9, 13.8%), and clinical data or laboratory findings (n = 4, 6.2%). Some studies have evaluated the effect of an RMI on the initiation of new treatments (i.e., incident drug users),[20, 23, 46, 48, 59, 60] whereas others focused on the prescribing rate for all users (incident and prevalent).[13, 21, 24, 36, 41, 45, 49, 53, 61-65, 77] Some studies defined new use as the absence of a prescription record during the preceding 4 months[23, 48] or 12 months.[20, 46] Among studies that examined the effectiveness of an RMI on the rate of concomitant use of contraindicated medications, concomitant use was most frequently defined as two contraindicated products with active prescriptions that overlap on at least one day.[15, 26, 27, 29, 30, 58, 66]
Three surveys cited in the literature targeted different healthcare professionals, namely physicians, nurses, pharmacist, and social workers.[11, 12, 67] Two surveys targeted only pharmacists,[18, 19] and two surveys only specialists, pediatricians, and neurologists. The completion rate for the surveys ranged from 50% to 100%. All surveys were comprised of multiple-choice questions. One survey utilized a scale to rate the healthcare professionals' attitude toward RMI, and one collected doctors' comments.
Results of assessment of risk minimization interventions
Among the studies reviewed, RMIs appeared to be successful or partly successful (n = 40, 61.5%) or 15 (23.1%) were not, and for nine studies (13.8%), it was not possible to draw a conclusion regarding their effectiveness because either the study was descriptive, had no comparison group, or had no threshold for effectiveness defined. After the implementation of the RMI, consisting mainly of warnings or DHCP letters, studies found a decrease in prescribing or co-prescribing rate of the study drug (n = 24),[16, 20, 21, 23-30, 40, 41, 44-46, 49, 51, 54, 62-64, 66, 68] a decrease in frequency of AEs (n = 10),[33, 34, 54, 55, 60, 61, 69-71, 78] a decrease in pregnancy rate while on thalidomide (n = 1), an increase in laboratory testing rate (n = 6),[31, 35, 44, 50, 65, 72] or a change in medication dosage (n = 1).
When successive communication interventions were implemented over time,[24, 26-31, 49, 51, 62, 63, 72] the first ones were often found unsuccessful (e.g., not reaching the expected decrease in prescription rate), whereas the latter interventions seemed more effective.[26-28, 30, 50] Wilkinson et al. showed that multiple safety alerts are required to yield a significant decrease in the co-prescription rate of cisapride and contraindicated drugs.