Do drug‐related safety warnings have the expected impact on drug therapy? A systematic review

The need for drug‐related safety warnings is undisputed, and their impact should also be evaluated. This systematic review investigates and assesses the impact of safety warnings on drug therapy.

The need for safety warnings as a part of pharmacovigilance processes is undisputed. Safety warnings should contribute to minimising risks in drug therapy. Assessing their effectiveness is essential to establish whether warnings have been effective or not, and if not, why and which further interventions are required. 3 The impact/effectiveness can be assessed for example, based on changes in drug use or drug monitoring. As mentioned in Module XVI of the GVP "The evaluation of effectiveness should facilitate early corrective actions if needed and may require modifications over time. It is recognised that this is an evolving area of medical sciences with no universally agreed standards and approaches." 3 To date, the impact of drug-related safety warnings has been examined by three systematic reviews. [5][6][7] A comparison of the most important aspects of these three systematic reviews is presented in Data S4. In summary, the search period for the review by Piening et al 5

| METHODS
In order to answer the question "Do drug-related safety warnings have the expected impact on drug therapy?" a systematic literature search and an assessment of the included studies have been carried out. Our review includes studies investigating the impacts of drugrelated safety warnings on drug therapy. The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) 8 were followed.
The study protocol was registered in Prospero under record number CRD42018084154 and is available at https://www.crd.york.ac. uk/prospero/display_record.php?RecordID=84154

| Eligibility criteria
The inclusion and exclusion criteria defined a priori were based on Participant, Intervention, Comparison, Outcome, and Study type (PICOS scheme) and were as follows.

| Participants
Outpatients and/or inpatients (of all ages) undergoing drug therapy using prescription substances or substance groups worldwide. Animal studies were excluded.

| Interventions
Information letters with safety warnings on prescription substances or substance groups addressed to health care professionals, for example, Because of potential bias, studies were excluded that defined as intervention the marketing suspension or the withdrawal of the marketing authorisation, or additional demands in a safety warning, such as a patient's declaration of consent or the mandatory use of checklists prior to prescription.

| Comparison
Possible comparison groups were similar regions, or comparable substances or substance groups. For randomised controlled trials (RCTs), a comparison group is essential by definition. However, the existence of comparative groups was not defined as an inclusion criterion for the included case-control studies and cohort studies with a before/ after design.

| Outcome
Outcome was defined as change, partial change, or unchanged continuation of the drug therapy following the safety warning on the concerned human drug or following a supplementary requirement by the competent authority concerning the dissemination of the safety warning information. "Change" meaning the safety warning led to the expected impact following the investigated safety warning. "Partial change" was defined as only a part of the safety warning was realised, and "unchanged" means that the safety warning did not lead to a change in drug therapy. These results were categorised in terms of effectiveness in, firstly, effectiveness of intervention according to the author of the study, yes (1), ie, the safety warning led to a change in drug therapy. The impact of the investigated safety warning on drug therapy was being effective. Secondly, effectiveness of intervention, no (0), ie, no change in drug therapy after the safety warning (no impact). Thirdly, effectiveness of intervention, partial (2), means that not all contents of the investigated safety warning were implemented, but only parts of it (partial implementation). If multiple safety warnings on the same topic were examined in one study and the contents of one safety warning were implemented, but not the contents of another, these results were also grouped as partially.
The changes in drug therapy may be intended or unintended. In line with GVP modules XV and XVI, 3 "intended" means that the possible changes occurring in correspondence with the safety warnings are usually drug-related and otherwise without negative consequences for the patient. As described above, their effectiveness can be derived from the change in drug therapy. "Unintended" means that the changes do not represent the contents of the safety warnings but are affected by them. The effects are not drug-related and associated with negative consequences for the patients. Therefore, studies investigating unintended effects cannot be grouped according to the effectiveness of the intervention. Figure 1 demonstrates a possible approach to systematically assess the impact/efficacy of drug-related safety warnings. An example for an unintended change is the increase in attempted suicides linked with the safety warnings on antidepressants in adolescents and young adults. 9 The safety warnings investigated informed about the increased risk of suicide in young people taking antidepressants. The authors hypothesised "decreasing rates of overall antidepressant treatment after the warnings would be associated with a net increase in suicide attempts among young people." The study results confirmed the hypothesis for adolescents and young adults. Another example, the "pill scare," caused an increase in abortions in young women in Norway. 10 The use of oral contraceptives dropped massively after a warning informed about an increased risk for adverse vascular events in users of special third-generation oral contraceptives. Subsequently, the rate of abortions increased.
Whether a safety warning led, did not lead, or partially led to an intended or unintended change in drug therapy was derived from the results presented by the authors.

KEY POINTS
• Drug-related safety warnings influence drug therapy; their impact can be intended or unintended.
• Safety warnings regarding psychotropic drugs seem to influence drug therapy less than safety warnings on other substance groups.
• The included studies are of broadly varying methodological quality. Standardised reporting and assessment procedure are desirable in order to investigate the effectiveness of drug-related safety warnings more specifically.

| Information sources and search
We carried out a systematic electronic literature search using the databases MEDLINE and EMBASE (via the Ovid interface) for the period from January 1, 1998, to December 12, 2017. This search was updated on January 12, 2019. For this purpose, a search string with keywords corresponding to the research questions and related to the above mentioned PICOS criteria to find studies published since 1998, MESH terms and free-text search were used. The search strategy for MEDLINE is available as Data S1. During manual search, references of included studies (backward citation tracking) and relevant reviews [4][5][6] were screened to identify further relevant studies. In addition, the Federal Institute for Drugs and Medical Devices (BfArM), the EMA, and the FDA websites were searched for relevant studies.

| Study selection and data extraction
Taking into account the eligibility criteria of this systematic review, two researchers (UG and JL) independently selected relevant articles based on titles and abstracts. Following this, two researchers (UG and JL) independently assessed the full-text articles of the preselected potentially relevant articles for their inclusion with disagreements being discussed between both reviewers. Where disagreements could not be resolved, a third reviewer (JS) was consulted to enable a decision.  I G U R E 1 Approach to systematically assess the impact/efficacy of drug-related safety warnings (e.g. 0.035 rather than <0.05) for the main outcomes except where the probability value is less than 0.001?") was answered with "yes" in case confidence intervals were specified. Item 27 ("Did the study have sufficient power to detect a clinically important effect where the probability value for a difference being due to chance is less than 5%?") was supplemented by the question whether a priory sample size planning had been carried out. The items of this checklist were answered either with "yes," "no," "unable to determine," or "not applicable." A numerical evaluation of these items was not performed, because the rating of individual items would not or not sufficiently be represented in the overall assessment of the study. 15 "The Cochrane Collaboration's tool for assessing risk of bias in randomised trials" 16 is to be used to assess RCTs. Two researchers (UG and JL) independently assessed the methodological quality of the studies using the described instruments. Divergent assessments were discussed until consensus could be reached or mediated by a third reviewer (TD).
Besides that, a qualitative synthesis based on results of the included studies was performed as follows: • The implementation of a safety warning (effectiveness) was grouped based on the authors' findings into a safety warning to prescription drugs "lead," "do not lead," or "partially lead" to an "intended or unintended" change in drug therapy.
• Based on the contents of investigated safety warnings, the safety warnings were categorised into new adverse reactions, indication limitations, new contraindications, dosage changes, additional laboratory monitoring, and new interactions. Potential patterns are to be deduced from the clustering.
• Based on the investigated substances or substance groups, the substances or substance groups were clustered into "psychotropic drugs" and "others." Potential patterns are to be deduced from the clustering.
In the presentation of possible patterns, the designs as well as the risk of a bias evaluation (items 14 to 26 of the checklist by Downs and Black) of the included studies are to be incorporated. An assessment of the risk of bias across studies was not planned. In order to minimise the selection of bias, a broad search is conducted as described under

| Study selection
After removing all duplicates, electronic searches resulted in the screening of 6423 titles and abstracts and 146 full texts. Additionally, 28 relevant papers were identified by manual search. A flow chart illustrating the entire study selection process is given in Figure 1. While 70 publications met our inclusion criteria, 76 studies were excluded mainly because of a lack of examined impact on the drug therapy and missing analysis of safety warnings or nonapplicable study types. A list of the studies excluded after full-text screening with reasons for their exclusion is available from the corresponding author ( Figure 2).
All of the 70 publications included 9,17-85 are observational studies based on routinely collected health data, which included health insurance or hospital records. Thus, no RCTs were found for the defined search period.
Two publications 25,33 examined safety warnings for two mutually independent substances each, that is, 70 publications reported 72 studies. If a study contained multiple characteristics, the characteristics of these studies were counted multiple times in the following analyses, for example, studies examining more than one safety warn-

| Study characteristics
General study characteristics are presented in Table 1

| Contents of intervention
The contents of the safety warnings were divided into six catego-  Unintended effects of a safety warning were examined in six out of 72 studies. 9,28,60,72,75,80 These six studies could not be analysed regarding the effectiveness of safety warnings, because an unintended effect occurred in connection with a safety warning, but not as a direct consequence of the content. One example of this was the increased rate of hospitalisations due to depression after the recommended dosage limit (content of the safety warning) of citalopram in adults was implemented. 60 After clustering the substances into a "psychotropic drugs" group and "others" group, a pattern was derived between the investigated substance and the effectiveness of a safety warning. As mentioned above, 47% (34)  concerning psychotropic drugs, and 53% (38) studied other substances and substance groups.

| Study quality
The reporting quality of the included intervention studies was assessed on the basis of "The Strengthening the reporting of observational studies in epidemiology (STROBE) explanation and elaboration" 11  closer patient monitoring after starting the therapy, respectively, were not implemented. But also unintended effects may occur in connection with drug-related safety warnings. One example is the decline in depressions diagnosed in patients <18 following warnings on suicidal ideation and suicide attempts in connection with the intake of paroxetine and other selective serotonin reuptake inhibitors except fluoxetine. 80 This contradicted the intention of safety warnings to make drug therapy safer. A risk assessment of safety warnings in terms of wording as well as interpretation possibilities would be conceivable in order to detect possible unintended effects in advance.
Secondly, a difference between the groups "psychotropic drugs" and "others" should be assumed regarding the effectiveness as outlined by the authors. Safety warnings regarding psychotropic drugs seem to influence drug therapy less than safety warnings on other substance groups. Compared with 53% of the studies in the "others" group, only a total of 23% of the studies in the "psychotropic drugs" group described the impact on drug therapy as effective. Especially the study design but as well the risk of bias assessment seems to support these two points. In the "psychotropic drugs" group, 53% of the studies used an ITS design as compared with 34% in the "others" group. Equally, comparison groups (35% vs 24%) were used more frequently in the studies on "psychotropic drugs." Moreover, 26% of the most important outcome measures were not correctly represented in the "psychotropic drugs" group and 32% in the "others" group. Some items of the risk of bias assessment could not be answered (items 14, 15, 19, 24, and 26)

| Strengths and limitations
The strength of this review is that it provides a current and compre-    publication timing of the safety warning in the respective countries. It seems difficult to adjust for these factors. Safety warnings on drugs represent only one of many sources of information for health care professionals so that influences from other sources cannot be ruled out. Moreover, there are indications that the credibility of a safety warning is already influenced by the sender. 87 In addition, a study by Bjerre et al 88 showed that safety warnings concerning one and the same substance were published at different times and in different formats in the United States, in Canada, and the United Kingdom, and that the information provided in the safety warnings failed to be consistent at all times.
Furthermore, this review is limited to English publications, so a language bias cannot be ruled out. In addition, more than half of the studies included were published in the United States. Therefore, the presentation of the results has a clear focus on US American data.
In summary, our review raises a number of questions: How can the effects of safety warnings on drug therapy be investigated independently of, for example, guidelines or media influences? What causes a change, no change, or partial change in drug therapy as a result of a safety warning? What leads to unintended changes in drug therapy? Which parameters influence the implementation of safety warnings from "psychotropic drugs" or "other" drugs? Do the parameters differ depending on the group of substances studied? The importance of evaluating the effectiveness of safety warnings is undisputed. It can be assumed that answers to these questions can contribute to an optimisation of the pharmacovigilance process.

| Conclusion
Out of 72 included studies, the impact of drug-related safety warnings on drug therapy was shown by a large body of evidence. Safety warnings lead, do not lead, or partially lead to an intended or unintended change in drug therapy. Furthermore, safety warnings regarding psychotropic drugs seem to influence drug therapy less than safety warnings on other substance groups. The included studies investigating the impact of safety warnings are of broadly varying methodological quality. In order to be able to investigate the effectiveness of safety warnings on drug therapy, a uniform evaluation procedure is desirable.
Further research is needed to clarify which parameters influence the impact of safety warnings to what extent.