• incremental benefits;
  • multicriteria decision analysis;
  • QALYs;
  • RVALYs;
  • benefit risk


  1. Top of page
  2. Introduction
  3. Multicriteria Decision Analysis
  4. Closing Remarks
  5. References

New medicines have therapeutic benefits, but also are potentially harmful for subgroups of patients. Longer-term safety issues often only become apparent during postmarketing surveillance and have in some cases led to key decisions to withdraw products over safety concerns, e.g., Vioxx and Lotronex, the latter subsequently being made available again in response to patient pressure through a restricted access drug distribution program. The issue of harms and benefits is very visible to licensing bodies and the public; however, decision-making in this context lacks transparency and consistency, and appears to neglect the application of modeling and other techniques that have proven useful in cost-effectiveness analyses to address uncertainty around outcomes similar to those used in benefit–harm analyses. Indeed, as this field develops, it is becoming clear that benefit–harm analysis holds great potential as a tool for informing decision-making by both regulatory and health technology assessment bodies that are faced with an increasing number of situations in which there is a trade-off between positive and negative health effects. Developments in this field are discussed. We follow the convention of referring to these trade-offs as “benefit risk analysis,” although we share Aaronson's view [1] that benefit–harm is a more appropriate description.

Trading Benefits and Risks

Methods for quantifying incremental benefits and risks have been described in the literature and reviewed by Cross and Garrison [2] and by Mussen et al. [3]. We focus on three. In the first example discussed here, Briggs and Levy [4] use data collected by Lynd and O’Brien [5] to present a very simple way of how one might trade-off benefits and risks of two interventions, unfractionated heparin and enoxaparin, for the treatment of deep vein thrombosis (DVT) after major trauma. The basic assumption is that as the numbers of patients in the two arms of the trial are broadly equivalent, the absolute numbers of events that occurred can be compared to assess whether the extra benefits are worth the extra risks. One product is producing more benefits, but also potentially more risks (Table 1).

Table 1.  Summary of events from a clinical trial of low-dose unfractionated heparin versus enoxaparin for the prevention of deep vein thrombosis after major trauma
EventNumbers experiencing an event*
Unfractionated heparinEnoxaparinDifference
  • *

    Focus on numbers (rather than probabilities) ignores the slight imbalance between arms of the trial low-dose unfractionated heparin, n = 136; enoxaparin, n = 129).

  • Negative numbers are events avoided, positive numbers are excess events [5].

  • DVT, deep vein thrombosis.

 Distal DVT4032−8
 Proximal DVT208−12
 Major bleeds15+4

Is that a good trade-off? When these positive and negative effects are placed on a cost-effectiveness plane that illustrates the joint distribution of risk of a major bleed and the probability of a reduction in DVT (Fig. 1), the incremental risk–benefit ratio is approximately one-third; in other words, if the benefit of averting a proximal DVT is valued at one-third or more than the consequences of an adverse event, then the technology should be used. And so, this represents a simple quantitative approach to defining a risk–benefit acceptability threshold. The decision-maker has to decide the positive and negative values, respectively, of the benefit “event” and the risk “event” and his or her willingness to trade one for the other.


Figure 1. Proximal deep vein thrombosis (DVT) versus major bleed: (µ = threshold ratio) shown as a joint distribution of risk and benefit on a cost-effectiveness plane. Results of the second-order Monte Carlo simulation of the DVT trial plotted on the risk–benefit plane: the incremental probability of a DVT (ΔB) versus the incremental probability of major bleed (ΔR). Lines extending from the origin into the NE quadrant represent different risk–benefit threshold values (µ). The • marks the point estimate of the risk–benefit ratio (if DVTs and major bleeds are valued equally, the risk–benefit ratio = 1) [5].

Download figure to PowerPoint

The second approach uses quality-adjusted life-years (QALYs) to create a “common currency” for comparing benefits and risks. The simplest version of this approach involves adding up positive and negative QALYs [6]. As shown in Table 2, the means of the expected benefits and the expected negative effects (measured in QALYs) were netted to generate an overall population effect in terms of the incremental net health benefit or potential net benefit of the drug. In this case, the authors positioned the article to demonstrate the potential of using value of information approaches within a licensing context (i.e., to consider when safety data should be collected alongside use of the product, rather than in advance of its use) and the merit of considering QALYs as an appropriate measure of absolute health effects. The authors noted that although they had combined positive and negative QALYs on a 1:1 basis in their example, there was no need for decision-makers to do this. Negative QALYs could be given a higher weight if licensing bodies attached a higher value to avoiding adverse effects than to achieving health gain. Nevertheless, such a judgment would be explicit.

Table 2.  Estimated aggregate incremental net health benefits [6])
  1. Note: Hypothetical assumptions and calculations. Numbers in parentheses are negative quantities.

  2. INHB, incremental net health benefit; QALY, quality-adjusted life-year.

Expected benefit per treated patient (QALY)0.200.05–0.36
Expected risks (harms) per treated patient (QALY)(0.05)(0.08)–(0.03)
Difference: INHB (net QALY per patient)0.15(0.03)–0.33
Population parameters  
 Total: 300,000,000  
 Annual cohort entering treatment: 150,000  
Aggregate annual INHB  
 Benefit (in QALYs)30,0007,500–54,000
 Risk (in QALYs)(7,500)(12,000)–(4,500)
 Difference (annual INHB)22,500(4,500)–49,500

Another study using QALYs to inform benefit–risk assessments was undertaken by Lynd [7] in a simulation analysis of Vioxx compared to naproxen. The first step was to attach negative (“disutility”) incremental QALY values for, and durations to, the events that might occur, ranging from dyspepsia to mycardial infarction. A discreet event simulation of 10,000 patients then produced the numbers of the different types of adverse events that occurred in each treatment arm. Focusing only on side effects, the simulation demonstrated that, on the assumption that the drugs were of equal efficacy, there was a 96% chance that the incremental net benefit of Vioxx as measured by QALYs was positive relative to the use of naproxen, calling into question whether the evidence that led to the removal of Vioxx from the market had been put into an appropriate context. Again, decision-makers may choose to give different weights to QALYs arising from different side effects. Such judgments can be made explicitly.

A variant of the use of QALYs to inform benefit–risk assessments is the use of “relative value adjusted life years” or RVALYs. In a second analysis, Lynd et al. [8] quantified incremental net health benefits of alosetron, originally indicated for irritable bowel syndrome (IBS), withdrawn from the market, and subsequently re-introduced with a restricted indication, using preference weights derived using conjoint analysis, generating a measure of health benefit that the authors called RVALYs. The preference weights (relative values of different benefits and side effects) were obtained from 565 IBS patients using a discrete choice experiment. By combing the preferences attached to the events with the probabilities that these events might occur, according to the original clinical trial data, they were able to calculate a net effect of 34.1 RVALYs gained per 1000 patients relative to placebo, comprising 34.9 RVALYs of benefit and −0.8 RVALYs of disutility. A probability sensitivity analysis indicated that if positive and negative RVALYs were regarded as equally desirable by decision-makers (a 1:1 trade-off), then there was a 99% chance of alosteron having a positive incremental net benefit. The RVALY approach of combining negative and positive health effects showed on the available data and a 1:1 trade-off that there is an overall positive health effect associated with using this intervention. Again, decision-makers could treat some RVALYs as more important than others and use weightings other than a 1:1 trade-off. This framework would enable such decisions to be explicit and the weightings to be transparent.

The third method is to inform decision-makers about patients’ willingness to trade the benefits and risks associated with treatment. This goes beyond the patient preferences incorporated in the RVALY variant of QALYs we considered previously to focus only on patient willingness to trade real risks against clinically realistic improvements in health. Not unsurprisingly we find that patients who can expect to gain the most are more willing to put up with the risk of a serious adverse event. Johnson et al. [9] use conjoint or stated preference techniques to evaluate women's willingness to accept adverse events from the use of hormone therapy in return for a reduction in vasomotor symptoms. In Figure 2, we see the maximum acceptable risks for heart attack by treatment benefit and it is shown that those who can gain most are most willing to accept the risk of heart attack. This study is consistent with accumulating evidence associating subgroups of patients expecting larger anticipated benefit with increasing maximum acceptable risk of a given side effect. So, despite adverse events occurring fairly consistently across all patient groups, some sub-groups will get more benefit from the treatment than others, and from the perspective of a licensing decision, it is those patients for whom the product should be made available. Again, decision-makers can attach their own values to potential benefits and risks. Nevertheless, an understanding of patient perception of benefit and risk is likely to assist in any such weighting by decision-makers.


Figure 2. Maximum acceptable risks for heart attack by treatment benefit [9]. The horizontal broken line corresponds to absolute rates of adverse events in the treatment group, as initially reported by the Women's Health Institute trial investigators.

Download figure to PowerPoint

Multicriteria Decision Analysis

  1. Top of page
  2. Introduction
  3. Multicriteria Decision Analysis
  4. Closing Remarks
  5. References

Finally, to multi-criteria decision analysis (MCDA), which offers a mechanism to structure decision-making such that it is very clear how the decision was made [3]? This is not a method to bring quantitative information inside from outside as with our three methods discussed previously. Rather, this technique helps decision-makers be more explicit in their own deliberations, to understand the importance (high or low) they are attaching to particular factors. This enables them to: better understand which factors are driving their decisions; change their thinking if the results do not make intuitive sense to them when their implicit assumptions are made clear to them; and explain to the public the basis on which decisions have been made. It is particularly important in regulatory review or health technology assessment, as it lends consistency and transparency, which is often lacking, in cases where there are negative and positive health effects. Of course, there may be concerns that the use of MCDA will make decision-making too complex or too mechanistic, removing the element of deliberation [10]. In adopting this approach, regulatory review committees and health technology assessment committees would be advised to include modeling experts who can guide the application of models that mathematically analyze complex trade-offs between different criteria and that are necessary to provide structure to the deliberative decision-making process that is required. The objective, however, is not to replace the judgment of experts but to make it better informed, more considered, more likely to be “right” and, whatever decision is made, easier to explain to patients, clinicians, payers, and pharmaceutical companies.

Closing Remarks

  1. Top of page
  2. Introduction
  3. Multicriteria Decision Analysis
  4. Closing Remarks
  5. References

We are now at a point where decision support tools and techniques that incorporate combined positive and negative health effects, and weighting systems based on absolute measures of health have demonstrated utility in informing regulatory decision-making. These tools potentially offer decision-makers an opportunity to be more consistent and transparent. Nevertheless, decision-makers themselves must understand and accept the risk–benefit analysis techniques that are being applied to generate these analyses if they are to accept or reject assumptions that support the use of an intervention in specific circumstances. On a cautionary note, it should be recognized that this field is in its infancy and many more studies of original data are required to ascertain how benefit–risk analyses can lead to better health-care decisions.

Source of financial support: This paper was given at a Symposium funded by the National Pharmaceutical Council and Shire Pharmaceuticals. The author has previously received funding from GSK to organize a workshop in this area. The Office of Health Economics receives funding from the Association of the British Pharmaceutical Industry.


  1. Top of page
  2. Introduction
  3. Multicriteria Decision Analysis
  4. Closing Remarks
  5. References
  • 1
    Aronson J. Balancing benefits and harms in health care. BMJ 2004;329:30.
  • 2
    Cross JT, Garrison LP. Challenges and opportunities for improving benefit-risk assessment of pharmaceuticals from an economic perspective. Office of Health Economics 2008. Available from:[Accessed May 27, 2010.
  • 3
    Mussen F, Salek S, Walker S. Benefit-Risk Appraisal of Medicines. Oxford: Wiley-Blackwell, 2009.
  • 4
    Briggs AH, Levy AR. Pharmacoeconomics and pharmacoepidemiology: curious bedfellows or a match made in heaven? Pharmacoeconomics 2006;24:107986.
  • 5
    Lynd LD, O’Brien BJ. Advances in risk-benefit evaluation using probabilistic simulation methods: an application to the prophylaxis of deep vein thrombosis. J Clin Epidemiol 2004;57:795803.
  • 6
    Garrison LP Jr, Towse A, Bresnahan BW. Assessing a structured, quantitative health outcomes approach to drug risk-benefit analysis. Health Aff (Millwood) 2007;26:68495.
  • 7
    Lynd LD. Using incremental net benefit for quantitative benefit-risk analysis case studies of Vioxx® and Lotronex®. Presented at the Benefit-Risk Assessments for Drugs Workshop. Office of Health Economics. October 24, 2007.
  • 8
    Lynd LD, Najafzadeh M, Colley L, et al. Using the incremental net benefit framework for quantitative benefit-risk analysis in regulatory decision-making—a case study of alosetron in irritable bowel syndrome. Value Health 2009;[Epub ahead of print.
  • 9
    Johnson R, Özdemir S, Hauber B, Kauf TL. Women's willingness to accept perceived risks for vasomotor symptom relief. J Womens Health 2007;16:102840.
  • 10
    Culyer AJ. Deliberative processes. Office of Health Economics 2009.