- Top of page
Objective: There is consensus that a more transparent, explicit, and rigorous approach to benefit–risk evaluation is required. The objective of this study is to evaluate the incremental net benefit (INB) framework for undertaking quantitative benefit–risk assessment by performing a quantitative benefit–risk analysis of alosetron for the treatment of irritable bowel syndrome from the patients’ perspective.
Methods: A discrete event simulation model was developed to determine the INB of alosetron relative to placebo, calculated as “relative value-adjusted life-years (RVALYs).”
Results: In the base case analysis, alosetron resulted in a mean INB of 34.1 RVALYs per 1000 patients treated relative to placebo over 52 weeks of treatment. Incorporating parameter uncertainty into the model, probabilistic sensitivity analysis revealed a mean INB of 30.4 (95% confidence interval 15.9–45.4) RVALYs per 1000 patients treated relative to placebo over 52 weeks of treatment. Overall, there was >99% chance that both the incremental benefit and incremental risk associated with alosetron are greater than placebo. As hypothesized, the INB of alosetron was greatest in patients with the worst quality of life experienced at baseline. The mean INB associated with alosetron in patients with mild, moderate, and severe symptoms at baseline was 17.97 (−0.55 to 36.23), 29.98 (17.05–43.37), and 35.98 (23.49–48.77) RVALYs per 1000 patients treated, respectively.
Conclusions: This study demonstrates the potential utility of applying the INB framework to real-life decision-making, and the ability to use simulation modeling incorporating outcomes data from different sources as a benefit–risk decision aid.
- Top of page
The traditional approach to benefit–risk analysis generally involves the sequential evaluation of the potential harms and benefits within the classical statistical (i.e., frequentist) paradigm of hypothesis testing. From the regulatory perspective, the mandate of the decision-maker is to review the available evidence and draw conclusions about the safety and efficacy of a product in both the pre- and postmarketing phases by considering the weight of the available evidence. This process does not currently include, or require, an explicit, transparent quantitative benefit–risk analysis that facilitates the joint consideration of the potential harms and benefits and incorporates some measurement of risk preference. Given the degree of complexity and uncertainty associated with making trade-offs across multiple harms and benefits, this decision-making process is not straightforward.
The US Institute of Medicine recommends that researchers investigate new approaches to conceptualizing, measuring, and applying benefit–risk analysis, and that the Center for Drug Evaluation and Research “develop and continually improve a systematic approach to benefit–risk analysis for use throughout the US Food and Drug Administration (FDA) in the pre- and postapproval settings”. The European Medicines Agency's Committee for Medicinal Products for Human Use (CHMP) recommended that the best method would be a structured approach that explicitly incorporates the importance of benefits and risks and describes uncertainties and their effect on the benefit–risk assessment . Although the CHMP currently recommends a mainly qualitative approach, it also recommends further research to explore further methodological development. Thus, there is consensus that a more transparent, explicit, and rigorous approach to benefit–risk evaluation is required.
Although a number of quantitative methods of benefit–risk assessment have been proposed, none have been universally adopted [3–8]. Lynd and O’Brien demonstrated the application of the incremental net benefit (INB) framework to benefit–risk analysis in deep-vein thrombosis prophylaxis . More recently, Garrison et al. also demonstrated the use of the INB framework for quantitative benefit–risk analysis using a hypothetical decision regarding a new weight loss drug, with quality-adjusted life-years (QALYs) as the outcome . Further empirical evidence of its application to benefit–risk analysis is required.
The primary objective of this study was to evaluate the INB framework for undertaking quantitative benefit–risk assessment by performing a quantitative benefit–risk analysis of alosetron for the treatment of irritable bowel syndrome (IBS), from the patients’ perspective using data available at the time of the regulatory benefit–risk decision. Alosetron was chosen for this evaluation based on its regulatory history, having been voluntarily withdrawn from the market in November 2000 due to concerns related to potential serious adverse reactions but was reintroduced in June 2002 after further analysis and extensive consultation with the US FDA, the implementation of a detailed risk management plan, and a strong patient lobby . Secondary objectives of this study were: 1) to determine the INB of alosetron relative to placebo from the patients’ perspective over a 52-week time horizon and 2) to determine if the INB of alosetron differs based on symptom severity.
- Top of page
Using a computer simulation model, the impact of alosetron on patient's quality of life was evaluated relative to placebo. The potential benefits included decreased frequency of abdominal pain, urgency, and diarrhea, and the primary potential adverse events were constipation (mild, moderate, and severe), ischemic colitis, and impacted or perforated bowel. The risk of death secondary to ischemic colitis and impacted or perforated bowel was also incorporated into the model based on external observational data. The primary clinical outcomes pertaining to both potential harms and potential benefits were derived from patient-level, premarketing randomized controlled trials (RCTs) in patients with moderate to severe IBS. Any RCT that evaluated alosetron 1 mg twice daily for the treatment of IBS in adults (aged 18 years and over) with at least 12 weeks of treatment, and where each of these outcomes was recorded daily for the duration of the trial, was included. Patient-level data from all studies that met the inclusion criteria were then merged to facilitate a pooled analysis. All potential outcomes were weighted using patients’ preferences to facilitate the calculation of INB.
In all included RCTs, patients were required to keep a daily diary of all IBS-associated symptoms. Adverse events, including constipation, were only recorded when they occurred. The number of days a patient reported constipation in a given week was determined and then classified as none, mild (1–2 days), moderate (3–5 days), or severe (>5 days). For ischemic colitis, and impacted or perforated bowel, the week the adverse event occurred and the time required for resolution were determined from the reported data.
We developed a discrete event simulation model using Arena version 9.00 (Rockwell Software, Inc., Milwaukee, WI) to calculate the INB of alosetron relative to placebo in a two hypothetical cohorts of 10,000 patients with moderate to severe IBS, over a 1-year time horizon (Fig. 1). In the base case, one hypothetical patient was randomly assigned specific baseline characteristics determined from the RCT data (i.e., age, IBS type, and IBS severity) and then hypothetically exposed to alosetron, while an identical clone in the other cohort was simultaneously exposed to placebo (Fig. 1). The base-case analysis was based on the point estimates of all model parameters including regression coefficients. The model consisted of 52 1-week cycles resulting in a 1-year time horizon. Over the time horizon of the model, each patient experienced improvements and decrements in his or her health-related quality of life (HRQoL) (i.e., increased or decreased utility) based on the changes in the frequency of his or her symptoms and the occurrence of adverse events. Baseline IBS symptoms were determined based on symptoms reported during the 2-week pretreatment run-in phase and classified as either mild (abdominal pain ≤5 days per week, urgency ≤2 days a week, and diarrhea ≤2 times a day) or severe (abdominal pain and urgency ≥3 days per week and diarrhea ≥3 times a day); patients not classified as mild or severe were deemed to have moderate symptoms.
Translating all model outcomes into HRQoL impact facilitated the calculation of the INB in “relative value-adjusted life-years” (RVALYs) [9,10]. RVALYs incorporate both the duration of time spent in a given health state, and the quality of life (i.e., utility or disutility) associated with that health state. Nevertheless, whereas QALYs are generally calculated using von Neumann–Morgenstern utilities, RVALYs were calculated using relative utilities derived using conjoint analysis. To calculate the INB, the net benefit of alosetron and placebo was calculated separately based on the difference between the mean RVALYs gained and lost due to benefit and adverse events, respectively. The mean INB associated with alosetron relative to placebo was then calculated by subtracting the mean net benefit of placebo from the mean net benefit of alosetron for each patient pair, and then averaging over the entire cohort.
Utilities Derived Using Conjoint Preference Weights
The calculation of the INB requires that all potential harms and benefits be on a common metric (i.e., RVALYs), which is achieved by incorporating the relative weighting of each outcome. The multidimensionality and complex interrelationship of the benefits and harms also required that the preference weights be additive. Therefore, conjoint preference weights (utilities) for IBS symptoms and potential treatment-related adverse events were elicited from 565 patients with IBS using a discrete choice experiment (Table 1). Preference weights were scaled between 0 and 1 in which 0 and 1 represent the combination of the best and worst levels of all attributes, respectively. The conjoint event-specific preference weights for each adverse outcome were determined by converting the preference weights for the different levels of risk for each probabilistic outcome to a continuous preference weight by linearly interpolating between categorical risk levels. Ninety-five percent confidence intervals (CIs) for the preference weights were then calculated using the technique proposed by Krinsky and Robb with 10,000 random draws . The rescaled preference weight for each outcome level can be interpreted as the decrease in utility associated with that outcome.
Table 1. Preference weights (scaled to range from 0 to 1) for treatment outcomes derived using conjoint analysis
|Abdominal pain||1–2 days a week||0.0151|
|3–5 days a week||0.0378|
|6–7 days a week||0.0529|
|Urgency||1–2 days a week||0.0198|
|3–5 days a week||0.0753|
|6–7 days a week||0.0817|
|Diarrhea||1–2 times a day||0.0269|
|3–4 times a day||0.0827|
|>4 times a day||0.1042|
|Constipation||1–2 days a week||0.0143|
|3–5 days a week||0.0359|
|6–7 days a week||0.0502|
|Moderate colitis|| ||0.0177|
|Severe colitis|| ||0.1258|
|Impacted bowel|| ||0.0987|
|Perforated bowel|| ||0.3072|
Extrapolation of Outcomes beyond 12 Weeks
The mean change in utility over the first 12 weeks revealed that symptomatic improvement was greatest in the early weeks and plateaued by week 12. To incorporate the time dependency of symptom improvement and the differences between treatments, a difference model with time-dependent coefficients was fitted to the 12-week patient-level data for both alosetron and placebo, such that
where ΔUit represents the change in utility for patient i in time interval t relative to time interval t − 1 (i.e., ΔUit = Uit − Uit−1), and S is a categorical variable that represents IBS symptoms at baseline (i.e., mild, moderate, or severe). By defining the baseline symptoms as a categorical variable, we estimated three different intercepts for patients with mild, moderate, and severe symptoms. With utility as the independent variable, we estimated time-dependent coefficients (i.e., β0t, β1t, β2t) for each independent variable in SAS (SAS v.9, Cary, NC). This model was then used to predict the utility for each patient for weeks 13 to 52. The regression coefficient for time was not statistically significant for weeks 7 to 12 for either the placebo or alosetron groups, suggesting that the effect of both treatments had plateaued. We therefore assumed that the estimated regression coefficient for week 12 was the best predictor of the subsequent 40 weeks in both groups, which is consistent with previously published data .
Similarly, the decrement in HRQoL associated with constipation was estimated by fitting a multinomial mixed logit model to the 12 weeks of patient-level outcome data with severity of constipation (i.e., mild, moderate, or severe) each week as the dependent variable, so that
where Lit represents the cumulative logit function for mild, moderate, and severe constipation for patient i during time t (where t = 0–12 weeks), age is age of patient, and type represents IBS subtype. These models revealed that the risk of constipation associated with placebo was constant over time, whereas the risk associated with alosetron increased over the first 6 weeks of treatment and then returned to baseline by week 12, which is also consistent with previously published data . We therefore assumed that the risk of constipation in week 12 persisted over the remaining 40 weeks of the model.
The rate and duration of disutility associated with the observed serious adverse events were determined directly from the RCT data. The time to occurrence of both ischemic colitis and impacted bowel was modeled using a Weibull distribution over 52 weeks. We assumed that patients experiencing constipation continued therapy (and therefore continued to experience an improvement in HRQoL), while patients experiencing colitis and ischemic bowel discontinued therapy.
No alosetron-related deaths occurred in any premarketing RCTs. Nevertheless, because of the potential additional increased risk of ischemic colitis and impacted bowel associated with alosetron, and the underlying mortality risk associated with these events independent of treatment, we included the potential for mortality associated with each of these events in the model. Although Wolfe et al. reported two cases of death out of 640 patients receiving alosetron for more than 6 months, neither of these deaths were attributed to treatment based on the investigators’ judgment . To achieve this, we used the estimated mortality rate for ischemic colitis of 0.023 (2 out of 84) and for impacted bowel of 0.015 (2 out of 133) that Ladabaum used  that were derived from FDA documentation (Table 2). Given their clinical similarities and the potential for misclassification, we assumed that the probability of death associated with severe constipation was the same as for impacted bowel.
Table 2. Model parameters
|Variable||Alostron||Placebo||Method of derivation or distribution|
|Change in utility over time as a result of improvement of symptoms (abdominal pain, urgency, and diarrhea)||ΔUit = β0t + β1t(1 − Uit−1) + β2tSi + εit||Mixed linear regression|
|Probability of constipation (severe, moderate, and mild) as a function of time||Logitit = β0i + β1it * t + β2 * age + β3 * IBS_subtype + εit||Longitudinal multinomial logit model|
|Probability of ischemic colitis||3 out of 1792||0||Beta distribution|
|Probability of impacted bowel||2 out of 1792||0||Beta distribution|
|Probability of death conditional on ischemic colitis||2 out of 84||0||Weibull distribution|
|Probability of death conditional on impacted bowel||2 out of 133||0||Weibull distribution|
To evaluate the impact of this uncertainty on the INB, we incorporated beta distributions around all probabilities and standard errors for all coefficients from each regression model into the simulation. A cohort of 10,000 hypothetical patients and their clones were simulated through the model with one set of model parameters. This process was repeated 10,000 times, with new parameter estimates randomly selected from the distributions for each model parameter each time. The mean net benefit and INB was then calculated for each iteration of the model from which the overall mean INB of alosetron relative to placebo over 10,000 iterations of the model was determined. The uncertainty in the results associated with variation in model parameters is illustrated on the incremental risk–benefit plane .
Stratified and Probabilistic Sensitivity Analysis (PSA)
We hypothesized that the model would demonstrate that alosetron results in a greater INB in IBS patients with more severe baseline symptoms. To test this hypothesis, we stratified the simulated cohort based on baseline symptom severity into mild, moderate, and severe subgroups, and calculated the INB of alosetron relative to placebo for each subgroup by running a strata-specific model for 10,000 patients in each cohort and comparing the INB between strata. The mean INB and 95% credibility interval were determined for each stratified cohort.
- Top of page
Eighteen phase III RCTs were evaluated and considered for inclusion in the analysis; seven studies met all inclusion criteria for this study (S3BA3001, S3BA3002, S3BA2001, S3B20023, S3B30013, S3BB3001, and S3B23002). Eleven studies were excluded because abdominal pain (two studies), urgency (one study), or symptoms (three studies) were not measured daily, the study was open label (two studies), treatment was <12 weeks (two studies), and one study involved patients <18 years. From the seven eligible studies, some patients were excluded due to lack of baseline data (n = 22), no recorded start date for treatment (n = 9), or no recorded data after baseline (n = 25). Thus, the final analysis includes 1792 patients randomized to alosetron and 1106 randomized to placebo, providing 18,667 and 11,936 person-weeks of exposure, respectively. Both groups were comparable with respect to baseline demographics, IBS subtype, and symptom frequency, and approximately 90% of the patients in both groups were female (Table 3). There were three cases of ischemic colitis and two cases of impacted bowel reported in all alosetron-treated patients over a mean follow-up of 10.7 weeks, which resolved over 9.3 days and 5 days, respectively. Neither of these outcomes was reported in the placebo arm of any trial.
Table 3. Baseline demographics and symptoms of patients from eligible randomized controlled trials
| ||Placebo (N = 1106)||Alosetron 1 mg (N = 1792)|
|Age [mean (SD)]||45.3 (13.4)||45.5 (13.5)|
|Sex (female) [n (%)]||960 (86.8%)||1647 (91.9%)|
|Height (cm) [mean (SD)]||158.9 (25.3)||161.2 (19.3)|
|Weight (kg) [mean (SD)]||82.4 (29.7)||76.7 (25.7)|
|Treatment duration (weeks)|| || |
| Mean (SD)||8.9 (3.7)||8.1 (4.2)|
|Person weeks of follow-up||11,936||18,667|
|Ethnicity [n (%)]|| || |
| Asian||5 (0.5%)||4 (0.2%)|
| Black||62 (5.6%)||39 (2.2%)|
| White||1,002 (90.6%)||1,671 (93.2%)|
| Hispanic||32 (2.9%)||70 (3.9%)|
| Other||5 (0.5%)||8 (0.4%)|
|Irritable bowel syndrome subtype|| || |
| Alternating||489 (44.2%)||623 (34.8%)|
| Diarrhea||595 (53.8%)||1,130 (63.1%)|
| Constipation||22 (2.0%)||39 (2.2%)|
|Frequency of abdominal pain|| || |
| 1–2 days a week||24 (2.2%)||25 (1.4%)|
| 3–5 days a week||329 (29.7%)||501 (28.0%)|
| 6–7 days a week||753 (68.1%)||1,266 (70.6%)|
|Frequency of urgency|| || |
| None||14 (1.3%)||38 (2.1%)|
| 1–2 days a week||123 (11.1%)||216 (12.1%)|
| 3–5 days a week||462 (41.8%)||728 (40.6%)|
| 6–7 days a week||507 (45.8%)||810 (45.2%)|
|Frequency of diarrhea|| || |
| None||76 (6.9%)||131 (7.3%)|
| 1–2 times a day||660 (59.7%)||1,064 (59.4%)|
| 3–4 times a day||299 (27.0%)||458 (25.6%)|
| >4 times a day||71 (6.4%)||139 (7.8%)|
In the base-case analysis, alosetron resulted in a mean incremental benefit of 34.9 RVALYs and a mean incremental harm of −0.8 RVALYs per 1000 patients relative to placebo. Therefore, the potential benefit of alosetron exceeded the potential harm (INB 34.1 RVALYs per 1000 patients treated) relative to placebo over 52 weeks of treatment. In the base case of 10,000 hypothetical patients treated with alosetron (520,000 weeks of follow-up), constipation occurred in 23,854 (4.6%) weeks (4642 mild, 6833 moderate, and 12,379 severe), and there were 59 cases of impacted bowel (0.11 per 1000 weeks of treatment), 76 cases of ischemic colitis (0.15 cases per 1000 weeks of treatment), and 4 deaths. Conversely, constipation occurred in 4037 (0.8%) weeks in patients receiving placebo (830 mild, 1715 moderate, and 1492 severe cases), and there were no cases of ischemic colitis, impacted bowel, or death.
PSA revealed a mean incremental benefit of 31.2 (95% CI 16.7–46.2) RVALYs and a mean incremental risk of −0.8 (95% CI −0.5 to −1.1) RVALYs per 1000 patients treated with alosetron relative to placebo. Therefore, the potential benefit of alosetron exceeded the potential risk with an INB of 30.4 (95% CI 15.9–45.4) RVALYs per 1000 patients treated relative to placebo over 52 weeks of treatment.
The primary etiologic factor contributing to the decrement in HRQoL (and consequentially RVALYs) with both treatments was constipation. Averaging over 10,000 iterations of the model in the probabilistic analysis, constipation occurred in 23,761 of 520,000 (4.6%) weeks of follow-up in the alosetron arm (95% CI 19,590–28,705) versus 3849 (0.7%) weeks (95% CI 1342–8844) in the placebo arm. Consistent with the base-case analysis, on average there were 53 cases of impacted bowel (95% CI 7–139), 83 cases of ischemic colitis (95% CI 18–204), and 3 deaths (95% CI 0–8) associated with these events per 10,000 hypothetical patients treated with alosetron, and no events associated with placebo.
The results of the PSA incorporating all second-order uncertainty of model parameters are illustrated in Figure 2. This figure illustrates that there is >99% change that the INB of alosetron relative to placebo is >0. The results of the PSA can be further depicted on the risk–benefit plane (Fig. 3). Each point on the plane represents the mean incremental benefit and risk based on 10,000 patients simulated with one set of model parameters. Based on 10,000 sets of randomly selected sets of model parameters, essentially, the entire joint density of the distribution of incremental risk relative to benefit lies in the northeast quadrant of the risk–benefit plane. If we assume that decision-makers would trade off HRQoL (i.e., RVALYs) associated with benefit and risk 1:1, the risk–benefit threshold (μ = 1) can be depicted by a line through the origin in the northeast quadrant that has a slope of 1. This model therefore implies >99% chance that both the incremental benefit and incremental risk associated with alosetron are greater than placebo. Assuming a risk threshold of μ = 1, there is also >99.9% chance that the INB of alosetron relative to placebo is below this threshold and therefore favors the use of alosetron.
Figure 2. Distribution of the mean incremental net benefit of alosetron relative to placebo derived from the probabilistic sensitivity analysis. RVALYs, relative value-adjusted life-years.
Download figure to PowerPoint
Figure 3. Mean incremental harm versus mean incremental benefit of alosetron relative to placebo over 52 weeks based on 10,000 patients simulated 10,000 each. Note: line illustrated in the northeast quadrant with slope of μ = 1 indicates a benefit–harm trade-off of 1:1. This would be depicted as a 45% degree line passing through the origin if vertical and horizontal axes were drawn on the same scale. RVALYs, relative value-adjusted life-years.
Download figure to PowerPoint
As hypothesized, the stratification of patients by baseline quality of life revealed a greater INB associated with alosetron in patients with more severe symptoms at baseline. The mean INBs (95% credibility interval) in patients with mild, moderate, and severe baseline symptoms were 17.97 (−0.55 to 36.23), 29.98 (17.05–43.37), and 35.98 (23.49–48.77) RVALYs per 1000 patients treated, respectively.
- Top of page
This study demonstrates the potential utility of applying the INB framework to real-life decision-making, and the ability to use simulation modeling incorporating outcomes data from different sources as a benefit−risk decision aid. Additionally, in this model, we were able to extrapolate the analysis beyond the follow-up period of the RCTs.
INB and QALYs have been adopted as the standard for evaluating the cost-effectiveness of new therapeutic interventions to help inform reimbursement and insurance decisions [17,18]. Other than data on costs, the data requirements for calculating the INB for cost-effectiveness or risk−benefit analysis are the same. Although no specific methods have been adopted for quantitative benefit−risk analysis, it has been recommended that regulators develop, adopt, and improve quantitative approaches . Although many benefit−risk analyses have been published previously, most have not incorporated an explicit, quantitative method to simultaneously evaluate potential benefit and risk [19–24].
This study and others provide empirical evidence supporting the use of INB without including costs to facilitate more explicit and transparent regulatory and clinical decision-making [9,25].
These results are also consistent with the eventual regulatory decision that the potential benefits of alosetron outweigh the potential risks from the perspective of the patient, and that the INB is greater in IBS patients with more severe symptoms. Although alosetron was voluntarily withdrawn from the market due to safety concerns, this was followed by a strong lobby from patients who had benefited from the drug for continued access despite safety concerns . This illustrates the revealed preferences of at least some IBS patients who were willing to accept the risk of a serious adverse event in exchange for potential benefit. Although we would argue that the regulatory decisions to remove alosetron, and subsequently to reintroduce it, were made considering different risk preferences (e.g., regulator's vs. patients’), the risk preferences applied in the decision-making process were not explicit. This analysis explicitly incorporates patients’ risk preferences and therefore only informs a decision from the perspective of the patient.
This analysis illustrates a number of advantages of using this benefit−risk modeling approach: 1) all data elements of the model are explicit and transparent; 2) it provides a quantification of the uncertainty surrounding a decision, given the available data; and 3) it facilitates stratified analysis to evaluate differences in INB among different strata. These results are consistent with the US FDA decision to reintroduce alosetron with an indication restricted to severe IBS, and therefore, it can be hypothesized that if this model had been available at the time, the regulatory decision might have been made more expeditiously. Conversely, had the results of this analysis been available at the time of the initial FDA regulatory review, the initial approved indication might have been more restrictive, which might have prevented the need for withdrawal, rereview, and reintroduction.
This study is somewhat limited by the inability to incorporate all RCT data into the analysis. Only seven studies met the inclusion criteria due to the requirement that all outcomes be measured at all time points over at least 12 weeks. This limitation is a consequence of performing the analysis retrospectively as opposed to planning both the benefit−risk analysis and the RCTs a priori so that all studies could be included. Nonetheless, this proof of concept study aptly demonstrates the feasibility and practicality of using phase III trials to evaluate the INB.
Regulatory decisions must be made based on phase III RCT data and are therefore “risk-efficacy” decisions, potentially resulting in undetected or unmeasured long-term outcomes and rare events. Although there is no way to evaluate the impact of any unknown or unanticipated adverse outcomes into an analysis, using a modeling approach, we were able to incorporate potential anticipated adverse outcomes not directly observed in the RCTs (i.e., death). Although modeling requires that certain assumptions be made, the robustness of the model to these assumptions can be tested in a sensitivity analysis. We assumed that the benefit of treatment with both alosetron and placebo was maintained from week 12 to week 52, which is consistent with one published long-term study , and where possible, we biased the model against alosetron. Despite these potential biases, the results still favored alosetron.
This analysis illustrates the utility of a modeling application of INB following phase III; however, this methodology could also be applied in the postmarketing phase using observational, real-world effectiveness data, if available. Different perspectives (e.g., societal) could also be taken, different methods for preference elicitation could be employed, and different preferences for different types of patients (e.g., mild vs. severe) could be applied, which could change the results. Given that quantitative evaluation should not be seen as a replacement for expert judgment but rather viewed as a decision aid, making the elements of the decision more explicit and transparent, the next steps will be to continue to develop, apply, and evaluate this methodology, and for regulators and decision-makers to further evaluate its utility in decision-making.