### Abstract

- Top of page
- Abstract
- Introduction
- Decision analysis
- Markov modelling
- Discounting
- Uncertainty analyses
- Commonly used software packages for pharmacoeconomic analyses
- Suggested approach to reading and appraising published pharmacoeconomic analyses
- Acknowledgements and Conflicts of Interest
- References

In the current climate of burgeoning health care costs, pharmacoeconomics is becoming increasingly important, but knowledge about pharmacoeconomic methods is limited among most clinicians. This review provides an introduction to, and overview of, common methods used in pharmacoeconomic modelling: decision analysis, Markov modelling, discounting and uncertainty analyses via Monte Carlo simulation. It will conclude with a suggested approach to reading and appraising published pharmacoeconomic analyses.

### Introduction

- Top of page
- Abstract
- Introduction
- Decision analysis
- Markov modelling
- Discounting
- Uncertainty analyses
- Commonly used software packages for pharmacoeconomic analyses
- Suggested approach to reading and appraising published pharmacoeconomic analyses
- Acknowledgements and Conflicts of Interest
- References

Properly designed clinical trials (or meta-analyses of such trials) provide the highest level of evidence regarding the *efficacy* of drugs, but they seldom afford insight into potential *effectiveness* and *cost-effectiveness* [1, 2]. Efficacy refers to the question of whether or not a drug provides benefit in a research setting (‘Can it work?’), while effectiveness and cost-effectiveness refer to questions of whether or not a drug provides benefit in the real world setting (‘Does it work?’ and ‘Is it worth it?’) [3, 4].

There is often a large gap between efficacy and effectiveness/cost-effectiveness because clinical trials are by necessity undertaken on highly circumscribed populations in tightly-controlled study environments, are often short in duration (up to 5 or so years) and do not consider competing mortality and morbidity from conditions not of interest. Furthermore, it is uncommon for clinical trials to consider costs, despite the fact that cost-effectiveness is a crucial determinant of the feasibility of pharmacotherapy.

Issues regarding the costs of health care may have more direct relevance to health policy makers and financiers of health care than clinicians, but in the current climate of burgeoning health care costs, the onus should be on all health professionals to ensure responsible health expenditure. Hence it is important for clinicians to be able to read and appraise the pharmacoeconomic literature. This review will provide an introduction to, and overview of, common methods used in pharmacoeconomic modelling: decision analysis, Markov modelling, discounting and uncertainty analyses via Monte Carlo simulation. It will conclude with a suggested approach to reading and appraising published pharmacoeconomic analyses.

### Decision analysis

- Top of page
- Abstract
- Introduction
- Decision analysis
- Markov modelling
- Discounting
- Uncertainty analyses
- Commonly used software packages for pharmacoeconomic analyses
- Suggested approach to reading and appraising published pharmacoeconomic analyses
- Acknowledgements and Conflicts of Interest
- References

Decision analysis [5-7] is used to quantify and compare explicitly various health strategies, including drug therapy, in terms of their likely health effects and/or costs, thus informing clinical practice as well as health policy. It is useful especially in situations where there is uncertainty about the balance of potential benefits and risks, and costs, associated with various health strategies.

Decision analysis is usually conceptualized as a *decision analysis tree*, which outlines and quantifies the consequences of the two or more options of a decision to be made. Figure 1 provides a simple, hypothetical example of a decision about whether or not to start drug therapy. Therapy is associated with both potential benefit (decrease in the risk of ‘disease A’) and harm (increase in the risk of ‘disease B’).

The square is the *decision node*, the point where alternative treatment options are defined. In this example, two choices are defined: ‘No drug therapy’ and ‘Drug therapy’. The circles represent *chance nodes*, from which emanate the possible consequences of each choice. These are called *transition states*. Both options in the example have the same four possible transition states: ‘Remain in good health’, ‘Develop ‘disease A’ (but stay alive)’, ‘Develop ‘disease B’ (but stay alive)’ and ‘Die’ (from any cause). The underlying likelihoods of their occurring are indicated below the relevant sub-branches, and are called *transition probabilities*. The sum of all transition probabilities emanating from a chance node is always one. In the example, compared with no drug therapy, drug therapy reduces the likelihood of developing non-fatal ‘disease A’ by 20% (from 20% to 16%) and of dying by 10% (from 10% to 9%), but increases the likelihood of developing non-fatal ‘disease B’ by 10% (from 20% to 22%). The triangles are *terminal nodes*, where the health impact of each consequence, called a *payoff*, is quantified.

In the example (Figure 1), as is often the case in pharmacoeconomic analyses, payoffs are expressed as *utilities*. Utilities provide a measure of health value and applies penalties to time spent in less-than-perfect health [8]. Utilities are useful because they provide a summary measure of health, and allow for comparison of disparate conditions. In the example, ‘disease A’ is ascribed a utility of 0.8. This means that ‘disease A’ is considered to be equivalent to 80% of full health. By the same token, ‘disease B’ is assigned a utility of 0.6, and hence equivalent to 60% of good health. Death has been ascribed a utility of zero, as is common practice.

To analyze (evaluate) a decision analysis tree, the *expected value* of each branch is calculated by multiplying the payoff associated with each transition by the probability of it occurring, and summing these. That is, the expected value of each main branch (‘No drug therapy’ and ‘Drug therapy’) is: Σ (payoff_{n} × transition probability_{n}).

In effect, the expected value is a weighted-average payoff associated with the option. In the example, the expected value of ‘No drug therapy’ is 0.78 utility (Σ[utility_{n} × transition probability_{n}] = 0.50 × 1.0 + 0.20 × 0.80 + 0.20 × 0.60 + 0.10 × 0) and the expected value of ‘Drug therapy'is 0.79 utility (0.53 × 1.0 + 0.16 × 0.80 + 0.22 × 0.60 + 0.09 × 0). Therefore on average, drug therapy would provide a (slightly) more favorable health return compared with no drug therapy. Even though it increases the risk of the more severe ‘disease B’, this is insufficient to offset its beneficial effects on the risk of non-fatal ‘disease A’ and death.

The consequences of an option can also be quantified in terms of costs. In Figure 2, the costs of diseases ‘A’ and ‘B’ are indicated in place of utilities from Figure 1. Costs for ‘disease B’ are higher because this condition is more disabling. The expected values of each branch are calculated in the same way. That is: Σ (cost_{n} × transition probability_{n}).

Analysis of the tree reveals the expected dollar values of ‘No drug therapy’ and ‘Drug therapy’ to be $120 (0.50 × $0 + 0.20 × $100 + 0.20 × $500 + 0.10 × $0) and $126 (0.53 × $0 + 0.16 × $100 + 0.22 × $500 + 0.09 × $0), respectively. On average, drug therapy would lead to greater downstream cost, even though it is associated with improved health outcomes.

So far, the expected dollar value of the drug therapy has not considered the cost of the drug itself. If this is assumed to be a once-off cost of $300, then the *net cost* of drug therapy would be: $300 + ($126–$120) = $306. The net costs of health interventions always take into account downstream related costs.

In the example, on average, if drug therapy was to be delivered, it would increase utility by 0.01 (from 0.78 to 0.79), but cost $306 dollars more.

The net cost per unit of benefit gained from intervention is the *incremental cost-effectiveness ratio* (ICER), which is the main outcome of interest in cost-effectiveness, including pharmacoeconomic, analyses. In the example, the ICER would be $306/0.01 utility gained = $30 600 per utility gained.

Utilities are multiplied by the years to which they apply to derive *quality-adjusted life years* (QALYs) [9-12]. In the example above, if it is assumed that the utilities stayed constant for 1 year (that is, the health benefit procured by drug therapy lasted for a whole year), then drug therapy would lead to 0.01 QALY gained (0.01 utility multiplied by 1 year), and the ICER would be $30 600 per QALY gained over 1 year.

Basic decision analysis considers only one sequence of events within one time-frame. This is a limitation if the conditions being simulated have different sequential stages, and/or data inputs (such as transition probabilities or costs) evolve with time. To overcome this inherent limitation of basic decision analysis, *Markov modelling* [10, 13] is often employed.

### Markov modelling

- Top of page
- Abstract
- Introduction
- Decision analysis
- Markov modelling
- Discounting
- Uncertainty analyses
- Commonly used software packages for pharmacoeconomic analyses
- Suggested approach to reading and appraising published pharmacoeconomic analyses
- Acknowledgements and Conflicts of Interest
- References

Figure 3 illustrates an extension of the situation depicted in Figures 1 and 2, in which multiple sequential events over multiple time periods can be considered.

After a person develops ‘disease A’, that person may later develop ‘disease B’, and *vice versa*. Recovery from either disease is not assumed to be possible (and hence there is no transition back to ‘healthy’). Any person is also susceptible to death at any time. To capture all possible events, multiple health states need to be modelled, each with its own set of possible transitions. The exception is death, from which no transitions can be made. Death is considered an absorbing state.

Analysis of Markov decision analysis trees does not involve just a single transition of subjects from left to right across the branches, but. rather involves repeated recycling between the terminal nodes (except in the case of absorbing states) and the purple encircled ‘M's, which are called *Markov nodes*. Markov nodes represent the points to which individuals return at the end of each cycle, and are channelled to one of the health states to begin the next cycle, either one previously occupied or another, depending on the transition they just made.

The above process describes Markov modelling [10, 13], which was named after the Russian mathematician Andrei Markov (1856–1922). A Markov chain describes a sequence of events in which the likelihood of an event occurring is dependent on the preceding event. A Markov chain has no length limit, but will end in an event that is absorbing. A Markov model often comprises more than one Markov chain. Figure 4 illustrates in a state-transition format the Markov model that is depicted as a decision analysis tree in Figure 3.

The key features of a Markov model are:

- that individuals reside in one of a finite set of mutually exclusive health states.
- that time is represented by discrete periods called
*cycles*, and individuals move between health states, or remain in their current health states, at the end of each cycle. - that movements are governed by transition probabilities that are specific to each health state and each cycle.

Evaluation of a decision analysis tree which incorporates Markov modelling is straight forward: expected values are calculated at the end of each cycle and totalled at the end of the simulation.

The main advantage of incorporating Markov modelling into a decision analysis is that is allows for the simulation of more complex consequences of an option, as is illustrated above. Not only can a greater number of possible events be simulated, they can be simulated for lengthier periods downstream.

The main limitation of Markov models is that with any cycle, the possible transitions which individuals can make depend only on the state they are in, and not on how they arrived at that state. That is, Markov modelling lacks memory. This characteristic is often known as the *Markov assumption*. This problem with the Markov assumption is especially highlighted in two settings. First, when more than one disease is being considered, because many diseases are inter-dependent and the probability of one is often affected by the prior occurrence of another. Secondly, when model inputs (such as transition probabilities, utilities and costs) evolve over time. For example, as the age of model subjects increases with repeated cycles, their risks of diseases will increase.

The first problem is overcome by including as many health states as is required to capture important disease combinations, as is exemplified by Figures 3 and 4. However, inclusion of too many health states into a Markov model can render it unwieldy. Paradoxically, a complex Markov model that attempts to reflect more accurately multiple related conditions may be subject to more uncertainty because of the need for more data inputs and assumptions to be made about them. For example, In Figures 3 and 4, transition probabilities need to be estimated for diseases A and B that are *conditional* on the presence or otherwise of the other disease.

The second problem of evolving data inputs is overcome by including cycle specific inputs. For some inputs, cycle specificity is easily estimated, such as age-related changes to risks of disease, but for others it poses a challenge due to a lack of data, such as age specificity of disease costs and utilities.

### Uncertainty analyses

- Top of page
- Abstract
- Introduction
- Decision analysis
- Markov modelling
- Discounting
- Uncertainty analyses
- Commonly used software packages for pharmacoeconomic analyses
- Suggested approach to reading and appraising published pharmacoeconomic analyses
- Acknowledgements and Conflicts of Interest
- References

Because pharmacoeconomic modelling is based on conjecture, it is important that uncertainty be expressed in the modelling outputs. Indeed, the expression of uncertainty is a fundamental part of any clinical and applied research, as epitomized by the 95% confidence interval that surrounds most estimates. Models are vulnerable to uncertainty because they rely on multiple data inputs, and as the number of inputs and modelling steps increase, so too will uncertainty in outputs. In addition, assumptions have to be made in many instances, which serve to increase uncertainty even further.

*Monte Carlo simulation* [16] refers to a method of undertaking multiple simulations of a model, each time taking samples from specified uncertainty ranges of the model's inputs (as opposed to point estimates). These uncertainty ranges are most often expressed as probability distributions. Probability distributions describe the range of possible values for a parameter as well as the probability of each value occurring. Common types of probability distributions include ‘gamma’, ‘uniform’, ‘normal’ and ‘triangular’ (Figure 5). Probability distributions that relate to data inputs for a model are often called *input distributions*.

As Monte Carlo simulation involves multiple simulations, it therefore generates multiple outputs, from which probability distributions can also be derived. These are often called *output distributions*. With only a small number of iterations in a Monte Carlo simulation, the output distribution is not well defined. However, as more iterations are run, the output distribution becomes more definable and is said to become stable. This phenomenon is known as *convergence* [17]. A sufficient number of iterations must be undertaken in Monte Carlo simulation such that convergence of an output distribution occurs.

Monte Carlo simulation modelling is also known as *stochastic* modelling. This defines a type of modelling in which there is a range of possible outputs that is dependent on the probabilistic nature of inputs. In contrast, models whose outputs are fixed because their inputs do not vary are known as *deterministic* models.

### Commonly used software packages for pharmacoeconomic analyses

- Top of page
- Abstract
- Introduction
- Decision analysis
- Markov modelling
- Discounting
- Uncertainty analyses
- Commonly used software packages for pharmacoeconomic analyses
- Suggested approach to reading and appraising published pharmacoeconomic analyses
- Acknowledgements and Conflicts of Interest
- References

*TreeAge* (TreeAge Software Inc, Williamstown, MA, USA) is a commonly-used software package designed specifically for decision analyses. It is a powerful program that also allows for construction of complex Markov models and incorporates uncertainty analyses via Monte Carlo simulation.

*Microsoft Excel* (Microsoft Corporation, Redmond, WA, USA) is also widely used to undertake pharmacoeconomic analyses. Being entirely flexible, it can be programmed to simulate decision analysis and Markov modelling. Monte Carlo simulation is not possible in Excel itself, but this feature can be added with installation of the macro *@RISK* (Palisade Corporation, Ithaca, NY, USA).

### Suggested approach to reading and appraising published pharmacoeconomic analyses

- Top of page
- Abstract
- Introduction
- Decision analysis
- Markov modelling
- Discounting
- Uncertainty analyses
- Commonly used software packages for pharmacoeconomic analyses
- Suggested approach to reading and appraising published pharmacoeconomic analyses
- Acknowledgements and Conflicts of Interest
- References

The parameters of a pharmacoeconomic analysis should be considered in the same way as the parameters of a clinical trial: population, intervention, comparator, outcome and timing (PICOT).

The population comprises the modelled population, sources of input data and assumptions for which must be clearly articulated so that its generalizability and applicability can be ascertained.

The intervention is the drug of interest, and all assumptions made about its use (such as dose over time and adherence) should be clearly described. The comparator should also clearly be described, as it is the reference against which health and economic outcomes associated with the drug of interest are compared.

Outcomes are usually net costs, net health benefits expressed as life years gained or QALYs gained and ICERs (the ratio of net costs to net health benefits). Costs will depend on the *perspective* adopted for the pharmacoeconomic evaluation, which should be clearly defined. The two common perspectives are the *health system perspective*, which captures only costs met by healthcare financiers, and the *societal perspective*, which captures all costs, including those outside of the healthcare system (such as occurs with loss of productivity). Life years and QALYs gained are extrapolated from the reduction in the risk of the target disease and/or death, and all assumptions made about the translation of treatment benefit to health gain need to be carefully considered.

The time frame considered in a pharmacoeconomic model is called the *time horizon.* Expression of the time horizon in any economic evaluation is important because ICERs vary with time. They generally decrease (that is, the intervention becomes more cost-effective) as the time horizon increases. Another key consideration related to the time horizon is the assumption made regarding the benefits of the drug compared with the comparator over time. This is especially important in instances where the modelled time horizon exceeds the time frame of the clinical trial(s) from which efficacy measures are derived.

### Acknowledgements and Conflicts of Interest

- Top of page
- Abstract
- Introduction
- Decision analysis
- Markov modelling
- Discounting
- Uncertainty analyses
- Commonly used software packages for pharmacoeconomic analyses
- Suggested approach to reading and appraising published pharmacoeconomic analyses
- Acknowledgements and Conflicts of Interest
- References

All authors have completed the Unified Competing Interest form at http://www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare no support from any organization for the submitted work (ZA, CRM, BH and DL) or had support from the National Health and Medical Research Council of Australia (HK) and the Australian Research Council (EZ) for the submitted work, no financial relationships with any organizations that might have an interest in the submitted work in the previous 3 years (ZA, EZ and BH) or received research grant support from Servier, sanofi-aventis, Bristol-Myers-Squibb and Pfizer (CRM) or received research grant support from sanofi-aventis and Pfizer (DL) or received honoraria from sanofi-aventis, AstraZeneca, Pfizer, Abbott, Bayer and Glaxo-Smith-Kline (DL) or employment at Bristol-Myers-Squibb and Glaxo-Smith-Kline (HK) in the previous 3 years and no other relationships or activities that could appear to have influenced the submitted work (ZA, HK, EZ, BH, CRM and DL).