SEARCH

SEARCH BY CITATION

Introduction

  1. Top of page
  2. Introduction
  3. Health Care Economic Evaluations: A Basic Anatomy
  4. Measurement of Health Outcomes
  5. Measurement of Costs
  6. Data Collection and Analysis for Health Outcomes and Costs
  7. Evaluation of Uncertainty in Results
  8. Checklist for Reviewing a Health Care Economic Evaluation
  9. Relevance of Health Care Economic Comparisons With Clinical Practice
  10. Conclusions
  11. REFERENCES

Musculoskeletal conditions are the most common cause of severe long-term pain and physical disability in the world, affecting hundreds of millions of individuals and incurring enormous, and rising, costs upon society (1). Costs climbed from 0.5% to almost 3% of the gross domestic product in the US during 1963–1995, and appear to have risen through today (2). Two powerful and persistent forces of supply and demand have helped drive this trend: the aging of the population and the continuing march of scientific progress in medical care (1). In the context of inflammatory rheumatic diseases, a new and far more expensive era has been brought forth by the arrival of new biologic therapies (e.g., infliximab, etanercept, adalimumab, and rituximab) and higher priced nonsteroidal antiinflammatory drugs. Currently, the cost of treating rheumatoid arthritis (RA) with biologic agents is more than triple other RA treatments (1, 3), and the economic burden of RA on society has been compared with that of coronary artery disease (3). With expanding indications for biologic agents and new biologic agents under development, a natural question of growing prominence has been what is the best way to use our health care dollars? A proliferation of health care economic evaluations has wrestled with this question in many contexts, especially in the area of RA (4–25). This article provides a primer for rheumatologists to assist in understanding what these types of studies are, their strengths and limitations, and how they may influence policy and access to treatments.

Health Care Economic Evaluations: A Basic Anatomy

  1. Top of page
  2. Introduction
  3. Health Care Economic Evaluations: A Basic Anatomy
  4. Measurement of Health Outcomes
  5. Measurement of Costs
  6. Data Collection and Analysis for Health Outcomes and Costs
  7. Evaluation of Uncertainty in Results
  8. Checklist for Reviewing a Health Care Economic Evaluation
  9. Relevance of Health Care Economic Comparisons With Clinical Practice
  10. Conclusions
  11. REFERENCES

Health care economic evaluations (health care evaluations) are based upon the principles of welfare economics, a subfield within economics that has explored the question of how to maximize consumer welfare within a limited budget. Accordingly, these research studies are concerned with the comparison of a set of strategies (2 or more available choices of action) and not the appraisal of any single strategy in isolation. Health care evaluations are intended to help decision-makers make choices.

Health care evaluations compare such choices with respect to expected consequences resulting from the adoption of one strategy over another. Health outcomes are consequences that affect the well-being, or quality of life, of individuals. Cost outcomes (costs) relate to the consumption and production of resources (26, 27).

There are 4 main types of health care evaluation. Each characterizes costs in the same terms, i.e., in monetary units (such as dollars). However, each has a different approach to characterizing health outcomes (26, 27).

Cost-minimization analysis assumes that all strategies under comparison are identical with respect to health outcomes of interest. Only costs matter, and the best strategy is the least expensive one.

Cost-benefit analysis characterizes costs and health outcomes in monetary terms. Strategies are compared on the basis of net monetary value (net benefit), which is defined as benefits minus costs.

Cost-effectiveness analysis is concerned with costs and health outcomes, and describes health outcomes in naturalistic or disease-specific units (e.g., the proportion of patients showing improvement according to a standard set of criteria). Strategies are evaluated on cost-effectiveness (28). The standard measure of cost-effectiveness is the incremental cost-effectiveness ratio (ICER), a summary statistic defined as ΔC/ΔE, where ΔC is the incremental cost (i.e., the difference in costs) and ΔE is the incremental effectiveness (i.e., the difference in health outcomes) in the comparison of a strategy of interest versus a baseline, or reference, strategy. When one strategy produces greater health outcomes than another but at greater cost, threshold (or tradeoff) ratios, or societal willingness-to-pay cutoff points, are used to arrive at a choice among strategies.

Cost-utility analysis, often considered a subtype of cost-effectiveness analysis, also addresses costs and health outcomes. But whereas health outcome measures in cost-effectiveness analysis relate to one aspect of a patient's well-being, health outcome measures in cost-utility analysis attempt to capture all aspects of well-being in a single composite value (26, 27, 29). The most widely accepted measure of this kind is the quality-adjusted life year (QALY) (27, 30), so the ICER estimates generated by cost-utility analysis studies are typically stated in terms of incremental cost per QALY gained. The comprehensiveness of the QALY as a health outcome measure helps explain the growth in acceptance of cost-utility analysis by health economists.

All 4 methods serve the objective of welfare maximization. Cost-minimization analysis can be thought of as a type of cost-effectiveness analysis. In turn, with the use of an ICER threshold value, cost-effectiveness analysis becomes identical to cost-benefit analysis. For example, if the threshold value was $50,000 per QALY gained, then a strategy with an incremental cost-effectiveness of $30,000 per QALY gained (with respect to a baseline strategy) would not be cost-effective. Equivalently, if the maximum willingness-to-pay cutoff point for each QALY gained from adopting the strategy (versus the baseline) was $50,000, the same conclusion would follow.

Note that cost analysis is an inherently different type of study. This involves the simple assessment of the cost of a health care strategy (e.g., the cost to treat a person with RA for 1 year) and, in fact, is not a true economic evaluation because it does not involve the side-by-side evaluation of competing strategies (26). For this reason, and because health outcomes are left out of consideration, decisions that rely solely on cost analysis can be erroneous. Some authorities have argued that cost-minimization analysis should never be used (31).

Guidelines for conducting economic evaluations for pharmaceuticals have been published by Drummond et al (26), the US Panel on Cost-Effectiveness in Health and Medicine (27), the Outcome Measures in Rheumatoid Arthritis Clinical Trials group (32), and the Canadian Coordinating Office for Health Technology Assessment (33). Nevertheless, evaluations can vary considerably in the specific methods used to assess the same or similar questions. Therefore, it can be difficult to compare the findings from different studies, and it is all the more important for decision-makers to know how to examine economic evaluations critically.

Economic evaluations of all types must address a set of common methodologic issues: How should health outcomes and costs be measured? How should data on health outcomes and costs be collected and analyzed? How should uncertainty in results be evaluated? Each of these topics and an overview of the challenges of translating the findings of economic evaluations to the practice of clinical medicine will be discussed below.

Measurement of Health Outcomes

  1. Top of page
  2. Introduction
  3. Health Care Economic Evaluations: A Basic Anatomy
  4. Measurement of Health Outcomes
  5. Measurement of Costs
  6. Data Collection and Analysis for Health Outcomes and Costs
  7. Evaluation of Uncertainty in Results
  8. Checklist for Reviewing a Health Care Economic Evaluation
  9. Relevance of Health Care Economic Comparisons With Clinical Practice
  10. Conclusions
  11. REFERENCES

Because the 4 types of health care economic evaluations adopt different approaches to characterizing health outcomes, they also differ in how they measure outcomes. In cost-minimization analysis, the task is relatively simple. Only costs are relevant, so measuring health outcomes is not required. In rheumatology, this study design is rarely used because the health benefit of one intervention or maneuver versus another is usually different.

In cost-benefit analysis, health outcomes are measured in monetary currency. Sometimes this will involve evaluating intangible costs, the monetary value individuals attach to different states of health, e.g., the physical, emotional, and psychological distress associated with being ill versus being healthy. Willingness-to-pay methods exist for evaluating intangible costs (8, 33), but these methods are controversial. Determining the economic value of saving a life is especially problematic, which is a chief reason that cost-benefit analysis is used more commonly in settings where determining the monetary value of a human life is not required.

In cost-utility analysis, the use of the QALY as a health outcome measure along with the inclusion of intangible costs in total costs has also stirred debate. The problem of double counting arises when a change in a person's well-being is accounted for twice: once as a component of costs in the numerator of a cost-utility analysis ICER, and a second time as an embedded component of QALYs in the denominator. An example of double counting is when an absence from work is entered into total costs as a loss in wages when, arguably, the absence has already been allowed for in the assessment of QALYs. Although the US Panel on Cost-Effectiveness in Health and Medicine advises against the inclusion of lost wages in the calculation of cost-utility ICERs (27), the practice of doing so is commonplace.

In cost-effectiveness analysis, as mentioned already, the choice of health outcome measure depends on disease context. For example, Choi et al (8) have measured health outcomes in terms of the proportion of patients with RA satisfying the American College of Rheumatology (ACR) criteria for disease-specific quality of life improvement (34). The ACR20 standard requires a 20% improvement in tender and swollen joint count, among other improvements (34). The ACR50 and ACR70 standards are similarly defined, but they involve 50% and 70% improvement thresholds, respectively. For a cost-effectiveness study of RA management strategies, the use of ACR criteria in defining health outcome measures is a natural choice because the criteria are already accepted as the gold standard for measuring efficacy in placebo-controlled, randomized clinical trials of new treatments (34).

In cost-utility analysis, QALYs represent expected years of future life weighted by values, called utilities, representing quality of life at a given time (35). If future life may be thought of as a succession of health states, then the calculation of QALYs involves multiplying the utility weight by the duration of each health state and then adding up the results. In this way, QALYs can account for a spectrum of health states through which patients may pass, including the states of full health (utility value = 1), death (utility value = 0), and health states valued worse than death (utility value <0). For example, in a cost-utility analysis of infliximab plus methotrexate combination treatment versus methotrexate alone for the treatment of RA over 1 year, the costs increased by €863 and there was a gain of 0.248 QALYs; the cost per QALY gained was €3,440 (i.e., 863/0.248 = 3,440) (11). A key advantage of using the QALY as a health outcome measure is that it enables the comparison of cost-effectiveness estimates from different disease settings. Therefore, one could compare the cost-utility of a given treatment for RA with the cost-utility of the same treatment or a different treatment for systemic lupus erythematosus, or even the cost-utility of interventions for heart disease, kidney disease, and so on. A body of literature, reviewed elsewhere (26, 30, 36), has explored the issue of utilities assessment. Two general approaches are available: direct and indirect. Direct preference measurement techniques, such as the standard gamble and time-tradeoff methods, obtain utilities by asking patients to provide assessments of their overall quality of life. Indirect preference measurement techniques, such as the Health Utilities Index (HUI) 2, the HUI 3, and the Short Form 6D, obtain utilities using questionnaires to elicit individuals' valuations of multiple attributes, or domains, of their quality of life, e.g., mobility, emotional well-being, and cognitive ability; responses are then converted into utility values using preestablished formulas. The relative strengths of using a direct versus indirect approach to preference measurement are briefly described in Figure 1. Indirect patient preference measures commonly used in rheumatology are listed in Table 1, along with the number of possible health states that these measures are, in principle, designed to account for and the range of utility values that these instruments can produce.

thumbnail image

Figure 1. Direct versus indirect preference measurement. Direct preference techniques are considered to give more theoretically robust (better grounded in theoretical principles) estimates of utility than indirect preference techniques. However, indirect preference techniques enable the study of the relationship between health state attributes and observed utilities values. Indirect preference techniques are also easier for respondents to understand and use. For instance, respondents may have to provide a self-rating for mobility. For many, this would be far simpler than participating in one of the thought experiments, such as the standard gamble, used in direct preference measurement. In a standard gamble, respondents are presented with a choice between continuing in their current state of well-being and receiving a treatment with a probability P of bringing them to a perfect state of well-being and a probability 1−P of death. Values of P are then adjusted until the respondent feels indifferent between the 2 options. The value of P corresponding to the point of indifference is recorded as a measurement of utility.

Download figure to PowerPoint

Table 1. Examples of multiattribute utility instruments*
InstrumentDomains (component attributes)No. possible health statesRange of values
  • *

    HUI = Health Utilities Index; SF-6D = Short Form 6D; EQ-5D = EuroQol 5D.

HUI2Sensation (vision, hearing, speech), mobility, emotion, cognition, self-care, pain24,000−0.03–1.00
HUI3Vision, hearing, speech, ambulation, dexterity, emotion, cognition, pain972,000−0.36–1.00
SF-6DPhysical function, role limitation, social function, payment, mental health, vitality18,0000.30–1.00
EQ-5DMobility, usual activities, self-care, pain, anxiety243−0.59–1.00

Measurement of Costs

  1. Top of page
  2. Introduction
  3. Health Care Economic Evaluations: A Basic Anatomy
  4. Measurement of Health Outcomes
  5. Measurement of Costs
  6. Data Collection and Analysis for Health Outcomes and Costs
  7. Evaluation of Uncertainty in Results
  8. Checklist for Reviewing a Health Care Economic Evaluation
  9. Relevance of Health Care Economic Comparisons With Clinical Practice
  10. Conclusions
  11. REFERENCES

Although economic evaluations of all types characterize costs in monetary units, they can differ considerably in what cost components are included in the definition of total costs (e.g., program setup, administrative, overhead costs, medication and hospitalization costs, physician followup costs, and costs relating to unpaid work absenteeism). The analytical perspective of a study determines fundamentally what costs are pertinent to the evaluation. For example, if the adopted perspective were that of a health insurer reimbursing drug costs, only the drug costs might be considered. From the broader perspective of the entire health system, the total cost of a new RA drug therapy would include the costs of medication, adverse events, pharmacy and nursing services (for the preparation of medication and monitoring), physician followup visits, and costs of traveling to keep appointments. Health system costs would not include losses incurred by the patient due to missed work, which economists refer to as lost productivity. From a patient perspective, lost salary due to missed work could be a major missing cost component if only health system costs were evaluated. In arthritis, costs due to missed work may exceed health system costs. Clearly, study perspective is a key aspect of study design that a reader should be aware of when critically reviewing an economic evaluation. An overview of different costing perspectives and the cost components that could fall under these different viewpoints is provided in Table 2.

Table 2. Examples of costing perspectives and their cost components*
PerspectiveDrug costsProgram setupAdministrationHospital costsPhysician costsLost productivity
  • *

    √ indicates that a given cost component may be included in the calculation of total costs under a given costing perspective, depending on the decision strategy being evaluated and the setting.

Patient   
Formulary   
Health insurer 
Hospital   
Health system  
Societal

Costs relating to lost productivity are called indirect costs. All other costs, such as health system costs, are called direct costs. Costing from a societal perspective involves adding together both direct and indirect costs, and “never counts as a gain what is really someone else's loss” (27). (As an illustration of this last point, government transfers to individuals should not be counted.) Established guidelines regard the societal perspective as the most comprehensive approach to costing and recommend its adoption (26, 27, 32).

Note that the distinction between direct and indirect costs, although established in well-regarded publications (26, 27), is not universally followed. For this reason, caution is advised in reviewing the costing methodology of any health care economic evaluation, and early studies in particular (37, 38).

Data Collection and Analysis for Health Outcomes and Costs

  1. Top of page
  2. Introduction
  3. Health Care Economic Evaluations: A Basic Anatomy
  4. Measurement of Health Outcomes
  5. Measurement of Costs
  6. Data Collection and Analysis for Health Outcomes and Costs
  7. Evaluation of Uncertainty in Results
  8. Checklist for Reviewing a Health Care Economic Evaluation
  9. Relevance of Health Care Economic Comparisons With Clinical Practice
  10. Conclusions
  11. REFERENCES

Once an approach to health outcomes and cost measurement has been decided upon, it is possible to contemplate an approach to data collection and analysis. The evaluation of health outcomes and costs depends very much on whether the analysis is based on a real cohort or a hypothetically defined cohort (hypothetical cohort) of study subjects. In a real cohort analysis, individual-level data on costs and health outcomes are collected on a group of patients. For example, the study sample may consist of 100 patients recruited into a placebo-controlled, randomized clinical trial for a new therapy. In a cost-effectiveness analysis designed to piggy back on the research experiment, the incremental cost and incremental effectiveness for the comparison could be calculated by summing up the costs and health outcomes across all patients in both the treatment arm and the placebo arm of the trial. Once the incremental cost and incremental effectiveness are known, of course, the ICER can also be calculated. Recent advances in cost-benefit methodology have made it possible to use patient-level data to generate results, i.e., net-benefit values, which are adjusted for potential confounders such as demographic and clinical characteristics (34, 35).

In a hypothetical cohort-based study, researchers ask, “What would happen if one strategy were chosen instead of another for a given study population?” The hypothetical nature of these investigations calls for the use of a decision-analytic model, a characterization of the world built upon assumptions made about the demographic and clinical profile of the study population of interest, and the health and cost outcomes expected to follow from the adoption of different strategies. The assumption set is typically based on the published findings of other studies, data extracted from outside administrative databases, and expert opinion.

A decision-analytic model is commonly depicted using a decision tree, a visual representation of the choices and potential consequences facing decision-makers. As shown in Figure 2, the starting point in a decision tree is a single root decision node situated on the far left side of the diagram, and the first branches extending from it represent the set of strategies under evaluation. Further subbranching lays out the blueprint of all the pathways, or possible sequences of events, resulting from the choice of a strategy. Each pathway has a probability that an individual will enter that pathway, a cost, and a value for every health outcome measure. Calculating the probability of the pathway involves multiplying together the conditional probabilities stated under the constituent subbranches that define each pathway, where a conditional probability is the likelihood that, having arrived at a given location in the tree, a certain event will occur. (For the whole group of subbranches that emerge from a given probability node, or junction in the decision tree, the sum of conditional probabilities always equals 1.)

thumbnail image

Figure 2. Decision analytic model example. In this decision tree, a choice is represented between strategy A and strategy B. Both strategies lead to 1 of 2 possible health states, labeled 1 and 2. Under strategy A, health state 2 is more likely to be the result and health state 1 progresses further to 1 of 2 possible substates. Overall, strategy A is more costly than B, but yields more quality-adjusted life years (QALYs). The incremental cost per QALY gained for strategy A, as compared with B, is $68.28 per QALY gained.

Download figure to PowerPoint

In the evaluation of costs and health outcomes in a model-based study comparing strategies with uncertain consequences, the concept of expected value is commonly relied upon. The expected value of an uncertain situation is the probability-weighted average of all possible outcomes. For example, for a lottery ticket (received as a birthday gift) with a 1 in 100 chance of winning $1 and a 1 in 1 million chance of winning $2 million, the expected value would be calculated as follows:

  • equation image

Figure 2 illustrates these concepts with a cost-utility analysis example involving the comparison of a strategy of interest, strategy A, and the baseline strategy defined as strategy B.

A modeling component is often incorporated into a real cohort analysis to address the problem of missing individual-level data. For instance, because RA is a chronic, incurable disease, the costs and health effects of RA management strategies are spread over a lengthy time horizon. Patient-level data on the cost and effectiveness of new RA drug therapies, however, are typically available only from short-term clinical trials (28, 32). In such situations, modeling is often used to estimate long-term costs and health outcomes, thereby filling the gap.

Future discounting, another basic aspect of outcomes evaluation, is the practice of assigning less value to health and cost outcomes depending on how distant into the future they occur. Future discounting is important because people generally prefer that desirable events (e.g., a 1-year remission from disease) occur sooner rather than later, and the reverse generally holds for undesirable events (e.g., paying off the balance on one's credit cards).

Technically, the future discounted value of an outcome occurring n years from now is calculated by multiplying the outcome's value (e.g., $12,000 for a hospitalization) by the factor (1 + r)−n. The variable r is the future discount rate. Often, outcomes of interest are probabilistically distributed over time, such as the incidence of progression to advanced-stage disease. In these situations, Markov modeling is a common technique for keeping a running tally of outcome values while accounting for future discounting and competing events such as background mortality (39). Markov modeling is described in greater detail in Figure 3.

thumbnail image

Figure 3. Markov process modeling. In a Markov model typically found in a health care evaluation, a hypothetical cohort of subjects is considered to move, in a stylized way, through a number of well-defined, nonoverlapping health states over time. Three elements define the Markov process: a set of health states (e.g., levels of rheumatoid arthritis severity defined by Health Assessment Questionnaire score increments); an initial statistical distribution across these states; and a set of transition probabilities for the likelihood that an individual, in a given period (e.g., 1 year), will move from one state to another or remain in the same state. With these 3 elements known, the health state distribution of the cohort can be calculated for any desired point in the future. The different types of circles represent 3 different Markov states. The 2-headed arrows drawn between the circles represent the transition of individuals between different Markov states, and values inside each circle correspond to the percentage of the population found in a given state at a given moment in between transitions.

Download figure to PowerPoint

Summary statistics serve as the touchstone for comparing strategies and drawing conclusions from the results, and different types of economic evaluations use different types of summary statistics. In cost-benefit analysis, the summary statistic is net monetary benefit. For example, if the lottery ticket described above were on sale for $3, would it be worth buying? From a cost-benefit perspective, the answer is determined by subtracting the expected benefit of the ticket ($2.01) from the expected cost ($3). The result, −$0.99, indicates that buying the ticket would be a bad deal. (The example ignores the value that individuals may place on the inherent riskiness of making a wager, but willingness-to-pay elicitation methods, referred to already, offer a way of capturing this value as well.)

To interpret the summary statistic of a cost-effectiveness study, the ICER, more explanation is needed. The Results section of a cost-effectiveness analysis contained the following statement: “The incremental cost of etanercept over a 6-month period was US $41,900 per ACR20 versus sulfasalazine” (7). What this implies is that etanercept was more effective but more costly than sulfasalazine. Is $41,900 an acceptable price? As mentioned earlier, answering this question requires a decision-maker to formulate, either explicitly or not, an ICER threshold (λ) representing the maximum cost considered acceptable for an incremental unit of health benefit. Hence, the etanercept treatment strategy would be cost-effective for values of λ greater than $41,900. In contrast, the strategy would not be cost-effective for values of λ less than $41,900. Determining a value for λ typically involves a review of studies that evaluate the cost-effectiveness of commonly funded strategies and report ICER values in relevant health outcome terms, e.g., of incremental cost per ACR20.

In a cost-effectiveness or cost-utility analysis of multiple new drug treatments that are both more costly and more effective than the reference treatment, the new drug with the lowest ICER value would be the best. Of course, if a new drug were more effective and less costly than the current treatment, its ICER would be negative and it would be called dominant because of its categorical superiority over the reference treatment. For technical reasons, ranking dominant strategies by comparing their negative ICER values alone is not possible (37).

Evaluation of Uncertainty in Results

  1. Top of page
  2. Introduction
  3. Health Care Economic Evaluations: A Basic Anatomy
  4. Measurement of Health Outcomes
  5. Measurement of Costs
  6. Data Collection and Analysis for Health Outcomes and Costs
  7. Evaluation of Uncertainty in Results
  8. Checklist for Reviewing a Health Care Economic Evaluation
  9. Relevance of Health Care Economic Comparisons With Clinical Practice
  10. Conclusions
  11. REFERENCES

The findings of any health care economic evaluation, whether based on patient-level data or a decision analytic model, have uncertainty. Older studies have tended to disregard the importance of uncertainty in results, often presenting ICER values as single fixed values, or point estimates. Recent methodologic advances, however, have motivated the gradual adoption of more sophisticated approaches to the measurement and reporting of uncertainty in results.

In a real cohort-based health economic evaluation, uncertainty exists in making inferences from only a sample of individuals from a study population. To measure this uncertainty, a common practice is to construct 95% confidence intervals (95% CIs) around the estimates of incremental cost and incremental effectiveness calculated using the patient-level data from the study sample. Estimating a 95% CI for ICER estimates is also of interest, but is more technically challenging because incremental cost and incremental effectiveness are often correlated. Fortunately, quantitative methods exist for overcoming this hurdle (i.e., nonparametric bootstrapping and Fieller's theorem) (37, 40, 41).

Hypothetical cohort-based evaluations have traditionally used sensitivity analysis to assess uncertainty. In one-way sensitivity analysis, 1 variable is varied at a time through a range of values. The resulting set of new outcome and summary statistic estimates is then examined. In two-way sensitivity analysis, 2 variables are varied simultaneously instead of just 1. In probabilistic sensitivity analysis, all model parameters are varied simultaneously over a large number of draws, or simulated trials called Monte Carlo simulations (42). This makes possible the quantification of overall uncertainty in the results, and the construction of 95% CIs for the results. The use of probabilistic sensitivity analysis has grown over recent years (42, 43).

Among cost-effectiveness and cost-utility studies, the cost-effectiveness acceptability curve has also grown in use as a means of representing uncertainty, which complements, if not improves upon, the use of confidence intervals. As shown in Figure 4, an acceptability curve plots the probability that a strategy of interest is cost-effective compared with baseline over a range of ICER thresholds (37, 44).

thumbnail image

Figure 4. Cost-effectiveness acceptability curve. Once the uncertainty of an incremental cost-effectiveness ratio (ICER) estimate has been evaluated, an acceptability curve can be drawn showing the relationship between ICER threshold values (on the x-axis) and cost-effectiveness probabilities (on the y-axis). According to this acceptability curve, if a decision-maker determines that the maximum willingness-to-pay (e.g., per quality-adjusted life year gained) is $500, the probability that strategy A, compared with strategy B, is cost-effective is 90%.

Download figure to PowerPoint

One advantage of the acceptability curve is that it provides a way of presenting results without having to report negative ICER values, which are open to misinterpretation because (among other reasons) they can be the result of a negative incremental cost and a positive incremental effect (favoring a given strategy over baseline) or a positive incremental cost and negative incremental effect (favoring the baseline strategy instead). Acceptability curves are also arguably easier to interpret than confidence intervals for decision-makers. A 95% CI provides a range of values that may or may not include the true ICER value of interest. If the experiment of interest were repeated a large number of times (by resampling from the study population, or running a simulation experiment), the set of 95% CIs constructed from the results of each repetition would include the true ICER value 19 out of every 20 times.

Because of the awkwardness of this interpretation, and the fact that ICER confidence intervals do not incorporate the willingness to pay of decision-makers for health outcomes, results reported in terms of ICER values and confidence intervals do not provide decision-makers with clear recommendations for action. In contrast, acceptability curves concisely state the likelihood that, given a willingness-to-pay threshold for an incremental unit of health outcome benefit, a given strategy would be cost-effective (37).

Checklist for Reviewing a Health Care Economic Evaluation

  1. Top of page
  2. Introduction
  3. Health Care Economic Evaluations: A Basic Anatomy
  4. Measurement of Health Outcomes
  5. Measurement of Costs
  6. Data Collection and Analysis for Health Outcomes and Costs
  7. Evaluation of Uncertainty in Results
  8. Checklist for Reviewing a Health Care Economic Evaluation
  9. Relevance of Health Care Economic Comparisons With Clinical Practice
  10. Conclusions
  11. REFERENCES

A checklist of basic questions to ask when dissecting a health care economic evaluation is presented in Table 3. Drummond et al have a similar checklist that is much more comprehensive (26).

Table 3. Checklist of things to look for in an economic analysis
What strategies were compared?
How were they compared (cost-minimization analysis, cost-benefit analysis, cost-effectiveness analysis, or cost-utility analysis)?
What was the study setting? (Who were the study subjects and what population were they intended to represent? What information was provided about the location and time of the study?)
How were health outcomes defined and measured?
How were costs defined and measured? What perspective (e.g., societal) was adopted for costing? Were all relevant costs for each strategy captured?
Was discounting applied to future health outcomes and costs?
What summary measure (e.g., incremental cost-effectiveness ratio) was used to evaluate the strategies of interest and, based on this measure, how did the strategies compare with each other?
What variables (e.g., the risk and cost of hospitalization) were key drivers of results? For hypothetical cohort studies, was uncertainty in these variables appropriately accounted for in sensitivity analysis?
What were the implications of the study findings to other study populations (e.g., higher-risk or lower-risk study populations)?
How was the overall uncertainty in results measured and reported, if at all?
What should decision-makers conclude from the results of the study?

Relevance of Health Care Economic Comparisons With Clinical Practice

  1. Top of page
  2. Introduction
  3. Health Care Economic Evaluations: A Basic Anatomy
  4. Measurement of Health Outcomes
  5. Measurement of Costs
  6. Data Collection and Analysis for Health Outcomes and Costs
  7. Evaluation of Uncertainty in Results
  8. Checklist for Reviewing a Health Care Economic Evaluation
  9. Relevance of Health Care Economic Comparisons With Clinical Practice
  10. Conclusions
  11. REFERENCES

The task of translating health care economic comparison research to the practice of clinical medicine is challenging for many reasons, and is itself an area of continuing discussion and debate (45, 46). First, for clinicians who regard that demanding the best possible care for every patient whatever the cost is a professional responsibility, there is reluctance to allow monetary considerations to enter into the management of patient care at all (47). A counterargument to this position, however, is that physician decisions have consequences for the use of limited resources affecting other physicians' patients, if not their own patients. Hence, the role of economic evaluation in clinical medicine may be justified on the grounds that the medical decisions of health care professionals have opportunity costs that fall upon all patients as a whole. Opportunity cost is a term from economics meaning the value of the best available alternative to a given decision (48).

A second issue is resistance to the increasing relative prominence of evidence-based medicine, “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients” (49), compared with the more traditional practice of medicine grounded in the physician-patient relationship and the seasoned judgment and expertise of medical practitioners earned through experience in clinical settings (50). A common concern is that study populations in published studies do not resemble the typical individual patient under a physician's care (51, 52). Another concern is that evidence-based medicine will encroach upon physician autonomy and patient choice (53, 54). These same issues are relevant to all health care economic evaluations because these evaluations belong to the body of current best evidence on which evidence-based medicine is founded.

A final challenge lies in the interpretability and validity of health care economic evaluation studies. Cost-minimization analysis has strong validity because it sidesteps the problem of having to measure health outcomes, but its inapplicability to most economic comparisons is a critical limitation (31). Cost-benefit analysis values health outcomes in monetary units, but no consensus exists on how this should be done, if it should be done at all (30, 43, 55). Cost-effectiveness analyses compare strategies according to cost-effectiveness and construct ICER estimates based on context-specific health outcome measures, thereby circumventing the problems associated with measuring health outcomes monetarily as in cost-benefit analysis. But a key drawback is that results are not comparable across disease settings, i.e., one cannot compare the cost per ACR20 for RA with the cost of 1 averted renal failure or with the value of 1 life year gained. In cost-utility analysis, the QALY is a versatile health outcome measure designed to capture differences in life expectancy and quality of life. In the context of arthritis, where interventions improve quality of life more than they prolong life, the QALY may seem an attractive health outcome measure. But QALYs have been criticized for being difficult to interpret and too dependent on the choice of approach used for eliciting utilities, among other shortcomings (43, 56–59).

Despite the challenges mentioned above, a transition to a more evidence-based model of clinical practice has advanced gradually, propelled by the potential of evidence-based medicine and health care economic evaluation research to improve the quality, consistency, and efficiency of patient care. For the rheumatologist, these changes may lead to desired improvements in health care equity as well, because cost-benefit and cost-utility analyses provide means of more objectively weighing the societal value of interventions in RA, which otherwise might seem to have less glamorous health outcomes (e.g., long-term quality of life improvements and improvements in productivity) than interventions in other disease settings (e.g., human immunodeficiency virus/acquired immunodeficiency syndrome, cancer, or coronary artery disease).

Conclusions

  1. Top of page
  2. Introduction
  3. Health Care Economic Evaluations: A Basic Anatomy
  4. Measurement of Health Outcomes
  5. Measurement of Costs
  6. Data Collection and Analysis for Health Outcomes and Costs
  7. Evaluation of Uncertainty in Results
  8. Checklist for Reviewing a Health Care Economic Evaluation
  9. Relevance of Health Care Economic Comparisons With Clinical Practice
  10. Conclusions
  11. REFERENCES

Health care economic evaluations offer important guidance to the management of limited health care resources and medical practice. Whether this carries implications for day-to-day clinical decision-making directly, or through clinical practice guidelines formulated by panels of experts, there is a need for rheumatology clinicians to be able to understand the methods and interpret and critique the findings of these studies. The purpose of this primer was to provide an introduction to these issues, and in doing so provide for rheumatology clinicians greater familiarity with a set of research approaches used increasingly to investigate the relationship between costs and health consequences of medical decisions.

REFERENCES

  1. Top of page
  2. Introduction
  3. Health Care Economic Evaluations: A Basic Anatomy
  4. Measurement of Health Outcomes
  5. Measurement of Costs
  6. Data Collection and Analysis for Health Outcomes and Costs
  7. Evaluation of Uncertainty in Results
  8. Checklist for Reviewing a Health Care Economic Evaluation
  9. Relevance of Health Care Economic Comparisons With Clinical Practice
  10. Conclusions
  11. REFERENCES
  • 1
    WHO Scientific Group on the Burden of Musculoskeletal Conditions at the Start of the New Millennium. The burden of musculoskeletal conditions at the start of the new millennium. World Health Organ Tech Rep Ser 2003; 919: i218.
  • 2
    Yelin E, Callahan LF, and the National Arthritis Data Work Group. The economic cost and social and psychological impact of musculoskeletal conditions. Arthritis Rheum 1995; 38: 135162.
  • 3
    Ward MM, Javitz HS, Yelin EH. The direct cost of rheumatoid arthritis. Value Health 2000; 3: 24352.
  • 4
    Bae SC, Corzillius M, Kuntz KM, Liang MH. Cost-effectiveness of low dose corticosteroids versus non-steroidal anti-inflammatory drugs and COX-2 specific inhibitors in the long-term treatment of rheumatoid arthritis. Rheumatology (Oxford) 2003; 42: 4653.
  • 5
    Brennan A, Bansback N, Reynolds A, Conway P. Modelling the cost-effectiveness of etanercept in adults with rheumatoid arthritis in the UK. Rheumatology (Oxford) 2004; 43: 6272.
  • 6
    Bansback NJ, Brennan A, Ghatnekar O. Cost effectiveness of adalimumab in the treatment of patients with moderate to severe rheumatoid arthritis in Sweden. Ann Rheum Dis 2005; 64: 9951002.
  • 7
    Choi HK, Seeger JD, Kuntz KM. A cost-effectiveness analysis of treatment options for patients with methotrexate-resistant rheumatoid arthritis. Arthritis Rheum 2000; 43: 231627.
  • 8
    Choi HK, Seeger JD, Kuntz KM. A cost effectiveness analysis of treatment options for methotrexate-naive rheumatoid arthritis. J Rheumatol 2002; 29: 115665.
  • 9
    Hartman M, van Ede A, Severens JL, Laan RF, van de Putte L, van der Wilt GJ. Economic evaluation of folate supplementation during methotrexate treatment in rheumatoid arthritis. J Rheumatol 2004; 31: 9028.
  • 10
    Jobanputra P, Barton P, Bryan S, Burls A. The effectiveness of infliximab and etanercept for the treatment of rheumatoid arthritis: a systematic review and economic evaluation. Health Technol Assess 2002; 6: 1110.
  • 11
    Kobelt G, Jonsson L, Young A, Eberhardt K. The cost-effectiveness of infliximab (Remicade) in the treatment of rheumatoid arthritis in Sweden and the United Kingdom based on the ATTRACT study. Rheumatology (Oxford) 2003; 42: 32635.
  • 12
    Kobelt G, Eberhardt K, Geborek P. TNF inhibitors in the treatment of rheumatoid arthritis in clinical practice: costs and outcomes in a follow up study of patients with RA treated with etanercept or infliximab in southern Sweden. Ann Rheum Dis 2004; 63: 410.
  • 13
    Kobelt G, Lindgren P, Singh A, Klareskog L. Cost effectiveness of etanercept (Enbrel) in combination with methotrexate in the treatment of active rheumatoid arthritis based on the TEMPO trial. Ann Rheum Dis 2005; 64: 11749.
  • 14
    Lee KK, You JH, Ho JT, Suen BY, Yung MY, Lau WH, et al. Economic analysis of celecoxib versus diclofenac plus omeprazole for the treatment of arthritis in patients at risk of ulcer disease. Aliment Pharmacol Ther 2003; 18: 21722.
  • 15
    Maetzel A, Krahn M, Naglie G. The cost effectiveness of rofecoxib and celecoxib in patients with osteoarthritis or rheumatoid arthritis. Arthritis Rheum 2003; 49: 28392.
  • 16
    Marra CA, Esdaile JM, Anis AH. Practical pharmacogenetics: the cost effectiveness of screening for thiopurine s-methyltransferase polymorphisms in patients with rheumatological conditions treated with azathioprine. J Rheumatol 2002; 29: 250712.
  • 17
    Merkesdal S, Ruof J, Mittendorf T, Zeidler H. Cost-effectiveness of TNF-α-blocking agents in the treatment of rheumatoid arthritis. Expert Opin Pharmacother 2004; 5: 18816.
  • 18
    Moore A, Phillips C, Hunsche E, Pellissier J, Crespi S. Economic evaluation of etoricoxib versus non-selective NSAIDs in the treatment of osteoarthritis and rheumatoid arthritis patients in the UK. Pharmacoeconomics 2004; 22: 64360.
  • 19
    Oh KT, Anis AH, Bae SC. Pharmacoeconomic analysis of thiopurine methyltransferase polymorphism screening by polymerase chain reaction for treatment with azathioprine in Korea. Rheumatology (Oxford) 2004; 43: 15663.
  • 20
    Pellissier JM, Watson DJ, Kong SX, Straus WL. Cost-effectiveness of cyclooxygenase-2 inhibitors in chronic arthritis. Ann Intern Med 2004; 140: 7612.
  • 21
    Van den Hout WB, Tijhuis GJ, Hazes JM, Breedveld FC, Vliet Vlieland TP. Cost effectiveness and cost utility analysis of multidisciplinary care in patients with rheumatoid arthritis: a randomised comparison of clinical nurse specialist care, inpatient team care, and day patient team care. Ann Rheum Dis 2003; 62: 30815.
  • 22
    Van den Hout WB, de Jong Z, Munneke M, Hazes JM, Breedveld FC, Vliet Vlieland TP. Cost-utility and cost-effectiveness analyses of a long-term, high-intensity exercise program compared with conventional physical therapy in patients with rheumatoid arthritis. Arthritis Rheum 2005; 53: 3947.
  • 23
    Welsing PM, Severens JL, Hartman M, van Riel PL, Laan RF. Modeling the 5-year cost effectiveness of treatment strategies including tumor necrosis factor-blocking agents and leflunomide for treating rheumatoid arthritis in the Netherlands. Arthritis Rheum 2004; 51: 96473.
  • 24
    Wong JB, Singh G, Kavanaugh A. Estimating the cost-effectiveness of 54 weeks of infliximab for rheumatoid arthritis. Am J Med 2002; 113: 4008.
  • 25
    Yun HR, Bae SC. Cost-effectiveness analysis of NSAIDs, NSAIDs with concomitant therapy to prevent gastrointestinal toxicity, and COX-2 specific inhibitors in the treatment of rheumatoid arthritis. Rheumatol Int 2005; 25: 914.
  • 26
    Drummond M, O'Brien B, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes. 2nd ed. Oxford: Oxford University Press; 2001.
  • 27
    GoldMR, SiegelJE, RussellLB, WeinsteinMC, editors. Cost-effectiveness in health and medicine. New York: Oxford University Press; 1996.
  • 28
    Bansback NJ, Regier DA, Ara R, Brennan A, Shojania K, Esdaile JM, et al. An overview of economic evaluations for drugs used in rheumatoid arthritis: focus on tumour necrosis factor-α antagonists. Drugs 2005; 65: 47396.
  • 29
    Dolan P. The measurement of health-related quality of life for use in resource allocation decisions in health care. In: CulyerAJ, NewhouseJP, editors. Handbook of health economics. London (UK): Elsevier Science; 2000.
  • 30
    Neumann PJ, Goldie SJ, Weinstein MC. Preference-based measures in economic evaluation in health care. Annu Rev Public Health 2000; 21: 587611.
  • 31
    Briggs AH, O'Brien BJ. The death of cost-minimization analysis? Health Econ 2001; 10: 17984.
  • 32
    Maetzel A, Tugwell P, Boers M, Guillemin F, Coyle D, Drummond M, et al, and the OMERACT 6 Economics Research Group. Economic evaluation of programs or interventions in the management of rheumatoid arthritis: defining a consensus-based reference case. J Rheumatol 2003; 30: 8916.
  • 33
    Guidelines for economic evaluation of pharmaceuticals: Canada. 2nd ed. Ottawa: The Canadian Coordinating Office for Health Technology Assessment; 1997.
  • 34
    Felson DT, Anderson JJ, Boers M, Bombardier C, Furst D, Goldsmith C, et al. American College of Rheumatology preliminary definition of improvement in rheumatoid arthritis. Arthritis Rheum 1995; 38: 72735.
  • 35
    Weinstein MC, Stason WB. Foundations of cost-effectiveness analysis for health and medical practices. N Engl J Med 1977; 296: 71621.
  • 36
    Kopec JA, Willison KD. A comparative review of four preference-weighted measures of health-related quality of life. J Clin Epidemiol 2003; 56: 31725.
  • 37
    Briggs AH, O'Brien BJ, Blackhouse G. Thinking outside the box: recent advances in the analysis and presentation of uncertainty in cost-effectiveness studies. Annu Rev Public Health 2002; 23: 377401.
  • 38
    Hoch JS, Briggs AH, Willan AR. Something old, something new, something borrowed, something blue: a framework for the marriage of health econometrics and cost-effectiveness analysis. Health Econ 2002; 11: 41530.
  • 39
    Weinstein MC, Fineberg HV, Elstein AS, Frazier HS, Neuhauser D, Neutra RR, et al. Clinical decision analysis. Philadelphia: WB Saunders; 1980.
  • 40
    Efron B, Tibshirani R. An introduction to the bootstrap. New York: Chapman & Hall; 1993.
  • 41
    Fieller EC. Some problems in interval estimation. J R Stat Soc Ser B 1954; 16: 17583.
  • 42
    Briggs AH. Handling uncertainty in cost-effectiveness models. Pharmacoeconomics 2000; 17: 479500.
  • 43
    Coast J. Is economic evaluation in touch with society's health values? BMJ 2004; 329: 12336.
  • 44
    Fenwick E, Claxton K, Sculpher M. Representing uncertainty: the role of cost-effectiveness acceptability curves. Health Econ 2001; 10: 77987.
  • 45
    Maetzel A. Cost-effectiveness analysis: out of touch with clinical reality? Arthritis Rheum 2005; 53: 34.
  • 46
    Wolfe F, Michaud K, Pincus T. Do rheumatology cost-effectiveness analyses make sense? Rheumatology (Oxford) 2004; 43: 46.
  • 47
    Detsky AS, Naglie IG. A clinician's guide to cost-effectiveness analysis. Ann Intern Med 1990; 113: 14754.
  • 48
    Varian H. Microeconomic analysis. 3rd ed. W. W. Norton & Company; 1992.
  • 49
    Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ 1996; 312: 712.
  • 50
    Patterson K. What doctors don't know (almost everything). N Y Times Mag [serial online]. 2002; May 5. URL: http://www.nytimes.com.
  • 51
    Charlton BG. Restoring the balance: evidence-based medicine put in its place. J Eval Clin Pract 1997; 3: 8798.
  • 52
    Sleigh JW. Logical limits of randomized controlled trials. J Eval Clin Pract 1997; 3: 1458.
  • 53
    Ginsburg ME, Kravitz RL, Sandberg WA. A survey of physician attitudes and practices concerning cost-effectiveness in patient care. West J Med 2000; 173: 3904.
  • 54
    Timmermans S, Mauck A. The promises and pitfalls of evidence-based medicine. Health Aff (Millwood) 2005; 24: 1828.
  • 55
    Moayyedi P, Duffet S, Soo S, Axon AT. The appropriateness of measuring the benefit of dyspepsia management in terms of quality adjusted life years [abstract]. Gastroenterology 2001; 120: A239.
  • 56
    Marra CA, Esdaile JM, Guh D, Kopec JA, Brazier JE, Koehler BE, et al. A comparison of four indirect methods of assessing utility values in rheumatoid arthritis. Med Care 2004; 42: 112531.
  • 57
    Marra CA, Woolcott JC, Kopec JA, Shojania K, Offer R, Brazier JE, et al. A comparison of generic, indirect utility measures (the HUI2, HUI3, SF-6D, and the EQ-5D) and disease-specific instruments (the RAQoL and the HAQ) in rheumatoid arthritis. Soc Sci Med 2005; 60: 157182.
  • 58
    Moayyedi P, Mason J. Cost-utility and cost-benefit analyses: how did we get here and where are we going? Eur J Gastroenterol Hepatol 2004; 16: 52734.
  • 59
    Ubel PA, Loewenstein G, Scanlon D, Kamlet M. Individual utilities are inconsistent with rationing choices: a partial explanation of why Oregon's cost-effectiveness list failed. Med Decis Making 1996; 16: 10816.