Overview of methods in economic analyses of behavioral interventions to promote oral health

Authors

  • Joan M. O'Connell PhD,

    Corresponding author
    1. Centers for American Indian and Alaska Native Health, Colorado School of Public Health, University of Colorado Denver, Aurora, CO
    Search for more papers by this author
  • Susan Griffin PhD

    1. Division of Oral Health, National Center for Chronic Disease Prevention and Promotion, Centers for Disease Control and Prevention, Atlanta, GA
    Search for more papers by this author

Dr. Joan M. O'Connell, Centers for American Indian and Alaska Native Health, Colorado School of Public Health, University ofColorado Denver, Aurora, CO 80045. Tel.: 303-724-1459; Fax: 303-724-1461; e-mail: joan.oconnell@ucdenver.edu. Joan M. O'Connell is with the, Centers for American Indian and Alaska Native Health, Colorado School of Public Health, University of Colorado Denver. Susan Griffin is with the Division of Oral Health, National Center for Chronic Disease Prevention and Promotion, Centers for Disease Control and Prevention.

Abstract

Background: Broad adoption of interventions that prove effective in randomized clinical trials or comparative effectiveness research may depend to a great extent on their costs and cost-effectiveness (CE). Many studies of behavioral health interventions for oral health promotion and disease prevention lack robust economic assessments of costs and CE.

Objective: To describe methodologies employed to assess intervention costs, potential savings, net costs, CE, and the financial sustainability of behavioral health interventions to promote oral health.

Methods: We provide an overview of terminology and strategies for conducting economic evaluations of behavioral interventions to improve oral health based on the recommendations of the Panel of Cost-Effectiveness in Health and Medicine. To illustrate these approaches, we summarize methodologies and findings from a limited number of published studies. The strategies include methods for assessing intervention costs, potential savings, net costs, CE, and financial sustainability from various perspectives (e.g., health-care provider, health system, health payer, employer, society). Statistical methods for estimating short-term and long-term economic outcomes and for examining the sensitivity of economic outcomes to cost parameters are described.

Discussion: Through the use of established protocols for evaluating costs and savings, it is possible to assess and compare intervention costs, net costs, CE, and financial sustainability. The addition of economic outcomes to outcomes reflecting effectiveness, appropriateness, acceptability, and organizational sustainability strengthens evaluations of oral health interventions and increases the potential that those found to be successful in research settings will be disseminated more broadly.

Introduction

In addition to understanding the effectiveness of interventions to prevent and control oral disease, health providers, payers, and policymakers need reliable information about intervention costs and cost-effectiveness (CE) if they are to make informed decisions about allocating resources. With health-care costs increasing rapidly, translation of interventions documented to be effective in research settings may be limited if reliable and accurate estimates of costs and CE are not available. Although the CE of community water fluoridation and dental sealants has been documented (1-9), and CE research on dental procedures in clinical settings is growing (10-15), research on the CE of interventions implemented to improve oral health knowledge, attitudes, and behaviors is limited. Because health behaviors and lifestyle influence oral health and long-term health, and have economic and social consequences, it is important to identify effective and cost-effective behavioral interventions to promote oral health and to reduce documented oral health disparities (16).

Economic analyses may vary with regard to types of costs measured, how costs are determined, methods employed to assess potential intervention savings, and documentation. This variability may reflect differences in resources allocated to cost analyses, or unfamiliarity with methods employed in conducting such studies, as there are few such studies of behavioral interventions implemented to improve oral health. The Guide to Community Preventive Services (17) typically conducts systematic reviews of the CE of an intervention after the effectiveness of the intervention has been established. Synthesizing CE ratios has been proven problematic because methods and reporting may vary across studies (18). Conducting the economic analysis while studying the effectiveness of the intervention may encourage the use of established protocols consistent with the recommendations of the Panel of Cost-Effectiveness in Health and Medicine (Panel) (19); in turn, such practices could improve the quality and uniformity of economic analyses (18,20). The findings may be used for dissemination and translation of effective interventions and may contribute to the understanding of factors influencing intervention effectiveness, CE, and components that may be altered to improve both.

This paper provides an overview of strategies for conducting economic evaluations of behavioral interventions for oral health promotion and disease prevention. Because it will not be possible to address every aspect of this topic or to provide detailed case studies, we reference books on economic analyses of health interventions (19,21-26). As this paper is structured to provide a framework for readers who may have little familiarity with economic analyses, the section Overview of economic terminology presents key concepts, including an overview of four types of economic analyses. The section Factors that influence the study design describes factors that influence the study design. Then we follow a project lifecycle and examine issues related to research design in the section Research design; a discussion of methods follows in the section Discussion. The section Research design includes information on measures, data collection, and data analysis. To illustrate the described research methods, we refer to selected studies in the analysis section. These studies include behavioral interventions addressing oral health in clinical and nonclinical settings (27-31) and two studies, one of community water fluoridation programs (CWFP) (2) and another of school-based dental sealant programs (SBSP) (5), that do not focus on behavior change yet include strategies for assessing oral health costs and savings (see Table 1).

Table 1.  Clinical and Economic Effectiveness of Interventions Implemented to Improve Oral Health
Author and yearGrant et al. 2007 (30)Wennhall et al., 2008 and 2010 (29,53)Kobayashi et al., 2005 (27)O'Connell et al., 2005 (2)Scherrer et al., 2007 (5)Hietasalo et al., 2009 (31)Kowash et al., 2006 (28)
  • * 

    Intervention savings are derived from estimates of the difference in costs for education, preventive services, and treatment services for children in both study populations.

  • † 

    Information concerning an inflation adjustment was not provided.

  • CE, cost-effectiveness; CI, 95% confidence interval; CR, credible range, estimated from a probabilistic sensitivity analysis conducted using boot-strapped resamples or simulations conducted with enough repetitions to ensure the standard error of the estimates is less than 3%; DEFS, decayed, extracted, filled surfaces; DMFS, decayed, missing, filled surfaces; DMFT, decayed, missing, filled teeth; DFS, decayed, filled surfaces; SCHIP, State Children's Health Insurance Program.

General Information       
 Type of studyObservationalObservationalObservationalDecision cost modelDecision cost modelRandomized clinical trialRandomized clinical trial
 InterventionIntervention: Parent oral health counseling, child dental screening and fluoride varnish applicationIntervention: Comprehensive oral health outreach and preventive program that included education on diet and oral hygiene, Comparison: Care as usual (historic reference group)Intervention: Community-based program included community outreach, parent and dental professional education, child referrals for services, preventive and treatment services, and higher reimbursement for trained dentists; Comparison: Usual careIntervention: Community water fluoridation programs, Comparison: No programIntervention: School-based dental sealant programs, Comparison: no programExperimental group: Patient-centered education on oral hygiene and nutrition, preventive services, clinical exams, and referrals for treatment; children averaged 3-4 visits over a 12-month period, community and school oral health promotion; Control group: Usual care, community and school oral health promotionExperimental group: Health educators provided education on oral hygiene and nutrition and dental screenings during home visits conducted at varying frequency (Groups A-D); Control group: Usual care
 LocationNorth Carolina, United StatesMalmo, SwedenState of Washington, United StatesColorado, United States7 states in the United StatesPori, FinlandLeeds, United Kingdom
 SettingUniversity pediatric clinicOutreach facilityCommunityCommunitySchoolsPublic dental clinicsHome-based
 Target populationChildren aged 6-36 months, Intervention group n = 665High-risk children living in a low-socioeconomic multicultural area, Intervention group: Children aged 2 years n = 651, Historic controls n = 201Children aged 6 years and younger, Intervention group: children in the intervention county (approximately 20,000 served 1997-1999), Comparison group: children in a county without the interventionPersons age 5 years and olderChildren 7-9 years old (children in second grade)Children ages 11-12 years with at least one active initial caries lesion, Experimental group n = 250, Control group n = 247Mothers of infants age 8 months living in a low-socioeconomic area with high caries prevalence, Intervention group n = 228 children, Control group n = 55 children
 Health outcomeNot providedOral health status of children age 5 years, Intervention group: 8.2 DEFS, Reference group: 11.2 DEFSOral health status of children in third grade in each county in 2002 (n = 453), Intervention county: ratio of DFS to all erupted surfaces: 0.1, Comparison county ratio: 0.2CWFPs reduced the decay increment by approximately 25%Averted caries estimated from an annual DMFT attack rate of 0.132 over the 9 year period, adjusted for the annual sealant retention rate of 90%Experimental group: 2.56 DMFS, Control group: 4.60 DMFS, Incremental effectiveness: 2.04 averted DMFS (CI: 1.26-2.82)Intervention groups: 0.29 DMFS for Group A, 0 DMFS for Groups B-D, 3% of children in Groups A-D had gingivitis; Control group: 1.75 DMFS, 16% had gingivitis
 Time frame and analytic horizonTime frame and analytic horizon: 31 months (12/2001-7/2004)Time frame and analytic horizon: 3 years (1998-2000)Time frame and analytic horizon: 7 years (1995-2001)Time frame: 1 year, Analytic Horizon: LifetimeTime frame: 1 year, Analytic Horizon: 9 yearsTime frame and analytic horizon: 3.4 years (2001-2005)Time frame and analytic horizon: 3 years, participants recruited in 1995
Economic Information       
 PerspectiveHealth-care providerHealth provider, health payer (government)Health-care provider, health payerSocietyHealth-care provider, health payer (government), societyHealth-care provider, health payer (government)Health-care provider, health payer (government)
 Economic outcomesIntervention costs: $4,951, reimbursement for intervention services: $51,992, net program costs: −$47,041Intervention cost per child: €310, Net cost per child including treatment revenue: €30 (CI: €109 to −61 (cost savings))Mean annual intervention costs per child for birth cohort (born in 1994 or 1995): $5.33, Mean annual Medicaid dental expenditures for birth cohort: intervention county −$207 and comparison county −$199, Mean annual net costs of health-care provider and Medicaid: $13.50Annual net costs $148.9 million (CR: $115.1-187.2 million), net costs per person $60.78 (CR: $46.97-76.41)Results for Wisconsin: Health-care provider cost per child sealed $20.51, Annual state net cost savings (includes Medicaid/SCHIP reimbursement) −$55,290, Annual societal net cost savings: $295,421-393,628Experimental group cost per child: €496.45, Control group cost per child: €426.95, Incremental cost per child: €69.50 (CR: 28.25-110.75), Incremental cost-effectiveness: €34.07 per averted DMFSAnnual cost estimates for a steady-state year: intervention costs: £6,445, Intervention savings: £36,386, Benefit-cost (intervention costs/savings) ratio: 5.6, Intervention costs per averted DMFS: £1.8
 Measurement of intervention costsMicro-costing: Obtained intervention costs related to labor and dental supplies from a retrospective chart audit of encounter forms to obtain clinical and financial (reimbursement) dataMicro-costing: Obtained prevention and treatment service costs for rental facilities, equipment, and supplies; personnel costs were estimated based on dental procedure data, salaries, and estimates for program management; overhead costs estimated to be 50% of salariesMicro-costing: Obtained intervention costs for dental professional training and community outreach and marketing; Cost estimates for preventive and treatment services and increased Medicaid reimbursement for trained providers obtained from Medicaid administrative records for birth cohortGross-costing: Used published results on program costs and state data on water system fluoride levels and population sizeGross-costing: Used published data and data obtained from data on sealant program utilization and costs (e.g., screening rates, direct and indirect costs of labor, equipment, dental supplies, and travel), used published information for 4 states and conducted interviews with personnel in 3 other states, excluded administrative costsMicro-costing: Assessed intervention costs and dental treatment costs; costs for labor (salaries and benefits), dental supplies, capital equipment, and overhead were included; costs allocated to services based on assigned treatment weights; costs of community health promotion excludedMicro-costing: Assessed intervention costs for labor (salaries), dental and education supplies, and travel
 Measurement of intervention savingsNot addressedThe methodology described above included costs for treatment services.* Additional information obtained to estimate revenue for avoided treatmentThe methodology described above included costs for treatment services. Thus, no additional data collection was conducted*Gross-costing: Published results and findings from secondary data analysis were used to estimate averted treatment costs for applying and maintaining a restoration (e.g., single-surface amalgam, multi-surface resin-based composite, crown) over a lifetime, household direct and indirect costs related to time and travel were includedGross-costing: Published results used to estimate averted treatment costs over the average sealant life (9 years) based on use of a single surface amalgam or resin-based composite restoration, household direct and indirect costs related to time and travel were includedThe methodology described above included costs for treatment services. Thus, no additional data collection was conducted*Averted treatment costs were estimated from child DMFS results, assumptions concerning treatment, and published fees for dental procedures
 Base year and currencyUS dollar2008, Euro1995, US dollar2003, US dollar2003-2004 academic year, US dollar2004, EuroPound
 Discount rateNot employed3%Not employed3%3%Not employedNot employed
 Sensitivity analysesNoYesNoYesYesYesNo
 Decision analysis and/or probabilistic sensitivity analysis softwareNot employedNot employedNot employedTreeAge Pro 2005AutoMod 12.0R version 2.8.1Not employed

This overview aims to provide information both for those planning and conducting such studies and for experts in health planning and policy who would like an improved understanding of economic findings reported in published studies. The approaches described in this paper may also be used in studies of other oral health interventions and of health policy and reimbursement changes that influence the provision of oral health, as well as for program management, to clarify the impact of existing or planned resource allocations.

Overview of economic terminology

This section briefly describes four major types of economic analyses and different approaches to calculating costs. It then reviews questions that must be addressed prior to initiating the analysis: from whose perspective should the study be conducted?; how long will the study last and is the time frame sufficient to capture all outcomes associated with the intervention?; and what is the best strategy to collect data for the analysis?

Types of economic analyses

Economic analyses provide information to help decision-makers select among competing alternatives when resources are limited. The four types of economic analyses commonly used to assess health interventions include cost, CE, cost-utility, and cost-benefit(19,21-26). All collect information on costs (defined as the value of what is foregone when resources are used in a particular manner). All four types typically include measures of: a) intervention costs – the value of resources used to deliver the intervention; b) intervention savings – averted treatment and other costs attributable to the intervention; and c) net costs – the intervention costs netting out intervention savings.

The first type of economic analysis, a cost analysis, measures net costs. One aspect related to net costs is financial sustainability (the ability of the program to endure after the initial funding has stopped), which may be measured by including reimbursement and other sources of program revenue in estimates of net costs. Although cost analyses provide a good estimate of resource use, they are limited in their ability to assess efficiency because they do not provide a good measure of health outcomes gained from the intervention. The remaining three types of analyses compare costs to some measure of the intervention benefits.

A CE analysis (CEA) measures the net cost per health outcome achieved such as cases of disease prevented (19,22,24) and years of life saved (19,22,24). Oral health outcomes for CEA may include averted caries (12,28), tooth years gained (13), pocket probing depth reduction (13-15), and clinical attachment level gain (14,15). CEA provides good estimates of health outcome gained per dollar spent and is appropriate when comparing different interventions that influence the same health outcome. It is limited, however, in that the health outcome measure may not include a measure of quality, and economic studies of interventions with different health outcomes cannot be compared to determine which intervention provides the best health investment.

A cost-utility analysis (CUA), which is a type of CEA, measures net costs per unit increase in a quality of life measure. The measure most commonly used for CUA is a quality-adjusted life-year (QALY) (19,22,24). QALY measures health with the value “1” representing a year of perfect health and the value “0” representing death. Oral health-specific quality measures include a quality-adjusted tooth year (6,32) and oral health-related quality of life (OHrQOL) (33-35). Through the use of a common outcome in the denominator (e.g., QALY), CUA may be used to compare interventions that address different conditions (e.g., diabetes, heart disease, cancer).

A number of instruments have been developed to measure the morbidity associated with different conditions (i.e., QALY >0). All solicit information from individual study subjects on their relative valuation of living with the ill health associated with the condition vs. perfect health. There are little data on the relationship between oral disease and QALYs (36). OHrQOL involves a similar approach, using dimensions of oral health instead of general health to examine the impact of oral disease. OHrQOL can be decomposed into dimensions such as function, pain, appearance, and psychosocial impact and role functioning (33-35).

When comparing two or more interventions, researchers may conduct an incremental CEA or CUA analysis. An incremental CE ratio includes the difference in net costs between the two interventions in the numerator and the difference in outcomes in the denominator (21,24).

Cost-benefit analyses (CBA) compare the intervention costs to the monetary value of the achieved health benefits based on how much a person values the averted disease or how much he/she would be willing to pay to avoid the disease. Obtaining estimates of a person's valuation of a health outcome [e.g., use of contingent valuation (37,38)] is typically resource intensive, and assigning a monetary value to health benefits such as a year of life is problematic (21). Because CBA is not used as frequently as the other types of economic analyses, we focus on cost, cost-effective, and CUAs in this paper.

Calculating costs

Two common approaches for assessing intervention costs and savings are to measure accounting costs and economic costs. Accounting costs, often referred to as financial costs or direct costs, are explicit monetary outlays for resources to provide or obtain services. They include medical costs (e.g., salaries and benefits for intervention personnel, medical supplies, household payments for health services) and nonmedical costs (e.g., travel costs). Such costs are generally recorded in an organization's accounting system.

Economic costs include both accounting and indirect costs (i.e., implicit costs such as the market value of resources for which no money was spent). They include productivity losses and resources provided in-kind (e.g., office space and other capital resources). Examples of productivity losses include time spent by unpaid intervention personnel and the time costs associated with traveling to, waiting for, and receiving dental services as well as time not spent conducting usual activities due to poor health. These are considered productivity losses, as they represent time away from regular work, household, and leisure activities.

Study perspective

Economic outcomes may be estimated from different perspectives (e.g., a health-care provider, a health system, a health payer, households, an employer, and society) (19,21-26). A health-care provider perspective may be that of a dental office, dental clinic, or group practice that includes dental and medical providers. In this paper, we employ the term health system to include private or public health-care providers that offer a range of medical, dental, pharmacy, outpatient, inpatient, and other services (e.g., Kaiser-Permanente, Indian Health Service). Health payers are public or private organizations that provide reimbursement or payment for health services. Examples of publicly funded programs include Medicare, Medicaid, and State Children's Health Insurance Program (SCHIP). It is important to note that a health system may function as both a health-care provider and a health payer, and that a government entity (e.g., federal government) may be a health-care provider and health payer. Household costs may include direct and indirect costs associated with obtaining intervention services, using related health services, and poor health. The employer perspective may include that of a health payer with the addition of indirect costs associated with employees' productivity losses due to poor health (39). Societal costs include all of the above costs – those of providers, payers, systems, households, and employers – and are generally estimated using economic rather than accounting costs.

To improve the conduct and reporting of CEA and CUA, the Panel (19) developed a list of recommendations that are analogous to the CONSORT statement (40) for medical researchers. The Panel recommended that CEA be conducted from the societal perspective so that findings may be used to determine the optimal allocation of scarce resources among competing alternatives. Also, this perspective provides a realistic estimate of the true cost of implementing an intervention.

Time frame and analytic horizon

Time frame refers to the duration of an intervention, whereas analytic horizon refers to the period over which all benefits and costs associated with an intervention are incurred. The analytic horizon is frequently longer than the time frame. For example, one study on the effectiveness of community water fluoridation found that exposure to water fluoridation in childhood (time frame) prevented tooth loss in adulthood (analytic horizon) (41).

Types of data collection

Economic studies of health interventions use different approaches to obtaining data. One approach, often referred to as micro-costing, tracks costs and savings associated with an intervention during the time frame of a study (19,24). For example, Hietasalo and colleagues (31) documented intervention costs, savings, and health benefits for each participant during the study time frame to estimate economic outcomes for an oral health behavioral health study.

Another approach is to construct a decision cost model, based on findings from previously published studies and/or secondary data, to estimate intervention costs, effectiveness, and potential savings over an analytic horizon (19,24). This strategy is referred to as gross-costing, as data used in the model are not derived from an intervention study. Decision cost models have been used to assess net savings associated with CWFP and SBSP (1,2,4,5).

A third, hybrid approach combines findings from an intervention study [e.g., randomized clinical trials (RCT) and comparative effectiveness research (CER) with data from other sources to develop a decision cost model to estimate short-term and long-term economic outcomes (42,43). The Panel recommends that all costs and health gains be included in a CEA. In situations where resources do not allow extending a CER or RCT time frame to capture all costs and benefits, modeling may be employed to estimate future costs and benefits.

Factors that influence the study design

Multiple factors are important to consider when designing economic studies of behavioral interventions. They include the service site, target population (e.g., an individual or family), data collection processes, phases of intervention implementation, and use of validated measures.

Intervention site of service and target population

The site of service for the intervention plays a key role, as intervention costs related to recruitment, participation, and retention may vary by service site. Oral health interventions for parents and caregivers of young children implemented at sites where such persons routinely come (e.g., Women, Infants, and Children clinics; Head Start centers; pediatric clinics) may have lower recruitment costs but may only reach those who access such services, and intervention frequency may be based on the schedule of services at the site. Programs implemented at other locations may have flexibility with regard to service frequency, yet may have higher recruitment and retention costs.

An intervention may be aimed at influencing behavior change at the family level (e.g., targeting caregivers of young children who may include parents, grandparents, other relatives, or guardians). Other interventions may focus services more specifically on individuals. Consequently, measures of costs and savings should be relevant to the defined target population.

Retrospective or prospective data collection

It is important to consider the study time horizon when assessing intervention costs and savings (e.g., whether data related to costs and savings will be collected prospectively or retrospectively). Prospective data collection throughout the intervention time frame allows for ongoing review of data and for modification of data collection methods to ensure accuracy and to address identified reporting issues.

Phases of intervention implementation

Interventions may include different phases during which costs are incurred yet service provision varies. They include pre-implementation, start-up, steady-state, and wind-down phases. Costs associated with program development or adaptation may be incurred before intervention services are provided to participants. There may be a start-up or pilot phase when participant enrollment is relatively low and intervention cost per participant is higher than during the steady-state phase due to fixed salary costs of intervention personnel and personnel experience providing services. During a steady-state period, study enrollment may remain fairly constant and personnel are experienced at providing intervention services. Finally, the number of study participants may be lower at the end of the intervention time frame due to participant loss or earlier intervention completion for some participants. The cost per participant may be higher during this phase as well. It is important to consider intervention costs for each phase when determining the frequency of data collection.

Validated measures

If possible, cost measures, like other study measures, should have been validated in other studies and pilot tested in the current population. Use of validated measures for survey instruments may minimize biases associated with self-report data (e.g., recall bias) and provide guidance on appropriate time periods to include in such measures (e.g., past 6 months or year). Pilot testing is especially important when the target population is culturally or economically different from that of previous studies (44).

Research design

Measures and data collection

In this section, we describe an array of cost measures, relevant for the economic analyses described above, and present methods and issues related to micro-costing such information. As with other study measures, it is important to ensure the fidelity of the data collection process while balancing accuracy and precision with data collection costs. It also is important to plan logistics related to data collection, tracking, quality checking, and storage.

Intervention costs

Table 2 provides a list of intervention costs (capital, noncapital one-time fixed, and operating costs) that may be included in cost estimates. If prices are not available through micro-costing, online references that provide cost data, such as those listed in Table 3, may be used. These sources may be used to estimate the current market price of materials and labor provided on an in-kind basis.

Table 2.  Examples of Intervention Non-Personnel Cost Categories
Type of CostCost CategoryType of Expense
  • * 

    Facility costs could be capitol or operating costs. In this table they are listed as operating costs.

  • † 

    Such costs may be one-time fixed costs or costs that are incurred throughout an intervention time frame.

  • ‡ 

    Overhead costs may include those associated with office space, utilities, information systems, human resources, and other activities necessary for program operations, and their costs may not be included as a line item in the intervention budget. Human resource costs may or may not include costs associated with hiring intervention personnel and related payroll and benefit services.

Capital  
 EquipmentOffice furniture
 Computers
 Large electronic (e.g., copiers, printers)
 Small electronic (e.g., cameras, cell phones, PDAs)
 Medical and dental equipment
 Laboratory equipment
Operating  
 Facilities*Office space
 Clinic space
 Space for events and classes
 Storage space
 Maintenance
 Insurance
 Real estate taxes
 Repairs and service costs
PersonnelIntervention personnel
 Clinical personnel
 Supervisory personnel
 Administrative personnel
 Other types of personnel
General office suppliesGeneral office supplies
 Printing and Xeroxing
 Postage and FedEx
 Books and manuals
 Information technology and computer supplies
UtilitiesTelephone (e.g., phone services, long distance calls and faxes)
 Internet services
 Energy
 Water
Clinical interventionMedical supplies
 Laboratory tests
 Pharmaceuticals
Nonclinical interventionPrinted materials
 Digital media (DVDs, tapes, videos CDs)
 Other intervention supplies
TransportationVehicle
 Fuel
 Repairs and service
 Personnel travel reimbursement or costs
 Consultant travel costs
TrainingTraining fees
 Training materials
External Consultants, SubcontractsConsultants
 Subcontractors
Information technologySoftware
 Information technology support
 Website design
OverheadFacility and administrative
 Human resource
Table 3.  References for Price Estimates for Intervention Personnel, Equipment, Productivity Losses, and Other Costs
CategoryType of costDescriptionSource
  1. Note that nearly all of the listed references may be accessed free of charge. Those marked with an “*” require payment to purchase.

Health servicesDental servicesThe American Dental Association (ADA) 2009 Survey of Dental Fees includes national summary statistics of fees charged by dentists.http://www.ada.org/1619.aspx*
 Medicaid reimbursement: The ADA Medicaid Compendium Update provides Medicaid reimbursements for all states for select dental services.2008 fees http://www.ada.org/2123.aspx (other source: 2004 fees http://multivu.prnewswire.com/mnr/ada/20973/)
Consumer price indexMedical services: Use the US Department of Labor, Bureau of Labor Statistics website and type in the series identification number. Use CUUR0000SAM for medical care. Use CUUR0000SEMC01 for physician services.http://data.bls.gov/cgi-bin/srgate
 Dental services Use the US Department of Labor, Bureau of Labor Statistics website and type in the series identification number. Use CUUR0000SEMC02 for dental services.http://data.bls.gov/cgi-bin/srgate
Productivity lossesWages foregoneAnnual and hourly wages for all US stateshttp://www.bls.gov/oes/current/oessrcst.htm
Benefits foregoneEstimates of benefits are only available at the federal level.http://www.bls.gov/data/#wages
Value of a lost dayEstimates for time of persons who work and who do not workTable I.1(a)-(c) in Prevention Effectiveness, A Guide to Decision Analysis and Economic Evaluation (19)*
CapitalEquipmentThere are several publically available online sources. Authors list two.http://www.buydentalequipment.com, http://www.ebay.com
Annuity factorTo estimate the annual cost of capital equipment, authors recommend using annuity factors based on a 3% discount rate, per recommendation of the US Panel on Cost-effectiveness in Health and Medicine (16), and equipment useful lives ranging from 1 to 25 years.Table 2 of Appendix C in SEALS Users Manual available at http://www.chawisconsin.org/sas.htm

Capital costs include expenditures for items that may be used over several years, including facilities and equipment (e.g., Xerox machines, printers). Although the costs of these items may occur in one-time period, their benefits will span over their useful life. Thus, it is necessary to estimate the annual cost for each year of the equipment's useful life by dividing the value of the equipment (i.e., purchase price) by the annuity factor that is based on the equipment's useful life and the discount rate (i.e., 3 percent; see Table 3).

Noncapital one-time fixed costs often include those associated with program development (e.g., costs associated with developing or adapting oral health education and promotion materials, developing or adapting training programs, developing logistics and processes). Expenditures for these costs need not be annuitized over their useful life span.

Operating costs are the ongoing costs required to provide intervention services that accrue over a budget period, usually calculated on an annual basis, such as personnel, program supplies, travel, reporting and documentation, and administrative costs. As with other costs, some vary by the volume of services provided and others do not.

When documenting costs associated with a study, it is important to distinguish research costs from those associated with actually providing an intervention. For example, some consultant costs may be associated with research methods, while others might be associated with the adaption of educational materials for use in the intervention. Similarly, personnel time may include time spent providing the intervention and time spent on research-related activities (e.g., writing a study protocol for institutional review board approval, attending a training to ensure measurement fidelity).

Personnel costs

For many behavioral interventions, personnel expenses constitute the majority of costs and will substantially influence intervention cost estimates. As such, detailed measurement is merited to ensure reliable and accurate data. This generally involves assessing personnel time for the intervention and assigning a dollar value to that time based on related costs (e.g., salaries, benefits). Two common strategies for assessing personnel time include use of time logs and time–motion studies (22,26). For both approaches, larger samples and collection of data throughout the intervention time frame increase the accuracy and precision of time estimates.

With either approach, personnel time spent on intervention activities should be accrued separately from time spent conducting research activities. Intervention activities may include travel to intervention sites, provision of oral health promotion and prevention services, scheduling intervention services, documentation and reporting, and management and supervision of intervention personnel. Research activities include trainings about research protocols, administration of study questionnaires that would not be utilized for an intervention in a non-research setting, and meetings about research methods. Some activities, such as study enrollment, may be considered both intervention and research activities. A uniform approach to assigning a proportion of time spent on such activities should be developed.

Figure 1 provides an example of a personnel time log for the pre-implementation phase of a study; Figure 2 provides a log for the intervention period. Time logs may be developed based on personnel job descriptions and responsibilities and should be pilot tested before being finalized to ensure that they accurately reflect activities. As with other intervention procedures, staff should be trained on their use. Logistical arrangements for their use may vary across implementation stages and by personnel types. For example, during pre-implementation, the personnel may be asked to estimate time spent on various activities, based on a review of their calendars at the end of each month, while more detailed data may be collected during the intervention time frame. Previous research indicates that personnel time spent completing logs at the end of a month may be no more than 5 minutes (45). The accuracy and costs of data collected retrospectively on a monthly basis, as compared to other time periods, should be evaluated based on personnel intervention activities. During the implementation phase, intervention personnel could be asked to prospectively complete a log at specific intervals (e.g., each work day during a representative week each quarter) to allow for data collection during the intervention start-up, steady-state, and wind-down phases.

Figure 1.

Example of an intervention time log for personnel during the preimplementation period.

Figure 2.

Example of an intervention time log for personnel during the intervention time frame.

When greater precision is needed, the best alternative to time logs usually will be a time–motion study involving use of observers to record time data (22,26). This approach is effective for assessing time costs associated with an intervention activity that is added to a health-care provider's routine array of services (e.g., the addition of oral health education and fluoride varnish application during a well-child visit for children less than 2 years of age). For either approach, the frequency of data collection (i.e., the sample size) may be determined based on information and estimates for the number of intervention staff, length of the intervention time frame, variety of intervention activities, variation in time spent on intervention activities, and reporting burden using a classical or Bayesian approach (25,26), and adjusted after an initial round of data collection.

Other personnel costs include those associated with recruitment, hiring, training, and staff turnover. While some may be included in organization overhead costs, direct costs such as training may be listed as a separate category. Indirect costs such as those associated with personnel experience are generally difficult to measure but may be represented in statistics concerning the number of services provided.

Other intervention costs

An extensive list of other intervention cost categories is provided in Table 2. Organizational overhead costs can vary substantially and thereby significantly influence intervention costs. Information on monetary outlays for these and other costs may be extracted from fiscal documents on a quarterly, semiannual, or annual basis.

Costs associated with utilization of other health services

Behavioral interventions implemented to improve oral health may alter utilization of other health services. Improvements in oral health may be associated with reductions in dental and medical service utilization for oral health problems, but intervention services may also increase utilization. For example, oral health screenings conducted as part of an intervention may lead to increased utilization of some dental services as persons are referred for follow-up services based on screening results. It is also possible, however, that any type of service utilization may be associated with adverse effects and thereby increase utilization of health services. Information on changes in use of other health services may be used to assess intervention costs and savings.

Data on dental, medical, and pharmacy service utilization for oral health problems may be obtained from self-reports, clinical exams, and data extracted from medical records or health provider, system, or payer databases. The feasibility, benefits, and costs associated with the use of these data sources vary. For example, data costs associated with adding utilization measures to existing study instruments are lower than they would be if an instrument were developed for this purpose. Two issues to consider when using self-report data are level of detail and recall bias. Although it is possible to include measures related to specific dental procedures such as tooth extractions and dental sealants, self-report data may lack detail related to the number and types of dental procedures provided in a dental clinic or office. Results of published studies indicate that the reliability of self-reports of dental service utilization varies by service type and frequency of data collection (46-48). It may be possible to examine potential reporting biases by comparing self-report data with data from an administrative database for a sample of intervention participants.

A study protocol may include use of a clinical oral health exam to assess oral health status. Exam data on the number and type of restorations may be used to assess costs associated with such treatments. However, these data do not include information on the site of service, types and costs of related procedures, and related household costs. It is possible to use a combination of self-report and clinical exam data to address weaknesses associated with the use of each data source on its own.

While dental and medical records include detailed information on procedures, health status, and dates of service, the costs of data extraction are significant. Electronic data stored in administrative databases for billing and other purposes include less detailed information, yet costs associated with data extraction are relatively lower. Some databases include information on the use of medical services (e.g., hospital emergency), as well as dental services, for oral health problems. Others, such as those of private dental insurers, generally do not.

Health service price estimates may be obtained from utilization data and published sources on provider service charges (e.g., American Dental Association report on dental fees) or reimbursement (e.g., Medicaid and SCHIP reimbursement rates; see Table 3). Although data on service charges and reimbursement are often used to estimate service costs, it is recognized that such measures may not reflect the true economic costs associated with the provided services.

Health provider, system, and payer revenue

Health providers and systems may be reimbursed for services by public or private insurers. To assess the impact of these and other revenue sources on an intervention's financial sustainability, information on participant insurance status may be obtained via study participant self-report and from revenue data obtained from health provider, system, and payer databases.

Household costs

Households may incur direct and indirect costs due to study participation, health service utilization, and poor health. Direct costs are costs that families pay out of pocket for dental and medical services, travel to obtain services, oral health supplies, and other related items. Indirect costs are productivity losses, such as those associated with time spent traveling to provider offices, waiting for intervention services to be provided, and receiving dental services. They also include time away from day-to-day activities due to poor health or to caring for someone in poor health. The dollar value of these costs may be estimated from measures of time spent and related prices. The price or value of a study participant's time may be estimated from information on participant wages or from published estimates for the value of 1 hour of activity for men and women employed both inside and outside the home (22) (see Table 3).

Other costs

Assessing economic outcomes from the employer and societal perspectives often involves no additional data collection. Economic outcomes from an employer perspective may be derived from data on costs and savings associated with employee health and dental insurance and participant's missed days of work (39). However, additional data may be collected to estimate human resource costs associated with hiring new employees should health status influence employment or other costs associated with employee benefits.

Data on the economic costs incurred by health providers, systems, and payers; households; and employers may be used to estimate economic outcomes from a societal perspective. However, there are many short-term and long-term costs associated with poor oral health that are intangible or difficult to assign a monetary value to (49). As noted above, OHrQOL may be used to assess these factors and a related change associated with improved oral health status (33-35). For example, the Pediatric Oral Health-related Quality of Life Parent Report on Child was found to be a valid and reliable measure for assessing parent-reported effects of oral conditions in preschool children on their physical, emotional, and role functioning (35). This OHrQOL measure may be useful in assessing benefits for behavioral interventions targeting caregivers of young children. Other OHrQOL measures, such as the Oral Health Impact Profile, may be used for interventions targeting adults (10,50-52).

Analysis

In this section, we refer to seven oral health studies to describe strategies for estimating economic outcomes from different perspectives. To locate relevant studies, we searched Pubmed for economic analyses on dental interventions published in English after 2004. Our search yielded 36 studies, of which 15 included economic analyses. Seven are included in Table 1. Of the remaining eight studies, three were on dental implants or treatment of dental fractures, one was on community water fluoridation, three were on delivering clinical dental care in settings other than the dental office, and one examined willingness to pay to prevent dental caries.

We begin with three studies that used micro-costing to estimate net costs associated with oral health education and preventive services targeting low-income, high-risk families (27,29,30,53). All three studies estimated net costs from the health-care provider perspective. Grant and colleagues (30) included salaries and dental supplies to estimate the costs of screening, parental counseling, and applying fluoride varnish in a dental clinic setting. Wennhall and colleagues (29) included facility, equipment, labor, and supply costs in their estimate of intervention costs; the cost and net cost per child were 310 and 30 euros, respectively. Kobayashi and colleagues (27) also included costs associated with training dental personnel, community outreach, and program marketing in their estimate of costs for a community-based program to increase preventive service utilization. Both Grant et al. (30) and Kobayashi et al. (27) reported net costs and an estimate of financial sustainability – net costs including Medicaid reimbursement revenue, as both studies targeted households with children enrolled in Medicaid.

We next examine CWFP and SBSP economic analyses conducted from a societal perspective. Although they do not address behavioral health interventions, these studies illustrate strategies useful for behavioral health studies. O'Connell and colleagues (2) reported CWFP net savings based on a decision cost model that incorporated secondary data to estimate intervention costs for a 12-month period, caries reductions during the same 12-month period, and intervention savings accrued over a lifetime due to caries reductions during the 12-month period. Through the use of decision analysis software, a Markov model was constructed to describe the probability of different types of initial and replacement restorations (e.g., single-surface amalgam, multisurface composite resin, crown) and extractions that may occur over a lifetime, and estimate related treatment costs which may be averted due to CWFP.

As the clinical effectiveness of dental sealants is well documented (54-56), a number of studies have examined sealant costs, net costs, and CE. In 2002, Griffin et al. reported net cost savings from the societal perspective using a decision cost model based on secondary data that provided a framework for other studies. More recently, Scherrer and colleagues (5) developed a cost model from data provided by seven-state SBSP and published findings to examine net costs from the health-care provider, state, and societal perspective. The model was used to examine the influence of program size (e.g., number of sealant stations, capital costs) and different combinations of personnel (i.e., dentists, dental hygienists, dental assistants) on program costs and savings. The authors reported that additional cost savings were associated with modifying Wisconsin's dental practice act to allow dental hygienists to prescribe sealants without a dentist's supervision.

Both studies (2,5) used software to construct cost models to conduct sensitivity analyses to examine the robustness of study findings to parameter estimates (i.e., the values of the measures) and to explore the impact of programmatic changes or unique community aspects on the economic outcomes. Decision analysis software was used for the CWFP study to develop a Markov model to incorporate probabilities of various health events occurring over a person's lifetime and to conduct sensitivity analyses. The Panel recommends that, at a minimum, one-way sensitivity analyses be conducted where uncertainty or lack of agreement about some key parameters (e.g., program size, personnel costs) exists, to understand their influence on the economic outcomes (19). Multiway sensitivity analyses for important parameters are recommended (19). Parameter values may be varied within realistic ranges of the parameters' distributions, such as those based on the mean and standard error of an estimate based on a normal distribution. Software may be used to estimate a confidence interval or credible range for a CE estimate. For example, a second-order Monte Carlo probabilistic sensitivity analysis may examine the influence of variability in all cost parameters on the estimated economic outcomes. Such analyses make possible the calculation of a 95 percent credible range (the 2.5 percent to 97.5 percent) for each economic outcome. Similar to a confidence interval, a 95 percent credible range provides information about the variability of the estimated economic outcomes due to inherent uncertainty of some cost measures.

Table 1 includes CEA findings for two RCTs of behavioral health interventions that included oral health education and preventive services. Both studies used micro-costing and assessed economic costs from a health-care provider perspective during the RCT time frame. Hietasalo et al. (31) reported the average incremental CE ratio as 34.07 euros per averted decayed, missing, and filled surface (DMFS) for a 3.4-year program serving children ages 11-12 years. The incremental net costs over the intervention time frame were reported to be 69.50 euros (95 percent credible range: 28.25-110.75). Annual incremental net costs during the later years of the intervention time frame were found to be lower as restorative costs for intervention children decreased. Using data for a 3-year intervention designed for mothers of infants living in low socioeconomic areas with high caries prevalence, Kowash et al. (28) estimated intervention costs and savings for a steady-state year and reported intervention costs per averted DMFS as 1.8 pounds. The intervention involved the provision of education focused on oral hygiene and nutrition through home-based visits of varying frequency over a 3-year period.

As intervention costs and benefits may accrue over several years, economic analyses include adjustments for inflation and discounting (see Table 3). Published inflation rates for the economy as a whole and for certain sectors (e.g., medical and dental) may be used to convert dollars from varying time periods to those of 1 year, often referred to as the base year. Even in the absence of inflation, a dollar or a health benefit received today is worth more than that received tomorrow due to time preferences. For example, most persons would be willing to pay more for a positive health outcome today than waiting for a year. As a result, future costs and health outcomes must be discounted. The Panel recommends that CEA use a discount rate of 3 percent per year (19).

A number of other statistical issues arise when analyzing oral health and economic outcomes. One issue is missing data for participants lost to follow-up. Some investigators use conservative estimates concerning the intervention's effectiveness for such participants (26). A second issue is the distribution of health and cost measures (25,26). The distribution of DMFS and treatment cost data is often highly skewed. For example, treatment costs can exhibit a large proportion of zero values for those who obtain no treatment and a limited number of very high values for those who obtain expensive types of treatment. Consequently, sample means may not fully account for such treatment costs, and statistical methods that address such distributions may be used. The third statistical issue concerns tooth loss. Oral health interventions for young children or older adults must account statistically for high rates of tooth loss associated with loss of primary teeth in the case of young children and permanent teeth among older adults. Finally, it is important not to double count costs, such as those for health services that may be incurred by households, health-care providers, and health payers. Because CUA includes a measure of time costs associated with poor health in the denominator (e.g., OHrQOL), such time costs should not be included in the numerator.

Discussion

This paper summarizes information and established protocols for conducting economic analyses of behavioral interventions implemented to improve oral health. Although the literature on costs and CE of such interventions is fairly limited, we illustrated various methodologies (e.g., micro-costing, gross-costing, decision cost models, sensitivity analysis) by describing methods and findings for seven oral health studies (2,5,27-31). Five of the studies examined behavioral interventions (27-31). While none included economic outcomes estimated from a societal perspective or potential savings for an analytic horizon longer than the intervention time frame (e.g., costs of maintaining a restoration overtime), the authors of two (28,31) discussed potential future savings that may accrue beyond the study time frame due to intervention effectiveness during that period.

In the description of research design considerations for conducting economic analyses, the recommendations of the Panel of Cost-Effectiveness in Health and Medicine's were highlighted (19). These recommendations include use of a reference case analysis to enhance comparability across studies. The reference case should be based on a societal perspective, compare the health intervention of interest to at least one relevant alternative including doing nothing, include all pertinent costs, use a health-related quality-of-life measure to assess health benefits, have a time horizon that is long enough to capture all relevant future effects, adjust all costs for inflation, discount future costs and health benefits to their present value, and conduct a meaningful set of sensitivity analyses.

We recognize, given resource constraints, that it may not be feasible to implement these recommendations in all studies and comment on three considerations. First, conducting analyses from a societal perspective is important so that all relevant costs and benefits are accounted for. However, we acknowledge the importance of also having estimates based on a provider, payer, or government perspective as decisions concerning dissemination or translation of findings may be made by such entities. If resources preclude the measurement of some costs or benefits, their exclusion and influence on the findings should be addressed as a study limitation. Second, we know it may be difficult to assess the health impact in terms of health-related quality of life. At present, there is limited data linking oral conditions to QALY and no universally accepted OHrQOL measure. Adding a OHrQOL measure to a research protocol may also involve significant resources. As research advances, information linking oral health improvements to changes in QALY will facilitate comparisons among oral health interventions and those for other conditions, and contribute to decisions on the broad allocation of health resources. Finally, documentation of intervention costs could be improved by assessing all costs associated with each phase of development and by including all pertinent costs associated with personnel training, intervention outreach, participant scheduling, missed appointments, administration, and overhead. Use of standard protocols and detailed documentation of methods contribute to the quality and uniformity of economic analyses and allow for comparisons across interventions.

Researchers conducting economic studies of oral health behavioral interventions may learn from strategies employed in studies of other types of oral health interventions (10-13) as well as those of other conditions (e.g., diabetes, heart disease, cancer) (42,43,57-61) because economic studies, including estimates of long-term costs and benefits from a societal perspective, are more commonly conducted for interventions targeting such diseases. While we addressed several important methodological issues in this paper, we excluded others, such as measurement of net health benefits and use of a CE plane or acceptability curve to graphically depict trade-offs between intervention costs and health benefits. Detailed information on these and other important issues may be found in books dedicated to this topic (19,21-26).

Costs and CE findings derived from RCT or CER studies inform decisions about the dissemination and translation of interventions found to be clinically effective, culturally acceptable, and organizationally sustainable. At the same time, information on costs and potential savings of interventions with undocumented health benefits may contribute to the assessment of intervention components that may be altered to improve clinical effectiveness. Knowledge of resources needed to implement behavioral health interventions and their influence on health providers, systems, payers, employers, and society may increase the provision of effective interventions which may not only improve oral health status but also reduce oral health disparities.

Conflict of interest

JMO has received an NIDCR grant and received an honorarium from NIDCR for writing or reviewing this manuscript. SG declares no conflict of interest.

Ancillary