There is a paucity of economic evaluations of prediction methods of caries and periodontitis—A systematic review

Abstract Objectives Direct cost for methods of prediction also named risk assessment in dentistry may be negligible compared with the cost of extensive constructions. On the other hand, as risk assessment is performed daily and for several patients in general dental practice, the costs may be considerable. The objective was to summarize evidence in studies of economic evaluation of prognostic prediction multivariable models and methods of caries and periodontitis and to identify knowledge gaps (PROSPERO registration number: CRD42020149763). Material and methods Four electronic databases (PubMed, Web of Science, The Cochrane Library, NHS Economic Evaluation Database) and reference lists of included studies were searched. Titles and abstracts were screened by two reviewers in parallel. Full‐text studies reporting resources used, costs and cost‐effectiveness of prediction models and methods were selected and critically appraised using a protocol based on items from the CHEERS checklist for economic evaluations and the CHARMS checklist for evaluation of prediction studies. Results From 38 selected studies, six studies on prediction fulfilled the eligibility criteria, four on caries and two on periodontitis. As the economic evaluations differed in method and perspective among the studies, the results could not be generalized. Our systematic review revealed methodological shortcomings regarding the description of predictive models and methods, and particularly of the economic evaluation. Conclusions The systematic review highlighted a paucity of economic evaluations regarding methods or multivariable models for prediction of caries and periodontitis. Our results indicate that what we currently practice using models and methods to predict caries and periodontitis lacks evidence regarding cost‐effectiveness.


| INTRODUCTION
Caries and periodontitis are among the most prevalent oral diseases.
The distribution and severity vary between individuals within the same region or country (Petersen et al., 2008) resulting in a skewed distribution of disease. The challenge today is to identify individuals not at risk and those with increased risk for future disease development. Identification of individuals not at risk, will enable reduced redundant recalls and examinations in the dental practice. For those with an increased risk tailor-made recalls and interventions will be enabled.
The identification of individuals with an increased risk of developing caries lesions using prognostic assessments, also called risk assessment, is either based on a method with one single variable, for example, previous caries experience, number of microbiota, salivary buffering capacity or salivary flow rate (Mejàre et al., 2014;Senneby et al., 2015), or on multivariable prediction models such as the Cariogram (Bratthall & Hänsel-Petersson, 2005) and Caries Management by Risk Assessment (CAMBRA; Featherstone et al., 2003). To predict the progression of periodontal disease, the presence or absence of bleeding on probing (Lang et al., 1990) or probing pocket depth (Haffajee et al., 1991) has been applied as single methods. For the risk assessment models of periodontal disease, age, smoking and diabetes status are most commonly included (Du et al., 2018).
The direct cost for a method or model used for risk assessment in dentistry may be negligible compared to the cost of extensive constructions. On the other hand, as risk assessment and decisions about recalls and intervals between examinations as well as interventions are performed for several patients daily in general dental practice, a comprehensive cost has to be taken into account, which from a societal perspective, means to allow for effective resource allocation. However, the societal cost of a method may not equal the cost that the clinicians face. Hence, health economic evaluation that is a set of formal methods is applied to analyze resource use and outcomes of interventions such as diagnostics, prediction, prevention and treatment. As public resources are limited, they should ideally only be used in programmes that are cost-effective. A recent scoping review indicated that the number of economic evaluations in dentistry is increasing (Eow et al., 2019) but the need is still great and thus more work in this field is necessary. The results of a systematic review of economic evaluation demonstrated that evidence is limited for diagnostic methods used in dentistry (Christell et al., 2014). Similar results may be assumed to apply for methods used for prognostic prediction.
The purpose of a systematic review (SR) is to provide evidence for an intervention based on the summary of current literature and to identify knowledge gaps to guide future research within a knowledge field. In SRs that include economic evaluations, information is compiled not only on whether interventions work but also on the economic aspects of the actual interventions. The objective of this SR was to further the understanding in this field by collecting data of studies of economic evaluations of prognostic multivariable models and methods for prediction of caries and periodontitis by using a SR approach.

| METHODS
Based on current method guidelines (Moher et al., 2010) we reviewed the principal findings of economic evaluations of multivariable models and methods for prognostic prediction of increased risk of caries as well as of periodontitis. To ensure a systematic approach, the SR was structured in the following steps: (a) formulating the review questions, (b) formulating eligibility criteria, (c) identification and selection of studies, (d) collection of data and appraisal of included studies and (e) synthesis and interpretation. PRISMA statement (Shemilt et al., 2019) was implemented, and the SR was registered in PROS-PERO (International prospective register of systematic reviews; PROSPERO, 2020) as CRD42020149763.

| Review questions
The SR of the literature on economic evaluations of multivariable models and methods for prognostic prediction of the increased risk of caries and periodontitis aimed to address the following questions: • Which multivariable models and methods for prognostic prediction have been analyzed?
• What are the estimated resources used, costs and/or costeffectiveness of the multivariable models and methods?
• What are the potential strengths and weaknesses of the methodologies of the included studies?
• To what extent may the evaluations help decision-makers to direct resources efficiently?
The following elements were defined prior as: • Economic evaluation: either full economic evaluation according to the definition by Husereau et al. (2013) as a comparative analysis of alternative courses of action in terms of costs (resource use) and consequences (outcomes, effects) or a partial economic evaluation defined as cost analyses and cost-description studies (Moher et al., 2010).
• Multivariable model: a model based on a mathematical equation that relates to at least two variables (Moons et al., 2014).
• Prediction model: a tool that combines multiple predictors by assigning relative weights to each predictor to obtain a risk or probability. They are developed to estimate the probability or risk that a specific disease or condition is present (diagnostic models) or that a specific event will occur in the future (prognostic model; Moons et al., 2014).
• Prognostic prediction: an estimation of the probability that an individual will experience a specific event or outcome within a certain time period (Collins et al., 2015).
• Risk: this is defined as the probability of an unwanted event that may or may not occur. Risk of dental caries is a probable development from either (a) a sound tooth surface to a lesion in enamel or dentin, that is, from health to disease or (b) an existing lesion to a more extensive lesion, that is, from disease to more severe disease (Senneby et al., 2015). The risk of periodontitis is a probable development from either (a) sound gum to bleeding on probing, increased pocket depth and loss of alveolar bone height or (b) increased pocket depth and loss of alveolar bone height to bleeding on probing, further increase of pocket depth and loss of alveolar bone height.

| Eligibility criteria
Eligibility criteria were determined a priori according to the review questions and based on PIRO (Population, Index test; Reference standard, Outcomes): Inclusion criteria: included studies had to meet a minimum set of criteria concerning • Population: humans of all age groups.
• Index test: clinically applicable multivariable model or method for prognostic prediction of caries and caries development or of periodontal disease.
• Reference standard: Caries progression also named caries increment/incidence/ experience presented at an individual basis.
Progression of periodontal disease and/or tooth loss at an individual basis.
• Outcomes of index test (predictive performance): Predictive accuracy efficacy expressed as sensitivity, specificity, positive and negative predictive values, positive and negative likelihood ratios, area under the receiver operating curve (AUC) or data required to populate 2 × 2 tables cross-classifying index test and reference standard.
Consequences of the prediction of progression of caries or periodontal disease such as recalls for examination.
• Outcomes of economic evaluation: resources use, costs or costeffectiveness data.
• Language: studies with English abstracts and written in English or German would be considered for inclusion in a pragmatic way, based on the availability of adequate translation skills within the authors' institutions.
Exclusion criteria: Studies were excluded if they contained no data of either multivariable model or method for prognostic prediction of caries and caries development or of periodontal disease together with an economic evaluation. Narrative reviews, editorials, case reports and individual (clinical) studies were excluded.

| Identification and selection of studies
The electronic databases searched were: MEDLINE via PubMed, the Web of Science, the Cochrane Library and the NHS Economic Evaluation Database (NHS EED) until it stopped at the end of 2014. MESH-terms as well as free-text terms were combined (Table S1). Search strings were developed together with librarians and adapted for use across databases, combining the MeSH using the "AND" operator with index terms (Table S1). The publication years ranged from database inception to September 2, 2020. In addition, the reference lists of included publications were screened for publications not captured by the electronic searches. We also searched the PROSPERO database (terms: economic evaluation/cost-effectiveness of prognostic prediction/predictive methods/risk-assessment of caries/periodontitis) on August 26, 2020 to allow identification of any forthcoming studies.
The selection of studies was completed in two phases. In phaseone, the retrieved records were assessed according to title and/or abstract by two reviewers in parallel and selected according to the review questions and eligibility criteria. Records selected by at least one reviewer were retrieved in full-text for further assessment. In phase-two, two reviewers in parallel applied the same eligibility criteria to the full-text studies. Any disagreement was resolved by consensus among the reviewers.

| Collection of data and appraisal of included studies
Four reviewers, one of them being a health economist, were involved in the data collection. A table for data collection regarding both the prognostic prediction method/model and the economic evaluation was developed a priori and piloted by the four reviewers in parallel using five studies. Discussion was held among the reviewers and discrepancies were resolved until consensus was reached. The data on the prediction method/model of each study considered to be important was the description and the predictive performance of the methods/model. For the economic evaluation data as proposed by Shemilt et al. (2019) was collected when available on (a) the analytic framework and type of economic evaluation, (b) the analytic perspective (whose costs and benefits a decision maker views as important) and time horizon, (c) main cost items includes in the analysis and (d) the setting (i.e., country, health care system), currency and price year. Data from included studies were tabulated by one reviewer and checked by the other reviewers, who read the studies in parallel to verify the accuracy of collected data and to ensure that relevant data was included. Any disagreement was resolved by consensus among the four reviewers. When predictive performance was not presented in the included study, data required populating tables cross-classifying index test and reference standard was searched in studies referenced to, enabling the calculation of predictive performance.
For the critical appraisal, a protocol was designed and applied in included studies. Items from CHARMS checklist for reviews of prediction modeling studies (Moons et al., 2014) and checklist for reporting economic evaluations (CHEERS; Husereau et al., 2013) were included. The protocol contains five domains with signaling questions: sample, prediction model or method, outcome to be predicted, economic evaluation, and analysis and interpretation of results (Table 1). Each signaling question is formulated to be answered by one of "Yes," "No," "Unclear" or "Not applicable." First, the four reviewers assessed included studies in parallel and then compared the answers of each signaling question. Discrepancies regarding answers for the signaling questions were discussed until consensus was reached.

| Analysis and interpretation
A narrative approach was used to present the analysis of collected data of the estimated resources used, costs and/or cost-effectiveness.
Based on the collected data and answers to the signaling questions of the five domains presented in Table 1, an overall judgment of "Strengths" and "Weaknesses" of included studies was made by the T A B L E 1 Tool for critical appraisal of studies of economic evaluation of prognostic multivariable methods and models for prediction of caries and periodontitis Was target population to whom prediction method/model applies described?
Was setting of recruitment described (e.g., primary care, secondary care, general population)?
Were number, age and sex of included individuals presented?
Was disease prevalence of included groups presented?
Was study dates presented?
Domain 2: Method/model for prediction of caries and/or periodontitis (index test) Was method for prediction described to permit replication?
Was model with types and handling of predictors described to permit replication?
Were examination method(s) and measurements presented?
Was number of examiners and their experience described?
If implemented was thresholds clearly described?
Was timing of measurement of predictor(s) described?
Domain 3: Outcome to be predicted (reference standard) and method/model performance Was type of outcome(s) to be predicted defined?
Were outcome thresholds and criteria for thresholds clearly described?
Were examination method(s) and measurements presented?
Was outcome assessed without knowledge of candidate predictors (blinded assessment)? Was performance of model or method presented adequately?

Domain 4: Economic evaluation
Was the study design appropriate?
If simulation was used was the model transparent?
Was the perspective of the evaluation presented?
Were all relevant costs identified, presented and valued?
Was time horizon long enough to reflect all relevant costs and effects?
Were outcome measures captured and quantified for all important effects?
Was discount rate(s) stated?
Was uncertainty considered?

Domain 5: Analysis and interpretation of results
Was the result presented as costs related to effects (ratio)?
Was sensitivity analysis with relevant variables performed?
Is the result robust in regard to results of sensitivity analysis?
Is the result transferable to other settings?
Is the analysis helpful for a decision-maker?

Comments
Not applicable four reviewers jointly. The generalizability and transferability of each economic evaluation was interpreted when answering the review question "To what extent can the evaluations help decision-makers to direct resources efficiently?" 3 | RESULTS

| Identification and selection of studies
As presented in Figure 1, 455 records were identified after removal of duplicates, 38 full-text publications were read, and 32 full-text publications were excluded because they contained no data on predictive performance or of an economic evaluation (Table S2). Six studies, published between 1994 and 2014 and listed in Table 2, were included.
3.2 | Which multivariable models and methods for prognostic prediction have been analyzed? Table 2 presents the methods and multivariable models for prognostic prediction of caries or periodontitis of the included studies. Four studies presented the prediction of caries in children aged 1-3 years with a follow-up of 1-3 years, two studies on methods (Jokela & Pienihäkkinen, 2003;Zavras et al., 2000) and two studies on multivariable models (Holst et al., 1997;Holst & Braune, 1994). The methods were based either on the amount of salivary mutans streptococci as the predictor (Zavras et al., 2000) or the presence of caries lesions together with the presence of mutans streptococci (Jokela & Pienihäkkinen, 2003). The studies of multivariable models for the prediction of caries (Holst et al., 1997;Holst & Braune, 1994)  According to our calculations based on Fig. 4 in Page et al. (2003).
periodontitis in middle-aged persons with a follow-up of 13 and 30 years, respectively. In one study (Higashi et al., 2002), the prediction was based on a model with a biomarker (interleukin-1), whilst the other study presented a model based on risk scores containing nine factors (Martin et al., 2014). Predictive performance was presented as sensitivity, specificity or predictive positive or negative values. For one study (Martin et al., 2014), we calculated the predictive performance based on a figure presented in a referred study (Page et al., 2003).
3.3 | What are the estimated resources used, costs and/or cost-effectiveness of the multivariable models and methods?
The estimated costs and/or cost-effectiveness of each study are presented in Table 2. Two studies (Jokela & Pienihäkkinen, 2003;Zavras et al., 2000) were cost analyses and hence only presented the costs, while the other four (Higashi et al., 2002;Holst et al., 1997;Holst & Braune, 1994;Martin et al., 2014) presented both costs (resources used) and outcomes. In two studies (Holst et al., 1997;Holst & Braune, 1994), the costs were constituted by the resources used at the clinics, presented as mean time spent per child, and the outcomes were measured as the difference in caries prevalence at follow-up between the test clinic and the comparator clinic. One study used QALYs as outcome (Higashi et al., 2002), where QALY weight decrements of 0.02 and 0.07 were used for mild and severe periodontitis, respectively. In one study number of teeth preserved was used as an outcome measure (Martin et al., 2014).
Four studies (Holst et al., 1997;Holst & Braune, 1994;Jokela & Pienihäkkinen, 2003;Zavras et al., 2000) showed cost savings due to the use of methods or multivariable models for the prediction of caries; hence, they were assumed to be cost-effective. The other studies (Higashi et al., 2002;Martin et al., 2014) showed increased costs but improved outcomes. Whether they were considered cost-effective depends on the willingness-to-pay of the decision-maker.
3.4 | What are the potential strengths and weaknesses of the methodologies of the included studies? Table 3 presents the strengths and weaknesses of each included study. Samples concerning target population, setting and time of recruitment, number, age and sex of included participants, disease prevalence at baseline were reported adequately in most studies. Several shortcomings were found regarding the predictive method/model and in particular for the economic evaluation. Overall, the prognostic prediction method/model was described, but not to permit replication. There was an incomplete description of the examination methods and measurements and of the outcomes to be predicted.
Also, the economic evaluations were often not completely performed or reported, for example, regarding the perspective of the analysis and sensitivity analyses, but the results were most often clearly reported regarding the main outcome.

| To what extent may the evaluations help decision-makers to direct resources efficiently?
All studies showed that the cost-effectiveness of the models or methods for the prediction of caries and periodontitis improved when the risk of caries or periodontitis is high. This finding is valuable for a decision-maker; thus, the decision-maker must consider the risk level in the target population before deciding on the effective methods.
Apart from this finding, the studies are of quite low relevance for a decision-maker, as they cannot easily be generalized or transferred to settings other than those studied. searched, which is considered to be an effective way to identify relevant additional publications (Whiting et al., 2008). It is implied that MEDLINE, Embase and NHS EED cover the majority of relevant economic studies (Glanville & Paisley, 2010).

| DISCUSSION
A major problem for the quality appraisal of economic evaluations lies in adapting methods to meet the challenges that arise as a result of the additional component of resources and costs to the prediction models and methods. As we did not find any published checklist for critical appraisal of prognostic prediction studies in combination with an economic evaluation, we designed and applied such a protocol and adapted it to the review questions.
T A B L E 3 Critical appraisal of included studies on economic evaluation of prognostic prediction methods and multivariable models of caries or periodontitis Results not transferable to a setting of men and women As description of method for prediction is limited it is questionable whether analysis is helpful for a decision-maker Note: Studies assessed using tool presented in Table 1 and results of appraisal summarized as strengths and weaknesses.

| Discussion of results
The result on paucity of studies was not unexpected, as the number of economic evaluations is also scarce regarding diagnostic methods (Christell et al., 2014). In other areas of oral health care, such as preventive and restorative interventions, significantly more health economic evaluations are found (Eow et al., 2019). One reason for the paucity of economic evaluations of prognostic predictive studies is probably that evidence is limited regarding the validity of existing prognostic predictive methods (Mejàre et al., 2014;Senneby et al., 2015) and multivariable prediction models (Cagetti et al., 2018;Tellez et al., 2013) to predict caries and caries development. Regarding periodontitis, a systematic review recently concluded that prediction models were deemed to be of "high risk" of bias due to the poor handling of data and the lack of validation (Du et al., 2018). Thus, valid and reliable data on the per- shown that a systematic approach, structured in several successive steps can be helpful in analyzing all relevant costs  and that the transferability of cost-data between settings in different countries is limited, as the circumstances can differ greatly (Christell, Birch, Hedesiu, et al., 2012). The estimation of resource use in minutes, as presented by two included studies (Holst et al., 1997;Holst & Braune, 1994), may increase the transferability of the results to other settings. Finally, agencies for health technology assessment in different countries often require the results of costeffectiveness in terms of cost per QALY gained to enable comparisons between methods. Only one study, which assessed the prediction of periodontitis, presented the results in this way (Higashi et al., 2002).
Major challenges exist when conducting economic evaluations of prognostic prediction models and methods used clinically. There is a need for economic evaluations of prediction models and methods with various perspectives, well-defined research questions, and measures of the cost and cost-effectiveness.

| CONCLUSIONS
Although prognostic methods and models for prediction of risk for caries or periodontitis are used daily in general dental practice, there is no evidence that it is cost-effective to do so. This SR identifies a wide knowledge gap and highlights not only a paucity of economic evaluations of these models and methods but also reveals methodological shortcomings regarding the reporting in particular, of the economic evaluations. There is certainly a need for studies with welldefined research questions, various perspectives and measures of cost-effectiveness to identify key determinants of resource use to draw out how these determinants may be distributed within and between dental settings.