Literature review on applications
Interest in MCDA has grown in health care over the last few years.[45, 46] A targeted literature search was conducted to identify published English language studies that used MCDA within the context of oncology. Electronic databases (Medline, PubMed, ProQuest, Ovid, Embase and Web of Knowledge) were searched between 1 January 1980 and 31 August 2013. The time period set for databases search seemed appropriate to identify the large majority of the oncology-related MCDA studies published during the last three decades. Databases were searched using the free text terms: ‘multicriteria decision analysis’, ‘multicriteria decision analysis’, ‘multiple criteria decision aiding’, ‘multicriteria decision making’, ‘multicriteria decision making’, ‘multicriteria analysis’, ‘multi-attribute utility’, ‘multi-attribute utility’ and ‘multiple objective problems’. The above free text terms were combined with ‘oncology’ and ‘cancer’ using Boolean operators when necessary. This list of terms reflects the different terms used to refer to MCDA. Hand-searching for the specified period was undertaken for the following journals: International Journal of Multicriteria Decision Making (IJMCDM), Journal of Multi-Criteria Decision Analysis (JMCDA), European Journal of Operational Research (EJOR), Decision Support Systems (DSS), International Journal of Technology Assessment in Health Care, BMC Medical Informatics and Decision Making, Medical Decision Making (MDM) and Operations Research for Health Care. Reference lists from identified articles were scanned to find additional studies not identified by the electronic searches. Authors and experts in the field were contacted for help in identifying relevant studies.
The literature search generated eight bibliographic references. Retrieved articles were reviewed by the first author (G.A) to identify both applied area and MCDA methods used.
A study by Miot et al., applied the Evidence and Value: Impact on Decision-Making (EVIDEM) framework to field test an MCDA framework as a support for coverage decision making on a cervical cancer screening test liquid-based cytology (LBC). The decision was to be made by a private health plan in South Africa. The study design included two major steps: (i) health technology assessment (HTA) report and (ii) field testing by a committee composed of medical doctors (specialists and general practitioners), pharmacists and nurses. During the first step, a literature review was conducted, and committee input was sought to develop an HTA report on LBC. The HTA investigated 14 MCDA decision criteria of the framework organized in a matrix. During workshop sessions, committee members assigned: (i) weights to each criterion of the MCDA matrix to express their perspectives, (ii) scores for LBC for each criterion of the MCDA matrix based on data from the by-criterion HTA report and (iii) the qualitative impact of system-related criteria on the appraisal. In this study, the HTA report was entered into an interactive web prototype (Tikiwiki v2.2), while weights, scores and impact obtained from committee members were entered into Excel software. Post-testing survey on the adoptability and utility of the EVIDEM approach indicated that the committee felt the framework brought greater clarity to the decision-making process and was easily adaptable to different types of health interventions.
Cunich and colleagues developed a grounded MCDA web-based decision support template, Annalisa© (AL). Annalisa was pilot tested as a decision support tool for prostate cancer screening (ALProst) in Australia. Being grounded in MCDA, AL ‘enables widely varying balances between intuition and analysis, rigour and relevance, and complexity and practicality’. The decision attributes specifically the benefit and potential harms of prostate cancer screening were included into the decision support tools. Attributes were identified by means of a literature review. Face-to-face interviews were conducted with a sample of primary care physicians referred to as general practitioners (GPs) in the study. During the course of the interviews, GPs nominated past decisions, stated prior preferences on PSA, watched an introductory video of ALProst, had hands-on experience with ALProst and rated statements on the template, AL and ALProst. Attributes weights were elicited during the interview. The authors believe that AL can be useful at all levels of the health care system.
Vidal et al., used the analytic hierarchy process (AHP) ‘to develop a decision support tool in order to assist pharmacists choosing the anticancer drugs that can be produced in advance’. The setting of this study was the pharmacy department of a French hospital. Criteria and subcriteria for the AHP framework were identified by means of expert interviews and literature review. The authors noted the advantage of the AHP framework and suggested opportunities for future research.
Liberatore et al., discussed the development and implementation of an AHP-based decision-counselling protocol for prostate cancer screening. The decision-counselling protocol was designed by a multidisciplinary research team. The protocol was tested in four primary care practices in Philadelphia. The protocol was tested to assist men in deciding whether they would undergo both a digital rectal examination (DRE) and prostate-specific antigen (PSA) testing. The study indicated that the successful application of the decision-counselling protocol is appropriate within primary care setting only if it is well structured and coordinated by an experienced analyst.
Richman et al., proposed ‘a strategic, computer-based, prostate cancer decision-making model based on the AHP’. A group of physician expert and a group of patients were sought to validate the model. The validation process consisted of comparing and ranking management choices of prostate cancer. A particular characteristic of this study is the analysis on quality of life issues. The model was found to be a good fit for weighting priorities while being superior to traditional models.
Dolan et al., conducted a pilot test of a decision aid designed to help patients choose among recommended colorectal cancer screening programs at two internal medicine practices in Rochester, New York. The study population consisted of patients at average risk of colon cancer being seen for routine appointments. The study was designed as a randomized controlled trial comparing a patient decision aid based on multicriteria decision-making theory with a simple educational intervention. The study made use of the AHP defined by the authors as ‘a multicriteria decision-making method that was specifically designed for decisions that require integration of quantitative data with less tangible, qualitative considerations such as values and preferences’. All ‘judgements’ were entered and tested by the collaboration decision software expert choice, so the total weight could be calculated. The study found that the decision support tool improved patient decision-making process. However, the decision support tool was found not to have an effect on decision implementation.
Carter et al., compared the analytic network process (ANP), the AHP and the Markov process all together in the evaluation of the optimal post-lumpectomy treatment strategy. The treatment alternatives considered in the study were as follows: observation, radiation, tamoxifen, radiation and tamoxifen combination, and simple mastectomy. The patient was a 74-year-old woman with a mammographically detected, non-palpable early-stage breast cancer. Combined radiation and tamoxifen was found to be the preferred treatment across all three methods. Of the three methods compared in the study, the AHP was the quickest to generate results. The Markov process and the two MCDA approaches (AHP, ANP) used in the study performed well. The study shows that the choice of a particular method depends on the context, as well as on the requirement set by the type analysis to be conducted.
Dolan determined whether patients are capable and willing to use the AHP to help make clinical decisions. The author used 20 volunteers to perform an AHP analysis of the choice among five screening regimens for colon cancer. Dolan hypothesized that ‘a substantial proportion of patients, arbitrarily defined as 50%, would be capable of using and willing to use the AHP to help make cancer screening decisions’. The analysis was conducted using the standard AHP software package Expert Choice. The results of the study suggest that a lot of patients were competent to use AHP to assess difficult clinical problem. Overall, Dolan showed that the AHP can assist in health care decision making.
In this literature review, six[47, 48, 50, 52-54] of the eight studies focused on decision making for cancer screening. Cancer screening decisions can be complex, and decision aids have been developed to assist patients and providers in these decisions. The debate over cancer screening is interconnected to the issues of overdiagnosis and overtreatment. These issues are part of an even larger concern over delivery of unnecessary medical care. Four studies[49, 50, 52, 54] demonstrated applicability and acceptability of the AHP technique for patients within the health care setting. The AHP is one to the widely used MCDA tool in health care because of its extensive applicability and user-friendly characteristic. Several applications of AHP have been published in different areas of health care such as performance assessment, discharge planning, performance measurement, resource planning,[58, 59] shared decision making and many more. Comprehensive reviews of the application of AHP both in health care and other fields are covered elsewhere.[61-66]
Imagine that the MDT from a cancer centre intends to select the best treatment among three potential alternatives (ai). The criteria (C) described below are those against which the decision has to be made.
- Overall survival (C1): C1 is measured on a quantitative scale (time in months) and is expected to be maximized;
- Safety profile (C2): C2 is measured on a qualitative scale (Low, Mild, Moderate, High) and is expected to be minimized;
- Compliance to treatment (C3): C3 is measured on a qualitative scale (Low, Fair, and High) and is expected to be maximized. This criterion applies only to oral drugs.
- Cost (C4): C4 is measured in dollars ($) on a quantitative scale. It is also expected to minimize this criterion.
The MDT has compiled information on the criteria that allowed for the construction of the following performance matrix (see Table 1).
Table 1. Performance matrix*
|Alternative||Overall survival (Months)||Safety profile (Low-High)||Compliance‡(Low-High)||Cost ($)|
|C1 (max)||C2 (min)||C3 (max)||C4 (min)|
There are many MCDA methods that can address choice problems. However, compensatory MCDA methods such as weighted sum models are not suitable for oncology decision-making problems. In fact, an MDT would be interested in treatments that are both effective (efficacy criterion) with and acceptable safety profile (safety criterion). A compensatory effect would arise if an alternative that exhibits a very good performance on efficacy and a poor performance on safety is preferred to another that has a good performance on efficacy criterion and a fair performance on safety, on the basis of a weighted sum score. Under these circumstances, it is worthwhile considering non-compensatory methods such as outranking models. The illustration of the application of outranking models is carried out using the ELECTRE I model.
ELECTRE I belongs to a family of methods (outranking models) that has been developed as an alternative to compensatory models. This type of models uses outranking relations to compare different alternatives pairwise. The application of ELECTRE I involves two steps. The first step consists in constructing a crisp (yes or no) outranking relation noted ‘S’, meaning ‘at least as good as’. As part of the comparison of a pair of alternatives (a,b), the following situation may arise:
- a is at least as good as b, but the inverse is not true: a is therefore strictly preferred to b (commonly noted a S b)
- b is at least as good as a, but the inverse is not true: b is therefore strictly preferred to a (commonly noted b S a)
- a is at least as good as b, and the inverse is true: a is therefore indifferent to b (commonly noted a I b)
- a is not at least as good as b, and the inverse is true: a is therefore incomparable to b (commonly noted a R b).
For each pair of alternatives (a, b), the following questions need to be addressed to establish the outranking relations.
- Does ax S ay?
- Does ay S ax?
Given (a, b), an alternative a is said to outrank another b if the following conditions are both true:
- Concordance condition: a outperforms b on enough criteria of sufficient importance, corresponding to the sum of the criteria weights (voting powers) kj.
- Veto condition: a is not outperformed by b in the sense of recording a significantly inferior performance on any one criterion. In other words, a veto represents a maximum difference, in terms of performance of alternatives against a criterion that cannot be compensated.
The second step in the application of ELECTRE I deals with exploiting the outranking relations to find the kernel, meaning the set of non-outranked alternatives. This can be achieved based on the algorithm presented in Fig. 2.
Figure 2. Algorithm for exploiting outranking relations to find the kernel. * A cycle contains alternatives that are considered indifferent.
Download figure to PowerPoint
A detailed application of ELECTRE I is presented in the next paragraphs. To apply ELECTRE I, concordance index (CI), voting powers (kj) and veto values (vj) have to be determined by the decision-makers. For the purpose of this hypothetical scenario, let us consider that the MDT was able to set the values, respectively, for CI, kj and vj. These values are included in the modified performance matrix as shown in Table 2.
Table 2. Performance matrix* for ELECTRE I
|CI= 0.55||Overall survival (%)||Safety profile (Low-High)||Compliance‡ (Low-High)||Cost ($)|
|Alternative||C1 (max)||C2 (min)||C3 (max)||C4 (min)|
|kj|| 0.35 || 0.30 || 0.15 || 0.20 |
|vj|| 5 || – || – || 500 |