Economic evaluation of applying the Canadian Syncope Risk Score in an Australian emergency department

To evaluate the Canadian Syncope Risk Score (CSRS) in syncope patients presenting to the ED from an economic perspective, using very‐low and low‐risk patients (CSRS −3 to 0) as a threshold for avoiding hospital admissions.


Introduction
Syncope is a common ED presentation estimated up to 3% of all ED visits. [1][2][3] In Australia alone, there were 90 000 syncope presentations in 2017-2018. 4 Due to the heterogeneity of underlying causes, with some potentially life threatening, the management of syncope has been problematic especially following nondiagnostic evaluation in ED. [5][6][7][8] This has led to a conservative approach and high hospitalisation rates. 9 This approach is not only costly but adds little to the overall management and prognosis of patients admitted with syncope. 10 Risk stratification using clinical decision rules are often employed to guide patient disposition and reduce unnecessary admissions; however, none have been robust enough for universal adoption. 11 The Canadian Syncope Risk Score (CSRS) has been the latest decision rule derived and validated in a large multicentre cohort demonstrating robust prognostication. 12 The CSRS encompasses nine predictors, producing a score between À3 and 11, each linked to a risk category, based on the likelihood of serious adverse event (SAE). 12 We hypothesise that if low-risk patients (CSRS scores À3 to 0) can be safely discharged from ED, substantial improvements to the efficiency of patient management and disposition could be achieved. This is important for decision-makers who need to strike a balance of safe service delivery within a fixed budgetary framework. 13,14 This evaluation has been undertaken alongside a clinical validation study assessing the accuracy and safety of the CSRS in the Australian public hospital setting, described in detail elsewhere. 5,6 The aim of the present study is to perform an economic evaluation using modelling to predict cost differences if low-risk patients (CSRS À3 to 0) were discharged compared to baseline usual practice described in our study cohort.

Methods
To ensure consistency and rigour when reporting findings, this evaluation follows the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist 15 (Table S1).

Study design
A 1-year, prospective observational study was undertaken at Redcliffe Hospital, an outer metropolitan public hospital near Brisbane which receives approximately 65 000 ED presentations per year. 5 Patients over 18 years, presenting to ED with undifferentiated syncope were enrolled. Patients were excluded if they had a persistently altered GCS, seizure, intoxication, language barrier, dementia, major trauma or a serious diagnosis made in ED. 5 Demographic and clinical assessment information used for calculating CSRS was prospectively collected from all participants. Patient management and disposition were left entirely to clinician discretion. Patients were followed up through phone call, 30 days after their initial ED presentation. Serious adverse outcomes of interest based on the original CSRS derivation study included death, arrhythmias, myocardial infarction, structural heart disease, aortic dissection, pulmonary embolism, severe pulmonary hypertension, subarachnoid haemorrhage, significant haemorrhage, procedural intervention and any other serious condition that required the patient to return to the ED for treatment, if not detected during the initial presentation. 12 Ethical approval was granted by The Prince Charles Hospital Human Research Ethics Committee (HREC/17/QPCH/48).

Modelling approach and rationale
As the study was purely observational, the CSRS was not utilised prospectively to influence decisionmaking or patient disposition. Hence, disposition from ED reflected current usual practice following individualised patient evaluation. The CSRS was calculated by a research nurse following data collection and applied retrospectively to determine each patient's risk category. These were then correlated with patient disposition and 30-day SAE. Accordingly, we simulated a hypothetical model comparing an intervention group, where the CSRS was used to discharge very-low and low-risk patients and a comparison group which reflected usual practice for managing syncope patients in our ED. Patient disposition according to usual practice relied on clinician judgement and risk tolerance but to our knowledge, did not utilise any structured approach. The rationale for discharging very-low and low-risk CSRS patients is based on a recent multicentre validation study which demonstrated 30-day SAE of 0.3% and 0.7% respectively. 16 This represents a cohort where discharge from ED is likely feasible without compromising safety in realworld clinical practice, acknowledging that a 0% miss rate is highly impractical when dealing with a complex condition like syncope.

Decision-analytic model
A decision-tree model was developed to evaluate application of the CSRS, where 'branches' of the tree represent possible outcomes for the patient cohort. 17 A pictorial representation of the model which shows the possible paths of movement for patients is shown in Figure 1. Each branch has a cost and health outcome associated with it. The model was used to estimate the cost of managing a hypothetical cohort of 1000 patients who presented to the ED with syncope and was constructed and analysed in MS Excel.
The estimates for length of stay (LOS) used in the model come from the data that was collected in the study. For usual care estimates, all patients from the dataset were included in the analyses of LOS, with no attention paid to their CSRS score. This reflects the clinical reality where the CSRS was not in use for usual care. For the intervention arm, we analysed LOS data from patients who had a CSRS of 'low' or 'very low'and were therefore deemed safe to dischargeto derive LOS estimates as if the tool was in use.
Patient characteristics and disposition for those presenting with syncope from the study population are shown in Table 1. The average age of patients was 55.6 years (standard deviation [SD] 22.7) with 37.1% being male. The majority (98.6%) of patients lived independently. Hypertension (37.1%), a predisposition to vasovagal symptoms (24.4%) and dizziness/ presyncopal on standing (22.7%) were the most common medical histories. The demographic and clinical characteristics of the study population are similar to those reported elsewhere for ED presentations in the general Australian population. 18 Input data for the model Movement throughout the decision tree is based on transition probabilities, which were estimated from various sources (Table S2). The model was run over a 1-year time horizon and as such, discounting was not applied to costs or health outcomes, as per published guidelines for this type of study. 19

Transition probabilities and costs
Data collected from the validation study cohort were used to estimate all transition probabilities. 6 For the estimation of usual care probabilities, data included all patients in the final dataset (n = 283), as this reflects what happened in the reallife setting with no application of the CSRS decision rule. To estimate probabilities for the intervention, we used data from patients who returned a CSRS score between À3 ('very low risk') and 0 ('low risk') (n = 203). Data were collected for 12 months, sources of evidence and values used in the model are shown in Table S2.
Costs associated with each branch of the decision-tree were measured and valued in 2018 Australian dollarsthe year the study beganand the National Hospital Costs Data Collection (NHCDC, Round 21) was used for costing syncope-related hospital utilisation. We used a previously published currency conversion method to convert costs from index values to 2022 Australian dollar values (Table S2). The costing perspective for the present study is from the hospitalcosts associated with treatment, administration and monitoring in the hospital are included, but we did not include any patient costs associated with their visit. 20 A detailed breakdown of how cost data was used in the model can be found in Table S3.

Model outputs and handling uncertainty
This evaluation is designed to estimate the expected value for money by applying the CSRS for patients who present to ED with syncope-related symptoms, compared to usual care. Cost-effectiveness is shown using the incremental cost-effectiveness ratio (ICER), where the mean change to costs associated with the new model of care is divided by the mean change in hospital admissions. 21 The ICER is compared against a cost-effectiveness threshold, which is assumed to be the willingness to pay (WTP) for an increase in health benefitin this case admissions avoidedwith ICERs that fall under the threshold deemed as 'cost-effective'. The threshold for the present study is AU$28 000, which is based on the most recently published estimates for the Australian healthcare setting and is represented in Figure 2 by the solid line fitted through the cost-effectiveness plane's origin. 22 To measure the impact of uncertainty relating to the model's inputs on its outputs, probabilistic sensitivity analysis was undertaken. One thousand iterations of the model were run using Monte Carlo simulation, where each simulation takes a random draw from each parameter's distribution. 23 However, given that the ratio of two numbers has awkward statistical properties, practical issues with using the ICER for decision-making exist. 24 To simplify this ratio information into a single number, the net monetary benefit (NMB) framework is used. Results are converted from the ICER to a NMB value, through the linear rearrangement of the ICER equation, as follows: The interpretation of costeffectiveness becomes particularly simple using the NMB and gives decision-makers a clear framework for choosing to adopt, or not adopt the intervention being evaluateda positive NMB indicates that a strategy is cost-effective and a negative NMB indicates that a strategy is not cost-effective. 25 Scenario analyses were undertaken to further explore uncertainty, and to test the model's robustness. This requires key parameter values to be changed, reflecting plausible decision-making scenarios.
Four alternate scenarios were examined: (i) where clinician acceptability of risk was altered to a CSRS score of À2 or lower; (ii) where a decision-maker's WTP for avoided admissions is $0; (iii) where costs associated with time in hospital were made equivalent by ensuring the average LOS was the same for both the usual care and the intervention; and (iv) the rate of admission was made equivalent for usual care and the intervention, to handle an over-estimate of reduced chance of admission associated with applying the CSRS.

Base case analysis
For a modelled cohort of 1000 patients, the estimated costs associated with applying the CSRS were $1 536 006, compared with $2 931 225 for usual care ( Table 2). The estimated number of admissions was 229 when the CSRS was applied and 398 for usual care. As a result, this means that for 1000 presenting patients, 169 admissions could be prevented, at a cost-saving of $8255 per admission.

Probabilistic analysis
The results of Monte Carlo simulation can be seen in Figure 2, with the majority (99.9%) of all ICERs being under the WTP threshold of $28 000.
More than three-quarters (78.6%) of all simulations showed that applying the CSRS is a cost-saving approach to managing syncope patients in the ED. A NMB analysis was also undertaken and provides easily interpreted resultsthe mean NMB for the application of the CSRS is $6 599 659, showing strong support for the intervention being better value for money compared to usual care. Put simply, applying the CSRS reduces costs and reduces the number of admissions.

Scenario analysis
Four scenarios were examined, testing the model's robustness and providing decision-makers with a wider breadth of new information. The results are summarised in Table 3.
In all scenarios, adopting the CSRS is the optimal strategy, compared to usual care. There was high certainty in these outcomes, with a greater than 99% probability that the CSRS was cost-effective in scenarios 1, 3 and 4. For scenario 2, when the decision-maker's WTP for an avoided admission is reduced to $0, the probability that applying the CSRS is cost-effective is still highwith 78.6% of simulations showing this.

Discussion
This economic evaluation shows that applying the CSRS is less costly and more efficient compared to the current practice of patient management. Cost-savings associated with the new approach exist because patients spend less time in the ED and there are fewer patients requiring admission to hospital for closer observation and a prolonged period of monitoring. Admission to hospital is associated with high service costs, particularly when LOS is also increased, so avoiding these costs has obvious system-wide benefits.
As with all health service innovations, there remains an implementation challenge associated with applying a new decision rule at 'the coalface' of a hospital. To encourage a smooth translation of this new evidence into practice, clinical staff working in the ED will require a period of training and consultation regarding the new approach. This is not unachievable, but those implementing new innovations should be mindful that many interventions found to be successful in research studies are not effective in translating to meaningful patient outcomes. 26 Our findings are robust to changes in the model's underlying assumptions, meaning that they provide decision-makers with pragmatic evidence across a wide range of potential uncertainties relevant to the healthcare setting. Significantly, the adoption of the CSRS remains costeffective despite alterations to key information, such as the decisionmaker's WTP for avoided admissions being $0. In this instance, decision-makers can, with a high degree of certainty, be sure that adoption of the decision rule will remain good value for money even in times when they have no capacity to pay for the introduction of an innovative approach to patient management. This is of particular importance in today's constrained economic climate, where hospital executives are under significant pressure to reduce expenditure and maintain high clinical outcomes. 14 We believe that the results are generalisable and useful for clinicians and decision-makers across Australia, particularly those who share similar resources and models of care.
The adoption of the CSRS is a useful bridging toolone which closes the gap between clinicians, who are focussed on patient outcomes; and hospital administrators who are interested in reducing low value care, minimising potentially preventable hospital admissions and maximising efficiency. Tools such as the CSRS have the capacity to empower clinicians to provide safe, acceptable and appropriate care, with the knowledge that the services being provided are high quality and efficient.

Limitations
Our study has limitations. As is common with modelling studies, the results are somewhat dependent on the assumptions made regarding the model's included parameters. We used a simplified decision model, acknowledging that trying to include every possible patient pathway would make the model unwieldy and impractical for decision-makers. Some key assumptions were made, including the decision to not differentiate between possible admission destinationsall admitted patients were treated as being admitted to the same place, which may not occur in reality, with some patients admitted to wards with telemetry capacity and others that are not. We also derived the definitions of what constituted a 'long' and a 'short' stay in hospital using expert opinion, in the absence of any published literature on the topic. We also assumed that the LOS associated with all SAE was equivalent, which may underestimate the true costs of major complications associated with syncope. With increased treatment and LOS costs associated with some AE and not others, it is likely that there is some underestimation of the true costs in our results, particularly for some of   the more SAE. This assumption was necessary due to a lack of reliable local data about AE costs, a model input that we could review when more precise data becomes available. We know that the underestimation has been applied to both usual care and the intervention, so does not unfairly disadvantage one approach compared to the other. Finally, due to variability in clinicians' risk tolerance and non-decision-rule factors, achieving a 100% discharge rate for patients is unlikely in real-world practice. A sensitivity analysis which explored different discharge rates and assesses the impact of these differences was not undertaken but is planned in future studies. Despite these limitations, the structure of the model is flexible and easily updated if new information or higher quality estimates become available. The model can be updated with data that is relevant to another jurisdiction, making our approach to evidence generation appropriate for decisionmakers outside Australia.

Conclusions
This evaluation provides new evidence regarding the economic benefit of applying the CSRS, for the management of people who present to ED with syncope. To our knowledge, the present study provides novel economic evidence and could serve as pilot evidence for sites in other jurisdictions who wish to adopt the decision-rule in their local context. This new evidence will help decision-makers choose cost-effective approaches for the management of patients presenting to the ED with syncope, as they search for efficient ways to maximise health gain from a finite budget.

Supporting information
Additional supporting information may be found in the online version of this article at the publisher's web site: Table S1. CHEERS checklist. Table S2. Input variables for the economic model. Table S3. Decision tree data source.