Implementing a virtual emergency department to avoid unnecessary emergency department presentations

EDs are necessary for urgent health concerns; however, many physical ED visits could be better treated in alternate settings. The present study aimed to describe the feasibility, acceptability and effectiveness of a Virtual ED to reduce unnecessary physical ED presentations at a large tertiary health service in Australia.

This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

Accepted 20 September 2023
Key findings Virtual care (or telehealth) involves remote healthcare interactions using technology for collecting, sharing and evaluating health information. 4Virtual ED is a strategy to manage large numbers of patients and target nonlife-threatening, 5 lower acuity presentations, making efficient use of resources and reducing wait times yet retaining appropriate management of patient outcomes. 6Virtual hospital care can lead to improved patient access, satisfaction, 7 clinical outcomes and health service efficiencies, including reduced costs. 8f a Virtual ED were to be effective and sustainable, it could reduce unnecessary ED visits, lower costs and alleviate overcrowding, although rigorous research is lacking to date.Recent international evidence shows Virtual ED is highly accepted and may reduce presentations to the physical ED. 5,9However, little is known in the Australian public health context, with observational studies underway to address these knowledge gaps, 10 The potential benefit of a Virtual ED is that it can be a more accessible way to connect patients with clinicians and support clinical decision-making at the point of care, enhancing timely interventions and accurate triage.However, whether alternative models of ED feasibly identifies the right patients and can align them with the right care in the right setting without the need for a physical ED presentation, remains unknown.Therefore, the present study aims to describe the feasibility, acceptability and effectiveness of a Virtual ED to reduce unnecessary ED presentations at a large health service in Australia.

Study design
This mixed-method observational study followed the Mixed Methods Studies in Health Services Research checklist reporting guidance. 11

Setting
The present study was conducted in a Virtual ED established during the SARS-CoV-2 (COVID-19) pandemic across Metro North Hospital and Health Service (MNHHS).The MNHHS is a metropolitan health service, encompassing six hospitals, and a large community and oral health directorate, approximately 18 000 staff and delivers services to a population over 1 million people, from north of the Brisbane River to north of Kilcoy in Queensland Australia. 12The study ran from April 2020 through to 31 March 2022.Ethics waiver was granted by MNHHS Human Research Ethics Committee (LNR/2020/QRBW/ 64414).

Inclusion criteria
The Virtual ED evaluation captured healthcare practitioner referrals only.Eligible referrers to the Virtual ED were general practitioners (GPs), Queensland Ambulance Service Officers and community-based health professionals whose patients meet the below eligibility criteria.
All patients of all ages were eligible to be referred to virtual triaging through the Virtual ED; however, to be deemed eligible they must (i) be referred by a health professional and (ii) live, are visiting or receiving treatment in the MNHHS catchment. 13rtual ED model of care The Virtual ED is a remote, triage and consultation service and is available to GPs, Queensland Ambulance Service Officers, hospital-based and community-based health professionals within the health service catchment area.
The Virtual ED model of care provides real-time clinician-to-clinician virtual consultations with an emergency medicine clinician (through telephone and/or video) for patients in the community, to impact avoidable ED presentations where clinically indicated by (i) building a pathway for healthcare practitioners in the community who are managing patients with emergency care needs to directly access expert emergency doctors and nurses; (ii) challenging traditional 'ED transition' pathways and enabling direct access from the community to hospital-based services instead of referring the patient to the physical ED; and (iii) identifying and developing opportunities for other referral pathways to expand the scope of services that are able to be accessed by community-based health professionals.
The Virtual ED is staffed with a team of emergency medicine consultants, emergency triage nurses and administration officer support.The telehealth calls are answered by a triage trained emergency nurse, who takes a brief history and performs triage in accordance with the Australasian Triage Scale.At this stage, an administration officer is also involved and will register patients on the Queensland Health management systems.The emergency medicine consultant then performs the patient's clinical consultation with the community-based health practitioner and develops a management plan for the patient.The Virtual ED team use the resources available to engage with its facility-based services, including, but not limited to, outpatient departments, rapid access clinics, inpatient teams and other specialty services to discuss the case and facilitate next steps in the patient's journey.These end points could include GP care, community services (provided by the Metro North Community and Oral Health Services), ambulance services, private specialist and residential aged care facilities.If the patient requires physical ED care, the Virtual ED team will liaise with the receiving facility's ED and inform them of the pending arrival and the patient's needs through a clinical handover process according to the National Safety and Quality Health Service standards.The patient's progress through to arrival to the department is also monitored through the Queensland Health Provider Portal ('The Viewer') to ensure quality and safety of the service.
Virtual ED commenced in April 2020 and was open 08.00-17.00hours, Monday to Friday until January 2022 when hours were extended to 08.00-18.00hours, 7 days per week (Fig. 1).

Data collection
Quantitative data were collected from administrative data sources (Virtual ED attendance record).Emergency triage nurses document quantitative data into a confidential Microsoft Forms (Microsoft Corporation, Office 365 for faculty).Online surveys were completed by GP referrers into the Virtual ED through online invitation.This survey was administered only to GPs who had referred a patient to the Virtual ED and not nonparticipating GPs.
The primary outcome for the present study was the feasibility of the Virtual ED model of care.This was evaluated using a process-evaluation which determines the intervention fidelity and adherence to the programme according to the five dimensions of the RE-AIM framework: Reach, Efficacy, Adoption, Implementation and Maintenance. 14The key evaluation dimensions and data sources used are outlined in Table 1.

Data analysis
All results are reported using the RE-AIM framework including descriptive statistics and non-numerical data which was used to describe in the context of the framework listed above as recommended for feasibility evaluation studies. 15

Reach and referrer characteristics
A total of 2080 patients with a mean age of 51.4 years (AE24.3)were referred to the Virtual ED service in the 24-month data collection period (Table 2).Referred patients were predominantly adults (n = 1846, 93%), with a minority of paediatric patients (n = 146, 7%) and n = 703 (35%) geriatric.Within the MNHHS catchment, 95% (n = 57) of postcodes were represented, referred by 235 (70%) GP practices and 877 (67%) referring GP clinicians.There were an average 212 referrals per month, with a 65.17% average growth rate of total referrals per month (Table 2).

Effectiveness and perceived benefit of the Virtual ED implementation
There were 2080 (100%) referrals to Virtual ED for management.Of which, 1992 (95.8%) were potentially avoidable ED presentations, with 1402 (70.4%) of calls being managed by Virtual ED and only 428 (21.5%) resulting in referral for a physical ED presentation.A total of 88 (4.2%) patients were calls from referrers simply to notify the Virtual ED clinician that their patient was on the way to the physical ED, and therefore these patients were excluded from the analysis pertaining to potential to avoid ED.The 428 referrals to the physical ED resulted in 178 (28.3%) admissions (Table 3).Of the 224 patients who were referred to Virtual ED by ambulance services, 54 (24.1%) were referred to the physical ED after Virtual ED consultation, resulting in 1377 (70%) cases being redirected away from the physical ED.
A total of 101 surveys were returned by 749 GP referrers (response rate 13.5%).Across these GP respondents, 89 (88.1%) perceived the Virtual ED service as helpful to very helpful.No referrers perceived the Virtual ED service to be unhelpful (Table 3).
After discharge, 578 (27.8%) of referred patients to the Virtual ED were primarily directed to follow up with their GP, followed by 446 (21.4%) in community services (n = 446, 21.4%), and 106 (5.1%) to the ambulance services and 76 (3.7%) to an outpatient department.A total of 224 (10.8%) patients were directly admitted to inpatient wards (Table 4).
Overall, six positions were required to deliver the Virtual ED service at a minimum for the duration of the project data collection period.This included five emergency medical consultants (1.25 full-time equivalent) to provide clinical consultations, assistance with pathway development, service expansion and clinical management planning.A single registered nurse performed triage, and case management on each shift, and when required oversaw clinical and operational service development.

Implementation and safety
All referrals conducted by the Virtual ED were triaged by the Australasian Triage Scale.A total of 18 (1.22%)patients unexpectedly required admission to hospital through the physical ED within 48 h, despite not been referred by the Virtual ED to hospital.No serious adverse outcomes occurred among patients who were diverted

Reach
The number and proportion of the total target population (GPs and primary care practices) who referred to the Virtual ED during the data collection period.We compared these metrics to the total GP and primary care practices within the Metro North Catchment.Reach also considered how the uptake of Virtual ED referrals changed each week.

Effectiveness
Whether the Virtual ED intervention avoided ED presentations, reduced ambulance presentations to ED, number of physical ED presentations, number of people managed by Virtual ED alone, number of presentations to ED in people redirected elsewhere, levels of referral and staff experience and satisfaction.

Adoption
The participation rate of the target clinicians engaged, and the staffing required to deliver the Virtual ED programme.Adoption also includes the number of referrals and consultations performed in and out of standard operating hours, and the location non-ED care was performed.

Implementation and safety
The percentage of patients requiring admittance to ED 48 h post-management by Virtual ED alone, and the staffing required to deliver the Virtual ED.

Maintenance and sustainability
Viability of the Virtual ED to continue to engage and service the target population particularly from a funding perspective through engagement with key stakeholders.This includes a descriptive summary of the approximate staffing, anticipated changes to service demand over time.
from the Virtual ED and subsequently admitted to the hospital within 24-48 h.All patients received appropriate medical care and accessed it correctly.

Maintenance of the Virtual ED programme
To ensure the viability and potential future sustainability of the Virtual ED, increased staff funding was required, given virtual emergency services is not currently within the public funding models.Expansion of the service to deliver extended hours to 14 h a day, 7 days a week emergency care has now occurred due to demand.Increased funding of medical officers, registered nurses and administration officers has been required to manage the service including Clinical Director and Clinical Nurse Consultant coverage, which was considered vital to the delivery and ongoing expansion of the service in a safe and efficient manner.

Discussion
The present study showed that a Virtual ED provides an alternate model which is safe and effective alternative to the physical ED model. 16The service enabled care to be delivered to the right patient, in the right setting and the right time, with only 24% of Virtual ED cases referred by participating GPs and other referral sources/ services resulting in an ED presentation.Only 18 (1.2%)patients unexpectedly required hospital admission within 48 h of the virtual consultation, showing a relatively low admission rate compared to unplanned hospital readmissions.Non-admitted patient data suggest that the rates of unplanned hospital readmissions generally range from as low as 0.3-11%, particularly for general medicine admissions. 17,18While not directly comparable, this highlights a relatively low admission rate for people initially managed by the Virtual ED alone.In addition, admissions were occasionally unrelated to the initial issue presented at the Virtual ED, but these data were not consistently recorded.These results provide a snapshot of case demographics that may present to a Virtual ED from community-based referrers (predominantly GPs).
The study's findings are supported by examples from international and Australian literature.A Canadian study found that 17% of 1036 paediatric patients referred to a Virtual ED required in-person assessment and less than 1% required hospital admission. 5In Victoria, Australia, a Virtual ED which followed a similar model of care, referral pathways and evaluation framework to our study resulted in over 70% preventable in-person ED presentations according to preliminary reports. 10Virtual community care models have also been effective in identifying early signs of clinical  7 These offer an overall lower cost solution which is feasible to manage acute care issues. 19he Virtual ED service had excellent reach within the MNHHS catchment, with referred patients from 96% of all postcodes and 60% from GPs and 27% from community care providers.The Virtual ED reached approximately 70% of all GP practices and practitioners, making it a feasible, safe and effective model for improving emergency care access.It is important to consider that this impressive reach may not be generalisable through the wider Australian community (i.e.outside the MNHHS catchment) or referrers from other areas of Australia, given the uncertain reach and adoption of telehealth for those with the greatest needs, including older populations, marginalised communities, those with limited technology options, digital literacy, rural and remote geographic locations, sociodemographic factors and poor connectivity. 20,21ur data do not include patient selfreferrals so we cannot determine the reach and adoption of the Virtual ED for these characteristics and demographics, but this is a focus of future research.Our GP reach was achieved through strong engagement with primary care networks and the GP liaison officers, allowing direct communication with GPs to disseminate information.Technology barriers were not a concern as referrals were made through phone calls instead of video.The main barrier GPs faced was limited time, and they were more likely to engage and reuse the service if they could connect immediately with a Virtual ED clinician.Unfortunately, COVID-19 restrictions prevented us from using our usual engagement strategies and fully understanding the barriers.Integrating information about Virtual ED into GP tools/software was another challenge, but improved integration led to increased engagement with the Virtual ED.
Virtual models of emergency care are highly acceptable to referring clinicians, notably GPs.Our study found over 87% of GP referrers found the Virtual ED to be helpful or very helpful for them and their patients.There is limited research in referring clinicians to Virtual ED; however, the results align with previous patient satisfaction reports in the literature (not clinicians), including a cross-sectional study in the USA that found 92% of patient respondents rated their experience with virtual urgent care as excellent, very good or good. 22Similarly, the Canadian observational study total of 1036 paediatric patients reported a 87% satisfaction rating. 5he acceptability and satisfaction of virtual care has grown enormously and now, based on over 30 years of literature, is strong to support its continued exploration as a patientcentred and clinically suitable alternative model to in-person care. 23However, we note that we surveyed only GP referrers and no other clinicians and services were surveyed, and we did not survey our patients.This limits our interpretation slightly around adoption and satisfaction.For example, our ambulance services have their own 'Clinical Hub' (external to the Virtual ED) which can manage some calls within the ambulance service and can divert away from hospital transport where indicated.We are unable to determine whether the awareness and uptake within these Unknown NA †Standard operating hours were 08.00-17.00hours Monday-Friday from April 2020 until January 2022, In January 2022 these hours extended to 08.00-18.00hours 7 days/week in line with funded service growth decisions.‡Other referral sources included any non-listed clinician/referral pathway, including but not limited to private specialists, nursing and allied health, inpatient registrars and medical officers, pharmacists.§Any non-listed clinician/ referral pathway, including but not limited to private specialists, nursing and allied health, in-patient registrars and medical officers, pharmacists.GP, general practitioner.
clinical groups was lower due to these reasons, or whether the 10% reflected the need for more clinical decision support from the experienced clinicians within the Virtual ED.This will be an area of future research which is important to evaluate.
The evolution of Virtual ED in Australia has been accelerated by the COVID-19 pandemic and has become topical due to unprecedented emergency service demand, clinician shortages and access block. 24While our Virtual ED model is viable, its long-term sustainability remains unclear, in-part due to their novel nature, but also due to lack of a clear model of funding through State, Federal, not-for-profit or private sources. 24Services where the patient is not present, and a community clinician is supported are also not currently within the current funding model.Therefore, significant stakeholder engagement, courageous leadership and investment is needed to continue the Virtual ED programme.Success of previous virtual wards in Australia hinges on experience in quality improvement, agile implementation, strong executive support, clinical leadership, trust and open communication. 19The results of this observational feasibility study lead to the hypothesis that continued investment would be cost-saving over time, by leveraging Virtual ED to help facilitate care for emergency cases, reduce travel requirements for patients and save healthcare dollars at the point of care. 24everal limitations should be noted regarding the present study.Firstly, due to the unique circumstances of the COVID-19 pandemic, the generalisability of the findings may be limited to this specific time period.The feasibility and acceptability of virtual care may have been influenced by the public health interventions in place to reduce community transmission and population movement.Future research should consider this and replicate the study under different conditions.Secondly, the lack of an action or historical control group limits the ability to draw causal inferences from the pre-post test results.Thirdly, the use of phone-only telehealth modalities during the study may not reflect current practices, where video consultations are available directly to patients.Fourthly, our acceptability survey response rate was modest and was explicitly sent to GP referrers and only those who had referred a patient to the Virtual ED.This increases the risk of a selection bias and does not explain or explore the acceptability or barriers to referring patients in GPs who did not refer a patient to the service.In addition, this does not consider other referrers to the Virtual ED, such as community services and ambulance services, which is important area for future research to evaluate acceptability of these refers, including clinicians who do not refer patients to Virtual ED services.Finally, the findings pertaining to the Virtual ED managing 70% of patients without referring them to the physical ED should be interpreted within the context of the referred cases involving communitybased referrers, primarily GPs.It remains uncertain how these outcomes might evolve with the inclusion of additional referrers and selfreferred patients.It is also important to acknowledge the possibility that our community-based referrers may already be engaging in self-triaging practices.Nonetheless, the observation period for our study coincided with the peak of the COVID-19 pandemic and stringent public health measures in place across Australia to discourage unnecessary hospital visits and in-person interactions.Consequently, self-triaging behaviours could have contributed to reduced referrals, providing the Virtual ED with a reasonably representative sample of patients who would typically seek care at a physical ED in the absence of a virtual alternative.However, further research and analysis, including the inclusion of patient self-referrals, are necessary to enhance the reliability of these findings, particularly to compare how referrals compare to typical walk-ins to the physical ED and ways these are managed.

Conclusion
In conclusion, the Virtual ED service was effective in preventing 70% of community triaged patients from presenting to the physical ED in a safe and effective way, with good uptake across GPs, community and ambulance services.Only 18 (1.2%) of these patients were unexpectedly admitted through the hospital ED within 48 h of being managed by the Virtual ED alone.While we cannot determine the acceptability of nonreferrers and how self-triaging may be influencing clinicians' decisions, our results suggest that a Virtual ED may be a suitable alternative for lower acuity patients for communitybased clinicians to seek additional support to make decisions about accessing secondary models of care.These findings highlight the potential of virtual care pathways in emergency management, providing an accessible and cost-effective solution to reduce ED waiting times and unnecessary presentations.These results can guide strategic planning towards long-term outcomes and aid decision-making.

Figure 1 .
Figure 1.Flow of patient through the Virtual ED clinician to clinician model of care.

TABLE 2 .
Total reach demographics †n = 53 postcodes are not included in this analysis as they were postcodes from patients not typically serviced by the health service catchment.‡Total calls to Virtual ED included 'enroute calls' referred to Virtual ED courtesy call only where the Virtual ED team was not involved in the decision to transport the patient to ED. GP, general practitioner.

TABLE 1 .
RE-AIM (Reach, Efficacy, Adoption, Implementation and Maintenance) domains used for the present study

TABLE 3 .
Effectiveness of the Virtual ED model © 2023 The Authors.Emergency Medicine Australasia published by John Wiley & Sons Australia, Ltd on behalf of Australasian College for Emergency Medicine.

TABLE 4 .
Referral rates, growth and uptake