ACADEMIC EMERGENCY MEDICINE 2012; 19:201–209 © 2012 by the Society for Academic Emergency Medicine
In 2007, the Institute of Medicine’s (IOM’s) Committee on the Future of Emergency Care recommended that a multidisciplinary panel establish a model for developing evidence-based protocols for the treatment of emergency medical systems (EMS) patients. In response, the National EMS Advisory Council (NEMSAC) and the Federal Interagency Committee on EMS (FICEMS) convened a panel of multidisciplinary experts to review current strategies for developing evidence-based guidelines (EBGs) and to propose a model for developing such guidelines for the prehospital milieu. This paper describes the eight-step model endorsed by FICEMS, NEMSAC, and a panel of EMS and evidence-based medicine experts.
According to the model, prehospital EBG development would begin with the input of evidence from various external sources. Potential EBG topics would be suggested following a preliminary evidentiary review; those topics with sufficient extant foundational evidence would be selected for development. Next, the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) methodology would be used to determine a quality-of-evidence rating and a strength of recommendation related to the patient care guidelines. More specific, contextualized patient care protocols would then be generated and disseminated to the EMS community. After educating EMS professionals using targeted teaching materials, the protocols would be implemented in local EMS systems. Finally, effectiveness and uptake would be measured with integrated quality improvement and outcomes monitoring systems.
The constituencies and experts involved in the model development process concluded that the use of such transparent, objective, and scientifically rigorous guidelines could significantly increase the quality of EMS care in the future.
Developing Evidence-Based Guidelines for Prehospital Care: A Call to Action
The past decade has brought growing recognition of the need to strengthen the evidence base in prehospital emergency medical services (EMS). In 2001, the EMS National Research Agenda recommended that EMS professionals apply the results of scientific research to improve patient care.1 Elaborating upon this call to action, the Institute of Medicine’s (IOM’s) Committee on the Future of Emergency Care in the United States Health System examined all aspects of emergency care in the nation, and published “Emergency Medicine at the Crossroads.”2 This 2007 report contained several recommendations (Table 1) aimed at linking evidence and practice in EMS.2
|• The Department of Health and Human Services and the National Highway Traffic Safety Administration, in partnership with professional organizations, should convene a panel of individuals with multidisciplinary expertise to develop evidence-based categorization systems for emergency medical services, emergency departments, and trauma centers based on adult and pediatric service capabilities. (Recommendation 3.1)|
|• The National Highway Traffic Safety Administration, in partnership with professional organizations, will convene a panel of individuals with multidisciplinary expertise to develop an evidence-based model for prehospital care protocols for the treatment, triage, and transport of patients. (Recommendation 3.2)|
Mindful of these two reports, the Federal Interagency Committee on EMS (FICEMS) and the National EMS Advisory Council (NEMSAC) set out to create a model for the creation of prehospital evidence-based guidelines (EBGs), using funding from the National Highway Traffic Safety Administration (NHTSA). The model generated outlines a structured eight-step process for the development, implementation, and evaluation of EBGs for local, national, and international EMS systems. The emphasis is on developing EBGs that are valid, reliable, clear, and readily implementable. It is intended that this model could reduce variations in care, synthesize existing evidence so as to be useful in a wide range of EMS settings, and ultimately improve patient outcomes and system efficiency.
The IOM’s recommendation to develop evidence-based protocols for prehospital emergency care responded to the historical overreliance on expert opinion in defining standards of practice. Such dependency on expertise-based guidance stemmed partially from a lack of other alternatives, chiefly high-quality, practice-changing research. This lack of evidence is not only an issue in the prehospital setting: it is estimated that in the 1980s, only 10% to 20% of all medical interventions were supported by evidence from randomized clinical trials.3 Further compounding the difficulties in establishing standards for “best” care was the existence of dramatic variations in practice among experts themselves.4,5 And even though the volume of scientifically derived medical knowledge has rapidly increased in the past several decades, this rapid crescendo in knowledge has outpaced its integration into bedside medical practice. Together, these factors prompted the development of evidence-based medicine (EBM) and motivated the medical community to use higher-quality evidence as the foundation for practice guidelines, rather than just consensus or expert opinion.6
In particular, in EMS there is a clear need for strengthening the relationship between scientific research and clinical practice. Many current EMS guidelines are based solely on preclinical research or on clinical studies performed in the hospital setting and lack validation for application in the field.7 The IOM found that half of EMS interventions are based on very weak evidence or no evidence at all, while only 4% are supported by high-quality evidence.2 This need to buttress EMS practice with additional scientific evidence is reflected in the inclusion of EBM objectives in the 2009 EMS educational standards.8 The proposed model for EBG development in EMS incorporates elements that have been developed and adopted by other health care agencies, as well as components that speak to the unique challenges and opportunities of implementation in the prehospital context.
There is also a risk that implementing new interventions prior to study in the EMS setting might consume valuable resources, while not necessarily yielding a survival benefit.9 For instance, the OPALS study group has suggested that advanced life support (ALS) practices are not necessarily correlated with improved survival in trauma and cardiac arrest.9–11 Nevertheless, ALS interventions are widely used by prehospital care providers.
The Model Development Process
The model development process was overseen by a steering committee of the Medical Oversight Committee of the FICEMS Technical Working Group. The steering committee was composed of members of the Medical Oversight Committee, representatives from NEMSAC, and five subject matter experts from the EMS community. The members of the steering committee were selected by virtue of their contributions to the field of EMS through seminal works that lay the groundwork for future research efforts or through the development of the field of EMS medical oversight. Other members were selected through work in the fields of guideline development particular to emergency medicine, expertise in critical appraisal and evidence-based medicine, and the arena of evidence implementation or knowledge translation. In September 2008, the steering committee held a multidisciplinary conference, with FICEMS and NEMSAC as cosponsors.12
In formulating the list of conference invitees, the steering committee considered other reports on EMS and the history of NHTSA’s interactions with other health care agencies, as well as all other relevant agencies and organizations that might be considered to have a stake in the process. The steering committee’s philosophy was to be as inclusive as was reasonably possible, so it waived registration costs and provided months of advance notification for the conference. The committee accepted that the outcome of the consensus conference would only be influenced by the interested and engaged agencies that agreed to participate. As all efforts were made to be inclusive, the steering committee deemed that it had, to the best of its ability and resources, done its utmost to mitigate bias with respect to the conference attendees. Data Supplement S1 (available as supporting information in the online version of this paper) lists the groups in attendance.
The conference planners agreed that given the diverse backgrounds and knowledge bases of the participants, it would be important to review the tenets of guideline development with all involved. Assuring that all participants had comparable baseline knowledge of the EBG approach was essential to the formulation of the proposed national EBG model. To these ends, a primer containing key information about EBM and the guideline development process was distributed to participants prior to the conference. A slate of lectures was developed to help focus the discussions during the small group feedback sessions (see Data Supplement S1).
Small group sessions were led by steering committee members and were designed to generate input for specific steps of the model. The primary goals of the sessions were to establish methods for evaluating the evidence for guideline development, to suggest approaches for translating the evidence into practice, to outline a method for revising and updating guidelines, and to discuss integration with other national EMS system development strategies.
The steering committee synthesized this information from the small groups and developed a preliminary EBG model, which was then presented to all conference participants for feedback. While mechanisms for resolution of disagreement were planned for, they were never invoked, as there was general support among attendees for the proposed EBG model. After this multistep process, the model was subsequently presented to, discussed by, and approved at separate public meetings of both FICEMS and NEMSAC, where there were scheduled periods for public comment.
There was strong consensus at the founding conference that a federally funded core program is needed to ensure success of the model. The concept of establishing an evidence-based practice center for EMS received enthusiastic support. The lack of an ongoing, consistent federal source of funding for the core EBG development program will clearly prevent the optimal use of this model.
The Evidence-Based Guidelines Model
After the multistakeholder conference, a final draft of the model was recommended by the steering committee and approved by NEMSAC and FICEMS (Figure 1). The presented model provides a framework for creating both guidelines and protocols.
Clinical guidelines are systematically developed statements that assist with decisions about the appropriate management of a given disease entity.13 They are “… intended to be flexible …” and should “… serve as reference points, not rigid criteria. They should be followed in most cases, but there is an understanding that they can and should be tailored to fit individual needs.”12
As conceived of by this national effort, there is a logical linkage between centrally developed EBGs and regional EMS protocols. The advantage of this contextualized approach is that it uses the best science available for guideline development while allowing flexibility in operationalizing evidence-based care at the individual system level. The deployment of EMS is inherently local, and there are wide variations in patient demographics, system size, financial resources, personnel training, and geography. Thus, while the development of “national” protocols for prehospital care would be possible, it would not be useful since local application necessarily creates variation. In fact, expecting complete homogeneity of care might lead to interventions that are inappropriate or even harmful in some settings. For example, there may be settings in which performing rapid sequence intubation in the field is appropriate (particularly in high-volume systems with extensive training and close medical oversight), while in other systems this may be ill-advised given the potential for complications and adverse events.
Thus, the national model focuses on the development of broad EMS guidelines based on the best available evidence. The intent is that these guidelines will be specifically contextualized at the regional and local level. Using the national EBGs as a starting point, EMS systems and agencies can develop more specific “blueprints” for patient care in the form of treatment protocols, algorithms, and standing orders. EMS care is generally directed by prespecified approaches, both evaluative and therapeutic, centering on particular patient presentations. Practically speaking, EMS medical oversight is operationalized by the step-by-step execution of mandated actions (treatment protocols) or by care maps that provide emergency medical technicians (EMTs) with decision trees based on clinical problems (algorithms). Finally, standing orders empower EMTs to administer therapies that would otherwise require a physician directive, such as the administration of opioid analgesics for acute pain. The eight steps of the national model are described in detail below.
Step 1: External Inputs
To maximize efficiency and minimize cost, a search for existing information resources would be the first step in EBG development. The ideal starting point would be high-quality systematic reviews, in which the evidence has already been evaluated for scientific rigor and synthesized in a standardized, clinically relevant format. Such reviews are often produced by evidence synthesis organizations such as the Cochrane Collaboration and other evidence-based practice centers, and their inclusion in the EBG process would help decrease redundancy.
Extant EBGs would be another possible source of input. Some regions have already developed prehospital guidelines using evidence-based methods.14,15 These would be evaluated from the model’s broader national perspective and potentially adapted and endorsed for widespread promulgation. Moreover, guidelines not geared to the prehospital setting might still have components that are useful with proper modifications and would be considered for inclusion in the EBG.14,16,17
Organizations such as the Guidelines International Network (http://www.g-i-n.net) and the U.S. National Guidelines Clearinghouse (http://www.guideline.gov) could assist with the identification of existing relevant EBGs. These organizations promote international collaboration and information sharing and help to maximize guideline dissemination and minimize redundancy of effort in EBG development.18 A final source of external inputs would be EMS scope of practice and educational standards, such as the National EMS Scope of Practice Model released by NHTSA in 2007.19
Step 2: Guideline Initiation and Evidence Review
Any EMS stakeholder could generate a proposal for a new guideline topic. Potential topics might be based on perceived clinical need or prompted by new findings in the scientific literature. Topics might relate to prehospital treatment, patient triage or transport by EMS systems, or other aspects of prehospital patient management. A centralized reviewing organization, or “core project team,” would receive and review proposals. Ideally the core project team would have representation from key national EMS organizations such as the National Association of EMS Physicians; their early involvement in the process could facilitate every subsequent step of guideline development. Proposals would ideally be submitted to the core project team in “PICO” format where the patient group, intervention under examination, control or comparison group, and outcomes of interest are specified a priori.20
In addition to external proposals, the core project team might identify areas of emerging interest in the EMS literature and generate topic proposals. Since systematic reviews form much of the substrate for EBGs, the core project team would also identify and catalog potentially applicable systematic reviews for future EBG development.
After identifying a suitable potential topic, the core project team would search the medical literature (including guideline repositories) for applicable information. This preliminary search would identify existing guidelines, systematic reviews, randomized controlled trials, and observational studies germane to the topic. When clinical trials of reasonable quality exist but have not yet been synthesized into a systematic review or meta-analysis, the core project team might recommend the need for (or conduct) such efforts. There would not be a minimum evidence threshold for proposals, but the decision to proceed with developing an EBG would undoubtedly be affected by the volume and quality of extant evidence uncovered in this preliminary literature search.
At the outset, it would be mandatory that all members of the core project team disclose affiliations and conflicts of interest, as incomplete disclosure threatens the integrity of the guideline development process.21,22 Experts called upon to write EBGs are often the same individuals who have researched a topic extensively. Thus, they have a vested interest in the outcome because they will naturally want to see their research conclusions affirmed. While this may not be a financial conflict, it must still be disclosed. The core project team would be thoughtfully composed to minimize potential conflicts of interest and maximize expertise in content, research, and guideline development methodology. To the extent possible, bias would be mitigated at every step in EBG development, and the final product should be assessed with a validated tool such as Appraisal of Guidelines for Research and Evaluation (AGREE II).23,24
Step 3: Evidence Appraisal
Once it is established that sufficient foundational data exists to develop an EBG, developers would perform a more formal and in-depth evaluation of the literature. Some of the inputs might have already been rigorously evaluated, while others might require varying amounts of additional evaluation. The core project team should generally apply recognized and validated quality evaluation tools such as AGREE II for guidelines, Meta-analysis of Observational Studies in Epidemiology (MOOSE) for observational studies, Assessment of Multiple Systematic Reviews (AMSTAR) and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) for systematic reviews, or Consolidated Standards of Reporting Trials (CONSORT) for randomized controlled trials.25–29 Guidelines already written and endorsed by other authoritative bodies would be rigorously evaluated. The AGREE II tool can be used to evaluate the quality of existing guidelines and can provide ongoing direction during guideline development.30 AGREE II analyzes guideline validity across six key domains such as editorial independence, stakeholder involvement, and applicability.25
High-quality, relevant existing EBGs would be endorsed by the core project team and would be contextualized, disseminated, and evaluated as described in Steps 4 through 8 of the model. Sometimes it may be appropriate to consider modifying EBGs that were developed for use in other environments. However, doing so through a nonstructured process may yield inappropriate applications in settings for which the original guidelines were not intended (e.g., modifying a hospital or clinic EBG for use in the prehospital environment). In this scenario, the ADAPTE tool might be used for the modification of existing guidelines. While it still requires validation, it was recently applied to asthma guidelines by the Canadian Thoracic Society with favorable results.31–33
Evidence-based guideline developers would proceed with developing guidelines de novo (Step 4) if the research substrate was solid and the topic was both clinically significant and relevant. Research on topics not selected would be archived for potential use in a future EBG.
Step 4: Guideline Development
Developing an EBG de novo is a labor- and resource-intensive process requiring broad stakeholder input. Contributions to the process from EMS personnel and from community members would be particularly valuable.34 Multiple national academic, physician, and paramedic organizations could potentially lend insight to EBG development. In addition, inclusion of international experts could lend a broader perspective and enhance the applicability of the finished product.
The Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) system should steer guideline development. The GRADE process uses a formalized, transparent, and reproducible system to assess the level of evidence pertaining to a specific clinical question and issues clear, executable recommendations, which are designated as either strong or weak.35–37 The GRADE process also gives significant weight to patient values and preferences when determining the strength of recommendations, and a broad consideration of potential benefits and harms is brought into play.38,39
As such, relevant, patient-centered outcomes should be prioritized by consensus early in the EBG development process and the risks and benefits of the proposed intervention should be carefully considered. Having used GRADE to assign a quality of evidence rating to the information gathered in Step 3, developers could then complete the EBG by issuing recommendations that are designated as either strong or weak as per the GRADE process. Of note, the GRADE process makes a clear separation between the grading of evidence quality and strength of recommendations. As a result, it is possible for an EBG panel to reach consensus on what is a strong recommendation for a strategy where benefits clearly outweigh risks and costs, and yet the quality of research evidence for this question may be limited.
Developers should be mindful of the EMS context when assigning a strength of recommendation. In some cases, an EBG might require several variations depending on system, demographic, or cultural issues. Contextualization should use a widely accepted, explicit approach to insure relevance of the EBGs to diverse settings, and prehospital situational factors may influence the strength of a recommendation as per the GRADE process.32 For instance, medications proven useful in hospital settings might be difficult to store on an ambulance. Such prehospital limitations may lead to a weak recommendation even if an intervention has been proven to be safe and effective in other settings. This may include consideration of taking a strong stand on not recommending some interventions that are commonly used in EMS.40
To enhance uptake and proper implementation at the local level, EBGs should offer clearly executable recommendations representing unambiguous guidance. Guidelines should be written so as to be applicable across widely diverse EMS settings. A common format should be used to describe levels of evidence and strength of recommendations.
To be worthwhile, EBGs must be implementable in “real-world” patient care settings. The Guideline Implementability Appraisal (GLIA) tool specifically evaluates and identifies barriers to guideline implementation.41 Developers should apply GLIA after the first draft is written to facilitate eventual implementation.42
Step 5: Model EMS Protocol Development
In Step 5, the broad guidelines that were developed in Step 4 will be “translated” into EMS protocols that are specific in their intent and application. A federally sponsored program could generate sample protocols that would then be tailored at the local EMS level to suit needs and resources.
The strength of a recommendation at the guideline level influences the likelihood that it will be implemented as a patient care protocol at the EMS system level. Strong recommendations should generally be a standard part of EMS practice, while weak recommendations are likely to be more selectively adopted.
Step 6: Guideline/Protocol Dissemination
A systematic strategy would be used to disseminate the guidelines and protocols generated in Steps 4 and 5 to targeted stakeholder organizations. In addition to more traditional publication methods such as print journals and textbooks, a multimedia approach using videos and podcasts would enhance user interest. Internet publication would ensure guidelines are readily accessible in a wide variety of practice environments. Furthermore, incorporating the guidelines into the national EMS education program accreditation and national EMS education standards would help to ensure influence at both the state and local system levels, as well as among individual providers. Adapting educational materials to the needs of the end-users would promote implementation and retention. Integration of EBGs with the continuing education requirements of the National Registry of EMTs would help to assure an ongoing method of integrating guidelines into the practice of EMS professionals.
Step 7: Implementation
If changes in EMS protocols do not result in changes in hands-on care, the entire motivation for developing protocols is undermined. Hence, once guidelines and protocols are disseminated as in Step 6, additional measures would be necessary to maximize their implementation.
Protocol implementation in the prehospital environment can be challenging. For instance, three studies have shown that paramedics have low rates of administering aspirin despite explicit protocols directing their use and that the causes for noncompliance are difficult to determine.43–45 This reflects the widespread difficulty with standardizing prehospital care and underscores the importance of the IOM’s call for improvement in quality evaluation in EMS systems.2
Multiple obstacles might challenge successful EBG implementation. Lack of common governance is a major hindrance: the deployment of EMS care is inherently jurisdictional, with different state and local authorities presiding over logistics, operational policies, medical oversight, and funding. In addition, insufficient training and lack of available resources might be barriers to local implementation of nationally endorsed guidelines.46 This is further compounded by the fact that conducting research that evaluates guideline implementation is difficult for a host of operational, political, financial, administrative, and ethical reasons. The few existing studies related to guideline implementation by health care professionals are of varying quality and have discrepant results. However, there is some evidence of improved patient and process outcomes with active, multifaceted, implementation interventions.7,47
Considering these obstacles, providing tools for implementation would be necessary to help ensure that the guidelines are translated into practice. Linking to EMS provider certification, agency accreditation, and funding could encourage practical uptake. Emphasis would need to be placed on the freedom for EMS systems to adapt the guidelines to their context using standardized tools (e.g., GLIA). Pilot testing of the implementation program and modifications based upon end-user feedback would likely improve uptake and implementation.47 Finally, health informatics and clinical decision support software that automatically integrate EBG usage, documentation, and evaluation into day-to-day EMS operations would facilitate initiation and promote ongoing usage and assessment of the guidelines.
Step 8: Evaluation of Quality of Care and Outcomes
Once guidelines and protocols are implemented, there must be follow-up with quality improvement evaluations, systems research, and outcomes research. To build on the implementability assessments of Step 4, pilot testing and feasibility studies should occur in the eighth step of development as well. Using data from multiple systems, pertinent quality improvement indicators should be monitored and benchmarks should be developed.
The National Emergency Medical Services Information System (NEMSIS) data set should be a key element, as uniform data definitions and recording form the foundation of any system evaluation.48 Optimally, systems implementing the EBGs would report their data to NEMSIS to allow pooling of information from many sources and system types. Some research projects would require additional data elements depending upon the nature of the investigation.
Building upon the foundation established by the EMS Outcomes Project (EMSOP) and the EMS Cost Analysis Project (EMSCAP), a variety of outcomes, including cost, should be examined.49–55 The EMSCAP team devised a framework to calculate EMS costs based on a wide variety of factors such as administrative overhead, communications, equipment, human resources, information systems, medical oversight, physical plant, and training.56 This standardized method could be applied to EBGs to determine the financial consequences of a given patient intervention. Its limitation is that is does not measure final patient outcome and is unable to evaluate cost effectiveness relative to other non-EMS medical interventions (refer to Data Supplement S2, available as supporting information in the online version of this paper, for more information).
The initial EBG implementation process outlined in Step 7 should include planning for outcomes whenever possible. Understanding that translating knowledge into practice is pivotal in improving quality of care, any barriers to implementation discovered in Step 8 should be identified, overcome, and reported in appropriate venues (e.g., peer-reviewed scientific journals).
A pilot study funded by NHTSA and Emergency Medical Services for Children evaluating the use of the national EBG development model is currently under way. While the original national consensus conference discussed use of the model by states, regions, and local EMS systems, the pilot study has already identified that even developing a relatively simple EBG is time- and resource-intensive. It is already apparent that authoring new EBGs will be largely beyond the means and expertise of unfunded “grassroots” efforts.
The National EMS Research Agenda and the Institute of Medicine’s “Emergency Medicine at the Crossroads” report clearly identified the need to directly link EMS practice to scientific evidence. However, increasing the presence of evidence-based guidelines in the prehospital setting will not be possible without the resources necessary to search, appraise, and contextualize the medical literature. Ultimately, implementing transparent, objective, and scientifically rigorous guidelines will be of immense benefit to patients and to the EMS professionals who care for them. The presented model outlines the necessary process and paves the way for future prehospital evidence-based guideline development, which has the potential to significantly increase the quality of EMS care in the future.
The authors acknowledge Dr. E. Brooke Lerner and Mr. Daniel Manz for their contributions to the EBG Model development process, as well as Ms. Angela Burba for her authorship of the Evidence-Based Guidelines Primer presented to conference participants.