Presented at the ICEM annual meeting, Halifax, Nova Scotia, Canada, June 3–7, 2006.
International Survey of Emergency Physicians’ Priorities for Clinical Decision Rules
Article first published online: 7 FEB 2008
© 2008 by the Society for Academic Emergency Medicine
Academic Emergency Medicine
Volume 15, Issue 2, pages 177–182, February 2008
How to Cite
Eagles, D., Stiell, I. G., Clement, C. M., Brehaut, J., Kelly, A.-M., Mason, S., Kellermann, A. and Perry, J. J. (2008), International Survey of Emergency Physicians’ Priorities for Clinical Decision Rules. Academic Emergency Medicine, 15: 177–182. doi: 10.1111/j.1553-2712.2008.00035.x
- Issue published online: 7 FEB 2008
- Article first published online: 7 FEB 2008
- Received August 9, 2007; revisions received October 3 and October 10, 2007; accepted October 12, 2007.
- clinical decision rules
Objectives: One of the first stages in the development of new clinical decision rules (CDRs) is determination of need. This study examined the clinical priorities of emergency physicians (EPs) working in Australasia, Canada, the United Kingdom, and the United States for the development of future CDRs.
Methods: The authors administered an e-mail and postal survey to members of the national emergency medicine (EM) associations in Australasia, Canada, the United Kingdom, and the United States. Results were analyzed via frequency distributions.
Results: The total response rate was 54.8% (1,150/2,100). The respondents were primarily male (74%), with a mean age of 42.5 years (SD ± 8), and a mean of 12 years of experience (SD ± 7). The top 10 clinical priorities (% selected) were: 1) investigation of febrile child < 36 months (62%); 2) identification of central or serious vertigo (42%); 3) lumbar puncture or admission of febrile child < 3 months (41%); 4) imaging for suspected transient ischemic attack (39%); 5) admission for anterior chest pain (37%); 6) computed tomography (CT) angiography for pulmonary embolus (30%); 7) admission for suicide risk (29%); 8) ultrasound for pain or bleeding in the first trimester of pregnancy (28%); 9) nonspecific weakness in elders (26%); and 10) CT for abdominal pain (25%). Between study countries, there was consistency in identification of clinical problems, but variation in prioritization.
Conclusions: This international survey identified the sampled EPs’ priorities for the future development of CDRs. The top priority overall was investigation of the febrile child < 36 months. These results will be valuable to researchers for future development of CDRs in EM that are relevant internationally.
Aclinical decision rule (CDR) is an algorithmic decision-making tool that is derived from original research using strict methodologic guidelines.1–3 Based on three or more variables from the history, physical examination, or simple tests, CDRs allow computation of a quantitative estimate of the probability of a certain clinical outcome or suggest whether or not a diagnostic or therapeutic intervention is required.1–3 CDRs are appealing because they can be adopted into clinical practice with relative ease and because they offer numerous potential benefits. For practitioners, they may reduce clinical uncertainty at the bedside, facilitate translation of clinical evidence to bedside practice, and improve patient flow. For patients, they may improve quality and consistency of care,4–7 and they usually decrease exposure to costly and potentially hazardous procedures.8 For the health care system, they may improve the utilization of resources.9–13
When determining the need for a new decision rule, several factors should be considered. These include: prevalence of the clinical condition, current use of diagnostic tests, the variability of practice patterns, the attitudes of clinicians, and the accuracy of their bedside practice.3 Ultimately, clinicians decide whether or not they will adopt a new CDR into their repertoire.14 Because perceived need is likely the most important predictor of ultimate adoption, we sought to identify the clinical problems emergency physicians (EPs) would most like to approach with a well-designed clinical rule. To identify these problems, we surveyed members of national emergency medicine (EM) physician associations in Australasia, Canada, the United Kingdom, and the United States.
Study Design and Population
We conducted a self-administered e-mail and postal survey of members of four national EP associations, using the tailored design method for survey design and administration.15 The research was conducted by the Clinical Epidemiology Program of the Ottawa Health Research Institute between July 2005 and April 2006. The project was approved by the Ottawa Hospital Research Ethics Board.
The survey included four countries/regions. We selected Australasia, Canada, the United Kingdom, and the United States, because the specialty of EM is well established in these areas. Furthermore, the majority of CDRs have been developed in English, so use of these tools would be most familiar to English-speaking physicians.
The original intent was to conduct an exclusive e-mail survey of 500 randomly selected members of four national EP organizations. We were unable to do so because of the constraints placed upon us by the individual associations. Thus our participants included all members of the Australasian College of Emergency Physicians (total membership n = 772), a random sample of 500 members of the Canadian Association of Emergency Physicians (total membership n = 1,675), a random sample of 350 members of the British Association of Emergency Physicians (total membership n = 1,700), and a random sample of 500 members of the American College of Emergency Physicians (total membership n = 23,000). Members who were not physicians or not currently practicing clinical EM were excluded. To ensure that each area was given equal weight when determining the rank of the priorities overall, a simple unadjusted average of proportions was performed.
Survey Content and Administration
The survey instrument was a four-page questionnaire primarily consisting of closed-ended questions. This study was part of a larger survey that contained questions pertaining to knowledge of, use of, and attitudes toward the Canadian C-Spine Rule and the Canadian Computed Tomography (CT) Head Rule; management of acute headaches; and physician demographic, professional, and practice setting characteristics. The section on clinical priorities for future development of CDRs was developed by the authors. It was pilot-tested on approximately 80 EPs at The Ottawa Hospital and the Children’s Hospital of Eastern Ontario. The list was subsequently revised to incorporate the feedback generated. The clinical priorities section consisted of a list of 26 clinical problems (21 adult and 5 pediatric). EPs were asked to select a maximum of five clinical problems for which they would find the development of a sensitive and well-validated CDR highly useful to their emergency department (ED) practice. We also included an open-ended question that gave respondents the opportunity to suggest clinical problems not previously listed. The questionnaire is available as an online Data Supplement at <link removed as it refers to this location>.
All participants received a prenotification letter that described the study and requested their participation. One week later, the survey instrument was sent along with a cover letter describing the study, assuring confidentiality, and providing instructions for completing the survey. Nonrespondents were sent a minimum of two reminder letters with surveys, at 4-week intervals. Members practicing in Australasia and the United States were contacted by postal mail only. Members practicing in Canada and the United Kingdom were sent the prenotification letter, cover letter, and two reminder letters by e-mail; two additional reminders were sent by post. To facilitate response, postage-paid, preaddressed reply envelopes were included with all postal surveys. No incentives were offered.
All data were entered into a SPSS (Windows Version 13.0, SPSS Inc., Chicago, IL) database. Three levels of data review were used prior to entering the data into the study database with single entry. During the data entry process, a validity check was done on a random sample of 5% of cases. Because no errors were noted, single data entry with three levels of review was maintained throughout the study.
Our primary outcome measure was the ranking of physician priorities for development of CDRs. The problems were divided into four subgroups: adult admission, adult imaging, adult management, and pediatrics. Priorities within each subgroup were identified.
Data were analyzed using SPSS (Version 13, SPSS Inc., Chicago, IL). Simple univariate descriptive statistics were utilized. Frequency distributions were generated for all closed-ended questions (the clinical priority of each problem and most physician demographic, professional, and practice setting characteristics). Continuous variables (e.g., age and years of practice) were collapsed into three categories whose parameters were set to facilitate a relatively equal distribution of respondents within each category. We also computed overall “top 10” clinical priorities based on unadjusted means by country.
In total, 1,150 of 2,100 EPs responded to the survey, yielding an aggregate response rate of 54.8%. Area-specific response rates were Australasia 54% (417/770), Canada 69% (339/491), the United Kingdom 45% (155/348), and the United States 49% (239/491). The numerator represents all returned surveys, while the denominator represents all surveys sent less those returned because the survey was undeliverable (n = 6) or the respondent failed to meet inclusion criteria (n = 16).
Physician demographic, professional, and practice setting characteristics are depicted in Table 1. For each region, more than 69% of respondents were male (overall 74%). Full-time employment in the ED varied from 52% in the United Kingdom to 88% in the United States. Fifty-two percent of American EPs had greater than 15 years’ experience, compared with roughly one-third of EPs in the other three study countries. With the exception of respondents from the United States, more than 74% worked at a teaching hospital. For selection of the respondents’ top five clinical priorities for the development of a CDR, our item nonresponse rate was 4.8% and our average response rate was 4.9 clinical problems selected.
|Demographic||Australasia (n = 417)||Canada (n = 339)||United Kingdom (n = 155)||United States (n = 239)|
|Employment status in ED|
|Years of practice|
|Annual ED visits|
Figure 1 illustrates the percentage of respondents that selected each of the 26 clinical problems as one of their top five. There was a large variation in selection rates between clinical problems, ranging from 4% for imaging of pelvic trauma to 62% for investigation of febrile child less than 36 months.
Figure 1 also organizes the 26 problems, subdivided into four categories of need. Admission for anterior chest pain, imaging for suspected transient ischemic attack, identification of serious vertigo, and investigation of febrile child less than 36 months were the top priorities in the categories of admission, imaging, management, and pediatrics, respectively. Two clinical problems in the categories of admission, management, and pediatrics were selected by greater than 25% of respondents, whereas four imaging problems were selected by greater than 25% of respondents.
Table 2 presents the top 10 clinical problems overall irrespective of category, categorized by area-specific rates of selection and rank. The top 3 clinical problems overall were investigation of febrile child less than 36 months (selected by 62% of respondents), identification of central or serious vertigo (selected by 42% of respondents), and lumbar puncture or admission of febrile child less than 3 months (selected by 41% of respondents). With the exception of United Kingdom physicians, respondents in different countries were fairly consistent with their prioritization of problems. When compared to the top 10 overall, as determined by the aggregate means, physicians from Australasia prioritized the same 10 problems as respondents overall. Respondents from Canada and the United States included 9 of the top 10 overall on their list of clinical priorities. United Kingdom respondents picked 7. While there was similarity in selection of clinical scenarios in general, the relative level of prioritization varied by country.
|Clinical Priority||% Selected by Physicians (Rank)|
|Overall||Australasia||Canada||United Kingdom||United States|
|Investigation of febrile child < 36 months||62 (1)||70 (1)||57 (1)||68 (1)||51 (2)|
|Identification of central/serious vertigo||42 (2)||47 (2)||47 (2)||30 (6)||45 (3)|
|LP/admission of febrile child < 3 months||41 (3)||34 (6)||43 (3)||24 (10)||61 (1)|
|Imaging for suspected TIA||39 (4)||39 (3)||41 (4)||34 (4)||40 (5)|
|Admission for anterior chest pain||37 (5)||36 (5)||29 (7)||43 (2)||41 (4)|
|CT angiography for pulmonary embolus||30 (6)||37 (4)||22 (12)||30 (5)||30 (9)|
|Admission for suicide risk||29 (7)||29 (9)||30 (6)||37 (3)||21 (13)|
|US for pain/bleeding in first trimester||28 (8)||31 (7)||29 (8)||19 (14)||35 (6)|
|Nonspecific weakness in elders||26 (9)||19 (10)||32 (5)||19 (13)||34 (7)|
|CT for abdominal pain||25 (10)||29 (8)||26 (9)||13 (19)||33 (8)|
A total of 157 respondents completed the open ended question that asked them to suggest other clinical problems for which they would find a CDR to be useful. The five most common suggestions were syncope/collapse (n = 34; 22%), deep vein thrombosis/pulmonary embolus (n = 14; 9%), back pain (n = 10; 6%), scaphoid injury (n = 9; 6%) and atrial fibrillation (n = 7; 4%).
Valid and reliable CDRs may provide widespread benefits to the patient, the physician, and the health care system. They potentially reduce needless variations in practice, lower health care costs, decrease physician liability for missed diagnoses or faulty treatment, and improve clinical outcomes. To prioritize the development of new rules, we conducted an international survey to identify which clinical problems are most vexing to practicing EPs. The top three priorities overall were investigation of febrile child less than 36 months, identification of central or serious vertigo and lumbar puncture, or admission of febrile child less than 3 months. There was consistency in the overall identification of clinical problems across the four surveyed areas, but they varied in the relative order of prioritization.
The scope of this survey was broad in terms of overall numbers, percentage of target audience sampled, and geographical area. The instrument we used was adapted from previously successful surveys,16,17 and new research that was pilot tested. With an item nonresponse rate of only 4.8% and an average response rate of 4.9%, we are confident that our results are a true reflection of the respondents’ priorities.
Whether due to their inherent benefits, or a paradigm shift toward evidence-based medicine, research on the development of CDRs has grown in recent years.1–3 This is reflected in a burgeoning medical literature, with the publication of many derivation and validation studies within various areas of medicine. Relatively few studies have examined the acceptability of specific rules18,19 or CDRs in general.17,19,20 Studies indicate that EPs are quite interested in CDRs, but there is no research on which CDRs practicing EPs would find most helpful. A pilot study that compared priorities in Europe and North America was presented in abstract form only.21
Emergency physician respondents from the four study areas were relatively consistent in their top 10 clinical priorities. This suggests that there are certain clinical problems that EPs find challenging regardless of their nationality. These problems are ideally suited for development of a sensitive and well-validated CDR.
For some of the clinical problems, there was striking consistency across areas. For example, the challenge of deciding when to admit a febrile child less than 36 months was ranked as the top priority by respondents from Australasia, Canada, and the United Kingdom and ranked as the second most important priority by respondents from the United States.
Conversely, certain clinical problems were ranked differently by nationality. For example, determining when to admit a patient for suicide risk was ranked as a “top five” priority by 30% of respondents overall and ranked 2nd among admission challenges and 6th among the problems overall. Respondents from the United Kingdom ranked this problem 3rd but their American counterparts ranked it 13th. This might reflect different perceived levels of risk; suicide rates are higher in the United Kingdom where suicide accounts for 18.1 deaths per 100,000 persons,22 compared with 10.7 per 100,000 in the United States.23 It could also reflect differences in access to psychiatric services or training in the assessment of patients at risk of suicide.
Similarly, U.K. respondents ranked lumbar puncture and admission of febrile child less than 3 months, and ultrasound for pain and bleeding in first trimester, considerably lower than respondents in the other three study areas. This is probably due to differences in clinical practice in the United Kingdom, where many EDs there do not provide services such as lumbar puncture for pediatric patients and ultrasound in threatened miscarriage, because generally these patients would be investigated by specialty teams.
Computed tomography angiography for pulmonary embolus was selected by 30% of respondents, making it the second highest clinical priority for imaging and 7th overall. Only 22% of Canadian respondents selected this priority, placing it 12th on their list. This could be due, in part, to increased awareness of the Wells criteria,24,25 a previously validated CDR for detecting pulmonary embolus, because this rule was developed in Canada. Observations like these suggest, but do not prove, that intercountry variation in prioritization of clinical scenarios are caused by a variety of factors including, but not limited to differences in patient demographics, models of health care and service delivery, education and training, resources, access to technology, and varying levels of medicolegal risk.
The principal implication of this research is that it could serve as a guide to prioritize further development of CDRs. This study has clearly identified physicians’ self-perceived needs. Adhering to the stages proposed for the development of CDRs, further research should be undertaken to determine prevalence of each perceived condition, inefficiency in the current use of diagnostic tests, variations in practice among physicians, and clinical accuracy of physicians in the management of the prioritized clinical problem. From there, derivation of CDRs according to strict methodologic guidelines may begin.
Of the top 10 clinical priorities, 4 relate to diagnostic imaging. Clearly there is great interest in appropriate use of imaging technologies. This finding raises interesting questions about the state of research on diagnostic imaging. As development of advanced imaging technology grows at a rapid rate, is clinical knowledge on optimal utilization of these technologies keeping up? Furthermore, what factors are driving the use of advanced imaging tools? Research should be conducted to assure that costly diagnostic tests are used in the most cost-effective manner. Otherwise, their development may do more harm, by rapidly increasing health care costs, than any good that might be realized by marginally improving diagnostic accuracy.
Other factors that influence prioritization of clinical problems could be explored. These include differences in models of health care and service delivery, patient demographics, resources, access to technology, training, and the medicolegal system.
The pediatric population, a distinct group of patients with unique needs,26 represents a significant percentage of ED visits. In a previous survey, EPs reported that they are less comfortable managing pediatric cardiopulmonary arrest, pediatric trauma resuscitation, and child abuse and evaluating the acutely febrile infant or child than they are handling the same problems in adults.27 Perhaps this lack of confidence in treating pediatric conditions is reflected in our study, where two of the top three clinical priorities were specific to the pediatric population.
Our target population was clinically active EPs, because CDRs are often utilized by these frontline providers. To reach these individuals, we sampled the members of four national EM associations. We have no information on how representative these doctors may be of all physicians who practice in the EDs of their respective countries. It is likely that EPs working in the same environment, regardless of affiliation with their national association, would have similar priorities. Thus, while the potential for sampling bias exists, it is unlikely to skew our findings to a significant degree.
To respect the privacy and confidentiality of respondents, the associations would not provide demographic information regarding nonresponders. Thus, an analysis to determine if responders differed from nonresponders could not be conducted.
Although response rates of greater than 80% are considered necessary in other realms of research, studies have shown that the average response rate for published physician surveys is 52%–54%.28,29 The United Kingdom and the United States response rates to our survey were slightly lower than this, but our primary outcome measure, the overall ranking of clinical priorities, achieved an aggregate response rate of 55%. This is slightly above the norm.
This international survey identified the sampled EPs’ priorities for the future development of CDRs. Respondents’ top clinical priorities overall were investigation of febrile child less than 36 months, identification of central or serious vertigo, and lumbar puncture or admission of febrile child less than 3 months. There was consistency in the overall identification of priority problems, but there were national variations within the priority list. These results should be useful to guide researchers in the development of CDRs for EM specialists.
- 15Mail and internet surveys: the tailored design method. New York, NY: Wiley, 2000..
- 16The International diffusion of the Ottawa Ankle and Knee Rules [abstract]. Acad Emerg Med. 1999; 6:434., , , et al.
- 18Clinicians required very high sensitivity of a bacteremia prediction rule. J Clin Epidemiol. 2001; 75:1104–6., , .
- 20The acceptability of clinical decision rules: validation of the Ottawa Acceptability of Decision Rules Scales (OADRS) [abstract]. Acad Emerg Med. 2007; 14(Supp 1):S49., , , .
- 21Potential areas for new clinical decision rules: comparison of North America and Europe [abstract]. Acad Emerg Med. 1999; 6:433., , , et al.
- 22Suicides: UK Numbers Reach 30 Year Low in 2003. London: Office of National Statistics, 2005.
- 24Excluding pulmonary embolism at the bedside without diagnostic imaging: management of patients with suspected pulmonary embolism presenting to the emergency department by using a simple clinical model and D-dimer. Ann Intern Med. 2001; 135:98–107., , , et al.
|ACEM_035_sm_data supplement.pdf||117K||Supporting info item|
Please note: Wiley Blackwell is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.