SEARCH

SEARCH BY CITATION

Keywords:

  • computerized tomography angiography;
  • D-dimer;
  • decision rule;
  • decision-making;
  • diagnosis;
  • medical malpractice;
  • pulmonary embolism;
  • venous thromboembolism

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Addendum
  8. Author contributions
  9. Acknowledgements
  10. Disclosure of Conflict of Interests
  11. References
  12. Appendix

Summary. Backgound: Over-investigation of low-risk patients with suspected pulmonary embolism (PE) represents a growing problem. The combination of gestalt estimate of low suspicion for PE, together with the PE rule-out criteria [PERC(−): age < 50 years, pulse < 100 beats min−1, SaO2 ≥ 95%, no hemoptysis, no estrogen use, no surgery/trauma requiring hospitalization within 4 weeks, no prior venous thromboembolism (VTE), and no unilateral leg swelling], may reduce speculative testing for PE. We hypothesized that low suspicion and PERC(−) would predict a post-test probability of VTE(+) or death below 2.0%. Methods: We enrolled outpatients with suspected PE in 13 emergency departments. Clinicians completed a 72-field, web-based data form at the time of test order. Low suspicion required a gestalt pretest probability estimate of <15%. The main outcome was the composite of image-proven VTE(+) or death from any cause within 45 days. Results: We enrolled 8138 patients, 85% of whom had a chief complaint of either dyspnea or chest pain. Clinicians reported a low suspicion for PE, together with PERC(−), in 1666 patients (20%). At initial testing and within 45 days, 561 patients (6.9%, 95% confidence interval 6.5–7.6) were VTE(+), and 56 others died. Among the low suspicion and PERC(−) patients, 15 were VTE(+) and one other patient died, yielding a false-negative rate of 16/1666 (1.0%, 0.6–1.6%). As a diagnostic test, low suspicion and PERC(−) had a sensitivity of 97.4% (95.8–98.5%) and a specificity of 21.9% (21.0–22.9%). Conclusions: The combination of gestalt estimate of low suspicion for PE and PERC(−) reduces the probability of VTE to below 2% in about 20% of outpatients with suspected PE.


Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Addendum
  8. Author contributions
  9. Acknowledgements
  10. Disclosure of Conflict of Interests
  11. References
  12. Appendix

Studies that employed autopsy as a criterion standard have found that pulmonary embolism (PE) follows acute coronary syndrome as the second most common cause of sudden unexpected death in outpatients [1–3]. PE manifests in a broad clinical spectrum of severity, ranging from vague chest discomfort with normal vital signs to sudden death [4–6]. At least one-quarter of outpatients diagnosed with PE have no overt risk factors [7,8]. Experts often suggest that physicians continue to miss the diagnosis of PE at an unacceptably high rate, and patients with a delayed diagnosis of PE have worsened outcomes [9,10]. Coincidentally, several reports have indicated that physicians hold an increasing perception of risk of medical malpractice secondary to failure to order a diagnostic test for PE [11,12]. These influences, together with the widened availability and acceptance of the D-dimer and computed tomography (CT) angiography as diagnostic tests for PE, may have combined to cause an increase in the frequency of testing for PE among very low risk outpatients [13]. Speculative investigation for PE that leads to large numbers of negative CT angiograms in young patients may have negative consequences in the form of increased risk of malignancy secondary to radiation exposure [14,15].

In 2004, we derived the pulmonary embolism rule-out criteria (PERC), an eight-factor decision rule to support the decision not to order a diagnostic test for PE in patients for whom the clinician already had a low clinical suspicion for PE based on a gestalt impression [16]. The PERC criteria negative [PERC(−)] requires the clinician to answer ‘no’ to the following eight questions:

  • 1
    Is the patient older than 49 years of age?
  • 2
    Is the pulse rate above 99 beats min−1?
  • 3
    Is the pulse oximetry reading <95% while the patient breathes room air?
  • 4
    Is there a present history of hemoptysis?
  • 5
    Is the patient taking exogenous estrogen?
  • 6
    Does the patient have a prior diagnosis of venous thromboembolism (VTE)?
  • 7
    Has the patient had recent surgery or trauma? (Requiring endotracheal intubation or hospitalization in the previous 4 weeks.)
  • 8
    Does the patient have unilateral leg swelling? (Visual observation of asymmetry of the calves.)

We use the term very low risk to describe patients with the combination of both gestalt low suspicion and PERC(−). Each patient in the very low risk group should have a pretest probability of less than the test threshold for PE. The test threshold represents a numeric estimate of the pretest probability, which defines a point of equipoise: patients with a pretest probability lower than the test threshold should not benefit (and may even be harmed) from diagnostic testing. Using the method of Pauker and Kassirer, we have previously estimated the point of equipoise to be 2% (rounded to the nearest whole per cent) [16,17]. The ultimate goal is to produce an actual outcome frequency (i.e. a posterior probability) of VTE that is below 1%, as has been suggested to be the goal of a PE rule-out protocol [18]. We hypothesized that patients undergoing evaluation for possible PE, but who satisfied our definition of very low risk, would have a measured 45-day incidence of VTE(+) or death with an upper-limit 95% confidence interval (95% CI) below 2.0%.

Methods

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Addendum
  8. Author contributions
  9. Acknowledgements
  10. Disclosure of Conflict of Interests
  11. References
  12. Appendix

This was a prospective, non-interventional, multicenter study of patients presenting to the emergency department (ED) in 12 hospitals in the USA and one in Christchurch, New Zealand. Prior to the study initiation, a project manager traveled to each site (except New Zealand) in order to train site investigators and research associates on protocol methodology. This individual was then available full-time by telephone and e-mail, and, if needed, for face-to-face visits throughout the study duration as a resource to investigators. The study protocol was approved by the Institutional Review Boards for the conduct of research on humans and the Privacy Boards of all 13 hospitals. Patients were enrolled from July 1, 2003 until November 30, 2006. The event that triggered eligibility for enrollment was an order for an objective diagnostic test for PE, written by or under the supervision of a board-certified emergency physician. The decision to order this test was based upon information obtained from the initial history and physical examination, and medical records immediately available in the ED. We defined objective diagnostic testing for PE as either a pulmonary vascular imaging study [CT angiography or scintillation ventilation–perfusion (VQ) lung scanning] or a D-dimer assay ordered to evaluate for possible PE. Tests performed for suspected DVT only did not trigger enrollment. Pre-enrollment exclusions were as follows. (i) Clinician’s knowledge of a diagnostic positive pulmonary vascular imaging study performed within the previous 7 days. (ii) The patient indicated that the enrollment hospital was not his or her hospital system of choice for follow-up. (iii) Any circumstance that suggested that the patient would be lost to follow-up (e.g. homeless patients, patients with severe psychiatric disorders, patients who could not provide a reliable telephone number, international travelers, inmates in State or Federal penitentiaries, persons arrested for felonies). These exclusions were determined in the ED by qualified study personnel.

Sites had the option of enrolling patients in one of two ways: either by using randomly selected 8-h time blocks or by using all patients consecutively with a defined target rate of >85%. Adherence to the 85% threshold was estimated by a retrospective review of computerized order-entry databases (see Appendix), which was used to locate and total the number of D-dimer studies ordered for adults with one or more of the symptoms shown in Table 1 (or obvious documentation of PE suspicion), all VQ scans and all CT pulmonary angiography studies ordered and carried out during the active hours of enrollment. These reviews were performed on a quarterly basis. A patient could be enrolled more than once if he or she had a test ordered for PE on separate days of presentation. Enrollment was completed after 500 patients or 18 months of data collection. If a site could not achieve >85% capture with consecutive enrollment, this triggered a medical record review of patients who were eligible for enrollment but who were missed (i.e. the intent-to-study group). The purpose of this intent-to-study group was to allow statistical comparison of the key demographics and frequency of VTE between enrolled vs. missed patients [19]. If the number of missed patients exceeded the number enrolled, then data were abstracted from a random sample of missed patients equal to the number enrolled. Data abstraction was performed using published methods [20].

Table 1.   Clinical features of 8138 emergency department patients who were tested for pulmonary embolism (PE)
 Mean or n±SD or %
  1. *Recorded as a race. Non-substernal and worse with breathing or cough; generalized immobility >72 h, therapeutic limb fixation causing immobility of >1 major joint, or continuous travel in seated position >6 h; §surgery or trauma within 4 weeks and requiring endotracheal intubation; patient report or medical record, including genetic variations, hemoglobinopathies, and antiphospholipid antibody syndromes. **Highest and lowest values prior to ordering the first test for PE. DVT, deep vein thrombosis.

Demographic data
 Age (years)49.1 ± 17.8
 Black race266932
 Caucasian race481159
 Latino or Hispanic descent*4616
 Asian race661
 Other race1312
 Female gender542867
Symptoms
 Pleuritic chest pain359844
 Substernal chest pain278534
 Dyspnea412951
 Syncope4786
 Cough238129
 Hemoptysis2433
Comorbid conditions
 Current smoker285035
 Active malignancy121615
 Immobility206725
 Recent surgery§5407
 Pregnant or postpartum <4 weeks80810
 Prior PE or DVT86011
 Congestive heart failure80410
 Chronic obstructive pulmonary disease4746
 Known coronary artery disease104113
 Connective tissue disease5297
 Exogenous estrogen87011
 Taking warfarin5437
 Hematological thrombophilia4125
Physical findings
 Highest pulse rate (beats min−1)**92 ± 21
 Highest respiratory rate (breaths min−1)**21 ± 9
 Lowest systolic blood pressure (mmHg)**131 ± 28
 Lowest room air pulse oximetry (%)**96 ± 4
 Temperature (°C)37 ± 1
 Body mass index (kg m−2)29 ± 8
 Wheezing78110
 Unilateral leg swelling7079

A key technical objective was to collect clinical data (including the gestalt estimate of clinical suspicion) that accurately represented the clinicians’ beliefs at the time when he or she decided to order a diagnostic test for PE, but prior to knowing the results. In order to achieve this objective, clinicians collected and recorded study data coincident with clinical care, as illustrated in Figure 1. Data were entered into a web-based, secure, electronic data collection form. This protocol specified that the form be completed before the results of testing were known, and the form required the clinician to input if he or she had knowledge of any test results pertinent to diagnosis of VTE at the time it was populated. A full description of the content and methodology for this data collection system has been published [21]. Physicians clicked one of three radio buttons to encode gestalt pretest probability estimates for PE as <15%, 15–40% or > 40%. The form then asked for the data required for the PERC rule as well as 65 other data points. All fields had explicit definitions, available in a pop-up box at the click of a mouse. This form employed automated checks in order to prevent the upload of missing or nonsensical data. Classification as very low risk required that the clinician encode a gestalt pretest probability of <15%, as well as all eight factors required for PERC(−), into the form. The protocol permitted the use of a personal digital assistant loaded with the data form, or a paper template to facilitate data transfer to the web-based form. Once the data form was uploaded, the patient was enrolled, and no patients were excluded. To help to ensure comprehensive follow-up, we recorded if the patient stated that the enrollment hospital was his or her preferred hospital. Figure 2 shows the suggested diagnostic algorithm to allow exclusion of PE in the ED that was provided to all clinicians who participated. Imaging protocols were overseen by the radiology department at each institution and all image results were from the final interpretations signed by a board-certified radiologist.

image

Figure 1.  Overview of research-related activities, relative to the main steps in clinical care. ED, emergency department, PE, pulmonary embolism.

Download figure to PowerPoint

image

Figure 2.  Overview of the diagnostic algorithm used by all 13 study hospitals to exclude the diagnosis of pulmonary embolism (PE) in the emergency department. All patients had either D-dimer, computed tomography (CT) angiography or ventilation–perfusion (VQ) lung scan performed. Solid lines depict primary pathways and dotted lines represent secondary pathways. *Yale University used a qualitative D-dimer.

Download figure to PowerPoint

We followed all enrolled patients for outcome at 45 days. We have found that a longer time interval reduces the probability of successful contact [20]. No patients were excluded on the basis of follow-up. The protocol required one of the following: documented follow-up history and a physical examination that contained sufficient information to allow the site investigator to determine the VTE status at 45 days, or telephone or mail contact with the patient, or a family member, or the patient’s clinician. If we were unable to obtain a medical record, and could not contact the patient, we searched the social security death index to determine vital status (http://ssdi.rootsweb.com). We obtained a death certificate for all decedents. We documented the results of tests, clinical diagnoses, treatments, death-related data, and findings from telephone interviews on a more extensive, web-based follow-up form that encoded 143 data fields [21]. Consistent with contemporaneous research and expert opinion [22–25], we used either PE or deep venous thrombosis (DVT) as evidence of VTE(+) in the reference standard. The VTE status was established by an adjudicated review of imaging results, medical records, and follow-up. The assignment of present or absent VTE within 45 days required agreement between two independent clinicians who used explicit criteria. Positive evidence of PE required either a high-probability VQ scan or a CT angiogram or conventional pulmonary angiogram that demonstrated a pulmonary arterial filling defect interpreted as positive for acute PE, or an autopsy positive for PE. Positive evidence of DVT required extremity venous duplex Doppler-ultrasonography or CT venography interpreted as positive for acute venous thrombosis in the popliteal, femoral or axillary (but not calf) veins. Diagnosis of VTE also required written documentation of a clinical plan to treat with anticoagulation using either a vitamin K antagonist, or parenteral low-molecular-weight heparin for ≥3 months, or insertion of a vena cava filtering device in a patient with a contraindication to anticoagulation. Exclusion of VTE required the absence of positive imaging for VTE and no treatment for VTE within 45 days. For the intent-to-study group (eligible patients not enrolled during the time of active enrollment and not included in the main analysis), the diagnosis of VTE(+) required one or more VTE-related ICD-9 (or -10) codes (415.X, 451.X and 453.X) to be documented within 45 days of having a diagnostic test ordered to rule out PE in the ED while the study was actively enrolling.

Statistical analysis

The hypothesis stated that the upper limit of the 95% CI for the proportion of very low risk patients who met the primary endpoints of VTE(+) or death from any cause within 45 days would not exceed 2.0%. For the sample size calculation, we estimated that approximately 20% of the cohort patients would fall into the very low risk category and that 1% of these patients would actually have the primary endpoint of VTE diagnosed within 45 days. Using the method described by Arkin, we computed that a sample size of 7000 patients would allow us reject the null hypothesis at α = 0.05 (i.e. <5% probability that the true incidence of VTE was >2.0% in very low risk patients) and to demonstrate an 80% power to narrow one side of the 95% CI to <1.0% [26]. We used the Clopper–Pearson exact method to compute confidence intervals for proportions (StatsDirect version 2.4.4, Cheshire, UK).

Results

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Addendum
  8. Author contributions
  9. Acknowledgements
  10. Disclosure of Conflict of Interests
  11. References
  12. Appendix

The study enrolled 8138 patients from 12 213 eligible patients. Figure 3 shows the outcomes of all patients. Table 1 describes the demographic data, age and clinical characteristics of the cohort. The mean age was 49 years, and women comprised approximately two-thirds of the cohort. The first symptom that prompted the ED visit was chest pain (53%), dyspnea (33%), cough (3%), syncope (2%), respiratory distress (2%), seizure (0.1%), and other (7%). Fifty-five per cent of patients indicated that their first symptom onset was ‘sudden’ and 45% said the onset was ‘gradual’. Seventy-six per cent of the cohort had one or more of the comorbid conditions listed in Table 1. Forty-two per cent had either a respiratory rate >22 breaths min−1 or a pulse rate >100 beats min−1 at the time of enrollment.

image

Figure 3.  Flow diagram showing outcomes of all patients [*telemetry, n=2158; unmonitored ward, n=958; emergency department (ED) observation, n=1015; intensive care unit, n=215]. **All patients indicated that the study hospital was his or her hospital of choice.

Download figure to PowerPoint

Table 2 summarizes the associated beliefs and actions of clinicians that are pertinent to assessing their clinical suspicion for PE. Clinicians documented a low clinical suspicion (i.e. gestalt impression <15%) in two-thirds of all patients, and 80% of clinicians recorded that they believed an alternative diagnosis was more likely than PE at the time they ordered a diagnostic test for PE.

Table 2.   Perceptions of clinicians pertinent to pretest probability of pulmonary embolism (PE)
Clinicians’ unstructured estimates of pretest probability of PEn%
  1. *This field indicated explicit plans to perform lower-extremity ultrasound if the results of pulmonary vascular imaging were negative.

 Less than 15%542567
 15–40%218227
 Greater than 40%5317
Other data documented at the time of testing
 Plan to administer heparin before imaging completed3514
 Alternative diagnosis more likely than PE651480
 Deep vein thrombosis imaging preplanned*86711

Table 3 shows the results of diagnostic testing for PE. Seventy-four per cent of patients had a D-dimer ordered; 55% had CT angiography, VQ scanning, or both performed. In 5999 (74%) patients, the results of PE-related diagnostic tests were completely normal.

Table 3.   Frequency of ordering and results of diagnostic testing for pulmonary embolism (PE)
Diagnostic testn%
  1. *Four of 13 sites used computed tomography (CT) venography as part of the CT angiography protocol.

  2. Includes 12 who had follow-up venous ultrasonographies that were negative and who were not treated. DVT, deep vein thrombosis.

D-dimer601974
Positive2797
Negative3222
CT angiography412751
PE365
PE and DVT*73
DVT only24
Pulmonary infiltrate, no PE230
Other non-PE finding1506
Normal1843
Indeterminate or inadequate imaging86
Scintillation lung scanning4866
High probability40
Intermediate probability47
Low probability273
Normal126
Venous ultrasound96212
Negative for DVT808
Calf or saphenous thrombosis86
Femoral or axillary vein thrombosis68

Table 4 shows the outcomes that occurred within 45 days of enrollment. Of 8138 patients tested for possible PE in the ED, a total of 513 were diagnosed with VTE during the index visit. Follow-up found that 47 patients (0.5% of the cohort) were diagnosed with VTE after discharge from the index visit but were not diagnosed at the time of the index ED visit or hospitalization. One patient died of undiagnosed PE discovered at autopsy. Thus, 561 patients (6.9%) met a criterion standard for the diagnosis of VTE(+) within 45 days. The median 45 day incidence of VTE(+) among the 13 sites was 6.9% (interquartile range 5.8–8.6%). The measured 45-day incidences of VTE(+) stratified by the gestalt pretest probability estimates of <15%, 15–40% and >40% were 3.0%, 10.4% and 31.1%, respectively.

Table 4.   Outcomes at 45 days for the cohort of 8138 emergency department patients
Outcome*n%95% confidence interval
  1. *Pulmonary embolism (PE) and deep vein thrombosis (DVT) required imaging evidence and a clinical plan to treat within 45 days of enrollment. VTE, venous thromboembolism.

Any VTE5616.96.3–7.5%
PE only3714.64.1–5.0%
DVT only801.00.8–1.2%
PE and DVT1091.31.1–1.6%
Death from PE10.00.0–0.1%
Death, no known VTE560.70.5–0.9%

Four sites (Northwestern Memorial, Pitt County Memorial, St. Vincent Mercy Medical and William Beaumont) enrolled consecutively but did not meet the prespecified 85% goal, and were thus considered as convenience samples. We therefore performed a medical record review of a random sample of missed patients, equal to the number enrolled at each of these four sites (n = 2040), and compared the demographics of these patients with those of the 8138 patients that we enrolled. The mean age of these 2040 patients was 52 ± 18.7 years, 64% were female, 55% were Caucasian, and 130 (6.4%) were diagnosed with VTE(+), including 3.9% with PE only, 1.1% with DVT only, and 1.3% with both PE and DVT. The 95% CI for the 0.5% difference between this 6.4% vs. the 6.9% in the study cohort was −1.7 to 0.7%, suggesting no significant difference in the VTE(+) rate between the study cohort and the intent-to-study group.

Table 5 shows the outcomes of patients based upon the results of the PERC rule. Clinicians recorded all eight variables of the PERC rule as negative in 1952 (24%) patients. The false-negative rate in this subgroup was 1.2% (95% CI 0.8–1.8%) for VTE(+) and the false-negative rate was 1.3% (95% CI 0.8–1.9%) for the combined endpoints of either VTE(+) or death from any cause within 45 days. If we were to treat PERC(−) as an independent diagnostic test, its sensitivity would have been 95.7% (95% CI 93.6–97.2%), the specificity would equal 25.4% (95% CI 24.4–26.4%), and the negative likelihood ratio would have been 0.17 (95% CI 0.11–0.25%).

Table 5.   Outcomes of low-risk patients
Outcome PERC(−)* (n = 1952)Very low risk (n = 1666)
n%95% CIn%95% CI
  1. 95% CI, 95% confidence interval; PE, pulmonary embolism; PERC, pulmonary embolism rule-out criteria; very low risk, no test needed; VTE, venous thromboembolism.

  2. *Age < 50 years, pulse < 100 beats min−1, SaO2 ≥ 95%, no hemoptysis, estrogen use, surgery/trauma requiring hospitalization within 4 weeks, or prior VTE, and no unilateral leg swelling.

PE191.00.6–1.5%150.90.5–1.5%
Any VTE241.20.8–1.8%150.90.5–1.5%
VTE or death251.30.8–1.9%161.00.6–1.6%

Sixteen hundred and sixty-six patients (20% of the cohort) were classified as very low risk. On a per-site basis, the proportion of patients in this very low risk group ranged from 10% to 36%, with a median value of 19% for all 13 sites. The false-negative rate in this very low risk subgroup was 0.9% (95% CI 0.5–1.5%) for VTE(+) and 1.0% (0.6–1.6%) for VTE(+) or death from any cause within 45 days. If it were treated as a diagnostic test, the very low risk classification would have yielded a diagnostic sensitivity equal to 97.4% (95% CI 95.8–98.5%), a specificity equal to 21.9% (95% CI 21.0–22.9%), and a negative likelihood ratio equal to 0.12 (95% CI 0.07–0.19%).

Additionally, 1745 patients were classified by clinicians as having both an alternative diagnosis more likely than PE, and PERC(−), and 16 of these were VTE(+) within 45 days, yielding a posterior probability of 0.9% (95% CI: 0.5–1.5%).

Discussion

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Addendum
  8. Author contributions
  9. Acknowledgements
  10. Disclosure of Conflict of Interests
  11. References
  12. Appendix

In this large, multicenter study, we found that clinicians employ a very low threshold to test for PE, resulting in a very high rate of negative testing and a low rate of PE diagnosis. We used rigorous methodology to ensure a patient sample and a dataset that would allow broad inferences about the perceptions of clinicians when they tested for PE. To our knowledge, this is the largest prospective study of patients tested for PE. We enrolled patients from urban and rural settings, from teaching and community hospitals, and at high altitudes. We recognized the potential roles of convenience sampling and the informed consent process as sources of bias, and we attempted to control for these by our sampling methods and by cross-checking patients who were eligible but were missed for enrollment [27]. The study population was diverse in racial and ethnic composition and the patients were drawn from a wide geographic range.

We found that in two-thirds of cases, emergency clinicians harbored a low suspicion for PE when they ordered a diagnostic test to evaluate for PE. In 80% of cases, clinicians ordered a diagnostic test for PE with the belief that an alternative diagnosis was more likely. Eighteen hundred and forty-three patients underwent a CT angiography that yielded images interpreted as completely normal by a board-certified radiologist. While we recorded the category of ‘other’ interpretations in 36% of CT angiograms, the majority of these were of questionable significance (e.g. enlarged lymph nodes or small pleural effusion). Taken together, the combination of low physician pretest suspicion and the preponderance of negative imaging results infer the opportunity for the PERC rule to save resources and time, and to reduce patient exposure to the potential negative impact of ionizing radiation and iodinated contrast [14,15,28,29]. Of relevance, the very low risk subgroup included 362 patients who had a completely normal CT angiography.

The clinician’s clinical suspicion for PE was low and the PERC rule was negative in 1666 patients, or 20% of the cohort. In this very low risk subgroup, 1.0% were VTE(+) or died within 45 days, and the top limit 95% CI of this proportion was 1.6%. Similarly, the clinician indicated that an alternative diagnosis was more likely than PE and the PERC rule was negative in 1745 patients, and only 16/1745 (0.9%, 95% CI 0.5–1.5%) had VTE within 45 days. Thus, the very low risk categorization produced a false-negative rate similar to that observed after a negative quantitative D-dimer, a normal scintillation lung scan, a negative multidetector CT chest angiography, or formal pulmonary angiography negative for PE [30,31]. Only one very low risk patient died (of end-stage cancer) within 45 days.

Three published studies have performed secondary analyses of existing databases to test the performance of the PERC rule, independently of gestalt reasoning [32–34]. In aggregate, these studies included 1542 patients, of whom 362 (24%) were PERC(−) and nine (2.4%) were VTE(+). The present study found the PERC rule had a negative likelihood ratio equal to 0.17 within the entire cohort, suggesting that the PERC(−) rule would afford a false-negative rate below 1.0% only if the initial prevalence of VTE was less than 6% in the population under consideration. These data underscore the need for gestalt reasoning to precede the PERC rule. We submit that in real practice, clinicians will use a validated clinical decision rule to rule out PE only when they believe that the clinical picture portrays a low-risk profile. Moreover, no sensible decision rule could exclude all possible high-risk PE cases. Runyon et al. previously found that unstructured gestalt categorization of pretest probability into three categories (low, moderate and high) yielded results that exactly mirrored those of a validated, structured scoring system [35]. Moreover, two studies independently found the interobserver reliability for gestalt reasoning to be very good, based upon a Cohen’s kappa value above 0.60 [35,36]. Regardless of its level of validation, we submit that most clinicians currently employ gestalt reasoning as their primary method of formulating pretest probability [37].

Limitations of our study could include the requirement for gestalt interpretation of low risk. Moreover, this was not a management study; clinicians did not use the PERC rule instead of diagnostic testing in any patient, therefore the true safety of the PERC rule cannot be fully inferred. Some may hesitate to use clinical criteria that are not 100% sensitive in order to exclude a potentially fatal disease. It could be speculated that patients in our cohort differed from ED patients in other countries. We found that only 5.9% of our cohort had PE within 45 days, and 6.9% had either PE or DVT within 45 days. The classification of VTE status by adjudicators also was dependent upon the judgment of their physicians. It should be emphasized that the underlying prevalence of VTE may be too high in other ED populations for this strategy to be able to safely rule out PE [23,38,39]. Thus, we would suggest that prior to implementation at any hospital, the tandem strategy of using gestalt <15% and PERC(−) should be pilot-tested in an observational quality assurance study, in order to ensure an acceptably low posterior probability (e.g. <1%) [18].

In summary, we followed a cohort of 8138 outpatients who were tested for PE in the ED, and found that 6.9% were VTE(+) within 45 days. One-fifth of the cohort could have been classified as very low risk, defined as a gestalt pretest probability <15% and PERC(−). In this very low risk subgroup, 1.0% (95% CI 0.6 –1.6%) had VTE within 45 days. The combination of a clinician’s gestalt estimate of a pretest probability of less than 15% and PERC(−) selects a subgroup of patients with a very low probability of VTE.

Addendum

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Addendum
  8. Author contributions
  9. Acknowledgements
  10. Disclosure of Conflict of Interests
  11. References
  12. Appendix

Other Authors

T. R. McCubbin, Exempla St. Joseph’s Hospital, Denver, CO (e-mail: trmccubbin@aol.com). D. M. Beam, Pitt County Memorial Hospital, Greenville, NC (e-mail: Dmb1107@ecu.edu). M. P. Than, Christchurch Hospital, Christchurch, New Zealand (e-mail: Martin.thancdhb.govt.nz). K. E. Nordenholz, University of Colorado School of Health Sciences, Denver, CO (e-mail: Kristen.nordenholz@uchsc.edu). A. M. Mitchell, Carolinas Medical Center, Charlotte, NC (e-mail: Alice.mitchell@carolinas.org). C. L. Johnson, Studymaker LLC, Newton, MA (e-mail: Charlie@studymaker.com). C. A. Camargo Jr, Massachusetts General Hospital, Boston, MA (e-mail:ccamargo@partners.org).

Author contributions

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Addendum
  8. Author contributions
  9. Acknowledgements
  10. Disclosure of Conflict of Interests
  11. References
  12. Appendix

J. A. Kline conceived and organized the study, obtained funding, collected and analyzed data and drafted the manuscript. D. M. Courtney, C. Kabrhel, C. L. Moore, H. A. Smithline, T. R. McCubbin, M. C. Plewa, P. B. Richman and B. J. O’Neil participated in writing the study protocol, obtaining funding, data collection and analysis, and drafting and revising the manuscript. D. M. Beam, M. P. Than, A. M. Mitchell and K. E. Nordenholz collected and analyzed data and helped to write the manuscript. C. A. Camargo Jr helped to write the protocol, analyze and interpret data, and draft and revise the manuscript. C. L. Johnson contributed to the overall study design based upon the design and operation of the web-based data collection system, and he helped to collect and analyze data and draft the manuscript. Presented at the Society for Academic Emergency Medicine Meeting, May, 2007, Chicago, IL. William B. Webb was the research project manager.

Acknowledgements

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Addendum
  8. Author contributions
  9. Acknowledgements
  10. Disclosure of Conflict of Interests
  11. References
  12. Appendix

Supported by Grants from the National Institutes of Health R41HL074415 and R42HL074415, K23HL077404 and R01 HL074384, and a Medical Student Award from the Emergency Medicine Foundation.

Disclosure of Conflict of Interests

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Addendum
  8. Author contributions
  9. Acknowledgements
  10. Disclosure of Conflict of Interests
  11. References
  12. Appendix

J. A. Kline owns stock in CP Diagnostics LLC and Studymaker LLC. C. L. Johnson owns stock in CP Diagnostics LLC and Studymaker LLC. All authors have access to all data using the rules of the data use agreement.

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Addendum
  8. Author contributions
  9. Acknowledgements
  10. Disclosure of Conflict of Interests
  11. References
  12. Appendix
  • 1
    Courtney DM, Sasser H, Pincus B, Kline JA. Pulseless electrical activity with witnessed arrest as a predictor of sudden death from massive pulmonary embolism in outpatients. Resuscitation 2001; 49: 26572.
  • 2
    Kurkciyan I, Meron G, Sterz F, Janata K, Domanovits H, Holzer M, Berzlanovich A, Bankl HC, Laggner AN. Pulmonary embolism as a cause of cardiac arrest. Arch Intern Med 2000; 160: 152935.
  • 3
    Manfredini R, Portaluppi F, Grandi E, Fersinsi C, Gallerani M. Out of hospital sudden death referring to an emergency department. J Clin Epidemiol 1996; 49: 8658.
  • 4
    Stein PD, Henry JW. Clinical characteristics of patients with acute pulmonary embolism stratified according to their presenting syndromes. Chest 1997; 112: 9749.
  • 5
    Susec O, Boudrow D, Kline J. The clinical features of acute pulmonary embolism in ambulatory patients. Acad Emerg Med 1997; 4: 8917.
  • 6
    Ryu JH, Olson EJ, Pellikka PA. Clinical recognition of pulmonary embolism: problem of unrecognized and asymptomatic cases. Mayo Clin Proc 1998; 73: 8739.
  • 7
    Heit JA, O’Fallon WM, Petterson TM, Lohse CM, Silverstein MD, Mohr DN, Melton LJ III. Relative impact of risk factors for deep vein thrombosis and pulmonary embolism: a population-based study. Arch Intern Med 2002; 162: 12458.
  • 8
    Cushman M, Tsai AW, White RH, Heckbert SR, Rosamond WD, Enright P, Folsom AR. Deep vein thrombosis and pulmonary embolism in two cohorts: the longitudinal investigation of thromboembolism etiology. Am J Med 2004; 117: 1925.
  • 9
    Ota M, Nakamura M, Yamada N, Yazu T, Ishikura K, Hiraoka N, Tanaka H, Fujioka H, Isaka N, Nakano T. Prognostic significance of early diagnosis in acute pulmonary thromboembolism with circulatory failure. Heart Vessels 2002; 17: 711.
  • 10
    Kline JA, Hernandez J, Jones AE, Rose GA, Norton HJ, Camargo CA Jr. Prospective study of the clinical features and outcomes of emergency department patients with delayed diagnosis of pulmonary embolism. Acad Emerg Med 2007; 14: 5928.
  • 11
    Studdert DM, Mello MM, Sage WM, DesRoches CM, Peugh J, Zapert K, Brennan TA. Defensive medicine among high-risk specialist physicians in a volatile malpractice environment. JAMA 2005; 293: 260917.
  • 12
    Taylor H. Doctors and other health professionals report that fear of malpractice has a big and mostly negative effect on clinical practice, uneccessary defensive medicine and openness to discussing medical errors. Taylor H, Leitman R, editors. Harris Interactive, Health Care News. 2 [3], 15. 2003.
  • 13
    Kabrhel C, Matts C, McNamara M, Katz J, Ptak T. A highly sensitive ELISA D-dimer increases testing but not diagnosis of pulmonary embolism. Acad Emerg Med 2006; 13: 51924.
  • 14
    Einstein AJ, Henzlova MJ, Rajagopalan S. Estimating risk of cancer associated with radiation exposure from 64-slice computed tomography coronary angiography. JAMA 2007; 298: 31723.
  • 15
    Brenner DJ, Hall EJ. Computed tomography--an increasing source of radiation exposure. N Engl J Med 2007; 357: 227784.
  • 16
    Kline JA, Mitchell AM, Kabrhel C., Richman PB, Courtney DM. Clinical criteria to prevent unnecessary diagnostic testing in emergency department patients with suspected pulmonary embolism. J Thromb Haemost 2004; 2: 124755.
  • 17
    Pauker SG, Kassirer JP. The threshold approach to clinical decision making. N Engl J Med 1980; 302: 110917.
  • 18
    Kline JA, Wells PS. Methodology for a rapid protocol to rule out pulmonary embolism in the emergency department. Ann Emerg Med 2003; 42: 26675.
  • 19
    Kline JA, Israel EG, Michelson EA, O’Neil BJ, Plewa MC, Portelli DC. Diagnostic accuracy of a bedside D-dimer assay and alveolar deadspace measurement as for rapid exclusion of pulmonary embolism. A multicenter study. JAMA 2001; 285: 7618.
  • 20
    Kline JA, Mitchell AM, Runyon MS, Jones AE, Webb WB. Electronic medical record review as a surrogate to telephone follow-up to establish outcome for diagnostic research studies in the emergency department. Acad Emerg Med 2005; 12: 1127232.
  • 21
    Kline JA, Johnson CL, Webb WB, Runyon MS. Prospective study of clinician-entered research data in the emergency department using an internet-based system after the HIPAA Privacy Rule. BMC Med Inform Decis Mak 2004 2005; 4: 1725.
  • 22
    Wolde MT, Hagen PJ, MacGillavry MR, Pollen IJ, Mairuhu ATA, Koopman MMW, Prins MH, Hoekstra OS, Brandjes DPM, Postmus PE, Buller HR. Non-invasive diagnostic work-up of patients with clinically suspected pulmonary embolism; results of a management study. J Thromb Haemost 2004; 2: 11107.
  • 23
    Perrier A, Roy PM, Aujesky D, Chagnon I, Howarth N, Gourdier AL, Leftheriotis G, Barghouth G, Cornuz J, Hayoz D, Bounameaux H, Perrier A, Roy PM, Aujesky D, Chagnon I, Howarth N, Gourdier AL, Leftheriotis G, Barghouth G, Cornuz J. Diagnosing pulmonary embolism in outpatients with clinical assessment, D-dimer measurement, venous ultrasound, and helical computed tomography: a multicenter management study. Am J Med 2004; 116: 2919.
  • 24
    Wolfe TR, Hartsell SC. Pulmonary embolism: making sense of the diagnostic algorithm. Ann Emerg Med 2001; 37: 50414.
  • 25
    Moores LK. Diagnosis and management of pulmonary embolism: are we moving toward an outcome standard? Arch Intern Med 2006; 166: 1478.
  • 26
    Arkin CF, Wachtel MS. How many patients are necessary to assess test performance? JAMA 1990; 263: 27580.
  • 27
    Mitchell AM, Kline JA. Systematic bias introduced by the informed consent process in pulmonary embolism prevalence. Acad Emerg Med 2008; 15: 22530.
  • 28
    Coche E, Vynckier S, Octave-Prignot M, Coche E, Vynckier S, Octave-Prignot M. Pulmonary embolism: radiation dose with multi-detector row CT and digital angiography for diagnosis. Radiology 2006; 240: 6907.
  • 29
    Mitchell AM, Kline J.A. Contrast nephropathy following computed tomography angiogrpahy of the chest for pulmonary embolism in the emergency department. J Thromb Haemost 2007; 5: 504.
  • 30
    PIOPED Investigators. Value of the ventilation/perfusion scan in acute pulmonary embolism. JAMA 1990; 263: 27539.
  • 31
    Stein PD, Fowler SE, Goodman LR, Gottschalk A, Hales CA, Hull RD, Leeper KV Jr, Popovich J Jr, Quinn DA, Sos TA, Sostman HD, Tapson VF, Wakefield TW, Weg JG, Woodard PK. Multidetector computed tomography for acute pulmonary embolism. N Engl J Med 2006; 354: 231727.
  • 32
    Righini M, Le Gal G, Perrier A, Bounameaux H. External evaluation of the performance of the pulmonary embolism research consortium rule (PERC rule). J Thromb Haemost 2004; 3: 1889.
  • 33
    Hogg KE, Dawson D, Kline JA. Application of pulmonary embolism rule-out criteria to the UK Manchester Investigation of Pulmonary Embolism Diagnosis (MIOPED) study cohort. J Thromb Haemost 2005; 3: 5923.
  • 34
    Wolf SJ, McCubbin TR, Naviaux NW, Nordenholz KE, Haukoos JS. External validation of the pulmonary embolism rule-out criteria rule. Acad Emerg Med 2005; 12: 40.
  • 35
    Runyon MS, Webb WB, Jones AE, Kline JA. Comparison of the unstructured clinician estimate of low clinical probability for pulmonary embolism to the Canadian score or the Charlotte rule. Acad Emerg Med 2005; 12: 58793.
  • 36
    Nordenholz KE, Naviaux NW, Stegelmeier K, Haukoos JS, Wolf SJ, McCubbin T, Heard K. Pulmonary embolism risk assessment screening tools: the interrater reliability of their criteria. Am J Emerg Med 2007; 25: 28590.
  • 37
    Runyon MS, Richman PB, Kline JA. Emergency medicine practitioner knowledge and use of decision rules for the evaluation of patients with suspected pulmonary embolism: variations by practice setting and training level. Acad Emerg Med 2007; 14: 537.
  • 38
    Van Belle A, Buller HR, Huisman MV, Huisman PM, Kaasjager K, Kamphuisen PW, Kramer MH, Kruip MJ, Kwakkel-van Erp JM, Leebeek FW, Nijkeuter M, Prins MH, Sohne M, Tick LW. Effectiveness of managing suspected pulmonary embolism using an algorithm combining clinical probability, D-dimer testing, and computed tomography. JAMA 2006; 295: 1729.
  • 39
    Musset D, Parent F, Meyer G, Maitre S, Girard P, Leroyer C, Revel MP, Carette MF, Laurent M, Charbonnier B, Laurent F, Mal H, Nonent M, Lancar R, Grenier P, Simonneau G, Evaluation du Scanner Spirale dans l’Embolie Pulmonaire study group. Diagnostic strategy for patients with suspected pulmonary embolism: a prospective multicentre outcome study. Lancet 2002; 360: 191420.

Appendix

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Addendum
  8. Author contributions
  9. Acknowledgements
  10. Disclosure of Conflict of Interests
  11. References
  12. Appendix

Table. Enrollment details for the 13 sites

Hospital nameLocationStart dateEnd dateMethod of enrollmentInformed consent Enrolled (n) Eligible (n)Computerized system(s) queried‡
  1. *70% waiver; initially started as consecutive, but fell short of 85%; ‡To determine number of eligible patients (see methods).

Carolinas Medical CenterCharlotte, NC5/1/20046/30/2006ConsecutiveWritten and waiver*26092838McKesson Horizon and Star
University HospitalCharlotte, NC9/2/20049/30/2005RandomWritten498641McKesson Horizon and Star
Northwestern Memorial HospitalChicago, IL10/27/20049/14/2006ConvenienceWritten10812285Cerner Powerchart
Yale School of MedicineNew Haven, CT9/1/20043/31/2005ConsecutiveVerbal551719Zixcorp (Dallas, TX)
Baystate Medical CenterSpringfield, MA12/2/20043/8/2006ConsecutiveWritten500581Cerner Millennium Clinical Information System (CIS).
St. Vincent Mercy MedicalToledo, OH5/10/20058/26/2006ConvenienceWaiver458500Invision Patient Information Net Access (Siemens)
Exempla St. JosephsDenver, CO3/29/20059/19/2005RandomWritten476627Meditech for CT/VQ and MiSys Laboratory System for Dimers
University of ColoradoDenver, CO8/9/20054/10/2006RandomWritten126202Pulsecheck (Picis)
Mayo ClinicPhoenix, AZ11/4/20049/30/2006RandomWritten432550IDX
Pitt County Memorial HospitalGreenville, NC2/1/200610/30/2006RandomWritten188210McKesson, Wellsoft and SMS
Christchurch HospitalChristchurch, NZ9/23/20039/30/2004ConsecutiveWritten427681Manual tracking
Massachusetts General HospitalBoston, MA10/1/20042/30/2006RandomVerbal492547EDIS and IDX
William Beaumont HospitalRoyal Oak, MI11/17/200511/16/2006ConvenienceWaiver3001832Medical record abstract system