Get access

Checklists improve experts’ diagnostic decisions

Authors

  • Matthew Sibbald,

    1. Ho Ping Kong Center for Excellence in Education and Practice, University Health Network, Toronto, Ontario, Canada
    2. Division of Cardiology, Department of Medicine, University of Toronto, Toronto, Ontario, Canada
    Search for more papers by this author
  • Anique B H de Bruin,

    1. Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands
    Search for more papers by this author
  • Jeroen J G van Merrienboer

    1. Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands
    Search for more papers by this author

Dr Matthew Sibbald, Toronto Western Hospital, East Wing 8-420, Division of Internal Medicine, Department of Medicine, University of Toronto, 399 Bathurst Street, Toronto, Ontario M5T 2S8, Canada. Tel: 00 1 416 603 6412; E-mail: mattsibbald@gmail.com

Abstract

Context  Checklists are commonly proposed tools to reduce error. However, when applied by experts, checklists have the potential to increase cognitive load and result in ‘expertise reversal’. One potential solution is to use checklists in the verification stage, rather than in the initial interpretation stage of diagnostic decisions. This may avoid expertise reversal by preserving the experts’ initial approach. Whether checklist use during the verification stage of diagnostic decision making improves experts’ diagnostic decisions is unknown.

Methods  Fifteen experts interpreted 18 electrocardiograms (ECGs) in four different conditions: undirected interpretation; verification without a checklist; verification with a checklist, and interpretation combined with verification with a checklist. Outcomes included the number of errors, cognitive load, interpretation time and interpretation length. Outcomes were compared in two analyses: (i) a comparison of verification conditions with and without a checklist, and (ii) a comparison of all four conditions. Standardised scores for each outcome were used to calculate the efficiency of a checklist and to weigh its relative benefit against its relative cost in terms of cognitive load imposed, interpretation time and interpretation length.

Results  In both analyses, checklist use was found to reduce error (more errors were corrected in verification conditions with checklists [0.29 ± 0.77 versus 0.03 ± 0.61 errors per ECG], and fewer net errors occurred in all conditions with checklists [0.39 ± 1.14 versus 1.04 ± 1.49 errors per ECG]; p < 0.01 for both). Checklists were not associated with increased cognitive load (verifications with and without checklists: 3.7 ± 1.9 and 3.3 ± 2.0, respectively; conditions with and without checklists: 4.0 ± 1.8 versus 3.9 ± 2.0, respectively [p = not significant for both]). Checklists resulted in greater interpretation times and lengths (p < 0.01 for all). However, checklists were efficient in terms of the cognitive load invested, interpretation time and interpretation length (p < 0.01 for all).

Conclusions  Among ECG interpretation experts, checklist use during the verification stage of diagnostic decisions did not increase cognitive load or cause expertise reversal, but did reduce diagnostic error.

Ancillary