Get access

Validating a script concordance test for assessing neurological localization and emergencies

Authors

  • K. Tan,

    Corresponding author
    1. Department of Neurology, Office of Neurological Education, National Neuroscience Institute, Singapore City, Singapore
    2. Yong Loo Lin School of Medicine, National University of Singapore, Singapore City, Singapore
    • Correspondence: K. Tan, Department of Neurology, National Neuroscience Institute, 11 Jalan Tan Tock Seng, Singapore 308433, Singapore (tel.:+65-6357-7171; fax: +65-6357-7137; e-mail: Kevin_Tan@nni.com.sg).

    Search for more papers by this author
  • N. C. K. Tan,

    1. Department of Neurology, Office of Neurological Education, National Neuroscience Institute, Singapore City, Singapore
    2. Yong Loo Lin School of Medicine, National University of Singapore, Singapore City, Singapore
    Search for more papers by this author
  • N. Kandiah,

    1. Department of Neurology, Office of Neurological Education, National Neuroscience Institute, Singapore City, Singapore
    2. Yong Loo Lin School of Medicine, National University of Singapore, Singapore City, Singapore
    Search for more papers by this author
  • D. Samarasekera,

    1. Yong Loo Lin School of Medicine, National University of Singapore, Singapore City, Singapore
    Search for more papers by this author
  • G. Ponnamperuma

    1. Faculty of Medicine, University of Colombo, Sri Lanka
    Search for more papers by this author

Abstract

Background and purpose

Clinical judgment is the ability to weigh clinical information and make decisions under conditions of uncertainty. Although neurological localization (NL) and neurological emergencies (NE) present such uncertainties, no validated method is reported to assess these decision-making skills. A script concordance test (SCT) was designed and validated to assess clinical judgment in NL and NE.

Methods

Our SCT comprised 14 clinical scenarios (53 questions). Candidates picked the response they considered the best for the questions in each scenario. Undergraduates and internal medicine residents completed the SCT; their responses were scored against the scoring key derived from an expert panel of accredited neurologists. Scores were expressed as a percentage of the maximum score.

Results

Mean total scores for undergraduates (n = 52), residents (n = 37) and experts (n = 15) were 61.0 ± 0.9, 68.3 ± 1.1 and 76.6 ± 1.1 (mean ± standard error of the mean, P < 0.001). Mean scores for undergraduates, residents and experts were 59.3 ± 1.1, 66.4 ± 1.4 and 76.1 ± 1.8 (P < 0.001) for NL, and 62.9 ± 1.3, 70.4 ± 1.3 and 77.2 ± 1.6 (P < 0.001) for NE. Senior residents scored higher than junior residents (postgraduate years 2–5 versus postgraduate year 1, 69.7 ± 1.4 vs. 65.3 ± 1.1, P = 0.035). The higher scores with increasing clinical experience supports the construct validity of the SCT. The SCT showed acceptable reliability (G coefficient 0.74 ± 0.05).

Conclusions

Our SCT is validated to reliably assess NL and NE in undergraduate and postgraduate learners; it is generalizable and feasible. It has potential as a valuable adjunct assessment tool for clinical judgment. Future plans to design SCTs to evaluate other topics in clinical neurology, as a multi-center study, are under way.

Ancillary