Validation of the Confusion Assessment Method for the Intensive Care Unit in Older Emergency Department Patients

Authors

  • Jin H. Han MD, MSc,

    1. Center for Quality Aging, Vanderbilt University School of Medicine, Nashville, TN
    2. Department of Emergency Medicine, Vanderbilt University School of Medicine, Nashville, TN
    Search for more papers by this author
  • Amanda Wilson MD,

    1. Department of Psychiatry, Vanderbilt University School of Medicine, Nashville, TN
    Search for more papers by this author
  • Amy J. Graves SM, MPH,

    1. Department of Biostatistics, Vanderbilt University School of Medicine, Nashville, TN
    Search for more papers by this author
  • Ayumi Shintani PhD, MPH,

    1. Department of Biostatistics, Vanderbilt University School of Medicine, Nashville, TN
    Search for more papers by this author
  • John F. Schnelle PhD,

    1. Center for Quality Aging, Vanderbilt University School of Medicine, Nashville, TN
    2. Division of General Internal Medicine and Public Health, Vanderbilt University School of Medicine, Nashville, TN
    3. Geriatric Research, Education, and Clinical Center, Department of Veterans Affairs Medical Center, Tennessee Valley Health Care Center, Nashville, TN
    Search for more papers by this author
  • Robert S. Dittus MD, MPH,

    1. Division of General Internal Medicine and Public Health, Vanderbilt University School of Medicine, Nashville, TN
    2. Center for Health Services Research, Vanderbilt University School of Medicine, Nashville, TN
    3. Geriatric Research, Education, and Clinical Center, Department of Veterans Affairs Medical Center, Tennessee Valley Health Care Center, Nashville, TN
    Search for more papers by this author
  • James S. Powers MD,

    1. Center for Quality Aging, Vanderbilt University School of Medicine, Nashville, TN
    2. Division of General Internal Medicine and Public Health, Vanderbilt University School of Medicine, Nashville, TN
    3. Geriatric Research, Education, and Clinical Center, Department of Veterans Affairs Medical Center, Tennessee Valley Health Care Center, Nashville, TN
    Search for more papers by this author
  • John Vernon MD,

    1. Department of Psychiatry, Virginia Commonwealth University Medical Center, Richmond, VA
    Search for more papers by this author
  • Alan B. Storrow MD,

    1. Department of Emergency Medicine, Vanderbilt University School of Medicine, Nashville, TN
    Search for more papers by this author
  • E. Wesley Ely MD, MPH

    1. Center for Quality Aging, Vanderbilt University School of Medicine, Nashville, TN
    2. Center for Health Services Research, Vanderbilt University School of Medicine, Nashville, TN
    3. Division of Allergy, Pulmonary, and Critical Care Medicine, Vanderbilt University School of Medicine, Nashville, TN
    4. Geriatric Research, Education, and Clinical Center, Department of Veterans Affairs Medical Center, Tennessee Valley Health Care Center, Nashville, TN
    Search for more papers by this author

  • Presented at the American College of Emergency Physicians Research Forum, Denver, CO, October 2012.
  • Research reported in this publication was supported by the National Institute on Aging of the National Institutes of Health under award number K23AG032355, the National Center for Research Resources of the National Institutes of Health under award number UL1 RR024975-01 (now at the National Center for Advancing Translational Sciences, Grant 2 UL1 TR000445-06), and the Emergency Medicine Foundation Career Development Award. Dr. Ely was supported in part by the National Institutes of Health R01AG027472 and R01AG035117 and a Veteran Affairs MERIT award. Drs. Schnelle, Dittus, Powers, and Ely are also supported by the Veteran Affairs Geriatric Research, Education, and Clinical Center (GRECC). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health, Emergency Medicine Foundation, and Department of Veterans Affairs.
  • The authors have no potential conflicts of interest to disclose.

Address for correspondence and reprints: Jin H. Han, MD; e-mail: jin.h.han@vanderbilt.edu.

Abstract

Objectives

In the emergency department (ED), health care providers miss delirium approximately 75% of the time, because they do not routinely screen for this syndrome. The Confusion Assessment Method for the Intensive Care Unit (CAM-ICU) is a brief (<1 minute) delirium assessment that may be feasible for use in the ED. The study objective was to determine its validity and reliability in older ED patients.

Methods

In this prospective observational cohort study, patients aged 65 years or older were enrolled at an academic, tertiary care ED from July 2009 to February 2012. An emergency physician (EP) and research assistants (RAs) performed the CAM-ICU. The reference standard for delirium was a comprehensive (~30 minutes) psychiatrist assessment using the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision criteria. All assessments were blinded to each other and were conducted within 3 hours. Sensitivities, specificities, and likelihood ratios were calculated for both the EP and the RAs using the psychiatrist's assessment as the reference standard. Kappa values between the EP and RAs were also calculated to measure reliability.

Results

Of 406 patients enrolled, 50 (12.3%) had delirium. The median age was 73.5 years old (interquartile range [IQR] = 69 to 80 years), 202 (49.8%) were female, and 57 (14.0%) were nonwhite. The CAM-ICU's sensitivities were 72.0% (95% confidence interval [CI] = 58.3% to 82.5%) and 68.0% (95% CI = 54.2% to 79.2%) in the EP and RAs, respectively. The CAM-ICU's specificity was 98.6% (95% CI = 96.8% to 99.4%) for both raters. The negative likelihood ratios (LR–) were 0.28 (95% CI = 0.18 to 0.44) and 0.32 (95% CI = 0.22 to 0.49) in the EP and RAs, respectively. The positive likelihood ratios (LR+) were 51.3 (95% CI = 21.1 to 124.5) and 48.4 (95% CI = 19.9 to 118.0), respectively. The kappa between the EP and RAs was 0.92 (95% CI = 0.85 to 0.98), indicating excellent interobserver reliability.

Conclusions

In older ED patients, the CAM-ICU is highly specific, and a positive test is nearly diagnostic for delirium when used by both the EP and RAs. However, the CAM-ICU's sensitivity was modest, and a negative test decreased the likelihood of delirium by a small amount. The consequences of a false-negative CAM-ICU are unknown and deserve further study.

Resumen

Objetivos

En el servicio de urgencias (SU), los profesionales sanitarios no identifican el síndrome confusional agudo en aproximadamente un 75% de las ocasiones, debido a que no criban este síndrome de manera rutinaria. El Confusion Assessment Method for the Intensive Care Unit (CAM-ICU) es una valoración rápida (<1 minuto) del síndrome confusional agudo cuya utilización puede ser factible en el SU. El objetivo fue determinar su validez y fiabilidad en los pacientes mayores del SU.

Metodología

Estudio observacional de cohorte prospectivo, que incluyó a los pacientes de 65 años o más de un SU terciario universitario atendidos de julio de 2009 a febrero de 2012. Los colaboradores de la investigación (CI) y un médico de urgencias (MU) realizaron el CAM-ICU. El estándar de referencia fue una valoración psiquiátrica global (aproximadamente 30 minutos) mediante los criterios del Manual Diagnóstico y Estadístico de los Trastornos Mentales, 4ª Edición (DSM-IV). Todas las evaluaciones fueron enmascaradas para cada uno de los miembros y se llevaron a cabo en las 3 primeras horas. Se calculó la sensibilidad, la especificidad y las razones de probabilidad tanto para el MU como para los CI usando como estándar de referencia la valoración psiquiátrica. Se calculó también el índice kappa entre el MU y los CI para cuantificar la fiabilidad.

Resultados

De los 406 pacientes incluidos, 50 (12,3%) tuvieron un cuadro confusional agudo. La mediana de edad fue de 73,5 años (RIC: 69 a 80 años), 202 (49,8%) eran mujeres, y 57 (14.0%) no blancos. Las sensibilidades del CAM-ICU fueron de un 72,0% (IC 95% = 58,3% a 82,5%) y un 68,0% (IC 95% = 54,2% a 79,2%) en el MU y los CI, respectivamente. La especificidad del CAM-ICU fue de 98,6% (IC 95% = 96,8% a 99,4%) para ambos. La razón de probabilidad negativa fue de 0,28 (IC 95% = 0,18 a 0,44) y de 0,32 (IC 95% = 0,22 a 0,49) en el MU y los CI, respectivamente. La razón de probabilidad positiva fue de 51,3 (IC 95% = 21,1 a 124,5) y de 48,4 (IC 95% = 19,9 a 118,0), respectivamente. El índice kappa entre el MU y los CI fue de 0,92 (IC 95% = 0,85 a 0,98), el cual es una excelente fiabilidad entre observadores.

Conclusiones

En los pacientes mayores del SU, el CAM-ICU es muy específico y una prueba positiva es casi diagnóstica de síndrome confusional agudo tanto cuando es realizada por los CI como por el MU. Sin embargo, la sensibilidad del CAM-ICU fue modesta, y una prueba negativa disminuyó la probabilidad de síndrome confusional agudo de forma discreta. Las consecuencias de un falso negativo de CAM-ICU son desconocidas y merecen futuros estudios.

Ancillary