Verifying the Japanese version of the Preschool Confusion Assessment Method for the ICU (psCAM‐ICU)
Funding information:
This work was supported by Tsukuba University education/research funds.
Abstract
Aim
Pediatric delirium has been well investigated and its prevalence is reported to be from 20% to 44%. For pediatric intensive care settings, several validated assessment tools for diagnosing delirium, including the Preschool Confusion Assessment Method for the Intensive Care Unit (psCAM‐ICU), are available in English. However, validated assessment tools for identifying pediatric delirium are unavailable in Japanese. Therefore, the aim of this study is to verify the Japanese translation of the psCAM‐ICU.
Methods
We enrolled patients at the Pediatric ICU at University of Tsukuba Hospital (Tsukuba, Japan) from May 2017 to February 2019. Enrollment criteria included patients aged 6 months to 5 years, and we excluded coma patients scoring under −4 on the Richmond Agitation–Sedation Scale or suffering from stroke. Pediatric patient delirium was simultaneously evaluated by three medical workers (pediatric intensivist and researchers). Psychiatrists then verified these findings against criteria of the Diagnostic and Statistical Manual of Mental Disorders – 5th Edition. We evaluated criterion validity (sensitivity and specificity) and reliability using Cohen's κ coefficient.
Results
We made a total of 56 independent assessments of 19 patients (42% female) with an average age of 18 (±15) weeks. Mechanical ventilation was used at least once in 73% of patients and the positive rate of delirium was 54% in total observation. Overall, the psCAM‐ICU showed high sensitivity, specificity (sensitivity, 0.90 [95% confidence interval [CI], 0.80–0.94]; specificity, 0.93 [95% CI, 0.83–0.97]), and high reliability within the researcher assessments (κ = 0.92; 95% CI, 0.82–1.0).
Conclusion
We verified the psCAM‐ICU and it shows high validity and reliability.
Introduction
Delirium frequently occurs as a treatment complication in intensive care patients and pediatric delirium has a reported prevalence of 20–44%.1, 2 Delirium rapidly arises due to a physiological transformation characterized by fluctuating course, attention deficits, disorganized thinking, and an altered level of consciousness.3 Within published works, delirium has also become recognized as a biomarker of acute brain dysfunction.4 This devastating condition is associated with higher mortality,5 longer length of hospital stay, and prolonged mechanical ventilation time, making it a serious health‐care issue.6 Moreover, delirium is usually underrecognized in daily routine care;7 using assessment tools will improve the diagnostic accuracy of delirium.8, 9 Also complicating diagnosis is the further issue that, within pediatric intensive care settings, there is a shortage of reports on the effect of mechanical ventilation with respect to sedation condition and the development of delirium. There are several validated assessment tools available in English for pediatric delirium to address this issue, including the Preschool Confusion Assessment Method for the Intensive Care Unit (psCAM‐ICU).2 The psCAM‐ICU is in accordance with the Diagnostic and Statistical Manual of Mental Disorders – 5th Edition (DSM‐5) and has been reported as a reliable and valid tool for infant and preschool‐aged children.2 We have previously published a back‐translated version of the psCAM‐ICU in Japanese.10 However, as there are no currently validated pediatric delirium assessment tools in Japanese, the aim of this study is to validate our back‐translated psCAM‐ICU for clinical use.
Method
Patient selection
We registered patients in our mixed‐use (eight‐bed post‐surgical and internal medicine) pediatric ICU at the University of Tsukuba Hospital (Tsukuba, Japan) from May 2017 to February 2019 on Wednesdays during morning and afternoon shifts (9.00 am and 2.00 pm). Enrollment criteria included patients aged 6 months to 5 years and we excluded coma patients scoring under −4 on the Richmond Agitation–Sedation Scale (RASS), or suffering from stroke. We extracted characteristics, including age, sex, mechanical ventilation status, and diagnosis for pediatric ICU admission, from medical records. This study was carried out under laws equivalent to or derived from the principles of the Declaration of Helsinki and was approved by the University of Tsukuba Institutional Review Board (approval # H28‐085).
Characteristics, prevalence, and distribution of delirium
We recorded age, sex, diagnosis, mechanical ventilation status, usage of dexmedetomidine, usage of midazolam, usage of opioids, length of ICU stay (in days), Pediatric Index of Mortality 2 score (PIM2),11 Pediatric Sequential Organ Failure Assessment Score (pSOFA),12 and sedation status on the RASS. Both PIM2 and pSOFA are generally accepted methods of scoring severity and already validated for prediction of in‐hospital mortality.12, 13 The RASS was originally developed for adult patients but has recently been validated in critically ill pediatric patients.14 As Smith and colleagues examined the validity of psCAM‐ICU by using RASS,2 we deemed it suitable for our purposes. Relationships between cohort characteristics and the distribution of delirium with and without mechanical ventilation with respect to sedation condition were then analyzed.
Pediatric delirium assessment
Pediatric patient delirium was simultaneously evaluated by three medical workers (researchers and a pediatric intensivist). Researchers evaluated pediatric delirium by the Japanese version of the psCAM‐ICU while the pediatric intensivist assessed delirium in an assistive capacity by the Vanderbilt Assessment for Delirium in Infants and Children (VADIC),15 which was then verified by a psychiatrist in accordance with DSM‐5 criteria.3 As psychiatrists in Japan have little training in assessment of the varying degrees of pediatric delirium, we therefore chose a pediatric intensivist for the assessment but relied on objective diagnosis by the psychiatrist (Fig. 1). Prior to the study, the first author learned the measurement procedures of psCAM‐ICU at the original author's hospital (The Monroe Carell Jr. Children's Hospital in Vanderbilt, Nashville, TN, USA).
Statistical analysis
To assess reliability between an observer's psCAM‐ICU assessment and the psychiatrist's diagnosis for pediatric delirium, we used Cohen's κ coefficient and bootstrapping method for evaluating a 95% confidence interval (CI). We also assessed reliability between two psychiatrists’ diagnoses. To assess criterion validity, we calculated sensitivity, specificity, negative predictive values (NPV), and positive predictive values (PPV) for psCAM‐ICU in our cohort.
Sample size calculations
Variability and required sample size fluctuate accordingly with the state of the cohort. Therefore, we evaluated sample size based on reliability as previously reported.16, 17 We undertook a pilot study for estimating sample size and we observed 24 paired, independent assessments out of a total of 74 with a good coefficient (κ = 0.90; 95% CI, 0.72–1.0). From this result, we estimated the κ coefficient between observers as 0.9 and the lower κ coefficient was also within the definition of Guilford's Rule of Thumb18 (lower κ = 0.7; upper κ coefficient = 0.99). The significance level (α) was 0.05 and we estimated the positive rate of delirium as 0.33 from the pilot study. Our calculations predicted that 56 observations were needed to reach significance for this study.19, 20
Results
Characteristics
From May 2017 to February 2019, we undertook a total of 162 observations (Fig. 2). Of these, 106 were excluded for the following reasons: 38 observations were in patients younger than 6 months of age, 15 observations were in patients over 5 years of age, and 53 observations were in comatose patients. With regard to the baseline characteristics of study patients, cardiac surgery patients accounted for 63% of enrollees. A total of 19 patients (42% female) were of an average age of 18 (±15) weeks. Approximately three‐quarters (73%) of patients had received mechanical ventilation at least once and the average severity score, as evaluated by PIM2, was 2.0 (±1.6). (Table 1) Of patients who participated in research evaluations, 55% were cardiac surgery patients. The average age of the 56 patients evaluated (42% female) was 12 (±11) months. Approximately half (46%) of patients had received mechanical ventilation at the time of observation and the average severity score, as evaluated by pSOFA, was 4.9 (±3.9). Nearly half of those evaluated received dexmedetomidine (48%), midazolam (60%), or opioids (53%) (Table 2).
| Variable | n = 19 |
|---|---|
| Age (months) ± SD | 18 ± 15 |
| Female, n (%) | 8 (42) |
| Diagnosis | |
| Cardiac surgical, n (%) | 12 (63) |
| Abdominal surgical, n (%) | 2 (10) |
| Thoracic surgical, n (%) | 1 (5) |
| Medical, n (%) | 4 (21) |
| PIM2 ± SD | 2.0 ± 1.6 |
| Mechanical ventilation††
Using mechanical ventilation during intensive care unit stay.
|
14 (73) |
- † Using mechanical ventilation during intensive care unit stay.
- PIM2, Pediatric Index of Mortality 2; SD, standard deviation.
| Variable | n = 56 |
|---|---|
| Age (months) ± SD | 12 ± 11 |
| Female, n (%) | 24 (42) |
| Diagnosis | |
| Cardiac surgical, n (%) | 31 (55) |
| Abdominal surgical, n (%) | 3 (5) |
| Thoracic surgical, n (%) | 2 (3) |
| Medical, n (%) | 20 (35) |
| Mechanical ventilation††
Using mechanical ventilation at the observation.
|
26 (46) |
| pSOFA ± SD | 4.9 ± 3.9 |
| Use of dexmedetomidine, n (%) | 27 (48) |
| Use of midazolam, n (%) | 34 (60) |
| Use of opioids, n (%) | 30 (53) |
| ICU days at observation | 45 ± 53 |
- † Using mechanical ventilation at the observation.
- ICU, intensive care unit; pSOFA, Pediatric Sequential Organ Failure Assessment; SD, standard deviation.
Distribution of delirium in all measurements
Evaluated pediatric delirium in total was 54% among 56 independent psychiatric assessments, mechanical ventilation was present in 79% of 24 independent assessments and absent among 34% of 32 independent assessments, all by DSM‐5 criteria. Sedated (RASS –1 to −3) patients tended towards high frequency of delirium compared with agitated (RASS +1 to +3) patients (RASS: −3, 100%; −1, 92%; 0, 32%; +1, 56%). There were no observations for patients with RASS scores of +2 to +4 (agitated to combative) in our cohort (Fig. 3).
Criterion validity
A total of 56 independent assessments were undertaken on 19 patients. Table 3 shows the criterion validity of the Japanese version of the psCAM‐ICU with regards to psychiatric diagnosis using DSM‐5 criteria. The psCAM‐ICU showed a sensitivity of 0.90 (95% CI, 0.80–0.94), specificity of 0.93 (95% CI, 0.83–0.97), PPV of 0.93 (95% CI, 0.83–0.97), NPV of 0.90 (95% CI, 0.80–0.94), likelihood ratio for positive results (LR+) of 13.0 (95% CI, 4.78–41.0), and a likelihood ratio for negative results (LR−) of 0.1 (95% CI, 0.05–0.22). When mechanical ventilation was present, the psCAM‐ICU showed a sensitivity of 0.89 (95% CI, 0.79–0.93), specificity of 0.80 (95% CI, 0.43–0.96), PPV of 0.94 (95% CI, 0.84–0.98), NPV of 0.66 (95% CI, 0.36–0.80), LR+ of 4.47 (95% CI, 1.40–23.14), and LR− of 0.13 (95% CI, 0.06–0.46) within 24 independent assessments. In patients without mechanical ventilation, the psCAM‐ICU showed a sensitivity of 0.91 (95% CI, 0.73–0.97), specificity of 0.95 (95% CI, 0.84–0.98), PPV of 0.91 (95% CI, 0.73–0.97), NPV of 0.95 (95% CI, 0.84–0.98), LR+ of 18.3 (95% CI, 4.59–70.49), and LR− of 0.08 (95% CI, 0.02–0.31) within 32 independent assessments.
| Variable | Without mechanical ventilation, n = 32 | With mechanical ventilation, n = 24 | Total number of observations, n = 56 |
|---|---|---|---|
| Sensitivity | 0.91 (0.73–0.97) | 0.89 (0.79–0.93) | 0.90 (0.80–0.94) |
| Specificity | 0.95 (0.84–0.98) | 0.80 (0.43–0.96) | 0.93 (0.83‐0.97) |
| PPV | 0.91 (0.73–0.97) | 0.94 (0.84–0.98) | 0.93 (0.83–0.97) |
| NPV | 0.95 (0.84–0.98) | 0.66 (0.36–0.80) | 0.90 (0.80–0.94) |
| LR+ | 18.3 (4.59–70.49) | 4.47 (1.40–23.14) | 13.0 (4.78–41.0) |
| LR− | 0.08 (0.02–0.31) | 0.13 (0.06–0.46) | 0.1 (0.05–0.22) |
- Data are shown as value (95% confidence interval).
- LR+, likelihood ratio for positive results; LR−, likelihood ratio for negative results; NPV, negative predictive values; PPV, positive predictive value.
Reliability
The Japanese version of the psCAM‐ICU showed high reliability for researchers with respect to total assessments (κ = 0.92; 95% CI, 0.82–1.0) (Table 4). This high reliability is unchanged between patients with mechanical ventilation (κ = 0.89; 95% CI, 0.68–1.0) and those without (κ = 0.93; 95% CI, 0.79–1.0). Reliability for the psychiatrist evaluation was high in the total population (κ = 1.0; 95% CI, 1.0–1.0), with mechanical ventilation (κ = 1.0; 95% CI, 1.0–1.0), and without mechanical ventilation (κ = 1.0; 95% CI, 1.0–1.0).
| Variable | Without mechanical ventilation, n = 32 | With mechanical ventilation, n = 24 | Total number of observations, n = 56 |
|---|---|---|---|
| Researcher to researcher‡‡
Psychiatrist using the Diagnostic and Statistical Manual of Mental Disorders – 5th Edition and comparison with psychiatrist diagnosis.
|
0.93 (0.79–1.0) | 0.89 (0.68–1.0) | 0.92 (0.82–1.0) |
| Psychiatrist to psychiatrist‡‡
Psychiatrist using the Diagnostic and Statistical Manual of Mental Disorders – 5th Edition and comparison with psychiatrist diagnosis.
,‡‡ Psychiatrist using the Diagnostic and Statistical Manual of Mental Disorders – 5th Edition and comparison with psychiatrist diagnosis. |
1.0 (1.0–1.0) | 1.0 (1.0–1.0) | 1.0 (1.0–1.0) |
- Data are shown as κ coefficient (95% confidence interval).
- † Researchers using psCAM‐ICU and comparison between researchers’ assessments.
- ‡ Psychiatrist using the Diagnostic and Statistical Manual of Mental Disorders – 5th Edition and comparison with psychiatrist diagnosis.
Discussion
The present study is the first to report the high criterion validity and reliability of the Japanese version of the PsCAM‐ICU (sensitivity of 0.90, specificity of 0.93, reliability κ = 0.92) in our cohort. As these results are in line with the psCAM‐ICU original study, we considered the back‐translated Japanese version to have valid accuracy. Methodologically, in this study, we modified the original method and used a pediatric intensivist to assess pediatric delirium using VADIC to support a psychiatric diagnosis under DSM‐5 criteria. This change was needed to effectively bridge gaps between two cultures; merely repeating the original method might not have been accurate for Japanese clinical usage.21 As the capability of Japanese psychiatry to assess pediatric delirium in Japan is limited, we posited that a pediatric intensivist, as an experienced specialist in pediatric observation, would be more suitable in our case. Additionally, the VADIC was suitable, in our judgement, to bridge the gap between Japanese and American cultural differences. Our observed delirium rate of 54% was variable based on the presence or absence of mechanical ventilation. Under light sedation (RASS −1), non‐mechanically ventilated patients trended towards delirium compared to those under mechanical ventilation. In contrast, in more agitated states (RASS +1 to +4), mechanically ventilated patients tended to experience a higher prevalence of delirium. This result indicates that patient agitation under mechanical ventilation, as well as drowsy patients without mechanical ventilation, need careful observation and assessment for signs of delirium. In our cohort, we did not observe RASS scores of +2 to +4 (agitated to combative) as a majority of our patients were admitted for cardiac surgery and sedation is strictly controlled in these patients to prevent negative hemodynamic effects. Nor did we observe RASS scores of −2 (light sedation) in any patient. As this state is defined by briefly (less than 10 s) awakens with eye contact to voice, our average age of 18 months might have enrolled patients who were not developmentally able to sustain light sedation. In spite of the above observations, RASS is still useful for estimating pediatric patient sedation levels and plays a role in deciding when to evaluate delirium assessment by psCAM‐ICU. However, even though children over the age of 5 years can be evaluated by the validated pCAM‐ICU,22 there has never been a validation study using a Japanese version until now. With this study, pediatricians now have validated tools available for evaluating delirium in all age ranges above 6 months.
Pediatric delirium in the Japanese pediatric ICU is still a relatively undiagnosed phenomenon,23 resulting in lapses of both diagnosis and management.24 For this reason, our study is the first to provide validation for the psCAM‐ICU tool, which could promote proper diagnosis of pediatric delirium as well as encourage the creation of screening and treatment guidelines to diagnose and manage this condition.
Limitation
We acknowledge several limitations of this study. This is a single center study consisting mostly of cardiac patients (due to the nature of our institute) selected by a non‐randomized participant pool that was chosen primarily by availability during the study period. This fact might reduce the generalizability of the results. Also, we applied repeated measurements in this study and this could generate bias in predicting delirium prevalence. However, as the definition of delirium indicates fluctuations over the course of the day, we believe this inherent variability prevented such bias. Additionally, some measurements could not be taken during emergency situations. Finally, we ran an additional experiment due to changes in the sample size estimation, dividing the study into two periods, which could affect the validity of the scale.
Conclusion
We VERIFIED THE Japanese version of psCAM‐ICU as having high validity and reliability.
Acknowledgments
The authors would like to thank the patients and their families for their participation in this study.
Disclosure
Approval of the research protocol: The Institutional Review Board of the University of Tsukuba Affiliated Hospital approved the present study (approval # H28‐085).
Informed consent: Informed consent was obtained from the guardians by opt‐out in publicity documents.
Registry and the registration no. of the study/trial: N/A.
Animal studies: N/A.
Conflict of interest: None declared.
