Norms of performance of sustained attention among a community sample: Continuous Performance Test study

Authors


Dr Yen Kuang Yang, Department of Psychiatry, College of Medicine, National Cheng Kung University, 138 Sheng Li Road, Tainan 70428, Taiwan. Email: ykyang@mail.ncku.edu.tw

Abstract

Abstract  The Continuous Performance Test (CPT) is a widely used measure of sustained attention, which is a preferred tool for assessing various mental functions. A well-established norm for CPT is essential when choosing a suitable threshold for classifying individuals as either case (CPT impairment) or non-case. The CPT performance of 900 adults who were randomly chosen from a community survey was measured to establish the norms for subgroups with different gender, age, and educational levels. The results revealed that age and educational level are significantly associated with the performance sensitivity (d′) of CPT. Male subjects perform better than female subjects. Seventeen percent of the subjects scored higher on the masked CPT than on the unmasked CPT. Subjects who could not finish the masked CPT had the characteristics of older age and fewer years of education. When classifying a patient as case or non-case, his/her CPT performance should be considered relative to the norms for his/her gender, age, and educational levels.

INTRODUCTION

The Continuous Performance Test (CPT) is a widely used measure of sustained attention. Designed for measuring deficits in sustained attention among brain-injured  patients,  the  CPT  has  been  well  established  as a preferred tool for measuring a number of mental functions.1–6 In addition, the CPT has been suggested as a valid phenotypic indicator of schizophrenia genotype.7–10 The CPT performance can vary with the format and indices used. The information process load of the CPT-AX (a character or number proceeded by another character or number as a target) is higher than that of the CPT-X (single character or number as target). If the stimulus is blurred during the masked or degraded CPT, poorer performance will result.2 The performance indices of the CPT have evolved from hit rate (HR) and false alarm rate (FAR), which are indices derived from signal detection theory, to lnβ (response criterion) and d′ (sensitivity criterion).11

A well-established norm for CPT is essential when choosing a suitable threshold to classify individuals as either case (CPT impairment) or non-case. Chen et al. reported a norm for CPT performance based on a study of 345 subjects in Taiwan.12 Because it is known that performance of CPT is significantly correlated with gender, age, and educational level of subjects,12 stratified comparisons among subgroups of different ages, genders and educational levels are essential.

The aim of the present study was to investigate the CPT performance of a larger sample randomly recruited using a well-designed community survey to explore the norms of CPT performance for different gender, age and educational subgroups.

METHODS

Sampling subjects

This study was performed in Tainan, the oldest city in southern Taiwan. Tainan has a population of more than 750 000. The sampling scheme is a three-stage process that generated a stratified systematic cluster sample of households throughout the city. In the first stage, areas of the city were divided into seven strata according to the administrative districts. One area (Li) was randomly selected from each stratum. In the second stage, every fifth household within each of the seven selected areas was systematically identified. In the third stage, the selected households were informed about the survey by letter and telephone from the medical center to obtain consent for participation and to schedule examinations. All of the subjects, whose ages were ≥20 years according to the government population register in 1995, were included in the study.13 This cohort was followed up every year. This is the follow up to a fifth-year study. Before the study began, informed consent was obtained. The Ethics Committee for Human Research at National Cheng Kung University Medical Center approved the study protocol.

At the time of the first year, 1638 residents were recruited. Due to loss of contact, only 900 subjects (425 men and 475 women) remained in our study group at year 5. The mean age of the men was 49.77 ± 15.15 years and that of the women was 47.85 ± 13.60 years. The percentage of the subjects in the 21–35, 36–50, and 50+-years-old groups was 18.6%, 39.1%, and 42.4% among the men and 20.6%, 39.2%, and 40.2% among the women, respectively. There was no significant age difference between male and female subjects.

As for the distribution of education-receiving years, the average was 11.73 ± 4.42 years among the men and 10.20 ± 5.08 years among the women. The men received more years of education than the women (t = 4.81, P < 0.001). The percentage of subjects with education-receiving years below 6, 6–12, and >12 years were 21.9%, 40.0%, and 38.1% among the men and 31.7%, 40.6%, 27.6% among the women, respec-tively. Negative correlation exists between age and education (male: r = −0.51, P < 0.001; female: r = −0.69, P < 0.001).

Continuous Performance Test procedures

The CPT or continuous performance task is a psychological test for humans that primarily measures attention. The task presents a time series of visual stimuli to the subject. The subject is instructed to press a switch button in response to a critical stimulus. The critical stimulus may be defined either as a particular single stimulus out of the available set (X task), or a particular sequence of two stimuli out of the available set (AX task). Only AX tasks were used in the present study. In this study, subject responses were recorded automatically on a diskette using the CPT machine (Sunrise Systems, version 2.20, Pembroke, MA, USA).

Visual stimuli are presented on a 2-in. (5.1-cm) square matrix of green light-emitting diodes, 21 high by 20 wide. The interstimulus interval is 800 ms and the stimulus duration is 200 ms. The available response time is 700 ms. Numbers from 0 to 9 are randomly presented. The target stimulus was the number 9 preceded by the number 1. Each subject underwent two CPT sessions: the unmasked and masked tasks. During the unmasked session, subjects responded to the target stimulus by pressing a button. A total of 341 trials with 31 targets was presented over 5 min. During the masked session, a pattern of snow was used to camouflage an image of dots so that the image was not distinct. Each session of the test began with 2 min of practising, if the subject required it. The rater monitored the subjects’ performance through a computer monitor to confirm that the subjects knew how to press the button correctly. The subjects were instructed to press a given button as correctly as possible.

According to the signal detection theory, the fundamental task of this test is to discriminate the signal (target) from the background noise (non-target).14 The d′ (sensitivity) is a measure of the subject's ability to discriminate a signal from the background noise. A higher d′ indicates better processing capabilities. In general, the attention test index (d′s) should be normally distributed. The response criterion index, lnβ, was computed as ln{y[z(HR)/y[z(FAR)]}, where y is the ordinate of the normal distribution.2,14,15 The response criterion (lnβ) was calculated after a total of 341 trials with 31 targets presented, in order to define the receiver operating characteristic (ROC) curve in signal detection theory.16 For studies with a small sample size, the non-parametric method (A′) is applied to estimate d′. In the present study, A′ was computed by using the equation from Grier17 as follows:

A′ = 1/2 + (HR − FAR) × (HR + 1 − FAR)/4 × (HR) × (1 − FAR)

Statistical analysis

Pearson product moment coefficient was used to analyze the correlation between two categorical variables. Paired t-tests were used to compare the performance indices of the unmasked CPT and the masked CPT. An independent t-test or anova was used to analyze the difference between the performances of two or more groups. Multiple linear regression was used to assess the effects of gender, age, and education on the CPT performance indices. P < 0.05 was considered significant (two-tailed). All analyses were performed using spss (SPSS, Chicago, IL, USA).

RESULTS

All 900 subjects completed the unmasked CPT and 828 participants (399 men, 429 womens) completed the masked CPT. The resulting CPT indices are shown in Table 1. Regarding the differences between the two sessions, paired t-tests revealed that subjects had a lower HR, d′ and higher FAR during the masked CPT (Table 1). However, there was no statistical difference in lnβ between the two CPT sessions. For the unmasked CPT, the percentage of subjects with 100% HR, 0% FAR, and the best d′ (d′ = 4.86) was 30.9%, 40.1%, 21.4%, respectively. When subjects took the masked CPT, their performance became worse probably because of the disturbance of the dots. The percentage with 100% HR, 0% FAR, the best d′ decreased to 11.1%, 16.6%, and 5.7%, respectively.

Table 1.  Distributions of CPT performance indices
 Unmasked (n = 900)
Mean ± SD
Masked (n = 828)
Mean ± SD
t
  • CPT, Continuous Performance Test.

  • Seventy-two subjects could not complete masked CPT.

  • Paired-t-tests were used.

  • ***

    P < 0.001 (two tailed).

Hit rate (%)91.04 ± 12.9280.49 ± 19.66 19.29***
False alarm (%)0.95 ± 1.781.54 ± 2.06−12.54***
d′4.05 ± 0.843.39 ± 1.03 23.95***
lnβ1.99 ± 0.832.02 ± 0.83 0.11

To examine whether d′ and lnβ are independent, the Pearson correlation test are performed. Although there are positive correlations between d′ and lnβ in both the unmasked (= 0.17, P < 0.001) and masked (= 0.18, P < 0.001) CPT, the correlations are weak. Additionally, there is a highly significant positive correlation between A′ and d′ both in the unmasked (= 0.94, P < 0.001) and masked CPT (= 0.95, P < 0.001).

The results of linear regression, which was performed to explore the association between demographic characteristics and the performance of CPT, are shown in Table 2. Gender, age, and education-receiving years, are correlated with the performance of CPT. Men who were younger and had more education-receiving years had higher HR, lower FAR, and better d′ in both the unmasked or the masked CPT. However, none of these demographic features had an effect on lnβ. Age has the strongest influence on d′ (unmasked, β= −0.237, P < 0.001; masked, β= −0.266, P < 0.001), followed by educational level (unmasked, β= 0.232, P < 0.001; masked, β = 0.19, P < 0.001) and gender (unmasked, β = 0.125, P < 0.001; masked, β = 0.127, P < 0.001).

Table 2.  Coefficients for the regression of CPT performance indices
 Unmasked (n = 900)Masked (n = 828)
Hit rate (%)False alarm (%)d′ln βHit rate (%)False alarm (%)d′ln β
  • CPT, Continuous Performance Test.

  • Seventy-two subjects could not complete masked CPT.

  • Dummy variable, male = 1 and female = 0.

  • **

    P < 0.01;

  • ***

    P < 0.001 (two tailed).

Gender 0.13 (0.01)***−0.10 (0.00)** 0.16 (0.05)*** 0.03 (0.06) 0.13 (0.01)***−0.10 (0.00)** 0.18 (0.07)*** 0.05 (0.06)
Age−0.24 (0.00)*** 0.16 (0.00)*** −0.31 (0.00)***−0.01 (0.00)−0.27 (0.00)*** 0.18 (0.00)***−0.31 (0.00)***−0.06 (0.00)
Education 0.23 (0.00)***−0.23 (0.00)*** 0.27 (0.01)*** 0.04 (0.01) 0.18 (0.00)***−0.17 (0.00)*** 0.19 (0.01)*** 0.01 (0.01)
F72.28***47.64***127.73*** 1.0758.50***32.71***81.57*** 1.81
R2adjusted 0.19 0.14 0.30 0.00 0.17 0.10 0.23 0.00

Because gender, age, and education level all influence the performance of both the unmasked and masked CPT, the subjects were divided into subgroups according to these three demographic factors, and the indices of CPT performance are listed in Table 3. Due to the 9-year compulsory education policy in Taiwan, there were few young subjects with low education level (<6 years). There were also few older subjects who were born before the 9-year compulsory education policy was enforced. Thus, both of these groups had fewer than 30 subjects while the remaining groups had more than 30.

Table 3.  Distribution of indices for unmasked and masked CPT
Age groupsUnmaskedMasked
nHit rate (%)False alarm (%)d′lnβnHit rate (%)False alarm (%)d′lnβ
  1. CPT, Continuous Performance Test.

Male
21–35 years
 Education years
 ≤61 104.862.021 0.970.014.191.03
 6–122992.10 (14.39)0.55 (1.26)4.24 (0.88)2.17 (0.55)2891.24 (11.87)0.45 (0.62)4.16 (0.74)2.23 (0.61)
 >124997.56 (3.13)0.24 (0.54)4.61 (0.33)2.04 (0.54)4892.74 (10.26)0.57 (0.92)4.20 (0.70)2.09 (0.73)
36–50 years
 Education years
 ≤61392.31 (8.38)0.69 (0.91)4.10 (0.65)2.09 (1.00)1173.31 (18.54)2.78 (3.07)2.93 (1.16)1.90 (0.92)
 6–127096.04 (7.16)0.55 (1.37)4.44 (0.56)1.95 (0.82)6785.51 (18.42)1.32 (2.46)3.71 (0.98)1.96 (0.95)
 >128397.05 (4.78)0.26 (0.45)4.56 (0.39)2.05 (0.59)8187.06 (14.90)0.73 (0.88)3.86 (0.87)2.20 (0.66)
>50 years
 Education years
 ≤67985.83 (13.39)1.56 (2.21)3.59 (0.82)2.00 (1.01)6868.45 (22.77)2.44 (3.20)2.77 (1.07)2.01 (1.02)
 6–127191.59 (8.72)0.84 (1.25)4.03 (0.69)2.04 (0.86)6683.04 (17.09)1.43 (1.55)3.44 (0.88)1.95 (0.94)
 >123090.86 (14.10)0.85 (1.33)4.05 (0.91)1.97 (0.67)2981.42 (16.33)1.60 (1.74)3.41 (1.03)2.01 (0.74)
Female
21–35 years
 Education years
 ≤600
 6–123194.90 (8.24)0.43 (0.89)4.39 (0.55)2.14 (0.82)3085.16 (12.95)1.41 (1.74)3.55 (0.89)2.04 (0.62)
 >126797.01 (5.34)0.30 (0.60)4.55 (0.44)2.01 (0.59)6589.88 (10.11)0.91 (1.09)3.89 (0.74)2.04 (0.74)
36–50 years
 Education years
 ≤62983.43 (19.50)1.23 (1.46)3.58 (0.86)1.96 (1.11)2562.58 (27.37)2.37 (2.17)2.54 (1.11)1.97 (0.97)
 6–1210692.30 (13.34)1.16 (2.19)4.05 (0.84)1.80 (0.92)9979.47 (17.89)1.60 (1.79)3.25 (0.86)2.04 (0.87)
 >125192.41 (9.78)0.61 (0.90)4.16 (0.67)2.09 (0.81)4783.67 (17.51)1.46 (2.21)3.56 (1.01)1.95 (0.68)
>50 years
 Education years
 ≤612281.70 (17.24)2.09 (2.83)3.28 (0.94)1.99 (0.98)9969.70 (21.70)2.47 (2.45)2.73 (0.95)1.94 (0.89)
 6–125690.09 (13.55)1.07 (1.36)3.89 (0.81)1.83 (0.77)5173.12 (22.85)2.00 (1.02)2.96 (1.02)1.85 (0.78)
 >121388.09 (9.89)1.09 (2.21)3.84 (0.80)2.32 (0.92)1381.89 (15.56)1.29 (1.36)3.42 (0.83)2.12 (0.90)

We further examined the differences in demographic variables between the three Δd′ (Δd′ = d′masked–d′unmasked) groups: Δd′ ≤ 0, Δd′ > 0, and unfinished masked CPT (Table 4). Most of the subjects performed worse in the masked CPT than in the unmasked version (Δd′ < 0) (χ2 = 710.6, d.f. = 2, P < 0.001). Some of the subjects (men 18%, women 16%, respectively) had better d′ in the masked CPT (Δd′ > 0). A small percentage of subjects (6% of the men and 10% of the women) could not complete the masked CPT. The results of one-way anova showed that there are significant differences among the three groups (Δd′ ≤ 0, Δd′ > 0, and those who could not complete the masked CPT) in age and education-receiving years. However, no significant differences in distribution of gender are observed between Δd′ ≤ 0 and Δd′ > 0. Scheffe multiple comparison tests further showed that subjects who could not complete the masked CPT were older and had fewer education-receiving years than either the Δd′ ≤ 0 (< 0.001) or Δd′ > 0 group (< 0.001). There was no significant difference in age or education-receiving years between the Δd′ ≤ 0 (= 0.96) and Δd′ > 0 (= 0.92) groups.

Table 4.  Comparison of subjects with decreased d′ and subjects with increased d′ during the masked CPT (mean ± SD)
GroupsnΔd′d′unmaskedGender
n (M/F)
Age (years)Education years
  1. CPT, Continuous Performance Test.

  2. Total: n (men) = 425, n (women) = 475.

  3. Δd′= d′masked–dunmasked.

Δd′ ≤ 0674−0.10 ± 0.734.21 ± 0.72321/35348.06 ± 13.8811.29 ± 4.61
Δd′ > 0154 0.43 ± 0.403.75 ± 0.8278/7647.69 ± 14.5811.12 ± 4.74
Unable to complete masked CPT 72 3.19 ± 1.1826/4657.56 ± 15.69 7.08 ± 5.50
Statistics   χ2 = 4.34, d.f. = 2,
P = 0.11
F2,897 = 15.15,
P < 0.001
F2,897 = 26.09,
P < 0.001

DISCUSSION

Indices of Continuous Performance Test

Our results for the unmasked CPT performance are similar to those of the Chen et al. report that included 345 adults as subjects.12 As did Chen et al., we also found a ceiling effect of the CPT performance because many of the subjects obtained perfect scores. In the current study, the ceiling effect was less in d′ than in HR and FAR, and less in the masked CPT than the unmasked one. However, HR and FAR are used more frequently because they are easily calculated.4

Theoretically, d′ should be independent of lnβ, although our results showed a weak but positive correlation between d′ and lnβ. That is, subjects with more conservative criterion (higher lnβ) also have a higher degree of sensitivity. The relationship between d′ and lnβ, which may be influenced by age and education-receiving years, needs further investigation.

According to the study by Nuechterlein, there is also a significantly high correlation between d′ and A′ (= 0.92).2 In the present study, the significant correlation between d′ and A′ was also significant in both the unmasked and masked CPT. A′ has the advantage of being easier to calculate than d′, especially in small sample studies. Thus, it might be useful and convenient to use A′ instead of d′ as an index of CPT performance in small sample clinical studies.

Effects of demographic variables on CPT performance

Gender, age and education-receiving years have a profound influence on the indices of CPT performance.12 The present results confirmed that the HR and d′ of the CPT are affected by gender, age, and education-receiving years while the response criterion, lnβ, is not.12 However, the fact that the FAR is affected by gender, age, and education-receiving years, has not been demonstrated in previous studies.12 In previous studies, older age is associated with a poorer performance for both sessions of the CPT in adults.12,18 For children, aged 9–17, the opposite is true. Men had higher HR and d′ than women in the masked CPT. Our results are consistent with all of these findings.

Different performances between two sessions of CPT

Theoretically, people may perform worse on the masked version of CPT due to disturbances of dots in the masked test. In the current study, most subjects had lower HR, higher FAR, and lower value of d′ when taking the masked CPT.12 There were as many as 3–8% of the subjects who had difficulty taking the masked CPT due to older age. In addition, the results also showed that some of the subjects had better d′ in the masked CPT than in the unmasked CPT. It is possible that this could have been due to practice effect or fatigue effect because subjects took the masked CPT after the unmasked CPT. The magnitude of the change between the two sessions of CPT was significantly smaller and the baseline values of d′ (i.e. the values of the unmasked CPT) were lower in those who had Δd′ ≤ 0 than those who had Δd′ > 0 both in men and in women. However, the results of linear regression showed that gender, age and education-receiving years were not associated with the CPT performance of these two groups of people. Because the practice effect or fatigue effect may exist between the two sessions of CPT in some participants, practice effect or fatigue effect should be considered. Our findings support those of Chen et al.12 Fatigue and practice effects may have confounded the results in the present study. Therefore, counterbalance for such effects is recommended for future studies.

Subjects who could not complete the masked CPT were older and had fewer years of education than those who finished both sessions of the CPT. Women had more difficulties completing the masked CPT. These results show that gender, age and education-receiving years can all affect the performance of CPT, especially in the masked CPT.

The gender-, age-, and education-receiving years-specific distributions of CPT performance indices obtained in the present study make it possible to choose a suitable threshold for classifying and stratifying an adult performance as either normal or deficit. However, ceiling effect and practice effect may exist in some participants. Thus, it is important to classify a participant as having deficits in these CPT performance indices with adjustment for the aforementioned factors. The data presented in the current report will be valuable for standardization of some demographic factors in CPT performance. Because we used a community sample for the present study, it is possible that mental problems, dementia, or physical problems may confound the results. Therefore, the results may not be generalizable to a healthy population.

ACKNOWLEDGMENTS

The authors wish to thank Ms Shu-Chuan Lin and Linda J. Chang and the staff of the Department of Psychiatry and Department of Family Medicine in National Cheng Kung University for their statistical and administrative assistance in the preparation of this manuscript.

Ancillary