This study was presented in part at the 26th annual meeting of the Japanese Society for Psychiatric Diagnosis, Kyoto, Japan, October 2006.
Brief Assessment of Cognition in Schizophrenia: Validation of the Japanese version
Article first published online: 7 DEC 2007
Psychiatry and Clinical Neurosciences
Volume 61, Issue 6, pages 602–609, December 2007
How to Cite
KANEDA, Y., SUMIYOSHI, T., KEEFE, R., ISHIMOTO, Y., NUMATA, S. and OHMORI, T. (2007), Brief Assessment of Cognition in Schizophrenia: Validation of the Japanese version. Psychiatry and Clinical Neurosciences, 61: 602–609. doi: 10.1111/j.1440-1819.2007.01725.x
- Issue published online: 7 DEC 2007
- Article first published online: 7 DEC 2007
- Received 10 October 2006; revised 24 July 2007; accepted 5 August 2007.
- Brief Assessment of Cognition in Schizophrenia (BACS);
- cognitive function;
- Japanese version;
Abstract This preliminary study was performed to test the reliability and validity of the Brief Assessment of Cognition in Schizophrenia (BACS) as an assessment tool in a Japanese-language version (BACS-J). The subjects for the present study were 30 outpatients with chronic schizophrenia. Each subject gave written informed consent to participate in the research. Cronbach's alpha for the BACS-J was 0.77. The BACS-J composite score was significantly correlated with all primary measures of BACS-J (verbal memory, working memory, motor speed, verbal fluency, attention, and executive function). All BACS-J primary measures and the composite score were significantly correlated between two assessments. The mean score of the Digit Sequencing Task and composite score on the second assessment were significantly larger than those on the first assessment. All BACS-J primary measures except the Symbol Coding Task were significantly correlated with relevant standard neurocognitive tests. Also, the BACS-J composite score was significantly correlated with all standard neurocognitive tests except the Continuous Performance Test. A principal components analysis with varimax rotation resulted in a three-factor solution (executive function and memory; motor speed and general cognitive functions; and working memory). This preliminary study indicates that the BACS-J is a reliable and practical scale to evaluate cognitive function.
The assessment of cognitive function is an important step in evaluating patients with schizophrenia because of the extensive evidence that cognitive impairment is a core feature of schizophrenia,1,2 as well as a key determinant of functional outcome.3,4 Cognitive domains that show differential impairment in schizophrenia include attention (vigilance), executive function, long-term and learning memory, working memory, and verbal fluency.3–7 The long-term and learning memory and attention, in particular, have been shown to predict functional outcome better than positive and negative symptoms.3,4
For assessing cognitive function in schizophrenia, an extensive neuropsychological test battery (NTB), usually consisting of 8–12 tests tapping each of several tests for the putative neurocognitive domains, was used. These NTB are, however, time-consuming, technically difficult, and costly. Moreover, differences in NTB chosen for various studies make a direct comparison across studies difficult.
The availability of a quick and efficient tool for measuring cognition in patients with schizophrenia would be extremely useful for clinicians in considering an indication of rehabilitation, as well as for researchers implementing clinical trials to assess cognitive improvement. The Brief Assessment of Cognition in Schizophrenia (BACS) has been developed for facilitation of these key features.8 In addition, the domains of cognitive function assessed by the BACS are those found to be consistently impaired and related to outcome in schizophrenia, that is, verbal memory, working memory, motor speed, verbal fluency, attention, and executive function. The BACS is fully portable, and is designed to be easily administered by a variety of testers, including nurses, clinicians, psychiatrists, neurologists, social workers, and other mental health personnel. The time required for the testing with the BACS is only approximately 30 min with minimal extra time for scoring and training demands. The BACS has been shown to be reliable, valid and practical. Therefore, for its clinical application, we prepared the Japanese-language version (BACS-J; Kaneda et al., unpubl. data, 2007). The aim of the present study was to test the validity and usefulness of the BACS-J for the evaluation of cognitive function in schizophrenia.
Translation of the BACS
In order to have an approved translation of the BACS, the following translation methodology was used: two independent forward translations, reconciliation, two independent back-translations, and pilot testing on five people with schizophrenia. The author of the scale checked the back-translation.
The subjects for the present study were those seen at the Department of Psychiatry, Tokushima University Hospital between 28 March 2005 and 31 March 2006. Thirty outpatients with chronic schizophrenia were recruited. The diagnosis was based on DSM-IV criteria for schizophrenia,9 a detailed clinical interview, and review of the prior records. Hepatic and renal functions were normal in the subjects, and they were excluded if they presented with any organic central nervous system disorder, significant substance abuse, or mental retardation. The study was approved by the relevant ethics committee and was performed in accordance with the Declaration of Helsinki II. All patients who gave informed consent participated in the present study. Table 1 lists the subject characteristics.
|Age (years)||38.9 ± 14.0|
|A/A||34.7 ± 11.2|
|A/B||41.2 ± 12.6|
|Education (patient, years)||12.9 ± 2.0|
|A/A||13.8 ± 2.1|
|A/B||12.2 ± 1.2|
|Education (father, years)||11.4 ± 3.1|
|A/A||11.9 ± 2.4|
|A/B||10.9 ± 3.9|
|Education (mother, years)||10.9 ± 2.4|
|A/A||11.3 ± 2.9|
|A/B||10.8 ± 2.0|
|Duration of illness (years)||13.9 ± 10.9|
|A/A||13.9 ± 8.9|
|A/B||16.6 ± 8.7|
|BPRS (total)||29.7 ± 7.2|
|A/A||30.7 ± 5.7|
|A/B||30.6 ± 9.2|
|CDSS (total)||3.6 ± 3.8|
|A/A||3.9 ± 4.1|
|A/B||3.9 ± 3.8|
Fifteen of the 30 patients were being treated with a single second-generation antipsychotic medication (olanzapine, n = 4; perospirone, n = 4; risperidone, n = 7), two were being treated with a single first-generation antipsychotic (with haloperidol), and 13, with a combination of antipsychotic drugs.
The BACS-J includes brief assessments of verbal memory, working memory, motor speed, verbal fluency, attention, and executive function as follows.
Patients are presented with 15 words and then asked to recall as many as possible. This procedure is repeated five times. The outcome measure is the total number of words recalled. There are two alternate forms.
Digit sequencing task (working memory)
Patients are presented with clusters of numbers in random order of increasing length. They are asked to tell the experimenter the numbers in order, from lowest to highest. The outcome measure is the total number of correct items.
Token motor task (motor speed)
Patients are given 100 plastic tokens and asked to place them into a container as quickly as possible for 60 s. The outcome measure is the total number of tokens placed in the container.
Category fluency (verbal fluency)
Patients are given 60 s to name as many words as possible within the category of animals. The animal category was chosen based on a previous study of category fluency in Japanese patients with schizophrenia.10 The outcome measure is the total number of unique words generated.
Letter fluency (verbal fluency)
In two separate trials, patients are given 60 s to generate as many words as possible that begin with the letters ka and ta. These letters were chosen based on a previous study of letter fluency in Japanese patients with schizophrenia.11 The outcome measure is the total number of unique words generated.
Symbol coding (attention)
As quickly as possible, patients write numerals 1–9 as matches to symbols on a response sheet for 90 s. The outcome measure is the total number of correct responses.
Tower of London (executive function)
Patients look at two pictures simultaneously. Each picture shows three different-colored balls arranged on three pegs, with the balls in a unique arrangement in each picture. The patients are told about the rules in the task and asked to provide the total number of times the balls in one picture would have to be moved in order to make the arrangement of balls identical to that of the other, opposing picture. The outcome measure is the number of trials on which the correct response is provided. There are two alternate forms.
A composite score was calculated by averaging all z-scores of the six primary measures from the BACS-J, using the first test patient means and the standard deviations.
Subjects were tested on 1 day. Subjects received the BACS-J version A followed by a standard battery, which included the following tests: Rey Auditory–Verbal Learning Test (RAVLT)12 for Verbal Memory; Consonant Trigram Test (CTT)13 for Working Memory; General Aptitude Test Battery (GATB)14 for Motor Speed; Continuous Performance Tests (CPT)15 for Attention; computerized Wisconsin Card-Sorting Test (WCST)16 for Executive Function; and Wechsler Adult Intelligence Scale–Revised (WAIS-R)17 subtests (Picture Completion and Information) for general intelligence. The Category Fluency and Letter Fluency tests in BACS-J are typically included in the clinical assessment in cognitive dysfunction in patients with schizophrenia or Alzheimer disease, and therefore we did not conduct a control test for verbal fluency. At the end of the test session the subjects were randomized to receive either the BACS-J version A or B.
The following scales were completed by trained psychiatrists along with the NTB: the 18-item Brief Psychiatric Rating Scale (BPRS,18 1–7 score), and Calgary Depression Scale for Schizophrenia (CDSS).19 Based on previous groupings of the BPRS subscales,20 the Positive Symptoms (conceptual disorganization, grandiosity, hallucinatory behavior, and unusual thought content) and Negative Symptoms (emotional withdrawal, motor retardation, blunted affect, and disorientation) were defined. The CDSS is a nine-item structured interview scale, in which each item has a 4-point measure, to assess depression in schizophrenia.
The data analysis was conducted using JMP-5.1.2.J for Mac software (SAS Institute, Cary, NC, USA). Descriptive statistics were used to report the patients in terms of sociodemographic and clinical data. Cronbach's alpha21 was used to determine the internal consistency of the BACS-J. The relationship among the BACS-J measures was determined by calculating Pearson correlations among the scores. For all BACS-J primary measures and composite score, test–retest reliability was measured with Pearson correlations. Practice effects were measured by comparing data collected at test session 1 to those collected at test session 2 with repeated measure analysis of variance (anova). To test the construct validity, Pearson correlation coefficients (r) were used between the BACS-J primary measures and composite score and neurocognitive tests. The factor structure of the scores was determined by performing a principal components analysis with varimax rotation. For data analysis, the BACS scores on the first assessment were used, and the BACS scores on the second assessment were used only for the measurement of the test–retest reliability.
Eighteen out of 30 patients completed all tests. For the remainder the full tests were too much, and they just stopped during a test. Table 2 lists the means and standard deviations for all of the measures from the BACS-J. The BACS-J required a mean of 39.6 ± 5.5 min to complete, while the NTB required 107.0 ± 11.2 min.
|Condition||n||First test||n||Second test||r†||P(F)‡|
|Verbal memory||30||35.5 ± 11.7||19||37.9 ± 13.7||0.80****||0.1282 (0.1)|
|A/A||10||35.8 ± 13.4||10||41.6 ± 15.7||0.78**||0.0962 (0.4)|
|A/B||9||33.8 ± 10.9||9||33.8 ± 10.6||0.94****||>0.9999 (0)|
|Digit sequencing||30||15.7 ± 4.3||18||17.7 ± 4.2||0.76***||0.0059 (0.6)|
|Token motor task||30||68.8 ± 12.4||18||73.0 ± 21.3||0.70**||0.7263 (0.0)|
|Verbal fluency||30||32.8 ± 11.8||18||38.8 ± 8.9||0.65**||0.3899 (0.0)|
|Symbol coding task||30||46.4 ± 10.9||18||52.1 ± 12.1||0.86****||0.0502 (0.3)|
|Tower of London||30||14.5 ± 4.7||18||16.1 ± 4.2||0.72***||0.1157 (0.2)|
|A/A||10||15.4 ± 5.5||10||16.0 ± 5.2||0.97****||0.2172 (0.2)|
|A/B||8||13.4 ± 5.4||8||16.3 ± 2.9||0.32||0.2211 (0.3)|
|BACS-J composite score§||30||0.00 ± 0.72||19||0.33 ± 0.74||0.93****||0.0004 (1.0)|
|A/A||10||0.25 ± 0.75||10||0.54 ± 0.82||0.96****||0.0035 (1.7)|
|A/B||9||−0.14 ± 0.66||9||0.10 ± 0.60||0.88**||0.0536 (0.6)|
Cronbach's alpha for the BACS-J was 0.77.
Intercorrelations between BACS-J primary measures and composite score
Table 3 presents the correlations among the BACS-J primary measures and composite score. The BACS-J composite score was significantly correlated with all primary BACS-J measures.
|Digit sequencing||Token motor task||Verbal fluency||Symbol coding task||Tower of London||BACS-J composite score|
|Tower of London||0.67***|
|Symbol coding task||0.47**||0.80****|
|Token motor task||0.49**||0.58***||0.32||0.71****|
The Pearson correlations between the first and second assessment for each measure are given in Table 2. All BACS-J primary measures and the composite score were significantly correlated between two assessments except for the tests using a different version (alternate form) in the Tower of London Test.
Among the BACS-J primary measures, only the Digit Sequencing Test had a significant practice effect as shown on repeated-measure anova. In addition, the composite score showed a significant practice effect. However, when alternate forms of the test were administered, the composite score did not show a statistically significant practice effect.
Correlations for BACS-J measures and standard NTB
Table 4 shows performance on the standard battery of neurocognitive tests, and Table 5 shows the Pearson correlations between BACS-J measures and the standard NTB. All BACS-J primary measures except the Symbol Coding Task were significantly correlated with relevant standard neurocognitive tests. Also, the BACS-J composite score was significantly correlated with all standard neurocognitive tests except the CPT.
|n||Mean ± SD|
|RAVLT||24||38.3 ± 13.0|
|CTT||21||35.0 ± 9.4|
|GATB –Peg.1||21||13.1 ± 5.8|
|GATB –Peg.2||21||14.4 ± 2.9|
|CPT (d′)||18||48.9 ± 9.1|
|CPT (β)||18||48.0 ± 13.6|
|WCST-CA||21||3.2 ± 2.0|
|WCST-PEM||21||3.3 ± 2.4|
|WCST-PEN||21||6.2 ± 5.1|
|WAIS-R||27||90.3 ± 15.2|
|Verbal memory||Digit sequencing||Token motor task||Verbal fluency||Symbol coding task||Tower of London||BACS-J composite score|
Factor analysis for BACS-J primary measures
Table 6 shows the factor loadings. The analysis resulted in a three-factor solution that accounted for 80.3% of the variance. Eigenvalues (and percentage of variance explained) for Factor 1 (executive function and memory), Factor 2 (motor speed and general cognitive functions), and Factor 3 (working memory) were 3.1 (52.3%), 1.0 (16.1%), and 0.7 (11.9%), respectively.
|Token motor task||0.22||0.82||0.09|
|Symbol coding task||0.53||0.73||0.05|
|Tower of London||0.87||0.13||0.14|
|Percentage of variance||52.3||16.1||11.9|
BACS-J and psychiatric symptoms
The Positive Symptom score was significantly correlated with the Digit Sequencing Test score (r = −0.45, P = 0.01, n = 30). The Negative Symptom score was significantly correlated with the Verbal Memory (r = −0.48, P < 0.01) and Digit Sequencing (r = −0.41, P = 0.02) Test scores. The Depressive Symptom score was not significantly correlated with any of the BACS-J primary measures, or composite score.
Overall, the results of the present study indicate that the BACS-J is a useful tool to evaluate cognitive function in schizophrenia.
The BACS-J required a mean of 39.6 min to complete, whereas the original BACS was completed in a mean of 34.2 min. It is expected that the testing time could be shortened by becoming more familiar with the administration of the tests.
As for internal consistency, Cronbach's alpha was 0.77, which is in the acceptable range for internal consistency (>0.60).
The Pearson correlations calculated from the present data and repeated-measure anova suggest that the BACS-J primary measures had good test–retest reliability as shown in the original version. Only the Digit Sequencing Test had a significant practice effect, and this was also shown in the original BACS Study. Because the potential effect of practice was maximized by testing twice within 5 days,8 it is highly recommended that when assessing change over time, such as in clinical trials, subjects are tested twice with more than 1 week between assessments to minimize any effects of practice.
The comparisons of the subtests for different versions suggested that different versions of the Verbal Memory Test are highly recommended rather than applying the same version, because the practice effect across different versions was very small compared to the practice effect using the same version. Thus, the use of alternate forms for the Verbal Memory Test might facilitate the assessment of changes in verbal memory abilities that are independent from learning the words in a previous administration.
Regarding verbal memory, the lists proved to be very similar in difficulty and highly reliable. The practice effect across these versions was very small compared to the practice effect using the same version.
Regarding the Tower of London Test, the Pearson correlation showed that the test with an alternate form had poor test–retest reliability. Meanwhile, in the original version, the reliability of the Tower of London Test was high, even when different versions were administered on consecutive test sessions. Because the difference between the two versions is hard to explain by the difference in languages, the lack of a significant correlation in the Tower of London Test (A/B) in the present study may have been due to the small number of samples. Further evaluation with a larger sample is needed.
The composite score derived from each BACS-J primary measure was strongly correlated between two assessments, and showed a significant practice effect, as was noted in the original BACS Study. The practice effect may be explained by the fact that the time interval between the two assessments, that is within 1 day, was too short. When different versions of the test were administered, the composite score did not show a statistically significant practice effect, and had very good test–retest reliability. These data suggest that using an alternate form is recommended to reduce practice effect.
Pearson correlations between the BACS-J measures and the standard NTB showed that all BACS-J primary measures except the Symbol Coding Task were significantly correlated with relevant standard neurocognitive tests. Also, the BACS-J composite score was significantly correlated with all standard neurocognitive tests except the CPT. Processing speed tasks such as the Symbol Coding Task are well-known neurocognitive tests, and similar versions are included in some NTB, such as WAIS-R and Battery for the Assessment of Neuropsychological Status.22 In addition, the CPT was correlated with none of the standard neurocognitive tests in the present study. Based on these facts, the CPT data in the present study may have been confounded by, for example, technically inaccurate administration (e.g. some subjects were not familiar with computers), while the validity of the Symbol Coding Task could not be denied. Another reason for the lack of a correlation between the Symbol Coding Task and CPT might be that these two tests assess slightly different aspects of cognitive functions: the Symbol Coding Task assesses processing speed/executive function rather than attention, and the CPT assesses vigilance/sustained attention.
Data from the factor analyses and correlation analyses suggest that while there is a single factor of cognitive performance that can be derived from the BACS-J raw scores, indicated by the high correlation between each individual measure of the BACS-J and the composite score, there are two other relatively dissociable cognitive domains of performance measured by the BACS-J. A three-factor solution was also reported in the original BACS study,8 that is, motor speed and general cognitive functions load on the first factor, the memory and working memory measures load on the second factor, and executive function loads on the third factor. However, in the present study executive function and verbal memory loaded on the first factor, which accounted for the largest amount of variance. It is possible that the differences in clinical backgrounds or the sample size account for the different results between the previous and present studies. To test this hypothesis, further cross-national evaluation with a larger sample is needed.
We found some significant correlations between both positive and negative symptoms versus the BACS-J primary measures. This is consistent with the previous observation that psychiatric symptoms, particularly negative symptoms, are associated with distinct neuropsychological deficits.23
The definitive validation of the BACS-J will require further study. First, further comparisons need to be made of the two versions of the Verbal Memory and Tower of London Test, because we could not exactly assess the comparability of the two test forms for these subtests in the present study, due to the expected practice effects. Second, the BACS-J will need to be assessed in various populations of patients with schizophrenia. It is also important to determine whether the BACS-J is sensitive to cognitive changes during clinical trials. Finally, the small sample size was a major limitation of the present study, and further work on a larger sample is required.
The present study indicates that the BACS-J is a promising tool for assessing the major constructs of cognition that have been found to be most impaired and strongly correlated with outcome in patients with schizophrenia.
This work was supported in part by a grant from Eli Lilly Japan, Research Group for Schizophrenia (2005), and a Grant-in-Aid for Scientific Research from Japan Society for the Promotion of Science (No. 19591345). Drs Sumiko Shibuya-Tayoshi, Shu-ichi Ueno, Ken Yamauchi, Yumiko Izaki, Shin-Ya Tayoshi, Jun-ichi Iga, and Sawako Kinouchi helped to recruit patients. Neurocognitive data were collected by Hajime Momose.
- 9American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, 4th edn (DSM-IV). American Psychiatric Association, Washington, DC, 1994.
- 14US Department of Labour. General Aptitude Test Battery, Manpower Administration. Intran Corporation, Minneapolis, MN, 1982.
- 15Conners' Continuous Performance Test. Multi-Health Systems, Toronto, 1995..
- 16Wisconsin Card Sorting Test Manual. Psychological Resources, Odessa, FL, 1993., , , , .
- 17Manual for the Wechsler Adult Intelligence Scale-Revised. Psychological Corporation, New York, 1981..
- 20ECDEU. Assessment Manual for Psychopharmacology (Publication ADM 76-338). US Department of Health, Education, and Welfare, Washington, DC, 1976.(ed.).