Fax: (011) 49 89-70957582
Cognitive function during neoadjuvant chemotherapy for breast cancer
Results of a prospective, multicenter, longitudinal study
Article first published online: 9 MAR 2007
Copyright © 2007 American Cancer Society
Volume 109, Issue 9, pages 1905–1913, 1 May 2007
How to Cite
Hermelink, K., Untch, M., Lux, M. P., Kreienberg, R., Beck, T., Bauerfeind, I. and Münzel, K. (2007), Cognitive function during neoadjuvant chemotherapy for breast cancer. Cancer, 109: 1905–1913. doi: 10.1002/cncr.22610
Fax: (011) 49 89-70957582
M.U. and R.K. receive research funding and honoraria from Bristol Myers Squibb and Amgen.
- Issue published online: 18 APR 2007
- Article first published online: 9 MAR 2007
- Manuscript Accepted: 23 JAN 2007
- Manuscript Received: 13 NOV 2006
- Amgen Ltd
- chemotherapy, adjuvant/adverse effects;
- breast neoplasms;
- cognition disorders;
- menopause, premature;
- psychology, medical
It is believed widely that chemotherapy-induced cognitive impairment occurs in a subgroup of patients with breast cancer. However, recent reports have provided no evidence that chemotherapy affects cognition. In this study, the authors questioned whether cognitive compromise in patients with breast cancer is attributable to chemotherapy. In addition, the effects of therapy-induced menopause and of the erythropoiesis-stimulating factor darbepoetin α on cognitive performance were assessed.
A battery of neuropsychological tests was used to assess cognitive performance in 101 patients with breast cancer before neoadjuvant chemotherapy (T1) and toward the end of neoadjuvant chemotherapy (T2) with combined epirubicin, paclitaxel, and cyclophosphamide with concomitant darbepoetin α. Repeated-measures multiple analyses of variance and a reliable-change approach were used for statistical analyses.
At T1, the group means ranged below the test norms in 5 of 12 cognitive tests. At T2, multiple analyses of variance (MANOVA) indicated a significant overall improvement in the test results (P < .001). After correcting for practice effects, cognitive decline predominated in 27% of patients, whereas improvement predominated in 28% of patients. Cognitive performance was not related significantly to self-reported cognitive problems, anxiety and depression, menopause, or darbepoetin α administration.
Even before chemotherapy, a subgroup of patients with breast cancer showed cognitive compromise that was unrelated to anxiety or depression. During chemotherapy, cognitive function remained stable in most patients, improved in a subgroup, and deteriorated in another subgroup. The deterioration may have been caused by side effects of chemotherapy, but it also may have been related to currently unidentified factors that cause prechemotherapy cognitive compromise. Therapy-induced menopause and darbepoetin α did not appear to influence cognition. Cancer 2007. © 2007 American Cancer Society.
Many patients with breast cancer report chemotherapy-associated cognitive compromise. In 1995, in the first of a series of cross-sectional studies with early breast cancer patients, cognitive impairment was observed after cytostatic treatment in 75% of patients.1 Although the results of subsequent cross-sectional trials assessing cognitive function during or after chemotherapy2–7 were less dramatic, all of them reported substantial cognitive impairment rates of 16% to 50%, suggesting detrimental cytostatic side effects on cognitive function in a subgroup of patients with breast cancer. The first 3 reports that were published on longitudinal studies provided further support for this hypothesis.8–10 Thus, the existence of chemo brain has become almost universally accepted,11 although many details of the concept are controversial or obscure. The reported rates of cognitively impaired patients vary widely, reflecting different definitions of impairment.11 Data regarding affected cognitive domains and the duration of cognitive disturbance are far from being congruent.5, 12, 13 It remains unknown which cytostatic agents are responsible for cognitive impairment, which characteristics make patients vulnerable,14 and which biologic mechanisms are involved.15
Some doubts regarding the evidence for chemotherapy-induced cognitive impairment have been raised by 3 recently published studies that failed to confirm the adverse effects of chemotherapy on cognitive function.16–18 Meta-analyses also consistently have stated that it depends on the study design whether chemotherapy-related cognitive impairment is observed, with more appropriate designs resulting in less clear evidence or no evidence of chemotherapy-induced cognitive impairment.19–21
Currently, we are conducting a multicenter, prospective study entitled Cognitive Impairment in Therapy of Breast Cancer (COGITO). Cognitive function in early breast cancer patients is assessed before the start of neoadjuvant chemotherapy, toward the end of chemotherapy, and 1 year after the baseline assessment. The study is ongoing; this report describes the findings from the first 2 assessments.
The objective of the study was to assess the course of cognitive function before and after exposure to cytostatic treatment and, thus, to test the hypothesis that the cognitive dysfunction observed in patients with breast cancer is induced by chemotherapy. Our secondary objectives were to determine the impact of therapy-induced menopause and the effect of concomitant medication with darbepoetin α on cognition. Darbepoetin α is a hyperglycosylated, human-recombinant erythropoietin, and favorable effects of erythropoietin on cognitive performance during adjuvant chemotherapy have been reported.22
MATERIALS AND METHODS
All patients who had been included in the Preoperative Epirubicin Paclitaxel Aranesp (PREPARE) trial between July 2003 and March 2005 at 5 gynecologic or oncologic centers in Germany were eligible for participation in COGITO if they were fluent in German. Participants in the PREPARE trial are between ages ≥18 years and ≤65 years, have breast carcinomas that measure ≥2 cm or inflammatory lesions, and have no evidence of metastases. In the PREPARE trial, patients are assigned randomly to receive standard chemotherapy (epirubicin 90 mg/m2 and cyclophosphamide 600 mg/m2 on Day 1 every 21 days for 4 cycles; paclitaxel 175 mg/m2 on Day 1 every 21 days for 4 cycles) or dose-intensified therapy (epirubicin 150 mg/m2 on Day 1 every 14 days for 3 cycles; paclitaxel 225 mg/m2 on Day 1 every 14 days for 3 cycles; combined cyclophosphamide, methotrexate, fluorouracil on Days 1 and 8 every 28 days for 3 cycles), and are subrandomized to receive concomitant anemia prophylaxis with darbepoetin α (Aranesp; Amgen Ltd., Munich, Germany) (1 × 4.5 μg/kg every 2 weeks; medication is suspended if hemoglobin exceeds 14 g/dL). Patients who terminated chemotherapy prematurely continued to be included in COGITO if they received ≥4 cycles of chemotherapy.
For the sake of representativeness, we did not establish rigid exclusion criteria. Instead, we performed analyses twice—including all patients and including only patients who were not affected by potential confounders. Features that were considered to be confounding factors were daily alcohol intake of >2 drinks, use of medication known to affect cognition, and a history of a central nervous system disorder. The nonnative speakers who participate in the study, although they are fluent in German, still may be handicapped slightly on the language-related tests; thus, not being a native speaker of German also was regarded as a confounder.
Written informed consent was provided by all patients. The study was approved by the ethics committees at all of the universities involved.
The first assessment (T1; baseline) was conducted before the start of preoperative chemotherapy and 1 week after diagnosis at the earliest. The second assessment (T2) was scheduled between the penultimate and the final chemotherapy session—ie, approximately 5 months after the baseline assessment.
A battery of cognitive tests was compiled that was sensitive for detecting dysfunction in a wide range of cognitive domains. Twelve standardized and widely used tests were included: Logical Memory I and II (verbal memory) from the Wechsler Memory Scale-Revised (WMS-R),23 Digit Span Forward (verbal short-term memory, attention) and Digit Span Backward (verbal working memory) from the WMS-R, Digit Symbol (psychomotor function, information processing speed) from the Wechsler Adult Intelligence Scale-Revised (WAIS-R),24 the D2 Test25 (concentration), Trail Making Test Part A (TMT-A)26–28 (psychomotor function, selective attention29), Trail Making Test Part B (TMT-B)26–28 (psychomotor function, divided attention, cognitive flexibility29), and the Regensburg Word Fluency Test (RWT)30 with the subtests Lexical Search, Semantic Search, Lexical Search with Change of Category, and Semantic Search with Change of Category (executive function). With the word-fluency tests, parallel forms were used to minimize practice effects.
The MWT-B,31 a validated, language-related test, was used to estimate premorbid intelligence. Testees are required to find correct words among nonsense neologisms. The test was taken once at T2.
Patients' self-reports of cognitive functioning were covered by the Fragebogen erlebter Defizite der Aufmerksamkeit [Questionnaire of Experienced Attention Deficits (FEDA)],32 a validated questionnaire of attention deficits experienced in daily life, and the Cognitive Function Scale of the European Organization for Research and Treatment of Cancer (EORTC) Quality-of-life Questionnaire C30, version 3.0.33, 34 The Hospital Anxiety and Depression Scale (HADS)35 was used to measure current levels of depression and anxiety symptoms.
All of the assessments were conducted by 1 of 2 university-qualified psychologists, either in patients' homes or in a hospital. Tests and questionnaires were administered in a fixed order.
All statistical analyses were carried out using the Statistical Package for the Social Sciences (SPSS), version 13 or 14. Statistical tests were 2-sided. Significance was calculated with a 5% probability of error (except for reliable-change index calculations; see below).
For comparisons with test norms, either 1-sample t tests or binomial tests, as appropriate, were performed. Test norms for TMT-A and TMT-B were taken from a United States normative study.28 For all other tests, we used norms that were representative of the German population.23–25, 30 All norms were stratified for age.
For group comparisons of the T1 and T2 results of cognitive tests, a repeated-measures MANOVA was carried out, thus avoiding inflation of α error by multiple testing. To determine whether differences between an individual patient's T2 and T1 test results were caused by measurement error or reflected a change in cognitive ability, we used the reliable-change index (RCI) developed by Jacobson and Truax.36 Setting the probability of error at 10%, we calculated the RCI for each pair of T1 and T2 test results, using the standard deviation of the T1 test results, and published test-retest reliabilities.23, 24, 27, 37
Practice effects were to be expected for Logical Memory I and II, TMT-A, TMT-B, and Digit Symbol,38, 39 but not for Digit Spans Forward and Backward.23, 39 To correct the RCI for practice effects, we used practice effects observed in healthy control groups in normative studies (Table 1).23, 40, 41 The D2 Test had to be excluded from this analysis because of a lack of data about the practice effects. In addition, the word-fluency tests were excepted from RCI analyses; because, with these tests, parallel forms were applied that could not be compared directly.
|Test||Correction||Based on source||Group||Test-retest interval|
|Logical Memory I||−3.15*||Test manual†||Normative sample, representative (age, education) for German population, N = 40||6 mo|
|Logical Memory II||−4.37*||Same as Logical Memory I|
|Digit Span Forward||No correction||Same as Logical Memory I|
|Digit Span Backward||No correction||Same as Logical Memory I|
|Digit Symbol||−2.88*||Study on test-retest reliability of diverse cognitive tests‡||Control group, N = 115 (mean age ± SD, 48.9 ± 19.3 y)||21 d|
|Trail Making Test Part A||4.91§||Study on test-retest reliability and practice effects‖||Control group, N = 63 (mean age ± SD, 56.2 ± SD 11.0 y)||6 mo|
|Trail Making Test Part B||10.06§||Same as Trail Making Test Part A|
Spearman rank correlations were calculated for each patient's number of deteriorated test results corrected for practice effects with self-reported cognitive function, anxiety, and depression. Repeated-measures MANOVAs were performed to assess effects of darbepoetin α and of therapy-induced menopause.
The participation criteria were met by 160 patients. Of these, 26 patients (16%) declined to participate. For another 21 patients (13%), it was not possible to arrange an assessment session before the start of therapy. Of 113 patients who were recruited into the study, 4 patients were excluded as screening failures, and 109 patients completed the T1 assessment. At T2, 2 patients declined further participation, and 6 patients had to be excluded because they discontinued chemotherapy before completing the fourth cycle (attrition rate, 7.3%). Four of the remaining 101 patients had received fewer chemotherapy cycles than scheduled.
At T1, 18 patients had potential confounders (see Materials and Methods, above), including alcohol intake in 2 patients, central nervous system disorders in 6 patients, medications in 7 patients, and 3 patients who were nonnative German-language speakers. At T2, another 6 patients had started medications that were considered confounding (mostly antidepressants), and 1 patient had acquired an unclear neurologic problem (symptoms of paralysis).
With regard to premorbid intelligence and educational levels, the sample was in the above-average range (mean intelligence quotient, 108), and 34% had ≥12 years of education (compared with 22% in the general German population matched for age). Table 2 lists the demographic data for the sample.
|Characteristic||Mean ± SD||No. of patients (%)|
|Age (N = 109), y||48.6 ± 9.7|
|Years of education (N = 109)|
|Premorbid IQ (N = 101)*||107.7 ± 14.7|
|Menopausal status (N = 109)†|
Before the Start of Therapy—T1
The group mean scores for 5 cognitive tests showed significantly poorer scores than the test norms, whereas the group mean score for 1 test was significantly better than the norm (Table 3). If 18 patients with confounders were excluded, then the scores for 4 tests remained significantly poorer than the norms (no longer including the RWT Semantic Search), whereas 3 tests were significantly better (in addition, the Digit Span Forward and the RWT Semantic Search with Change of Category).
|Test||Mean (SD)||Z-score/PR*||Compared with test norms||P||No.|
|Logical Memory 1|
|Logical Memory 2|
|Digit Span Forward|
|Digit Span Backward|
|Trail Making Test Part A|
|Trail Making Test Part B|
|RWT Lexical Search|
|RWT Semantic Search|
|RWT Lexical Search With Change of Category|
|RWT Semantic Search With Change of Category|
On an individual level, 33 patients (31%) had ≥2 test results in the lower 5% range, and 29 patients (27%) had ≥2 test results in the upper 5% range (Fig. 1). The percentages changed marginally (30% and 27%, respectively) if patients with confounders were excluded. The number of test results in the lower 5% range was correlated significantly with premorbid intelligence (Spearman ρ = −0.34; P = .001), and education (Spearman ρ = −0.39; P < .001), but not with anxiety, depression, or self-reported cognitive function.
If the method of determining cognitive impairment used in the frequently cited study by Wieneke and Dienst had been applied,1 then we would have had to classify 61 patients (56%) with at least mild cognitive impairment (>1 test result ≤1 standard deviation [SD]), with 35 patients (32%) demonstrating moderate cognitive impairment (in addition, ≥1 test result ≤2 SD). If patients with confounders present at T1 were excluded, then the percentages changed marginally (31% moderately impaired).
Toward the End of Chemotherapy—T2
The mean interval between the T1 and the T2 assessments was 20.9 weeks (SD, 1.3 weeks). With the exception of 2 tests, the group mean scores were within or above the test norms at T2 (Table 3).
Repeated-measures MANOVA of 93 data sets that were complete for T1 and T2 indicated a significant overall improvement [P < .001; effect size (partial Eta squared) η2p = 0.59]. Improvement reached significance in 6 tests (Table 4). Deterioration was observed in only 1 test (RWT Semantic Search with Change of Category) and reached significance. With that test, and in all RWT subtests, parallel forms were used at T1 and T2 to minimize practice effects.
|Logical Memory 1||5.772||.018†||.059|
|Logical Memory 2||25.656||.000†||.218|
|Digit Span Forward||0.718||.399||.008|
|Digit Span Backward||4.255||.042†||.044|
|Trail Making Test Part A||2.840||.095||.030|
|Trail Making Test Part B||2.156||.145||.023|
|RWT Lexical Search||0.720||.398||.008|
|RWT Semantic Search||6.506||.012†||.066|
|RWT Lexical Search with Change of Category||3.908||.051||.041|
|RWT Semantic Search with Change of Category||6.075||.016‡||.062|
If patients who had confounders were excluded, the overall improvement was more pronounced (P < .001; η = 0.75). There was no consistent pattern of affected cognitive domains. The only overall deterioration was observed in 1 of 4 word-fluency tests. However, the other word-fluency tests showed either improvement or no change.
Forty-eight patients received standard chemotherapy, and 53 patients received dose-intensified chemotherapy. The results from MANOVA showed no effects associated with the treatment arm (interaction time*treatment arm: P = .64).
In the individual RCI analyses, 100 patients with ≤1 missing test result were included. Deterioration predominated in a subset of 27 patients (27%), whereas improvement predominated in 28 patients (28%) (Fig. 2). Excluding 24 patients who had confounders at T1, at T2, or at both T1 and T2 resulted in a lower percentage of predominant deterioration (22%) and a higher percentage of predominant improvement (32%).
Neither declines nor improvements were related significantly to the number of test results in the lower 5% range at T1. However, the number of test results in the upper 5% range at T1 was correlated positively with declines and negatively with further improvements; ie, patients who had performed particularly well at T1 were more likely to show decline and were unlikely to improve their performance further. Age and intelligence were not associated significantly with the number of deteriorated and improved test results.
A subgroup of 5 patients (5%) showed deterioration on 3 tests (n = 4 patients) or 4 tests (n = 1 patient). Three of those patients were using antidepressant medications at T2. The remaining 2 patients had performed very well at T1 and still performed well at T2 (3 and 2 test results in the upper 5% range, respectively; no test results in the lower 5% range), suggesting regression to the mean.
Both measures of subjective cognitive function showed a significant increase in cognitive problems at T2 (FEDA: P < .001; EORTC Cognitive Function Scale: P < .001). Correlations with the number of deteriorated test results were small and nonsignificant (FEDA: Spearman ρ = −0.13; P = .20; EORTC-Cognitive Function Scale: Spearman ρ = 0.11; P = .30). In addition, changes in self-reported cognitive problems were not associated significantly with the number of deteriorated test results.
Anxiety and depression were not correlated significantly with the number of deteriorated or improved test results, whereas all correlations of anxiety and depression with self-reported cognitive complaints were significant and of moderate size (eg, HADS-A and FEDA: Spearman ρ = 0.41; P < .001; HADS-D and FEDA: Spearman ρ = 0.58; P < .001).
Impact of Therapy-induced Menopause and Darbepoetin α
Among 52 patients who were premenopausal at T1, 3 still were premenopausal at T2, menses had ceased in 45 patients, the menopausal status of 1 patient was indeterminable, and 3 patients had terminated participation in the study. MANOVA with age and intelligence as covariates did not identify any significant effect of therapy-induced menopause (interaction time*induced menopause: P = .29; η = 0.16).
There were no differences between patients who received (n = 51 patients) or did not receive (n = 50 patients) concomitant darbepoetin α therapy with regard to age, intelligence, or education. MANOVA did not identify any significant influence of darbepoetin α on changes in cognitive performance (interaction of time*darbepoetin α: P = .78; η = 0.09).
Even before the start of chemotherapy, the performances of the patients—who had above-average educational and intelligence levels—ranged below the test norms on 5 of 12 tests. Approximately one-third of patients showed cognitive compromise even before treatment. This impairment rate is similar to that reported after chemotherapy in cross-sectional studies. Prechemotherapy cognitive compromise has also been observed in other studies.8, 42, 43
Toward the completion of chemotherapy, approximately a quarter of patients showed a decline, whereas another quarter demonstrated improvement of cognitive function. Practice effects were taken into account. There was no consistent pattern of affected cognitive domains.
Very few patients exhibited a pronounced cognitive decline. All of them were either taking antidepressant medication known to affect cognitive function or had performed extraordinarily well at T1 and still had above-average performance at T2, which may be interpreted as a regression to the mean.
The patients' reports of cognitive problems increased during chemotherapy, and those problems were not related significantly to cognitive test results but, rather, to anxiety and depression. This observation is in accordance with the findings of virtually all studies that have reported on the relations between cognitive performance and cognitive complaints.3, 5, 7, 18, 44 Neither chemotherapy-induced menopause nor concomitant darbepoetin α administration had a significant influence on cognitive change.
The current study was limited by the absence of a control group. Test norms were used to rate the baseline test results, and cognitive performances during chemotherapy were compared only with the patients' own baseline performance. Practice effects were estimated by using effects observed in normative samples that were matched imperfectly to our patient group. One advantage of this study was the homogeneity of treatment. In addition, the attrition rate was very low, and we controlled for confounding factors—even those that emerged after the start of study participation, such as psychoactive medication that commenced during chemotherapy.
Our findings cast some additional doubt on the evidence for chemotherapy-induced cognitive impairment. In cross-sectional studies that used test norms1 or control groups consisting of healthy participants,4, 6, 13 cognitive compromise in the patients may have originated before chemotherapy, like it did in the current study. However, several cross-sectional studies have used control groups consisting of patients with breast cancer who were treated without chemotherapy. The results were mixed: Poorer performances were reported among patients who received chemotherapy,2, 3, 5, 7 although others reported no differences16, 17; the latter findings also were supported by a meta-analysis.21
Two longitudinal studies that provided evidence of chemotherapy effects on cognitive function were limited by their rather small sample sizes (Wefel et al.: N = 18 patients; Bender et al.: N = 36 patients at the T2 assessment). A recently published, large-scale, longitudinal trial by Jenkins et al.18 did not identify any significant differences in cognitive changes between chemotherapy patients, nonchemotherapy patients, and healthy control individuals. The patients were heterogeneous with regard to the chemotherapy regimens administered, with most patients having received only relatively low-dose combined 5-fluorouracil, epirubicin, and cyclophosphamide treatment. The authors considered the regimen a possible reason why cognitive impairment did not become apparent. However, in the current study, all of our patients received quite high doses of chemotherapy that contained anthracycline and taxane, and the rate of decline during chemotherapy still did not exceed the rate of simultaneous improvement.
In contrast to Jenkins et al., we observed cognitive compromise before the start of chemotherapy. This discrepancy between the findings may have been caused by differences in the assessment points: Whereas the baseline assessment took place several weeks after surgery (mean, 41 days) in the study by Jenkins et al., we already had assessed cognitive function after diagnosis—ie, at a time that is known as especially stressful. Cognitive compromise at this particular time may indicate that the still unidentified factor affecting cognitive function before chemotherapy is related to stress-response symptoms. Without necessarily coinciding with symptoms of depression, stress-response symptoms may interfere with performance during cognitive testing.
Conceivably, persistent stress-response symptoms also may have caused the cognitive compromise that we observed during and after chemotherapy in several studies. Posttraumatic symptoms have been identified in subgroups of breast cancer patients even long after the completion of treatment.45 There is some evidence that posttraumatic stress disorder may be associated with memory and concentration problems.46 Further research is needed to determine whether the phenomenon of cognitive compromise associated with breast cancer is dubbed chemo brain correctly or, instead, should be renamed crisis brain.
The results from this study do not corroborate the hypothesis that chemotherapy is the cause of cognitive dysfunction in patients with breast cancer. The possibility cannot be ruled out that the cognitive deterioration evident in a subgroup of patients during chemotherapy is attributable to the side effects of chemotherapy. However, in the light of the apparent cognitive compromise before chemotherapy and improvements observed in a subgroup of patients during chemotherapy, an alternative explanation seems more plausible—namely, that as yet unidentified factors affect cognition in patients with breast cancer even before chemotherapy and affect it further during chemotherapy in a subgroup of patients, when another subgroup starts to recover.
We are grateful to Franziska Neufeld for conducting many of the assessments, and to the patients, physicians, and research nurses who participated in the Cognitive Impairment in Therapy of Breast Cancer study.
- 23WMS-R. Wechsler Gedachtnistest-Revidierte Fassung [in German]. 2nd ed. Bern: Huber, 2004., , , , , .
- 24HAWIE-R. Hamburg-Wechsler Intelligen Test for Erwachsene, revision 1991, 2nd ed [in German]. Bern: Huber; 1994..
- 25Test D2. Aufmerksamkeits-Belastungs-Test, 9th ed [in German]. Gottingen: Hogrefe; 2002..
- 26Trail Making Test. ed. 2. Tucson, Ariz: Reitan Neuropsychology Laboratory; 1992..
- 29Neuropsychological Assessment. 4th ed. New York, NY: Oxford University Press; 2004., , .
- 30RWT. Regensburger Wortflussigkeits-Test [in German]. Gottingen: Hogrefe; 2000., , .
- 31Mehrfachwahl-Wortschatz-Intelligenztest MWT-B. Balingen: Spitta; 1999..
- 32Differentielle Validitat des Fragebogen erlebter Defizite der Aufmerksamkeit (FEDA): konkurrente Validierungsergebnisse bei schizophrenen und depressiven Patienten [in German]. Z Klin Psychol Psychiatr Psychother. 1998; 46: 152–165., , .
- 34The EORTC QLQ-C30 Scoring Manual. 3rd ed. Brussels: European Organization for Research and Treatment of Cancer; 2001., , , et al.
- 35HADS-D. Hospital Anxiety and Depression Scale. German Version, 1st ed. Berne: Huber; 1995., ,
- 38Practitioner's Guide to Evaluating Change With Neuropsychological Assessment Instruments. New York, NY: Kluwer Academic/Plenum Publishers; 2000., , .
- 39Practitioner's Guide to Evaluating Change With Intellectual Assessment Instruments. New York, NY: Kluwer Academic/Plenum Publishers; 2000., , .
- 40Test-retest reliability of computerized, everyday memory measures and traditional memory tests. Clin Neuropsychol. 1992; 6: 276–286., , .