Cognitive dysfunction and neuropsychiatric disturbance are common in systemic lupus erythematosus (SLE). This study addressed the ability of the Automated Neuropsychological Assessment Metrics (ANAM), a computerized cognitive testing battery consisting of cognitive subtests, a sleepiness rating scale, and a mood scale, to predict neuropsychological status in patients with SLE.
Sixty individuals with SLE and no overt neuropsychiatric symptoms were administered ANAM to determine its validity as a screening measure of cognitive dysfunction and emotional distress in SLE.
Performance on ANAM was compared with results of a consecutively administered, 2-hour battery of traditional neuropsychological tests and the Beck Depression Inventory II (BDI-II). Individual ANAM cognitive test scores were significantly correlated with most neuropsychological tests, particularly those measuring psychomotor processing speed and executive functioning. Using logistic regression, ANAM cognitive subtests successfully predicted individuals with SLE who had probable versus no impairment after controlling for premorbid levels of cognitive ability. Sensitivity of group classification was 76.2% and specificity was 82.8%, with 80% correct classification overall. ANAM's ability to predict neuropsychological functioning remained even after controlling for subjective reports of depressed mood and current sleepiness. Further, the ANAM mood scale was significantly correlated with the BDI-II (r = 0.67, P < 0.001), indicating its potential future use as a screening tool for emotional distress.
ANAM shows promise as a time- and cost-efficient tool for screening and monitoring cognitive and emotional functioning in SLE, and can indicate when a more thorough neuropsychological investigation is warranted.
Systemic lupus erythematosus (SLE) is a chronic, relapsing/remitting, multisystem, autoimmune disease. Its effects on the central nervous system can manifest in many ways, with psychiatric disturbance, headache, and cognitive dysfunction being the most commonly reported symptoms (1–3). In fact, the preponderance with which patients with SLE demonstrate neuropsychiatric symptoms has led the American College of Rheumatology (ACR) to provide case definitions for this subset of patients (4).
The prevalence of cognitive disturbance in SLE is high, ranging from 21% to 66% across studies (5–9), and can occur in the absence of active systemic disease and major neurologic or psychiatric events (5, 7, 10–12). There is no specific profile of cognitive dysfunction in patients with SLE, but commonly reported areas of impairment include verbal and nonverbal learning and memory, verbal fluency, visuospatial constructional skills, psychomotor speed, and executive functioning (6, 7, 9, 13–18). The course of cognitive dysfunction in lupus has been described as fluctuating, with deficits shown to wax and wane over time, but, at least for a subset of patients, cognitive impairment may be progressive in nature (7, 11, 15, 19). The precise causes of cognitive dysfunction in patients with SLE remain unclear, although they are likely multifactorial, including direct effects of disease mechanisms on the central nervous system and indirect effects through fatigue or psychiatric disturbances.
Assessment of cognitive function in SLE has traditionally been conducted with batteries of standard neuropsychological tests (11, 13), and the ACR has recommended the use of a standard battery of neuropsychological tests believed to assess core areas of impairment seen in SLE (20). Conventional neuropsychological assessment batteries of tests, such as the one recommended by the ACR, are optimal because they include assessment of all major cognitive domains and provide detailed information about the magnitude and nature of cognitive impairment. However, there are many factors that prevent widespread use of conventional neuropsychological testing in the outpatient medical setting. For instance, such batteries require specialized training to administer, are time consuming, and often are cost prohibitive. Moreover, conventional tests are sensitive to patient fatigue (21), have large practice effects upon repeated testing (22), and are often limited in their ability to detect very subtle impairment compared with computerized assessment (23). A cost- and time-efficient screening tool would allow for routine evaluations of cognitive impairment over extended periods and would indicate when a more thorough neuropsychological evaluation was needed.
The Automated Neuropsychological Assessment Metrics (ANAM) is a 30-minute computerized cognitive battery of tests that can be self administered and has been used successfully to assess subtle cognitive dysfunction in other medical conditions with neurobehavioral sequelae (24–30). Specifically, ANAM has been used to detect the detrimental effects of migraine headache on cognitive functioning and the return to preheadache levels following migraine treatment (25, 26). ANAM is a good predictor of neuropsychological functioning in patients with relapsing, remitting multiple sclerosis (30). ANAM has also been used to track subtle cognitive dysfunction and subsequent cognitive recovery following sports concussion (29, 31). Factor analyses have demonstrated that both ANAM and traditional neuropsychological tests measure similar underlying cognitive domains, including processing speed, working memory, and resistance to interference (32, 33). Therefore, ANAM has the potential to be used both as a screening tool to help physicians detect subtle cognitive dysfunction and as an evaluative tool in repeated assessments to monitor cognitive change in patients with SLE.
Previous studies using ANAM in patients with SLE have demonstrated that ANAM performance is less affected by education, ethnicity, and English proficiency than traditional neuropsychological tests (1, 34). Holliday and colleagues (34) further demonstrated that ANAM performance was significantly correlated with many scores from a traditional neuropsychological battery of tests and that, in a regression model, selected ANAM scores together with age accounted for ∼61% of the variance in traditional neuropsychological test scores. The current study sought to expand upon these earlier studies by replicating the association between ANAM and traditional neuropsychological tests, and by specifically exploring ANAM's ability to predict which individuals with SLE perform in an impaired range on neuropsychological testing.
Depression and sleepiness, 2 noncognitive factors that can affect performance on cognitive tests, are commonly reported symptoms in patients with SLE (35–38). It is plausible that these factors may impact performance on cognitive testing, particularly on measures of psychomotor speed, attention, and working memory (21, 39, 40). However, the extent to which these factors affect performance on ANAM is not well understood. Therefore, a secondary aim of the current study was to examine relationships between self ratings of mood and sleepiness and ANAM performance, and to determine whether these noncognitive factors confound ANAM's ability to measure cognitive functioning.
PATIENTS AND METHODS
This study was conducted as part of a larger study of biomarkers of cognitive dysfunction in patients with SLE. Sixty individuals with SLE participated in the study based on the following inclusion criteria: age ≥18 years, willing and able to provide informed consent, and fulfillment of the 1997 updated ACR criteria for SLE (41). Based on an interview with the study participants, individuals were not included in the study if they reported any history of neurologic disease, including history of head injury resulting in loss of consciousness, stroke, seizures, or toxic exposure. Further, individuals with a history of clinically documented transient ischemic attacks within 6 months or individuals currently taking anticonvulsant medication were excluded from the study. Participants were prospectively recruited from the Rheumatology Clinic of the National Institutes of Health (NIH) Clinical Center or the National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIAMS) Satellite Clinics in Washington, DC. Consecutive patients meeting the inclusion/exclusion criteria were recruited. All patients signed an informed consent form that was approved by the Institutional Review Board of the National Institute of Diabetes and Digestive and Kidney Diseases/NIAMS, NIH.
All participants were administered a 2-hour battery of traditional neuropsychological tests followed by administration of the ANAM battery. Individual ANAM subtests are listed and described in Table 1 in the order that they were presented to the participants. Neuropsychological tests that were chosen were similar to and in many cases the same as those recommended by the ACR case definitions (4), and were administered in the following order: the reading subtest of the Wide Range Achievement Test, Third Edition (WRAT-3) (42), an estimate of premorbid intellectual functioning (43); the California Verbal Learning Test (CVLT) (44), a test of verbal learning and memory; the Rey Complex Figure Test (RCFT) (45), a test of visual spatial construction and visual memory; the Trail Making Test, Part B (46), a test of sequencing and executive functioning; the Stroop Color and Word Test (47), a test of information processing speed and executive functioning; the Digit Symbol subtest of the Wechsler Adult Intelligence Scale, Third Edition (WAIS-III) (48), a test of information processing speed; the Controlled Oral Word Association Test (COWA) (46), a test of verbal fluency and executive functioning; and the Grooved Pegboard Test (46), a test of fine motor speed and coordination.
Respond by pressing the mouse button as quickly as possible when a stimulus (*) appears.
Decide if a symbol/digit pairing is consistent with pairings presented in a “code” above.
Determine if 2 designs are the same/different.
Matching to sample
Select which of 2 designs match a target design presented 5 seconds earlier.
Decide whether the solution to a 3-step arithmetic problem (e.g., 5 + 2 – 3 = ?) is greater than or less than 5.
Continuous performance test
Continuously monitor numbers and identify if each number is the same/different from the preceding letter.
Code substitution delayed
Presented 20 minutes after code substitution. Decide if a presented symbol/digit pairing is consistent with the earlier “code.”
Sternberg memory test
Memorize a string of 6 letters and later determine whether individually presented letters were included in the original string.
Self-reported depression symptomatology over the past 2 weeks was assessed with the Beck Depression Inventory, Second Edition (BDI-II) (49). Current level of sleepiness was assessed with a computerized version of the Stanford Sleepiness Scale (50). Current mood was assessed by the ANAM mood scale, which was derived from a paper and pencil adjective checklist developed by Ryman et al (51). The ANAM mood scale consists of a listing of 36 adjectives that are rated on a 10-point scale with respect to how the participant feels at that moment. Six scales are produced from the response set, including depressed mood, happiness, anxiety, energy level, anger, and fatigue.
All tests were administered during 1 or 2 visits to the NIH Clinical Center. Participants were asked not to take medications with sedating properties in the morning of the study visit. All tests were administered according to standard protocol by a trained research nurse.
Each ANAM subtest produces 3 primary scores: mean reaction time; accuracy; and throughput, a score representing the product of speed and accuracy used to account for speed/accuracy tradeoffs. The throughput variable was analyzed for each ANAM subtest, with one exception. Mean reaction time, instead of throughput, was used for the simple reaction time subtest because accuracy was always 100% in this group. Performance on the battery of traditional neuropsychological tests was compared with published norms and transformed to Z scores, as necessary. Performance on traditional neuropsychological tests was then compared with performance on ANAM cognitive subtests using Pearson's product-moment correlations. Next, the ability of ANAM cognitive subtests to predict neuropsychological status was examined using logistic regression techniques. Neuropsychological status was defined as either probable impairment or no impairment according to the following classification system. Z scores from each neuropsychological test were averaged into 5 cognitive domains, as follows: memory (CVLT and RCFT delayed memory scores), attention/executive functioning (Trail Making Test, Stroop Color and Word, COWA, and CVLT semantic clustering scores), visuospatial skills (RCFT copy score), psychomotor speed (WAIS-III Digit Symbol subtest and Stroop Color and Word scores), and fine-motor speed and coordination (Grooved Pegboard scores). Individuals whose average Z score performance fell between −1.0 and −1.99 on 2 cognitive domains or individuals whose average Z score was less than −2.0 on 1 cognitive domain were classified in the probable impairment group. All other individuals were classified in the no impairment group. Additionally, the ability of ANAM cognitive subtests to predict overall neuropsychological functioning (defined by the average Z score of all neuropsychological tests) was examined after covarying self-reported depression and sleepiness. Finally, the association between ANAM mood scales and the BDI-II, a previously validated measure of depressed mood, was examined using Pearson's product-moment correlations.
The sample patients were primarily women, right handed, and college educated. The majority of participants were white, with the remaining individuals classifying themselves as African American, Asian American, or Latino/Hispanic. A breakdown of sample demographics and clinical characteristics is presented in Table 2. This sample had a mean ± SD disease duration of 13.8 ± 10.2 years, and 93% of the participants had Systemic Lupus Erythematosus Disease Activity Index scores <4. Sixty-three percent of individuals were taking prednisone (median dosage 2.5 mg/day, range 0–30 mg/day), and 81% were receiving concurrent immunosuppressive therapy. Please note that a separate article currently submitted for publication (52) examined the association between cognitive functioning and various clinical and biomarkers in this sample. Nine participants were missing from the current analyses, 2 due to computer error and 7 due to an invalid response style on at least 1 ANAM subtest, suggestive of either failure to understand task directions or poor motivation on testing (subtest performance <65% accuracy is rare and has been used to exclude poor effort in other studies using ANAM [26, 29]). Please note that the overall outcome of the following analyses did not change as a result of excluding these participants.
Values are the number (percentage) unless otherwise indicated. BDI-II = Beck Depression Inventory, Second Edition.
Age, mean ± SD years
41.3 ± 12.8
Education, mean ± SD years
14.98 ± 3.06
Correlations between ANAM and traditional neuropsychological tests.
ANAM subtest variables were correlated with raw neuropsychological test scores using Pearson's product-moment correlations. Partial correlations were run with age as a covariate, given the influence of age on ANAM scores (53). ANAM subtests were significantly correlated (P < 0.05) with many of the traditional neuropsychological tests, particularly those assessing executive functioning, information processing speed, and fine-motor speed and coordination. Further, ANAM subtests requiring learning and short-term memory (i.e., code substitution immediate and delayed, matching to sample, and Sternberg memory subtests) were correlated with traditional tests known to measure those domains. Individual r values are displayed in Table 3. In all, we found 58 statistically significant correlations between ANAM cognitive tests and individual traditional neuropsychological tests. Based on the number of correlations run (112), we would expect ∼5.6 to be significant by chance alone using an alpha level of 0.05. Therefore, we have chosen to focus on the overall pattern of significant correlations rather than to interpret any single test by test correlation. Although many correlations were of fairly weak magnitude (i.e., r < 0.35), the pattern of correlations indicates that, overall, ANAM cognitive subtests are associated with traditional neuropsychological testing.
Table 3. Pearson's correlations between individual ANAM cognitive tests and individual neuropsychological tests after controlling for age*
ANAM = Automated Neuropsychological Assessment Metrics; CDD = code substitution delayed; CDS = code substitution; MTG = matching grids; MSP = matching to sample; MTH = mathematical processing; STN = Sternberg memory test; CPT = continuous performance test; SRT = simple reaction time; WRAT-3 = Wide Range Achievement Test, Third Edition; CVLT = California Verbal Learning Test; RCFT = Rey Complex Figure Test; COWA = Controlled Oral Word Association Test; WAIS-III = Wechsler Adult Intelligence Scale, Third Edition; DH = dominant hand; NDH = nondominant hand.
ANAM's ability to predict neuropsychological status.
Although group performance on most neuropsychological test scores fell within the average or low average range, relative difficulties were seen for the CVLT learning and RCFT copy scores, which were in the mildly impaired range with Z scores of −1.2 and −1.3, respectively. A graph illustrating average Z scores for each administered neuropsychological test is presented in Figure 1. The ability of the ANAM cognitive battery of tests to predict neuropsychological status was assessed using logistic regression analyses. In this sample, 46.7% of individuals were classified as having probable cognitive impairment for the purposes of this analysis. The WRAT-3 was forced into the model first as a control for premorbid ability. Then the mean reaction time for the simple reaction time subtest and throughput scores from the remaining ANAM subtests were entered into the model. The model with WRAT-3 alone was statistically significant (χ2 = 6.5, P = 0.01) and correctly classified 74% of individuals. With ANAM subtests included, the overall model remained significant (χ2 = 23.45, P < 0.01) and the inclusion of ANAM subtests improved accuracy of prediction above and beyond that of the WRAT-3 alone (Block χ2 = 16.9, P < 0.05). The model including ANAM subtests correctly predicted the group status of 80% of individuals. Additionally, 24 of 29 individuals in the no impairment group were predicted accurately (specificity 82.8%), and 16 of 21 individuals in the probable impairment group were predicted accurately by ANAM performance (sensitivity 76.2%). Variables in the regression model are shown in Table 4.
Table 4. Summary of logistic regression using individual ANAM cognitive tests to predict neuropsychological status*
Overall model: χ2(8) = 34.77, P < 0.001. See Table 3 for definitions.
No single ANAM subtest emerged as a statistically significant predictor above and beyond the other subtests in this model. This is most likely due to a moderate degree of multicollinearity between ANAM subtests (32% of between-subtest correlation coefficients were >0.38, with a maximum correlation of 0.77). A backward-stepwise regression with WRAT-3 forced into the model revealed that the matching to sample subtest (odds ratio [OR] 0.884, P = 0.07) and matching grids subtest (OR 0.895, P = 0.11) were the best ANAM subtest predictors of neuropsychological status. However, given the multicollinearity noted previously, relative contribution of individual ANAM subtests to prediction of neuropsychological status may be sample specific and should be replicated with future samples.
To better examine the potential confounds of depression, the ability of ANAM cognitive tests to independently predict global neuropsychological test scores was examined using a hierarchical regression model in which BDI-II scores were entered into the model first followed by entry of ANAM cognitive tests as a block. BDI-II scores alone explained only 6% of variance in global neuropsychological functioning, meeting only marginal statistical significance (F[1,55] = 3.7, P < 0.06). ANAM cognitive subtests were entered into the model second and independently explained an additional 44% of the variance (change in F[8,47] = 6.2, P < 0.0001), indicating that ANAM cognitive subtests were related to global neuropsychological functioning even when the potential effects of depression were accounted for. In similar analyses, scores from the Stanford Sleepiness Scale alone explained only 5% of the variance in global neuropsychological functioning (F[1,55] = 3.1, P < 0.06, R2 = 0.082). When entered second in the model, ANAM cognitive tests independently explained an additional 51% of variance (change in F[8,47] = 6.8, P < 0.0001).
Association of ANAM's mood scale with validated measures.
The ANAM mood subscales of activity, fatigue, happiness, depression, anger, and anxiety were correlated with the BDI-II and the Stanford Sleepiness Scale using Pearson's product-moment correlations. BDI-II scores were significantly correlated with all ANAM mood subscales, with the exception of fatigue (P < 0.01). As expected, positive correlations were seen between BDI-II scores and the depression, anger, and anxiety ANAM mood subscales, and negative correlations were seen between BDI-II and the activity and happiness ANAM mood subscales. Correlations were found between the Stanford Sleepiness Scale and ANAM mood subscales for all but the anger subscale. Importantly, an overall ANAM score averaged across cognitive tests was not significantly correlated with the BDI-II, the Stanford Sleepiness Scale, or any of the ANAM mood subscales. Individual r values are presented in Table 5.
Table 5. Pearson's correlations between ANAM mood scale, BDI-II, Stanford Sleepiness Scale, and total ANAM cognitive score*
ANAM total cognitive
Stanford Sleepiness Scale
ANAM = Automated Neuropsychological Assessment Metrics; BDI-II = Beck Depression Inventory, Second Edition.
The current study sought to determine ANAM's usefulness and validity as a screening measure for cognitive functioning in a sample of patients with SLE by first examining ANAM's associations with traditional neuropsychological tests and then determining ANAM's ability to predict neuropsychological status in patients with SLE. Earlier studies have examined ANAM's association with traditional neuropsychological tests, and have validated its usefulness in diverse groups of patients with SLE (1, 34). Consistent with these earlier studies, we found that individual ANAM subtests were significantly correlated with many traditional neuropsychological tests. Similar to the Holliday et al study (34), our study found only 1 association between the simple reaction time subtest and traditional neuropsychological tests, but found a large number of significant associations between ANAM subtests, the WAIS-III Digit Symbol subtest, and Part B of the Trail Making Test. This is not surprising, given that, similar to ANAM, these latter 2 neuropsychological tests require psychomotor speed and complex attention, and the ANAM simple reaction time subtest requires simple motor speed with limited demands on complex attentional skills. Associations were found, however, between most ANAM subtests and the Grooved Pegboard test, which requires fine-motor speed and coordination. Finally, we found that many ANAM subtests, particularly those requiring learning and memory (matching to sample and code substitution delayed) were related to memory tests in the traditional neuropsychological battery.
As a group, patients within this sample showed relatively subtle cognitive dysfunction as determined by traditional neuropsychological testing compared with published norms. Their greatest areas of difficulty, with performances in the mildly impaired range, were observed on tests of verbal learning and visuoconstruction. These findings are consistent with other studies demonstrating cognitive dysfunction in patients with SLE (6, 7, 16).
Due to the variable nature and reported fluctuating course of cognitive impairment in SLE (7, 11, 15, 19), it is reasonable to suspect that, at least for some patients with SLE, cognitive dysfunction is occurring over time. Therefore, ongoing screening of cognitive functioning may prove to be a good tool for monitoring the effect of disease progression on the central nervous system, and may serve to alert physicians when a clinically meaningful change in disease state has occurred. Relying on patient report of cognitive status change may not be sufficient given reported discrepancies between subjective report of cognitive difficulties and objective findings on cognitive testing in patients with SLE (5). Because it is not feasible to repeat neuropsychological evaluations at each clinic visit, reliable cost- and time-efficient screening assessment tools are needed to determine when a more comprehensive neuropsychological evaluation is needed. Therefore, this study sought to build upon existing literature using ANAM in SLE by examining the ability of ANAM to predict neuropsychological status in patients with SLE.
Results of the current study revealed that ANAM cognitive subtests were good predictors of neuropsychological status in general, demonstrating both good sensitivity (76.2%) and specificity (82.8%) in classifying individuals with probable versus no impairment on neuropsychological testing. This finding was true even when premorbid cognitive functioning was controlled. Although the matching to sample and matching grids subtests appeared to be the best predictors of neuropsychological status in this sample, given that ANAM subtests tend to measure overlapping cognitive functions, this finding may be sample specific and should be replicated in future studies. Also, because the cognitive deficits seen in patients with SLE are varied, and some ANAM subtests appear to have specific associations with individual neuropsychological tests, a battery approach is recommended at this time rather than administration of one ANAM subtest over another.
Importantly, ANAM cognitive tests as a whole were not significantly associated with measures of self-reported sleepiness or depressed mood. Further, ANAM cognitive tests remained independent predictors of neuropsychological functioning, even when depression and sleepiness were covaried. Therefore, this study provides good support for ANAM's ability to screen for cognitive dysfunction without being confounded by noncognitive factors that are commonly comorbid in patients with SLE. A limitation of the current study, however, is that most individuals reported only minimal to mild symptoms of depression on the BDI-II; therefore, it is unclear whether a greater degree of depression symptomatology would account for more variance in ANAM performance.
These results also support ANAM's mood scale as a potential screening measure for emotional distress. Analyses demonstrated significant correlations between ANAM's mood scales assessing current mood and the BDI-II assessing mood symptoms over the last 2 weeks. However, these results should be considered preliminary, and future studies are needed to determine ANAM mood scale's sensitivity and specificity to depressed mood in a sample with and without clinically diagnosed depression.
The ANAM computerized battery of cognitive tests has been administered in many settings as a screening tool for cognitive impairment and to track changes in cognition related to disease progression or treatment. ANAM is both cost- and time-efficient and requires less training to administer than traditional neuropsychological tests. This study confirmed the relationship between ANAM cognitive subtests and traditional neuropsychological tests in a group of individuals with SLE (34). Moreover, our data demonstrate ANAM's ability to accurately predict neuropsychological status, even after controlling for estimates of premorbid functioning. ANAM's relationship to neuropsychological functioning remained strong even when confounds of depression and sleepiness were accounted for. ANAM is an easy to use tool for screening neuropsychological impairment in patients with lupus that is capable of identifying subtle cognitive impairments. Diagnosing cognitive dysfunction earlier will lead to better understanding of the pathogenesis of this process and ultimately to earlier and more effective therapy.