Value of Questionnaire-Based Screening as a Proxy for Neurocognitive Testing in Childhood-Onset Systemic Lupus Erythematosus

Authors


  • The contents of this study are solely the responsibility of the authors and do not necessarily represent the official views of the NIH.

Abstract

Objective

To investigate the utility of questionnaire-based assessment of cognitive function and behavioral/emotional symptoms to screen for neurocognitive dysfunction in childhood-onset systemic lupus erythematosus (cSLE).

Methods

Forty children with cSLE and 24 healthy controls ages 10–16 years were enrolled. Formal neurocognitive testing (FNCT) was done to determine cognitive performance in 4 key areas that appear to be sensitive to the adverse effects of cSLE: attention, working memory, psychomotor speed, and visuoconstructional ability. Paper and pencil questionnaires sampling cognitive functioning and behavioral/emotional symptoms were also completed: the Subjective Awareness of Neuropsychological Deficits for Children (SAND-C) questionnaire by patients, and the Child Behavioral Checklist and the Behavior Rating Inventory of Executive Function (BRIEF) by parents.

Results

Domain and summary scores of the BRIEF and SAND-C correlated modestly with participants' performance on FNCT. Questionnaire ratings did not discriminate subjects with different levels of cognitive ability as measured by FNCT.

Conclusion

Contrary to some reports in adults with SLE, self-administered questionnaires of cognitive functioning and parent ratings of executive functioning do not appear well suited to replace FNCT in screening for neurocognitive impairment of children and adolescents with cSLE. However, they may provide information that is complementary to FNCT and therefore play a useful role in clinical followup.

Introduction

Patients with systemic lupus erythematosus (SLE) frequently report cognitive problems, and many studies have documented cognitive deficits using neuropsychological testing ([1]). Most studies find cognitive deficits on tests measuring attention/concentration, cognitive flexibility, free recall memory, and speed of information processing, leading some to suggest the presence of a subcortical cognitive syndrome ([2]). Neurocognitive dysfunction (NCD) in SLE is often subtle and difficult to ascertain in daily clinical practice. Nonetheless, NCD can adversely affect quality of life ([3]); therefore, it is important to consider in routine care of these patients.

Diagnosis of NCD is typically made by formal neurocognitive testing (FNCT). However, FNCT is costly, time consuming, and not readily accessible in daily clinical practice. Furthermore, neurocognitive functioning can fluctuate over time, depending on disease activity and treatment. Although controversial ([4]), some studies in adult SLE suggest that self-report questionnaires may be useful as screening tools for NCD ([5, 6]). In pediatric populations, parent/proxy reporting is often employed to obtain information on patient outcomes and is relatively inexpensive, quick, and easy to integrate into busy clinics. However, no study has assessed whether it is empirically sound clinical practice to use parent/proxy or self-report questionnaire data to screen for NCD in childhood-onset SLE (cSLE).

The objective of this study was to examine relationships between FNCT and 1) patient self-report using the Subjective Awareness of Neuropsychological Deficits for Children (SAND-C) questionnaire and 2) parent/proxy report as measured by the Behavior Rating Inventory of Executive Function (BRIEF) and subscales of the Child Behavioral Checklist (CBCL) in children and adolescents with cSLE.

Box 1. Significance & Innovations

  • This is the first study assessing the role of questionnaires sampling cognition and behavioral/emotional symptoms as screening devices in children with systemic lupus erythematosus (SLE).
  • In children with SLE, self-administered and parent/proxy report questionnaires of cognitive function and behavioral/emotional symptoms more likely complement information obtained from formal neurocognitive testing, rather than supplant it.

Patients and methods

As part of a larger investigation at 2 tertiary pediatric rheumatology centers, patients with cSLE and healthy controls of similar demographic characteristics were studied cross-sectionally at baseline ([7]). Participants underwent FNCT, and participants and their parents also completed questionnaires assessing constructs related to cognitive ability and behavioral/emotional symptoms. This study was approved by the institutional review boards of both institutions, with written consent and assent obtained as appropriate.

cSLE participants fulfilled the updated American College of Rheumatology classification criteria prior to age 17 years ([8]) and were ages 9–18 years at the time of study enrollment. Patients were excluded from participation if they had a history of comorbid conditions affecting their neurocognitive functioning prior to cSLE diagnosis or known structural brain abnormalities, neuropathies, or movement disorders.

Each cSLE patient was asked to identify a friend within 1 year of their age, of the same sex, and in the same school grade to serve as a control. Controls needed to be healthy and without NCD to be considered for analyses. Disease activity was measured by the Systemic Lupus Erythematosus Disease Activity Index and disease damage was measured by the Systemic Lupus International Collaborating Clinics/American College of Rheumatology Damage Index ([9]). Participants spoke English as their primary language.

Measures

The SAND-C, which was completed by participants, is a 47-item self-report questionnaire developed to assess awareness of cognitive dysfunction ([10]). An overall SAND-C summary score is calculated ([10]), with higher scores representing greater acknowledgment of cognitive limitations.

The BRIEF, which was completed by parents/caretakers of participants, is an 86-item parent questionnaire sampling children's executive functioning in home and school environments ([11]). It produces 3 narrow-band scales (inhibit, shift, and emotional control) that cluster on a composite Behavioral Regulation Index, 5 narrow-band scales (initiate, working memory, plan/organize, organization of materials, and monitor) that cluster on a composite Metacognition Index, and an omnibus composite of all 8 narrow-band scales, the Global Executive Composite. Ratings on BRIEF scales and composite indices are expressed in age-standardized scores (T scores) having a normative mean ± SD of 50 ± 10, with higher scores indicating greater executive dysfunction.

The CBCL, which was completed by parents/caretakers of participants, is a 118-item questionnaire sampling behavioral and emotional symptoms and adaptive competencies. It produces 3 narrow-band scales (social withdrawal, somatic complaints, and anxiety/depression) that cluster on a composite Internalizing Problems index, 2 narrow-band scales (rule-breaking behavior and aggressive behavior) that cluster on a composite Externalizing Problems index, and 3 additional narrow-band scales (social problems, thought problems, and attention problems). The 8 narrow-band scales also combine to produce an omnibus Total Problems index. CBCL ratings also produce 3 narrow-band competency scales (activities, social, and school) that combine to form a Total Competency index. Ratings on all CBCL scales and composite indices are expressed in age-standardized scores (T scores) having a normative mean ± SD of 50 ± 10. For behavioral and emotional symptom scales, higher scores indicate greater concerns; for competency scales, lower scores indicate greater concerns.

FNCT and determination of cognitive function

All participants underwent FNCT by a trained psychometrician using a standardized test battery designed for neurocognitive followup of cSLE ([12]). Using published norms, participants' performance on each of the neuropsychological tests was expressed as Z scores with a normative mean ± SD of 0 ± 1. The tests were clustered into 4 cognitive domains using a previously reported framework ([7]): working memory, psychomotor speed, visuoconstructional ability, and attention/executive functioning (see Supplementary Table 1, available in the online version of this article at http://onlinelibrary.wiley.com/doi/10.1002/acr.22247/abstract). Functioning in each cognitive domain was estimated by calculating a mean Z score for the tests clustered in that domain. In the absence of a generally accepted definition for NCD, cognitive dysfunction was operationally defined by ≥2 domain Z scores falling below −1 ([13]).

Participants were categorized into 1 of 3 groups based upon disease presence and NCD status: healthy controls without NCD (HC), cSLE without NCD (cSLE-NL), and cSLE with NCD (cSLE-NCD).

Statistical analysis

Before formal statistical analyses, the score distributions of the SAND-C, BRIEF, CBCL, and FNCT were inspected and found to approximate the normal distribution. Therefore, parametric statistical models were used as the primary analytical methods. Associations of the SAND-C, BRIEF, and specific CBCL scale T scores with group membership (HC versus cSLE-NL versus cSLE-NCD) were tested using fixed-effect models. Because CBCL is primarily a behavioral questionnaire, only 3 scales sampling aspects of cognition were examined in relation to FNCT: attention problems, thought problems, and school competency. Both adjusted (co-varying for parent education level) and unadjusted methods were used in the fixed-effect models. Comparable results were obtained with both methods and, therefore, only adjusted methods are reported herein. Post hoc comparisons of group means were performed using Tukey's method, while accounting for multiple comparisons. Pearson's and Spearman's correlation coefficients were calculated to assess relationships between SAND-C, BRIEF, and CBCL scale scores and each of the NCD domain Z scores. Results with these methods were similar; therefore, Pearson's correlation coefficients are reported. Strength of association was considered based on cognitive and behavioral science criteria to be strong, moderate, or weak if the magnitude of the correlation coefficient was 0.5–1.0, 0.3–0.5, or 0.1–0.3, respectively ([14]). Group effects for all other numerical variables were assessed using analysis of variance models. When only cSLE groups were of interest, means and medians of the numerical variable were compared between groups using a 2-sample t-test and Wilcoxon's rank sum test, respectively. The Satterthwaite method was used in the t-test when sample variances were not the same between groups. Chi-square tests were used to compare the groups on categorical variables. All statistical analyses were performed using SAS 9.3 software, with P values less than 0.05 considered statistically significant.

Results

Nine cSLE patients had NCD as determined by FNCT and 31 had normal cognition. Because not all cSLE patients provided a control, data on only 25 were available, including 24 controls with normal cognition. One control was newly diagnosed with NCD and excluded from further consideration in the statistical analysis. Mothers of the HC subjects and cSLE-NL patients had a higher level of education than mothers of the cSLE-NCD subjects (P = 0.019), suggesting lower socioeconomic status in the cSLE-NCD group. Disease activity and daily dose of prednisone did not differ in the cSLE-NCD and cSLE-NL groups (Table 1).

Table 1. Demographics and clinical characteristics at baseline*
CategoryHealthy controls (n = 24)cSLE-NL (n = 31)cSLE-NCD (n = 9)P
  1. cSLE-NL = childhood-onset systemic lupus erythematosus without neurocognitive dysfunction (NCD); cSLE-NCD = cSLE with NCD; SLEDAI = Systemic Lupus Erythematosus Disease Activity Index; SDI = Systemic Lupus International Collaborating Clinics/American College of Rheumatology Damage Index.
  2. aAn analysis of variance F test was used to assess the group effect.
  3. bA chi-square test was used to assess the association between the categorical variable and the group effect.
  4. cThe post hoc comparison between a cSLE group and the control group is significant.
  5. dThe post hoc comparison between cSLE groups is significant, after adjusting for multiple comparisons using Tukey's test.
  6. eA 2-sample t-test and Wilcoxon's rank sum test were used to compare means and medians between groups. The Satterthwaite method was used in the t-test when variances were not the same between groups in a folded F test.
Age at enrollment, mean ± SD yearsa14.17 ± 2.2414.58 ± 2.3515.56 ± 1.940.297
Race/ethnicity, no. (%)b   0.120
White11 (46)11 (35)1 (11) 
African American8 (33)10 (32)8 (89) 
Hispanic4 (17)7 (23) 
Asian1 (4)1 (3) 
Other2 (6) 
Maternal education level, no. (%)b   0.019
No high school3 (13)2 (6) 
High school diploma4 (17)5 (16)6 (67) 
Additional education post–high school17 (71)24 (77)3 (33) 
Annual family income, no. (%)b   0.354
<$25,0003 (13)5 (16)3 (33) 
$25,000–50,0007 (30)9 (29)5 (56) 
$51,000–75,0006 (26)7 (23)1 (11) 
>$75,0007 (30)10 (32) 
Cognitive domain performance, mean ± SD Z scorea    
Working memory0.01 ± 0.54−0.11 ± 0.63c−0.98 ± 0.52d< 0.001
Psychomotor speed0.26 ± 0.720.22 ± 0.66c−1.11 ± 0.50d< 0.001
Attention0.10 ± 0.560.16 ± 0.73−0.20 ± 1.020.444
Visuoconstructional ability0.09 ± 0.640.22 ± 0.71c−1.28 ± 0.78d< 0.001
cSLE features, mean ± SD (median, range)e    
cSLE duration, months 23.99 ± 24.24 (12.2, 2.7–92.0)22.78 ± 19.82 (17.3, 1.5–55.2)0.892, 0.974
Physician assessment of disease activity 2.42 ± 1.98 (2.0, 0.0–9.0)2.33 ± 2.00 (2.0, 0.0–6.0)0.909, 0.831
Disease activity (SLEDAI) 3.90 ± 2.66 (4.0, 0.0–10.0)8.22 ± 7.03 (6.0, 0.0–20.0)0.106, 0.095
Disease damage (SDI) 0.35 ± 0.75 (0.0, 0.0–3.0)0.56 ± 1.01 (0.0, 0.0–3.0)0.520, 0.539
Prednisone daily dose, mg 15.92 ± 11.92 (15.0, 3.0–60.0)33.29 ± 26.54 (30.0, 0.5–80.0)0.138, 0.090

Based on parent reports on the CBCL, the cSLE-NCD group had more social withdrawal compared to the HC (P = 0.038) (Table 2). In addition, the cSLE-NCD subjects had higher ratings of somatic complaints than the HC (P = 0.014), higher ratings of externalizing problems than the cSLE-NL subjects (P = 0.01), and lower activities competency ratings than the cSLE-NL subjects (P = 0.021). Notably, the groups did not differ significantly on the majority of the BRIEF and CBCL scales. Additionally, patient self-reports of cognitive deficits (SAND-C) did not differ significantly between groups.

Table 2. Comparisons of means of the BRIEF, CBCL, and SAND-C among healthy control, cSLE-NL, and cSLE-NCD groups at baseline*
 Healthy controlscSLE-NLcSLE-NCD
  1. Values are the mean ± SE. P values are from post hoc comparisons using Tukey's method under adjusted fixed-effect models. The adjusted model used parent's highest education level as the controlling covariate in addition to the group effect (control vs. cSLE-NL vs. cSLE-NCD). BRIEF = Behavior Rating Inventory of Executive Function; CBCL = Child Behavioral Checklist; SAND-C = Subjective Awareness of Neuropsychological Deficits for Children; cSLE-NL = childhood-onset systemic lupus erythematosus without neurocognitive dysfunction (NCD); cSLE-NCD = cSLE with NCD.
  2. aP < 0.05 for healthy controls vs. cSLE-NCD (adjusted model).
  3. bCBCL scales that sample aspects of cognition.
  4. cP < 0.05 for cSLE-NL vs. cSLE-NCD (adjusted model).
BRIEF   
Index scores   
General Executive Composite47.79 ± 1.6347.52 ± 1.4448.78 ± 2.67
Behavioral Regulation Index47.33 ± 1.4146.61 ± 1.2445.67 ± 2.30
Metacognition Index48.63 ± 1.8948.45 ± 1.6651.00 ± 3.09
Clinical scores   
Inhibit47.96 ± 1.4647.13 ± 1.2846.00 ± 2.38
Shift48.04 ± 1.7347.61 ± 1.5246.67 ± 2.83
Emotional control47.17 ± 1.6346.48 ± 1.4445.78 ± 2.67
Initiate49.54 ± 1.7548.52 ± 1.5450.44 ± 2.86
Working memory49.88 ± 2.0451.39 ± 1.8052.33 ± 3.33
Plan/organize49.04 ± 1.8448.06 ± 1.6250.67 ± 3.01
Organization of materials47.25 ± 1.9447.94 ± 1.7047.22 ± 3.16
Monitor46.67 ± 1.8545.90 ± 1.6350.56 ± 3.03
CBCL   
Summary scales   
Total Problems47.25 ± 1.9148.00 ± 1.6849.22 ± 3.11
Total Competency49.26 ± 2.4747.17 ± 2.1740.78 ± 3.95
Syndrome/narrow-band scales   
Social withdrawal52.71 ± 1.1554.13 ± 1.0258.33 ± 1.88a
Somatic complaints56.21 ± 1.8660.26 ± 1.6459.56 ± 3.05a
Anxiety/depression52.88 ± 0.8352.32 ± 0.7351.56 ± 1.36
Social problems52.67 ± 0.9753.65 ± 0.8653.22 ± 1.59
Thought problemsb53.21 ± 1.0153.97 ± 0.8856.22 ± 1.64
Attention problemsb52.00 ± 0.8453.52 ± 0.7454.11 ± 1.37
Rule-breaking behavior52.46 ± 0.7051.90 ± 0.6152.00 ± 1.14
Aggressive behavior51.58 ± 0.6752.06 ± 0.5951.44 ± 1.09
Broad-band groupings of psychopathology   
Internalizing Problems49.58 ± 1.9552.81 ± 1.7253.33 ± 3.18
Externalizing Problems46.00 ± 1.6545.90 ± 1.4546.44 ± 2.70c
Competency scales   
Activities competency45.83 ± 2.3943.19 ± 2.0645.67 ± 3.82c
Social competency50.21 ± 1.8850.80 ± 1.6844.11 ± 3.07
School competencyb51.33 ± 1.5650.26 ± 1.3842.11 ± 2.55
SAND-C total score104.13 ± 2.48104.61 ± 2.1994.78 ± 4.06

The tests of association between questionnaire ratings and neurocognitive function in the 4 domains measured by FNCT are shown in Table 3. The majority of these correlations were weak (r <0.2) and nonsignificant. However, a few moderate but significant correlations ranging in magnitude from 0.26–0.31 were noted. Higher attention domain FNCT scores were associated with higher CBCL school competency scores (indicating better school functioning). Higher psychomotor domain FNCT scores were associated with lower scores (indicating better functioning) on the BRIEF monitor and initiate narrow-band scales and the BRIEF Metacognition Index and General Executive Composite. Similarly, higher working memory domain FNCT scores were associated with lower scores (indicating better functioning) on the BRIEF monitoring narrow-band scale.

Table 3. Pearson's correlations of BRIEF, CBCL, and SAND-C with neurocognitive dysfunction subdomain Z scores at baseline (n = 64)*
 AttentionPsychomotor speedVisuoconstructional abilityWorking memory
  1. Only adjusted correlation coefficients with an absolute magnitude ≥0.2 are shown. Correlation coefficients have been adjusted for parent's education level. BRIEF = Behavior Rating Inventory of Executive Function; CBCL = Child Behavioral Checklist; SAND-C = Subjective Awareness of Neuropsychological Deficits for Children.
  2. aP < 0.05.
  3. bOnly CBCL scales that sample aspects of cognition are shown.
  4. cP < 0.01.
BRIEF    
Index scores    
General Executive Composite −0.26a  
Behavioral Regulation Index    
Metacognition Index −0.27  
Clinical scores    
Inhibit    
Shift    
Emotional control    
Initiate −0.27a  
Working memory    
Plan/organize −0.2  
Organization of materials −0.22  
Monitor −0.31a −0.25a
CBCLb    
Thought problems    
Attention problems    
School competency0.31a 0.35c0.2
SAND-C total score  0.32a 

Discussion

NCD appears to be common among adults and children with SLE, with the potential to decrease quality of life, making the availability of an easy to use screening tool for NCD highly desirable. Self-administered questionnaires have been proposed by some as suitable screening tools for NCD in adults with SLE ([5]). Given developmental differences between adult and pediatric populations, we studied associations in cSLE between FNCT and self-administered and parent/proxy ratings using the SAND-C, BRIEF, and CBCL. Overall, the 3 questionnaires were weakly associated with results of FNCT, either when considered as continuous scores or when used to group our cSLE patients by neurocognitive status.

Our results are in line with those published by Hanly et al, who found that the self-report Cognitive Symptom Inventory was unsuitable for NCD screening in adults with SLE ([4]). However, results of FNCT were unavailable in that study and instead, cognitive functioning was estimated using the Automatic Neuropsychological Assessment Metrics, an abbreviated computerized test of cognition whose childhood adaptation holds promise as a screening tool for NCD in cSLE ([7]).

A recent systematic review ([15]) of studies in adults and children is consistent with our findings in indicating that there are, at best, modest associations between questionnaire-based ratings of executive function and results of formal office-based tests of executive skills. However, although FNCT provides measurements of information processing and response efficiency in an office setting that is typically quiet and highly controlled, paper and pencil questionnaires may provide valid and complementary information as indicators of functioning under “real-world” unstructured conditions.

Our study has some limitations, including our choice of a limited number of questionnaires that we considered representative of self-report and parent/proxy report data-gathering techniques. We cannot exclude the possibility that other questionnaires might be more closely aligned with performance on FNCT. We did not assess the cognitive function of caregivers, which might have influenced their ratings on paper-based questionnaires. However, in ordinary clinical use, caregivers would not be subject to such assessments. Also, given the maternal educational level reported, we do not anticipate that the cognitive functioning of caregivers significantly affected the overall study result. Furthermore, although the sample size of our study is limited, ours is the first investigation of associations between self-report and parent/proxy report questionnaires and FNCT in cSLE patients. The modest magnitude of correlations shown in Table 3 suggests that a larger sample would not yield more clinically significant findings.

Our study suggests that FNCT and questionnaire measures of cognition, behavior, and executive function capture different aspects of functioning. Therefore, paper and pencil questionnaires cannot be considered a proxy or replacement for FNCT. Nonetheless, we agree with Toplak and colleagues ([15]) that self-administered questionnaires and FNCT are potentially useful as complementary methods for assessing neurocognitive function and its impact upon daily behavior and adaptive functioning at school and in other performance contexts.

AUTHOR CONTRIBUTIONS

All authors were involved in drafting the article or revising it critically for important intellectual content, and all authors approved the final version to be published. Dr. Brunner had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study conception and design. Vega-Fernandez, Zelko, Ying, Beebe, Brunner.

Acquisition of data. Vega-Fernandez, Zelko, Klein-Gitelman, Lee, Hummel, Nelson, Thomas, Beebe, Brunner.

Analysis and interpretation of data. Vega-Fernandez, Zelko, Klein-Gitelman, Ying, Beebe, Brunner.

Acknowledgments

This study would not have been possible without the dedicated clinical research personnel, namely Aimee Baker and Dina Blair. We would also like to thank Meredith Amaya, Allison Clarke, Kate Dahl, Antoinette Dezzutti, Lev Gottlieb, Jennifer Heil, Jennifer Keller, Andrew Phillips, Michal Rischall, Rebecca Wasserman Lieb, and Mariah Wells for their assistance with neuropsychological testing. A special thanks to Ms Elaine Holtkamp for her administrative support of the study and assistance with the manuscript.

Ancillary