The Mini Mental State Examination does not accurately screen for objective cognitive impairment in Fabry Disease

Abstract Fabry disease (FD) patients may suffer from objective cognitive impairment (OCI). This study assessed the accuracy of the Mini Mental State Examination (MMSE) to screen for OCI in FD patients. Presence or absence of OCI was established using a neuropsychological test battery. For different MMSE cutoffs sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and clinical utility index (CUI) to identify OCI were calculated. Eighty‐one patients were included (mean age 44.5 ± 14.3, 35% men, 74% classical phenotype) of which 13 patients (16%) had OCI. The median MMSE score was 29 (range: 25‐30). MMSE cutoffs ≤28 and ≤29 had the highest sensitivity and specificity, with higher specificity reached at cutoff ≤28 (sensitivity: .46, specificity: .73) and higher sensitivity at cutoff ≤29 (sensitivity: .92, specificity: .40). PPV was low for both cutoffs (PPV ≤28: .25, PPV ≤29: .23) resulting in a low positive CUI (case finding ability). The results of our study indicate that the MMSE does not accurately screen for OCI in FD, with poor sensitivity‐specificity trade‐off at all cutoffs. The low PPV shows that the majority of FD patients that score below the cutoffs do not suffer from OCI. Administering the MMSE as a screening test will lead to unnecessary referrals for neuropsychological testing, which is time consuming and burdensome. Screening tools designed to accurately detect mild (executive) impairment might prove more appropriate to screen for OCI in FD.


| INTRODUCTION
Fabry Disease (FD; OMIM 301500) is a rare lysosomal storage disorder caused by mutations in the GLA-gene, which codes for the enzyme α-galactosidase A (enzyme commission no. 3.2.1.22). 1 Reduced or absent activity of this enzyme results in the accumulation of glycosphingolipids such as globotriaosylceramide (Gb3) in various cells types throughout the body. This leads to cardiac, renal, and cerebral involvement and complications. 2 Common cerebrovascular manifestations of FD are white matter lesions, early transient ischemic attacks (TIA), and stroke. 3 In the general population, these cerebrovascular disorders cause cognitive deficits such as impaired executive functioning and vascular dementia. 4 Several studies have shown a relation between FD and objective cognitive impairment (OCI). [5][6][7] In addition, we recently established that stroke is independently related to OCI in FD. 6 Interestingly, while subjective cognitive complaints are often mentioned by FD patients, 6 these seem to be related to depressive symptoms rather than OCI. 6,7 Subjective cognitive complaints therefore probably provide little information on the presence of OCI in FD, complicating the estimation of cognition by clinicians.
Neuropsychological examination, the golden standard in the assessment of cognitive function, is time consuming and burdensome. 8 The administration of cognitive screening instruments is a method to select patients that are likely to have OCI. The most widely used cognitive screening instrument is the Mini Mental State Examination (MMSE 9 ). The MMSE was designed for clinicians to get a quick indication of cognitive performance. 9 It is most commonly used to screen for dementia for which it works reasonably well, with a sensitivity of 0.85 and specificity of 0.90 in elderly community samples. 10 Its accuracy for the detection of subtle cognitive deficits is less impressive, with sensitivity dropping to 0.60. 11 Studies using the MMSE to assess cognitive functioning in FD 12,13 reported that OCI was not present. Later studies, using a full neuropsychological test battery, have shown that the prevalence of OCI in FD is probably increased compared to the general population, 6,7 suggesting that the MMSE might not be sensitive enough to detect the cognitive deficits found in FD.
The purpose of the present study was to assess the usefulness of the MMSE to screen for OCI in FD.

| Study design and participants
This study used the baseline data of a prospective cohort study assessing cognition in a cohort of FD patients at the Amsterdam University Medical Centre (Amsterdam UMC, location Academic Medical Centre [AMC]). The neuropsychological data were previously described in relation to predictors of OCI. 6 All adult FD patients (≥18 years) known at the AMC (n = 154), the national referral center for FD, were screened for eligibility. Ten patients were excluded according to preset criteria ( Figure S1). Patients were phenotypically classified as classical or nonclassical in accordance with previously published criteria. 6,14 The study was approved by the Human Research Ethics Committee of the AMC. All participants signed informed consent prior to inclusion. This manuscript was written in accordance with criteria for appropriate reporting in diagnostic accuracy studies: the STARD 15 and STARDdem. 16 All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2000. Informed consent was obtained from all patients for being included in the study.

| Data collection
Data collection for this study was performed at the AMC outpatient clinic or during a home visit (see Data S1 for additional information on data collection). The MMSE was administered on the same day as the neuropsychological test battery, always before the battery. Additional data, such as patient characteristics, were collected from the local Fabry database and cross-checked with digital medical records (see Data S1 for additional information on questionnaires and patient characteristics in Table 1).

| The Mini Mental State Examination
The MMSE screens general cognitive functioning with a score ranging from 0 to 30, with higher scores indicating better cognitive functioning. 9 The MMSE includes measures of memory, orientation in time and place, working memory, visuospatial skills, object naming, writing, reading, and complex motor operation. The cutoff most often used for presence of dementia is ≤23/30. 17

| Neuropsychological test battery
Neuropsychological functioning was assessed across the following five domains: language, memory, visuospatial perception, processing speed, and executive functioning. Raw test scores were converted to normative T-scores (mean = 50, SD = 10, corrected for age, education, and sex where possible) using extensive normative data (median sample size = 471, range 121-1000). 6 Language skills were assessed using the 30-item short form of the Boston Naming Test (BNT) 18,19 and the Wechsler Adult Intelligence Scale IV: Similarities (WAIS-IV: Sim). 20 Memory was assessed with the Rey Auditory Verbal Learning Test (RAVLT) 21  .004 b < a, c, d

Note:
Continuous variables were displayed as mean ± SD or median (range) and discrete variables as n (%).
Abbreviations In one 48-year-old woman with a classical phenotype and without objective cognitive impairment, the MMSE was not administered due to logistical issues; n.s., not significant after correcting for multiple comparisons.
c Inability to work was defined as an official statement from the Dutch government that one is unfit for work. d Imaging data of seven patients (four classical women, three nonclassical men) were not available (presence of non-MRI compatible ICD/pacemaker (n = 6), claustrophobia (n = 1)). skills were assessed using the WAIS IV: Block Design (WAIS-IV: BD) 20 ; and the Judgement of Line Orientation (JLO). 23 Processing speed was assessed using the Trail Making Test Part A (TMTA), 24 Stroop Word (W), and Color (C). 25 Executive functioning was assessed using the TMT part B (TMTB), 24 Stroop Colour-Word (CW), 25 Category Fluency (categories: animals and occupations), 26 and Letter Fluency. 27 2.5 | Objective cognitive impairment OCI was defined as a T-score ≤33 on two or more distinct cognitive tests, resembling statistical significance of two one-tailed tests with P < .05 (T-scores ≤33 imply scoring <5th percentile or 1.67 SD below the mean T-score of the normative population of 50

| Data analysis
Statistical analyses were performed using R (version 3.4.3). Patient characteristics and questionnaire scores for the different patient groups were compared using one-way ANOVAs, Kruskal Wallis tests and Fisher's exact tests where appropriate. For significant effects, post-hoc tests (Tukeys HSD, Dunn Test and 2x2 Fisher exact tests) were performed and corrected for multiple comparisons. The diagnostic properties of the MMSE to screen for OCI at different cutoff scores of were assessed by calculating sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and the clinical utility index (CUI). The CUI takes both the discriminative ability of the test and the prevalence of the disease into account with a CUI ≥0.81 being excellent, ≥0.64 good, ≥0.49 satisfactory, and <0.49 poor. 28 Positive CUI (CUI+: sensitivity*PPV) displays the case finding ability of the test. Negative CUI (CUI-: specificity*NPV) displays the ruling out ability of the test. An ROC-curve was plotted and the area under the curve (AUC) was calculated.

| Patient characteristics
Eighty-one FD patients were included in the study (flow chart in Figure S1). Participants and nonparticipants did not differ in age, sex, phenotype, median Fazekas score, and the occurrence of TIA or stroke. 6 Participating patients' mean age was 44.5 years (SD: 14.3, range: 19-76), 53 were women (65.4%), and 60 (74.1%) were classified as having a classical phenotype (Table 1).
Depressive symptoms were present in 31 patients (38.3%), with no significant differences between the subgroups divided by sex and phenotype. Disease severity as assessed by the Mainz severity score index ranged from mild in women with a nonclassical phenotype (median: 6.5, range: 2-20) to moderate in men with a classical phenotype (median: 32, range: 15-68). Deep white matter lesions were present in 47.3% of all patients.

| MMSE and OCI
The median MMSE score of the sample was 29 (range [25][26][27][28][29][30], with no differences across subgroups divided by sex and phenotype. In the neuropsychological test battery, reduced T-scores were predominantly found in male patients in the executive domain. 6 Thirteen patients were classified as having OCI of whom four had severe OCI. Men with a classical phenotype had the highest prevalence of OCI (n = 7; 41.2%), while in women with a nonclassical phenotype OCI was not present. In the other two subgroups (men with a nonclassical and women with a classical disease phenotype), an intermediate prevalence of OCI was found (27.3% and 7.0% respectively).

| Post hoc analyses: MMSE and OCI in patient subgroups
We calculated the discriminant properties of the MMSE for different patient subgroups to evaluate whether the MMSE performed better between subgroups divided by sex or phenotype or when screening for severe OCI. The discriminant properties of the MMSE for women, men, classical and nonclassical phenotype showed a similar pattern as for the patient group as a whole (Table S1-S4).

| DISCUSSION
The results of our study indicate that the MMSE does not accurately screen for OCI in FD, with poor sensitivity-specificity trade-off at all cutoffs. Thirteen patients had OCI according to our preset criteria. The poor PPV, case finding ability (CUI+), and ruling out ability (CUI-) disqualify the MMSE as a cognitive screening instrument to determine which patients need comprehensive neuropsychological testing, as the majority of patients would still be referred for further testing, which is time consuming and burdensome.
Our results are in line with the consensus that the MMSE cannot accurately differentiate subtle cognitively impaired from cognitively unimpaired patients 11 and does not detect executive dysfunction. 29 Studies suggest that the MMSE is an adequate screening instrument in a setting with a high prevalence of disorders resulting in severe cognitive impairment. It loses predictive value when cognitive disturbances are milder, less prevalent, and mainly occur in the executive domain, 30-32 as seems to be the case in FD. 5,6 This is, to our knowledge, the first study on the accuracy and effectiveness of using a cognitive screening instrument in a FD population. Previous studies have used the MMSE to assess global cognitive functioning in FD patients 12,13,33 (Table S2). The conclusion reached in these studies, namely that cognition is unaffected when the MMSE scores are in the normal range (≥24/30), 12 is in disagreement with the results of the current study, in which we validated MMSE scores using individual neuropsychological test scores.
Although we assessed cognition using the gold standard, a neuropsychological test battery, the cutoff for the presence of OCI is an arbitrary one. After reviewing FD literature we expected that most cognitive impairment found in this disorder would be mild. 5 As such, a cutoff T-score of ≤33 on two tests assessing different cognitive domains limited the number of false positives, while still including patients with milder cognitive impairment.
An alternative to using the MMSE could be to use alternative screening instruments such as the Montreal Cognitive Assessment (MoCA). 34 This screening instrument includes more cognitive domains that seem to be affected in FD, 5 like executive functioning and sustained attention. Also, the MoCA is advised for use in populations with mild cognitive impairment or early stage dementia. 11  cognitive impairment was found in FD patients at group level using the MoCA, 12 the MoCA classified 21% of FD patients as possibly having mild cognitive impairment compared to 11% of controls. Nonetheless, it remains to be investigated whether the MoCA is able to accurately detect individual FD patients that show OCI in comparison to a neuropsychological test battery.
In conclusion, this study showed a poor ability of the MMSE to screen for OCI in patients with FD. Clinicians should be cautious in using the MMSE, as it is probably not time-or cost-effective as a screening tool and could burden patients with unnecessary assessments. Future research should find out whether alternatives show better accuracy to screen for OCI in FD.

CONFLICT OF INTERESTS
Simon Körver, Sara A.J. van de Schraaf, and Gert J. Geurtsen report no conflict of interest.
Carla Hollak is involved in premarketing studies with Genzyme, Protalix, and Idorsia. Financial arrangements are made through AMC Research BV. No fees, travel support, or grants are obtained from Pharmaceutical Industries. She reports no nonfinancial conflicts of interest.
Mirjam Langeveld is involved in premarketing studies with Genzyme, Protalix, and Idorsia. Financial arrangements are made through AMC Research BV. No fees, travel support, or grants are obtained from Pharmaceutical Industries. She too reports no nonfinancial conflicts of interest.
Ivo N. van Schaik chairs a steering committee for CSL Behring and received departmental honoraria for serving on scientific advisory boards for CSL Behring and Baxter. All lecturing and consulting fees for INS were donated to the Stichting Klinische Neurologie, a local foundation that supports research in the field of neurological disorders. He too reports no nonfinancial competing interests.

DATA ACCESSIBILITY
The data sets generated and analyzed during the current study are not publicly available. Because of the rarity of the disease, even anonymized can be linked to a specific individual. In case of a specific scientific question, requests to make part of the data set available will be reviewed.