Predicting conversion to Alzheimer's disease among individual high‐risk patients using the Characterizing AD Risk Events index model

Abstract Aims Both amnestic mild cognitive impairment (aMCI) and remitted late‐onset depression (rLOD) confer a high risk of developing Alzheimer's disease (AD). This study aims to determine whether the Characterizing AD Risk Events (CARE) index model can effectively predict conversion in individuals at high risk for AD development either in an independent aMCI population or in an rLOD population. Methods The CARE index model was constructed based on the event‐based probabilistic framework fusion of AD biomarkers to differentiate individuals progressing to AD from cognitively stable individuals in the aMCI population (27 stable subjects, 6 progressive subjects) and rLOD population (29 stable subjects, 10 progressive subjects) during the follow‐up period. Results AD diagnoses were predicted in the aMCI population with a balanced accuracy of 80.6%, a sensitivity of 83.3%, and a specificity of 77.8%. They were also predicted in the rLOD population with a balanced accuracy of 74.5%, a sensitivity of 80.0%, and a specificity of 69.0%. In addition, the CARE index scores were observed to be negatively correlated with the composite Z scores for episodic memory (R 2 = .17, P < .001) at baseline in the combined high‐risk population (N = 72). Conclusions The CARE index model can be used for the prediction of conversion to AD in both aMCI and rLOD populations effectively. Additionally, it can be used to monitor the disease severity of patients.

from 10% to 15% per year, depending on the country and population studied. 4,5 Late-onset depression (LOD), one type of late-life depression (LLD), represents a major depressive episode that initially occurs after the age of 50-65. 6 Importantly, the LOD is suggested to exhibit higher sensitivity to expedited brain aging and likely to predispose to AD through the exhaustion of the brain's structural and functional reserve. 7 Individuals with LOD commonly display cognitive deficits, and LOD confers a nearly 50% upregulated risk for dementia growth. 8,9 Furthermore, cognitive dysfunction is reported to be persistent in LOD cases even when remission had been achieved. 10 In certain aspects, aMCI and LOD confer a high risk of developing AD and they might represent a potential clinical continuum. 11 Many studies have supported the notion that LOD and aMCI share common AD pathology biomarkers (eg, amyloid-β[Aβ]), 12 common genetic risk factors (eg, APOE ε4, clusterin-T allele), 13,14 common regions of brain atrophy, 15,16 common brain network disruptions, 17 and common impairments in cognitive domains. 18,19 Recently, an amyloid/tau/neurodegeneration (A/T/N) research framework has been published to advance the accurate diagnosis of AD. 20 With this framework, AD can be identified earlier through abnormalities in amyloid and tau, even without the presence of clinical symptoms. However, molecular positron emission tomography or cerebrospinal fluid (CSF) biomarkers are often not available in the clinic, especially in the early stages for individual high-risk patients. 21 Therefore, researchers are constantly looking for objective biomarkers that are noninvasive and convenient to the screening of AD in its early stage. Combinations of in vivo biomarkers using the machine learning (ML) framework for predicting the progression of patients with mild cognitive impairment (MCI) to dementia has been focus of attention in recent years.
Although some of these studies have achieved good predictive performance, most of the models have been validated with the leave-one-out approach, 22,23 or K-fold cross-validation loop, 24,25 which may overestimate the models' performance. Thus, external validation of the model in an independent cohort is a crucial step in its extensive promotion. 26 On the other hand, to our knowledge, in the field of predicting dementia conversion in LLD patients, only Lebedeva and his colleagues have used multidimensional biomarkers by employing the random forests (RF) framework of the ML method. 27 The sensitivity of their Alzheimer's Disease Neuroimaging Initiative (ADNI) model validation in the LLD cohort was determined to be 48.0%-65.5%, the specificity was calculated to be 62.0%-68.6%, and the area under the curve (AUC) was 0.623-0.737. In addition, they trained new models on the cohort of LLD patients with an AUC performance of 0.678-0.905. However, this result is likely to be biased and overestimated by the out-of-bag method which was used for internally estimating the prediction performance. 28 Therefore, a sensitive and generalizable model for identifying individuals with LOD who have a high risk of progressing to AD is lacking.
In our previous study, 29 we chose several AD biomarkers of cognitive assessments, brain structure and function, and CSF from the ADNI cohort, 30 to appraise their optimal sequence with the help of the event-based probabilistic (EBP) framework. On the basis of this, we have developed the Characterizing AD Risk Events (CARE) index to measure AD progression in each individual. This CARE index model has been demonstrated to identify AD from healthy controls and accurately differentiate late MCI from early MCI. 29 In addition, we used it to predict aMCI-to-AD in two longitudinal cohorts with highly accuracy, robustness, and generalization. 31 The CARE index model works by calculating the risk scores of AD at the individual level based on the optimal temporal sequence of the biomarkers instead of the clinical diagnosis information, so it was supposed to be able to overcome the heterogeneity in the populations at high risk for AD. In this study, we applied the CARE index model to an independent aMCI population and an remitted LOD (rLOD) population to further assess its robustness and generalization and to assess whether the CARE index model can be used to monitor disease severity of the patients thereby supporting its clinical promotion.

| Participants
Written informed consent was obtained from all participants, and the study was approved by the responsible Human Participants Ethics Committee of the Affiliated ZhongDa Hospital, Nanjing, China. Initially, 106 elderly subjects were recruited, including 48 aMCI subjects and 58 rLOD subjects. The inclusion criteria of the aMCI subjects were proposed by Petersen et al 32 and others, 33 and the inclusion criteria for rLOD subjects were described previously 14 (see S.1 in Appendix S1). The included subjects underwent follow-up after an average of 27 months (range 16-39 months).
Twenty-seven subjects (12 aMCI and 15 rLOD) withdrew from the study due to the development of neurological or psychiatric disease (including one rLOD subject who developed vascular dementia), withdrawal of consent, death, or relocation to other cities. Ultimately, 36 aMCI and 43 rLOD subjects were included in the present study. Of these subjects, 4 rLOD subjects and 1 aMCI subject were excluded due to head motion artefacts of more than 2.5 mm translation or 2.5° rotation, and 2 aMCI subjects were excluded for incomplete imaging coverage. The remaining 33 aMCI and 39 rLOD subjects were included in further analysis. The participants received the clinical diagnosis of possible AD with the National Institute of NINCDS-ADRDA criteria 34 ; during the followup period, those patients who progressed to AD were marked as progressive subjects, while the others were marked as stable subjects. Therefore, the included participants were categorized into a progressive aMCI group (P-aMCI) (n = 6), a stable aMCI group (S-aMCI) (n = 27), a progressive rLOD group (P-rLOD) (n = 10), and a stable rLOD group (S-rLOD) (n = 29) (see Table 1). The combined high-risk population was defined as the entire sample (N = 72) in present study, including all the aMCI subjects and rLOD subjects.

| Neuropsychological assessment
The participants completed a series of neuropsychological tests containing assessments of multiple cognitive domains, and the raw scores were Z-transformed to composite each cognitive domain. See S.2 in Appendix S1.

| MRI data acquisition and image preprocessing
The participants assessed at the Affiliated Brain Hospital of Nanjing Medical University (33 aMCI and 12 rLOD subjects) underwent 1.

| Individual CARE index
Based on our previously published research, 29 we selected a total of 10 well-studied AD biomarkers. Moreover, each biomarker represents an event affected by AD progression. These biomarkers consist of four levels: (a) biomarkers from brain structure: gray matter concentration indices (GMI) of the hippocampus (HIP GMI ) and fusi- (d) biomarkers from CSF: Aβ and p-tau levels. Z-standardization is performed to control the site effects of different magnetic field strengths. 35 Elaborated approaches to extract FCI and GMI are presented in S.4 in Appendix S1 along with S.5. We obtained the CARE index by numbering each of the 10 biomarker events according to their optimal sequence of occurrence. Then, we obtained the CARE index score of each individual according to the maximum likelihood value of the optimal sequence (S.6-S.9 in Appendix S1).The mathematical specifics of the optimal sequence for missing biomarker events (Aβ and p-tau levels as well as ADAS-Cog scores), which were not considered in the present study, are described in S.10-S.12 in Appendix S1. The workflow is represented in Figure 1.  Note: Values are presented as the means ± standard deviations (SD). Two-way ANOVAs or Scheirer-Ray-Hare tests were applied to examine the effects of disease and AD progression on age, years of education, and neuropsychological data. Fisher's exact test was applied in the comparisons of sex. Abbreviations: AD: Alzheimer's disease; aMCI, amnesic mild cognitive impairment; ANOVA, analysis of variance; rLOD, remitted late-onset depression.

| Demographic and neuropsychological data
*P < .05. When comparing each cognitive domain, Bonferroni correction for multiple comparisons was performed at a significance level of P < .0125 (P = .05/4 composite scores).
demographic data. Fisher's exact test was applied for the comparisons of sex. We mainly adopted Mann-Whitney U tests to compare the differences in the CARE index scores between the P-aMCI and the S-aMCI subjects and between the P-rLOD and the S-rLOD subjects, respectively. Statistical significance was considered at P < .05.

| Conversion prediction
We appraised the CARE index as well as the power of individual biomarkers to differentiate progressive individuals from stable individuals using receiver operating characteristic (ROC) curves. Because there is a large difference in the number of progressive individuals and stable individuals in our study, which means the data had unbalanced labels, we introduced balanced accuracy ([sensitivity + specificity]/2) for evaluating the prediction capability of the models. 36 In addition, we used logistic regression (LR) models in which the events were conversion to AD in the aMCI population, rLOD population, and combined high-risk population, respectively. The independent variables were CARE index scores and demographic factors: age, sex, and years of education. Statistical significance was considered at P < .05. Moreover, when comparing the power of the classification of the CARE index with each single selected biomarker, we introduced the net reclassification improvement (NRI) index, which is considered to be a sensitive evaluation method to assess the improvements in the performance of the models 37,38  Table 1 suggests that there is no significant difference in age, sex, years of education or MMSE scores among the four groups of subjects. Significant main effects of disease were observed on the composite Z scores for EM, executive function (EF). The aMCI subjects exhibited lower performances on EM and EF than the rLOD subjects.

| Demographic and neuropsychological data at baseline
Significant main effects of AD progression were found on the EM, EF, and information processing speed tests. Finally, no significant interactions of disease and AD progression in the neuropsychological data were found. Furthermore, no significant differences in followup durations were observed between stable subjects and progressive subjects within each population (Ps > 0.05) (Table S1).

| Discriminating stable/progressive individuals
As shown in Figure 2, the CARE index can differentiate progressive subjects from stable subjects at baseline on an individual basis. Significant differences in the CARE index scores were noted between the P-aMCI and the S-aMCI subjects and between the P-rLOD and the S-rLOD subjects (Ps < 0.05). AD diagnoses were predicted in the aMCI population with a balanced accuracy of  Figure 2C and Table 2). Hazard ratio (HR) and statistical significance of each variable in the LR models are listed in Table 3. Goodness of fit of the logistic models were evaluated by the Hosmer-Lemeshow tests (Ps > 0.05).Increasing

| Robustness and power of the CARE index compared to a single biomarker
As explained in Table 2, the CARE index outperforms each of the selected biomarkers in distinguishing the progressive and stable subjects either in the aMCI or in rLOD populations. Although the balanced accuracy of AVLT scores is high in the aMCI population, its AUC is not as high as that of the CARE index. In addition, AVLT scores did not show high generalization and stability across datasets in our previous study. 31 Besides, the sensitivity of AVLT scores at the optimal cutoff is low in the rLOD population.

| Behavioral significance of the CARE index measured at baseline
As illustrated in Figure 3A, in the combined high-risk population, a negative correlation was detected between the CARE index scores and the composite Z scores of the EM at baseline (R 2 = .17, P < .001).
The higher the CARE index score is, the lower the composite Z scores of the EM and the more severe the disease.

| Predictive value of the CARE index scores measured at baseline
As illustrated in Figure 3B, in the combined high-risk popula-

| D ISCUSS I ON
In the present study, we demonstrated that the CARE index model can effectively differentiate individuals progressing to AD from stable individuals in both the MCI population and the rLOD population with satisfactory generalization. In addition, the CARE index demonstrated better performance than each single collected biomarker in our study. Besides, the CARE index scores can be used to monitor disease severity, and the scores at baseline can be used to predict the long-term performance of high-risk patients.
In this study, the CARE index model was applied to the aMCI population assessed using 1.5T MRI scanners with high robustness. Prior to this study, we validated the CARE index model in an independent Nanjing Aging and Dementia Study cohort with 3.0T MRI scanners with an AUC of 0.861, a sensitivity of 81.3%, a specificity of 90.0%, and a balanced accuracy of 85.7%. 31 These data show that the model has satisfactory generalizability and robustness in independent cohorts, even in different MRI scan modes.

F I G U R E 2
Applying the CARE index model to the independent aMCI population and rLOD population for prediction of progressive subjects and stable subjects. A, Percentages of cases in each CARE index stage at baseline in the entire sample (N = 72). Stable subjects (covering S-aMCI and S-rLOD) are represented in blue and progressive subjects (covering P-aMCI and P-rLOD) in red. B, Bars of the CARE index score differences between groups. The Mann-Whitney U tests between S-aMCI and P-aMCI, and between S-rLOD and P-rLOD showed significant differences. Another principal novelty of this study was that the CARE index model can effectively identify progressive rLOD patients from stable rLOD patients. It is worth noting that it is the first time applied the model based on EBP framework to LOD population. As we mentioned, only Lebedeva and colleagues have used an RF framework to predict dementia conversion in LLD patients. 27 In our study, the AUC, sensitivity, and specificity of the CARE index model were better than those values in their research when they validated the ADNI model in the LLD population. In addition, the subjects in their study included both early-onset depression (EOD) and LOD. EOD is likely to denote an independent risk factor that predisposes patients to dementia rather than a prodrome of AD. 11 We also noted that the specificity of the CARE index was not high enough in the rLOD population. We inferred that the reason was that, except for the common biomarkers of AD progression with MCI, some biomarkers may be more specifically related to the conversion of rLOD to AD, such as EF, 47 functional connectivity of the amygdala, 48 and white matter damage in the middle anterior corpus callosum. 27 However, sensitivity is a key indicator for distinguishing potential AD patients for early intervention and is more crucial than specificity. In addition, it is important that our result comes from an independent validation of the model. Therefore, the CARE index may still be considered a good risk-screening tool to identify progressive individuals in the rLOD population.
TA B L E 2 Classification results of CARE index and each selected biomarker indices in aMCI population and rLOD population separately  49,50 Meanwhile, the rLOD group of patients had noticeably poorer performance in the AVLT scores than the healthy group. 51 Many studies have been conducted on the abnormalities of the structure and function of the hippocampus associated with conversion to AD in both MCI 52 and LOD patients. 53, 54 The PCC was thought to be a structural and functional core hub of the default mode network (DMN), and the DMN abnormalities have been associated with AD disease progression. 55 In addition, according to previous studies, the DMN was considered the neural foundation of the link between AD and LOD. 6 Structural and functional indicators of the fusiform gyrus were also incorporated into the CARE index model. Young and colleagues found that brain atrophy in particular regions occurred in a distinct sequence in the AD spectrum; in particular, the hippocampus and entorhinal cortex are initially affected with a progression to the fusiform gyrus and the middle temporal gyrus. 56 The functional connectivity of fusiform gyrus has also been thought to be a part of the DMN. 57 It is interesting that in a task-based fMRI study, the hippocampus and fusiform gyrus were the important brain regions associated with cognitive decline in LLD patients. 58 Second, we believe that the methodological advan- As demonstrated in Table 2 in the context of the entire progression of the disease. Therefore, it could be used to monitor disease progression as an AD staging system for individual high-risk patients. In addition, a negative predictive relationship was observed between CARE index scores measured at baseline and MMSE scores measured at the follow-up, which also proves that the CARE index is a good predictor for the progression of AD.
In the present study, the CARE index model was found to be replicable and accurate for identifying patients with a high risk of progressing to AD. Additionally, this model based on the EBP framework was found not to be limited by the heterogeneity of the population, which will greatly facilitate its application. In addition, this model also has the advantages of being noninvasive, convenient, and especially capable of monitoring patients' clinic progression, which is essential for clinicians. In summary, this model is potentially important and clinically valuable.

| Limitations
First, the sample size in our study was relatively small, particularly in the progressive groups. Second, the MRI scanners used in the rLOD group displayed varied magnetic field strengths. However, zstandardization is performed to control for the site effects of different magnetic field strengths. Third, although our study showed that the CARE index can well overcome the disease heterogeneity, further studies should be considered to incorporate more reliable biomarkers, such as EF and diffusion tensor imaging, to improve the performance of the model.

| CON CLUS ION
In conclusion, the CARE index model was successfully used to predict AD in an independent aMCI population and an rLOD population with satisfactory generalizability and balanced accuracy. In particular, the CARE index model should be used to monitor patients' clinical progression. Our findings suggest that the CARE index can be effectively applied to detect individual high-risk patients in clinical practice to develop early treatment strategies for preventing AD progression.

ACK N OWLED G M ENTS
We sincerely thank PhD. Shi-jiang Li, PhD. Gang Chen, PhD.
Guang-yu Chen (Department of Biophysics, Medical College of Wisconsin, Milwaukee, USA) for supporting the code of the CARE index model.

CO N FLI C T O F I NTE R E S T
The authors declare that they have no conflict of interest.