Deep Learning for Brain MRI Confirms Patterned Pathological Progression in Alzheimer's Disease

Abstract Deep learning (DL) on brain magnetic resonance imaging (MRI) data has shown excellent performance in differentiating individuals with Alzheimer's disease (AD). However, the value of DL in detecting progressive structural MRI (sMRI) abnormalities linked to AD pathology has yet to be established. In this study, an interpretable DL algorithm named the Ensemble of 3‐dimensional convolutional neural network (Ensemble 3DCNN) with enhanced parsing techniques is proposed to investigate the longitudinal trajectories of whole‐brain sMRI changes denoting AD onset and progression. A set of 2369 T1‐weighted images from the multi‐centre Alzheimer's Disease Neuroimaging Initiative and Open Access Series of Imaging Studies cohorts are applied to model derivation, validation, testing, and pattern analysis. An Ensemble‐3DCNN‐based P‐score is generated, based on which multiple brain regions, including amygdala, insular, parahippocampal, and temporal gyrus, exhibit early and connected progressive neurodegeneration. Complex individual variability in the sMRI is also observed. This study combining non‐invasive sMRI and interpretable DL in detecting patterned sMRI changes confirmed AD pathological progression, shedding new light on predicting AD progression using whole‐brain sMRI.

Supplementary Material -Deep Learning for Brain MRI Confirms Patterned Pathological Progression in Alzheimer's Disease 1 AD-related regions in the Brainnetome Atlas [1] The abbreviations and locations of a part of the neurodegenerative brain regions associated with AD in the Brainnetome Atlas [1] are summarized in Table 1 Table 1: Detailed information on a part of neurodegenerative brain regions associated with AD in the Brainnetome Atlas [1] 2 AD Classification Performance of Ensemble 3D Convolutional Neural Networks (En- As a proof of concept, we posit the machine learning model could effectively distinguish sMRI images of AD subjects from those of the healthy controls (HC). The neuroimaging biomarker (P -score) is detected in the sMRI images on the basis of a proposed machine learning model, named Ensemble 3DCNN.
In this section, before we demonstrate the effectiveness of P -score in indicating the degree of neurodegenerative in the AD brain, the capability of the Ensemble 3DCNN to identify AD subjects from the healthy controls (HC) is verified. With the three evaluation metrics, i.e., classification accuracy (ACC), Area under the curve (AUC), and Matthews Correlation Coefficient (MCC) [2], Ensemble 3DCNN is evaluated on the validation and testing datasets retrieved from the ADNI database [3] and OASIS database [4], respectively. The experimental results are detailed in Table 2. The performance of Ensemble 3DCNN is superior to other machine learning models (PCA+SVM [5],3D-SENet [6], and wH-FCN [7]). More importantly, with Ensemble 3DCNN, an AUC value of 88.4% is achieved on the testing dataset retrieved from the OASIS database [4], which verifies the generalization ability of Ensemble 3DCNN when it is applied across databases. 3 Probability map of brain neurodegeneration in the AD population In our paper, some common neurodegenerative brain regions in AD are extracted, as listed in Table 3 in the main text of this manuscript. According to the frequency (neurodegenerative probability) of each brain region, we draw a probability map in Figure 1 to show the distribution of the common neurodegenerative regions in the brain intuitively. It can be observed that the common degenerative regions locate in the area around the amygdala, nucleus accumbens, agranular insular cortex, and hippocampus, which are approximately corresponding to the isocortex, basal magnocellular complex, and transentorhinal regions in the brain. 4 Scatter diagrams of the P -score whole and MMSE [8] values It is found that the P -score whole is more sensitively indicates less severe cognitive impairment (MMSE [9] > 20) in comparison with the existing radiomic features [10]. The correlation between P -score whole and MMSE [9] value is shown in the scatter diagram of Figure 2. It is noticeable that with the decrease in MMSE value, the smallest P -score whole value among all AD sMRI images corresponding to the same MMSE value increases. Among the subjects with P -score whole lower than 40, only a small percentage (less than 16%) of subjects are in moderate or worse cognitive impairment (i.e., (M M SE < 20)). The Chi-Squared test is adopted to verify this observation among AD sMRI images. Hypothesize that M M SE ≥ 20 is independent of P -score whole < 40. The Chi-Squared value is 7.15, which means the probability that M M SE ≥ 20 and P -score whole < 40 are irrelevant is lower than 2.5% (Freedom Degree=1). When P -score whole is rounded to the nearest integer, the Pearson's correlation coefficient between a P -score whole and the corresponding smallest MMSE value is -0.80 (p-value=3.06 × 10 −6 ), indicating a fairly strong negative relationship between P -score whole and the lower bound of MMSE. Therefore, P -score whole can be used as a preliminary measurement to exclude moderate or worse cognitive impairment. For comparison, we also measure the eight radiomic features [10] for sMRI images and plot the scatter diagram of these features and MMSE values. The eight diagrams corresponding to the eight measures are displayed in Figure 3. Unlike P -score whole , the radiomic features (such as gray matter volume, cortical area, and cortical thickness) take similar values for the AD sMRI images in different cognitive impairment levels. The reason for this result might be the individual diversity in brain size among AD subjects.  5 P -score calculation at the four levels In this paper, we derive a neuroimaging biomarker, named P -score, to assess the degree of neurodegeneration for AD subjects. P -score is a multi-level measurement that can evaluate the neurodegeneration at the four levels (cube, voxel, region, and whole-brain level). The higher P -score value represents the higher degree of neurodegeneration. 1) P -score calculation for sMRI cubes: Each sMRI image is divided into non-overlapping small cubes. P -score of each small cube is defined as the weighted AD predictive score of its corresponding base classifier with the weights from the metaclassifier. Mathematically, P -score cube (i, c) = w c · p i,c , where i and c are indexes of sMRI images and cubes, respectively. p i,c denotes the AD predictive result (softmax score) obtained by the c-th base classifier of Ensemble 3DCNN with the c-th cube in the i-th sMRI image as an input. w c represents the contribution of the c-th base classifier to AD predictive scores, i.e., the final decision of Ensemble 3DCNN, and is a parameter in the parameter vector in the meta-classifier of Ensemble 3DCNN. Here, the weights of base classifiers whose accuracies in the validation dataset are less than 70% are set to 0 to ensure the reliability of neurodegeneration measurement.
2) P -score allocation at voxel level: In each sMRI cube, we evenly distribute P -score cube to the voxels covered by the cerebral tissue. We denote P -score of the j-th voxel, i.e., voxel j , in the i-th sMRI image as P -score voxel (i, j). That is, P -score voxel (i, j) = P -score cube (i, c)/|S act i,c |. Here, |S act i,c | denotes the number of voxels covered by the cerebral tissue within an sMRI cube X i c , i.e., the c-th cube in the i-th sMRI image, where voxel j is located, i.e., voxel j ∈ X i c . 3) P -score calculation for brain regions: Based on the Brainnetome Atlas [1], the human brain is parceled into 246 regions that reflect the whole brain's anatomical and functional connections. P -score of each brain region (before normalization) is the sum of P -score voxel (i, j) of each voxel it contains. We further divided P -score region by the size of the brain region (i.e., the number of voxels in the brain region) to eliminate the effect of differences in the size of regions. For observation convenience, P -score region (before normalization) is scaled to the range of [0, 1] using the Min-Max normalization a . The calculation of P -score region (after normalization) for each brain region can be written as where i, j and k represent the index of an image, a voxel and a region, respectively; Ω k denotes a set of voxels within the k-th brain region in the Brainnetome Atlas, and |Ω k | stands for the number of voxels within the k-th brain region in the Brainnetome Atlas. N orm(·) represents a Min-Max normalization function. By default, both P -score and P -score region represent P -score region (after normalization) .
4) The whole-brain P -score: Finally, we sum up P -score region of all regions to get the whole-brain P -score to evaluate the degree of neurodegeneration at the whole-brain level.
Thus, according to those mentioned above, The P -score region at the region level is used for subsequent pattern analysis in this paper. Pseudo-codes for P -score calculation are presented in Algorithm 1.
a In the Min-Max normalization, the minimum and maximum values are 1.157×10 −5 and 0.1241, respectively. They are the maximal/minimal P -score region value (before normalization) of all brain regions in all the 720 sMRI images used for subsequent filtering with Criterion 3 and 2 and pattern analysis of AD neurodegenerative progression, shown in Table 2 in the main text of this article. Algorithm 1 P -score calculation for each sMRI image Input: The i-th sMRI image, which is divided into C small cubes, X i = {X i c }c=1,...,C ; The base classifiers of the Ensemble 3DCNN, M = {Mc}c=1,...,C ; The weights for each cube (base classifier), w = {wc}c=1,...,C ; The Brainnetome Atlas [1], Output: For the i-th sMRI image, P -score of each brain region, P -scoreregion(i) ∈ R 1×K , and the whole-brain P -score, Pscore whole (i). 1: function GET_P-score(X i , M, w, Ω)

2:
Initialize P -scoreregion(i) as a K-dimension zero vector. P -score cube (i, c) = wc · pic; {Calculate P -score for each small cube X i c in the i-th sMRI image} 6: for Voxel j within X i c do 7: if Voxel j is covered by the cerebral tissue then for Each Ω k in Ω do 13: for Voxel j within Ω k do 14: P -scoreregion(i, k) + = P -score voxel (i, j); P -score whole (i) = k P -scoreregion(i, k); 20: return P -scoreregion(i), P -score whole (i) 21: end function 6 Associations between P -score at whole-brain level and AD predictive scores obtained by Ensemble 3DCNN Since P -score is a neuroimaging biomarker derived from the results of the trained Ensemble 3DCNN, for each sMRI image, its P -score at whole-brain level should be associated with its AD probability output from Ensemble 3DCNN, i.e., AD (softmax) predictive scores. In Figure 4, we plot the scatter diagram of P -score whole and AD (softmax) predictive score obtained by Ensemble 3DCNN for 638 AD sMRI images in the analysis dataset, as shown in Table 2 in the main text of this article. Their correlation coefficient (Pearson coefficient) is 0.60 with p-value of 3.843 × 10 −63 (<0.01), indicating a moderately to strongly positive correlation. In Figure 4(b), the lower bound of P -score whole increases with the increase of AD predictive score. Moreover, many scattered points with P -score whole larger than 40 are gathered around the vertical line with the AD predictive score of 1. This is because Ensemble 3DCNN is trained by the beginning sMRIs among each longitudinal AD sMRI sequence and the outputs of Ensemble 3DCNN are compressed by the softmax function, resulting in the fact that the AD predictive scores of many subsequent AD sMRI images in a longitudinal AD sMRI sequence are almost equal to 1 via the compressive saturation response of the softmax function. We stretch the axis of the AD predictive score to observe its correlation with P -score whole in its range of [0.99, 1]. As illustrated in Figure  4(b), the moderately to strongly positive correlation between P -score whole and AD predictive score in its range of [0.99, 1] still holds and remains statistically significant. In contrast, P -score whole is based on the original output results from base classifiers of Ensemble 3DCNN, which provide relatively distinguishable and contrastive values for the AD sMRI images in a longitudinal AD sMRI sequence.

Connected component analysis of neurodegenerative brain regions
For each sMRI image, we save the neurodegenerative status of the brain regions detected with P -score as a binary vector v i ∈ R 1×K , where v i (k) = 1 if the k-th brain region is labeled as neurodegeneration in the i-th sMR image. The neighborhood around each brain region is saved as a label set, denoted as Depth First Search (DFS) algorithm [11] is employed to explore the connected components in the detected neurodegenerative brain regions, i.e., v i , for each AD sMRI image. The pseudo-codes for connected component analysis are presented in Algorithm 2. When analyzing the spatial-temporal connectivity for each AD subject, each AD subject's longitudinal 3D sMRI images are concatenated along the time axis to generate a 4D sample for connected component analysis. The binary vector v i and its corresponding neighbour N eighborList(k) now record the neurodegenerative brain regions in the 4D sample and the spatial-temporal adjacent regions of each neurodegenerative brain region, respectively. The algorithm for the spatial-temporal connected component analysis is similar to Algorithm 2. visit ←− Region {Insert Region into visit, mean that Region has been visited} 6: for Each brain region j ∈ N eighborList(Region) do For each sMRI images from the 167 AD subjects in the analysis dataset retrieved from ADNI database [3] as shown in Table 2 in the main text of this article, we label the neurodegenerative brain regions with the detection of P -score. Here, the 167 AD subjects take MRI examinations at multiple time points, forming 167 longitudinal sMRI sequences. Each sequence is denoted as } is an item-set recording the neurodegenerative brain regions in the t i -th sMRI image of the longitudinal sequence S i . r t i kt i represents the k t i -th neurodegenerative brain region in R t i i . K t i denotes the number of neurodegenerative brain regions in the t i -th sMRI image in the sequence S i , and T i is the number of sMRI images in the sequence S i . We search for the sequential patterns that frequently occur in these sequences to explore the patterns of neurodegeneration in AD progression. Three main steps are included: 1) preparing longitudinal sequences, 2) mining frequent sequential patterns, and 3) post-screening frequent sequential patterns.

1) Preparing longitudinal sequences
Medical studies [12] have indicated that AD tends to progress in a continuous and irreversible manner. For each longitudinal sequence recording neurodegenerative brain regions, we add the regions in the antecedent item-sets to the consequent item-sets to preserve the irreversibility, if necessary. For each sequence S i , we have R p i ⊆ R q i , if q > p. For the sake of simplicity in expression, regions that have appeared in the antecedent item-set are omitted in the consequent itemset for the same sequence. The sequence S i is updated as Finally, the original long sequences are separated into subsequences {S j } j=1,2,...,n , where n represents the total number of subsequences. Each subsequencẽ S j = {R p j −→ R q j } contains only one antecedent item-set R p j and one consequent item-set R q j . We save the subjects' IDs corresponding to these subsequences and use them as the inputs for mining frequent sequential patterns in the AD progression.
2) Mining frequent sequential patterns The SPADE algorithm [13] is adopted to mine the frequent sequential patterns among the neurodegenerative brain region sequences, i.e., {S j } j=1,2,...,n , where n represents the number of subsequences. The support is set to 0.05 to obtain as many frequent items as possible in an acceptable time period. More than 163,000 frequent sequential patterns are extracted.
3) Post-screening frequent sequential patterns A series of post-screening processes are applied to explore the effective neurodegeneration patterns from thousands of frequent sequential patterns. Here, the support for each frequent sequential pattern is updated with the number of subjects that this sequential pattern is applicable to. The sequential patterns with small support (< 20) are considered infrequent items, which are excluded. Finally, for two sequential patterns ({R p1 −→ R q1 } and {R p2 −→ R q2 }) that share the same supporting subject IDs and are the same in terms of consequent item sets (R q1 = R q2 ), we further analyze their antecedent item sets R p1 and R p2 . When R p1 ⊆ R p2 , only the sequential patterns of {R p2 −→ R q2 } are retained in the final result. After the post-screening, 87 frequent sequential patterns (with a supporting rate greater than 16%) that reflect the AD progression patterns are extracted from the longitudinal sequences of neurodegenerative brain regions. The most common sequential patterns are listed in Table 3. Table 3 also reports the frequency (supporting rate b ) of each sequential pattern in the OASIS database [4] to verify the generalization of the neurodegenerative patterns in the ADNI database detected via P -score. In the OASIS database [4], 26 AD subjects have longitudinal sMRI image sequences and are correctly identified by ensemble 3DCNN. Among these 26 AD subjects, 22 subjects (with 46 sMRI images), as shown in Table 2 in the main text of this article, contain at least one neurodegenerative brain region detected by P -score and are selected for mining frequent sequential patterns. Although the supporting rates of frequent sequential patterns in the OASIS database b The calculation of λ oasis for degenerative region detection in the OASIS database is the same as that in the ADNI database [3]. In specific, λ oasis = mean + 2std = 0.733, which is determined by the mean and standard deviation (std) of all brain regions' P -score among the AD sMRI images correctly identified by the Ensemble 3DCNN in the OASIS database [4]. [4] are lower than those in the ADNI database [3], most (92%, 80 of 87 frequent sequential patterns) frequent sequential patterns mined in the ADNI database [3] can be detected in the OASIS database [4] as well. The average supporting rate of the top 20 frequent sequential patterns can reach 13.33% in the OASIS database [4].   Table 3: Frequent sequential patterns of the neurodegenerative brain regions in the longitudinal sMRI image sequences of AD subjects.
Symbol # represents the elements in the antecedent.