A Deep Learning System to Predict Recurrence and Disability Outcomes in Patients with Transient Ischemic Attack or Ischemic Stroke

Ischemic strokes (IS) and transient ischemic attacks (TIA) account for approximately 80% of all strokes and are leading causes of death worldwide. Assessing the risk of recurrence or functional impairment in IS and TIA patients is essential to both acute phase treatment and secondary prevention. Current risk prediction systems that rely on clinical parameters alone without leveraging imaging data have only modest performance. Herein, a deep learning‐based risk prediction system (RPS) is developed to predict the probability of stroke recurrence or disability (i.e., deep‐learning stroke recurrence risk score, SRR score). Then, Kaplan–Meier analysis to evaluate the ability of SRR score to stratify patients at stroke recurrence risk is discussed. Using 15 166 Third China National Stroke Registry (CNSR‐III) cases, the RPS's receiver operating characteristic curve (AUC) values of 0.850 for 14 day TIA recurrence prediction and 0.837 for 3 month IS disability prediction are used. Among patients deemed high risk by SRR score, 22.9% and 24.4% of individuals with TIA and IS respectively have stroke recurrence within 1 year, which are significantly higher than the rates in low‐risk individuals. Deep learning‐based RPS can outperform conventional risk scores and has the potential to assist accurate prognostication in stroke patients to optimize management.

are based on clinical parameters, such as age, sex, and medical history, and do not include other valuable prognostic parameters, such as medical image data, potentially compromising their performance. [19] Many studies have emphasized the prognostic value of imaging features in TIA or IS patients. By introducing ipsilateral stenosis of internal carotid artery and acute diffusion-weighted imaging (DWI) hyperintensity to ABCD2 scoring, the ABCD3-I score better predicted early stroke recurrence after TIA, [4a] which was further validated by meta-analysis (the ABCD3-I score improved the identification of high-risk TIA patients compared with ABCD2 at 90 days of follow-up, c-statistics from 0.61 to 0.76). [20] Large artery stenosis and acute infarction on DWI have been shown to be independent risk factors for recurrent stroke after TIA and minor IS. [20,21] Multiple acute infarcts, multiple infarcts of different ages, watershed infarcts, and other image features have also shown excellent prognostic performance in IS patients as part of the RRE90 score. [9] Thus, image-based risk prediction models can contribute to the prognostic assessment of TIA or IS patients. [19e] Artificial intelligence (AI) has already shown promise for complex neuroimaging analyses, with many stroke studies successfully applying AI to triage radiology workflow. [22] For example, deep learning models can screen head CT images for acute neurological events [22a] and predict the final area of IS lesions from baseline images. [22b] Machine learning models using imaging features were developed to identify stroke patients requiring acute thrombolysis [22g] and to predict stroke outcomes and risk of recurrence. [23] Although AI-equipped prognostic algorithms show potential for improving clinical decision-making and outcomes, they have generally been limited by their development using small study cohorts and underutilization of brain imaging and expert radiology. More sophisticated and predictive AI frameworks developed on a large populations of stroke patients are required.
To address these issues, here, this study proposes a deep learning-based risk prediction system (RPS) to predict stroke recurrence and disability in patients after an index TIA or IS. The RPS model innovates in several ways. First, RPS is developed based on a large-scale clinical cohort, the Third China National Stroke Registry (CNSR-III), [24] which includes 15 166 IS and TIA patients. Second, it combines image features and clinical information. Third, RPS leverages graph neural networks and the attention mechanism to explore the spatial correlations of stroke lesions. Fourth, based on the probability output from RPS (termed deep-learning stroke recurrence risk score and SRR score), a new simplified stroke recurrence risk stratification system is proposed. Finally, RPS is compared to and outperforms several existing clinical risk prediction models (e.g., SPI, ESRS, RRE-90, ABCD3-I, and ASTRAL).

Overview of the Approach
The proposed RPS has two key modules ( Figure 1): an imagebased risk prediction module (RPS-I) and a joint risk prediction module that includes clinical data (RPS-J). RPS-I consists of a 3D U-Net for segmenting infarct areas from DWI b1000 images (b ¼ 1000 s mm À2 ) and a deep multi-parameter graph network (DMG-Net) that combines DWI b0 images (b ¼ 0), DWI b1000 images, and lesion segmentations as inputs to predict recurrence/disability risk. RPS-J fuses the deep image features and clinical information (Table S1, Supporting Information) to predict stroke recurrence/disability risk in IS and TIA patients. Several existing popular used clinical risk prediction models (e.g., SPI, ESRS, RRE-90, ABCD3-I, and ASTRAL) and a clinical information (Table S1, Supporting Information, same as clinical information used in RPS-J)-based logistic regression (called Clinical_logist) for comparison. All models were evaluated to predict the risk of stroke recurrence/disability in TIA and IS patients over both the short and long terms.

Study Population
This study was conducted on CNSR-III, [24] a nationwide, prospective, hospital-based registry of 15 166 TIA and IS patients (1,020 TIA and 14 146 IS patients) with stroke recurrence and disability outcomes. Stroke recurrence was defined as a new stroke (IS or hemorrhagic stroke) after an index TIA or IS. [25] Disability was defined as a neurological condition with a modified Rankin Scale (mRS) between 3 and 6, as previously. [13] Details of the image analysis protocols and methods were previously reported. [24] The stroke recurrence risk was assessed over 1 year at five timepoints (14 days, 1 month, 3, 6 months, and 1 year), and the recurrence status was used as the label for recurrence risk prediction. The patients' disability outcomes were assessed at 3, 6 months, and 1 year after admission. Of the 15 166 stroke patients recruited to CNSR-III, 13 012 had brain magnetic resonance imaging (MRI) scans collected during hospitalization. The patient cohort characteristics are summarized in Table 1, and the patient enrollment process is shown in Figure 2.

RPS for TIA Recurrence Risk Prediction
RPS-I outperformed other conventional methods in predicting both short-and long-term recurrence risks in TIA patients. For 14 day recurrence prediction, RPS-I achieved an area under the receiver operating characteristics (ROC) and curve (AUC) value of 0.795 (95% CI 0.739-0.886) ( Figure 3a and Table 2) compared to an AUC of 0.525 (95% CI 0.343-0.651) for ResNet18. For 1 year prediction, the AUC of RPS-I was 0.667 (95% CI 0.634-0.739)] (Table 2 and Figure S1, Supporting Information), significantly higher than that of the ResNet18 (p < 0.005).
Clinical_logist model used clinical information to achieve competitive performance, with 14 day AUC 0.818 (95% CI 0.757-0.853). Combining deep image features and clinical information in RPS-J further improved the prediction of recurrence in TIA patients. RPS-J achieved a 14 day and 1 month predictive AUC of 0.850 (95% CI 0.827-0.865) and 0.846 (95% CI 0.807-0.873), respectively (Table 2). Compared with ABCD2 and ABCD3-I scores derived from the same data set, the 14 day recurrence risk AUC of ABCD3-I was 0.737 (95% CI 0.734-0.742) and the 14 day recurrence risk AUC of ABCD2 was 0.625 (95% CI 0.566-0.660) (see Table 2 for complete results).
Overall, RPS-I consistently outperformed conventional risk prediction models and RPS-J provided further prognostic accuracy over both the short and long terms (p < 0.005), but particularly over the short term compared to existing clinical risk prediction models (p < 0.005; Figure 3a and Figure S1, Supporting Information). Over longer-time frames, the predictive accuracy decreased: an AUC of 0.850 (95% CI 0.827-0.865) at 14 days for RPS-J but 0.737 (95% CI 0.705-0.773) and 0.689 (95% CI 0.634-0.739) by 6 months and 1 year, respectively (Table 2 and Figure S1, Supporting Information), a feature common to all predictive models.
The SPI-I, SPI-II, ESRS, and RRE-90 risk scores and Clinical_logist model were implemented to predict recurrence risk in IS patients for comparison. At 14 days, for example, the AUCs of SPI-II, RRE-90 and Clinical_logist were 0.552 (95% CI 0.536-0.565), 0.514 (95% CI (0.503-0.523) and 0.633 [0.616, 0.663], respectively (Table 2), confirming the superiority of RPS over conventional risk scores and ResNet. Furthermore, RPS-J AUCs were significantly higher than those of RPS-I for both short-and long-term predictions, with the performance gap again decreasing over time (Table 2).
The ASTRAL score [13] was developed based on registry data to predict 3 months functional outcomes in patients with acute IS.  The predictive performance of ASTRAL and Clinical_logist is shown in Table 2 for comparison. Although the AUCs of RPS-I were slightly lower than those of ASTRAL for predicting disability in TIA patients and competitive with Clinical_logist, RPS-J outperformed conventional clinical risk scores and RPS-I (p < 0.005).
RPS-I outperformed both conventional clinical scores and ResNet18, and RPS-J was most predictive, significantly outperforming conventional clinical risk scores and RPS-I (p < 0.005). RPS showed a clear advantage for predicting disability risk in patients with IS, and the RPS was relatively stable for both short-and long-term disability risk predictions.  Table S1 and Figure S5-S7, Supporting Information). Only age, NIHSS admission score, and diagnosis at admission were correlated with deep image   [26] was next applied to generate an interpretable map. Specifically, features' maps from the final convolutional layer of the deep learning model were used to generate an activation map, which highlighted important regions relevant to prediction. Representative examples are shown in Figure S8, Supporting Information.

Conclusions and Discussion
Here, this study developed a deep learning-based RPS to predict the risk of stroke recurrence/disability in IS and TIA patients. Using DWI b0 and b1000 images, it developed an image-based risk prediction module (RPS-I) and subsequently a joint risk prediction module (RPS-J) that combined deep image features and clinical parameters. When compared to conventional risk prediction scores, RPS-I was more predictive than existing scoring systems. Benefiting from the combination of deep image features and clinical information, RPS-J was most predictive of both recurrence and disability over both the short and long terms.
The system has several advantages. First, RPS-J can overcome performance bottlenecks of conventional risk prediction scores by combining medical image and clinical parameters, providing neurologists with useful information to stratify patients at high risk of stroke recurrence or disability. Second, RPS-I achieved highly competitive predictions based on MR images alone. This model could complete the analysis of DWI MR images in less than 1 min to produce accurate prognostic risk assessments. When clinical parameters cannot be collected in time and only DWI is available, RPS-I can quickly predict risk in individual patients, greatly improving the efficiency of prognostic risk assessment in the emergency setting. Finally, the risk stratification tool could be used by clinicians to optimize assessing the risk of future recurrence or functional impairment in IS and TIA patients. Patients with TIA or IS identified by the algorithm as being at high risk could be hospitalized to improve the outcome of stroke, with patients at lower risk followed up less frequently to reduce the economic burden.
To visualize RPS-I decisions for interpretation, this study applied Gradient-based Class Activation Mapping (Grad-CAM) [26] to generate heatmaps highlighting important regions relevant to the prediction. As shown in the generated activation maps ( Figure S8, Supporting Information), the network accurately focused on lesional areas, which can be attributed to the application of graph convolutional networks and a double attention mechanism in DMG-Net. It also investigated the activation maps generated from ResNet18, and although ResNet18 captured some lesions, areas of noise may have impacted model performance.
It also interpreted the predictive value of RPS-J by Pearson correlation analysis, which showed that deep image features are correlated with manually extracted image information. However, correlations between deep image features, fundamental clinical information, drug information, and medical history were very weak. The relatively low correlation is implying that the clinical information and the deep image features are of different dimensions or aspects, so the two may have some complementary properties. These may also be the reasons for the improvement of the performance of RPS-J after combing deep image feature and clinical information, compared with RPS-I and clinical information.
An important advantage of this study was developing RPS using a large-scale national cohort of TIA and IS in CNSR-III. CNSR-III has complete baseline information and >95% of patients completed a 1 year follow-up assessment, assuring the effectiveness and robustness of the model. [24] Moreover, in contrast to the previous stroke cohort studies, CNSR-III focuses on imaging data, collecting DWI MR scans from 13 012 IS and TIA patients during their hospitalization. Most patients also had a complete stroke etiological evaluation (including intracranial artery imaging, extracranial artery imaging, cardiac rhythm examination, and cardiac structure imaging) with centralized interpretation. [27] Furthermore, all the risk prediction models were centrally evaluated according to baseline data instead of sub-center evaluation, which ensured data consistency.
There were also some limitations. First, the number of TIA patients in the cohort was relatively small (895 and 882 for recurrence and disability risk prediction, respectively). Second, although the predictive accuracies of the models were good, the interpretability of deep learning-based models is still limited and no risk factors directly beneficial to clinical assessment were discovered. Finally, despite the promising predictive performance, there is still room for improvement.
In conclusion, here, this study proposes a deep learning-based RPS for clinical prognostication after an index TIA and IS. Using DWI images and clinical parameters, the RPS significantly outperformed conventional risk scores. Leveraging AI to combine medical image and clinical parameters is a new direction to overcome the shortcomings of the performance of traditional risk prediction scores. www.advancedsciencenews.com www.advintellsyst.com

Experimental Section
Data Set: CNSR-III is a nationwide, prospective, hospital-based registry of 15 166 patients with TIA or IS recruited from 201 participating hospitals in 26 provinces and municipalities in China between August 2015 and March 2018. Patients aged ≥18 years were enrolled in CNSR-III if they had a TIA or IS within 7 days of symptom onset. Baseline data on demographic characteristics, cardiovascular risk factors, medical history, physical examination, laboratory tests, medical treatments, and pre-stroke modified Rankin Scale (mRS), etc., were collected after admission through face-to-face interviews by neurologists at participating hospitals, following a standardized data collection protocol developed by the steering committee. All patients underwent routine imaging during hospitalization, including brain MRI with DWI (1.5 T or 3.0 T), intracranial artery imaging, extracranial artery imaging, cardiac rhythm examination, and cardiac structure imaging. All data were collected and analyzed centrally at Tinatin Neuroimaging Center of Excellence, Beijing Tiantan Hospital. In CNSR-III, all patients were followed up at 3, 6, and 12 months and then annually up to 5 years. Details of the image analysis protocol and methods were previously reported. [24] Among the 15 166 IS and TIA patients recruited to CNSR-III, brain MRI scans were collected during hospitalization in 13 012 patients. After quality control and image review (DWI sequence must be included), the recurrence RPS was developed using data from 11 975 patients (mean age 62.4 AE 11.2 years), of whom 895 had TIA (mean age 61.3 AE 11.6 years) and 11 080 had IS (mean age 62.5 AE 11.2 years). Among these patients, 3.5% TIA and 4.4% IS patients had recurrences within 14 days. The disability RPS was developed using data from 11 627 patients (mean age 62.4 AE 11.2 years), of whom 882 had TIA (mean age 61.2 AE 11.5 years) and 10 745 had IS (mean age 62.5 AE 11.2 years). At 3 months followup, 3.1% TIA and 14.5% IS patients had disabilities.
MR Imaging Acquirement and Preprocessing: Brain MRI examinations were completed using 1.5 T or 3 T MRI scanners. Sequences included T1-weighted images, T2-weighted images, fluid-attenuated inversion recovery (FLAIR) images and DWI. All patients were reviewed for the availability of DWI sequences with b values of 0 and 1000 s mm À2 (DWI b0 and DWI b1000). Both DWI b0 and DWI b1000 sequences were preprocessed as described in the following section.
All DWI b1000 scans from CNSR-III were registered to the FLAIR template [28] (GG-FLAIR-366, available at http://brainder.org). It used the FLIRT [29] tool available within the FSL package to align the scans with respect to the GG-FLAIR-366 template. DWI b0 scans and lesion segmentation maps were registered to the individual's DWI b1000 scan using affine linear transformation with twelve degrees of freedom (12 DOF). A careful manual check of the registered images revealed that the automatic registration was reasonable for a large majority of CNSR-III cases. For cases that were not well-registered, the degrees of freedom were changed from 12 to 7 and the cost function was set to mutual information. Considering that there is no universal registration method for all MRI scans, after the two-step process, scans with poor preprocessing quality were excluded. The quality assessment was evaluated by two specialized radiologists with significant clinical experience. The main reasons why images were excluded due to poor pre-processing quality include poor scan quality, severe head-motion, severe artifacts, MRI incomplete, distorted image, etc. After linear registration, all images had an isotropic spatial resolution of 2 mm 3 with voxel size 91 Â 109 Â 91 and all axial images were centrally cropped to 72 Â 96 Â 72.
After image registration, brain voxel intensities were normalized to mean ¼ 0 and standard deviation ¼ 1. It then performed background removal, where all voxels in background regions outside of the skull were set to À1 to ensure uniform background intensity. Voxels with intensities less than the threshold (set to 0) were regarded as background.
3D U-Net Lesion Segmentation Model: U-Net and its derived models have been widely used in medical image segmentation tasks and have achieved excellent performance. [30] Here, it trained a modified 3D U-Net model to segment stroke lesions from DWI b1000 images to evaluate the presence and location of infarcts. Apart from the classic U-Net, to preserve spatial information, for each slice, the upper and lower slices were concatenated to form a 3D patch. It randomly selected 4345 DWIs from the SPACE data set [31] and used 80% for training and 20% for validation of the lesion segmentation model after manually labeling the lesional areas. SPACE data set [31] is a multi-center, randomized, double-blind, placebocontrolled, parallel-group, superiority trial. All patients included in the current study routinely completed image evaluation during hospitalization as follows: brain MRI (3.0 T or 1.5 T) and DWI and FLAIR sequence must be included as the Digital Imaging and Communications in Medicine (DICOM) format. In general, it used the SPACE data set without special preprocessing operation, only converting the medical images from DICOM format to NIFIT formation. The network was trained to optimize the binary cross-entropy loss. The Adam optimizer was used by setting the initial learning rate and weighted decay coefficient to 0.001 and 0.0001, respectively. The network attained dice similarity coefficients (DSC) of 0.976 and 0.871 for the training and validation sets. The trained model achieved a DSC of 0.831 in 600 manually annotated samples randomly selected from CNSR-III. In the inference stage, all DWI b1000 images from the CNSR-III data set were fed to the trained 3D CNN lesion segmentation model to obtain the stroke lesion segmentation results. The training process was performed across 100 epochs with a batch size of 12 on a single NVIDIA GTX1080 graphic processing unit using the PyTorch framework.
Deep Multi-Parameter graph network (DMG-Net): To fully explore the multi-parameter images (DWI b0, DWI b1000 and segmentations) for better prediction, this study proposes the deep multi-parameter graph network (DMG-Net). It designed the network by mimicking human diagnostic reasoning, interpreting the images based on infarct distribution and pattern. Specifically, it divided the brain image according to the blood supply to the brain, i.e., the anterior, middle, and posterior cerebral arteries. Images were first divided into the left and the right hemispheres. Then, along the sagittal axis, each hemisphere was divided into three patches covering the distributions of the cerebral arteries. Most importantly, the divided patches contained border ("watershed") zones vulnerable to infarction. The division process generated six partly overlapping patches for each brain image. It also used another patch down sampled from the whole image to preserve global information. In summary, seven image patches for each data modality were used for each patient. The network extracted visual features from those patches and reasoned using the spatial relationship derived from these features. The network architecture of DMG-Net is shown in Figure S9a, Supporting Information. The network had four main parts: the backbone feature extractor for both DWI and segmentation; the higher-level feature extractors with separate weights for each patch; the patch graphs and subsequent graph attention block; and the last feature fusion and fully connected layer. Joint risk prediction system (RPS-J): The proposed joint RPS (RPS-J)'s inputs had two main components, deep image features extracted by RPS-I and clinical information (demographic information and medical history). For each case, all images were first processed by RPS-I, a 65 525dimensional feature vector formed in the bilinear pooling step of DMG-Net. The deep image features were obtained by reducing the dimensionality of this high-dimensional feature vector to 16 dimensions via principal component analysis. The clinical information was featurized using one-hot encoding for all categorical features. Age and other numerical features were used as a float number normalized to [0,1] based on the range in the training set. These two components were concatenated into a joint feature and fit into the logistic regression to make recurrence or disability risk predictions.

Supporting Information
Supporting Information is available from the Wiley Online Library or from the author.