Learning to see the invisible: A data‐driven approach to finding the underlying patterns of abnormality in visually normal brain magnetic resonance images in patients with temporal lobe epilepsy

Abstract Objective To find the covert patterns of abnormality in patients with unilateral temporal lobe epilepsy (TLE) and visually normal brain magnetic resonance images (MRI‐negative), comparing them to those with visible abnormalities (MRI‐positive). Methods We used multimodal brain MRI from patients with unilateral TLE and employed contemporary machine learning methods to predict the known laterality of seizure onset in 104 subjects (82 MRI‐positive, 22 MRI‐negative). A visualization approach entitled "Importance Maps" was developed to highlight image features predictive of seizure laterality in both the MRI‐positive and MRI‐negative cases. Results Seizure laterality could be predicted with an area under the receiver operating characteristic curve of 0.981 (95% confidence interval [CI] =0.974‐0.989) in MRI‐positive and 0.842 (95% CI = 0.736‐0.949) in MRI‐negative cases. The known image features arising from the hippocampus were the leading predictors of seizure laterality in the MRI‐positive cases, whereas widespread temporal lobe abnormalities were revealed in the MRI‐negative cases. Significance Covert abnormalities not discerned on visual reading were detected in MRI‐negative TLE, with a spatial pattern involving the whole temporal lobe, rather than just the hippocampus. This suggests that MRI‐negative TLE may be associated with subtle but widespread temporal lobe abnormalities. These abnormalities merit close inspection and postacquisition processing if there is no overt lesion.


| INTRODUCTION
Epilepsy is a common and serious neurological disease that can have a devastating effect on a person's life if not adequately treated. However, one-third of patients continue to have frequent seizures despite optimal medical therapy. 1,2 Focal epilepsy accounts for a significant proportion (around two-thirds) of those cases that do not respond to medication, and the temporal lobe is the most common site of onset of focal seizures. If focal epilepsy does not respond to medication, neurosurgical procedures can be considered. If the underlying epileptogenic abnormality can be located in a patient's brain and surgically removed, there is a good chance that a patient can have freedom from seizures postoperatively; up to a 82% chance of postoperative remission for at least 1 year and 41% sustained complete seizure freedom have been demonstrated. 3 The identification of an underlying abnormality on magnetic resonance imaging (MRI) is a key goal of the surgical workup, but such abnormalities are only demonstrated on conventional MRI in about two-thirds of cases (the so-called MRI-positive cases). 2 This leaves one-third of pharmacoresistant patients with no visually identifiable target for neurosurgery (the MRI-negative cases). This MRI-negative subgroup of patients represents a significant clinical challenge and is far less likely to experience seizure freedom after surgery. 3,4 In these patients, further investigations are necessary to localize the epileptogenic focus. This often involves intracranial electroencephalographic (EEG) monitoring, which carries a significant risk to the patient as well as considerable cost. It would be very beneficial to develop means of localizing underlying cerebral abnormalities using noninvasive methods such as MRI and scalp EEG alone.
Detection of brain abnormalities on magnetic resonance (MR) images using contemporary methods such as machine learning is an emerging and promising area of research. [5][6][7][8][9][10] Focke et al generated maps of gray matter voxel relevance across the brain by visualizing the weights assigned to each feature by a support vector machine (SVM) classifier trained to lateralize temporal lobe epilepsy (TLE) in 38 MRI-positive patients. 6 These relevance maps provided an interesting demonstration of the distribution of morphological gray matter changes found in MRI-positive TLE patients.
Keihaninejad et al reported a distribution of brain abnormalities using an SVM classifier trained to lateralize TLE, and also included MRI-negative cases. 7 However, only regional brain volumes were used in the analysis, and no other relevant imaging parameters were included such as T1 or fluid-attenuated inversion recovery (FLAIR) signal intensity. Keller and Roberts reviewed voxel-based morphometry approaches to reporting the patterns of abnormality associated with TLE, limiting the assessment to the measurement of brain volume changes. 8 Cantor-Rivera et al studied correlation and analysis of variance-based feature selection methods and presented a collection of anatomical regions of interest that were found to be relevant for the purpose of TLE lateralization, 9 but did not compare MRI-positive and MRI-negative cases.
Several different approaches have shown promise in lateralizing seizure origin in TLE, although mostly in MRIpositive cases. In the case of MRI-negative TLE, if a trained classifier is demonstrating reasonable predictive capability, then these apparently normal images must contain abnormalities, albeit subtle or poorly understood ones. Determining these abnormalities would provide valuable insight into why visual inspection might fail in the MRI-negative cases and may guide more principled and accurate visual image interpretation in the future.
In this study, we trained a random forest classifier (RFC) on image features from multimodal MRI (T1, T2, and FLAIR) and carried out feature importance measurements, demonstrating the patterns of abnormality across the temporal lobe in both MRI-positive and MRI-negative cases. We created visualizations of these results, which we call "Importance Maps," that show the spatial pattern of abnormalities across the temporal lobes in these two patient groups.

| MATERIALS AND METHODS
The study cohort included patients diagnosed with TLE by a consultant epileptologist. The patients underwent routine clinical scans including T1, T2, and FLAIR volumes on a 3T Signa HDx scanner (General Electric Company) at the Epilepsy Society MRI Unit, Chalfont St Peter between 2009 and 2013. Patients were included if they had provided written, informed consent and clinical consensus at the multidisciplinary epilepsy surgery meeting or careful review of notes and investigations (OFB, GPW) demonstrated findings consistent with unilateral TLE. The study was approved by the National Hospital for Neurology and Neurosurgery and the

Key Points
• This study was conducted to determine the covert patterns of abnormality in patients with unilateral TLE and visually normal brain MR images • Multimodal brain MRI and contemporary machine learning methods were used, and a visualization approach entitled "Importance Maps" was developed • Covert abnormalities in MRI-negative TLE had a spatial pattern involving the whole temporal lobe, rather than just the hippocampus, and assisted lateralization  Table 1, and detailed clinical information for the MRI-negative cases is given in Table S1.
For the MRI-positive group, 68 of 82 patients had imaging findings consistent with hippocampal sclerosis, whereas 5 of 82 patients had findings consistent with focal cortical dysplasia. The remaining 9 of 82 patients had a visually recognizable abnormality within the temporal lobe that allowed lateralization of the seizure focus but without a confident underlying diagnosis of the lesion from the radiologist.
The numbers of patients in the MRI-positive and MRInegative groups who had surgery and histology were 55 of 82 and 10 of 22, respectively. International League Against Epilepsy outcomes at 12 months after surgery are shown in Table 1. In the MRI-positive group, 34 of the 55 patients who had surgery were seizure-free at 12 months, whereas this number was 5 of 10 for the MRI-negative patients.

| Workflow
Our analysis used T1-weighted, T2-weighted, and FLAIR MR image volumes. The T2-weighted and FLAIR volumes were rigidly registered to the T1, and each T1-weighted image volume was parcellated into its constituent anatomical regions using the Geodesic Information Flows (GIF) method. 11 A junction map (JM) was also generated from each T1-weighted volume using SPM12 12 and locally developed MATLAB (MathWorks) scripts. This allowed us to incorporate into the analysis regions of gray-white matter boundary blurring, an image feature often seen in focal cortical dysplasia. 13,14 Example volumes are shown in Figure 1.
The following measurements were then made for regions of the temporal lobe parcellated by GIF: This provided a set of 117 features per patient. These features were used to train an RFC 15 comprising an ensemble of 5000 decision trees to predict the lateralization of the seizure origin. The importance of each of the 117 features described above for determining the seizure laterality was evaluated by calculating the reduction in Gini impurity provided by a split in that feature (weighted by the chance of reaching that node) averaged across all trees in the ensemble. Put simply, this measured the degree to which each image feature was able to separate the two classes (left TLE and right TLE) in the dataset. Once these importance measurements were calculated, visualizations of the results were generated.
To ensure the validity of the feature importances determined above, we evaluated the predictive capability of these features using SVMs. The type of kernel used for the MRI-positive cases was linear, whereas for the MRInegative cases the radial basis function was chosen due to laterality detection in MRI-negative cases being a significantly more difficult problem. We used a different type of classifier (SVM) from that used for determining the feature importances (RFC) to ensure the generalizability of the results. Hyperparameters of the SVM classifier were optimized using the grid-search method, and its capability to lateralize the seizure origin was assessed using bolstering, 16 a technique that provides an unbiased estimation of a classifier's performance by carrying out multiple train/test runs using multiple random partitions of the dataset into training and testing sets. We carried out 100 train/test runs, and the dataset was randomly partitioned into four folds during each run (three folds used for training, and the remaining fold used for testing). A binomial distribution was assumed on the collection of performance figures across subjects, and then the Agresti-Coull method of interval estimation 17 was used to produce confidence intervals (CIs).

| RESULTS
The performance of the SVM classifier at lateralizing the seizure origin was as follows: a maximum area under the receiver operating characteristic (ROC) curve of 0.981 (95% CI = 0.974-0.989) was achieved in the MRI-positive and 0.842 (95% CI = 0.736-0.949) in the MRI-negative cases ( Figure 2). The corresponding classification accuracies (the agreement between the laterality determined using our method and the clinical lateralization) were 0.926 (95% CI = 0.900-0.951) and 0.756 (95% CI = 0.636-0.877), respectively. The most important features for predicting seizure laterality in the MRI-positive and MRI-negative cases are as shown in Tables 2 and 3, and visualization of these features is displayed in Figure 3.  Four observations were made. First, hippocampal volume changes seen in MRI-positive cases were essentially absent in the MRI-negative group. Second, the typical hippocampal signal abnormalities seen in the MRI-positive cases, and which are often focused on by radiologists performing this task visually, 14 were less important for predicting seizure laterality in the MRI-negative cases. Third, signal abnormalities within the amygdala, and signal abnormalities arising from within the white matter of the temporal lobe, along with inferior regions of the cortex provided useful information in the MRI-negative group. Last, JMs, although not informative for the MRI-positive cases, were helpful in predicting seizure laterality in MRI-negative patients.
A re-review of the MRI-negative images, guided by a neuroradiologist, revealed subtle signal abnormalities in 8 of the 22 cases (Table 3), supporting our findings (Table S2 and Figure S1).

| DISCUSSION
The results of our study support the hypothesis that subtle abnormalities exist in MR scans in MRI-negative TLE, which may be detectable using contemporary machine learning approaches. The areas under the ROC curve of 0.981 (95% CI = 0.974-0.989) and 0.842 (95% CI = 0.736-0.949) obtained in determining seizure laterality in MRI-positive and MRI-negative cases, respectively, are similar to those reported in other studies addressing TLE lateralization. 7,9 However, a direct comparison cannot be made due to differences in the study cohorts.
Maximum areas under the ROC curve for lateralization in the MRI-positive and MRI-negative cases were obtained using the top three and 52 imaging features, respectively ( Figure 2). The higher number of imaging features required for lateralization of the MRI-negative cases is indicative of more diffuse, and perhaps more complex, imaging abnormalities in MRI-negative epilepsy.
The abnormality patterns demonstrated in the Importance Maps are the most significant results in our study. A comparison between them suggests that there is a fundamental difference in the pattern of imaging abnormalities within the MRI-negative cases compared to the visually recognizable patterns in the MRI-positive cases. In other words, the MRI-negative abnormalities are not simply a more subtle version of those seen in MRI-positive patients.
This observed difference is in accord with other investigations. For example, MRI-negative TLE subjects have been demonstrated to exhibit significantly different connectivity patterns from those of MRI-positive TLE subjects, 18 suggesting a underlying difference in the pathophysiology between the two groups.
A prominent difference in our study was the contrasting significance of the hippocampus and the other temporal lobe regions in the MRI-positive and MRI-negative cases, respectively. The hippocampus is conventionally focused on by radiologists reporting MRI scans in patients with TLE, as hippocampal sclerosis is a common cause of TLE. 2 Hippocampal sclerosis is generally visually recognized from a reduced hippocampal volume and a higher T2/FLAIR signal arising from within the hippocampus. 14 It is of note that these features top the list of important features in the MRI-positive cases (Table  2). However, these features appear to be less important in MRI-negative cases. Instead, multiple imaging features arising from widespread temporal lobe areas seem to be much more important in predicting seizure laterality in MRI-negative TLE. Of note are the signal abnormalities arising from the amygdala, the temporal white matter, and the inferior regions of the cortex (Table 3, Figure 3). Our results suggest that lateralization of MRI-negative cases in clinical practice may require a comparison of the signal intensities ("mean" features), and signal heterogeneities ("standard deviation" features) between the two brain hemispheres for widespread regions of the temporal lobe, in particular the amygdala, the temporal white matter, and the inferior regions of the cortex. Our results provide a novel insight into why human readers may not visually identify the patterns of abnormality in certain images. The results suggest that, in cases that are difficult to lateralize using the conventional approach, it would be useful to carefully examine images for possible abnormalities and asymmetry within the amygdala, the temporal white matter, and inferior cortical regions of the temporal lobe. Postacquisition processing of these regions in such cases may also prove useful and help to unveil the subtle changes that elude visual inspection. F I G U R E 3 Importance Maps for regional volume asymmetry (top), mean intensity asymmetry (middle), and intensity standard deviation asymmetry (bottom) features in each image type across the temporal lobe for magnetic resonance imaging (MRI)-positive (left) and MRInegative cases (right). The intensity of red within the maps signifies the importance assigned to each region. FLAIR, fluidattenuated inversion recovery; JM, junction map; L, left; MR, magnetic resonance; R, right There are limitations to our approach. First, it is not possible to compare the pattern of detected image abnormalities with the histological ground truth in every case, because surgery is not always performed. Second, the use of simple imaging features such as regional volumes and signal variations remain fairly basic when compared to the myriad of more sophisticated image features that are available in the field of medical image computing. 19,20 However, this is also an advantage, as the use of basic image features results in Importance Maps that are more readily interpretable. More complicated features could be used, but a balance between complexity and interpretability needs to be struck. It is our aspiration that our results could be used by radiologists (perhaps aided by some simple postacquisition processing) to guide the interpretation of MRI-negative images, and this sort of translation would only be possible with fairly simple image features like the ones used in this study.
A radiological re-review of the MRI-negative cases in our study already revealed subtle signal abnormalities aiding lateralization in 8 of the 22 cases. We expect further work in this area, particularly appropriate postprocessing methods enabled by machine learning, could help highlight signal abnormalities, and enable higher rates of successful radiological assessment.

| CONCLUSION
This study has demonstrated that covert abnormalities exist in multimodal MR scans in MRI-negative TLE, and that these abnormalities follow a different pattern from those in MRI-positive TLE. These findings could aid clinical reading of MRI-negative images in the future. The study also introduced the concept of Importance Maps, which may provide an elegant and principled way to characterize patterns of image abnormality that have previously eluded visual detection. They could be used by radiologists to help guide image evaluation in these more challenging cases.