Detecting Tumor Infiltration in Diffuse Gliomas with Deep Learning

Glioblastoma tumor recurrences often occur in brain tissue areas harboring infiltrating tumor cells, resembling healthy tissue in brain imaging. Demarcating infiltrative regions for aggressive resections is critical for improving prognostic outcomes but is challenging in neurosurgery. Herein, a multilayer sigmoid‐activated convolutional neural network (MLS‐CNN) is developed for rapidly distinguishing glioma tumor infiltration in brain tissue histology. Unlike conventional multiclass classifiers, the MLS‐CNN employs sigmoidal activation to accommodate coexisting classes within patch images. 59 811 image patches (25 807 infiltrating edge, 15 178 normal brain, 18 826 cellular tumor) from 73 brain tissue samples are extracted to train the classifier. The model achieves an accuracy of 91.70% (sensitivity: 91.62%; specificity: 91.78%) and area under the curve (AUC) of 0.964 in distinguishing infiltrating edges, outperforming the current state‐of‐the‐art Vision Transformer (ViT) (accuracy: 89.45; AUC: 0.947). The MLS‐CNN is computationally efficient, generating predictions within 11.5 s in comparison to 81.4 s for ViT. The predictions strongly correlate with In Situ Hybridization expression intensities, validating the utility of the MLS‐CNN model in spatial genomics investigations in gliomas. The robust model can therefore serve as an automatic and accurate classifier to help pathologists identify infiltrative glioma for better diagnosis and patient outcomes in brain oncology.


Introduction
Glioblastoma (GBM) is the most aggressive and common form of primary brain tumor in adults, with an average survival of only 14-16 months despite surgical removal and chemoradiation. [1]90% of GBM recurrences occur at the peritumoral brain zone (PBZ) which is known to harbor infiltrative tumor cells, commonly manifesting radiologically as noncontrast-enhanced regions around the enhanced tumor. [2,3]Indeed, recent studies have shown that a supra-total tumor removal including the PBZ is associated with improved survival. [4,5]As the PBZ may appear macroscopically normal, differentiating infiltrating cancers from normal brain tissue intraoperatively is challenging and requires advanced diagnostic techniques, such as magnetic resonance imaging (MRI) methods or fluorescencebased biomarkers. [2,4,6,7]This, however, involves expensive investment and maintenance of the imaging modalities in operating theatres.A minimum concentration of tumor cells in the PBZ is also necessary to produce detectable alterations in imaging or fluorescence uptake.Similarly, the low tumor cell content in the PBZ also poses a great challenge to neuropathologists.The challenge is further aggravated by the presence of reactive gliosis, where healthy brain tissue deceptively resembles tumor infiltration due to the increased cellularity. [8]These factors would collectively exacerbate delays that are already introduced by the traditionally manual diagnosis of frozen sections during intraoperative consults. [9]igital pathology has emerged as a promising tool to replace the traditional process of microscopic examination of glass slides.The digitization of glass slides also allows for examination through advanced computer algorithms for image classification.12][13] CNNs provide an avenue to investigate large quantities of images quickly and simultaneously, assessing their differentiability.[16] In the context of neuropathology, a transfer learning model with a DenseNet architecture as the backbone and a modified dense layer with squeezeand-excitation blocks have been used for patch-wise classification of glioma subtypes within hematoxylin and eosin (H&E)-stained tissue images. [17]These deeper network architectures have depths ranging from 16 to more than 100 layers.They can therefore be more computationally intensive to utilize and induce the risk of the vanishing gradient phenomenon, in which the gradients of network weights become severely small and hinder the optimization of the network.Vanishing gradients can be averted in models with reduced layers, but their effectiveness in the domain of neuropathology has not been demonstrated in existing literature.
More recently, Vision Transformer (ViT) models have achieved SOTA performance in several image classification tasks. [18]Leveraging the self-attention mechanism originally purposed for natural language processing, ViT models have been shown to outperform complex CNN architectures, such as ResNets. [18,19]Nevertheless, the complexity of ViT architectures and their workflow, which involves processing a number of image segments to study the relationships between them, results in expensive computational costs, limiting their applicability. [20]Simpler architectures, including a model built completely upon multilayer perceptrons, have been shown to outperform ViT-based models. [21]A study in the medical imaging domain has also shown that CNNs can outperform ViTs when trained from scratch. [22]Conversely, a recent investigation into the use of ViT-based models for brain tumor histopathology observed that a pretrained ViT outperforms CNNs for image classification. [23]However, the CNN architectures utilized, including a pretrained ResNet-50 and Inception v3, are complex, constituting 50 and 159 layers, respectively. [23]The large number of convolutional layers implies that the amount of computational processing required for both models is considerable.This reduces prediction speed and may prove especially detrimental in bedside examinations where diagnostic efficiency is key.
The study on the viability of CNNs in aiding pathologists with differentiating infiltrating edges from cellular tumors and normal brain tissue for GBMs has also not been reported in the literature.Research into the delineation of infiltrative margins in glioblastoma histology is also very limited which may be attributed to difficulties in acquiring healthy brain tissue for controls in lieu of the risk of impaired neurological functions due to excessive resections. [9]An eight-layer CNN has recently been utilized in the detection of GBMs within hyperspectral images of H&E-stained tissue. [24]A second evaluation of samples in the study identified that the nontumor regions initially selected for analysis contained infiltrating tumor cells. [24]These findings highlight the obscurity of the infiltrative region and emphasize the need for a robust detection algorithm that can differentiate it from healthy brain tissue.
In this work, we aim to develop an automated diagnostic model with deep learning as an aid for neuropathologists to delineate infiltrative tumors precisely and efficiently for better surgical outcomes in diffuse gliomas.With diagnostic speed and accuracy as our paramount objectives, we introduce a unique multilayer sigmoid-activated convolutional neural network (MLS-CNN) deep learning model for the categorical classification of H&E-stained brain tissue images.We train and evaluate the model at the patch level to ensure diagnostic precision irrespective of the size of the biopsy.Our classifier thus circumvents a major challenge presented by smaller biopsies with limited features that are hard for neuropathologists to diagnose.The MLS-CNN uses the sigmoid activation function in its final dense layer to account for the nonmutual exclusivity of classes within the images, challenging the common practice of using softmax for categorical classification involving more than two classes.To ascertain whether our model achieves stronger diagnostic accuracy than the present SOTA transformer-based architectures, we compare the MLS-CNN's performance with a pretrained ViT-based classification model.Finally, we provide a second layer of validation for the model's performance by correlating classification results with gene expression intensities from In Situ Hybridized (ISH) adjacent tissue sections.We thus further validate the model's capability in differentiating the PBZ from the tumor core and assess its utility for identifying the spatial genomic heterogeneity of gliomas.

Classification Method
The unique MLS-CNN model developed (Figure 1) consists of four 3 Â 3 convolutional layers, each followed by a 2 Â 2 max pooling layer.The feature extraction layer is not pretrained and relies solely on processing training data for the detection of low-level features.The output from the final convolutional layer is flattened and sent to a 2048 dense layer with ReLU activation, followed by a 3-neuron dense layer for categorical classification.A dropout layer with a rate of 50% is added between the dense layers to discard some layer inputs to avoid overfitting.The final dense layer in the MLS-CNN model employs sigmoid as the activation function for the nodes corresponding to each class.Although softmax is regularly used in categorical classification problems, its use implies that the classes are mutually exclusive.A single patch in our context depicts multiple nuclei and may therefore possess multiple tissue classes in its vicinity.Sigmoid accounts for this characteristic by producing independent probability outputs for each node and is thus adopted in our study.We have also reduced the number of layers in our network to prevent the vanishing gradient problem from occurring, due to the increased risk presented by our choice of activation function.For SOTA comparison, a second model with a VIT-B/16 backbone pretrained on the ImageNet database, that achieved strong classification performance on large datasets including CIFAR-10 is fine-tuned on our training dataset. [18]he model adopts the same dense layer configuration and activation functions used by the MLS-CNN following its convolutional operations for fair comparison.
Training and fine-tuning tasks for the networks were performed for 100 epochs, with batch sizes of 64 for training and 32 for validation.Both tasks were conducted at an initial learning late of 0.0001 and the models were optimized with the Adam algorithm. [25]Categorical cross-entropy was chosen as an appropriate loss function for three classes, as follows: where p i is the prediction and y i is the ground truth label.

Tissue Samples, Data Acquisition, and Preprocessing
This research project was developed in compliance with ethical guidelines and was approved by the Ethics Committees of our institutions (National Healthcare Group DSRB Ref: 2019/00068).H&E-stained whole slide images (WSIs) of adult infiltrating gliomas were obtained from the Department of Pathology, Tan Tock Seng Hospital (TTSH), Singapore (n = 36) and the Ivy Glioblastoma Atlas Project (Ivy GAP) online database (n = 38). [26]The histopathological images and annotations from the Ivy GAP portal are accessible via https://glioblastoma.alleninstitute.org/ish.Seventeen H&E-stained WSIs containing portions of normal brain tissue acquired from resections of noninfiltrating brain tumors (such as meningiomas, hemangioblastomas, metastatic carcinomas, etc.) were obtained from the Department of Pathology, National University Health System (NUHS), Singapore to serve as study control.Labels for normal, infiltrative, and cellular tumor regions for samples acquired from TTSH and NUHS were derived through crossexamining and aggregating annotations performed by the two experienced neuropathologists (B.C.Wu, C.L. Tan) from the Department of Pathology at NUHS, with each pathologist blinded to the other's annotations.Sections for training were chosen only if a consensus between the annotations presented by both pathologists was ensured.Annotations for infiltrative and cellular tumor regions for images from the Ivy GAP database were acquired from the public database itself.It should be noted that tissue sections annotated as "leading edge" within the database are classified as infiltrating edges in this study due to the observed abnormal cellularity within the sections.
WSIs from the Ivy Glioblastoma project were downloaded in the ".jpeg" format, with a downsampling factor of 2 selected for processing allowance within existing computational capabilities.WSIs from both hospitals were scanned using the Hamamatsu NanoZoomer S60 digital slide scanner and extracted in the ".ndpi" format.Qupath, an open-source suite for digital pathology analysis, was used to extract ".jpeg" images from regions of interest based on pathologist annotations. [27]Depending on the original resolution, the images were downsampled by a factor of %2 or 4. The resulting resolution of all images after downsampling is approximately 1.0 μm pixel À1 .The removal of background artifacts such as dirt and white balancing was done through the GNU Manipulation Program (GIMP).
We randomly allocated 20% of the WSIs from each institute for testing, ensuring that there was no overlap at the patient level for samples within the training and testing cohorts.Training data were also augmented through horizontal and vertical flipping to increase its diversity, to improve the performance of the trained networks.
To assess the utility of MLS-CNN classifier in studying the genomic properties of the peritumoral and core tumor zones, 169 tissue sub-blocks with cellular tumor or infiltrative tumor classes present in at least 50% of the area within original annotations were acquired from the Ivy GAP database.Expression intensities for ISH samples within the dataset were extracted through color masking and normalized to a range of [0,1] after grayscale conversion.Intensity-based patches from the WSIs were then separated into 200 Â 200 pixel patches, with mean intensities and therefore expression levels calculated for each patch.To obtain ground truth annotations for the patches, the trained MLS-CNN classifier's predictions on image patches from adjacent H&E tissue sections were collated.Mean intensities for the ISH patches were then compared with H&E classification outputs to study the variation of gene expression levels between the tumor core and PBZ.

Evaluation Indices
Our study adopted accuracy, sensitivity, specificity, Matthews correlation coefficient (MCC) and area under receiver operating characteristic curve (AUC) as evaluation indexes.The calculations for accuracy, sensitivity, and specificity are as follows: where TP (true positive) represents the proportion of image patches of the positive class that are correctly classified, FP (false positive) represents the proportion of image patches of the negative class that are incorrectly classified, TN (true negative) represents the proportion of image patches of the negative class that are correctly classified, and FN (false negative) represents the proportion of image patches of the positive class that are incorrectly classified.Accuracy, sensitivity, and specificity and AUC were calculated on a binary basis (i.e., one class versus the other two) resulting in three sets of values.ROC plotting and AUC calculation were performed in MATLAB.MCC was measured using the Scikit-Learn library in Python as a single value that simultaneously accounts for predictions for all three individual classes.

Model Deployment
To demonstrate the deployment of the MLS-CNN and the independent probability outputs generated in a bedside scenario, a graphic user interface (GUI) was developed (Supporting Video).When a whole tissue image is uploaded through the GUI, patches are automatically generated and classified with the trained model to generate independent sigmoid probabilities for each class.With the probabilities generated, several visualization options are made available through the interface, which is built using the Gradio package in Python. [28]Users can either visualize patch-level probability heatmaps for each class or choose to display a classification map that showcases the class with the highest sigmoid probability for each patch.To provide an additional layer of explain-ability for the classification outputs, thresholding options are also available in the interface.When enabled, threshold sliders allow the user to set custom thresholds to identify positive patches for each class, as well as positive patches for combined class thresholds.

Results
The

infiltrating þ cellular tumor).
There is no overlap at the patient level for samples used for training and testing the models.Classification results from the trained models are output as probability values ranging from 0 to 1 for each class.As the annotations do not include a confidence score for each class, the outputs from the nodes in the final dense layer were compared and the class with the highest probability was taken as the predicted class for evaluation.For a better representation of MLS-CNN classifications in the spatial domain for easier interpretation, classification maps were generated in Python.The maps can also be displayed through the developed GUI (Supporting Video).The highest probable class is represented as a colored overlay for patches not belonging to the background of the image, identified through the average pixel values of each patch.
Figure 2 depicts classification result maps for the whole tissue images assigned a single annotation throughout the spatial domain.The MLS-CNN classifier exhibited strong (MCC %0.824) concordance with annotations across all three classes.
Figure 3 shows classification result maps for whole tissue images with coexisting annotations of cellular tumor and infiltrating edge classes, thereby allowing the evaluation of the classifier's precision in establishing margins between the tissue types.The MLS-CNN classifier demonstrates a high classification performance (MCC = 0.721) for different pathologic brain tissue types.
Table 1 shows the comparison of the diagnostic accuracy, sensitivity, and specificity of utilizing the trained MLS-CNN model and the fine-tuned VIT-B/16-based model on the testing cohort.Patch-level prediction results are output as probability values for each class.The accuracy achieved by the MLS-CNN model was 91.70% (91.62% sensitivity, 91.78% specificity) in the identification of infiltrating edges, 97.04% (97.78% sensitivity, 96.31% specificity) in the identification of normal brain tissue, and 91.62% (87.39% sensitivity, 95.84% specificity) in the identification of cellular tumors.The MLS-CNN thus achieves stronger performance across all classes, with the VIT-B/16-based model only achieving 89.45% accuracy (91.85% sensitivity, 86.00% specificity) in the identification of infiltrating edges, 95.53% accuracy (85.71% sensitivity, 96.10% specificity) in the identification of  Examination into classification maps revealed further strengths of our method.For example, the granular layer of the cerebellum that may be interpreted as cellular tumor and normal brain tissue with reactive gliosis that may be interpreted as an infiltrating edge were both correctly classified as normal brain tissue (Figure 2A,B).As shown in Figure 2C,D, the MLS-CNN classifier was also able to achieve almost consistent classification outcomes despite the distinct variation of stain qualities between samples, which tends to occur due to differing staining conditions or protocols between various pathology departments.This validates the robustness of the model in lieu of the diverse range of stain qualities in the multi-institute training cohort that it has learnt from.
To demonstrate the model's applicability in genomic investigations, 3235 ISH images representing 330 genes with 1949 adjacent H&E-stained tissue images were acquired from 27 patients through the Ivy GAP database.The images were chosen based on original annotations from the database with the criteria that at least 50% of the image space contains infiltrative tumors, cellular tumors, or a combination of both classes.A total of 6 genes were chosen for study on the premise of the larger specimen numbers available within the acquired cohort.These include CD44 (100 ISH specimens from 23 patients), HIF1A (99 ISH specimens from 23 patients), ID1 (98 ISH specimens from 23 patients), EZH2 (98 ISH specimens from 23 patients), OLIG2 (97 ISH specimens from 23 patients), and PDGFRA (99 ISH specimens from 23 patients).Images for the ISH specimens were tiled into 200 Â 200 pixel patches, with mean expressions for each patch obtained through color deconvolution.Mean expression intensities for the image patches were then compared with the MLS-CNN's classifications of patch images from adjacent H&E-stained sections, to measure expression levels for each gene within cellular tumors and infiltrating edges.Figure 5 shows examples of the spatial representation of the classification results and detected expressions in brain tumor.
Our analysis revealed significantly stronger expression levels for six genes (CD44, HIF1A, ID1, EZH2, OLIG2, and PDGFRA) within the cellular tumors (Mann-Whitney U: p-value < 0.01).Violin plots for the tile-based results are displayed in Figure 6.Overlaps between the plots for both classes as well as outliers with significantly stronger gene enrichment however suggest the presence of intratumoral heterogeneity, especially within the tumor core.
The selected specimens in Figure 5 possess both cellular tumor and infiltrative tumor classifications within the spatial domain and showcase differences in expression levels that are consistent with the results in Figure 6.While it is observed that the tumor core is more strongly enriched for the ID1, HIF1A, and PDGFRA genes, it is also shown that the expression strength varies within the region.A further examination of the ISH image for PDGFRA in Figure 5 indicates that expression strength is enriched only where specific features such as blood vessels are present.A correlation of patches of 200 Â 200 pixel dimensions is thus shown to provide sufficient precision to morphologically delineate these features while ensuring that each patch possesses sufficient biological information for CNN classification.

Discussion
In this work, we addressed the hypothesis that the MLS-CNN classifier can accurately distinguish the infiltrating edge of GBMs from normal brain tissue and cellular tumor tissue in H&E-stained brain tissue images.To validate this, the performance of a trained four-layer MLS-CNN with sigmoid activation was analyzed.Our results are promising, with the MLS-CNN model achieving a 91.70% patch-level accuracy in differentiating infiltrating edges from normal brain and cellular tumor and a good correlation (MCC = 0.824) between classifications and original annotations for all three classes.Our deep learning model with H&E-stained images shows that glioma infiltration can be distinguished at the patch level to precisely remove infiltrating tumor residuals for better survival outcomes.
CNNs are advantageous over traditional machine learning methods that employ handcrafted features in lieu of the automated nature of learning features within images for classification.Transfer learning CNN models are popular in image analysis and classification as the use of pretrained weights allows for reduced training durations and strong performance even with small training datasets.However, these complex pretrained networks may not always yield stronger results. [29]In the diagnosis of actinic keratosis, a shallow CNN comprising two convolutional layers has been used to achieve a diagnostic accuracy of 92.5% and outperformed popular and transfer learning models such as AlexNet and Googlenet. [30]Higher performance has also been achieved in the classification of breast cancer histopathological images with a six-layer CNN (Accuracy = 91.28%at 40Â magnification) than with a pretrained network (Accuracy = 89.12 at 40Â magnification). [31]Similarly, we have demonstrated that training from scratch on a four-layer shallow model yields a strong test performance (Accuracy = 91.70%,AUC = 0.964) without the need for pretraining.
We have also demonstrated that the MLS-CNN outperforms a SOTA ViT-based model that was pretrained on the ImageNet database (Accuracy = 89.45%,AUC = 0.947).Transformer-based models do not possess the advantage of inductive bias that CNN networks do, thereby performing relatively poorly on smaller datasets. [32]The large data required for a transformer-based network to perform optimally may be difficult to acquire in the medical domain. [22]In brain histology, obtaining healthy control tissue is exceptionally challenging due to the potential for detrimental effects on a patient's neural function resulting from aggressive resections beyond the tumor margin. [9]This may render transformer-based architectures ineffective in the neuropathological domain.The perceived stature of transformer models as SOTA architectures, however, highlights the importance of our study, in which we demonstrate that a CNN model trained from scratch is able to extract features and predict more effectively on our curated dataset.
Transformer models are also computationally expensive and may be difficult to deploy in medical institutions when the computational power is unavailable, especially in workstations provided to each pathologist. [20]The deployment of such models on systems without powerful and expensive graphic processing units (GPUs), which are thereby not specifically catered to deep learning applications, may severely hamper computational efficiency.In our study, we found that the MLS-CNN model takes approximately 11.5 s to evaluate on our testing cohort, achieving significantly faster prediction speeds than the ViT-B/16-based model, which takes approximately 81.4 s.Intriguingly, a recent study involving whole-slide brain tumor classification adopted a ViT-L/16 model pretrained on ImageNet for patch-based feature extraction, which received inputs of 1024 Â 1024 pixels. [23]ine-tuning the ViT-L/16 for a single epoch on a RTX 3060 GPU (12 GB memory) with a maximum allowable batch size of 1 took approximately 16 h.In contrast, it was possible to train the MLS-CNN model for 100 epochs in less than 4 h with the same hardware.This efficiency would prove valuable in retraining scenarios with increased amounts of acquired histological data.Notably, we also found that the MLS-CNN performs more efficiently than complex and popular CNN networks.Our model predicts almost twice as fast as a pretrained ResNet50 (21.8 s) and almost 4 times as fast as a pretrained VGG-19 network (46.1 s).The demonstrated prediction efficiency of the MLS-CNN would thus prove especially useful when rapid diagnosis is required for frozen tissue sections in bedside consults, given that the model has already demonstrated superior diagnostic accuracy.
Our study also investigates the performance of sigmoid activation in the final output layer of a CNN model.The sigmoid activation function is conventionally used in binary classification problems, with softmax typically used for categorical classification.Recently proposed models, including a MobileNetV2-based classifier that demonstrated 91-92% accuracy in the classification of breast cancer images and an Inception-V3-based model that achieved AUC values of more than 0.889 in the classification of rhabdomyosarcoma have adopted softmax activation in the final layer. [33,34]While this may be relevant when the classes are mutually exclusive, the presence of infiltration implies that tissue images can contain both benign and malignant cells.Multiple classes may therefore coexist within the images.Our model thus addresses the inherent heterogeneity by producing independent sigmoid-derived confidence measures for each class.The visualization of the independent confidence scores has been demonstrated in the Supporting Video.With custom thresholds set for the outputs, it is possible to identify patches with mixed classes.The heatmap and custom threshold-based visualizations empower clinicians to make well-informed decisions concerning a diagnosis or surgical intervention.To our knowledge, such an approach has not been explored in the A key advantage in this study is presented through the derivation of control tissue from resections of noninfiltrating brain tumors.Recent studies utilizing deep learning on glioma tissue images have designated tissue adjacent to the tumors as nonneoplastic. [24,35]It was found that normal samples originally selected for analysis could not be considered nontumor in lieu of their close proximity to the tumor area. [24]This finding further exemplified the infiltrating nature of GBM beyond the macroscopic and radiologic tumor epicenter, negating their viability in developing reliable classifiers and acting as reference standards for imaging methods or clinical procedures.In the study that employed multiphoton microscopy with deep learning techniques for the histopathological assessment of gliomas, control tissue was obtained from brain autopsies on individuals without neurologic diseases. [36]While this approach avoids the presence of tumor infiltration in control tissue, it does not provide a realistic representation of the reactive normal tissue in brain tumor cases.In contrast, the use of adjacent brain tissue from nondiffuse brain tumor samples as control tissue in our study avoids the risk of unwanted infiltration, but also addresses the challenge of reactive gliosis.Tissue sections with this abnormality are difficult for pathologists to distinguish from low-grade diffuse glioma (WHO Grade II) or the infiltrating edge of GBMs. [8]In our study that utilizes H&E staining alone, regions containing reactive gliosis were not misdiagnosed as regions of infiltrating edge and successfully classified as normal tissue (Figure 2B).This affirms our MLS-CNN classifier's potential as a diagnostic tool in generating classification references rapidly to aid pathologists in tumor identification for challenging cases.
We have further demonstrated the generation of easily interpretable classification maps from probability outputs of the trained model.Patch-level examination of tissue sections introduces the advantage of precision, thereby allowing the delineation of margins between different pathologic tissues.Patch extraction and subsequent augmentation techniques further allow for larger training datasets and more discerning features to be learned by a classifier, thereby increasing classification accuracy. [37]The extraction and processing of more samples from resections in glioblastoma in future studies may help ascertain the margins between infiltrating edges and normal tissue within GBM cases and further examine the precision of the trained model.Extracting patches from samples obtained from multiple institutes has also improved the robustness of the MLS-CNN model, enabling it to achieve strong performance despite dissimilar stain qualities which were encountered in this study, as shown in Figure 2C,D.This finding is concordant with results presented in a similar study that yielded strong results (AUC of 0.9021) in identifying invasive breast tumors, having been trained on multicenter data with varying stain intensities. [38]he MLS-CNN's present automated patch level-based examination capability also implies that resected tissue samples with a range of sizes can be examined accurately.This is important as smaller samples may not possess the visible transitions of cellular tumor to infiltrating edge, or infiltrating edge to normal brain tissue.These transitions provide pathologists with visual cues such as changes in cellularity or morphological irregularity, alerting them to tumor infiltration in WSIs.Where such cues are unavailable, our rapidly generated classification maps by the MLS-CNN model would mitigate diagnostic challenges for pathologists as reference tools for tumor identification.
H&E staining currently remains the gold standard and gives quantitative and structural cues to identify diseases in tissues at the microscopic level, with a precision that cannot be achieved by ex vivo radiomic imaging.One weakness of the technique is the need for biopsies and tissue processing.[41][42] Images generated from the label-free SRS technique possess similarities to H&E staining and may thus be diagnosed in a similar manner.To improve the efficiency of diagnosis within the operating setting, particularly where a skilled neuropathologist may not be immediately available, an automated method like the proposed MLS-CNN model will certainly be applicable with imaging equipment, especially because histopathology-based classifications would correspond well with small field of view (FOV) images, thereby allowing the real-time delineation of tumor margins for better resections and recurrence prevention.As H&E images continue to serve as the gold standard for the provision of ground truths in evaluating label free methods, automated classifiers may also be of aid to researchers in assessing their technology development quickly and accurately.
We also demonstrated the use of the MLS-CNN's patch-based classification for the genetic analysis for ISH images with ground truths from classifications on adjacent H&E-stained sections.Of the 330 genes available in the Ivy GAP database, our investigation revealed notably stronger expression levels for the CD44, HIF1A, ID1, EZH2, OLIG2, and PDGFRA genes within cellular tumors over infiltrative tumors, suggesting that the increased expression levels of these markers can be utilized to aid the delineation of both zones, as they have important biological implications on the behavior of GBMs.The expression of HIF1A (Hypoxia-inducible factor 1-alpha) has been shown to correlate with the increased vascular permeability and low oxygenation content that are understood to promote tumor infiltration. [43,44]As shown in Figure 5 for the HIF1A gene, the region classified by the MLS-CNN as cellular tumor is shown to be enriched for HIF1A and is surrounded by a significant area of tissue classified as infiltrating tumor, affirming that tumor cells proliferate to elude the adverse microenvironment within the tumor core.We have also observed the existence of genetic heterogeneity in both the tumor core and peritumoral edema, as illustrated statistically and visually in Figure 5 and 6, which is consistent with the observation of molecular heterogeneity of GBM tissues using the advanced imaging techniques. [42,45,46]With the larger datasets contributing to the brain tumor landscape being built, an automated MLS-CNN tool that can provide ground-truth annotations based on the gold standard will certainly facilitate the accurate genomic studies of brain tumors, as demonstrated by our workflow.
In summary, we have developed the MLS-CNN classification model for accurately distinguishing tissue regions with infiltrating glioblastoma cells from normal brain tissue and cellular tumors.Our model adopts sigmoid as opposed to the conventional softmax to account for the nonmutual exclusivity in patch images, which presents a unique challenge in the histopathology of diffuse gliomas.The automated MLS-CNN method has achieved a high AUC and diagnostic accuracy.We have also shown that the MLS-CNN, trained from scratch, outperforms a pretrained SOTA transformer-based model in both classification efficacy and efficiency.In the clinical setting for WSIs, classification maps provide readily interpretable representations of model predictions that can be utilized for rapid bed-side diagnosis.Our patch-based workflow can assist pathologists in examining challenging tissue samples with abnormalities like reactive gliosis or small samples lacking sufficient visual cues for infiltrative tumor identification.Finally, we have shown that MLS-CNN classifier can be utilized to ensure that spatial genomic evaluations of gliomas are supported by accurate ground truths in an automated fashion, suggesting its potential of facilitating the diagnostic workflow in neuropathology and neuro-oncology for better brain tumor patient outcomes.

Figure 1 .
Figure1.Architecture and potential workflow incorporating the MLS-CNN as a diagnostic aid for pathologic reference and genomic studies.The MLS-CNN comprises four 3 Â 3 convolutional layers and 4 2 Â 2 max pooling layers.Outputs from feature extractor for are flattened and processed with a 2048 neuron dense layer, dropout with a rate of 0.5 and a dense layer with 3 sigmoid-activated neurons.Classifications are released as probability values for each neuron, which are then used to determine the predicted class for each image.The predictions can be generated and visualized through a GUI as combined or single-class classification maps, or classification maps based on user-defined thresholding per-class.By cross-referencing expression intensities obtained from ISH images with the MLS-CNN's classifications for adjacent H&E images, an accurate genetic comparison between the peritumoral and core tumor regions can also be performed.
200 Â 200 pixel image patches were then extracted from the training and test images through the MATLAB environment.This resulted in 59 811 image patches available for training (25 807 infiltrating edge, 15 178 normal brain, 18 826 cellular tumor) and 11 435 image patches available for testing (6728 infiltrating edge, 630 normal brain, 4078 cellular tumor).Images were normalized through rescaling by a factor of 255 to the range [0,1].

Figure 2 .
Figure2.Generated MLS-CNN classification maps for whole tissue samples and selected 500 Â 500 pixel regions of interest with one annotation: A,B) normal brain; C,D) infiltrating edge; E,F) cellular tumor throughout the tissue space.The maps were generated from patch-level predictions produced by the MLS-CNN model and provide easily interpretable diagnostic aids for pathologists while showcasing the precision that can be accomplished at the patch level.The MLS-CNN's classifications strongly correspond with annotations across all three classes (MCC % 0.824).
normal brain tissue, and 91.22% (82.27% sensitivity, 96.18% specificity) in the identification of cellular tumors.Table1also showcases the duration taken by both models to predict on the test dataset.While the MLS-CNN took only 11.5 s for the prediction run, the VIT-B/16 model took 81.4 s.We further generated receiver operator characteristic (ROC) curves to evaluate the performance of the models on the three classes.As shown by the ROC plot in Figure 4, the MLS-CNN classifier demonstrated strong classification performance (area under the curve (AUC) = 0.964 (95% confidence interval, 0.956-0.972)) in differentiating infiltrating edges from cellular tumors and normal tissue.The ROC plots again demonstrate the superiority of the MLS-CNN over the VIT-B/16-based model, with the latter achieving an AUC of 0.947 (95% confidence interval, 0.940-0.954).

Figure 3 .
Figure 3. Classification maps for whole tissue samples annotated with infiltrating edge and cellular tumor regions.Samples displayed are obtained from A,B) TTS and annotated by an experienced neuropathologist and from the C,D) Ivy GAP with annotations provided through the repository.Samples labeled as "leading edge" (blue annotation) within Ivy GAP the database are also classified as infiltrating edges (pink annotation) due to the cellularity observed by the pathologists.The MLS-CNN's classifications correspond well with original annotations (MCC = 0.721) for different pathologic brain tissue types.

Figure 4 .
Figure 4. ROC curves, comparing the classification performances of the MLS-CNN and VIT-B/16-based models.Abbreviations used: IT = infiltrating tumor, CT = cellular tumor, NB = normal brain, ROC = receiver operating curve, AUC = area under the curve, CI = confidence Interval).The plots show that the MLS-CNN model accomplishes stronger performance (AUC = 0.964) than the VIT-B/16 model (AUC = 0.947) in differentiating infiltrating tumors from cellular tumors and healthy brain tissue.

Figure 5 .
Figure 5.The generated ISH intensity maps, plotted alongside classification maps generated from the MLS-CNN's classification of adjacent H&E-stained sections.Abbreviations used: ID-1 = inhibitor of DNA binding 1, HIF1A = hypoxia inducible factor 1 subunit alpha, PDGFRA = platelet-derived growth factor receptor alpha.The results showcase stronger expression levels for the ID1, HIF1A, and PDGFRA genes within areas demarcated by the MLS-CNN's predictions as cellular tumors over infiltrative tumors.
99% and 96.97% at peak performance during the training cycle.The trained models were then evaluated on a test set consisting of 11 435 (6728 infiltrating edge, 630 normal brain, 4077 cellular tumor) image patches from 18 samples (3 normal brain, 4 infiltrating edge, 4 cellular tumor, 7 database for SOTA comparison.Training and fine-tuning of the models were performed over 100 epochs, with a call-back function implemented to coerce weights for the highest validation accuracy achieved throughout the training cycle.The training and validation accuracies (infiltrating edge vs normal brain and cellular tumor) obtained by the MLS-CNN model are 99.67% and 96.82%, with the VIT-B/16-based model achieving 98.

Table 1 .
Classification performance comparison of the MLS-CNN and pretrained ViT-B/16 model on patch images in the test cohort.Percentages for prediction of each classification task are indicated in parenthesis.The MLS-CNN model achieves stronger classification accuracy across all three classes (infiltrating edge: 91.8%, normal brain: 96.39%, cellular tumor: 92.83%) than the VIT-B/16-based model (infiltrating edge: 89.45%, normal brain: 95.53%, cellular tumor: 91.22%).The model also achieves a prediction speed that is 69.9s faster than the speed of the VIT-B/16 model.