Noninvasive Nonlinear Optical Computational Histology

Abstract Cancer remains a global health challenge, demanding early detection and accurate diagnosis for improved patient outcomes. An intelligent paradigm is introduced that elevates label‐free nonlinear optical imaging with contrastive patch‐wise learning, yielding stain‐free nonlinear optical computational histology (NOCH). NOCH enables swift, precise diagnostic analysis of fresh tissues, reducing patient anxiety and healthcare costs. Nonlinear modalities are evaluated, including stimulated Raman scattering and multiphoton imaging, for their ability to enhance tumor microenvironment sensitivity, pathological analysis, and cancer examination. Quantitative analysis confirmed that NOCH images accurately reproduce nuclear morphometric features across different cancer stages. Key diagnostic features, such as nuclear morphology, size, and nuclear‐cytoplasmic contrast, are well preserved. NOCH models also demonstrate promising generalization when applied to other pathological tissues. The study unites label‐free nonlinear optical imaging with histopathology using contrastive learning to establish stain‐free computational histology. NOCH provides a rapid, non‐invasive, and precise approach to surgical pathology, holding immense potential for revolutionizing cancer diagnosis and surgical interventions.


Note S1 The key module of Cycle-GAN network
The Cycle-GAN model 1-3 (Fig. S1b) transfers the input image  from the domain  to the output image ′ in domain  via the forward generator ().Then the newly generated image ′ is transferred back to domain  as the image ′ via the backward generator (′) .The first consistency loss is to ensure  ≈ ′.Similarly, the reference target image  can be transferred from the domain  to the fake image  * in domain  via the backward generator ().Then this image  * is transferred back to domain  as the image  * via the forward generator ( * ).The second consistency loss is to ensure  ≈  * .The two similarities define a meaningful mapping that does not exist in the paired dataset.

Note S2 Transfer learning to breast and liver cancers
We demonstrated the transfer learning capability of NOCH by applying it to the conversion of other pathological tissues to match their corresponding H&E images (Fig. S6).We employed selfcontrastive learning to translate large-field MP images of unprocessed breast tissue obtained through aspiration biopsy into histopathological morphology, utilizing the pretrained NOCH model developed for ovarian tissue.As depicted in Fig. S6a-c, the network-inferred histopathological states closely aligned with ground truth observations, despite variations in the nuclear positions between the NOCH and H&E images.This misalignment was because we examined adjacent sections obtained through continuous cutting.Notably, the fibrous structure identified by SHG and the dense connective tissue indicated by 2PA FAD in breast tissue were faithfully translated into virtual histopathological morphology at the network output, exhibiting a high degree of concordance with H&E staining.
Expanding beyond breast tissue, we also applied the network to intraoperative liver tissue (Fig. S6d-g) without necessitating retraining.Following this learning process, MP images of hepatocytes were rapidly transformed into histopathological morphology.Notably, the cancer-related features (Fig. S6f) closely resembled their counterparts in H&E staining images (Fig. S6g).These results unequivocally establish the network's transfer learning ability.

Note S3 Denoising-enabled faster pathological diagnosis
To further realize a faster workflow for cancer diagnosis and analysis, as well as reduce stored detrimental heats in the samples, we applied contrastive learning in image denoising (Fig. S10a).The network improved the image quality of 30-kHz galvo-resonant scanning (GRS) to compete with 0.47-Hz dual-galvo scanning (DGS), which enables higher speed image acquisition.Despite the indistinct 3PA morphology of the ovarian tissues, after learning, the network largely suppressed the background and noise of the input GRS images and reconstructed clear texture (Fig. S10b) in a negligible time.
The acquisition time of the GRS image (33 ms) and inference time of the network (50 ms) were much shorter compared to the DGS acquisition time (2,100 ms), which suggests that the network allows an approximately 24-fold speed-up for the slow MP imaging (Fig. S10c).Nevertheless, eventual clinical translation needs to consider practical workflow integration and costs compared to routine H&E histopathology.
The raw GRS images and reconstructed output images were compared in Fig. S10d, which demonstrates that the noisy fluctuations in the intensity profile of the GRS images, especially the 3PA NADH image with a higher noise level, were greatly reduced.The peak signal-to-noise ratio (PSNR) on average across more than 100 high-noise images exhibit an increase of approximately 7.6 dB for 3PA NADH, 5.3 dB for SHG (with 3PA FAD crosstalk 4 ), and 3.0 dB for 2PA FAD (Fig. S10e), which verifies the capability of the network in image denoising.
With the image denoising, scanning fringe artifacts (SFA) resulting from the coupling of the scan and leakage of ambient light) were also removed (Fig. S10f).We compared computational histology performance using the raw GRS images, network enhanced-GRS images, and DGS images as shown in Fig. S10g.The resulting NOCH image translated from the inferior GRS image has much obvious HTA, yet the denoising network conduces to correction of these mistakes.The histopathological morphology produced from the network enhanced-GRS image ( ID +  HT ) might even possibly have less HTA compared to the NOCH result of the DGS image in some cases.We used colour deconvolution to reveal the composition of hematoxylin and eosin in the NOCH images (donut insets in Fig. S10g) and histologically stained image (donut inset in Fig. S10h).The hematoxylin-eosin proportion of NOCH image of the enhanced-GRS (0.69) and DGS (0.77) were closer to that of the ground truth (0.71), while the proportion obtained from the original raw GRS (1.30) had a large bias toward hematoxylin that revealed a false translation.
We then performed pathological classification using the output NOCH images and the pretrained classification model.The class prediction accuracies (Fig. S10i) indicate that the inferior NOCH image of GRS was prone to misdiagnosis in cancer stages with a low class accuracy of 54%, which could be improved to 84% after the denoising, approaching the accuracy attained by NOCH of DGS.Thus, the network denoising largely reduced the classification errors for the NOCH images of GRS.A tradeoff exists between information acquisition speed (including network inference) and image quality (associated with the prediction accuracy), while the combination of the fast scanning and image denoising provides a possibility to mitigate this conditionality.

Fig. S1
Fig. S1 Comparison of NOCH results of different networks.a,The SRS spectrum of lipid (blue) and protein (green), which compose the SRS images.b, Cycle-consistent module which transforms images between the two domains.c, Self-contrastive loss associating the sampled query and its positive, in contrast to negatives within the same image.d, Cross-contrastive loss associating the sampled query and its positive, in contrast to negatives

Fig. S2
Fig. S2 Large-field virtual stained results of autopsy normal samples.a, SRS image.b, Virtual histological image by Cycle-GAN.c, NOCH image by self-contrastive learning.d, H&E histological image.White dashed lines indicate the margin.Cycle-GAN generated an inverted staining result compared to H&E, while contrast learning generated a contrast-enhanced histological slice.Scale bars, 400 μm.

Fig. S3
Fig. S3 Comparison of NOCH performance with different SRS modalities.a, SRS images (blue) at around 2845 cm -1 .b, NOCH images of a. c, SRS images (green) at around 2930 cm -1 .d, NOCH images of c. e, SRS bimodal images merged by blue and green.f, NOCH images of e using self-contrastive learning.g, H&E histological images.h, Histogram distribution of nuclear cross-sectional areas.NOCH of SRS bimodal images can achieve higher staining accuracy across different tissues compared to single modality.Scale bars, 40 μm.

Fig. S4
Fig. S4 Virtual stained nuclei of autopsy normal samples.Recurrent/residual glioblastoma IV.The nuclear contrast-enhanced images parallel with the nonadjacent histologically stained slices.Scale bars, 150 μm.

Fig. S5
Fig. S5 Comparison of the nuclear characteristics.MP image (a), virtually histological image by Cycle-GAN (b) and self-contrastive learning (c), and real H&E image (d) of Stage IC are presented in the top panel, while the corresponding nuclear extractions are shown in the bottom panel.The nuclei were extracted using StarDist 5 .e, Histogram distribution of nuclear cross-sectional areas.f, Histogram distribution of internuclear nearest neighbor distances.Scale bars, 150 μm.

Fig. S6
Fig. S6 Histopathological transfer learning.Large-field MP (a), NOCH (b), and H&E (c) images of human breast cancer samples.d, MP image of human liver cancer samples.e, The corresponding NOCH image (left) and the adjacent H&E (right) image.Black boxes indicate the zoom-in views magnified in f for NOCH and g for H&E.The NOCH images were obtained using the pretrained model for ovary.Scale bar: 100 μm for the insets in a-c and 100 μm in f,g.

Fig. S7
Fig. S7 Comparison of NOCH performance with different modalities.The histograms in the last column present the hematoxylin proportion of the tri-modal NOCH and reference for each cancer stage.Scale bars, 200 μm.

Fig. S8
Fig. S8 Nuclear segmentation of different NOCH modalities.a-h correspond to the seven modal permutations and combinations.i, Comparison of the nuclear count between different NOCH modalities and H&E stain for Stage IIIC.Scale bars, 200 μm.

Fig. S9
Fig. S9 Comparison of prediction accuracy of different models.The p values of Wilcoxon matched-pairs signed rank test between the accuracy of the model results and that of the H&E reference were presented.n = 128.The results of our contrastive learning exhibit no significantly different with the reference.

Fig.
Fig. S10 High-speed acquisition and image denoising enables faster pathological diagnosis.a, GRS and DGS configurations and the acquisition images.b, Denoised image using contrastive learning.c, Comparison of GRS acquisition time, network inference time, and DGS acquisition time.d, Input (upper) and output (bottom) images of three nonlinear modalities.The color solid line in each image refers to the line of the shown cross-section of background and noise.e, PSNR of the input and output images with mean ± SD and two-tailed Wilcoxon matched-pairs test.n = 124.f, Left to right: GRS, network enhanced-GRS, and DGS images.g, The corresponding NOCH images. ID and  HT represent the network generator of contrastive learning for image denoising and computational histology, respectively.h, H&E histologically stained image.Donuts in g and h indicate deconvolution portion of hematoxylin (blue), eosin (pink), and residual (cyan) components.i, Overall prediction accuracy for the pathological classes.A repeated measures one-way ANOVA, with the Geisser-Greenhouse correction was applied.Scale bars, 30 μm in a-d and 100 μm in f-i.

Table S2 Size, speed and accuracy of different classification models.
*These networks were trained on the H&E histological images and tested on the NOCH data.