Noninvasive Imaging Through Scattering Media with Enlarged FOV Using PSF Estimations and Correlations

Noninvasive imaging through scattering media using memory effect (ME) normally suffers from the failure of imaging when the object is beyond the ME region, limiting its field of view (FOV). Herein, a method to enlarge the FOV is proposed by utilizing point‐spread‐function (PSF) estimations and correlations. For the first time, to the best of the knowledge, it is successfully demonstrated that a whole object or multiple objects with the right relative positions and orientations, although they are beyond the ME region, can be noninvasively well reconstructed using the proposed method, which suggests that it can realize noninvasive imaging through scattering media with enlarged FOV. Further, it is experimentally verified that the method can promote one of the previous works, making it capable of noninvasive super‐resolution imaging through scattering media beyond the ME region. The technique paves the way to noninvasively visualize objects behind scattering media beyond the ME region with enlarged FOV and unprecedented clarity.


Introduction
Scattering media, which disturbs the light field from an object making the image to be a speckle pattern, is normally regarded as a hinder to traditional imaging modalities. However, the seemingly noise-like speckle pattern does have information about the object behind the scattering media. Recently, to overcome or utilize light scattering, lots of efforts have been made; as a result, many effective imaging techniques have been developed, which can be mainly categorized as follows. Imaging techniques using ballistic light, such as optical coherence tomography [1][2][3][4] and two-photon microscopy, [5,6] in which light scattering is considered to be detrimental, require a sufficiently large amount of ballistic photons such that imaging through moderately scattering samples is achievable. Imaging techniques utilizing scattering light involve wave-front shaping, [7,8] transmission matrix measurement, [9,10] optical-phase conjugation, [11,12] and utilization of the memory effect (ME). [13][14][15][16] For imaging techniques using wave-front shaping or transmission matrix measurement, a lot of iterations or measurements are needed before the imaging process, which makes it time-consuming. Imaging techniques using optical-phase conjugation need not only a "guide-star" behind scattering media but also interference devices resulting in their sensitivity and complexity. Imaging techniques utilizing ME include deconvolution-based [17][18][19][20][21][22][23] or speckle autocorrelation-based [24][25][26][27][28] techniques. For the former, a point spread function (PSF), which is usually obtained by placing a "guide-star" behind the scattering samples, is needed making this type of method cannot work in a noninvasive way, limiting its applications. The latter one, that is, a speckle autocorrelationbased technique, removes the requirement of a "guide-star" for prior measuring the PSF, which is capable of imaging through scattering samples noninvasively. However, it still suffers from the failure of imaging when the object is beyond the ME region, limiting its field of view (FOV), as the deconvolution-based imaging technique does.
Methods reported in refs. [29][30][31][32] can only get rid of the FOV limitation for imaging through scattering media by the deconvolution-based technique, however suffering from invasive PSF measurement or the necessity of the prior information.
To solve this problem, several ingenious approaches were proposed. [33][34][35][36][37] Based on the work in ref. [38], Wang et al. utilized Fourier spectrum guessing and iterative energy-constrained compensation for separating the autocorrelations of dual targets exceeding the ME region to image them through scattering media, [33] however, such a method may not be capable of reconstructing more than three complicated targets. And the more objects there are the more iterative calculations are needed. Assuming one of the two objects kept stationary while the other one was gradually moving, [34,35] He et al. utilized the speckle-differential-based method to DOI: 10.1002/adpr.202300100 Noninvasive imaging through scattering media using memory effect (ME) normally suffers from the failure of imaging when the object is beyond the ME region, limiting its field of view (FOV). Herein, a method to enlarge the FOV is proposed by utilizing point-spread-function (PSF) estimations and correlations. For the first time, to the best of the knowledge, it is successfully demonstrated that a whole object or multiple objects with the right relative positions and orientations, although they are beyond the ME region, can be noninvasively well reconstructed using the proposed method, which suggests that it can realize noninvasive imaging through scattering media with enlarged FOV. Further, it is experimentally verified that the method can promote one of the previous works, making it capable of noninvasive super-resolution imaging through scattering media beyond the ME region. The technique paves the way to noninvasively visualize objects behind scattering media beyond the ME region with enlarged FOV and unprecedented clarity.
image the two isolated objects beyond the ME region through scattering media, [36] which needs the special assumption that is normally hard to be satisfied in practice. By knowing the number of objects, Li et al. employed the independent component analysis for dealiasing, decomposition, and speckle patterns separating to image four isolated targets exceeding the ME region through a scattering layer, [37] however, it still needs prior information about the number of targets behind scattering media and suffers from lacking the right relative positions and orientations of the reconstructed multi-targets. Thus far, how to do noninvasive scattering imaging of a whole object or multiple objects with the right relative positions and orientations without the need for prior information is still an important but challenging issue.
To solve this issue, here, we propose a method to image an object beyond the ME region through scattering media using PSFs estimations and correlations, in which the object beyond the ME region is assumed to be illuminated and scanned separately, which could be satisfied in the near further since noninvasive optical focusing through scattering media using the fluorescence signal has been realized. [39][40][41][42] The method only required a camera to capture speckle patterns produced by each illuminated part of the object behind a scattering medium. The relative position and orientation of each part of an object can be determined, allowing noninvasive imaging through scattering media beyond the ME region and thus with enlarged FOV (Figure 1b). More interestingly and inspiringly, the method can be applied to further develop our previous work, [43] making it capable of noninvasive super-resolution imaging through scattering media beyond the ME region ( Figure 1c). The technique provides an effective solution to see through turbid media such as frosted glass or translucent biological tissues with enlarged FOV and high clarity.

Principle
An object O beyond the ME region ( Figure 2a where O i is the i th part of the object and N is the total number of the different parts. Each part is assumed to be illuminated and scanned (Figure 2a1-a5 or 4a). The light emitted from each O i passes through scattering media and produces a speckle pattern S i recorded by a camera (Figure 2c). Although the whole object O is beyond the ME region, the size of each O i can be illuminated within the ME region; therefore, the autocorrelation of each speckle pattern S i (Figure 2c) is approximately equal to the autocorrelation of the corresponding O i . [25] Then, the image O 0 i of each O i can be reconstructed from its autocorrelation by a phase-retrieval algorithm [38] (Figure 2a0 or 4a) while losing the exact relative positions and orientations between O i and O iþ1 in the object O since O 0 i is only reconstructed from the autocorrelation: Superimposing the retrieved images of the different parts cannot obtain the exact whole image of the object (Figure 2h).
If the Euclidean distance between the adjacent parts O i and O iþ1 can be less than the diameter of the ME region, which means the corresponding adjacent ME regions have some overlap with each other (Figure 2a0 or 4a). Then, the exact relative positions between O 0 i and O 0 iþ1 in the image of the object O can be determined by calculating the cross correlations with the speckle pattern S i and S iþ1 . Despite all this, phase-retrieval algorithms cannot distinguish the exact solution with the flips via a central inversion. Inspired by our previous work, [43] the PSFs for the different ME regions in which the different O i sit there can be estimated by deconvolution: in which ★ indicates the 2D correlation calculation, and positionfg gives the 2D coordinate of the location of the correlation peak. Therefore, the exact relative orientations of a series of retrieved images O 0 j can be determined for a corresponding series of illuminated parts O j by deconvolution of the adjacent speckle pattern S jþ1 with the determined PSF 0 j . Following the relative positions suggested by the shift vectors calculated with Equation (1), an image (Figure 2g) of the whole object beyond the ME region can be stitched by superposing all individually retrieved images Figure 1. a) Imaging of a target consisting of five characters beyond the memory effect (ME) region behind a scattering medium by the speckle autocorrelation-based technique directly; [25] Noninvasive imaging b) or super-resolution imaging c) the target by our method.

Experimental Section
The schematic of the setup is presented in Figure 2b. A light-emitting diode (LED, Thorlabs, M625L4) with a center wavelength of 625 nm illuminates onto the digital micromirror device (DLP4500, 912 Â 1140 pixels), which is used to simulate the hidden target objects beyond the ME region. The light emitted from each part of a hidden object passes through a 4.5 mm iris and a scattering medium, and then generates a speckle image recorded by the camera (Andor Zyla 4.2 PLUS, 2048 Â 2048, pixel size 6.5 μm). Here, a ground glass (DG10-120, Thorlabs, optical diffuser) is utilized as the scattering medium. The iris is used for controlling the aperture of the optical system, which determines the grain size of the recorded speckle patterns, and guarantees the speckle patterns can be sampled by the camera appropriately. The distance from the object to the ground glass and that from the ground glass to the camera are u = 94 mm and v = 80 mm, respectively. With the aforementioned experimental setup, we demonstrate the proposed method for noninvasive imaging of multiple characters beyond the ME region through scattering media. The five characters are illuminated and scanned, respectively, as shown in Figure 2a. The speckle patterns recorded by the camera are shown in Figure 2c. Then, the phase-retrieval algorithm can be used to retrieve the images (Figure 2d) since each character is within the corresponding ME region. The direct superimposition of the five retrieved characters is shown in Figure 2h, which loses the relative positions and orientations among the five retrieved characters. To obtain the right relative positions, PSF estimations and correlations need to be applied. The estimated PSFs are calculated by deconvolution of the speckle patterns (Figure 2c) with the five retrieved characters (Figure 2d), as shown in Figure 2e and 3a1-a5. As the red-dashed circle in Figure 2a0 indicates, there are overlaps of the ME regions among the adjacent characters; as a result, the relative positions of these characters are equal to the shifts of their corresponding PSFs indicated by the shift vectors as shown in  Figure 3b1. With such a shift vector, the corresponding retrieved characters can be stitched to their correct relative positions. It can be seen in Figure 3c1 that the relative position between character "O" and character "T" is calculated correctly. After calculating the leftover shift vectors of the estimated PSFs (Figure 3b2-b4), the rest of the characters are stitched with the right relative positions (Figure 3c2-c4). It shows that the cross-correlation operations can be used for multiple characters, and all the shift vectors are shown in Figure 3b5. The exact relative orientations of the retrieved characters can be determined for the corresponding illuminated parts by deconvolution of the adjacent speckle pattern with a determined PSF. Finally, as shown in Figure 3c5, Figure 2. Principle of imaging an object beyond the ME region through scattering media. a0) The object consists of five characters beyond the ME region, the red-dashed circles indicate the ME region. a1-a5) the illuminated area. b) Experimental setup. c) Raw camera images. d) Reconstruction results of the corresponding (c) using the phase-retrieval algorithm. [38] e) The estimated point spread functions (PSFs) from the raw camera images (c) and their corresponding reconstruction results (d). f ) The relative positions obtained from autocorrelation and cross correlation with the estimated PSFs (e). g) The reconstructed object using our method. h) Superimposing the reconstructed images of the corresponding (d).
www.advancedsciencenews.com www.adpr-journal.com the whole five characters beyond the ME region can be reconstructed. This suggests that multiple objects beyond the ME region through scattering media can be imaged by the proposed method and the correct relative positions and orientations among the multiple objects can be well determined.
To further verify the performance of the proposed method, we imaged a more complicated object, which consists of five continuous stripes and is beyond the ME region. The object is illuminated and scanned 25 times, respectively, as shown in Figure 4a. Each of the 25 parts of the object can be retrieved using the phase-retrieval algorithm. As shown in Figure 4b, the order indicated by the red arrows can guarantee that the cross-correlation calculations can be applied in adjacent parts. Following such an order, the 24 shift vectors are calculated. Finally, as shown in Figure 4c, the whole object consisting of five continuous stripes beyond the ME region can be reconstructed. The results suggest that the proposed method can image a whole object beyond the ME region through scattering media, successfully enlarging the FOV of the speckle correlation imaging technique. [25] Up to now, we have successfully demonstrated the proposed method can be used for noninvasive imaging through scattering Figure 3. Noninvasive imaging of five characters beyond the ME region through scattering media using our method. a1-a5) The estimated PSFs. b1-b4) The relative positions between a character and its adjacent one, as indicated by the shift vectors. b5) the relative shifts for the five characters. c1-c4) Shift and superimpose the adjacent retrieved characters. c5) The object is stitched by the superposition of all the characters. www.advancedsciencenews.com www.adpr-journal.com media with enlarged FOV. In many practical applications, it is also critical to achieving high imaging resolution. Based on our previous work, [43] we further experimentally demonstrate the method for noninvasive super-resolution imaging through scattering media beyond the ME region with enlarged FOV. Figure 5a1-a5 shows the five characters reconstructed by the super-resolution stochastic optical scattering localization imaging (SOSLI) technique, [43] and the insets are the ground truths, each of them was obtained by randomly blinking 500 times. Compared with the five recovered characters of the same size in Figure 2d, the results in Figure 5a1-a5 show that the SOSLI technique achieves higher-resolution images with vivid and sharp edges. As compared with Figure 2g, the object, stitched by the superposition of the five retrieved parts (Figure 5a1-a5), can achieve higher imaging resolution as shown in Figure 5b5, which suggests that the SOSLI technique can be further extended to image an object beyond the ME region at super-resolution by the proposed method. Figure 6 gives the results for a long dashed line object consisting of 17 sections (see the inset on the top right of Figure 6a) reconstructed using Wiener deconvolution and the proposed method, respectively. To compare the imaging FOV of the proposed method with that of the Wiener deconvolution fairly, the background with an intensity less than 15% of the maximum intensity is removed. It can be seen from the blue solid line in Figure 6c that the intensity from the middle to the two sides gradually decreases, and only ten sections in the center of the object have a normalized intensity value over 0.5. By contrast, using the proposed method, the whole 17 sections of the object are well reconstructed as indicated by the red dashed line in Figure 6c. This further suggests that the proposed method can eliminate the limitation that the ME restricts the imaging FOV of the speckle autocorrelation-based imaging technique, [25,43] enabling it can do noninvasive imaging through scattering media with an enlarged FOV, which can be larger than that of the Wiener deconvolution method. Theoretically, the FOV can be enlarged infinitely. However, in practice, we found that the detection part consisting of the iris and the camera further limits the imaging FOV.
Through all the previous experiments, a method using PSF estimations and correlations for noninvasive imaging through Figure 5. Noninvasive super-resolution imaging of an object consisting of five characters beyond the ME region through scattering media using our method. a1-a5) The five characters are reconstructed using the stochastic optical scattering localization imaging (SOSLI) technique. [43] b1-b4) The reconstructed adjacent characters. b5) The reconstructed object at super-resolution. www.advancedsciencenews.com www.adpr-journal.com scattering media with enlarged FOV is demonstrated. We first demonstrate the proposed method for noninvasive imaging of multiple characters beyond the ME region as well as a whole object consisting of five continuous stripes beyond the ME region, which are placed behind a 120-grit ground glass. Experimental Section results suggest that the method can successfully do noninvasive imaging of a whole object or multiple objects with the right relative positions and orientations beyond the ME region without the need for prior information.
As compared with the methods reported in refs. [29][30][31][32], our method can image an object beyond the ME region through scattering media without the need for an invasive "guide-star" or prior information about the imaging system. Unlike the methods reported in refs. [33][34][35][36][37], which need time-consuming iterations for separating the autocorrelations of dual targets exceeding the ME region, [33,38] or need one of the two objects kept stationary while the other one was gradually moving, [34][35][36] or need prior information about the number of targets behind scattering media and suffer from lacking the right relative positions and orientations of the reconstructed multi-targets, [37] our method only requires a camera to capture speckle patterns produced by each illuminated part of the object behind scattering media, the whole object or multiple objects beyond the ME region can be noninvasively reconstructed, and each part of the reconstructed object has the right relative position and orientation. In addition, if the axial sectioning could be achieved, the method might also be applied to expand the depth of field of the scattering imaging, and realize noninvasive imaging of an object beyond the 3D ME region through scattering media. During the process of completing this work, we noticed ref. [45] reported a different ingenious approach for large FOV noninvasive imaging through scattering media using fluctuating random illumination, in which they captured a set of images while randomly changing the illumination, and used a Non-negative Matrix Factorization (NMF) algorithm to demix the set of acquired frames to retrieve each individual fingerprint. Nevertheless, thus far, it cannot realize large FOV noninvasive super-resolution imaging through scattering media.
Here, based on our previous work, [43] we demonstrate the proposed method for noninvasive super-resolution imaging of multiple characters beyond the ME region through scattering media. Inspiringly, the reconstructed multiple characters have the right relative positions and orientations with high clarity, which suggest that the method can be applied to further develop the previous work, [43] making it capable of noninvasive super-resolution imaging through scattering media beyond the ME region. Such a technique provides an effective solution to see through turbid media such as frosted glass or translucent biological tissues with enlarged FOV and unprecedented clarity.
Despite all this, the method requires an assumption that the object beyond the ME region hidden behind scattering media is illuminated and scanned separately. Fortunately, such an assumption have the potential to be satisfied, since noninvasive optical focusing through scattering media using the fluorescence signal has been realized. [39][40][41][42] Therefore, one can manipulate the focusing spot by steering a phase grating within the ME region, after the focusing spot approaches the edge of the ME region, the optimization for optical focusing through scattering media can be redone, such that the object beyond the ME region hidden behind scattering media could be illuminated and scanned separately.

Conclusion
In summary, we propose and experimentally demonstrate an effective scheme using PSF estimations and correlations for noninvasive imaging through scattering media with enlarged FOV. It can successfully do noninvasive imaging of a whole object or multiple objects beyond the ME region without the need for prior information, and each part of the reconstructed object has the right relative position and orientation. More interestingly and inspiringly, the proposed method can also be applied to enhance our previous work, making it capable of noninvasive super-resolution imaging of multiple objects beyond the ME region through scattering media. The technique provides an effective solution to see objects beyond the ME region through turbid media, such as frosted glass or translucent biological tissues, with enlarged FOV and high clarity.