High-speed 4-dimensional scanning transmission electron microscopy using compressive sensing techniques

Here we show that compressive sensing allows 4-dimensional (4-D) STEM data to be obtained and accurately reconstructed with both high-speed and reduced electron fluence. The methodology needed to achieve these results compared to conventional 4-D approaches requires only that a random subset of probe locations is acquired from the typical regular scanning grid, which immediately generates both higher speed and the lower fluence experimentally. We also consider downsampling of the detector, showing that oversampling is inherent within convergent beam electron diffraction (CBED) patterns and that detector downsampling does not reduce precision but allows faster experimental data acquisition. Analysis of an experimental atomic resolution yttrium silicide dataset shows that it is possible to recover over 25 dB peak signal-to-noise ratio in the recovered phase using 0.3% of the total data.

are recorded in the far field on a 2D pixelated detector (Figure 1). 1 Subsequently a variety of signals can be extracted by suitable geometric integration of regions at the detector.
Prior to the widespread use of aberration correctors, Nellist et al. demonstrated one of the earliest cases of 4-D STEM where coherent microdiffraction patterns were collected as a function of probe position and used for a superresolved ptychographic reconstruction. 2This allowed the resolution of the Si {004} at 0.136 nm; a much higher spatial resolution than was achieveable using high-angle annular dark field (HAADF) STEM on the instrument used.Another early demonstration by Zaluzec et al. used position resolved diffraction to image distributions of magnetic induction in a Lorentz STEM imaging mode. 3,44-D STEM has progressed significantly since these early demonstrations, with more recent examples of its application in ptychography having been used to recover the complex object wavefunction of weakly scattering objects, such as lithium ion cathode materials 5 and biological samples. 6STEM ptychography has also been used to resolve praseodymium dumbbells at the limit set by thermal atomic motion. 71][12][13][14] A major limitation in the application of 4-D STEM has been the need for long integration times to a achieve significant signal-to-noise ratio (SNR) in the presence of noise and dark current.Although most commercially available direct electron detectors that operate in counting mode have effective frame rates of less than 10 kHz, there have been recently announced direct electron detectors [15][16][17][18] operating at between 100 kHz and 1 MHz, albeit with small pixel array sizes.Using these detectors, CBED patterns can be acquired with little or no noise at an effective dwell time of 10 μs per probe position. 18,19hile these are significant improvements over earlier indirect scintillator coupled detectors operating at fewer than 30 fps, 20,21 it remains the case that only the most recent detectors match the dwell time of traditional solid state monolithic STEM detectors.Importantly, our approach can also be used with slower large pixel array detectors to provide the required matching speed increase.Hence, 4-D STEM experiments remain susceptible to drift and beam induced damage 22 which limits its applicability to studies of, for example, beam sensitive organic and hybrid materials or to investigations of materials dynamics.
One option to overcome beam damage is to reduce the electron fluence at the sample. 23,24By reducing the fluence below a materials dependent threshold, 25 or by using cryogenic temperatures, 6 beam damage can be reduced.Furthermore, if combined with alternative methods to increase acquisition speeds such as low bit-depth electron counting, 26,27 the acquisition speed can be increased and sample drift can be reduced.However, given that the SNR is related to the number of detected electrons, and hence, with the fluence per probe position, a combination of fluence and fast acquisition quickly transitions the experiment to conditions that are below the minimum signal-to-noise requirements for 4-D methods such as ptychography. 28An alternative method to overcome beam damage (as well as to increase the effective frame rate of an existing detector) in STEM is by using techniques based on the theory of compressive sensing (CS), 29,30 which is referred to here as probe subsampling.Probe subsampling in this context refers to controlling the set of positions of the STEM probe visits within a raster scan to reduce the number of acquisition points -thereby directly creating a faster scan and a lower fluence and flux at the sample.This can be modelled (details in the Supplemental Material) and a demonstration of this is given in Figure 2, which also shows that subsampling directly reduces the maximum fluence at the sample.1][42] The key benefit for probe subsampling in STEM is that by acquiring less data, acquisition rates can be increased, which in turn reduces drift artefacts as well as reducing the total cumulative electron fluence of the entire field of view.Thus, samples which are susceptible to beam damage can be imaged at usable SNRs, without over exposure to the incident beam.Although the dose at any acquired probe location is independent of the scan pattern, work by Nicholls et al. 35 has shown that the diffusion of radicals due to beam interactions at neighbouring probe locations compounds the damage of samples.By taking larger steps in a random fashion, this cumulative dose can be reduced since radicals are not propagated between successive probe locations.In this paper, we will demonstrate a focused probe acquisition method which reduces beam damage and increases acquisition rate by probe subsampling.We acquire only a subset of the CBED patterns and use a Bayesian dictionary learning technique known as Beta Process Factor Analysis (BPFA) to recover the full 4-D STEM dataset from the subsampled measurements.The BPFA has been shown as a robust inpainting algorithm to data containing complex structures such as defects, 42 and further evidence is given in the Supplemental Material.We describe simulations of this method to a 4-D STEM dataset of yttrium silicide, and demonstrate that 4-D STEM data acquisition can be reduced by at least 256× without significant quality loss in all imaging modes.
Previous work by Stevens et al. 34 demonstrated that probe subsampling and detector subsampling can be employed and that by inpainting followed by phase retrieval, one can recover functionally identical 1 results to a fully sampled experiment.In this work, the inpainting of the 4-D data used a Kruskal-factor analysis technique. 43e extend this approach by using a new implementation of the BPFA algorithm, which takes advantage of GPU acceleration.We will also build on the work of Zhang et al. 44 who showed that the number of detector pixels required for ptychographic reconstruction can be reduced significantly without loss of resolution.Recent work by Ni et al. 45 show that it is possible to reconstruct subsampled nanodiffraction 4-D STEM data in both the real and reciprocal space using a spatial-spectral compressed reconstruction based on the spectral unmixing, adapted from Wang et al. 46 Their results indicate a 100× decrease in data storage for their example dataset.

PROPOSED METHOD FOR SUBSAMPLED 4-D STEM
The experimental set-up for the acquisition of a subsampled dataset is shown in Figure 1.We assume a pixelated detector with  d and  d pixels in the vertical and horizontal axis, respectively, collecting 2-D CBED patterns of size  d ×  d .Let Ω d ∶= {1, … ,  d } × {1, … ,  d } be the set of all detector pixel locations and  d ∶= ( h d ,  w d ) ∈ Ω d denote the coordinates of a detector pixel.We fur- 1 Functionally identical results are defined as the preservation of features compared to the ground truth, such that the analysis is preserved in determining properties of the sample.We achieve our compressed 4-D STEM by subsampling  p ≪  p probe locations acquired in the subsampling set Ω ⊂ Ω p , which is equivalent to subsampling each of the virtual images (sharing a common mask determined by Ω).This defines our acquisition model as where  vi  d is the subsampled measurements at detector pixel  d and  Ω is a mask operator with ( Ω ()) (,) =  (,) if (, ) ∈ Ω and ( Ω ()) (,) = 0 otherwise, and   d is an additive noise.
We now estimate virtual images Xvi in (1) for  d ∈ Ω d , which defines the inpainting problem.In this work we assume that virtual images are sparse or compressible 3 in an unknown dictionary that can be learned during the recovery process.This leads to the development of dictionary learning adopting a Bayesian non-parametric method called Beta Process Factor Analysis (BPFA) as introduced in Ref. ( 47 ).The advantages of this approach include the ability to infer both the noise variance and sparsity level of the signal in the dictionary, and allows for the learning of dictionary elements directly from subsampled data.This approach has been tested in previous reports [37][38][39][40]42 and has shown success when applied to electron microscopy data. Not that this approach learns a different dictionary for each virtual image and a BPFA instance is applied to every virtual image.This is not necessarily optimal; however, we will leave the concept of learning a shared dictionary for all virtual images and applying a single instance of BPFA directly on the subsampled 4-D data to a future study (a full description of the BPFA process is provided in the Supplemental Material 4 ).
In addition to probe subsampling, we can also downsample the detector pixels to eliminate redundancy.This can also be inferred as the optimisation of our reciprocal space sampling, Δ d , which can be carried out by only reading out the set of rows which are within the sampling set.This is different to conventional detector pixel binning (which still requires reading of all rows within the total CBED pattern), since we do not consider nor acquire rows which do not belong to the sampling set.
Given the detector downsampling factor  d ∈ ℕ, we first uniformly read-out every  th d row on the detector.This results in faster acquisition of CBED patterns of size  d ∕ d ×  d pixels.To further reduce the size of the dataset, we then keep only the data from every  th d column on the detector; resulting in CBED patterns with  d =  d ⋅  d ∕ 2 d entries.In this paper, we define detector downsampling ratio as  d ∕ d = 1∕2 d (i.e., the sampling ratio on the detector).In practice, the camera length could also be varied to optimise Δ d since the camera length is inversely proportional to the reciprocal space sampling.This would account for detectors which cannot read-out rows/pixels independently.It would also effectively bin the signal on the detector where hardware binning is limited, improving signal-to-noise ratio.

RESULTS
In order to model experimental acquisition, an experimental 4-D STEM dataset of Y 5 Si 3 was used (with all scan positions) and applied random subsampling of the probe positions and downsampling of the CBED patterns.Y 5 Si 3 is an electride framework composed of cation and anion sublattices.These sublattices have a net positive electric charge which are balanced by loosely bonded, interstitial anionic electrons. 48Y 5 Si 3 has been proposed as a low Schottky barrier material for n-type silicon semiconductors due to its low Schottky barrier height of 0.27eV. 49eaders are referred to Zheng et al. 48for details on practical applications.The experimental data was acquired using a probe forming aperture semi-angle of 30 mrad from a 100 kV electron electron source with a probe current of 20 pA with a dwell time of 1.3 ms.A Δ p of 0.0108 nm was 4 See Supplemental Material for a full description of the BPFA process.1C).In addition we simulated the recovered ptychographic phase (Figure 3).For this, there are a number of analytical and iterative algorithms [51][52][53][54][55][56] that recover the complex ptychographic wavefunction, and here we used a modification of the Wigner distribution deconvolution (WDD) algorithm 5,57-61 within the ptychoSTEM package for MATLAB. 14Details on the analysis methods used can be found in the Supplemental Material.
Figure 4 is a direct comparison of reconstructed CBED patterns at different probe locations and probe subsampling ratios.The inpainting strategy is able to faithfully recover the CBEDs, which may have been missing from the acquired dataset, indicating a continuity passed through the reconstruction of the virtual images.This continuity should be preserved through to the CBED patterns and no further constraint is required to enforce this, as is seen in the figure .Figure 5 shows the quality of the ptychographic phase (using the structural similarity index measure (SSIM) 62 as our chosen metric) with respect to different probe subsampling and detector downsampling ratios.There is only a small degradation in the quality as the sampling at the detector is decreased; this implies the detector is oversampled.We further observe that probe subsampling can be used with BPFA to recover visually identical results in the phase.
Similarly, Figure 6A shows a comparison of the quality of CoM field analysis as a function of subsampling ratio, where visually identical results are achieved with respect to the reference data.Comparing Figure 5 and Figure 6A suggests that ptychographic phase recovery is more robust in this case.This is possibly due to the fact that the WDD operates on a full 4-D dataset, while the CoM field is computed from individual CBED patterns.
Figure 6B is a direct image comparison between our reference data and reduced sampling data ( p ∕ p =  d ∕ d = 6.25%) when applied to CoM field analysis, DPC, ABF and LAADF.It is clear that there is very little difference in the quality of the images from a visual perspective, and this is supported comparison of the corresponding peak signal-to-noise ratio (PSNR) and SSIM values corresponding to each. Figure 3 is a visual comparison of the data in Figure 5.As can be seen, the recovered phase data are almost indistinguishable, with all showing the expected location of yttrium and silicon atoms.

CONCLUSIONS
Our results demonstrate the inherent redundancy within the 4-D STEM dataset.By utilising inpainting algorithms, it is possible to discard over 99.6% (see Figure F I G U R E 5 SSIM of phases with to probe and detector sampling ratios.As the probe subsampling ratio increases, the quality of the phase increases.However, there is only a small difference in the phase quality as the detector downsampling ratio is decreased.This indicates significant redundancy within the 4-D dataset, which can be omitted through detector downsampling and probe subsampling.Example images of this experiment are shown in Figure 3. bottom-right) of the original dataset while still recovering qualitatively identical results in the reconstructed phase, CoM field, DPC and VD images, to those obtained from processing the full dataset.
The efficacy of this method applied to samples containing defects is important for correct analysis of complex samples.One key aspect of the BPFA is that it operates on a patch-wise basis; that is, each patch of the image is independent to others and only connected through a shared dictionary.This is different to reconstruction algorithms which leverage periodicity, or generative techniques such as generative adversarial networks.The BPFA is inpainting each overlapping patch independently, without knowledge of neighbouring patches.This means that if a patch contains a defect, for example, then as long as this patch is sufficiently sampled, the solution which BPFA provides will return a defect.This method has also been shown as robust to 4-D STEM data containing an interface, and the results are given in Figure S5 in the Supplemental Material.Furthermore, an example of an image containing multiple types of defects is provided in Figure S6 in the Supplemental Material.
Given the inherent redundancy in 4-D STEM data, we propose that even lower sampling ratios could be employed using a multidimensional recovery algorithm.The benefit of this is that by using a multidimensional recovery algorithm, we can leverage more data during the training process as well as the similarity between virtual images during the recovery step.It may be possible to also include sparse detector sampling followed by inpainting the 4-D STEM dataset with minor modifications to the acquisition model.This could further increase acquisition speeds by assuming that each pixel has a fixed read-out time and potentially allow for multiple 4-D STEM datasets to be acquired rapidly.
In order to implement these strategies in experiment, a scan engine which has its timings in sync with the camera is required.This scan engine can be given a desired scanning pattern (such as a randomly sampled pattern) and this is converted to voltages for the scan coils.The diffraction pattern captured on the camera is then the diffraction pattern corresponding to that probe location, and as such a 4-D STEM dataset can be acquired.In order to achieve the effective downsampling as described here, it would be recommended to reduce the camera length such that the maximum collection angle is restricted to a size of [N × M] pixels.Then by only reading those N rows from the camera, the speed up in acquisition due to detector downsampling can be achieved.If an event driven detector is used, then we suggest that a desired dwell time is set and then the signal is binned to [N × M] pixels.This would reduce the size of the data as well as allow for faster 4-D STEM acquisition without compromising the quality of the analysis.This is demonstrated during live acquisition in Ref. [63].We postulate that time-resolved 4-D STEM is now not limited by the detector read-out speed but can instead be acquired through reduced sampling strategies.

F I G U R E 1
Operating principles of 4-D STEM are demonstrated.(A) Electrons are converged to form a probe which is rastered in 2-D across the sample plane.The transmitted electrons are collected using a 2-D detector in the far field for each probe position.(B) Inpainting of the 4-D STEM dataset by sequentially inpainting each virtual image using the BPFA algorithm.(C) Application of VDs and DPC at the detector plane.

13652818, 2024, 3 ,
Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jmi.13315by University Of Glasgow, Wiley Online Library on [12/08/2024].See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions)on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License F I G U R E 2 Electron fluence distributions for both full sampled and subsampled acquisitions using a focused electron probe with a scan step of 0.02nm and a dwell time of 10 μs.The simulated electron probe (left-most) had a convergence semiangle of 30 mrad, a current density of 12 pA, and an accelerating voltage of 200 kV.This would correspond to a Nyquist sampling rate of 0.0208 nm.
13652818, 2024, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jmi.13315by University Of Glasgow, Wiley Online Library on [12/08/2024].See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions)on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License ther assume an electron probe scanning a regular grid of  p and  p locations in the vertical and horizontal axis, respectively 2 , collected in a probe locations set Ω p ∶= {1, … ,  p } × {1, … ,  p }. Let  p ∶= ( h p ,  w p ) ∈ Ω p denote the coordinates of a probe location.Moreover, the total number of detector pixels and probe locations are denoted by, respectively,  p =  p  p and  d =  d  d .Finally, given a scan step parameter Δ p , in m, of the electron probe and detector pixel size Δ d , in mrad, the location of the scanning probe and detector pixel can be converted from their index units to real units.Let  ∈ ℝ  p × p × d × d be the discretised 4-D representation of fully sampled 4-D STEM data and ( p ,  d ) be the 4-D STEM data observed at probe location  p and detector pixel  d .A CBED pattern collected at probe location  p is denoted by  dp  p ∶= ( p , ⋅) ∈ ℝ  d × d .In this paper, the virtual image corresponding to a detector pixel  d , represented as  vi  d ∶= (⋅,  d ) ∈ ℝ  p × p , refers to a matrix collecting the data observed at detector pixel  d for all probe positions.

F
I G U R E 4 A comparison of diffraction patterns at different probe positions ( h p ,  w p ) (as indexed on the left side of the figure) for various probe subsampling ratios (as given at the top of the figure).

F
I G U R E 6 (A) SSIM values as a quality metric for CoM field images.(B) CoM field, DPC, ABF and LAADF images for 6.25% probe sampling and 6.25% detector downsampling after inpainting.The RGB pixel values in the CoM field images are directly correlated to the magnitude and phase of the CoM shift with respect to the calibrated centre (see Supplemental Material for a detailed description).The reference data are the full data passed through the BPFA algorithm (top row).The PSNR and SSIM values are overlaid, the spatial scale bar indicates 0.5 nm, and the detector scale bar indicates 30 mrad.The left-most column is an example data point from the plot in (A), and the corresponding plots similar to (A) for DPC, ABF and LAADF can be found in the Supplemental Material.
This work was performed at the Albert Crewe Centre (ACC) for Electron Microscopy, a shared research facility (SRF) fully supported by the University of Liverpool.This work was also funded by the EPSRC Centre for Doctoral Training in Distributed Algorithms (EP/S023445/1), Sivananthan Labs, and the Rosalind Franklin Institute.M.C. would like to acknowledge the support by the US DOE Office of Science Early Career project FWP# ERKCZ55 and the Center for Nanophase Materials Sciences (CNMS), a US DOE Office of Science User Facility.Initial experiments were carried out using MagTEM, a JEOL ARM200F STEM in the Kelvin Nanocharacterisation Centre, which was installed with support from the University of Glasgow and the Scottish Universities Physics Alliance.A.W.R. would like to thank Jordan A. Hatchel (ORNL) for his knowledge and insights of 4-D STEM analysis.O R C I D Alex W. Robinson https://orcid.org/0000-0002-1901-2509Amirafshar Moshtaghpour https://orcid.org/0000-0002-6751-2698Ian MacLaren https://orcid.org/0000-0002-5334-3010R E F E R E N C E S