Scattering transform in microstructure reconstruction

Descriptor‐based microstructure characterization plays a crucial role in the field of reversed material engineering for random heterogeneous media. With the advent of differentiable microstructure characterization and reconstruction, there has been a growing interest in the development of differentiable formulations of descriptors. The search for effective descriptors becomes indispensable to adequately characterize a wide range of microstructures. This work proposes a novel approach to construct a descriptor by utilizing a wavelet‐based transformation called the scattering transformation on microstructure images. The characterization and reconstruction capabilities of this newly developed descriptor are compared to a benchmark descriptor based on spatial correlation functions using various 2D microstructure images. The comparative analysis aims to evaluate the effectiveness and potential advantages of the proposed wavelet‐based descriptor.

autoencoders [13], to diffusion models [14].However, this work focuses on a descriptor based approach, since the lack of decent training data is a well known problem.
The first major step in MCR was the Yeong-Torquato algorithm [15], where the microstructure is reconstructed by iteratively adapting a random instance to closer match a desired structure.One major advantage of the result was its simplicity in implementation and the possibility of incorporating more complex information about the morphology of the microstructure.The general randomness of the underlying morphology is described by so called descriptors, which can be either based on physical properties like volume fractions, or statistical properties like the lineal path function [16] or point spatial correlation functions [17].With translation-invariant descriptors the comparative analysis of microstructures can be raised to another level, since one can check for morphological similarities and differences or even average and interpolate between microstructures.Since descriptors capture the essential morphological properties they are often called the microstructure characterization.
Since then much progress has been made in the development and formulation of descriptors, as well as in improving optimization algorithms, as for example in [18], where the reconstructing algorithm is constructed as an differentiable optimization problem.The inclusion of the gradient in the optimization problem reduces the number of needed iterations significantly and therefore allows for a more accurate calculation of the descriptors while still reducing computational time, see [28] for a comparative overview.
In the upcoming work, a overview of the general model is given, before a transfer-learning approach from [19] is shortly reviewed, where essentially the local features of a pre trained convolutional neural network (CNN) are used to perform a descriptor based reconstruction.Inspired by this approach, a descriptor for DMCR based on a coupled wavelet transform of the microstructure is constructed and its reconstruction properties are investigated.

MODEL
Only the very essentials of the model are reviewed in this section, alongside some of its strengths.The differential microstructure reconstruction algorithm introduced in [18] is used, that is, given a differentiable descriptor   the microstructure is reconstructed by where  = {0, 1} × is the space of all  ×  two-phase microstructure images,   is the corresponding descriptor of  and  is a loss function, for example, the Euclidean distance.
Note that this algorithm can be extended to reconstruct 3D microstructure voxels from a 2D microstructure image, by minimizing over every slice of the voxel in ,  and  direction.See [20] for further information.

Software
The characterization and reconstruction of all microstructures in the numerical experiments is performed using the open source software MCRpy.As illustrated in Figure 1, the core function of MCRpy is the interplay between descriptors and microstructures, that is, characterization and reconstruction.All other functions, such as optimizer, loss function and descriptor are implemented as plug-ins to the core function.This makes the software easy to extend, since only the new plug-in has to be implemented and can then be combined with all other parts of the software.This will be exploited to test a new wavelet-based descriptor.

MCR using correlation functions
In the past, reconstruction properties of many different descriptors have been investigated, see for example, [3] and the references therein, where spatial correlation functions have shown promising results and therefore will act as a benchmark for new descriptors.The reconstruction setting is taken as follows: Structural overview of MCRpy: microstructures can be characterized and reconstructed, whereas the preferred optimizer, loss function and descriptor are provided as plug ins.The illustration was created by Paul Seibert et al. [21] and is used, without modifications made by the author, under the Creative Commons BY license.
A comparative overview of the reconstruction properties for some example microstructures will be given in the latter part of the work, alongside a comparison with a newly developed scattering descriptor.

MCR using local feature extraction
In [19], a reconstruction based on local features of a CNN is performed, that is, the microstructure is forward propagated through a pre-trained CNN and then the vectorized feature maps of of each layer in the network are extracted.In this case a trimmed VGG19-architecture as network and the ImageNet dataset as training data were used.
For the sake of simplicity, let's consider the case where a given microstructure is reconstructed.Therefore let  ∈ ℝ × denote the original microstructure.Passing  through the CNN will activate in Layer  the feature maps   ∈ ℝ   ×  , where   denotes the number of filters and   the the size of the vectorized feature maps in layer , see [19].The Gram matrix of layer  for microstructure  is then defined by where    is the -th row of   .Taking a random initialization M ∈ ℝ × the Gram Matrices are defined analogously and denoted by G .The optimization is then performed by taking the sum of the layer-losses that is, a weighted sum of the element wise matrix difference.Due to the structure of the CNN, the gradient of the loss function can be easily incorporated in the optimization, since it can be automatically computed by back propagation using standard python packages, for example, Tensor Flow.This follows the same idea as the above introduced DMCR.Although Li et al. [19] obtained impressive results for the reconstruction of different samples, the choice of weights in the Gram matrices seems a bit arbitrary, since the ImageNet dataset contains no microstructure images, but pictures of everyday life.Henceforth, in the forthcoming chapter, the identical methodology shall be employed, but with the utilization of pre-defined wavelet kernels within the convolutional network.

SCATTERING DESCRIPTOR
The foundation of this part is a signal presentation called the Scattering Transformation, which is a widely used tool in image processing and has been thoroughly investigated, see for example, [22,23] and [24].This representation is obtained  by propagating the signal, in this case the microstructure image, through wavelet transformations with nonlinear modulus and averaging operators.This leads to a computational structure that is identical with one of a convolutional neural network, see Figure 3.
The main difference is that the convolution kernels are not learned by using training data and back propagation but are all predefined rotated and dilated Morlet wavelets [26] and the averaging kernel is taken as a Gaussian, see Figure 2 for a visualization.Depending on the number of dilations  and the number of rotations ,  wavelet kernels are computed and used in every layer, which leads to a number of ()  nodes for the -th layer.Fortunately, one can show that most information of the signal is captured in the first three layers [23], so the exponential growth of the convolutions is not an issue for the computational feasibility.
In this work, the parameters were chosen as  = 2,  = 8 and a network depth of three.Extracting the scattering coefficients, that is, the averaged output of every node in the network, see Figure 3, the scattering descriptor  is defined by where    denotes the scattering coefficients corresponding to the -th node of layer  in the scattering network.Again, the differentiability is a natural property of the scattering descriptor by it's construction via a convolution network.Reconstructions using the scattering descriptor and spatial correlation functions as descriptors are summarized in Figure 4.The same reconstruction setting as in Section 2.2 was used, that is, L-BFGS-B as optimizer and mean squared error as loss function.The samples of alloy, sandstone and ceramics are reconstructed perfectly with the spatial correlation descriptor, that is, translated versions of the original sample are produced.The fingerprint-like structure of the copolymer is not captured by the spatial correlation functions at all, which can be seen in the inaccurate reconstruction.For this type of structure the scattering descriptor shows promising results, which displays its huge potential.Furthermore, except for the alloy, the other samples show satisfactory reconstruction results.

CONCLUSION AND OUTLOOK
In this work the essentials of differentiable microstructure characterization and reconstruction are reviewed, as well as properties of commonly used descriptors, such as spatial correlation functions and Gram matrices.
Inspired by the Gram matrix approach, a new descriptor, which is based on the scattering transformation, is formulated.The scattering transformation in turn is a coupled wavelet transformation which captures local features of the input signal by iteratively applying a wavelet transform, a complex modulus and an averaging.The reconstruction results show that the scattering descriptor is able to capture the morphology and characteristics of most tested microstructures, namely three out of four.The reconstruction of the copolymer, which could not be fully captured by spatial correlation functions, is particularly well using the scattering descriptor.This could be due to the fact, that the convolution with wavelets captures the many edges of the structure well.This can be seen even more clearly, when taking microstructure images of size 256 × 256 pixels into account.Figure 5 gives an overview of a 256 × 256 pixel copolymer reconstructed by correlation functions and the scattering descriptor.
It should be noted, that this is only a first idea on how to use the scattering transformation in MCR and does not capture the full potential.The exact scattering transformation defined in [24], that is, the limit of the discrete transform presented here, has some very interesting properties, one example being its stability to the action of small diffeomorphisms.Technically speaking, a diffeomorphisms is a deformation of the underlying structure.This stability could prove beneficial in the context of MCR and will be investigated in future work.In this work, the reconstruction properties of the scattering descriptor are only assessed by the eyeball-norm.Of course, this should be extended to the simulation and comparison of effective values of the corresponding microstructure.Overall, this work gives a new possibility to characterize microstructures, which shows promising potential and is a valuable extension to the existing set of microstructure descriptors.

A C K N O W L E D G M E N T S
The groups of M. Kästner and D. Peterseim would like to thank the German Research Foundation (DFG), which funded this work as part of a collaborative project (Grant Numbers KA 3309/18-1 and PE 2143/7-1).
Open access funding enabled and organized by Projekt DEAL.

2 F I G U R E 3
Examples of 64 × 64 pixel convolution kernels used in the scattering transformation.Computational Structure of the Scattering Transformation. denotes the wavelet kernel and  the gaussian kernel.In this example three different wavelets are used for the convolution, which gives 3  nodes for layer .The illustration was created by Varun Khemani et al.[25] and is used, without modifications made by the author, under the Creative Commons BY license.

F I G U R E 4
Comparative overview of the reconstruction properties using a spatial correlation function as descriptor and a scattering descriptor.The figure reads as follows: (A)-(D) are original microstructure samples of pixel size 64 × 64 taken from [19], (E)-(H) are reconstructions using a spatial correlation function and (I)-(L) are reconstructions using the above introduced scattering descriptor.