Using conditional generative adversarial networks for the prediction of stresses in an adhesive composite

Fiber reinforced adhesives are an integral part of wind energy turbine blades and play an important role in evaluation of structural integrity of the blades. Digitization of manufacturing processes and demand for fast and efficient stress analysis argues for the application of deep learning. As a step forward in using deep learning to predict structural properties in composite, the work would explore the applicability of deep learning methods to 2D fiber reinforced composite. In this work, a composite adhesive used in wind energy turbine blades is used as an illustrative example. The composite adhesive when viewed using a CT scan machine shows a heterogeneous structure with two phases, a softer matrix phase and a much stiff fiber phase. For this purpose, several CT scan images of the fiber adhesive composite are used. The images on subsequent FEA analysis serves as the training data for the deep learning framework. Once trained, the neural network would be able to predict the stress distributions in the structure. We apply conditional adversarial networks as general purpose solutions to problems focused on stress prediction, which could be thought of as an image‐to‐image translation from the microstructure domain to the stress domain. These networks in addition to learning the mapping from input image to output image also learn a loss function to train for this mapping. This makes the method more generic because otherwise different loss formulations for different problems have to be formulated.


INTRODUCTION
Wind turbine rotor blade manufacturing involves joining two aerodynamic half shells and multiple shear webs using structural adhesives.For the reasons of economic constraints and the size of the separate parts, the adhesive also functions as a mitigation for tolerances in manufacturing.All of this results in a variation of bond line geometry along the length of the rotor blade.The rotation of the wind rotor induces a gravitational load in the edge wise direction and a highly stochastic wind load in the flap wise direction.In addition to that fatigue load is also present and all of these loadings result in a complex multiaxial loading scenario over the life time of the blade.This makes the effective characterization of the material properties of the wind blade adhesive extremely important.The adhesives are usually a two-phase epoxybased resins with considerable viscosity.A prior work [1] involves employing a machine mixed specimen unlike the hand mixed specimen usually used to characterize such specimens for the reasons of enhanced porosity and diminished material properties not representative of the real behavior.During the study, several micro CT scans of the adhesive samples were obtained and that serve as the starting point in the current work.As a first step in developing a microscale numerical model for the obtained adhesive microstructures, FEM-based models for the composite adhesive microstructures are developed.Traditionally, when structures are solved numerically as a boundary-value problem, two types of equations are combined, equations based on conservation laws, which are inherently uncertainty-free and equations derived from physical modeling and observations, which are empirical and uncertain.The classical numerical paradigm is based on making use of the observed experimental data and then using the calibrated material model in other calculations.Often times the method is expensive and time-consuming while being empirical in nature [2][3][4].A new direction based on data-driven techniques using machine learning architectures opened up new methods for computational solid mechanics [5][6][7][8][9][10].Conditional Generative Adversarial Networks (cGANs), which are an extension of the more classical Generative Adversarial Networks (GANs) [11], are suited to mechanics-related tasks owing to the conditional generative abilities that these algorithms possess.In this study, cGANs will be used for stress predictions in a wind energy fiber composite adhesive microstructure.

DATA ACQUISITION AND MESH PREPARATION
Several CT Scan studies conducted at different magnifications at six different locations in the test region with Zeiss Xradia 410 Versa, reveal white fibers (lighter) embedded in an adhesive (dark) matrix with a 7%-9% fiber volume fraction.Depending on the flow conditions and the nature of the adhesive flow, fiber anisotropy is reported, which affects the structural properties and hence the mechanical behavior of the adhesive.The computed tomography scans are acquired, however, with a lot of noise and fluctuations for reasons of variation in specimen sizes, materials, and other operational modalities, hence a lot of noise cleaning operations have to be performed before they can be meshed for subsequent FEA analysis.
The approach borrows a lot of concepts from the reconstruction approach used for tomographic data of heterogeneous fiber systems [12].

FINITE ELEMENT ANALYSIS
After the image data is acquired, segmented, and subsequently meshed (Figure 1), suitable values of the material parameters are chosen from prior studies on fiber reinforced adhesives in wind energy [14,15].Specifically, Moduli of 79 050 MPa for fiber and 2798 MPa for matrix are used, respectively.The Poisson's ratio for fiber was taken as 0.22 and for matrix, the value was set as 0.40.The material assignments are followed by establishing linear elastic and hyper-elastic models with a variety of boundary (uniaxial, biaxial) and loading conditions (2 and 4% tension and compression).In addition, the images are acquired at different magnifications of 4X, 10X, and 20X, respectively.This creates a relatively small but diverse dataset (Figure 2).For the reasons of brevity, only the compressible Neo Hookean material model is shown here as an example, defined by a free energy function .If the deformation gradient is represented as  and the right Cauchy-Green tensor as , then the free energy function could be written as where   = trace() = trace(  ) =  ∶  and  = det() , and  and  are the Lame's parameters, respectively.The Cauchy stress tensor  is obtained by pushing forward the second Piola-Kirchhoff stress tensor , which is computed when the free energy function is derived with respect to .
The Frobenius norm of the Cauchy stress tensor, which is defined as the square root of the sum of the absolute squares of elements of the Cauchy stress tensor referred to here as  eq is chosen as the measure of obtaining a stress map and would hereafter be used in all the comparisons.

CONDITIONAL GENERATIVE ADVERSARIAL NETWORKS
Conditional image synthesis tasks are tasks where photorealistic images are generated conditioned on certain input data.
Recent methods employed to do these tasks include GANs [11] and variational auto-encoders (VAE) [16], both falling under the broad category of deep generative models.Deep generative modeling assumes that the way data are created or generated is related to a distribution and generally that distribution can be described by parametric and nonparametric variants, respectively [17,18].The goal in generative modeling is then to approximate this underlying distribution with various algorithms and techniques and finally be able to synthesize images.The current study is a similar conditional image-based synthesis task, but instead of converting a semantic segmentation mask to a photorealistic image, stress distributions will be produced.The goal of the deep learning network should be to output a stress distribution provided a microstructure, loading, and boundary conditions, thus making the output conditioned on these inputs.In other words, a mapping needs to be trained such that an input image   from the input domain  is translated to an output image in the stress domain .Mathematically, a mapping  → needs to be trained such that it could generate an image   ∈ , which is identical to the target image in the stress domain   ∈ , given the input image   ∈ .
∈  ∶   =  → (  ) The current work is based on GANs, which in the most classic setting consist of a generator  and a discriminator  where the aim of the generator is to produce realistic images () from random noise  so that the discriminator cannot tell the synthesized images () apart from the real ones  [11].A variation of GANs called cGANs, where the output is conditioned on some input variable (input microstructure, boundary conditions, loads), seems to be well-suited to the problem under study as it provides more control [16].Two most popular cGANs namely Pix to Pix [19] and GauGAN [20] are used in this work with little modifications.
Pix to Pix uses standard Convolution-Batch Normalization-ReLU blocks of layers in its generator and discriminator models.The generator uses a popular encoder-decoder network "U-Net" architecture with skip connections [21].A Patch-GAN network is used by the discriminator which instead of predicting the whole image as fake or real, takes an image patch and then subsequently makes a prediction for every pixel in that patch for its authenticity.The final objective  * then becomes a summation of adversarial loss   and loss  1 defined as The adversarial loss   controls whether the generator in the network has the power to produce images that are plausible in the target FEM domain.The 1 loss  1 regulates the generator to output images that are a possible translation of the image from the source domain.The parameter  gives the user control as to how much importance needs to be given to the loss  1 acting as a weighting factor between the adversarial loss   and the loss  1 .
GauGAN [20] employs a different normalization technique called the Spatially Adaptive Normalization technique (SPADE), which is specifically more suitable for learning affine parameters, that is, scale and bias, respectively.Because these affine parameters are spatially adaptive, learning different sets of scaling and bias parameters for each semantic label makes it more efficient compared to unconditional normalization like batch normalization used in Pix to Pix, which leads to the loss of semantic knowledge.Pix to Pix also suffers from the problem that stochasticity is only incorporated in the form of dropout, applied on several layers of the generator and because of that the model has limited randomness, which limits its generalizability.GauGAN tries to tackle this problem by letting the encoder  take the stress distribution and encode it into two vectors (Figure 3).These two vectors are then employed as the mean and standard deviation of a normal gaussian distribution.From this distribution, a random vector is then sampled and consequently concatenated along with the microstructure and boundary condition information as an input to the generator .The input to the generator  is different as it consists of a one-hot encoded semantic segmentation map and optionally an edge map and an encoded feature vector.The generator  of GauGAN is a fully convolutional decoder consisting of SPADE residual blocks.Compared to Pix to Pix, there is no downsampling and upsampling is performed by a nearest neighbor algorithm as opposed to by transposed convolutional layers in Pix to Pix.The discriminator  here also is a type of PatchGAN network but with multiscale abilities, which improves the receptive field and attention to finer detail.The loss function  * * is a summation of Hinge Adversarial loss   , which creates an image pyramid of the generated image resizing it to a number of scales, backpropagating the loss and calculating a realness score for each of these scales, a Feature Matching loss   and a Visual Geometry Group loss   , both ensuring that the generated images have the same statistical properties as of the real FEM images, and the Kullback-Leibler divergence loss   for effective regularization, respectively.A more detailed discussion on the losses can be found in Refs.[17,18].Mathematically, this could be summed as  * * =   +   +   +   (5)

RESULTS AND DISCUSSIONS
The deep learning model used and proposed was evaluated on different loading and boundary conditions.Here for the reasons of brevity, only biaxial extensions of 4% are shown and a comparison of the performance of Pix to Pix and GauGAN with the FEM generated ground truth is made.Figure 4 shows results of using conditional GANs on data out of the training dataset and it could be seen that even with very limited training data, good quality predictions are made.Some blurriness is noted in Pix to Pix results and some deviations between the ground FEM and the conditional GAN-based predictions near the boundaries of the fibers but overall the predictions are promising considering how intricate the problem on hand is and how paltry the dataset was during training.The results also argue for incorporating attention-based learning schemes where the fiber matrix interface is given more importance.However, this is left for future work.Nonetheless, the diversity of fiber shapes, small sizes, and a variety of volume fractions combined with the stresses they carry which could be different for each fiber and loading condition make the task of predicting stresses quite complicated especially when viewed from a computer vision lens as it becomes essentially a colorization problem of the microstructures but with the added constraint that each fiber and the neighborhood could have an entirely different stress value.The results open up broad applicability and transferability to other use cases.However, for more trustworthy results and confidence in the predictions of these methods, it is important to have a much more diverse dataset.If the dataset covers the whole spectrum of microstructures used in wind adhesives then in future, we can immediately know the mechanical behavior of the adhesive composite if provided the microstructure and the loading conditions.The work could be extended to cracks and studies involving crack propagation in wind turbine blades.In addition, porosity and other defects can be incorporated.Also, this could be extended in the framework of multiscale modeling.

A C K N O W L E D G M E N T S
This work was supported by the German Federal Ministry for Economic Affairs and Climate Action (BMWK) in the "Add2ReliaBlade" project (Grant no.0324335C).
Open access funding enabled and organized by Projekt DEAL.

FF I G U R E 2
I G U R E 1 (A) Scan locations of microstructure acquisitions.(B) A microstructure cell cropped and enlarged using Lanczos interpolation [13].(C) Noise reduction using nonlocal means denoising (NLM), the red box is the research window and the blue box is the comparison window.(D) Denoised image.(E) Microstructures after binarization using OTSU's method followed by open and close operation.(F) Watershed segmentation.(G) Suzuki algorithm.(H) Meshed microstructure.Creation of a FEA database of adhesives with diversity.(A) Microstructures at different magnifications were acquired.(B) For each microstructure, a stress distribution at different loads and boundary conditions was obtained.(C) Borders were drawn around the images as a method of letting the structure expand or contract.(D) Meshed microstructures were subjected to both uniaxial and biaxial boundary conditions with both tension and compression represented.

F I G U R E 3
Image encoder  takes in the stress distribution and outputs a latent representation for obtaining a mean and a variance vector.These two vectors are then used to compute the noise input via reparameterization to the generator .Another input to the generator  is from the microstructure and boundary condition detail via the SPADE residual blocks.The output of the generator  and the microstructure image is further concatenated in  and that serves as an input to the discriminator  for authenticity.F I G U R E 4 Pix to Pix versus GauGAN, a comparison of predictions based on the Frobenius norm of the Cauchy stress tensor   as a metric.(A) Microstructures subjected to a biaxial extension of 4% with a hyper-elastic material model.(B) FEM stress maps obtained, which act as the ground truth.(C) Predictions by Pix to Pix. (D) Predictions by GauGAN.