Advanced Deep Learning-Based 3D Microstructural Characterization of Multiphase Metal Matrix Composites

Cast near eutectic Al–Si alloys with addition of transition elements such as Cu, Fe, and Ni are commonly used materials in the aerospace and automotive industries. The microstructure of these alloys is characterized by a 3D interconnected network formed by eutectic and primary Si and several Ni-, Fe-, and Cu-rich aluminides embedded in the Al matrix. Under prolonged service time at high temperature (up to around 300–350 C), the aluminum matrix is overaged, what deteriorates its strength and creep properties. To improve the strength and creep resistance of these Al–Si alloys, additional ceramic reinforcements such as short fibers and particles can be used. It has been shown that the micromechanical behavior of such composites strongly depends on the orientation of the fibers, the spatial distribution of the particles, the individual volume fractions of all reinforcement phases, as well as on their morphology and interconnectivity. One of the most powerful tools for analyzing the micromechanical behavior and predicting the mechanical properties of such complex materials is micromechanical modeling. It is therefore of high interest to use all available microstructural information as an input to analytical models, to improve their performance. The most suitable tool to provide this kind of information is X-ray computed tomography (CT), because of its 3D nature. However, image segmentation of the CT data, especially in the case of multiphase materials, remains a highly challenging task. Ceramic reinforcements, as well as some intermetallic (IM) phases, have similar X-ray linear attenuation coefficients, and hence similar gray level in reconstructed CT data. The same holds for the eutectic Si and Al-alloy matrix. This makes a threshold-based segmentation of individual phases simply impossible. Moreover, high interconnectivity and clustering of all phases also excludes the application of shape-based classification. The complexity of the microstructure requires manual segmentation, which is at its turn impractical and time consuming. In addition, in the case of 4D CT with high temporal resolution, manual segmentation and successive analysis can only be done for small subvolumes, leading to statistically insufficient results. Recently, increasing interest in the application of deep learning (DL)-based algorithms has been observed for image classification and segmentation tasks, due to their strong ability to learn high-level features from raw input data. These methods have been very successfully applied to solve material science problems. Azimi et al. used DL for microstructural classification of steel by semantic segmentation of different phases with accuracy exceeding 93%. Iglesias et al. applied deep convolutional neural network (CNN) for the discrimination of quartz and resin in iron ore optical micrographs. Chowdhury et al. used DL for the recognition of dendritic morphologies, and Furat et al. showed the application of CNNs for grain-wise segmentation of 3D CT data on Al–Cu alloy. In this study, we investigate the 3D microstructure of an AlSi12CuMgNi alloy reinforced with 7%vol of Al2O3–Saffil short fibers and 15%vol of SiC particles. We show the application of an advanced DLmethod for the semantic segmentation of all phases present in this multiphase material. The investigated composite was produced by squeeze casting. Saffil short fibers, which consist of 96–97% of Al2O3 and 3–4% of SiO2 binders, have planar-random orientation, whereas SiC particles are expected to be randomly distributed within the volume. S. Evsevleev, Prof. S. Paciornik, Prof. G. Bruno Bundesanstalt für Materialforschung und -prüfung (BAM) Unter den Eichen 87, 12205 Berlin, Germany E-mail: sergei.evsevleev@bam.de Prof. S. Paciornik Department of Materials Engineering DEQM PUC-Rio Rua Marquês de São Vicente 225, Rio de Janeiro, Brazil Prof. G. Bruno Institute of Physics and Astronomy University of Potsdam Karl-Liebknecht-Str.24-25, 14476 Potsdam, Germany

the individual volume fractions of all reinforcement phases, as well as on their morphology and interconnectivity. [3,11,12] One of the most powerful tools for analyzing the micromechanical behavior and predicting the mechanical properties of such complex materials is micromechanical modeling. [6,12,13] It is therefore of high interest to use all available microstructural information as an input to analytical models, to improve their performance. The most suitable tool to provide this kind of information is X-ray computed tomography (CT), because of its 3D nature. However, image segmentation of the CT data, especially in the case of multiphase materials, remains a highly challenging task. Ceramic reinforcements, as well as some intermetallic (IM) phases, have similar X-ray linear attenuation coefficients, and hence similar gray level in reconstructed CT data. The same holds for the eutectic Si and Al-alloy matrix. This makes a threshold-based segmentation of individual phases simply impossible. [12] Moreover, high interconnectivity and clustering of all phases also excludes the application of shape-based classification. The complexity of the microstructure requires manual segmentation, which is at its turn impractical and time consuming. In addition, in the case of 4D CT with high temporal resolution, manual segmentation and successive analysis can only be done for small subvolumes, leading to statistically insufficient results.
Recently, increasing interest in the application of deep learning (DL)-based algorithms [14] has been observed for image classification and segmentation tasks, due to their strong ability to learn high-level features from raw input data. These methods have been very successfully applied to solve material science problems. Azimi et al. used DL for microstructural classification of steel by semantic segmentation of different phases with accuracy exceeding 93%. [15] Iglesias et al. applied deep convolutional neural network (CNN) for the discrimination of quartz and resin in iron ore optical micrographs. [16] Chowdhury et al. used DL for the recognition of dendritic morphologies, [17] and Furat et al. showed the application of CNNs for grain-wise segmentation of 3D CT data on Al-Cu alloy. [18] In this study, we investigate the 3D microstructure of an AlSi 12 CuMgNi alloy reinforced with 7%vol of Al 2 O 3 -Saffil short fibers and 15%vol of SiC particles. We show the application of an advanced DL method for the semantic segmentation of all phases present in this multiphase material.
The investigated composite was produced by squeeze casting. [12] Saffil short fibers, which consist of 96-97% of Al 2 O 3 and 3-4% of SiO 2 binders, have planar-random orientation, whereas SiC particles are expected to be randomly distributed within the volume.   Five different phases can be identified in scanning electron microscopy (SEM) micrographs of a polished composite sample: Al 2 O 3 fibers and SiC particles in dark gray, the eutectic Si in light gray, and IMs in white (Figure 1a, b). It is shown that SiC particles tend to agglomerate and build clusters between Al 2 O 3 fibers. Micrographs of the sample with chemically removed Al-alloy matrix (deep-etched using a solution of H 2 O and HCl with a ratio of 80:20 during 90 min) reveal the complexity of the reinforcing network (Figure 1c,d). Different IM particles are interconnected with eutectic Si and ceramic reinforcements.
The eutectic Si appears between SiC particles and in the form of bridges between Al 2 O 3 fibers, building an interconnected network. However, these observations are only qualitative, as SEM does not allow extracting any volumetric quantitative information concerning distribution, orientation, or interconnectivity of different phases. Truly 3D imaging methods are therefore needed. We used synchrotron X-ray computed tomography (SXCT).
The reconstructed SXCT dataset is shown in Figure 2a. Similar to the SEM micrograph all five phases are visually distinguishable. However, as mentioned earlier, individual phases cannot be segmented based on the gray-level threshold ( Figure 2b). The normalized histogram of the gray-intensity levels ( Figure 2c) shows only one peak, corresponding to the Al matrix. Clear peaks for gray values of any individual reinforcement phase or voids (related to the fabrication process) could not be found.
One of the main segmentation challenges in this work is to distinguish between Al 2 O 3 fibers and SiC particles, which have exactly the same gray value. Moreover, the cross-section of most fibers is highly similar to the cross-section of SiC particles. This similarity would limit the application of any 2D CNN in case of composites with randomly oriented fibers. Fortunately, in our material, fibers possess planar random orientation, the crosssections of fibers and particles differ in the SXCT slices parallel to the fiber plane (XY section in Figure 2a). This allows slice-wise segmentation by the application of 2D DL algorithms, which in comparison with 3D algorithms, require significantly less computation resources and training data. Therefore, for the semantic segmentation of individual phases in the composite, a 2D fully convolutional neural network (FCNN) with a U-Net encoderdecoder architecture implemented in FIJI ImageJ was used, [19] see Figure 3. The network architecture consists of two parts. First, the encoder (contracting) part allows analyzing the whole image, and second, decoder (expanding) part is used to produce a full-resolution segmentation (Figure 3c). The contracting path follows the typical architecture of a convolutional network. [21] It consists of the repeated application of two 3 Â 3 convolutions,  [20] d) The result of the segmentation using the U-Net.
www.advancedsciencenews.com www.aem-journal.com each followed by a rectified linear unit (ReLU) and a 2 Â 2 max pooling operation. At each downsampling step, the number of feature channels is doubled. Every step in the expansive path consists of an upsampling of the feature map followed by a 2 Â 2 deconvolution that halves the number of feature channels, a concatenation with the correspondingly cropped feature map from the contracting path, and two 3 Â 3 deconvolutions, each followed by a ReLU. At the final layer, a 1 Â 1 convolution is used to map each feature vector to the desired number of classes. [20] The schematic segmentation workflow is shown in Figure 3. As training data, one manually annotated SXCT slice of 2000 Â 2000 pixels was used. The network was trained with 5000 epochs using the GPU Nvidia GeForce GTX 1050 Ti with 4 GB GDDR5 memory. The full training process took %1 h. As only a limited amount of training data was available, we used excessive data augmentation by applying random rotation around the z-axis in the full 360 range and elastic deformation with random seed displacement vectors, resulting in a total amount of 46 images. After training, the segmentation of the reconstructed volume was performed slice by slice. The segmentation of 1 slice with the dimensions of 2000 Â 2000 pixels took %20 s.
The assessment of the DL segmentation results is shown in Figure 4. In comparison with a manually annotated slice (Figure 4b), the DL segmentation (Figure 4c) shows only a few discrepancies. Note that this SXCT slice with manual annotations was only used for the evaluation of the DL segmentation and not for training of the CNN. A qualitative analysis of the DL segmentation (Figure 4d) shows that most objects, except a few fibers, are detected, and the difference against manual annotations occurs only at the object's borders.
For quantitative assessment of the U-Net segmentation result, we used the DICE coefficient, which is a common evaluation metric to assess the performance of convolutional networks in semantic segmentation tasks. It measures how well the segmentation matches the ground-truth annotations and is defined as where TP is the number of true positive pixels, FP is the number of false-positive pixels and FN is the number of false-negative pixels. Note that DICE ¼ 0 means no overlap; DICE ¼ 1 means   Table 1). In the case of IMs, which have the best contrast and unique shape, it even exceeded 0.8. Despite the fact that Al 2 O 3 fibers, SiC particles, and some IMs could only be differentiated by their morphology, the U-Net was able to correctly assign the class to most of the elements. Another method to assess the segmentation quality is the comparison of the volume fractions of phases with nominal values. In addition, all phases were manually segmented in available SEM micrographs and added to the comparison ( Table 1). The nominal and calculated volume fractions of Al 2 O 3 fibers are in good agreement. The same holds for the values obtained from DL segmentation of SXCT and manually segmented SEM data for IM and Si phases. A significant discrepancy between nominal (supplier) and experimental values was found for the SiC phase. This can be caused by the inhomogeneous distribution of the SiC particles in the cast billet. There is little alternative to check whether the supplier's data are correct, other than investigating a large number of specimens all throughout the billet.
As shown by a 3D rendering of the segmented reinforcing phases (Figure 5), the eutectic Si builds bonds between SiC particles and Al 2 O 3 fibers, creating additional 3D interconnectivity. This leads to a significant improvement of the material's strength. The interconnectivity of individual phases can be assessed based on the segmentation result by considering the ratio of the volume of the biggest particle to the total volume of the phase. The interconnectivity of eutectic Si in the investigated composite was around 78% and of the IM phase was 61%. The interconnectivity of the IM phase appeared to be lower than the one for the matrix alloy (previously calculated and reported in the study by Evsevleev et al. [22] ). This can be caused by the presence of the large amount of ceramic reinforcement in the composite, which hinders the free growth and connection of the IM particles.
The SiC particles are partially agglomerated in small clusters, which are nevertheless homogeneously distributed and randomly oriented within the Al-alloy matrix. The data in Table 1 represent a radical improvement over the state-of-the-art, [23] and can be directly fed into micromechanical models. [12,23] In summary, we showed the successful application of the DL approach to a segmentation problem, which could not be solved by any conventional method (apart from lengthy manual approach). The achieved accuracy is sufficient to estimate the volumetric characteristics of every individual phase in a fivephase composite. The 3D rendering of the phases revealed the formation of an eutectic Si phase between ceramic  www.advancedsciencenews.com www.aem-journal.com reinforcements and IMs. Similar observations have been done by SXCT in Al alloy matrix composite with single ceramic reinforcement phase. [24] The results obtained in this work open a host of possibilities in quantitative 3D microstructural characterization of complex materials by means of CT. As an outlook for future work, we envisage the use of important microstructural features extracted from segmented CT data (e.g., volume fractions, fiber orientation and distribution, and interconnectivity of phases) to refine the previously developed analytical models and create new finite element models. This will lead to a significant improvement in the ability to predict the micromechanical behavior of multiphase metal matrix composites, which is a key for their use in practice.

Experimental Section
The AlSi 12 CuMgNi alloy reinforced with 7%vol of Al 2 O 3 and 15%vol SiC particles was produced by squeeze casting. The molten alloy was infiltrated in a hybrid preform with planar-random-oriented Al 2 O 3 short fibers and SiC particles randomly distributed in the volume. [12] Microstructural characterization of the composite was performed by both SEM and SXCT. Two different sample preparations were used for the microstructural characterization by the SEM. On one side, a sample was embedded in bakelite and finally polished with a 1 μm diamond paste. On the other side, a sample was chemically deep etched using a solution of H 2 O and HCl with a ratio of 80:20 for 90 min. Both types of samples were then studied in a HITACHI S 4800 scanning electron microscope.
The 3D microstructural characterization was conducted by means of SXCT. Cylindrical samples with a diameter of 1 mm were prepared from the bulk material using electrical discharge machining (EDM) with the planes of randomly oriented Al 2 O 3 fibers perpendicular to the rotation axis. SXCT experiments were conducted on the BAMline (BESSY II, HZB Berlin, Germany). The energy of the monochromatic X-ray beam was set to 25 keV, and an effective pixel size of (0.44 μm) 2 was achieved using an optical detector system with a charge-coupled device camera and a 10Â objective. 2400 projections were acquired with an exposure time of 3 s per projection. The reconstruction of 3D volumes was performed by an in-house developed software based on Paganin's phase retrieval method and the filtered back-projection algorithm. The achieved spatial resolution was %1 μm. Prior to segmentation, noise in the reconstructed CT data was suppressed by the application of a nonlocal means filter.