Lasers that learn: The interface of laser machining and machine learning

Laser machining is a highly flexible non-contact fabrication method used extensively across academia and industry. Whilst simulations based on fundamental understanding offer some insight into the processes, the highly nonlinear interactions between laser light and matter, and the variety of materials involved, mean that theoretical modelling is not particularly applicable to practical experimentation. However, recent breakthroughs in the field of machine learning now mean that neural networks are capable of accurate and rapid modelling of laser machining at a scale, speed, and precision well-beyond existing theoretical approaches, with applications including 3D surface visualisation and real-time error correction. In this review, a perspective at the intersection of laser machining and machine learning is presented, followed by a discussion of the future milestones and challenges for this field.

This impressive number of industrial and academic applications is due in part to significant flexibility with respect to parameters such as pulse length, pulse energy, wavelength, and beam size. However, this flexibility comes at a price, namely, the significant amount of time and effort that must be dedicated to finding the optimal set of parameters for each manufacturing process. For example, if you need to vary five parameters with 10 steps each, that corresponds to a total of 10^5 experimental trials. Because of the often highly non-linear relationships amongst parameters, the standard practice is generally the systematic collection of laser-machining data for all parameter combinations to identify the optimal combination. However, this process is both time-consuming and unfocussed and can take days or weeks, hence costing unnecessary effort, time, and money. Even when the optimal parameters have been determined, small changes during manufacturing, for example in laser power or beam shape, can result in final product quality that is below the required standard, again with associated costs in time and money. What is needed, therefore, is a set of modelling methodologies for identifying optimal parameters and providing real-time monitoring and error correction during manufacturing.
However, the processes that describe laser machining, such as the light-matter interaction, heat conduction, phase change of material, and material removal, are particularly complex and hence challenging to model precisely and directly from a theoretical standpoint. The exact details of these processes depend on many factors, including laser parameters (e.g. wavelength, fluence, and pulse length), associated diffraction effects, and sample properties (e.g. absorption, reflectivity, melting temperature, and ablation threshold). In addition, different interaction effects become dominant as the temporal interaction length varies from continuous wave to ultrafast (i.e. femtosecond pulses), and the additional non-linear effects that can occur for short pulses, such as multiphoton absorption and ionisation, further increase this complexity.
In general, theoretical modelling of laser machining can be divided into analytical and numerical treatment [58]. Analytical modelling is usually based on a specific property of interest along with an associated set of assumptions, with examples including heat conduction and material phase change [59], residual stress [60], striation and surface roughness [61], analysis of assisting gas jet modelling [62,63], reaction of oxygen with molten surface [64], energy balance [65] and melt geometry [66]. Numerical modelling approaches such as the finite element method (FEM) [67][68][69][70] can in general offer a broader solution, albeit approximated because of the requirement for calculation nodes whilst also being heavily reliant on significant computing resources.
The FEM is a widely used numerical method for solving partial differential equations in two or three space variables whilst considering specific boundary conditions. The approach is used widely in domains including structural analysis, fluid flow, electromagnetic field, and heat transfer, and hence has an obvious application in the modelling of laser machining. Typically, a larger system is divided into basic parts, known as finite elements, through a discretisation algorithm that effectively transforms the system into a mesh-like structure. A set of algebraic equations can be devised for each element and then assembled to describe the entire system. Through the introduction of boundary conditions, the equations can then be solved to provide a solution for a specific simulation.
The FEM has been applied to a wide range of laser and laser-assisted machining, with particular emphasis on the heataffected zone. As a general overview, Shetty et al. [71] have shown the application of FEMs for a range of conventional manufacturing process machining, including turning, milling, trimming, and drilling. Van Elson et al. [72] demonstrated analytical and numerical solutions for heat conduction and diffusion for a localised laser-produced heat source, with applications in laser machining, welding, layered manufacturing, and laser alloying. Dai et al. [73] showed how FEM can be used to solve heat transport at microscale.
Specifically for laser machining, Negarestani et al. [74] presented an improvement to the quality of laser machining for cutting and drilling carbon-fibre-reinforced polymer composites by understanding the mechanisms of transient thermal behaviour and its effect on material removal. Using FEMs, a three-dimensional model for simulating the temperature field and subsequent material removal was developed that was able to predict the dimensions of the heat-affected zone during machining. Similarly, Dixit et al. [75] showed the application for investigating thermal stress and temperature distribution of laser power for laser materials processing of aluminium oxide ceramic, and Shalahim et al. [76] conducted similar work with acrylic sheet for a range of material models and process parameters. Martinez-Calderon et al. [77] showed the application of finite difference time domain modelling for optimisation of laser materials processing to produce tailored optical properties of diamonds through the fabrication of surface nanostructures.
Similarly, work by Romer et al. [78] showed the finite difference time domain for modelling laser-produced nanostructures on a range of materials. Afrasiabi et al. [79] showed the application of a particle strength exchange model that offers some advantages over more conventional FEM for modelling metal removal in laser drilling, and they later showed a mesh-free method for accurate three-dimensional thermal simulation during laser drilling [80].
Laser machining is a key enabling technique for a range of industrial manufacturing techniques [81][82][83] including laserassisted machining, which uses laser energy to locally preheat and soften materials before applying a conventional cutting tool, and has seen many applications of finite element modelling. Tagliaferri et al. [84] used the FEM for modelling the laser heating process for laser power, scanning speed, focus distance, and surface roughness to enable reduced cutting-tool wear and cutting forces. Yang et al. [85] used the FEM to model the depth and width of the heat-affected zone when laser-heating a titanium alloy. Tian et al. [86] showed the application of the FEM for modelling the progression of defects, such as cracks, in the shear zone during laser-assisted machining of silicon nitride ceramics. Singh et al. [87] presented the FEM for optimisation of laser-assisted machining of steel via modelling temperature distribution, flow stress reduction due to laser heating, and cutting forces. Kim et al. [88] used the FEM for laser-assisted machining of 3-D shapes for the optimisation of preheating temperature. Finally, Shi et al. [89] used the FEM for the optimisation of laser-assisted machining of nickel chromium alloy.
Whilst effects that are deterministic and can be described by equations such as absorption and heat transfer can be accurately modelled via theoretical approaches, as the domain grows in size (i.e. a larger target sample), the computational requirements similarly increase. For FEMs, techniques to reduce this computational complexity include using a variable mesh or symmetry arguments to reduce dimensions [90], or in the case of laser machining, using the edge of the mesh as the laser cut position [74]. Even with such techniques, however, there will always be a practical limit to the size of the domain that can be accurately modelled. Effects that are nondeterministic, such as experimental variations in laser power and beam shape, sample defects, and the production of debris [91,92], are even more theoretically challenging to the model, as they require intimate knowledge of the statistical variations of the experiment. Rather than starting from a theoretical understanding of the system, the machine-learning modelling approach (as discussed in this review) starts directly from the experimental data and hence alleviates the challenges associated with acquiring a complete understanding of the experimental environment. Of course, such a data-driven approach also ensures that all physical effects can be included in the model, including those that are not fully understood.
Laser micromachining, which generally refers to the machining of microscale and nanoscale features, has its own set of modelling challenges [29,93]. Typically, femtosecond pulses are used, which results in a reduced heat-affected zone compared with using longer pulses [94] but also introduces many associated non-linear effects. Whilst these effects unlock capabilities such as surface nanoscale structuring [95] and machining inside transparent volumes [96,97], the increased complexity means a data-driven machine-learning approach for modelling becomes even more advantageous. The most critical factor in enabling effective modelling via machine learning is sufficient collection of appropriate training data. In the case of micromachining, this appears to be even more critical owing to the highly non-linear response of the target material during machining. The training data must also provide information about the diffraction limit (e.g. see [98], where the neural network was shown to learn an effect equivalent to the diffraction limit). Finally, if the objective of the machinelearning approach is to provide a 3-D surface profile, then an appropriate high-resolution imaging device, such as a white light interferometer, may be needed for collecting the training data.
Rather than using a theoretically driven solution for modelling laser machining, this review describes the application of machine learning for enabling a data-driven solution (i.e. directly from experimental data), as highlighted in Figure 1. This approach is based on the premise that machine learning can be used to empirically identify the relationships amongst experimental parameters directly from experimental data and hence can offer a capability for modelling laser-machining processes. Critically, as discussed in this review, the calculation speed offered by the machine-learning approach leads to the potential for real-time modelling of parameters and process optimisation. Section 2 introduces often-used machine-learning approaches, with a focus on neural networks. Section 3 reviews the literature at the interface of laser machining and machine learning. Section 4 draws together the trends in the literature and discusses potential milestones and challenges for the field. Section 5 presents the conclusions.

| INTRODUCTION TO MACHINE LEARNING
Machine learning is a broad field that encompasses a wide variety of algorithms that automatically improve through experience. Critically, such approaches do not require an explicit programmatical description of the relationship and the rules that govern the data-instead, the algorithms learn data properties directly from the data. The field of machine learning can be divided broadly into supervised, unsupervised and reinforcement learning. Supervised learning [99][100][101] involves the generation of a mathematical model that maps input data onto the associated output data and hence can be used to make predictions. Unsupervised learning [102][103][104] is used to discover structure within data and to find clustering or grouping of data points. Reinforcement learning [105][106][107][108] governs how a software agent makes decisions within an environment to maximise a reward. Whilst all three are important areas of research, supervised learning has been the predominant focus of laser-machining investigations to date and hence is the focus here. As shown in Figure 2, within the field of machine learning (which can be described as a type of artificial intelligence) exists the field of deep learning. Further, within the field of deep learning exists the critically important convolutional neural network (CNN). A further subtype, known as the conditional generative adversarial network (cGAN), is a neural network architecture that involves two CNNs that assist each other during the training process.

| Machine learning models
Within the field of machine learning, there is a wide range of models [109][110][111], with the most prominent briefly discussed here. An artificial neural network (ANN) is a network of connected units called artificial neurons, which are loosely inspired by the neurons in biological intelligence. As shown in Figure 3, an ANN typically has an input layer, a hidden layer, and an output layer. Internally, each artificial neuron receives information from adjacent neurons, and after processing the sum of information using a non-linear function, transmits the processed information forwards through the network. The learning process typically involves automatic optimisation of the non-linear functions at each neuron via an algorithm known as backpropagation [112,113]. The network therefore operates as a transfer function and can be used to transform input data into output data.
Decision trees [114,115] utilise a network of evaluations to make predictions about the value of an item directly from observations of that item. Support vector machines [116][117][118] allow representation of the sample data as points in space mapped such that there is a clear divider between two or more subsets of the sample data. New data items can then be mapped to the same space, and hence a category prediction can be made based on their location with respect to the divider. Regression analysis [119] includes a wide range of statistical methods used to assess the relationships between data inputs and associated outputs and includes both linear (e.g. using a least-squares error metric to calculate a best-fit line through a data set) and nonlinear regression (e.g. using a quadratic of higher-order polynomial fit). Bayesian networks [120,121] are a method for creating a probabilistic graphical connecting model that allows F I G U R E 1 Contrasting the practical application of (top) theoretically driven and (bottom) data-driven approaches for modelling laser machining MILLS AND GRANT-JACOB -3 the relationships amongst variables to be identified and hence for a probabilistic distribution of outcomes to be predicted for any given input. Genetic algorithms [122][123][124] are a type of search algorithm inspired by the process of natural selection whereby algorithmic equivalents to biological processes such as mutation and recombination effects are used to create new genotypes to encourage the likelihood of identifying effective solutions to a given problem.

| Deep learning
A fundamentally important type of ANN is the deep neural network [125], which is generally defined as an ANN with multiple hidden layers. Critically, owing to the nature of nonlinear functions, as the number of hidden layers in an ANN increases, the network will mathematically tend towards a universal function approximator [126,127], and as such is able to approximate any function, hence enabling practical application to considerably more complex tasks.
A CNN is a type of deep neural network that is generally applied to image data because of its capability for processing spatially represented data. CNNs are loosely inspired by the neuronal architecture in the biological visual cortex, where cortical neurons respond to a visual stimulus in a restricted area of the vision where these areas overlap. Similarly, the computational neurons in a CNN process a single region of data (although overlapping with adjacent regions) through a series of convolutional processes. A convolution is a mathematical operator used to calculate the integral of the product of two functions, as one is shifted, for all possible shift values. In a CNN, the input data for each layer is convolved with a generally much smaller array, known as a filter, to produce data for the next layer. The consequence of this process is that the output data contains a definitive description of the similarity of the filter and the input data. The convolutional process therefore acts as a pattern-matching algorithm that identifies where (and how strongly) the filter data matches the input data. In practice, if a filter contained a circular shape, this process would identify all locations in the input image where a circular shape with the same properties existed. Consecutive convolutional layers therefore introduce the capability for higher and higher abstraction of pattern recognition, whilst increasing the number of filters at each layer improves the capability to recognise a greater number of patterns in the input data. Critically, these filters are randomly initialised and optimised automatically during the training process. CNNs have been used widely for both the classification of images (i.e. describing the content in images) and image-to-image processing (i.e. transforming one image into another). The concept of a classification CNN is shown in Figure 4, which shows the transformation from 2-D data input (e.g. a photograph of a scene) to output that describes the content of the input data (e.g. what objects are in the photo) [128][129][130]. The concept for an image-to-image CNN is shown in Figure 5, which shows the transformation of input 2-D data to output 2-D data with a different appearance through encoding and decoding processes (e.g. for image enhancement or to provide spatial labelling [131]). The principles of both approaches are similar and involve the aforementioned convolutional processes in addition to a range of many other algorithmic processes including pooling, batch normalisation and dropout [132].
A cGAN [133,134] can be envisaged as two CNNs that are trained simultaneously. Specifically, the network consists of a generator, which transforms an experimental input image into a generated output image, and a discriminator, which receives an experimental input image along with either the associated experimental output image or the associated generated output image, as shown in Figure 6. The discriminator then judges whether the output image provided to it is experimental or generated. During training, the discriminator becomes more F I G U R E 2 Relationships amongst the different types of machine learning discussed in this review. cGAN, conditional generative adversarial network; CNN, convolutional neural network F I G U R E 3 Schematic of an artificial neural network showing the input, hidden, and output layers effective at judging whether the generator output is realistic, which enables the generator to become more effective at producing realistic images. The nature of this adversarial training approach is that both the generator and the discriminator improve in effectiveness during the training process [135,136] until a Nash equilibria is reached [137]. Once training is complete, the generator can be used as an imageto-image processing function for the transformation of images that were not in the training data set.

| MACHINE LEARNING AND LASER MACHINING
As highlighted in Section 1, laser machining is extremely challenging for accurate modelling when starting from the fundamental physical rules of light-matter interaction, particularly for ultrashort pulses. Whilst there is a wealth of modelling approaches, the challenges associated with scaling up such models to practical sizes and the inclusion of all experimental characteristics has driven the search for alternative modelling solutions. Machine learning has recently shown itself to be an important alternative, as it offers the capability to produce an experimental model directly from experimental data. In other words, machine learning enables a data-driven modelling approach for laser machining.
As shown in this section, machine learning can be applied to laser machining to provide a modelling environment that enables a range of capabilities, such as the optimisation of processing parameters, identification of the relationships amongst parameters, and predictive visualisation of the outcome of laser machining under different experimental conditions. A critically important advantage of using a datadriven modelling approach is that all experimental characteristics, including those that may be challenging to model programmatically such as beam shape inhomogeneity or experimental noise, are automatically included in the modelling system. In general, the results in the literature have focussed on neural networks, and hence the discussion here focusses on this machine-learning technique.
In general, the practical application of neural networks for modelling laser machining includes three stages, as shown in Figure 7. The first stage is the collection of experimental data in the form of data inputs and their associated data outputs. As an example, this could be laser fluence and repetition rate as the inputs and a measure of the depth of the laser machined features as the output. The second stage is the training of the neural network, where the internal weights of the network are automatically adjusted via backpropagation. The neural network accuracy is typically measured during the training process to optimise training progress. The third stage involves providing the trained neural network with a different set of input values, where the neural network then provides a prediction for the associated output values. Once trained, the neural network can thus be used as a 'black box' modelling unit that transforms input data into the associated output data. Notably, the nature of the transformation from input data to output data is specifically determined by the format of the input and output data used for training. For this example, the neural network would be able to predict the feature depth for any laser fluence and repetition rate including, of course, those that were not included in the collected experimental data.
As shown in Figure 7, once the neural network is trained, it can be applied to the modelling of input data that was not included in the training data. Here, a key strategy in testing the accuracy of the predictions from a neural network is separating the collected experimental data into training and validation pairs. In practice, this ensures that the data used for validation is not processed by the neural network during training and hence operates as 'unseen' data. It is important to note that as the training occurs on experimental data, bias (or inaccuracies) in the data will be replicated within the neural network. It is also true that a machine-learning system trained on a set of data from one environment may not be able to accurately predict the case for another set of data, if the properties of the latter are not represented in the original training data.
To highlight the relevance of the interface of laser machining and machine learning, Figure 8 shows the yearly frequency of publications for a range of search keywords (laser machining, machine learning, ANN, CNN and generative adversarial network) in Google Scholar, with data for 2020 estimated based on the number of days remaining in the year at the point of data collection. This figure therefore provides a snapshot of publication activity for laser machining and all machine-learning research fields over the last 40 years -'laser machining' has seen an exponential rise, representing a steady improvement in manufacturing processes and techniques over this period. 'Machine learning' saw exponential growth from 1990 until 2018, followed by a rapid drop, which could be argued to be the result of a preference for more descriptive terminology, as this coincides with the advent of rapid growth in the use of 'convolutional neural network' as a search term.

| Steel
The most common material to be investigated has been steel, as this has many important industrial applications. The literature presents a series of approaches for identifying the optimal laser parameters for specific tasks. In general, this involved using an ANN with a single hidden layer, with a set of laser and sample parameters as the input and a measure of materials processing quality as the output. Ciarana et al. [138] used an ANN with a single hidden layer (network shape of 4-5-1) to optimise the machining parameters when machining straight and tapered walls in hardened steel (AISI H13) with an Nd: YAG with a 20 µm spot size. The input parameters for the ANN were pulse intensity, scanning speed, pulse frequency and cutting time, and the output measures were dimensional prediction and surface roughness. Sivarao et al. [139] used an ANN (shape of 3-8-1) to predict end product quality and F I G U R E 7 Concept of practical application of machine learning for laser machining F I G U R E 8 Yearly frequency of publications with content corresponding to displayed search terms surface roughness when cutting 3 mm thick manganese molybdenum with a 4 kW CO 2 laser (Helius Hybrid CO 2 laser cutting machine). The three input parameters were cutting speed, gas pressure and power, and the ANN achieved ∼90% prediction accuracy. Dhara et al. [140] used an ANN (shape 4-25-2) for identifying the parameters needed to increase the depth of a machined groove and reduce the recast layer when using an Nd:YAG with a spot size of 100 µm for micromachining of tungsten-molybdenum general-purpose highspeed steel. The input parameters were lamp current, laser repetition rate, pulse width and air pressure, and the outputs were depth of the groove and height of the recast layer.
Rajamam et al. [141] used regression (a type of machine learning) to model the effects of power and feed speed on the quality of machining, in terms of kerf width, surface roughness, striation frequency and the size of the heat-affected zone, when machining steel with a CO 2 laser. Teixidor et al. [142] explored a range of machine-learning approaches to model the machining of microscale geometries in hardened steel using an Nd:YAG. The parameters of scanning speed, pulse intensity and repetition rate were optimised to control the depth and width of channels, the surface roughness and the material removal rate. The ANN and decision tree approach was found to be superior to the k-nearest neighbours and linear regression approaches. Zhang et al. [143] used an ANN to predict the cut quality for a 3 KW CO 2 laser cutting 1 mm thick steel, with inputs of laser power and cutting speed and outputs of dross, roughness and width. Radovanovic et al. [144] demonstrated the application of regression and an ANN for the modelling of surface roughness when cutting mild steel with a CO 2 laser, and results showed that the ANN approach provided considerably higher accuracy and was effective in modelling the non-linear and complex relationships amongst machining parameters.
Nakhjavani et al. [145] investigated the application of an ANN for optimising process parameters when using an Nd: YAG laser for percussive drilling into 2.5 mm mild steel EN3 sheets. As shown in Figure 9, the input parameters of peak power, pulse width, pulse frequency, number of pulses, assist gas pressure and focal plane position were optimised to predict the hole entrance diameter, circularity of hole entrance and hole exit, and hole taper. Sohrabpoor et al. [146] applied an ANN and adaptive inference model to predict the surface quality of laser-processed 316L stainless steel cylindrical pins when machined with a CO 2 laser. Velli et al. [147] showed that the laser-induced periodic surface structures [148][149][150] produced via a Yb:KBW femtosecond source on stainless steel (and titanium alloy and crystalline silicon) could be predicted via a range of machine-learning models.
Laser-based material modification without material removal is a technique often applied to steel for manufacturing applications. One such example, laser hardening, which involves the heating and cooling of the target metal to introduce metallurgical modification, is inherently complex, and thus the attraction for an ANN prediction tool. Lambiase et al. [151] used an ANN with three hidden layers to predict the optimal laser parameters for the laser hardening of steel. Maji et al. [152] used an ANN (shape 3-11-1) to predict the optimal parameters for manufacturing dome profiles through surface melting of AISI 304 stainless steel sheets when using a Yb fibre laser, with inputs of laser power, scan speed and spot diameter, and associated output of dome height. A second example of material modification is the laser-based bending of steel. Cheng et al. [153] used an ANN to predict the bending angle of steel sheets when irradiated by a CO 2 laser for different values of scan speeds, laser powers, and laser spot sizes along with the thickness and length of the metal workpiece. The authors found that an ANN with a single hidden layer of five neurons produced the optimal answer and was favoured over the regression model.

| Aluminium and other metals
Aluminium features less frequently than steel but in general has a similar approach to parameter optimisation through the use of an ANN. Campanelli et al. [154] used an ANN to determine the optimal scanning velocity and pulse frequency for the F I G U R E 9 Adapted from [145]. Architecture of artificial neural network used for identification of optimal machining of steel MILLS AND GRANT-JACOB -7 desired ablation depth in an aluminium alloy when using a Nd: YVO 4 laser source with focus diameter of 40 μm. Rahimi et al. [155] investigated the optimisation of laser engraving of an Al-SiC composite via a Q-switched Nd:YAG laser via an ANN with shape of 4-9-1. The ANN inputs were gas flow, distance between workpiece and beam focus, laser repetition rate, and laser pump value, and associated ANN outputs were depth, width, and contrast of engraved zone. Feng et al. [156] used an ANN to model the key processing parameters for underwater laser milling of RSA-905 aluminium.
Results for machining in other metals include the following. Yousef et al. [157] used a set of two ANNs to model the mean depth and diameter of the crater in the materials of brass, copper and stainless steel when machined with an Nd: YAG with varying pulse energy. As shown by the architecture in Figure 10, Klancnik et al. [158] used an ANN for predicting roughness and kerf width when using a CO 2 laser for cutting tungsten. Barletta et al. [159] demonstrated that an ANN can be applied for the optimisation of paint stripping of epoxypolyester coatings on aluminium substrates when using a 1.5 kW diode laser source. As shown by the network architecture in Figure 11, Biswas et al. [160] used an ANN to model laser microdrilling of titanium nitride-aluminium composite using an Nd:YAG, with ANN inputs of lamp current, pulse frequency, pulse width, air pressure and focal length.

| Plastic, ceramic, glass and other nonmetals
The manufacturing approach for machining plastics is generally the use of a CO 2 laser to produce melting. However, melting is still a complex process that can be challenging to optimise. Desai et al. [161] used an ANN to predict the cutting depth when machining a variety of thermoplastics with a CO 2 laser for varying laser power, cutting speed, absorptivity, and latent heat of vaporisation. Noor et al. [162] used an ANN (shape 3-7-1) to optimise the surface roughness when cutting 3 mm thick acrylic sheets with a CO 2 laser for ANN inputs of laser power, cutting speed and tip distance. Yang et al. [163] demonstrated the application of the Taguchi method (a statistical model) for improving predictive accuracy when using an ANN to model CO 2 laser cutting of PMMA. Tsai et al. [164] optimised the cutting of the epoxy on quad-flat no-leads packages (used for attaching integrated circuits) using a 1 μm wavelength solid-state laser. Dhupal et al. [165] used an ANN to optimise the machining of microgrooves in aluminium titanate ceramic when using an Nd:YAG laser machining microgrooves in aluminium titanate ceramic. The ANN (shape of 5-9-9-3) had inputs of lamp current, repetition rate, pulse width, assist air pressure, and cutting speed and outputs of upper width, lower width and depth of the trapezoidal microgroove. Karazi et al. [166] used an ANN to predict the width and depth of microchannels in glass when using a CO 2 laser. Song et al. [167] showed that an ANN can be used to optimise the process of laser-assisted machining to preheat and F I G U R E 1 0 Adapted from [158]. Network architecture used for the optimisation of CO 2 laser parameters for cutting tungsten F I G U R E 1 1 Adapted from [160]. Architecture used for optimisation of parameters for the microdrilling of titanium nitride-aluminium hence soften the target material to reduce machining time and wear on mechanical cutting tools. Liao et al. [168] used decision tree methods to predict the damage growth on fused silica optics in the case of an Nd:YAG laser.
Jacob et al. [169] used an ANN to predict etch depth for photoresist removal via a 248 nm excimer laser. Yildiz et al. [170] modelled thermal damage using an ANN for a range of lasers for biological applications. Laser materials processing also has application in the clothing industry, including the production of faded and torn products such as jeans [171]. Hung et al. [172] showed that an ANN can be used to predict the colour properties of cotton fabrics under various material parameters. The ANN (shape 8-25-4) had inputs including fibre composition, fabric density, mass of fabric, fabric thickness and density of yarn and outputs including a range of parameters used to define colour yield. The same author [173] also showed that two hidden layers assisted in the colour prediction when varying laser processing parameters related to intensity, resolution and speed.

| Deep learning
In general, the literature associated with the application of deep learning for laser machining presents capabilities well beyond those associated with ANNs. Of particular importance here is the capability for applying image-based data (i.e. 2-D data) as an input for processing by the neural network. Oh et al. [174] used a cGAN and CNN to provide a thermal simulation to predict the hardness distribution when using a 2 kW fibre laser for heat-treatment of steel, clearly showing improvement over previous modelling approaches. Sun et al. [175] used a CNN to identify and predict the effectiveness of laser cleaning of rusty iron through analysis of images of the samples. In food science, Chen et al. [176] used a CNN to predict the visual appearance of dough when irradiated by a CO 2 laser for different laser and dough parameters.
A critical result in ultrafast laser machining with deep learning has been the integration of camera images during manufacturing to assist in real-time feedback, particularly given the additional complexities arising from the use of femtosecond pulses. As shown in Figure 12, Mills et al. [177] used images of the sample during laser machining to identify the material type (silica and nickel), the number of laser pulses used for machining, and the laser fluence. The motivation here was that the effects of experimental noise, such as laser fluence fluctuations, could be identified in real time during laser machining. Similarly, Tani et al. [178] used the interferometric speckle pattern from the surface of the laser-machined region as an input to a CNN in order to identify sample parameters including the depth of machining and the type of material (aluminium, copper and nickel). Here, the anticipation is that real-time feedback, in combination with the capability for a neural network to identify the material type, has significant potential for the enhanced laser-based processing of structures consisting of layers of different materials.
An important direction in this field has been the demonstration of deep learning for predictive visualisation of the 3-D surface profile of a target material, when using a variety of spatial-intensity profiles for ultrafast laser machining, as shown by the concept in Figure 13. This has been achieved through the application of neural networks that can process arrays of data as both the input (e.g. spatial-intensity profile) and the output (e.g. 3-D depth profile). The complexity of modelling such a dynamic from a theoretical understanding of lightmatter interaction is significant because of a plethora of effects that include both the non-linear interactions typical of ultrafast laser machining and the effects of diffraction associated with non-Gaussian spatial-intensity profiles. Whilst liquid crystal spatial light modulators have seen many applications in ultrafast laser machining [179][180][181], the focus on machine learning applications has seen the use of digital micromirror devices (DMDs) [182][183][184] for beam shaping. DMDs have seen extensive use as spatial light modulators for F I G U R E 1 2 Adapted from [177]. The potential for using a neural network for real-time feedback MILLS AND GRANT-JACOB -9 both additive [185][186][187][188][189] and subtractive [190][191][192][193][194] laser-based manufacturing.
Mills et al. [98] showed that a cGAN can be used to transform the spatial-intensity profile used for ultrafast laser machining into a generated scanning electron microscope (SEM) image of the machined sample, hence providing a predictive visualisation of what the sample would look like if imaged via an SEM. Analysis of the generated SEM images showed that the cGAN had learnt properties of diffraction. Heath et al. [195] used a cGAN to predict the 3-D depth profile of the sample for a single laser pulse with a range of spatial-intensity profiles. The capability of this neural network is shown in Figure 14, where the (a)(c) 3-D surface profile predicted via the neural network can be compared with the (b)(d) experimentally measured 3-D surface profile for femtosecond laser machining with a single pulse with one of two different spatial-intensity profiles. Analysis of the outputs from the neural network model was used to predict a range of properties, such as the degree of debris deposition for different beam sizes. This work was enhanced by McDonnell et al. [196], who extended it to include three laser pulses and showed that the cGAN was able to predict that multiple pulses could enable a higher machining resolution than a single laser pulse when each pulse had a specific spatialintensity profile. In combining the concepts of real-time correction and DMD-based beam shaping, Xie et al. [197] showed that a DMD can be used in a real-time feedback loop to provide corrections to the beam shape and position during laser machining, along with the demonstration of the real-time ceasing of laser machining at task completion, despite not knowing the task length beforehand. As presented here, recent results have demonstrated the potential for using ANNs to model thermal and structural effects during laser machining [198,199] and using cGANs for 3-D visualisation of surfaces, such as those produced for single [98] and multiple laser pulses [196]. The authors anticipate that further breakthroughs in datadriven machine learning for laser machining will assist in the development of a new understanding of the transient processes that occur during laser machining. F I G U R E 1 3 Adapted from [195]. Concept of using a neural network for predictive visualisation of laser machining with varying spatial-intensity profiles F I G U R E 1 4 From [195]. Demonstration of using a conditional generative adversarial network for predictive visualisation of the 3-D surface profile when laser machining with different spatial-intensity profiles Whilst this review has provided an insight into the application of machine learning for laser machining, machine learning has similarly already transformed many aspects of manufacturing in general (see e.g. reviews for optimisation tools [200], smart machining [201]. and additive manufacturing [202,203]). Machine learning has also been applied to other laser-based manufacturing approaches [204][205][206][207] and for the control and optimisation of the laser sources themselves [208,209]. Techniques for imaging the sample have also developed rapidly, from using a camera and CNN for real-time monitoring of the laser beam [210]. to using a CNN for positional correction during manufacturing [211]. The application of machine learning to support the use of a laser for sample observation includes feature detection in wood [212], surface roughness measurements [213], laser-based odometry [214] and laser-induced breakdown spectroscopy [215]. Machine learning has also repeatedly shown the capability to enhance computational imaging and microscopy [216][217][218][219][220][221][222]. In summary, machine learning is being applied to the entire spectrum of laser machining, from lasers to material interaction, imaging and sample analysis. Whilst this review up to this point has focussed on the progress of this field to date, this section offers the authors' views on future milestones for this field along with some of the anticipated critical challenges.

| The all-encompassing neural network
To date, and as discussed in Section 3, the application demonstrations at the interface of laser machining and machine learning have been extremely broad. For example, neural networks have been applied to a very wide range of individual materials, such as steel, aluminium, and plastics. However, whilst a neural network trained on experimental data from steel may be extremely effective at predicting the effect of machining steel with different laser parameters, the network in general will be less effective at modelling other materials. A similar argument can be applied to the case for laser parameters, such as laser wavelength. This is simply a consequence of the property that neural networks are generally much more effective at interpolation (i.e. within the training data) than extrapolation (i.e. beyond the remit of the training data set). This leads to the important realisation that neural networks need to be trained on data appropriate to the problem one desires to solve.
Ultimately, it could be argued that the capability of neural networks is based on the three factors of (i) availability of computing power, (ii) quality and amount of training data, and (iii) developments in neural network architecture and algorithmic processes. As the likelihood is that all three factors will continue to increase (in fact, computing power has historically increased exponentially [223]), the anticipation is that neural network capability for laser machining will continue to increase over time. As historical evidence for this hypothesis, the literature around the year 2010 generally shows the optimisation of individual laser parameters, whilst the literature around 2020 describes applications of neural networks for producing predictive surface visualisations [98] and accounting for diffractive effects [196]. This period of time has clearly resulted in a significant development of neural network capability for modelling laser machining, and hence it an interesting thought experiment to extrapolate this progress to identify potential future capability. Here, it is the authors' view that a consummate milestone in this field will be a neural network that can be applied to all lasermachining applications. In practice, this could be a neural network that has been trained on experimental data for all possible materials and under all possible lasers and associated laser parameters, as shown by the concept in Figure 15. Such a neural network could therefore have encoded all light-matter interaction effects (e.g. heating, ablation, ionisation) in addition to light propagation effects (i.e. diffraction and interference) and hence be an all-encompassing model for laser machining. Critically, owing to the nature of neural network architecture, modelling calculations on such a neural network might take only a few milliseconds, which would be a fundamentally groundbreaking capability compared with calculations based on theoretical understanding that can take days or even weeks.

| Real-time error compensation
Neural networks have shown the capability for real-time observation and identification of laser and material parameters during laser machining [177,178]. In addition, neural networks been shown to achieve the modelling of laser machining under a range of laser parameters within milliseconds [98,195], hence demonstrating a real-time modelling capability. It is the authors' view that the synergy of real-time observation and real-time modelling will lead to the provision of the fundamentally important milestone of real-time error compensation, as highlighted by the diagram in Figure 16.
As an example, consider the case where laser fluence erroneously and unintentionally increases by 10% during laser machining, and consequently the target material begins to be machined incorrectly. The real-time observational component (e.g. a camera and neural network) recognises that the appearance of the target material is now different to the anticipated appearance of the material at this point in the machining, and F I G U R E 1 5 Anticipated milestone: A neural network that can model all aspects of laser machining, for all laser and material parameters, on a time scale of milliseconds MILLS AND GRANT-JACOB through processing of the observational data, provides the information that the laser fluence has increased by 10%. However, because of the highly non-linear nature of laser machining, reducing the laser fluence to 90% of the original value for a period would almost certainly not produce the desired correction. Instead, what is needed is a modelling approach that can identify the laser parameters that should be changed (and to what magnitude) to compensate for the previous manufacturing mistake. Critically, this modelling calculation should be completed in real time to immediately compensate for the lasermachining mistake. It is the authors' view that the implementation of such a technique would enable significant improvements to quality control in laser machining including the automation and control of tasks where process conditions are not known before machining commences [197]-for example, the laser cleaning of rust. Such process control may be of particular importance for the future of high-precision ultrafast laser machining, where extremely high non-linearities provide the opportunity for techniques such as sub-diffraction-limit fabrication [53,191,224,225] but with associated extreme sensitivity to fluctuations in the laser parameters (which could therefore be compensated in real time via machine learning).

| Assisting in developing physical understanding
As shown herein, neural networks have been shown to be capable of solving extremely complex problems directly from experimental data and hence without the need for physical understanding of the system. Therefore, it is critical that neural networks are able to provide insight and understanding of the problems that they are used to solve. In practice, this has already happened to a limited extent, where relationships amongst parameters have been identified, for example, between laser parameters and the associated sample surface quality. As a more specific example, Heath et al. [195] identified that neural network outputs were inconsistent with known theoretical calculations, until the theory was adapted to include a misaligned mirror model (the misaligned mirror was later discovered on the experimental setup). Whilst in specific cases the neural network parameters learnt during training can be analysed [226], the number of parameters (around 10^8 for the cGAN in [98]) makes this process extremely challenging because of obfuscation.
Whilst experimental data collection generally requires changing only a single parameter at a time whilst keeping all others constant (e.g. measuring the sample quality for all laser fluences whilst keeping all other parameters constant), the approach for data collection when using a neural network is fundamentally different. Here, there is no need for systematic data collection, and instead, sparse and unsystematic data collection across all parameters is sufficient. As an example, to create training data for the neural network capable of predictive visualisation of laser machining for any spatial-intensity profile (as shown in the concept in Figure 13, with associated results shown in Figure 14), the training data was formed of randomly generated spatial-intensity profiles, as shown in Figure 17. This may be considered in stark contrast to the generally accepted systematic approach for experimental data collection. Importantly, even though neural network training data may be associated with randomly chosen parameter values, the trained neural network can still provide an output prediction for any suitable input and hence can be used to produce a partial differential of data across any dimension of the parameter space. The anticipation is that this predictive capability could therefore be used to enable the construction of a more general set of rules and understanding for laser machining.

| Challenges
A fundamental challenge for deep learning is the significant amount of data that is generally considered necessary for accurate training. In the case of laser machining, this poses a critical challenge, as the accrual of each single data item physically involves parameter changes of experimental hardware along with subsequent laser materials processing in addition to an associated measurement to provide a quantification of the success or quality of the processing under the chosen parameters. Therefore, physically collecting tens or hundreds of thousands of data points could take weeks or longer. Clearly, this is not practically feasible, and hence, an approach known as data augmentation can be applied to increase the effective amount of training data. In the case of image-based data, this often includes approaches such as random cropping [129], flipping [227], dropout [228], batch normalisation [229] and selective data erasing [230]. Such methods have already been demonstrated in laser materials processing [197] and will likely be an important future direction.
To observe the sample faster and more accurately during machining, a key strategy may be reducing hardware processing times through advances in photonics. Several developments are in progress here, ranging from using lithography to create neural networks onto the camera sensor [231] to a plethora of techniques for integrating photonics with computing [232-F I G U R E 1 6 Anticipated milestone: Enabling real-time error compensation through the application of neural networks for simultaneous real-time error identification and real-time modelling 238]. Finally, whilst neural networks are generally designed as deterministic (i.e. constant output for constant input), in specific cases small changes in the input can have dramatic effects on the output. This effect is explored in the field of adversarial attacking [239,240], where it is shown that in extremely specific cases, changes to the intensity value of even just a single image pixel [241] can result in a fundamentally incorrect prediction by the neural network. Whilst research into adversarial attacking is assisting the development of techniques for making neural network predictions more robust [242], the application of neural networks for laser machining may therefore need to include additional levels of fail-safe such as setting hard limits of parameters or additional sanity checking algorithms to ensure the safety of human operators and equipment.

| CONCLUSIONS
This review article has discussed the rich and rapidly developing interface of laser machining and machine learning along with prospects for the future of this field. Laser machining is a well-established manufacturing process that has many applications across academia and industry. However, the flexibility inherent in the process, such as the plethora of different wavelengths, pulse energies and pulse lengths, means that optimisation of parameters for specific tasks can be challenging. Because of the highly non-linear light-matter interactions that occur during laser machining, modelling based on fundamental understanding can be limited in accuracy when applied to practical experiments. Machine learning offers a solution to this problem, namely, a data-driven modelling approach that can be used to optimise laser parameters and as a predictive visualisation tool for laser machining. Critically, machine learning can provide simulation capability on a time scale of milliseconds, hence enabling the capability for realtime modelling. In addition, machine learning has been shown to be capable of real-time observation of the sample being machined to identify laser and material parameters. The authors' postulation is therefore that neural networks will enable real-time error compensation for laser machining with associated future manufacturing breakthroughs in resolution, reliability, and speed. F I G U R E 1 7 Adapted from [98]. Examples of training data collected for the predictive visualisation of laser machining for varying spatialintensity profiles showing (left) programmed spatial-intensity profile and (right) scanning electron microscope image of the corresponding experimental laser-machined surfaces. Note that the training data is therefore formed of randomly generated spatial-intensity profiles MILLS AND GRANT-JACOB -13