Experimental Demonstration of High-Performance Physical Reservoir Computing with Nonlinear Interfered Spin Wave Multi-Detection

Physical reservoir computing, which is a promising method for the implementation of highly efficient artificial intelligence devices, requires a physical system with nonlinearity, fading memory, and the ability to map in high dimensions. Although it is expected that spin wave interference can perform as highly efficient reservoir computing in some micromagnetic simulations, there has been no experimental verification to date. Herein, we demonstrate reservoir computing that utilizes multidetected nonlinear spin wave interference in an yttrium iron garnet single crystal. The subject computing system achieved excellent performance when used for hand-written digit recognition, second-order nonlinear dynamical tasks, and nonlinear autoregressive moving average (NARMA). It is of particular note that normalized mean square errors (NMSEs) for NARMA2 and second-order nonlinear dynamical tasks were 1.81x10-2 and 8.37x10-5, respectively, which are the lowest figures for any experimental physical reservoir so far reported. Said high performance was achieved with higher nonlinearity and the large memory capacity of interfered spin wave multi-detection.

resonator [14][15][16][17] , although its capabilities are lacking (e.g., large volume, including resonator circuit, and low memory capacity).As a way of solving such problems, it is shown that spin wave interference in ferromagnetic materials satisfies the requirement for the three features of a physical reservoir through micromagnetic simulation [31][32][33][34] .However, there have been no experimental demonstrations that show that nonlinear interference of spin waves has been applied to a reservoir computing system.
Herein, we describe the first demonstration of a physical reservoir computing system utilizing interfered spin waves.In the subject system, an yttrium iron garnet (YIG) single crystal with multi-antennas, which are for the excitation and detection of multi-spin waves, is used as a homogeneous medium.A hand-written digit recognition task and nonlinear time series data prediction tasks were performed to evaluate the performance of the subject physical reservoir system.The maximum accuracy rate for the hand-written digit recognition task was 89.6 %, which is comparable to or higher than the score of the high performance physical reservoirs reported.Memory capacity and the solvability of nonlinear autoregressive moving average (NARMA) and second-order nonlinear dynamic tasks were dramatically improved by interfered spin wave multi-detection, which was used experimentally for the first time.Minimum errors of 8.37 ´ 10 -5 and 1.81´ 10 -2 were achieved, which are dramatically lower than the errors of any other experimental reservoir system reported to date.One of the most noteworthy points in this paper is that our reservoir computing system can predict NARMA10 more precisely than any experimental spintronics reservoir system due to its large shortterm memory capacity (CSTM ~ 26.98 per 100 nodes) and high nonlinearity, which were achieved utilizing interfered spin wave multi-detection.

The measurement configuration of interfered spin waves and the concept of reservoir computing
Figure 1(a) shows YIG single crystal deposited coplanar antennas and its experimental configuration.A static magnetic field was applied perpendicular to a YIG surface.Two excitors and two detectors are situated so as to detect multiple signals from interfered spin waves.Microwave current injected into the excitors by an arbitrary wave form generator (AWG) induces a microwave Oersted field, which drives the precession of the magnetic moments localized near the antenna.This precession is carried away from the antennas.Then, an inverse process occurs where the dynamic magnetic dipole field, produced by the precession of the moments, induces an electromotive force in the detectors, and an oscilloscope detects the induced wave forms.Here, in the experimental configuration shown in Fig. 1(b), detected spin waves are affected by two factors; (1) the interference of spin waves propagating from exciters A and B and (2) the historic effect of remnant spin precession originated from traveling spin wave.These effects give nonlinearity and fading memory properties to a reservoir computing system.Figure 1(c) shows the magnetic field dependence of a voltage induced spin wave.There are obvious variations between each signal.While the amplitude of a wave packet enhances with increasing magnetic field up to 186 mT, the amplitude of wave packets above 186 mT weaken due to damping of the spin wave during propagation, as it changes from an excitor to a detector.Figure 1(d) shows the nonlinearity of an interfered spin wave.As shown in the upper panel, a spin wave traveling between Excitor A and Detector A differs from that travelling between Excitor B and Detector A. Thus, a multi-detection technique can extract a variety of signals.As shown in the lower panel, the difference between interference and a linear combination of spin waves, shown in the upper panel after a time domain of 10 ns, shows that the interfered wave exhibits nonlinearity.This nonlinearity meets one of the necessary requirements of a physical reservoir.Figure 1(e) is a schematic illustration of a reservoir computing system with an interfered spin wave.Time-series data u(k) is input to the reservoir from input layer.The input data is then transformed nonlinear one due to spin wave interference and the historic effect.Thus, a detected signal is extracted as i-th virtual nodes Xi(k), namely neurons interacting with each other in the reservoir, to map input data to high dimensional space.A reservoir readout y(k) is expressed as the product of each Xi(k) and learning parameters Wi, as follow; where n and k are the number of nodes and the discrete time, respectively.

Hand-written digit recognition task
The first demonstration performed with our reservoir computing system was of a handwritten digit recognition task, as shown in Fig. 2 wave, and with spin wave propagating from excitors A and B, respectively.Although the accuracy rates depend on when the reservoir state is extracted, there is no systematic dependence on time.The best accuracy rate for an interfered spin wave is 89.31 % (39 ns) at 180 mT (indicated by a black arrow).The best scoring accuracy rate for a spin wave excited at excitor A is 89.6 % (36 ns) at 172 mT (indicated by a red arrow).The best scoring accuracy rate for a spin wave excited at excitor B is 89.47 % (38 ns) at 200 mT (indicated by a light blue arrow).As shown in Fig. 2(f), the accuracy rate of 69.59 % improved as the number of training samples increased.The best score in this experiment is superior to that of any other physical reservoir computing system (e.g., memristor 5 and magnetic devices 10 ) and is comparable to the excellent 91.3 % score achieved with an optical element 25 , which has a larger node number of 512, and with an ionic liquid device 26 .

Nonlinear dynamical system prediction task
Time series data prediction tasks are widely performed so as to evaluate the nonlinear transform function of a reservoir system.The process flow for such task is shown in Fig. 3(a).In a second-order nonlinear prediction task, a random wave is input to a second-order nonlinear dynamical system.The output d (k) from this dynamical system at k is described as follow; () = 0.4( − 1) + 0.4( − 1)( − 2) + 0.6 % () + 0.1.(2) d(k) depends on not only the current input u(k) but also on the past two states d(k -1) and d(k -2) at discrete times k -1 and k -2.Second term on the right-hand side of Eq. ( 2) is the cross term that makes it a second-order nonlinear system.Before being input to the reservoir system, the original random wave is processed to the pulsed signal, at intervals of 2, 5, 10, 15, and 20 ns, as pre-processing.Each of these signals is input to a reservoir computing system to which perpendicular magnetic fields of 70, 90, 97.5, 150, 169, and 176 mT are being applied.Figure 3(b) shows a comparison with the theoretical output of Eq. ( 2) and the predicted output reconstructed from the reservoir computing system in the training phase, which is measured under a magnetic field of 169 mT at an interval of 5 ns.The normalized mean square error (NMSE) for this task is described as follow, Here, T, d(k), and yp(k) are the lengths in the training phase (T = 3500) or test phase (T = 500), the target signal, and the predicted signal.NMSE at the training phase is 7.66 ´ 10 -5 .A new random input is prepared for the testing phase so as to verify that the trained reservoir computing system can predict output from Eq. ( 2).The compared results in the testing phase are shown in Fig. 3(c).NMSE for the testing phase exhibits a similar value of 8.37 ´ 10 -5 .Figure 3(d) shows NMSE changes at various intervals and magnetic fields.There is a tendency for the following to occur; while NMSE in stronger magnetic fields and at shorter intervals is lower, NSME in weaker magnetic fields and at longer intervals is higher under all measurement conditions.This can be seen in the comparison shown in the two upper panels (w/o interference, Detector A and Detector B) and the two lower panels (interference, Detector A and Detector B) of the figure, where NMSE is dropped overall by utilizing interfered spin waves.NMSE does not depend on the position of the detection antenna, although NMSE does change slightly in case of both interference and non-interference.However, NMSE drastically dropped to 8.37 ´ 10 -5 by utilizing interfered spin wave multi-detection, as is proved by the comparison shown in the two right-hand panels (Detectors A+B) and the remaining four panels (Detector A and Detector B) in the figure.This value is much lower than the values of other physical reservoir computing systems that have been reported, in which the NMSEs of a theoretical reservoir computing system with 24 spin torque oscillators 10 were ~ 1.31 ´ 10 -3 and an experimental reservoir computing system with 90 metaloxide memristors 5 were ~ 3.13 ´ 10 -3 , respectively.The nonlinear autoregressive moving average (NARMA) is a more difficult task than a second-order nonlinear dynamic task to perform, since in order to predict the output of an NARMA model, a reservoir system is required to not only perform nonlinear transform functions but also to exhibit fading memory.Here, we introduce NARMA2 and NARMA10, which need fading memory from the previous 2 and 10 steps, respectively, as defined below;  4) and predicted outputs reconstructed from the reservoir computing system in the training phase and testing phase, which are measured under a magnetic field of 169 mT and an interval of 5 ns, for NARMA2.NMSE variations at various magnetic fields and intervals are summarized in Fig. 4(c).Here, NMSE for NARMA tasks is described as follow; where dave. is the time average of d(k).The measurement condition dependence of NMSEvar is similar to that for a second-order nonlinear dynamics task.The lowest NMSEvar for this task was 1.81 ´ 10 -2 , which is dramatically lower than that of an experimental physical reservoir computing system previously reported [7][8][9] .Thus, reservoir computing systems with interfered spin wave multidetection shows excellent computational performance.Figures 4(d) and (e) show comparisons with the theoretical outputs of Eq. ( 5) and predicted outputs reconstructed from the reservoir computing system in the training phase and the testing phase, which are measured under a magnetic field of 176 mT and an interval of 20 ns, for NARMA10.The NMSEvar for NARMA10 was 2.43 ´ 10 -1 in the testing phase under a magnetic field of 176 mT and an interval of 20 ns when utilizing interfered spin wave multi-detection.Figure 4(f) shows the NMSE variation at various intervals and magnetic fields.
There is tendency, which differs from tendency for the NMSE for the NARMA10 prediction, which tendency is described as follows; while the NMSE is lower in stronger magnetic fields and longer intervals, the NSME at rest is higher.the figure.This result indicates that the large CSTM and high ability of mapping high dimension, achieved with 100 nodes extracted from the multi-detection, is critically important in solving NAMRA10 tasks.This result confirms that a reservoir computing system with interfered spin wave multi-detection exhibits the highest performance of all experimental reservoir computing system with spintronics phenomenon (NMSEvar of ~ 3.7 ´ 10 -1 ) 18 .

Short term memory capacity and nonlinearity of nonlinear interfered spin waves
A short-term memory task was performed to measure the ability of our reservoir computing system for predicting time series data output from a nonlinear dynamic model.The ability for prediction decreases as the step delay increases, as shown in Fig. 5(a).This behavior accurately shows the feature of short-term memory.While a reported experimental reservoir system with spin wave propagation, which does not utilize interference and multi-detection, is able to hold the memory of 8 steps [indicated by a black arrow in Fig. 5 reservoir computing systems that utilizes spin waves with an active ring resonator and anisotropic magnetoresistance [14][15][16][17][18] .Thus, our reservoir system has larger CSTM than said reservoir system with spin wave.Although the reservoir system under the condition labeled I in Fig. 5(b) can predict a NARMA10 model with the lowest NMSEvar of 2.43 ´ 10 -1 , the reservoir system cannot precisely predict second-order nonlinear dynamic task and NARMA2.On the other hand, the reservoir system under the condition labeled II, which has CSTM of 13.1, can precisely predict these tasks, while that in the condition labeled I cannot, even though it has a larger CSTM of 26.98,.This fact indicates that the system under condition I may not have enough nonlinearity to solve NARMA2.
To evaluate the nonlinearity of the system, Lyapunov exponents l were estimated.l is generally used to determine if the response of the system is orderly or disorderly (i.e., chaotic); when maximum l is negative (positive), the system is orderly (disorderly).Furthermore, the degree of nonlinearity of the system improves as l increases.As shown in Figs.5(c) and (d), the system under the conditions labeled I and II have maximum l of 0.30 and 0.56, respectively.Positive maximum l indicates the chaotic nature of an interfered spin wave in a YIG single crystal.The condition labeled II has larger nonlinearity due to a larger maximum l.Thus, the system under this condition has higher linearity than the condition labeled II.To eliminate the possibility of random response from the system, phase portraits of the interfered spin wave are plotted, as shown in Figs.5(e) and (f).The orbit in the figure exhibits aperiodicity, which can also be seen in the portrait describing the chaos of the spin wave in YIG 35 .From the result in this section, there is trade-off relationship between the nonlinearity and the memory capacity of the reservoir system, which is well known as a general rule 36 .To predict second-order nonlinear tasks and NARMA2, it is important for the system to possess high nonlinearity rather than large CSTM.On the other hand, to predict NARMA10, compatibility with large CSTM and nonlinearity is required, which is realized by utilizing interfered spin wave multi-detection.This is one of the unique advantages of the homogeneous, medium-based reservoir computing system in this study.g, trade-off relationship between the CSTM and nonlinearity of an interfered spin wave system.

Conclusions
We achieved the first demonstration of an experimental reservoir computing system with interfered spin wave multi-detection.Our reservoir computing system achieved a recognition rate of 89.6 % for hand-written digits.The NMSEs for a second-order nonlinear dynamic task and NARMA2 were 8.37 ´ 10 -5 and 1.81´ 10 -2 , respectively, which are dramatically lower than the NMSEs of any reservoir computing system reported to date.The subject reservoir system can also predict output from NARMA10 model, with an NMSE of 2.43 ´ 10 -1 , which is the lowest value in an experimental spintronic reservoir computing system.While the largest CSTM in this study was 26.98 per 100 nodes, which is larger than the CSTM for spin wave propagation reported to date [15][16][17] , this system has high nonlinearity, which is generally a trade-off relationship with CSTM.These high performance functions were achieved by utilizing the unique advantages of interfered spin wave multi-detection.Since this technique can be also applied to not only bulk crystal forms but also to thin film forms with extremely small volumes, the said system concept utilizes both spin wave interference and multi-detection to contribute to the implementation of integrated physical reservoir systems with practical uses.
(negative), proximity orbits are detached (asymptotic).In calculation, the Lyapunov exponent is calculated by taking a hypersphere (ε-sphere) in m-dimensional space of minute radius ε around a point, assuming that its change after one step can be approximated linearly, and then estimating the Jacobi matrix using that variation.
The displacement vector μi ∈ R N+L with respect to v(ki) as seen from v(t) is expressed as; != ( ! ) − ().(11)   Since v(t) and v(ki) transit v(t + s) and v(ki + s) when time s elapses, the displacement vector zi at time t + s can be described as; != ( !+ ) − ( + ).(12)   Under the consumption that ε and s are small enough to be negligible,  ! and  !can be linearly approximated as follow; where  S () is the Jacobi matrix to be estimated.Thus, the Jacobi matrix can be estimated as follows; U () =  ! ! .( ! ! .) +$ .(14 S (0) that is  S () at t = 0 is described by QR decomposition, as follows; where Q and R are the orthogonal matrix and the upper triangular matrix.At time t + 1, multiply  U () ∑ lnH ) !! H .
Here,  ) !! is the i-th diagonal element of the upper triangular matrix Rk.

Fig. 1|
Fig. 1| Concept of a reservoir computing system with interfered spin wave multi-detection a, YIG single crystal deposited coplanar antennas and its experimental configuration.b, Schematic (a).Hand-written digits are expressed by combinations of voltage signals with 16 different binary pulsed voltages.In the reservoir, these 16 different voltage signals are transformed to 16 different wave forms through the YIG single crystal, since excited spin waves also have various shapes.Thus, each hand-written digit is reconstructed by voltages extracted from the 16 wave forms.Figure 2(b) shows normalized induced-voltage variation at various pulsed 4bits of data.The individual values do not overlap each other, meaning that the reservoir system gives the input data sufficient diversity.The fading memory property in the reservoir system is important for this task, since the degree of dispersion of the 16 different induced voltages is based on past input pulses in a 4-bit pulse train.Therefore, as shown in Fig. 2 (b), it is revealed that the spin wave possesses a sufficiently large fading memory property to perform the hand-written digit recognition task.Figures 2(c), (d), and (e) show the accuracy rate for hand-written digit recognition tasks with interfered spin

Fig.
Fig. 2| Hand-written digit recognition tasks a, General concept of a process flow diagram of a hand-written digit recognition task using a reservoir computing system with interfered spin wave.b, Normalized induced voltage variation at various pulsed 4-bit data.c,d,e, Accuracy rate variations at various times when the value of the induced voltage is extracted from an interfered spin wave (c), spin wave excited from Exciter A (d), and spin wave excited from Exciter B (e). f, Number of training samples dependence of accuracy rate for handwritten digit recognition.

Fig. 3|
Fig. 3| Nonlinear dynamical system prediction task a, General concept process flow diagram of a time-series prediction task using a reservoir computing system with interfered spin wave.b,c, Predicted results at the training phase (b) and testing phase (c) of a second order nonlinear system.The black, green, and red lines denote the target, and the prediction results at the training and testing phases, respectively.d, NMSE variation for prediction of a second-order nonlinear system at various magnetic fields and intervals.(Upper-left) Detector A without interference.(Upper-middle) Detector B without interference.(Upper-right) Multi-detection without interference.(Lower-left) Detector A with interference.(Lower-middle) Detector B with

+ 1 .
Figs.4(a) and (b) show comparisons with the theoretical outputs of Eq. (4) and predicted outputs Fig. 4| The results of NARMA2 and NARMA10 prediction tasks a, b, Prediction results at training phase (a) and testing phase (b) of the NARMA2 system.The black, (a)]17 , the short-term memory of our reservoir system achieves longer short-term memory, above 20 steps [indicated by blue (condition I) and orange (condition II) arrows in Fig.5(a)], by utilizing interfered spin wave multi-detection.The respective memory capacities (CSTM) under various measurement conditions is calculated from the area under the forgetting curves, as summarized in Fig.5(b).While CSTM drops as interval length becomes longer at the region below a magnetic field of 150 mT, CSTM improves at the region above a magnetic field of 169 mT and an interval length of 10ns.The largest CSTM is 26.98 per 100 nodes, which is a much larger value than the respective CSTM below 4.85 per 20 nodes and 12.0 per 200 nodes of an experimental

Fig. 5|
Fig. 5| Short-term memory and nonlinearity of the reservoir system a, Forgetting curves corresponding to conditions I and II.The solid black line is the forgetting curve of a spin wave delay line active ring resonator 17 .b, The memory capacity variation for short term memory tasks at various magnetic fields and input intervals.I and II are the conditions that can achieve the lowest NMSE for NARMA2 and NARMA10, respectively.c, d, Lyapunov spectra (c) and phase portrait (d) of the conditions I. e, f, Lyapunov spectra (e) and phase portrait (f) of conditions II.
by the orthogonal matrix from one time earlier to obtain following relation; U () D =  DE$  DE$ .(16)Using the upper triangular matrix at each time obtained in this way, the λ is obtained as follows;