Nonlinear Behavior of Dendritic Polymer Networks for Reservoir Computing

Organic electrochemical devices are an emerging class of devices with synaptic properties that might allow for the implementation of next‐generation neuromorphic circuits for power‐efficient computing. Here, a brain‐inspired neural network approach, namely reservoir computing, which relies on a nonlinear transformation of a low‐dimensional input signal onto a high‐dimensional output space for information processing is utilized. The implementation of reservoir computing using dendritic networks of polymeric fibers is demonstrated and the nonlinear response of the polymer networks are analyzed and the sources of nonlinearity are identified. Furthermore, by adding a delayed feedback loop to the reservoir, it is proven that such a network can undergo a bifurcation into a chaotic state, proving sufficient complexity of the system for advanced classification tasks with time‐dependent data. Ultimately, a classification task is carried out and the accuracy is compared of the classification of different degrees of complexity of the system, showing an increase in accuracy from 60% for the base network to 80% when the delayed feedback loop is incorporated.


Introduction
The era of computer science and microelectronics fundamentally transformed our economy and society in the last 70 years. The ever-progressing miniaturization of electronic components in combination with increasing computational capacity [1] enabled the implementation of powerful machine-learning schemes and artificial intelligence (AI). These AI solutions are revolutionizing the way we use and analyze data, transforming the global economy and possibly helping us to solve some of the most pressing problems of our society. Using conventional electropolymerization in which the degree of complexity is significantly increased, thereby enabling advanced tasks of computation such as time-series prediction or heartbeat classification. [16] The computational power of these networks originates from the nonlinearity of their electrical response which is linked, in turn, to the geometry of the network. However, the mechanisms behind the nonlinear response and the connection to the network geometry have not been investigated yet. Moreover, so far random networks have been employed for OECT-based RC and the shape of the networks, and hence their degree of complexity, has not been optimized for specific tasks, which might enable even more efficient classification in future.
Here, we investigate the origin of the nonlinear response of dendritic polymer networks used for reservoir computing. We characterize the nonlinearity of the system by means of the total harmonic distortion (THD) and identify three main sources of the nonlinearity, which are 1) the global gating, 2) the self-gating, and 3) the local gating effects, caused by the coupling between adjacent fibers in the network. The coupling strength and its characteristic time-constants depend on the capacitance of the respective gate electrode as well as on the distance between gate electrode and the fiber carrying the signal. We study the complexity of the dendritic polymer network by implementing a delayed feedback loop and demonstrate by simulations and experiments that such systems can show chaotic behavior which is proving that the complexity is sufficient for advanced classification tasks. Finally, we carry out a classification task with a dendritic polymer reservoir and analyze the results of the classification for different degrees of complexity. In particular, we show how the nonlinearity of the reservoir is linked to the classification accuracy and how this can be improved by the implementation of a delayed feedback loop.

Results and Discussion
Reservoir computing is a well-established paradigm for AI-based information processing, [17] being its architecture composed of an input layer, an output layer, and in between the reservoir itself. The reservoir consists of an arbitrary number of randomly connected nonlinear nodes coupled either in a feed forward or in a recurrent fashion. The difference between RC and traditional recurrent neural networks lies in the fact that the internal nodes in the reservoir are not trained, but rather a random weight is assigned to each one of them and cannot be changed. [18] Only the weights between the reservoir and the output layer are trained which breaks down to a simple matrix multiplication, thus making training very efficient. The purpose of the reservoir is to map the low-dimensional input signals into a higher dimensional output space, where a linear separation between states becomes exponentially more likely. [19] This mapping process relies on one specific property of the reservoir which is its degree of nonlinearity. Furthermore, the reservoir is treated as a black box, meaning that its exact behavior may remain unknown, without making the system any less reliable. [20] The power consumption and the integration effort of a reservoir might be significantly reduced using the so-called singlenode reservoirs [19] in which a delayed feedback loop is employed for time multiplexing of the signals. In this way, the delayed feedback loop emulates the behavior of virtual nodes, which makes the implementation much easier as only one physical complex node is required (see Figure 1a). On the other hand, the use of virtual instead of real nodes limits the dynamics, and therefore, the processing capabilities compared to the one obtained by employing several real nonlinear nodes.
Besides the attractiveness of the delayed feedback loop in terms of increased computational power with a single node, it can be also used to characterize the properties of the reservoir at hand. In particular, as the reservoir is supposed to map the input signal to a high dimensional output space, it is necessary that it has a sufficient degree of complexity. This degree of complexity can be investigated in a delayed feedback system by analyzing the systems potential to show chaotic oscillations. [18,21,22] The mathematical description of a delayed feedback reservoir can be treated within the framework of delayed differential equations (DDE). For an input signal u(t) fed into the reservoir with an amplification factor λ, the reservoir applies a nonlinear transformation f() which is fed back to the input node with a delay time T D . The output state y(t) is then given by where τ is the response time of the reservoir. For τ ≪ T D this expression can be simplified in the so-called adiabatic limit to , describing a long delay system. If such a system shows chaotic oscillations for a certain function f(), and the parameters λ and T D , it is proven that the reservoir is able to map input data with a finite dimension into output data with an infinite dimension. [21,22] In the following, we study the function f() of the dendritic polymer networks by means of the THD and discuss the mechanisms behind the nonlinear response.

Origin and Characterization of the Nonlinear Response of Dendritic Networks
We form single fibers or even dendritic networks of polymer fibers by field-directed polymerization (FDP) [15] (see Figure 1b). To do so, the area between two electrodes (Cr (3 nm)/ Au (50 nm)) is immersed in a solution of acetonitrile containing 50 × 10 −3 m 3,4-Ethylenedioxythiophene (EDOT) and 1 × 10 −3 m tetrabutylammonium hexafluorophosphate (TBAPF 6 ). The presence of water and oxygen impurities in the solution cannot be discarded, as the experiment is run on ambient conditions. Additionally, PF -6 is stable with gold polarized up to our maximum applied voltage of 1.3 V, confirming redox stability, and it is also a suitable counterion to favor the formation of doped PEDOT:PF 6 dendrites in acetonitrile without adding an oxidizing agent in the setup. As a consequence, the reaction is not self-limited at the gold interface, rather, it continues until the fibers bridge the gap. [23] Due to the small gap between the electrodes, we are able to perform the polymerization at low voltages (sinusoidal signals with 2 to 5 V amplitudes and frequencies between 10 Hz and 1 kHz). The electrical characterization of the polymer networks is carried out in a phosphate-buffered saline (PBS) solution. Up to a voltage of 1.3 V no sign of electrolysis has been observed.
Each PEDOT:PF 6 fiber is an electronic-ionic mixed conductor and its state of conductance can be tuned by a gate electrode connected through the electrolyte. Hence, each fiber can be treated as an OECT (cf. Figure 2a). The gate electrode might be an external electrode (e.g., Ag/AgCl) or other fibers within the dendritic network. As we are interested in a nonlinear function f(), it is attractive to exploit the transfer characteristics of these fiber-based OECTs. If a voltage is applied between two electrodes bridged by a fiber, a hole drift current will flow through it. However, when a positive potential is set on the gate, cations from the electrolyte get injected into the organic layer lowering the polaron and current densities on the fiber. The opposite occurs when a negative bias is applied: cations are removed from the organic film, increasing the polaron density and therefore the total current. Analogous to p-type depletion mode MOS transistors, saturation follows for high negative values, here as a result of the fully depleted channel (IαV GS   2 ). For high positive bias, on the other hand, leakage and subthreshold currents become dominant, limiting the reduction of the fiber's current (usually I exp(V GS ), here for 0.3 V < V GS < 0.8 V). These two limit scenarios are highly nonlinear and prove to be exploitable for chaotic oscillations.
Aside from the conventional gating, we observe that OECTs experience a so-called self-gating effect (Figure 2b). Local potential drops along the fiber change the charge distribution in the electrolyte, doping and dedoping the channel progressively. This potential drop has been shown to be nonlinear [24] making each differentially sized length of fiber have a different (nonlinearly distributed) impedance, further affecting the current (given by the transfer characteristic). Moreover, a fiber connecting source and drain might not only be gated by an external metallic gate electrode but in principle by all other fibers in the dendritic network. Generalizing this effect, the aforementioned ion displacement in the electrolyte not only affects the conductivity of the fiber causing it, but also of the nearby fibers (see Figure 2c). This, in turn, induces further polaron drift through the electrolyte, coupling the fibers with one another. Thus, when considering a network of such fibers, each one can be seen as either an excitatory or inhibitory fiber (see Figure 2d). If a connection between input and output exists, any stimulus fed to it will result in a current which will flow to the output (excitatory). If, on the other hand, the fiber is disconnected, the behavior will be inhibitory: neighboring fibers get dedoped as positive potentials are applied to them, increasing the overall impedance and mitigating the effect of the input signal. This phenomenon is highly nonlinear as the degree of coupling is determined by the bias point of each differential length of fiber, the capacitance of the fiber (determined by its width), and the relative position between them. As a result, we obtain several nonlinear effects emerging from the same physics, but in different time-scales and strengths, contributing to the complexity of the function f(). Hence, an analytic expression for f() cannot be derived for such a complex configuration. Instead, we analyze the function f() by stimulating the dendritic networks under different biasing conditions and AC voltages and evaluate the response spectra, which in turn allow us to model f().
We apply sinusoidal signals to the gate electrode (here a PEDOT:PF 6 covered Au electrode) as well as to the drain electrode. Source and drain are connected by a dendritic polymer network as shown in Figure 2b. In order to study the effect of the biasing conditions, we analyze the response of the OECT network for different gate-source and drain-source potentials as well as for different frequencies. To illustrate the effect of signal level and bias on the output of the dendritic network (V o , voltage measured across a load resistor), Figure 3a shows the distortion of a sine wave injected on the gate (amplitude A  = 0.5 V and frequency f = 1 Hz) for different drain-source bias voltages. It can be observed that for the negative bias, the transistor operates in its most linear region, showing low distortion. On the other hand, as the drain-source bias is increased and the OECT starts operating like a diode, the distortion increases significantly. In order to quantify this effect, the THD is utilized, which is the ratio of sum of the root-mean-square (RMS) voltage values of the higher harmonics over the RMS-voltage of the fundamental frequency. [25] A high THD value means that the power in the harmonics generated by the nonlinear effects is significant compared to the fundamental frequency. In Figure 3b, the spectra of the transients from Figure 3a are shown. As the drain-source voltage increases from −0.5 to 0.5 V and 0.75 V, the THD changes from −21.5 to −16.9 dBc and −14.3 dBc, respectively, proving the higher signal distortion at high positive drain-source potential. The appearance of multiple higher harmonics in the spectrum proves the nonlinear character of the function f(). In principle, based on the spectra shown in Figure 3b, the function f() can be derived, e.g., in the form of a polynome where the coefficients determine the magnitude of the higher harmonics in the spectrum. In Figure 3c, we show how the THD depends on the frequency of the signal applied to the gate. Up to a frequency of ≈20 Hz, the magnitude of the THD is only weakly reduced, however, at frequencies above 20 Hz, it drops drastically. This finding indicates that the effects contribution to the nonlinear response of the system have different strength and time constants. While the coupling strength strongly depends on the in-series capacitance of the gate and channel, the time constant of coupling will mainly depend on the distance between the gate electrode and the fiber carrying the measurement signal, which has been proven by impedance spectroscopy. Unfortunately, since all coupling scenarios rely on the same physical principle, we cannot conclude which coupling is dominant in the given dendritic network and it is reasonable to assume that this will depend on the specific shape of each network. Connecting the dendritic-like properties of the fibers to their steady-state and time-dependent responses is challenging. Nevertheless, standard OECT models [26] capture the main properties well. In particular, the relative capacitance between gated fiber and channel is crucial in obtaining high or low transconductance (high voltage drop across the excitatory fiber-electrolyte interface required for efficient gating).

Implementation of a Delayed Feedback and Proof of Complexity
In order to perform computations with a reservoir computing architecture, a high (or rather infinite) dimensional system is needed. [19] One way to prove this in continuous time systems is by showing that the time evolution of the network presents a chaotic motion. [18] If so, the function f() is sufficiently nonlinear. In the following, we show in theory and experiment that, with the help of a delayed feedback loop, chaotic behavior can be observed for the degree of nonlinearity as discussed in Section 2.1. Mathematically speaking, the connection of a delayed feedback loop to the dendritic polymer network corresponds to a nonlinear delay differential equation (DDE). In this framework, our system can be divided into three main hardware blocks, shown schematically in Figure 4a. First, an element capable of performing a nonlinear transformation of the input data u(t), being in this implementation an OECT network. The output is delayed with a delay block by a time T D , constant for all relevant frequencies in the system. A third block, the derivative node, follows, setting the system's response time τ and providing a mathematical tool to obtain a derivative of the output ( ) dy t dt . This type of structure provides three control parameters which can be used to induce chaotic dynamics, namely the forward gain λ, the system's response time τ and the delay time T D (cf. Equation (1). Using a third-order Taylor expansion of a single OECT's transfer characteristic, we show via simulations that the DDE with the OECT as the nonlinear function has enough nonlinear effects to exhibit chaotic dynamics within the feedback loop. This can be observed from the circuit's phase diagram in Figure 4b: a complex, bound, aperiodic evolution of the output presents itself, strongly suggesting the presence of chaos. While it is true that the model presented only considers the device's static properties, it presents already enough complexity to show chaos. As such, it is expected that the physical device (comprising of both static and dynamic nonlinearities) also does. The degree of this effect depends on the function f() (i.e., the device's transfer characteristic) and the D T τ ratio. Additionally, the control of the device's dynamics can be achieved by a modification of the transconductance gm. Changing this value, the complexity can be reduced to a single mode oscillation for a small decrease of gm, or to a chaotic solution for a larger one. This is due to the fact that the transconductance acts not only as an additional gain control, but also governs the strength of the nonlinear term. This effect can be more clearly observed in the bifurcation diagram depicted in Figure 4c, where the forward gain λ parametrizes all solutions at the output node. A deviation from the stable solution occurs for a particular gain value; the more nonlinear the device is, the earlier the onset of this first bifurcation occurs.
A system which follows the block diagram from Figure 4a was implemented as a fully analog electronic circuit (see Supporting Information for details). As the nonlinear function, the transfer characteristic of a single fiber from the network was used by injecting the input u(t) into one drain and feeding back its delayed output back to one other fiber acting as a gate. The number of virtual nodes from conventional reservoir computing is given by the ratio D T τ , [19] both adjustable in the circuit.
Furthermore, the gain is controlled by an amplifier stage giving a third degree of freedom to set the operation point of the system. We demonstrate the capability of our circuit to map the input into an infinite dimensional output space by means of its phase diagram (Figure 4d-f). Following the approach shown for the simulated system, we adjust the complexity of the system by the gain factor λ acting as a bifurcation parameter. For lower gains λ 1 , the circuit presents a single stable solution (i.e., lowest complexity). As the gain increases to λ 2 , the stable solution disappears and single mode oscillations appears until reaching a point (gain equal λ 3 ) in which the system falls into a nonlinear aperiodic evolution, strongly suggesting chaotic dynamics, and thus high complexity. The appearance of chaotic oscillations was supported by mathematical tools such as fractional dimensional analysis, while effects such as drift were discarded by repeated measurements over long intervals. It is worth noting that albeit both simulation and implementation show chaotic behavior, the details of the dynamics are quite different. The reason for this is that in the simulation, only one type of nonlinearity was exploited, namely the transfer characteristic. In the physical implementation, other effects such as geometry-dependent coupling between fibers, transient effects and frequency dependent nonlinearities play a significant role in the overall shape of the function f(). All these aforementioned dynamic effects further enhance the chaotic evolution, as the static model has been shown to already provide enough complexity for a chaotic behavior. The lack of a detailed analysis of such effects is not a limiting factor, since the reservoir's specific dynamics do not define the computational power of the network, but rather the complexity of the system does.

Classification Tasks Using Different States of the Reservoir
In order to highlight the importance of the complexity of the system for classification, we carry out a classification task for different dynamical states using the fully analog electronic circuit and a dendritic fiber network (Figure 1b).
The classification task is carried out using a 5-input-3output network featuring 3 excitatory fibers, and 2 inhibitory fibers. The dataset we selected for the classification task is taken from, [27] in which 5 attributes are used to predict students' performances on a test. The performance is divided into 4 classes: high, medium, low, very low. The dataset entries are preprocessed, by transforming them into electric signals, in order to make them suitable for the network. Cucchi et al. [16] showed that PEDOT:PF 6 reservoir networks work best with time-dependent signals due to the fiber's dynamic and frequency-dependent behavior (See Figure 3c). Therefore, we chose to linearly map the input data into a frequency range spanning from 1 to 50 Hz. Accordingly, we generated 5 input vectors for each dataset entry using sine waves of amplitude 1 V and the appropriate frequency encoding. The generated signals are injected to the input layer, they are nonlinearly transformed by the reservoir and 3 time-dependent signals are read out at the output layer. The output channel is a 3xNt vector (where Nt is the number of timesteps), which can still be used for classification through linear regression even though the dataset has 4 classes, as the dimensions are given by the reservoir's complexity rather than the number of outputs. Ideally, the output vector should feature more channels than output classes in order to have a better mapping than the input vector. However, the growth technique limits the number of inputs and outputs possible. Therefore, we show here only a proof-of-concept which might be optimized further. Completing the setup, the delay line time was chosen such that it is longer than the period of the slowest input signal, and the system's response time faster than the delay (here, T D = 100 ms, τ = 5 ms) in order to obtain a long-delay system. To perform the classification task, a subset of 200 entries containing 50 samples for each class was utilized. 160 of these (randomly selected) were labelled and used for the training set. We then used the remaining 40 for the test phase. In the framework of RC, the training phase consists in performing the linear regression on the training set with labelled data. The linear function is then applied to the test set, and the result is compared to the label using a winner-takes-all approach.
Performing the classification using the input vectors results in 60% accuracy rate representing the situation where the function f() is linear see confusion plot in Figure 5a. Here, the majority of the error seems to come from similar inputs mapping to class "high" and "medium".
Using the output vectors, the accuracy rate raises to 75% (Figure 5b). This experiment shows that the nonlinearities of the network are able to increase the separability of the classes by generating very different outputs for different input signals. Note that, while the networks perform the nonlinear transformation, the linear regression needed for class separation in RC has been carried out afterward on software with the recorded data.
Finally, to further improve the complexity of the system we carry out the classification task with the delayed feedback loop (see Supporting Information). To do so, one of the readouts is amplified and reinjected into the network (to a previously unused inhibitory fiber) with a delay of 150 ms. With this configuration, the accuracy rate further improved to 80% (Figure 5c), proving the usefulness of operating the networks with a delay line for increased systems complexity. In particular, the initial miss-classification between "high" and "medium" is now almost completely resolved. Figure 5. Confusion plot resulting from the classification task performing a linear regression a) using input vectors, b) using output vectors, and c) using output vectors with a delay line (150 ms). Each block of the confusion plot reports the number of elements used for the testing phase, and the accuracy rate of each class. The total accuracy rate is reported at the top. The accuracy rate significantly increases in b) and further improvement is achieved by working close to chaotic conditions. Dataset taken from [27] in which 5 attributes are used to predict students' performances during a test. The performance is divided into 4 classes: high, medium, low, very low.