Viewpoint: Pavlovian Materials—Functional Biomimetics Inspired by Classical Conditioning

Herein, it is discussed whether the complex biological concepts of (associative) learning can inspire responsive artificial materials. It is argued that classical conditioning, being one of the most elementary forms of learning, inspires algorithmic realizations in synthetic materials, to allow stimuli‐responsive materials that learn to respond to a new stimulus, to which they are originally insensitive. Two synthetic model systems coined as “Pavlovian materials” are described, whose stimuli‐responsiveness algorithmically mimics programmable associative learning, inspired by classical conditioning. The concepts minimally need a stimulus‐triggerable memory, in addition to two stimuli, i.e., the unconditioned and the originally neutral stimuli. Importantly, the concept differs conceptually from the classic stimuli‐responsive and shape‐memory materials, as, upon association, Pavlovian materials obtain a given response using a new stimulus (the originally neutral one); i.e., the system evolves to a new state. This also enables the functionality to be described by a logic diagram. Ample room for generalization to different stimuli and memory combinations is foreseen, and opportunities to develop future adaptive materials with ever‐more intelligent functions are expected.


Introduction
Stimuli-responsive materials refer to synthetic material systems whose properties can be modulated externally through, e.g., chemical, thermal, optical, electrical, mechanical, or magnetic stimuli. [1][2][3][4] The response can take various forms, e.g., changes in the macroscopic shape and optical, mechanical, or surface properties. [4][5][6] Characteristic examples of stimuliresponsive materials are thermoresponsive polymers such as hydrogels made of poly(N-isopropylacrylamide), which undergo tuning the chemistry, responsive materials can be designed to be multi-responsive and to possess even more versatile shapeshifting capabilities and multiple shape-memory effects. [33] However, in all of the above concepts the original state is recovered, i.e., stimuli-responsive and shape-memory materials do not evolve as a function of time; the response remains similar under identical stimuli (albeit potential degradation phenomena).
Compared to the above-described artificial stimuli-responsive systems, biological systems involve complex combinations of responses and adaptations under the environmental changes. This has attracted an increasing interest for scientists to pursue advanced biomimetic material systems. Still, so far, the mainstream research has dealt with mimicking the extraordinary equilibrium biological properties, such as the mechanical properties of silk, nacre, muscle, bone, or phenomena such as wetting, structural colors, and catalysis. [6,[34][35][36] However, biological systems are inherently dissipative, i.e., out-of-equilibrium ( Figure 1d), allowing, for instance, chemotaxis, adaptation, learning, reproduction, and evolution. [37] Therefore, the overarching question arises, to which extent could one mimic more subtle dissipative dynamic biological properties using artificial materials? Before going to the specific topic of this "Viewpoint", associative learning in the context of inanimate materials, we discuss some of the related characteristics of biological systems and current efforts to mimic them in artificial systems.

Relevant Biological and Biomimetic Concepts
Adaptation is ubiquitous in biological systems, relevant to the present topic. A specific complication is that the word "adaptation" has different denotations in different contexts in biology and technology. In general, adaptation calls for a feedback loop. Related to biological evolution, evolutionary adaptation to changed environmental conditions takes place during the reproduction of the organisms over successive generations, based on selection of the genetic attributes among a species, thus shifting the gene frequencies in the gene pool. [38,39] Together with mutation and reshuffling of the genes, evolution and speciation take place. Extensive efforts have been taken in the laboratory environment to mimic these complex processes. Different chemical species that are capable of reproducing themselves have been developed. [40][41][42][43][44][45] Some of them show competition between species that can be used to select a certain chemical replicator by giving environmental pressures. [44] Others can form dynamic structures if fueled by proper chemicals. [43] Another type of biological adaptation takes place during shorter time scales within a given organism, where adaptive growth or reorganization allows the development of modified functional properties under the pressure of environmental changes during the growth of the organism. [46] It is relevant to emphasize that this type of adaptation involves structural changes of the biological matter. A classic example deals with bones or muscles that grow stronger under increased loading. [46] In artificial materials, concepts for the adaptive growth/reorganization are still rarely discussed. An interesting concept was recently presented, inspired by the muscle training. [47] Therein a double hydrogel network was constructed, where mechanical tension leads to fractures of the constituting polymer chains and the creation of radicals, which further trigger polymerization of the monomers available in the aqueous mixture, ultimately leading to mechanical reinforcement of the material. Another example also deals with gels where structural reorganization takes place during the deformation, suggesting the processes should be denoted in both cases as adaptive mechanics. [48] We finally point out that, different from the biological/bioinspired viewpoint, in materials science and robotics there exists a wide range of literature where the notion of adaptation, in fact, could also be interpreted as an alternative for stimuli-responsivity. For further information, we refer to several in-depth reviews. [49][50][51][52][53][54] Living systems are out-of-equilibrium, as energy is dissipated to sustain the functions characteristic to living organisms. From self-oscillation of chemical networks and dynamic formation/ destruction of microfilaments to transient self-assembly of nanoparticles, different dissipative processes have been achieved in artificial systems. [55][56][57][58][59] Eventually, artificial systems Hao Zeng earned his Ph.D. degree in photonics in 2015 from the University of Florence. In 2016 he moved to Finland to work at Tampere University in the group of Prof. Arri Priimägi. In 2018, he obtained a three-year postdoctoral fellowship funded by the Academy of Finland. His research interests center around "Light Robotics"-small robots based on soft responsive materials that can be fueled by light. Recently, the focus has shifted more from robotic locomotion to materials, soft mechanics like self-oscillation, self-regulation, and now even learning, to mimic natural complexity and adaptivity in synthetic materials. must be designed to be dissipative to achieve advanced dynamic functionalities inspired by the biological systems.
Inside organisms, homeostasis is the key to maintain suitable conditions for out-of-equilibrium enzymatic activities. The factors to be maintained are, e.g., temperature, concentration of electrolytes, and chemical gradients. [37,60] Oscillation around a certain level is a common form of such homeostasis, allowed by utilizing negative feedback loops, where chemical reactions are coupled with different delays. [61,62] In artificial systems, autonomous homeostasis without the use of external control such as electronic circuits is still a grand challenge. One prominent example is the utilization of chemo-mechano-chemical loops in a responsive micropillar array surface, where an exothermic chemical reaction is coupled to the thermally induced deformation of hydrogel to create a negative feedback loop, yielding an oscillating temperature and reaction rate in the system. [63] One important feature of living organisms, partly overlapping with the concepts of responsive materials, is the ability to self-repair, regenerate, or heal wounds. [64] Therein, the system is able to restore its original state after being damaged to maintain its normal functions, such as mobility or insulation. In biology, this invariably needs energy dissipation. Though mechanistically different, significant progress has been made in artificial materials possessing the self-healing ability, ranging from soft to hard materials, and from dry to wet states. [65][66][67] In artificial materials, self-healing provides Adv. Mater. 2020, 32, 1906619 Figure 1. Schematic illustrations of the classic responses in stimuli-responsive and shape-memory materials, and related notions in artificial and biological systems. a-c) Schematic distinction between stimulus-responsive (a), shape-memory (b), and multiple shape-memory (c) materials. For simplicity, multi-responsive and two-way shape-memory materials are not illustrated. d) Relevant notions and interconnections between artificial materials and biological systems under external stimuli. e) Pavlovian materials acquire the given responsivity to an originally neutral stimulus upon associating two (the unconditioned and neutral) stimuli, following the classical conditioning algorithm.
the system with additional robustness, nevertheless, it does not per se offer the opportunity to adapt to a new environment or improve the original performances, and does not typically involve dissipative processes implicit in the biological systems.
Finally, related to the core of the present viewpoint, we address the concept of biological learning. Generally, it involves modification of behavior of an organism based on experiences, training, and rewards/punishment. [68] As such, it deals with a complex combination of concepts at different length scales and different disciplines, from molecular biology to experimental psychology. Even if learning is related to responsive behavior, it is still fundamentally different, as it generally involves, e.g., perception, memory, reflex and motor functions, "mind", thought, consciousness, and reward-seeking, several of which are connected characteristically to living organisms, and the underlying mechanisms are often poorly understood. The complex psychological behavior in living organisms is enabled by the biological nervous systems. This, in turn, also inspired the development of machine learning based on artificial neural networks, yielding computer programs capable of decision making and task execution by learning from past experiences. [69] Although some demonstrations of basic logic gating have also been achieved in smart materials, [70][71][72] they still hardly reach the level of encoding/computing met in biological or computer-based systems. Therefore, taken the complexity of biological learning, it becomes meaningful to search for the simplest forms of learning to provide inspiration for inanimate materials. Notably, Kandel took a reductionist approach when he investigated a simple sea slug (Aplysia) with only about 20 000 neurons, which allowed in-depth studies of the nervous system, function, memory, and learning mechanisms. [73] He concluded that classical conditioning, habituation, and sensitization are the simplest forms of learning. [73] We could pose an even more simplified question: can artificial materials be developed to algorithmically show these simplified forms of learning? From another perspective, could one program learning algorithmically by selecting a priori an alternative stimulus for a material to trigger the intrinsic response? By algorithmically we mean that if we consider the material system as a black box (as Pavlov originally did with dogs), is it possible that upon feeding suitable stimuli, the material response would follow the logic of biological responses, despite the different underlying signal processing pathways?

Pavlovian Materials
Along these lines, we will describe existing artificial systems employing associative learning algorithms and in particular two recent synthetic material systems that we coin as Pavlovian materials, since their responses are inspired by the classical conditioning algorithm, a simple form of associative learning (Figure 1e). Associative learning takes place based on the association of different stimuli, which may additionally involve reinforcements or punishments. [74] In classical conditioning, a new behavior is learned simply by pairing two distinctive stimuli. [75,76] Here the new behavior refers to the responsivity toward a new stimulus, not a new response. A typical example is given by dogs that learn to salivate upon ringing a bell (neutral stimulus) by pairing the bell with a natural unconditioned stimulus, i.e., showing food. [76] The actual biological process is subtle, of course, involving interplay of different sensory and neuronal channels, forgetting, and recovery, thus combining perceptions of complementary stimuli. Related to artificial systems, many devices have been realized, inspired by neuronal, synaptic, and psychological behaviors. [77][78][79][80][81][82] In particular, classical conditioning has been emulated in machine learning, in electric circuits, using biochemical reaction networks, and in synthetic biology. [81,[83][84][85][86] In the field of machine learning, the classical conditioning process has been simulated by computational models in great extent. [87,88] Different models have been proposed, each achieving some aspects of the classical conditioning such as acquisition, extinction, blocking, and conditioned inhibition. [87] Inspired by the biological brain, neural networks have been developed, [89] where the network can acquire a learned response not originally programmed in the system upon repeated training (conditioning) procedures.
Furthermore, memristors provide another possibility of mimicking the classical conditioning process, a good example being a hybrid structure composed of aluminum-oxide nanoparticle array and polyimide, where periodic positive and negative voltages lead to dissolution or redeposition of silver filaments within the vertical aluminum-oxide/polyimide nanochannels. [81] The modulation of the silver filaments controls the electrical conduction of the memristor, and classical conditioning, including acquisition, extinction, recovery, and generalization, could be efficiently mimicked based on the association of pulses of positive (corresponding to food) and negative (corresponding to bell) voltages. It should be noted that the responses (current) here are not exactly the same before and after conditioning due to the change of the sign of the current.
Biochemical networks have also been employed to mimic the classical conditioning processes. [85,86] An associative memory system was demonstrated based on enzymatic cascade, [85] where the "correct" input (corresponding to food) consisting of maltose phosphorylase and glucose dehydrogenase results in the production of nicotinamide adenine dinucleotide (NADH) due to the presence of maltose inside the system. The "wrong" input (corresponding to bell) consisting of hexokinase and glucose-6-phosphate dehydrogenase does not lead to production of NADH, as no glucose is present in the original system. By applying both inputs simultaneously, glucose-6-phosphate is produced that serves as the memory, which can be later used to produce NADH upon feeding the "wrong" input after conditioning. Importantly, the "wrong" input consists of two components: the hexokinase is responsible for switching on the memory during conditioning, while glucose-6-phosphate dehydrogenase is responsible for retrieving the memory (production of NADH). In another report, [86] a simple biochemical model is proposed to mimic the associative learning process, potentially applicable for DNA, RNA, and polypeptides. The mechanism is mainly based on the replication and reversible polymerization of two species triggered by two different stimuli. This model demonstrates the theoretical possibility of associative learning within living cells.
Synthetic biology provides ample examples of programming on the cellular level, such as logic gates or computation. [90,91] It is thus not surprising that the conditioning algorithm can also be implemented in cells, as demonstrated in Escherichia coli. [84] In the report, two chemical triggers, salicylate and arabinose, were used as the food and bell, respectively. [84] When both chemicals are present, a genetic toggle switch (memory) can be switched on, which allows the production of green fluorescent protein (salivation) upon feeding the arabinose alone.
Instead of emulating psychological processes, synaptic processes have been mimicked using electric devices and neuromorphic devices. [78,79,82,92] Therein potentiation or depression can be achieved based on the tuning of spiking. In view of the above-mentioned literature on classical conditioning in artificial systems, we next review the newly identified concept of learning in synthetic materials by programming the material response with classical conditioning algorithm. [93,94]

Pavlovian Hydrogels
We have recently developed a model system consisting of hydrogel networks and gold nanoparticles that follows the classical conditioning algorithm. [93] The gel intrinsically melts upon heating above the gel-sol transition temperature (T m , 33 °C) and initially does not melt upon the used laser irradiation (a selected combination of blue and red light). The irradiation is therefore a neutral stimulus, and heating the unconditioned stimulus. Through a conditioning process, where both heating and irradiation are simultaneously applied, gold nanoparticles migrate inside the melted sol and assemble into linear chains, yielding an enhanced absorbance at the red part of the spectrum. The gel thus acquires the ability to respond to irradiation by melting after conditioning; see Figure 2a for the logic diagram. Importantly, neither heating nor the selected irradiation alone triggers the acquisition of the responsivity to irradiation. The process thus mimics the dog's salivation response to the ringing bell in Pavlov's seminal experiments. Intriguingly, other characteristic aspects in the classical conditioning framework can be realized, such as timing dependence of the stimuli during the conditioning process inspired by forward and backward conditioning, forgetting, and spontaneous recovery of the memory under out-of-equilibrium conditions (Figure 2b,c).
Though the Pavlovian hydrogel algorithmically shows apparent behaviors following the classical conditioning experiments, there are important mechanistic differences to be distinguished from the biological systems. First, the two stimuli (heating and irradiation) are not fully orthogonal, since the irradiation produces photothermal heating that could trigger also the melting of the gel before conditioning, if the intensity of irradiation is high enough. This calls for optimization in the present case. In contrast, the food and bell as stimuli in the case of dog are fully orthogonal as they undergo different pathways (visual vs auditory senses). Second, the irradiation used in the gel contains two components: 455 nm is mainly responsible for activation of the memory, while the 635 nm is used to produce the enhanced photothermal effect after the memory has been activated. In the case of the bell, there is no differentiation in the components of the sound. Considering these mechanistic differences, we wish that the Pavlovian hydrogel would lead to further investigations into artificial materials that would pursue full orthogonality and more complex learning functionalities.

Pavlovian Actuators
As another example, we implemented the classical conditioning algorithm in mechanically deformable liquid crystal networks (LCNs), by introducing a different kind of built-in triggerable memory. [94] LCNs are synthetic materials combining the anisotropy of liquid crystals and the elasticity of crosslinked polymer networks. [95,96] Intrinsically, all LCNs respond to heat due to temperature-induced changes in molecular alignment, as schematically shown in Figure 3a(i) and (ii) using order-disorder transition and the resultant shape change in LCNs as an example. The present LCN actuator comprises light-absorbing dyes initially dispersed randomly on one surface of the sample and treats irradiation as a neutral stimulus because of low absorption in the bulk (Figure 3a(iii)). During conditioning, heat and irradiation together boost up the sample temperature to allow dye diffusion into the bulk from the surface, resulting in enhanced optical absorption (Figure 3a(iv) and (v)). As a result of the conditioning, irradiation becomes the conditioned stimulus, rendering the LCN light responsive (Figure 3a(vi) and (vii)). This process allows an intrinsically thermoresponsive material to learn to respond to light, enabling the conversion from heat-fueled to light-fueled soft robotic devices. The diffusion process of the dye to fill the bulk volume from one surface constitutes the triggerable memory. By using splay-aligned Adv. Mater. 2020, 32, 1906619   Figure 3. Pavlovian actuator. a) Schematic illustration of classical conditioning in planar-aligned, nematic LCN using the order-disorder transition as an example. The unconditioned stimulus is heating beyond the isotropic transition. The originally neutral stimulus is light irradiation, which will be the conditioned stimulus after associating light with heat. b) The unconditioned response (upon heating) of the splay-aligned LCN before conditioning (bending) is identical to the conditioned response (upon irradiation) to the originally neutral stimulus due to the diffusion of the dyes to the bulk of the matter. c-e) Soft robotic demonstrations: a small-sized walker insensitive to light becomes photomechanically deformable after conditioning (c); a gripper learns to grab a light-scattering object with specific color (d); and an artificial Pavlov's dog showing jaw opening and salivation upon irradiation after conditioning (e). b-e) Adapted with permission. [94] Copyright 2019, Cell Press.
LCNs, bending motion (Figure 3b), which was used in our softrobotic demonstrations (Figure 3c,d), can be easily obtained by asymmetric thermal expansion of the actuator. Figure 3c shows a light-fueled walker that starts to translocate upon modulated irradiation only after conditioning. Furthermore, a small-sized gripper can be conditioned to grab a scattering object with specific color (Figure 3d), representing a simple case of tuning the neutral stimulus. Combining the actuator and the hydrogel, an artificial Pavlov's dog that shows conditioned salivation can be fabricated, where the melting hydrogel serves as the reservoir of saliva and the mechanical LCN as the dog's jaws (Figure 3e). Also in this case, heating corresponds to the food for the artificial dog and the light irradiation to the bell.
Compared to the programmable gel, the irradiation contains only one wavelength component for the LCN actuation, providing in that sense a more direct association between unconditioned (heat) and neutral stimuli (light). Although diverse dye options can be adopted to train the actuator to respond to different wavelengths, once conditioned, it cannot easily "forget" since the diffusion process is irreversible. However, by treating the conditioned LCN with an organic solvent, the dye can be extracted while the film remains intact, thus providing the possibility to extinguish the conditioned memory. Compared to biological systems, the above-demonstrated material still presents an extremely simplistic model. Indeed, again the two stimuli lack perfect orthogonality as light actuation is based on photothermal heating, which triggers the intrinsic thermal response of the material. This, together with the temperaturedependent diffusion process (triggering the memory), dictates that all the experimental parameters (intensity, temperature, and applied periods) must be carefully tuned in order to mimic the conditioning algorithm. In contrast, the stimuli in the original Pavlovian conditioning are intrinsically disconnected as discussed above, and the physical limits (e.g., stimuli strength and training time) exist over a much more flexible range.

Discussion and Perspective
To achieve associative learning in artificial materials, a memory and mechanisms (logic gates) to trigger and retrieve the memory are indispensable. [93,94] The behavior of the Pavlovian materials can in some sense be considered as "switching" of the material property, since it requires the memory to be switched on by the simultaneous exposure to two stimuli (in the present case, light + heat). Nevertheless, it is also a type of "prescribed" training, where the new responsivity to light, which is preselected as the neutral stimulus, is acquired by the association process (training). In principle, we can foresee a lot of opportunities for different material systems to be implemented with the classical conditioning algorithm, as long as the abovementioned criteria (Figure 2a) are met. For instance, various logic gates have been achieved in soft materials (e.g., hydrogels) to respond to different chemical cues, [70][71][72] and they can potentially be utilized to construct the conditioning algorithm and even more complex functionalities. Besides, DNA-or RNAbased systems have been utilized to synthesize macroscopic responsive materials, [97] which may serve as potential platforms for the Pavlovian conditioning, as proposed by Gandhi et al. [86] Introducing the memory elements for classical conditioning is, however, not trivial and should conceptually be distinguished from the memory in several existing materials. In shapememory materials, even if they also possess a material memory and the responsivity to several stimuli, the last stimulus always recovers the original state, and restores the original responsivity. Therefore the system does not evolve to respond to a new stimulus. As discussed earlier, the memory in shape-memory materials is essentially the permanent shape that is fixed by the fabrication process, which in principle does not change post synthetically. Besides, there is also no direct association between different stimuli for programmable shape-memory materials. Some artificial materials also possess memories based on hysteresis in the response, such as in ferromagnetic materials or soft materials such as hydrogels. [98,99] This kind of memory is essentially the bistability of the system and thus different from the conditioned memory in the "Pavlovian materials". To be emphasized is that the memory in classical conditioning should be only activated by AND gate (unconditioned and neutral stimulus) and, equally importantly, can be retrieved by the neutral stimulus alone to produce the response.
To mimic natural behaviors more accurately, the nonorthogonality of the stimuli in artificial materials should also be addressed. In biological systems, the classical conditioning helps an organism to prepare for a meaningful event that is coupled with an otherwise irrelevant (neutral) stimulus. Not surprisingly, classical conditioning exists not only in animals that possess central nervous systems, but also in plants, showing its ubiquity in the biological world. [100] It is thus vital for the organism to be able to associate a neutral stimulus with a totally different (orthogonal) stimulus. Admittedly, the abovedemonstrated artificial materials are very limited in the choice of neutral stimulus compared to biological systems and the neutral stimulus is not fully orthogonal to the unconditioned stimulus. We envision that fully orthogonal stimuli can be identified in the future, and that addressing the orthogonality of the stimuli and expansion of the range of neutral stimuli will significantly enhance the applicability of the concept, and therefore deserves future attention.
The intriguing conclusions here are twofold. First, already long we have been claimed to have "smart" or "intelligent" materials. Even if they have proved to be relevant in a wealth of applications, they are, however, typically responsive to the external environmental conditions. In contrast, our recent findings discussed here suggest that synthetic materials could show some simple learning behavior. It is vital that their behavior can be explained by the incorporation of a triggerable memory, in combination with response to (at least) two stimuli. The long history of polymer physics and biophysics already shows a wealth of different stable or metastable states that could be used as "memory" states. Therefore, one of the recommendations of the present article is to reconsider previous literature from a new perspective. We think that there are several combinations of potential permanent and volatile memory concepts allowing mimicry of Pavlovian behavior. The second point deals with the conditioning to achieve technologically useful properties after the present conceptual studies. In other words, having now established that classical conditioning can in principle be implemented in common materials systems "like a black box", it becomes relevant to ask how to turn it to be useful mechanism for functional materials. As we postulated above, so far the concept bears resemblance with supervised learning, since we specify a priori the stimuli to be used. This contrasts with the conditioning process in biological systems or in machine learning, where the neutral stimulus is not determined a priori. Toward applications, one could foresee that robots can be constructed using the Pavlovian materials, where repeated stimuli from their surrounding environment can induce a change in the property of its constituting material, which further promotes adaptation to environment or better performance in robotic functions. Besides, materials responding to different orthogonal stimuli and alternatively permanent and volatile memory concepts should be studied. Such concepts, inspired by the logic of associative learning, are being explored in our groups. We foresee that, ultimately, we could really enter the realm of smart materials.