Artificial Perception Built on Memristive System: Visual, Auditory, and Tactile Sensations

The widespread implementation and rapid development of autonomous systems pose stringent performance requirements on emerging sensory systems. In addition to the basic sensing requirements, leading sensory systems are required to process data and extract featured information from highly redundant data in real time. With the added edge‐computational capabilities, data shuttling is avoided, leading to significant reduction of computational burden and bandwidth pressure in the cloud. Among the different computing architectures, the neuromorphic sensory system stands out due to its high power efficiency, low latency, and excellent processing capability. Mimicking the biological neural network, the colocation of sensory, processor, and memory components of neuromorphic sensory systems enables the requirements for frequent data shuttles to be circumvented. In particular, artificial intelligent perceptions built on memristive neuromorphic systems exhibit outstanding characteristics of small footprint, low power consumption, 3D stacking ability, and high density. Herein, the two essential parts of the memristive artificial perceptron system are presented: 1) memristive systems for neuromorphic computing and 2) high‐performance sensors. Next, the current state of the art established on artificial perceptron systems covering visual, auditory, and tactile sensations is highlighted. To conclude, the current challenges and future direction in the area of advanced intelligent perceptions are presented.

computational approach to process real-time data with high throughputs. [6] In this way, the information or signals from sensors are processed in situ to significantly reduce the computational burden and bandwidth pressure. This computational approach is also well-known as edge computing or near sensor computing.
Great strides in the field of neuromorphic engineering have been made in the last decade by developing the platform technology necessary for constructing distributed multichip systems. Presently, one can build neuromorphic systems that integrate multiple sensors with artificial or spiking neural network chips. In these systems, each chip module specializes in a specific neural function, including visual sensing, auditory sensing, filtering, and spike-based learning. However, most of these existing commercialized neuromorphic chips use a digital or analog circuit to emulate neurons and use static random-access memory (SRAM) to emulate the synapse, which contain a number of active components to achieve the functionalities, and naturally have a large footprint. Moreover, as SRAM is volatile, it still needs to load data from off-chip memory before it is able to execute computing functions, leading to consumption of most of the system energy. Therefore, memristive neuromorphic sensors provide an attractive alternative to CMOS-based neuromorphic sensory systems. Memristive neuromorphic sensory systems use memristors as the basic building blocks, which stand out in terms of fast accessing speed, ultralow power consumption, excellent scalability, high endurance, and high density. In addition, memristors with analog switching behaviors can better mimic the typical biological synapse and neuron in both structure and switching dynamics.
In this article, we provide a review of the current state of the art in memristive neuromorphic implementation of visual, auditory, and tactile sensation and identify key contributions across these areas. An overview of the performance and progress of neuromorphic computing systems is first introduced, followed by a Mei Chee Tan is currently an associate professor at the Singapore University of Technology and Design. She graduated with a bachelor's degree in chemical engineering and earned her master's and doctorate degrees with the Singapore-MIT Alliance at the National University of Singapore. Later, she continued to expand her research experience as a postdoctoral researcher at Rutgers, The State University of New Jersey. She has extensive experience in synthesis, modification, and characterization of optical nanomaterials with controlled sizes and morphologies for biomedical applications and energy-efficient devices. Her future research activities focus on the study and engineering of tailored interfaces.
survey of the cutting-edge materials and device architectures at the frontier of high-performance sensor systems. Subsequently, a summary of the integration of these sensor-and memristorbased computation systems leading to the design and implementation of artificial perception (i.e., visual, auditory, and tactile) is highlighted. Finally, a perspective of the current challenges and future direction in the development of the neuromorphic sensory field is presented.

Current State of Memristive Systems for Neuromorphic Computing
Driven by Moore's law, modern computing systems have evolved to be increasingly more compact and powerful. The increase in computational performance of the rank first supercomputer (see Figure 2a) has not been able to keep pace with the rapid exponential increase in power consumption demands. The huge demand for power consumption arises from the frequent data shuttling between the processer and memory in the Von Neumann architecture, which is also frequently referred to as the Von Neumann bottleneck. Likewise, the improvements in processor speeds are unable to match the requisite demand for increased memory accessing speeds. The large performance gap between the processor and memory leads to huge latency and limits the data bandwidth, which is also known as the memory wall issue (Figure 2b). Together, the Von Neumann bottleneck and memory wall issue severely restrict the advancement of modern computing systems.
In recent years, neuromorphic computing has emerged as a preferred alternative to the conventional Von Neumann architecture. Inspired by the biological brain's function, neuromorphic computing does not differentiate between processor and memory, and computation is executed at where the data are located (Figure 2c). In this way, it successfully circumvents the Von Neumann bottleneck and breaks the memory wall limit. By combining the strengths of modern VLSI technology and mimicking the principle of the biological brain, neuromorphic machines are potentially more powerful in managing highdimensionality and unstructured data while operating with much lower power consumption. Currently, commercial neuromorphic systems are mostly built on complementary CMOS circuits, including TrueNorth by IBM, [12] SpiNNaker by the University of Manchester, [13] Neurogrid by Stanford University, [14] Loihi by Intel, [15] and Tianjic by Tsinghua University. [11,16] Compared to conventional processors based on Von Neumann architecture, these neuromorphic chips exhibit an exceptional information processing capability at much lower power consumption levels, especially for unstructured data (Figure 2d). However, since CMOS transistors are not created or optimized for the purposes of neuromorphic computing, existing CMOS-based neuromorphic chips do not completely mimic the biological computational elements (synapse and neuron), leading to the absence of intrinsic hardware learning capability. Consequently, these silicon synapses and neurons require complex circuits that are difficult to scale down or stack in 3D in the back end of line. In addition, bulky memory and frequent memory visits limit the energy and area efficiency of these systems. To overcome these technical challenges, the emerging memristor-based devices with superior computing ability and excellent scalability provide a strategic opportunity to advance neuromorphic system construction.
Although the concept of the memristor was introduced in 1971, [23] the functional memristor device was only experimentally discovered in 2008. [24] As its name indicates, memristors "remember" the operation history and exhibit controllable analogue conductance change. This unique switching behavior Figure 2. Overview of performance trends of neuromorphic computing systems. a) Performance and power consumption of rank first supercomputers. [9,10] b) Memory gap between processor and memory of modern computing systems. c) Schematic representation of the design philosophy of a representative neuromorphic chip. Reproduced with permission. [11] Copyright 2019, Springer Nature. d) CMOS-based neuromorphic chips and their key parameters. [11][12][13][14][15][16] e) The development of memristive neuromorphic chips shows an exponential scaling-up trend. [17][18][19][20][21][22] www.advancedsciencenews.com www.advintellsyst.com positions memristors as promising fundamental building blocks of leading neuromorphic computing systems. At the device level, memristors stand out as frontrunners in terms of fast accessing speed (<85 ps in Figure 3a), [25] ultralow power consumption (<100 fJ event À1 in Figure 3b), [26] excellent scalability (linewidth <2 nm), high density (4.5 Tbit in À2 in Figure 3c), [27] and high endurance (>10 12 in Figure 3d). [28] At the array level, the continuous analogue conductance tunability (Figure 3e) enables singlestep vector-matrix multiplication operation, which is the basis for designing a formidable calculation function for linear and differential equations. [30,31] In addition to their excellent switching behavior, memristors also resemble typical biological synapses in both structure and kinetic process. [32] Therefore, using memristors to emulate an artificial synapse, various synaptic functions, including long-term plasticity, short-term plasticity, spike-timing-dependent plasticity, and spike-rating-dependent plasticity, have been established. [33,34] By assimilating memristors in networks, complex computational functions, such as image classification, [17] image multilayer perceptron, [18] sparse coding, [19] reservoir computing, [20] deep neural networks, [21] and face classification, [22] have been demonstrated. These demonstrations have proved that most of the prevailing machine learning and deep learning algorithms, including feed forward and feedback propagation, [35] can be implemented using a memristive synapse network. With the array size increasing over time, the computing power of memristive systems has also increased rapidly (Figure 2e). In 2015, the most advanced array size of 12 Â 12 could only perform a 3 Â 3 white-and-black pixel image classification. [17] By 2019, the most advanced memristive array contains 204 900 synapses, which allows a throughput per unit area of 3.6 trillion operations per second per square millimeter (3.6 TOP s À1 mm À2 ). [21] In recent years, the memristor family has been greatly expanded with additional control factors, including diffusive memristors, [36] second-order memristors, [37] three-terminal memristors, [38][39][40][41] and gap-type atomic switches. [42,43] With the retention times of different conductance states ranging from several microseconds, tens of days, and up to several years, these memristors display enriched switching dynamics, which has greatly expanded the range of possible application scenarios. The recent advances of biomimicry devices Figure 3. State-of-the-art performance of memristors. a) Ultrafast switching of an AlN memristor. The memristor can be switched both off and on in %85 ps. Reproduced with permission. [25] Copyright 2016, John Wiley and Sons. b) A NbO x -based memristor showing a heating transience of 700 ps and a cooling transience of 2.3 nm, demonstrating a <100 fJ power consumption. Reproduced with permission. [26] Copyright 2012, IOPscience. c) Crosspoint structure with linewidth <2 nm and density >4.5 Tbit in À2 . Reproduced with permission. [27] Copyright 2019, Springer Nature. d) Endurance performance of TaO x memristors under 2.1%, 2.7%, and 3.0% oxygen partial-pressure conditions; >10 11 endurance was demonstrated based on TaO x with 2.1% oxygen partial pressure. Reproduced with permission. [28] Copyright 2011, Springer Nature. e) I-V curves with different conductances, showing good linearity over the selected conductance range. More than 64 levels of conductance or 6 bits of digital precision over the conductance range 100-900 μS. Reproduced with permission. [29] Copyright 2018, Springer Nature. with sophisticated behaviors (e.g., artificial neuron, short-term to long-term memory) reinforce the eventual possibility of building a full memristive system using minimal CMOS peripheral circuits.
Modern sensing systems have also benefitted from this rapid development of memristor devices and memristive systems. Directly interfacing the physical computing ability of a memristor with the analogue signals from sensors allows analogue computing without the use of analogue-to-digital or digitalto-analogue (ADC/DAC) conversions, which typically consume most of the energy in a mixed-signal neural network. Another important consequence of memristive switching dynamics is that memristor also serve as filters enabled by the spikingrate-dependent plasticity behavior, as shown in Figure 4a. [36] Memristors with a low initial conductance state function as high-pass filters, whereas memristors with a high conductance state act as low-pass filters. [45] Memristors with intermediate conductance could act as bandpass filters that are most effective at transmitting impulses with a specific frequency. [45] These unique features imply that memristors have an advantageous intrinsic information processing ability.
In conventional sensory systems, the sensors are always separated or independent from the information processing units, resulting in additional signal transmission time and energy, which restricts real-time applications. With a relatively simple device structure and CMOS compatibility, a memristor could be integrated with sensors to form compact and efficient memristive sensory systems. Memristors process and encode the signal from a sensor locally, significantly reducing the computing burden of central processors. For example, in Figure 4b, the memristor device was used as a memristive-integrated sensor and directly compressed information on neural spikes from retinal ganglion cells. [44] In Figure 4c,d, the memristor array was directly connected to the analogue output of a sensor or other edge device and configured as a discrete cosine transformation operator to provide a spectral analysis of a signal without needing to first digitize it. [29] Moreover, although a memristive system provide a totally new computing architecture, circuits for logic operation are still indispensable parts for sensing and data processing in artificial perception systems. A positive development is that logic circuits can also be constructed with memristors. Despite the hybrid memristor/CMOS approach, . Reproduced with permission. [36] Copyright 2017, Springer Nature. b) An arbitrary input waveform consisting of four concatenated copies of the same retinal cell recording and artificially inverted to produce spike trains with alternating polarities. This waveform was employed to validate the concept of memristive integrated sensors. Reproduced with permission. [44] Copyright 2016, Springer Nature. c) Differential discrete cosine transformation written into a 128 Â 64 memristor array, which provides a Fourier-related transform widely used in digital signal processing and image/video compression and processing. d) Experimental realization of a memristor crossbarbased spectrum analyzer for high-density, low-footprint computation. Reproduced with permission. [29] Copyright 2018, Springer Nature. all memristor-based logic and memristive-network-based logic computations have been developed. [46] With these novel design paradigms, all of the essential logic gates in digital systems can be realized based on memristors with faster speed, lower energy, and smaller form factor. These compelling unique features make memristors a natural choice as building blocks for artificial perceptron systems with a broad spectrum of applications such as computer vision, speech recognition, and artificial skin.

Materials and Devices for High-Performance Sensing
Memristive systems with intrinsic low power and learning ability successfully address the speed and energy efficiency issues related to the computational aspects. However, high-performance sensor arrays with small device form factors and low energy dissipation are still required to collect information from the external environment and interface with the computational units. In this section, we present an overview of cutting-edge materials and device architectures at the frontier of high-performance sensor systems for visual, auditory, and tactile sensations.

Sensitive and Responsive Photodetectors for Visual Sensing
Vision is the ability to detect light patterns from the external environment and subsequently interpret, translate, and convert these signals into images. In the biological system, the remarkable ability of visual perception that led to the perception and identification of color, shape, and motion as well as the differentiation between light intensities is enabled by a complex visual system. A brief description of the workings in the visual system is as follows: 1) external light is refracted onto a small image that is incident upon the retina. 2) Phototransduction occurs next, where the captured image is converted into electrical pulses via the rods and cones of the retina. To emulate this biological visual perception, highly sensitive and responsive visual sensors or photodetectors that detect light information are required. The photodetector detects the characteristics (e.g., wavelength and intensity) of electromagnetic waves through the photoelectric effect, where semiconductor materials absorb incident photons to generate electron-hole pairs which are separated to generate photocurrent. The key performance feature of the photodetector is its photosensitivity, which characterizes its capability to distinguish signal from noise. The high sensitivity is needed for visual sensors to operate at low levels of illumination. Additional performance descriptors to evaluate photodetectors include responsivity, external quantum efficiency (EQE), detectivity (D*) or noise-equivalent power (NEP) and linear dynamic range (LDR).
Light detection has been traditionally facilitated by inorganic semiconductor-based photodetectors (e.g., Si, Ge, GaN, and InGaAs) fabricated via matured semiconductor-manufacturing processes. The detection wavelength of these photodetectors extends from the ultraviolet (UV) to the long-wavelength infrared (LWIR) regions. For instance, the typical detection range of commercially available photodetectors is from %400 to 1100 nm for Si, 870 nm to 1.7 μm for InGaAs, and 0.7 to 25 μm for HgCdTe. However, the cost of fabricating these inorganic semiconductor-based photodetectors limits the potential for creating the complex device architectures that could be required to improve detection performance. In addition, the rigidity of these photodetectors restricts their placement possibilities on curved objects. Furthermore, these inorganic semiconductor-based photodetectors are often conventional inorganic semiconductorbased infrared photodetectors and often require cryogenic cooling to reduce thermal dark current and allow efficient sensing.
Consequently, the recent drive is toward the development of photodetectors based on other emerging classes of optoelectronic materials, including organic semiconductors, nanocrystals (e.g., perovskite nanocrystals and semiconducting quantum dots), and nanocomposites (e.g., rare-earth-doped nanocomposites). In contrast to the inorganic semiconductor counterpart, these optoelectronic materials could be fabricated on mechanically flexible substrates as large-area flexible photodetectors. The solution processability of these emerging materials would also improve the cost effectiveness of fabricating photodetectors with complex architectures. Despite the weaker overall photodetection performance of these emerging optoelectronic materials in some cases, their existing performance is adequate for select applications (e.g., intelligent prothesis and humanoid robots), especially in consideration of the ability of these flexible photodetectors to be placed and mounted on curved surfaces. Therefore, we would highlight the key developments and achievements of large-area flexible photodetectors using solution-processable organic semiconductor materials and semiconducting colloidal nanocrystals on mechanically flexible substrates. Broadly, the narrative is presented as two subsections based on the dimensions of the photoactive component: 1) nanoparticle-and 2) film-based photodetectors.

Nanoparticle-Based Photodetectors
The major benefit of using photoresponsive nanoparticles arises from mostly from their solution processability, which allows convenient integration with an almost limitless variety of substrates including postprocessing atop other integrated circuits. These photoresponsive nanoparticles are typically mixed with other optoelectronic materials (e.g., organic semiconductors) to enhance light absorption as well as to extend the light detection range. To date, flexible photodetectors have been fabricated using a myriad of photoresponsive nanoparticles such as quantum dot semiconductors and rare-earth-doped nanoparticles (RENPs). [47,48] The bandgap of quantum dots dictates the absorption/ detection wavelength ranging from the visible to the infrared region, which is tuned by controlling the particle size. Subsequently, with the addition of these quantum dots, the detection ranges of existing photodetectors are extended. For instance, the efficient exciton generation and transfer of visible-light-responsive CdS quantum dots to the semiconducting matrix led to increased photoconductivity and improved photodetection performance. [49] In the last two decades, lead sulfide (PbS) nanocrystals have been extensively investigated to create flexible infrared photodetectors. [50] The unique properties of infrared light have led to various technological advancements in night vision imaging, nondestructive machine inspection, environment monitoring, and small animal imaging platforms with longitudinal tracking capabilities. The advent of autonomous vehicles, wearable electronics, and humanoid robots that harness invisible infrared sensors for object detection, remote sensing and control, and scene recognition has also accelerated the development of infrared sensors. Well-dispersed PbS nanocrystals are typically synthesized via a solution-based method with controlled sizes (distribution <5%) to engineer an effective bandgap that is optimal for a selected application (e.g., the size-dependent absorption peak can be tuned from 1000 nm for a nanocrystal size of 4.3 nm-to 1800 nm for a size of 8.4 nm). [51] PbS nanocrystals have been used to fabricate photodetectors with different architectures (e.g., photoconductors, phototransistors, and photodiodes) that exhibit exceptional performance. For example, Konstantatos et al. reported ultrasensitive solution-cast quantum dot photodetectors with detectivity of 1.8 Â 10 13 Jones at 1.3 μm at room temperature (Figure 5a,b). In addition, it was also demonstrated that the transport and trap states of PbS nanocrystals are separately controlled by engineering of ligands and the oxidation of nanoparticle surfaces. [52] The main challenges of quantum-dot-based photodetectors are size uniformity and chemical stability. The size uniformity of quantum dots is critical to both device performance and detection range, while the poor chemical stability affects device reliability.
An alternative emerging candidate for optoelectronic nanocrystals is the RENPs, which generally exhibit large Stokes and anti-Stokes shifts, narrow and multipeak emission profiles, long lifetimes, and excellent photostability. In addition, the RENPs also function as charge-trapping centers, which is required for exciton separation. The state of the electron and hole traps is governed by the generally unchanged 4f energy levels of trivalent rare-earth ions. [56][57][58] Together, the unique optical characteristics and charge-trapping properties position RENPs as unique photonic converters that enhance light detection and extend the detection range. For example, photoconductor devices using an optically active nanostructured composites of RENPs and organic semiconductors exhibited a high conversion efficiency of near infrared ray and short-wave infrared ray (SWIR) light to electrical signals (Figure 5c-e). [47,48] The high conversion efficiency was ascribed to the high up-conversion efficiency, charge-trapping ability, and solid loading of RENPs in the composite. Due to the insulative nature of RENPs, the loading of RENPs is limited to allow efficient transport and conduction of electron-hole pairs. The design of these specific-wavelength photodetectors as a result of the narrow absorption characteristics of RENPs reduces the need for optical filters to separate signal from noise. To broaden the response range, a mixture of RENPs designed to respond to different wavelengths would need to be introduced within the composite active layer.

Film-Based Photodetectors
The alternative class of film-based photodetectors is built upon emerging organic optoelectronic materials such as organic semiconducting polymers, metal-halide perovskites (MHPs), and 2D materials. These film-based photodetectors are often built on donor-acceptor (DA) conjugated polymers comprising alternating electron-rich (donor) and electron-poor (acceptor) moieties. The state-of-the-art film-based photodetectors leverage mostly on the bulk heterojunction (BHJ) diode architecture, where the charge separation occurs due the BHJ structure, leading to high EQE. The photoresponsive spectral bands of organic semiconductors are tuned from the ultraviolet to the near infrared by regulating their conjugation length. The BHJ is formed via blending two organic semiconducting polymers (a donor and an acceptor) with different chemical potential energies. Existing BHJ organic photodetectors exhibit a relatively wide spectral response (0.3-1.8 μm), detectivities (D*) % 10 12 Jones, and a linear dynamic range over 100 dB in the visible range (0.5-0.8 μm). [59] For example, Gong et al. reported a high-detectivity polymer photodetector with wide spectral response from 300 to 1450 nm using a narrow-bandgap semiconducting polymer blended with a fullerene derivative. These organic semiconductor polymer film-based photodetectors exhibited detectivities greater than 10 12 Jones (Figure 5f-h). Optimization of photoactive layer thickness and BHJ morphology is required to achieve high responsivity and detectivity. Nevertheless, progress in semiconducting polymers responding to wavelengths ≥1000 nm has been limited due to various reasons, including structural unit design, chemical reactivity, and yield. [60] The material and process incompatibility with CMOS processing technology due to its low thermal stability limits the integration of these organic semiconductor-based photodetectors with other neuromorphic devices that are generally fabricated by CMOS-based technology.
Metal-halide perovskites (MHPs) are another promising candidate with exceptional optoelectronic properties. The basic crystal unit cell of perovskites is the type ABX 3 , where B is a cation and X is an anion, forming an octahedron [BX6] 4À group. In organohalide lead perovskites, the cation A is a small organic molecule (e.g., CH 3 NH 3 þ ), B is a metal ion (e.g., Pb), and X is a halide combination. MHPs are typically synthesized through a solution-based method and are easily processed by spin coating or evaporation using relatively low deposition temperatures. Their energy bandgaps are tuned by changing the halide ratio. MHPs combine the advantages of inorganic and organic semiconductors since thin single photojunctions can be formed in simple planar or mesoporous scaffold-based architectures due to their low exciton-binding energy. [61] Although the use of MHPs for photodetectors was first reported in 2014, [62] the reported performance is comparable, if not better, than that of inorganic semiconductor-based (e.g., Si) photodetectors in terms of responsivity, detectivity, and gain. For example, Dou et al reported a large detectivity approaching 10 14 Jones, a linear dynamic range over 100 dB, and fast photoresponsiveness with 3 dB bandwidth up to 3 MHz for an MHP-based photodetector operating at room temperature (Figure 5i,j). [54] However, current MHP photodetectors generally have a limited detection range (300-800 nm), suffer from poor chemical stability, and incompatible chemistries with the metal electrodes.
2D semiconducting materials such as graphene (zero bandgap), black phosphorus (narrow bandgap), and transition metal dichalcogenides (TMDs, narrow bandgap) characterized by their atomic-level thickness in the vertical direction are widely used. Upon absorbing bandgap-energy photons, the quantum confinement increases the probability of generating free electron-hole pairs, leading to the high absorption efficiency of 2D materials. The absence of dangling bonds on the surfaces of 2D materials facilitates the stacking of different materials beyond the constraint of crystal lattice mismatching through van der Waals interactions. At present, a multitude of Figure 5. Summary of photodetectors fabricated using different photoactive materials. a) Spectra of responsivity and normalized detectivity D* and b) device structure of photodetector fabricated using PbS colloidal quantum dot nanocrystals. Reproduced with permission. [52] Copyright 2006, Springer Nature. c) Schematic of the planar photodetector architecture in which the rare earth doped-nanoparticle-semiconducting polymer composite rests on a flexible PE substrate. [47] d) CoreÀshell N-NP efficiently converted the incident SWIR light to higher-photon-energy emissions (i.e., 654, 808 nm) that are effectively absorbed by poly(2,5-bis(2-decyltetradecyl)-pyrrolo[3,4-c]pyrrole-1,4(2H,5H)-dione-3,6-diyl-alt-3 00 ,4 0 -difluoro-2,2 0 :5 0 ,2 00 :5 00 ,2‴-quaterthiophene-5,5 0 -diyl to generate electrical signals. [47] e) Calculated R and EQE at different bending configurations show an unchanged detection response under varying degrees of mechanical stress. Reproduced with permission. [47] Copyright 2019, ACS Publications. f ) Molecular structures of PDDTT and phenyl-C61-butyric acid methyl ester (PC60BM) and energy-level diagrams of PDDTT, PC60BM, polyvinylcarbazole, polystyrene-bis(diphenylamino)biphenyl-PFCB, C60, ITO, poly(3,4-ethylenedioxythiophene), and Al. [53] g) Detectivities of Si photodetector, InGaAs photodetector, and polymer photodetector versus wavelength. [53] The high detectivities (10 12 Jones) of the InGaAs photodetectors require cooling the devices to 4.2 K. The detectivities of the polymer photodetector were calculated at λ ¼ 500 nm (point A) and λ ¼ 800 nm (point B) biased at À100 mV; h) EQE versus wavelength and calculated detectivities at λ ¼ 500 nm and λ ¼ 800 nm at various biases for the polymer photodetectors. Reproduced with permission. [53] Copyright 2009, American Association for the Advancement of Science. i) Device structure of the hybrid perovskite photodetector and energy diagram of the perovskite photodetector under a slight reverse bias [54] ; j) EQE and detectivity of the hybrid perovskite photodetector at different wavelengths. Reproduced with permission. [54] Copyright 2014, Springer Nature. The detectivity of a single-crystalline silicon diode (purple triangle) is shown for comparison. k) 3D schematic view of the single-layer MoS 2 photodetector and the focused laser beam used to probe the device; [55] l) spatial map of the photocurrent recorded as a focused laser beam is rasterscanned over the surface of the photodetector using a nanopositioning stage, scale bar, 5 μm, [55] ; (m) photoresponsivity of a similar monolayer MoS 2 device as a function of illumination wavelength. Reproduced with permission. [55] Copyright 2013, Springer Nature. photodetectors fabricated with 2D materials with detection range from ultraviolet to the near infrared have been reported. [55,[63][64][65] These 2D material-based photodetectors offer a large range of physical flexibility, high photoresponsivity and detectivity (e.g., MoS 2 : up to %10 3 A W À1 , [66] %10 12 Jones [67] ), and an ultrabroad detection spectrum (e.g., graphene from visible to terahertz). [68] The zero bandgap and ultrahigh carrier mobilities of up to 200 000 cm 2 V À1 s À1 at low temperature of graphene enable its detection from the visible to terahertz range. The relatively narrow bandgaps (1-2.5 eV; i.e., 500-1200 nm) and moderate carrier mobility (i.e., order of 100 cm 2 V À1 s À1 at room temperature) and strongly bounded excitons of TMDs lead to high responsivities. [64,65] For example, a monolayer MoS 2 phototransistor of 400-680 nm has shown high responsivity of 880 A W À1 (Figure 5k-m). [55] The synergistic combination of graphene with TMDs harnesses the individual strengths, leading to the high photovoltaic photoresponsivity of photodetectors fabricated from graphene-MoS 2 -graphene [69] and WSe 2 -graphene-MoS 2 (p-g-n) heterostructures. [63] However, the sensitivity of layered materials to the environment and the limited methods for large-scale material deposition remain as challenges for their application as flexible, large-area photodetectors.

Ultrasensitive Mechanosensors for Auditory Sensing
In the biological auditory sensory system, the sound wave vibrates the eardrums with a specific frequency and amplitude. Next, the mechanical vibrations are transmitted via ossicles to the cochlear hair cells and converted into electrical signals. The interpretation of auditory signals relies on the ability to analyze their frequency composition. These mechanoreceptors used for auditory sensing need ultrahigh sensitivity over a broad vibration frequency detection range. The human cochlea operates in a frequency band over three orders from 20 to 20 000 Hz, covering a 120 dB dynamic range, and distinguishes acoustic tones that differ by <0.5% in frequency. The selectivity of frequencies depends in part on the mechanical properties of the basilar membrane in the cochlea (Figure 6a), which has a trapeziform geometry. At the base of the cochlea, the narrow and rigid membrane responds to high-frequency stimuli, while at the cochlear apex, the wide and flexible membrane responds to low-frequency stimuli. As a consequence, hair cells at the cochlear base are preferentially stimulated by high sound frequencies, whereas hair cells along the cochlear coil respond best to lower sound frequencies, leading to the frequency-selective property of the cochlear, as shown in Figure 6b. Subsequently, the first step toward creating an artificial auditory system is to develop mechanosensors with suitable materials and delicate structures. The three broad classes of materials that emulate the frequency and intensity response of the basilar membrane are piezoelectric, triboelectric, and electromagnetic materials.

Electromagnetic Auditory Sensors
Electromagnetic materials have long been used to make acoustic devices and are therefore also used as sensors for sound detection. In recent years, a series of novel magnetic nanomaterials have been developed to improve the sensitivity of acoustic devices. For example, Lee et al. synthesized magnetic cubeshaped nanoparticles (c-MNPs, Zn 0.4 Fe 2.6 O 4 ) to modify the hair cells of amphibian inner ear. [71] With the high magnetization and colloidal stability, the oscillations of modified hair cells are remotely fine-tuned with an external magnetic field. Since the magnetic nanoparticle responds to electromagnetic stimuli ranging from 100 to 10 000 Hz, it meets the frequency detection range requirement for an auditory mechanotransduction system (Figure 6c). This method could also be useful toward the study of complex nonlinear dynamics of the auditory system. [77,78] By implanting a Ag-nanoparticle-infused silicone coil antenna into 3D printed bionic ears, sound signals were successfully captured (Figure 6d). [72] The clever combination of engineered tissue and flexible electronics offers a novel way to generate "off-the-shelf" cyborg organs. However, since these electromagnetic materials do not respond to the sound signal directly, a sound-to-electromagnetic signal convertor is still needed.

Piezoelectric Auditory Sensors
The ability of piezoelectric materials to generate electric signals from mechanical stimuli positions them as a natural candidate for mechanosensory design in auditory sensor systems. In Figure 6e, a 40 μm polyvinylidene difluoride piezoelectric film functions as an artificial basilar membrane and hair cells. Implanted into a guinea pig cochlea, the piezoelectric sensor successfully converted sound (with frequencies of 6.6-19.8 kHz) to create auditory brain-stem responses. [73,79] However, the electrical output power was not sufficient to stimulate signal processing units due to the low piezoelectric charge constant of polymeric piezomaterials. To augment the electrical performance, inorganic piezoelectric materials with high piezoelectric charge constant were used to emulate the basilar membrane and hair cells. As shown in Figure 6f, researchers transferred a crystallized Pb[Zr x Ti 1Àx ]O 3 (PZT) thin film onto a flexible plastic substrate to create a highly efficient and flexible mechanosensor to detect sound. [74] This piezoelectric material was capable of converting sound energy at even nanometer-level deformation into electrical energy. At 1000 Hz stimulation, the oscillatory displacement of their inorganic auditory sensor generated 59.7 μV piezoelectric signals. Nevertheless, the rigidity of the inorganic material limited the detection range such that the frequency range was reduced to 100-1600 Hz.

Triboelectric Auditory Sensors
Triboelectric materials also convert sound signals to electric signals by utilizing two kinds of membrane materials with different electron affinities. The acoustic wave vibrates both membranes and generates electron transport between them. Researchers have used a 125 μm thick, rollable, paper-based polytetrafluoroethylene/copper triboelectric nanogenerator to detect sound vibration, as shown in Figure 6g. [75] It is capable of delivering a maximum power density of 121 mW m À2 and 968 W m À3 under a sound pressure of 117 dB SPL , therefore simultaneously serving as a self-powered sensor for sound recording. [75] Inspired from the structure of the human eardrum, the researchers from the same team developed a multilayered (polyethylene www.advancedsciencenews.com www.advintellsyst.com terephthalate/indium tin oxide/nylon/polytetrafluoroethylene) bionic membrane sensor (Figure 6h), which exhibits high sensitivity (51 mV Pa À1 , in a range of 2.5-1200 Pa) and wide working bandwidth (0.1-3.2 kHz). [76] The bionic membrane sensor has been successfully demonstrated as a power source for throatattached microphones that pick up and recover human throat voice even in extremely noisy, windy environments, or even in a voiceless environment, through vibrations.

High-Performance Pressure Sensors for Tactile Sensing
In the biological system, tactile sensing of the skin relies heavily on neuronal sensor elements called mechanoreceptors, which are embedded at different depths below the skin surface and respond to forces on different timescales (Figure 7a). [80] Forces applied to the skin surface cause the flow of electrical currents into the mechanoreceptors. Compared to the Reproduced with permission. [70] Copyright 2001, Sinauer Associates. c) Schematic diagram of the electromagnetic probe stimulating a single hair bundle with c-MNPs attached, and Fourier transform spectrum of bundle motions, entrained at 10 000 Hz. Reproduced with permission. [71] Copyright 2014, ACS Publications. d) Schematic representation of the radio signal reception of two complementary bionic ears, and comparison between transmitted and received audio signal of the right and left bionic ears. Reproduced with permission. [72] Copyright 2013, ACS Publications. e) Auditory brain-stem response recording schematic using a polyvinylidene difluoride piezoelectric device, and its voltage output in response to sound stimuli with different frequencies. Reproduced with permission. [73] Copyright 2011, National Academy of Sciences. f ) Structure and working mechanism of the PZT flexible inorganic piezoelectric acoustic nanosensor and its frequency selectivity property. Reproduced with permission. [74] Copyright 2014, John Wiley and Sons. g) Structural design of the ultrathin paper-based triboelectric nanogenerator and its demonstration as a self-powered microphone for sound recording. Reproduced with permission. [75] Copyright 2015, ACS Publications. h) Schematic illustration of the eardrum-inspired active sensor, and the real-time voltage output in response to the throat vibration during speaking with low and high frequency. Reproduced with permission. [76] Copyright 2015, John Wiley and Sons.
www.advancedsciencenews.com www.advintellsyst.com Figure 7. Biological and electronic tactile sensors. a) Anatomy of human skin and tactile sensory system. Reproduced with permission. [80] Copyright 2015, Springer Nature. b) Schematic description of the Schottky junction tactile sensor and relative forward current increase of the sensor measured at 5 V under different external pressures. Reproduced with permission. [81] Copyright 2018, John Wiley and Sons. c) Photograph of the active matrix (12 Â 12 pixels) integrated with a large-area (9 Â 9 cm 2 ) pressure-sensor nanofiber sheet, and pressure versus resistance curve of the nanofiber sensor (red) and thin-film sensor (black) sandwiched between Au electrodes. Reproduced with permission. [82] Copyright 2016, Springer Nature. d) Schematic of the microstructured PDMS pyramid array film sensor and the pressure-response curve for different types of microstructured PDMS films, showing higher sensitivity than the unstructured film. Reproduced with permission. [83] Copyright 2010, Springer Nature. e) Structure and mechanism of the self-powered PZT pressure sensor and the normalized output voltage as a function of pressure and the enlarged output peak representing the response time of the pressure sensor (inset). Reproduced with permission. [84] Copyright 2017, John Wiley and Sons. f ) The basic structure of the TEAS and the summarized relationship and linear fitting between the relative variations of voltage and pressure applied on the TEAS device. Reproduced with permission. [85] Copyright 2013, ACS Publications. g) The structure and mechanism of the HPTS, and the electrical performance of the HPTS with different connection modes. Reproduced with permission. [86] Copyright 2019, Elsevier.
www.advancedsciencenews.com www.advintellsyst.com frequency-selectivity criterion of acoustic sensors, the development of tactile sensor systems focuses on the optimization of pressure sensitivity, response time, and stability. With the advent of the electronic skin, tactile sensors have been actively studied over the past 20 years. [87] Numerous pressure sensors for tactile sensing with different detection mechanisms have been developed, including resistive, capacitive, piezoelectric, triboelectric, electromagnetic, and optical mechanisms. The latest progress of resistive, capacitive, piezoelectric, triboelectric tactile sensors is presented here, since their development has also been the most promising in recent years.

Resistive Pressure Sensors
Resistive pressure sensors for tactile sensing that transduce a change in the resistance of a varistor device array into a measurement of pressure have been extensively explored due to their simple structure and working mechanism. Although conventional Si-based devices show pressure-sensitive property and exhibit high spatial resolution, their rigidity and brittleness restricted their application for tactile sensing, which generally requires extreme mechanical deformation. New material systems are needed to accommodate high-spatial-resolution tactile sensing while maintaining the same electrical performance. Flexible materials such as carbon nanotubes and graphene have been adopted to address the shortcomings of Si-based devices. [88,89] Recently, a pressure sensor matrix (100 pixels cm À2 ) made of position-registered elastic conductive microparticles was demonstrated with high sensitivity, good mechanical property, and high spatial resolution. [81] As shown in Figure 7b, the microparticles form a Schottky junction with the bottom electrode, where the contact area is changeable under pressure. It is worth mentioning that the formation of the Schottky junction successfully eliminates the electrical crosstalk and leakage current in the large sensor array, making it more practical compared to earlier reported structures. However, when using the pressure sensor to detect soft surfaces, the contact will inevitably cause mechanical deformation (e.g., bending). This means the normal pressure can hardly be measured independently from the mechanical stress. To address this issue, researchers from the University of Tokyo used composite nanofibers of carbon nanotubes and graphene as a bending-insensitive pressure sensor, which measures only the normal pressure even under extreme bending conditions. [82] As shown in Figure 7c, the resistance of composite nanofibers decreased drastically (>6 orders, from 6 GΩ to 1 kΩ) with application of a very small pressure (%800 Pa), exhibiting very high pressure sensitivity. Although resistive sensors commonly have a simple device structure and working mechanism, their high standby power consumption arising from the need to monitor resistive changes in real time and temperature sensitivity have limited their extensive application.

Capacitive Pressure Sensors
Capacitive pressure sensors have been widely used in various consumer electronic products and show mature craft and high reliability. Based on first principle, there are three variables that are sensitive to changes in pressure: relative permittivity ε r , area A, and distance d. The change in d is commonly used to measure normal forces, the changes in A are typically used to measure shear forces, and changes in ε r are used to measure forces with specially designed materials that have limited development. Improving sensitivity and response time for tactile sensing applications is a great technical challenge. A pyramid structured polydimethylsiloxane sensor has been developed and presents high sensitivity of 0.55 kPa À1 and a fast relaxation time in millisecond range, as shown in Figure 7d. [83] The increase in capacitance in the structured film arises from the change of air volume and distance between two parallel electrodes. The air voids in polydimethylsiloxane (PDMS) film induced far less elastic resistance in the structured films, and the air has a lower dielectric constant than PDMS. Together, these two factors determine the high sensitivity. In addition, the sensitivity and pressure range are highly tunable, ranging from 0.0001 to 0.55 kPa À1 , by designing the shape of the PDMS pyramidal microstructure. [90][91][92][93] Recently, an improved structure by adding a lamination layer was demonstrated to significantly reduce the variability from fabrication and achieved better performance in sensitivity and response time. [94] Compared to other sensing mechanisms, capacitive pressure sensors are suitable for a wide range of applications and stand out in terms of good sensitivity to both static pressure and dynamic pressure, low power consumption, and fast response time.

Piezoelectric and Triboelectric Pressure Sensors
Piezoelectric and triboelectric sensors are self-powered sensors that transform mechanical energy to an electrical signal via deformation. Therefore, they are natural candidates for pressure or strain sensing. Inorganic piezoelectric materials, such as PZT, ZnO, and BaTiO 3 have been actively studied for tactile sensing due to their large piezoelectric and excellent electromechanical coupling coefficients. [84,95,96] Researchers have demonstrated a self-powered piezoelectric sensor based on a PZT thin film (Figure 7e). [84] Using an inorganic-based laser lift-off process, a PZT film was transferred onto an ultrathin PET substrate, exhibiting excellent flexibility. With a sensitivity of 0.018 kPa À1 and a response time of 60 ms, the piezoelectric sensor adapts to multiple real-time biomedical monitoring applications. Triboelectric devices have also been widely adopted for selfpowered tactile sensing. By employing a pyramid-structured PDMS/Ag-nanowire nanoparticle on an Al dual layer structure, a triboelectric active sensor (TEAS) was demonstrated (Figure 7f ). [85] Relying on the coupling effect of contact electrification and electrostatic induction, the TEAS presents unique pressure response characteristics of the open-circuit voltage and short-circuit current and good performance in terms of high sensitivity (0.31 kPa À1 ), fast response time (<5 ms), and longterm stability (>30 000 cycles). Although most self-powered sensors achieve either high sensitivity or wide measurement range, they are hardly used for precise action identification. [86,97] Hybrid piezoelectric-triboelectric sensors (HPTSs) have been proposed to further enhance the sensitivity and measurement range. [86,98,99] For example, in Figure 7g, a piezoelectrictriboelectric sensor is composed of an ultraflexible nanopolarized microfrustum-array PZT and PDMS film and an m-Cu film.
The interaction (i.e., contact-separation) under pressure between the two films results in a triboelectric signal, while the deformation of the polarized PZT and PDMS film generates a piezoelectric signal. [86] With the combined effect of piezoelectric and triboelectric, piezoelectric-triboelectric sensors commonly present better comprehensive performance than sensors utilizing a single effect, exhibiting excellent sensitivity, wider measurement range, better linearity, and fast response time. The main strength of piezoelectric and triboelectric pressure sensors is that they do not require an external power source and are therefore very energy efficient. But piezoelectric pressure sensors experience high drift over time and are hardly able to detect applied static forces. Furthermore, since electrical spikes (positive/negative pulses) may exist in the output of these sensors due to the combined effects of contact electrification, electrostatic induction, and piezoelectric effect, additional rectifiers are required to preprocess the raw signal. However, since the amplitude and frequency of the spike trains also carry additional information about external forces, these unexpected spikes can be utilized for frequency or rate encoding to unlock a new avenue for tactile information coding. [86]

Intelligent Artificial Perception Using Integrated Memristive Systems
In existing autonomous artificial intelligent systems, such as selfdriving or unmanned automotive vehicles and humanoid robots, conventional sensory systems are separated from the data processing end due to the limited computing resource at the edge. [5] At present, the signal processing module remains bulky with high power consumption and high latency. In the human cognitive and memory system, the sensing systems are located at the front to collect information from the environment. Unlike widely used conventional electronics sensory systems, the sensing system for human cognition has added preprocessing functionalities in addition to the basic function of collecting information from the surrounding. The proximity of preprocessing and sensing functions significantly reduces the bandwidth demands and computational burden of the cerebral cortex. The possibility to collocate of preprocessing and sensing functions is attributed to the compact, high processing efficiency and low power consumption of the peripheral nervous system near the sensory organs. Considering the complexity of biological signals (i.e., visual, auditory, and tactile), which comprise various frequency or wavelength components, conventional CMOS signal processing approaches are incapable of providing sufficient computational power while still maintaining low power cost at the edge. Since memristive devices resemble synapses and neurons in both switching mechanism and structure, some key features of neuronal signal processing functions such as synaptic filtering, spike-timing-dependent plasticity, and spike-rate-dependent plasticity have recently been demonstrated on a single memristive device. Therefore, the integration of memristive devices with high-performance sensors to create integrated artificial perception systems is a promising approach toward achieving nearedge signal processing needed to advance autonomous artificial intelligent systems. In this section, the recent progresses on integrated artificial perception systems are presented.

Intelligent Color and Contrast Visualization with Integrated Artificial Vision Systems
A commercialized vision sensory system typically comprises a video camera and graph processor. The video cameras are responsible for capturing a sequence of still images (frames) by sensing the light intensities of visual reality. Next, the sequence of captured images is sent to the graph processor for analysis. Relevant information, such as objects, edge, contrast, are extracted for decision making. The higher the camera resolution or frame rate, the more computing power is required by the graph processor to perform the tasks required for artificial vision. Meanwhile, the complex circuitry of artificial visual systems based on conventional image sensors, memory, and processing units presents an overwhelming challenge in terms of system design and power consumption. In contrast, the signals are processed differently with higher efficiency in the biological vision system. Although the biological retinae also capture pixellevel light intensities, they perform powerful signal preprocessing before sending visual information to the visual cortex in the brain. Figure 8a shows the pathways from the eye to higher centers in the brain. Before the information goes to the next higher layer, there are six readily distinguishable layers of cells in the retina to convert an optical signal to an encoded electric signal, which is processed by the visual cortex (Figure 8b). Through the collaboration and synchronization of multiple areas in the visual cortex, the biological brain attains the visual perception of shape, color, position, direction, and more. Inspired by the biological vision system, neuromorphic vision systems are being developed with considerable potential to emulate functions of the human vision system, extending beyond the visible-light region. From a functional point of view, enthusiasm has been centered on two functions related to artificial vision, namely, color vision and contrast visualization.

Color Vision
In the human visual nervous system, color vision is generated by three types of sensory photoreceptors with different pigments in the retina, which are found with distinct but overlapping sensitivities in the blue, green, and red part of the spectrum, as shown in Figure 8c. For artificial color vision, there are two methods of integrating visual functionalities in a neuromorphic nanodevice. One is to design synaptic devices that directly respond to light signals, and the other is to connect a synaptic device to a photodetector. Many light-sensitive materials show excellent analog switching behavior. These materials exhibit not only light-sensing property to specific wavelengths but also light-tunable synaptic behaviors. In Figure 9, Zhou et al. used a simple two-terminal structure of Pd/MoO x /ITO to realize an optoelectronic resistive random-access memory (ORRAM), as shown in Figure 9a. [100] The device exhibits ultraviolet (UV) light sensing. Under UV illumination at 365 nm, it switches to a low-resistance state and it preserves its state upon removing the light illumination. A reset process can be triggered by electrical sweeping (Figure 9b). The transition of resistance states was attributed to the photogenerated electrons and protons that led to a change of the Mo ion www.advancedsciencenews.com www.advintellsyst.com Figure 8. Biological vision sensory system. [70] a) Signal pathway from eye to visual cortex in brain. b) Principal cell types and connections of primate retina. c) Spectral sensitivity curves of the three colored visual pigments showing absorbance peaks at wavelengths corresponding to blue, green, and red. d) The on-center receptive field and off-center receptive field. Reproduced with permission. [70] Copyright 2001, Sinauer Associates. Figure 9. UV-based visualization system. [100] a) Schematic of the Pd/MoO x /ITO ORRAM memory device. b) Optical set and electrical reset in sweeping mode. c) Light-intensity-dependent short-term plasticity. d) Long-term potentiation with the pulse number increased to 500 pulses. e) Examples of images before (left columns) and after (right columns) ORRAM-based preprocessing. f ) Comparisons of the image recognition rate with and without ORRAM-based image preprocessing. Reproduced with permission. [100] Copyright 2019, Springer Nature.
www.advancedsciencenews.com www.advintellsyst.com valence state (from 6þ to 5þ) and the formation of the low-resistance HyMoO x phase. Since their device can be programmed by optical stimulation and output light-dosagedependent and history-dependent resistance states (Figure 9c,d), this functionality enables the mimicry of the basic feature of synaptic plasticity in emulating the learning and memory functions of the human brain. Based on this property, the device can "see" pictures modulated in the UV range with improved image processing efficiency (Figure 9e,f ). It should be noted that unlike conventional image sensors, the light-tunable and time-dependent plasticity of the ORRAM mimics the neural signaling mechanisms to also function as a first-stage image processing system (e.g., for contrast enhancement and noise reduction). However, due to the material limitation, the device was responsive only to UV, which restricts its engineering applications, which usually require responsivity over a wide wavelength range.
Using an alternative approach, Seo et al. demonstrated an optic-neural synaptic device that features synaptic and opticalsensing function. [101] The device was created by integrating a synaptic device with an optical-sensing device on the same h-boron nitride (BN)/WS 2 heterostructure, as shown in Figure 10a. By adjusting the density of carriers trapped in the weight control layer (WCL) using light of different wavelengths, the conductance of the heterostructure is tunable. This device exhibited light-tunable synaptic plasticity (Figure 10b) and responded to a wide wavelength range, from red (%655 nm) to blue (%405 nm). Based on the optic-neural synaptic device, they formed an optic neural network to emulate colored and colormixed pattern recognition capability (Figure 10c). By training the optic neural network for 600 epochs, they achieved a >90% recognition rate for color-pattern recognition task, which is similar to a color-blindness test (Figure 10d,e). These results laid the foundation for future work on designing neural networks that comprise sensing and training functionalities for highly integrated complex pattern recognition tasks.
In another approach, Lee et al. also reported the design and fabrication of a novel memristive device created by combining a photodetector and a stretchable organic nanowire synaptic transistor, as shown in Figure 11. [102] Although their application scenario mimics biological synapses in the human sensorimotor nervous system (i.e., tactile sensation), the excellent lightsensitive property of their device warrants the mention of their system in this section on integrated artificial vision. As shown in Figure 11a, poly(3-hexylthiophene):phenyl-C61-butyric acid methyl ester was used as the photodetector to generate voltage spikes upon exposure to light signals. These voltage spikes from the photodetector would continuously modulate the conductance state of the organic nanowire synaptic transistor. Using this combinatorial approach, paired pulse facilitation, spike-voltagedependent plasticity, spike-number-dependent plasticity, and Figure 10. System for red green blue visualization. [101] a) Schematic of the human optic nerve system, the h-BN/WSe 2 synaptic device integrated with the h-BN/WSe 2 photodetector, and the simplified electrical circuit for the optic-neural synaptic device. b) Long-term potentiation and depression curves under different light conditions. c) Examples of the training and the testing datasets consisting of single-colored and color-mixed numeric pattern images, respectively. d) Weight mapping images after the 12th and 600th training epoch. e) Activation values of each output neuron in cases of a single-colored number (blue 4) and a color-mixed number (red/green-mixed 4) after the 600th training epoch. Reproduced with permission. [101] Copyright 2018, Springer Nature.
www.advancedsciencenews.com www.advintellsyst.com spike-frequency-dependent plasticity characteristics were successfully demonstrated. The most unique aspect of this work is that since electric signals are generated via the photodetector, no external power source is needed. Furthermore, their photodetector also responds to a wide range of wavelengths from the UV to infrared region. In Figure 11c,d, the reaction of their device to light patterns can be utilized to represent the international Morse code, highlighting the potential of this organic optoelectronic synaptic device as an optical wireless communication device.

Contrast Visualization
In the biological visual system, contrast visualization is enabled by the receptive field, which is defined as the area of the retina for which light influences the activity of that neuron. As shown in Figure 8d, there are basically two kinds of receptive fields: the on-center receptive field and the off-center receptive field, which are distinguished by different organization of photoreceptor cells and bipolar cells. The on-center cells are only excited when the center of the receptive field is illuminated. On the contrary, center illumination will suppress the off-center receptive field, whereas surrounding illumination will excite it. Therefore, the receptive fields are sensitive to the difference of light rather than the intensity of light. In this way, the contrast and the shape of objects are extracted and sent to the higher visual cortex, where more complicated spatial features are processed.
Bao et al. demonstrated a tunable memristive neuron design based on a 1-transistor-1-memristor (1T1M) structure, as shown in Figure 12a,b. [103] The 1T1M not only exhibits integrate-andfire functionality, but also shows the effect of light signal modulation in the environment. The voltage changes between two terminals of the memristor and transistor are used to mimic the changes of membrane potential caused by spikes and illumination, respectively (Figure 12c). Therefore, the behavior of photoreceptor cells and ganglion cells are fully emulated by the 1T1M structure. By integrating 1T1M neurons with peripheral circuits, a functional artificial receptive field was simulated, which has the ability to extract shape and contrast information from an image and transform it into spike frequency format (Figure 12d). It should be noted that although they used a peripheral circuit to emulate the firing activity of the artificial neuron, and the final shape detection was realized in simulation, this work demonstrated the feasibility of using a memristive system to emulate the receptive field for contrast visualization or shape detection applications in principle.
Gelencser et al. employed hexagonal memristive grids to emulate the smoothing effect occurring in the outer plexiform layer in the vertebrate retina (Figure 13a,b). [104] By utilizing a memristive-based thresholding scheme, they successfully demonstrated edge extraction from grayscale images. As shown in Figure 13b, single memristors were used to function as the bipolar cells' dendrites, and every node within the hexagonal grid was Figure 11. Novel memristive device created by combining a photodetector and a stretchable organic nanowire synaptic transistor. [102] a) Image of organic optoelectronic synapse on an internal human structure model. b) Configuration of organic optoelectronic synapse (photodetector and artificial synapse) and neuromuscular electronic system. c) Visible light-triggered excitatory postsynaptic current (EPSC) amplitudes of s-stretchable organic nanowire synaptic transistor (ONWST) with the International Morse Code of "SOS," which is the most common distress signal. d) Infrared (IR) and ultraviolet (UV) light-triggered EPSC amplitudes of s-ONWST with the International Morse Code of "HELLO UNIVERSE." Reproduced with permission. [102] Copyright 2018, American Association for the Advancement of Science.
www.advancedsciencenews.com www.advintellsyst.com linked with six neighboring nodes through memristive fuses. In this case, the photoreceptor function was not emulated but instead the light stimuli were translated into appropriate current or voltage biases (Figure 13c). The intensity contrast between adjacent pixels imposes the biasing of the underlying memristive fuses with corresponding potential differences. As such, the edges are easily detected by monitoring the outgoing current flow at the outer plexiform layer nodes, as shown in Figure 13d. When applying an image to the hexagonal memristive grids, the devices associated with the nodes were exposed to larger potentials at the edges. Subsequently, the memristors at the edges drifted to lower conductive states at a rate set by the overlying intensity contrast. By monitoring the transient memristance change, the edge information was extracted (Figure 13e). By optimizing the thresholding scheme, the edge image offers more feature details. Although this work did not connect the memristive grid to a photodetector, it emphasizes the powerful processing ability of a memristive system for contrast visualization. It is worth noting that, besides color and contrast sensing, visual perception encompasses a wider range of informative content and context. Emerging reports have focused on the application of memristive systems for spatial position and visual orientation perceptions. [105] Although these reports have yet to integrate memristors with visual sensors, the wider application potential of memristive systems for the construction of multiple functional visual perception has been successfully demonstrated.

Accurate Audio Perception with Integrated Artificial Auditory Systems
The functional objective of the auditory system is to detect and extract information from pressure waves in the surrounding medium, typically air or water. Sound waves with different amplitudes, frequencies, and components are generated by movements or collisions, which primarily tell the perceiver what is happening in the environment. The biological auditory systems present excellent frequency selectivity, where different species adapt to certain frequency ranges, e.g., humans 0.02-20 kHz. Arriving at the ears, sound waves are transmitted mechanically through the outer and middle ear to the sensory hair cells in the cochlea in the inner ear, which generate amplified electrical signals depending on the mechanical vibrations. Auditory nerve fibers transmit information about the receptor potentials to the auditory region of the cerebral cortex, as shown in Figure 14a.
Since sound waves are spatiotemporally encoded and additive, the signal processing of the auditory system is more complicated when compared to that for visual signal processing. Our ears receive a combination of all concurrently active sound sources, which contain many different frequency components. The complexity is further exacerbated since both the frequencies and amplitudes of the components can vary in a single sound. To isolate different frequency components, the ear must have Figure 12. Artificial shape perception retina network. [103] a) Transmission electron microscopy image of a HfO x -based memristor for ganglion cell emulation. b) The 1T1M structure of the tunable memristive neuron. The peripheral circuit is used to generate spikes and reset the memristor. c) The relationship between membrane potential (V 0 ) and input pulse number under different gate potentials. d) With artificial shape perception retina network, the grayscale matrix is transformed into the frequency matrix, and the edges in the image are detected. Reproduced with permission. [103] Copyright 2018, Springer Nature.
www.advancedsciencenews.com www.advintellsyst.com excellent sensitivity and frequency selectivity. In addition, since sounds unfold in time, the modelers of auditory processing cannot ignore time dependencies. Not ignoring the time dependencies is especially challenging upon considering the multiscale nature of the information contained in sounds. Biological cochleae use a space-to-rate encoding in which the Time evolution of all memristive fuse states affiliated with this node. Inset: distinct colors and marks correspond to the neighboring devices to the center node. e) Detection of the edges from an input image. Reproduced with permission. [104] Copyright 2012, APS Physics. Figure 14. Sound localization mechanism of the biological auditory system. a) Central auditory pathways are shown schematically on transverse sections of the medulla, midbrain, and thalamus, as well as on a coronal section of the cerebral cortex. Reproduced with permission. [70] Copyright 2001, Sinauer Associates. b) Representative example of sound-evoked spike train responses of a single gerbil calyx to the synapse/neuron. Reproduced with permission. [106] Copyright 2017, National Academy of Sciences. c) Stimuli from different sources with different acoustic timbre characteristics for natural sound recognition experiment. Reproduced with permission. [107] Copyright 2012, Acoustical Society of America. d) Schematic for sound localization in the horizontal plane, including ITD and interaural level difference. Reproduced with permission. [108] Copyright 2010, American Physiological Society.
www.advancedsciencenews.com www.advintellsyst.com input sound is encoded as trains of pulses created from the outputs of a set of broadly frequency-selective channels (Figure 14b). Such information encoding allows sparser sampling of frequency information according to the active frequency channels rather than the maximal sampling rate required to capture all information from a single audio source. So, the higher layers, like the cerebral cortex, can efficiently extract key information from complex sound wave signals. By using this efficient coding strategy, in addition to the basic sense of hearing, our auditory system produces higher-level perceptions, including acoustic timbre recognition, [107] pitch perception, rhythm perception, [109] and sound localization [108] (Figure 14c,d). Since the research on the use of memristive systems for auditory signal processing is at the nascent phase, there are much fewer reports on integrated memristive artificial auditory sensations, especially when compared with those from the field on visual or tactile sensations.
Here, one of the pioneering successful demonstrations of sound localization using the short-term plasticity behavior of a memristor is described.
The human brain detects sound location by using the interaural time difference (ITD), which is defined as the time difference of the sound arriving at the left and right ears. The ITD in the range of À0.6 to 0.6 ms is the most important clue for sound azimuth location. With the design of HfO x -based 1-transistor-1-resistor (1T1R) synapses based on a spiking neural network, sound location detection was achieved (Figure 15a). [110] To realize time-dependent control of the synaptic weight, the authors connected the memristor with a transistor in series (Figure 15b,c). By connecting the 1T1R synapses as a network, the spatiotemporal encodable characteristic was demonstrated by its ability to detect the precise spike timing interval, which is similar to the biological neural system. Two presynaptic neurons (representing the left ear and right ear, respectively) were employed as the input port and two postsynaptic neurons were designed to generate output internal voltage signals (Figure 15d). By measuring the difference between the internal potentials of the two postsynaptic neurons, the azimuth of the sounding source was precisely identified to allow ITD emulation (Figure 15e,f ).
In addition to using the precise detection of the spike timing interval to enable the sound azimuth and location detection, the intrinsic synaptic computing capability of short-term synapse was also used to process the acoustic signal. A monolayer MoS 2 lateral memristor was designed with a low energy consumption of %10 fJ to process a given acoustic signal (Figure 16a). [111] The resistive switching mechanism of their MoS 2 selector was attributed to the Joule heating effect and doping-induced metal-insulator transition, which results in short-term plasticity (Figure 16b,c). The resistive switching and resultant short-term plasticity enable synaptic computing capability. Without using a transistor, precise temporal computation for sound localization identification was demonstrated (Figure 16d).

Perceptive Tactile and Haptic Sensations with Integrated Artificial Tactile Systems
In the biological system, tactile receptors including mechanoreceptors, thermoreceptors, and nociceptors are located within the different layers of the skin. Tactile sensation and the haptic manipulation of objects begin with the activation of mechanoreceptors in the skin. When they are stimulated or pressed, electrical signals are generated and then flash to the brain, where an overall touch picture from these nerve signals is assembled, as shown in Figure 17a. Three types of mechanoreceptors in the human skin have been identified (Figure 17b): slowly adapting type 1 (SA), rapidly adapting (RA), and pacinian corpuscle (PC) afferents. These afferents encode the shape of an object grasped in the hand. SA afferents are sensitive to static Figure 15. Artificial auditory system for identifying sound propagation direction. [110] a) Illustration of a biological neural subsystem with presynaptic neurons (PREs) connected with a postsynaptic neuron (POST) via plastic synapses. A memristive device with resistive switching behavior can be used to emulate the artificial synapse. b) Typical current-voltage (I-V ) curves of the RRAM device. c) Schematic diagram of a 1T1R synapse, connecting a PRE axon to the POST. d) Schematic illustration of binaural effect, where the ITD provides an estimate of the direction of the sound propagation with respect to the listener, and schematic structure of a 2 Â 2 spiking neural network (SNN) to detect the sound direction from the ITD. e) Experimental sound waveforms of left and right ears, corresponding to the axon potential of the two PREs, and V int for the two POSTs with their corresponding difference. f ) Measured and calculated V int as a function of sound azimuth revealing analog information about the sound propagation direction. Reproduced with permission. [110] Copyright 2018, American Association for the Advancement of Science. stimulation and low-frequency vibrations (<10 Hz). RA afferents are responsive to skin vibrations at the flutter frequency of 10-40 Hz. PC afferents mainly respond to high-frequency vibrations. RA mechanoreceptors sense dynamic changes in force and respond with intense signals during the application or removal of force. In contrast, SA mechanoreceptors are able to sense static forces by responding continuously during prolonged stimulation. Signals from groups of receptors are processed in the brain, where final data interpretation occurs. This breakdown of tactile signal processing helps to reduce the brain's requirements for data processing. Through this signal-processing pathway, complex features from tactile sensors are primarily extracted. Next, perceptions, such as spatial resolution, static or dynamic force sensitivity, temporal resolution, and robustness, are generated. With the active research of tactile sensors in the e-skin community, tactile perception based on memristive systems has emerged as one of the most rapidly growing areas. In this section, the selected recent advancements and innovations are discussed.
By connecting and integrating a resistive pressure sensor with a synaptic transistor, Wan et al. demonstrated a neuromorphic tactile processing system (NeuTap), which mimics the sensory neuron in a biological tactile system and is also capable of perceptual learning (Figure 18a). [114] The pressure sensor with a dual layer structure of a carbon nanotube-coated PDMS/gold interdigital electrode converts pressure stimuli into electrical signals. The synaptic transistors using polyvinyl alcohol (PVA)gated indium-tungsten-oxide (IWO) exhibit short-term plasticity and are capable of extracting the spatiotemporal correlated tactile features (Figure 18b). When a force is applied to the pressure sensor, the resistance of the sensor decreases dramatically, inducing an increase to the voltage drop on the gate terminal of the synaptic transistor, which triggers ionic fluxes to reduce the channel resistance, as shown in Figure 18c. By attaching a NeuTap to a Finger, pattern recognition was also demonstrated (Figure 18d). The range of the output variations tends to be more stable after multiple training sets, indicating the enhanced recognition capability of NeuTap (Figure 18e). The recognition error of NeuTap drops from 44% to 0.4% after six times of learning, suggesting the fast learning rate of NeuTap (Figure 18f ).
In Figure 19a, an artificial haptic neuron system was demonstrated by integrating a piezoresistive sensor with a Nafion-based memristor. [112] The piezoresistive sensor serves as a sensory receptor to transform mechanical stimuli into electric signals, exhibiting a high sensitivity of 6.7 Â 10 7 kPa À1 within a low pressure range from 1 to 5 kPa (Figure 19b). The Nafion-based memristor serves as the synapse, showing repeatable potentiation and depression under an operation pulse train to process the information, as shown in Figure 19c. By connecting the piezoresistive sensor and the Nafion-based memristor, the whole system shows short-term plasticity under pressure stimuli (Figure 19d). As a proof of concept for practical use, the artificial haptic neuron system was adopted to identify English letters through a smart "Pen" (Figure 19e). The current output in Figure 19f indicates that each letter has a different character and is well recognized by monitoring the peak values. Based on this, supervised learning was performed for letter recognition and a 91.7% accuracy was achieved after ten times of training.
Inspired by SA type I afferent sensory neurons, Kim et al. proposed an artificial afferent nerve that consists of three parts: a Figure 16. Artificial auditory system for identifying sound localization. [111] a) Schematic diagram of the two-terminal memristor based on monolayer MoS 2 . b) Joule-heating-driven conductance facilitation with multiple voltage sweeps. c) Paired-pulse facilitation and paired-pulse depression index after two consecutive pulses as a function of interpulse interval. d) Schematic picture of the working mechanism of synaptic computation for ITD-based sound localization. "CA" means cochlea. Reproduced with permission. [111] Copyright 2018, ACS Publications.
www.advancedsciencenews.com www.advintellsyst.com resistive pressure sensor, organic ring oscillators, and a synaptic transistor, as shown in Figure 20a. [115] The ring oscillator acts as an artificial nerve fiber that converts the pressure information from a cluster of pressure sensors into action potentials. The application of nonzero pressure on the sensor results in a supply voltage to the ring oscillator, which causes a constant frequency output. Next, the electrical spike signals from multiple artificial nerve fibers are integrated and converted into a postsynaptic current by a synaptic transistor. With good response performance to external pressure, the artificial afferent nerves were used to identify braille characters pressed on a 3 Â 2 pixel array (Figure 20b). The peak frequency map of postsynaptic currents well reproduced the input image (Figure 20c). With the help of a synaptic transistor, the different letters are readily distinguished (a larger Victor-Purpura distance means more dissimilarity between two spike trains) (Figure 20d). In this work, the information from multiple pressure sensors was demonstrated to flow through a neuromorphic circuit to deliver biomimetic postsynaptic oscillating signals into biological efferent nerves in a detached cockroach leg, leading to the actuation of the tibial extensor muscle in the leg (Figure 20e-g).
In addition to the mechanoreceptor, the nociceptor was also emulated by memristive devices. Yoon et al. proposed and demonstrated an artificial nociceptor based on a volatile diffusive memristor with unique switching dynamics and avoided using bulk signal processing modules (Figure 21a). [116] In their demonstration, the external stimulus is correspondent with the input voltage applied to the device, and the threshold switching parameters of the device play the role of the threshold function of the nociceptor. As shown in Figure 21b, with the voltage stimuli, the threshold voltage shifted to the lower end whereas the output current shifted higher, reproducing the allodynia and hyperalgesia characteristics in a nociceptor. Further, by connecting a thermoelectric module to the diffusive memristor, they demonstrated an artificial thermal nociceptor (Figure 21c). When the thermoelectric plate is heated to different temperatures, a voltage with different amplitudes can be generated from the diffusive memristor (Figure 21d). In this work, their diffusive memristor faithfully resembles the key features of a biological nociceptor, including no adaptation, relaxation, and sensitization, which usually require multiple complicated CMOS circuit units to achieve.

Conclusions
Artificial perception built on memristive systems has risen as a vital strategic research area in recent years with potential as a disruptive technological platform that will revolutionize autonomous systems. The research on artificial perception involves many expertise across multiple engineering fields and scientific disciplines, such as innovative integrated sensor design, bionic Figure 17. Biological tactile sensing range and mechanisms. a) A schematic of the steps required for the transduction of sensory stimuli from natural (top) or artificial (bottom) receptors in the brain. Reproduced with permission. [112] Copyright 2019, John Wiley and Sons. b) Types of mechanoreceptors, their function, temporal response, and density in the hand. Reproduced with permission. [113] Copyright 2016, Springer Nature.
www.advancedsciencenews.com www.advintellsyst.com neuroscience, and advance signal processing (Figure 1). Sensors and processing units are two essential parts for artificial perception system construction. Although great progress and knowledge generation have been made in each of these domains, building an effective artificial perceptron system is not simply a physical connection of the individual parts (e.g., sensor and processor). In addition, growing challenges are continuously being presented from increasingly complex application scenarios (e.g., from autonomous systems, internet of things). At the device level, in the face of these multiple application scenarios,  . Artificial haptic perception system. [112] a) The artificial haptic perception system consisting of a pressure sensor and a Nafion-based memristor. b) Current variation of sensor under pressure ranging from 1 to 50 kPa. c) Device conductance variation under 500 consecutive potentiation and 500 depression pulses in turn for 20 times. d) Current response of the system under static pressure of different weight (0-160 g). e) Illustration of the writing of English characters by the sensory system assembled on a "Pen." f ) Current response of different characters. g) Recognition accuracy of each character after training 10 times. Reproduced with permission. [112] Copyright 2019, John Wiley and Sons.
www.advancedsciencenews.com www.advintellsyst.com novel materials and innovative sensor designs are required to have a wider detection range and higher sensitivity. Although current memristive systems address the speed and energy efficiency issues of computing hardware, several technical challenges on the construction and operation of neuromorphic sensory systems remain to be addressed: 1) the variability in device conductance, especially in high-resistance regimes, which is an intrinsic limiting factor to the bit precision in analogue computing; 2) the nonlinear dynamics in most memristive devices, which causes asymmetric potentiation/depression that degrades learning efficiency and computing accuracy; 3) the continuous conductance (or multiple analogue conductance states) modulation that is required for analog computing; and 4) for both computing and storage cells, the long-duration and highfrequency update and access that memristors experience, which put forward a significant challenge to the endurance performance. Since the development of memristive systems is still in its initial phase, the aforementioned challenges could be well addressed by novel device engineering approaches, together with the codesign and codevelopment of algorithms and peripheral circuits. At the system level, the fabrication of sensor arrays at high densities and good uniformity is needed, while memristive systems must provide high computing power with low latency to process real-time data with high throughputs. Moreover, the integrated system comprising two separated parts could incur additional electronic losses during data transmission. Figure 20. Artificial afferent nerve. [115] a) An artificial afferent nerve made of pressure sensors, an organic ring oscillator, and a synaptic transistor. Only one ring oscillator connected to a synaptic transistor is shown here for simplicity. b) Portion of the connections used for braille reading. c) Applied pressures on the pressure sensor array. Peak frequencies of postsynaptic currents from synaptic transistors connected to only one pixel. d) The smallest Victor-Purpura distance between the postsynaptic currents of different alphabets. e) Hybrid reflex arc made of an artificial afferent nerve and a biological efferent nerve. f ) Photograph of reference and stimulating electrodes, a detached cockroach leg, and a force gauge. g) Isometric contraction force of the tibial extensor muscle in response to pressure on the artificial afferent nerve. Reproduced with permission. [115] Copyright 2018, American Association for the Advancement of Science.
www.advancedsciencenews.com www.advintellsyst.com Therefore, designing a highly integrated device with both sensing and memristive switching functionalities using advanced integrated designs and methods is emerging as a critical imminent direction for low-power neuromorphic sensory systems. At the algorithm level, although a memristive system has intrinsic learning and signal-processing ability, significant development of the algorithm and mathematic fundamentals is required to bridge the gaps. To date, memristive artificial perception systems still have a long way to go before practical applications are possible. Nevertheless, research on memristive artificial perception is in its nascent phase and has just started. Increasing, accelerated efforts devoted to this study will promote the development of artificial intelligent technologies, which can significantly improve operation efficiency and autonomy.