All‐Optical Data Processing with Photon‐Avalanching Nanocrystalline Photonic Synapse

Data processing and storage in electronic devices are typically performed as a sequence of elementary binary operations. Alternative approaches, such as neuromorphic or reservoir computing, are rapidly gaining interest where data processing is relatively slow, but can be performed in a more comprehensive way or massively in parallel, like in neuronal circuits. Here, time‐domain all‐optical information processing capabilities of photon‐avalanching (PA) nanoparticles at room temperature are discovered. Demonstrated functionality resembles properties found in neuronal synapses, such as: paired‐pulse facilitation and short‐term internal memory, in situ plasticity, multiple inputs processing, and all‐or‐nothing threshold response. The PA‐memory‐like behavior shows capability of machine‐learning‐algorithm‐free feature extraction and further recognition of 2D patterns with simple 2 input artificial neural network. Additionally, high nonlinearity of luminescence intensity in response to photoexcitation mimics and enhances spike‐timing‐dependent plasticity that is coherent in nature with the way a sound source is localized in animal neuronal circuits. Not only are yet unexplored fundamental properties of photon‐avalanche luminescence kinetics studied, but this approach, combined with recent achievements in photonics, light confinement and guiding, promises all‐optical data processing, control, adaptive responsivity, and storage on photonic chips.


Introduction
While current microprocessors comprise billions of transistors and are computationally efficient, further progress using existing DOI: 10.1002/adma.202304390technologies is limited.Following the Moore's law itself, decreasing the size of transistors down to atomic scale may be insufficient to keep up with growing computing demands.In conjunction with time-and power-consuming computation and lack of parallelism resulting from separation of memory and processor in von Neumann architecture, new concepts in computing technologies are sought and desired. [1,2]Unlike electronic computers, biological neuronal network (BNN) works in time/frequency domain using electrically driven and (bio)chemically modulated spiking activity.Individual neurons are neither fast in signal processing nor transduction, but combined, they integrate and orchestrate activity in multiple neuronal cells in parallel.Neurons are electrically excitable cells that typically communicate between themselves through multiple synapses (≈7000-10 000 per cell) stimulating dendrites and then guiding the output electrical pulses through axons to next multiple neuronal cells (Figure 1a).Biological neural networks are thus complicated systems (the human brain incorporates ≈8.6 × 10 10 neurons), which interconnect thousands of cells, weight signals from many inputs and stimulate resulting output proportionally to biochemically fine-tuned synapses. [3]Brain, neurons, and synapses occur unbeatable in their ability to detect, filter, transduce, analyze, interpret, and memorize multiple information in a massively parallel way, as well as to learn and react specifically to the strength and type of stimulus.In particular, synapses exhibit paired-pulse facilitation (PPF), which enhances response according to prior stimulation. [4]Moreover, spatiotemporal summation, associativity (i.e., enhancement of weak presynaptic activity in one path by strong stimulation in another path), cooperativity (i.e., postsynaptic activity being activated by weak stimulation on many presynaptic paths), and in situ plasticity (the same presynaptic input generates different outputs in postsynaptic neurons depending on in situ mediated neurotransmitter variations) are featured by neuronal cells and synaptic junctions between them. [3]The responsivity of stimulated neurons varies not only in response to the previous stimulation but may further depend on numerous biochemical compounds, such as neurotransmitters and neuromodulators, what comprise neuroplasticity. [5]hese phenomena, together with other mechanisms like  3+ -doped NaYF 4 core passivated with undoped NaYF 4 shell avalanching nanoparticles (ANPs) as photon-avalanching synapse accepts ≈1060 nm photon pulses as input signal and emits ≈800 nm photons as output; energy scheme of Tm 3+ ion pair is presented in the inset with the processes k, W, s referring to Equations (S3)-(S5) (Supporting Information).c) ESA@1060 nm and energy looping through cross-relaxation (CR) processes lead to 800 nmphoton-avalanching phenomenon, i.e., very steep relationship between luminescence intensity I L at 800 nm (inset) under 1060 nm photoexcitation pump intensity I P above the pump threshold (I TH ).d) The pump-power-dependent energy looping leads to slow population buildup time of the looping (n 2 ) and emitting (n 3 ) levels (pulse width Δ = 200 ms), these two levels show slow luminescence decays under pulsed excitation.e) The oddment population of level 2 is responsible for significant paired-pulsed facilitation (PPF) index (pulse width Δ = 0.2 ms, time gap  = 2 ms) as compared to individual, displaced pulses ( = 20 ms); data in (d,e) were simulated for I P = 10 kW cm −2 .The colors in (b-e) correspond to each other.
excitatory and inhibitory of postsynaptic potentials, neurotransmitter susceptibility, short-and long-term potentiation, respectively, are believed to be the major source of adaptations and interpretation of biosensory inputs, transient changes in behavioral states, and neural circuit reprogramming, leading to short-and long-term memory formation in animal brains. [6]hese mentioned features and behaviors are important for neuronal data processing, thus synaptic-like functionality and neuronal circuits (NCs) comprising various artificial electronic and photonic devices [2,[7][8][9] have been studied recently aiming to perform signal processing and neuromorphic computing.For example, such artificial single computing units or their arrays were used to recognize neural firing patterns and to identify synchronization in cofiring between different neurons, [10] for image pattern recognition or time series forecasting. [11,12]Their behavior may additionally promise augmented understanding of nervous systems by real-time, on-site neural activity monitoring and interpretation, or provide feedback in a closed loop. [10]Such artificial synapses were based on electrically stimulated phasechange materials, [9] layered semiconductor microcavities, [13] ferroelectric materials, [14] field-effect transistors, [15,16] metalinsulator-metal heterostructures, [17] carbon nanotubes, [18] layered black phosphorus, [19] NbOx/TaOx memristors, [20] WOx, [12] organic conducting polymers, [21,22] and many others [2,20] including photonic devices too. [8]However, luminescent materials were rarely evaluated in this context [23,24] despite some clear and expected advantages of optical stimulation.Optical synapses (OSs) shall enable feasible remote volumetric addressing, [25] signal transduction by light or through surface plasmon polaritons, [26] as well as combining multiple and weighted input signals.Moreover, the operation of OSs remains unperturbed by electromagnetic field, and nanomaterials for OSs can be massively and reproducibly synthesized at nanoscale by simple chemical methods. [27,28]Among the luminescent nanostructures, hexagonal-NaYF 4 -lanthanide-doped luminescent nanoparticles (LnNPs) have emerged as highly promising materials characterized by low phonon energy, chemical stability, and high luminescence efficiency. [29]Moreover, extensive literature on the synthesis and properties of NaYF 4 nanocrystals describes ways to precisely control their morphology and internal architecture, i.e., core-shell structures. [30,31]The nanoparticle surface passivation with an undoped shell is required for many applications because such a coating shields the luminescent core from the influence of the surrounding environment diminishing possible surface quenching. [32,33]LnNPs exhibit many valuable and unique properties, such as narrowband near infrared (NIR) absorption and multicolor, efficient anti-Stokes (upconverted) emission.The synthesis of LnNPs and their structural and morphologic properties have been characterized in Figure S1 (Supporting Information).
Out of many different upconversion (UC) mechanisms, where low energy photons (800-1100 nm) are converted to higher energy ones (350-800 nm), photon avalanching (PA) is particularly interesting. [34]In photon-avalanching materials, some unusual processes are purposefully augmented and used, unlike in Stokes or conventional upconverting Ln 3+ -doped materials (Figure 1b).First, the photoexcitation wavelength matches the energy of the excited state absorption (ESA) with negligible ground state absorption (GSA).Second, unlike for conventional UC materials, very efficient energy looping is promoted owing to significantly higher doping levels (e.g., 3-20% Tm 3+ in PA vs 0.2% Tm for UC), which occurs through energy cross-relaxation (CR) between metastable levels of two neighbor lanthanide ions (Figure 1b).Thanks to these effects, which enable energy accumulation within avalanching nanoparticles (ANPs), the relationship between pump (I P ) power density and luminescence (I L ) intensity becomes very steep above the photon-avalanche threshold pump power (I TH ) and tends to saturate when rising pump intensity (Figure 1c).This behavior scales with a power law, I L = (I P − I TH ) S , but unlike in conventional UC emission, where S typically equals 2 or 3 and can reach up to max 5, the PA nonlinearity S reaches the values exceeding 10 or more.Moreover, the nonlinearity pertains to the same emission wavelength (Figure 1c, inset), in contrast to other UC mechanisms, where higher S shifts emission wavelengths into the UV region. [35]ecent reports demonstrated S up to 32 or 46, respectively, in singly Tm 3+ -doped and Yb 3+ -Pr 3+ -codoped nanocrystalline matrices at room temperature. [34,36]Originally, such photon ANPs were proposed [37] to bypass the limit of light diffraction and a direct, super-resolution imaging with single diffraction limited excitation beam with ≈70 nm spatial resolution was indeed successfully demonstrated. [34,36,37]Here, for experimental demonstration of PA luminescence in various configurations, we used coreshell hexagonal NaYF 4 :8% Tm 3+ @NaYF 4 nanocrystals that have been previously reported to show efficient PA phenomenon under excitation at 1060 nm. [34]The synthesized nanocrystals with 10.7 ± 1.2 nm shell have been characterized in terms of structure and morphology (Figure S1, Supporting Information) as well as for emission at 800 nm (corresponding to transition between 3 H 4 → 3 H 6 Tm 3+ energy levels) under excitation at 1060 nm (Figure 1c, inset).
While steady-state upconversion and photon avalanche have been relatively well understood and engineerable, [34,[38][39][40] the kinetics of PA emission is showing complex [41,38] and yet undiscovered behavior.The emission intensity and its dynamics result from a balance of rates of a few processes such as ground and excited state absorption, energy migration, multiphonon relaxation and quenching, as well as energy transfer between numerous, long-living electronic levels.These processes are complex, collective combinations of phenomena, whose dynamics is interdependent and reliant from codopant type and concentration, host type and composition, compositional architecture (dopant volumetric distribution), surface exposition, pump power, and population history. [28,42]o understand pump-power-and time-dependent population dynamics of the PA emission, phenomenological differential rate equation numerical simulations were solved (Figure 1d).The population of the intermediate (n 2 -cyan) level decays much slower than the luminescence of the emitting level (n 3 -blue).Because PA materials are originally transparent to the excitation beam, the PA luminescence must proceed by at least marginal intermediate n 2 level population buildup.Typically, this occurs through side-band GSA and it is critical for PA to occur, as the ESA becomes enabled only when this level is occupied.In the course of energy looping between excited and ground states of two neighbor ions (Figure 1b), the population of the n 2 level becomes doubled on every iteration of the energy loop. [40]These processes are the indispensable PA conditions that ultimately spread over the ANP volume and engage more and more intermediately populated activators and let these levels efficiently absorb incoming photons in the ESA process.In consequence, slow and pump-power-dependent luminescence risetime is a characteristic and immanent feature of photon avalanching (Figure 1d and Figures S3-S6 (Supporting Information)).In heavily Ln 3+doped PA materials and for absent ground state absorption, the interaction between neighbor dopant ions is rapidly augmented, which simultaneously leads to extreme susceptibility of the luminescence dynamics to tiny perturbation applied to this energy looping system. [33,40,43]Simultaneously, the luminescence kinetics of such ESA-pumped system should also be susceptible to the oddment population of the looping level (n 2 ).In consequence, the observed PA emission displays some key features typical for neuronal synapses and thus holds promise for all-optical reservoir, neuromorphic, and optical computing. [45]irst of all, the oddment population of the n 2 level enables PPF, which occurs when two stimuli are delivered within a short interval, respectively, leading to augmented response to the second stimulus (Figure 1e).The n 2 remnants (Figures S3-S6, Supporting Information) enable more efficient ESA excitation and allow to correlate enhanced PA emission with time difference between subsequent pulses (Figure 1e).This "fading memory" concept has laid foundations for recurrent neuronal networks to reshape original, complex, time domain input data aiming to simplify their transmission, digitization, and analysis by small, trained linear neural networks. [10]Further analysis of the developed numerical model of this system allows to predict that the ANPs display also other features found in BNN, such as associativity, cooperativity, and plasticity.Due to extreme power dependence nonlinearity in ANPs, a few weak input signals trigger strong response of PA emission (associativity), which holds true when those stimulating pulses overlap in time (cooperativity).We expect the ANPs also to feature photonic plasticity, i.e., adaptation to new input signal by previous stimulation patterns in ANP is observed as in situ dependence of avalanche PPF factor on the pump power and the number of pulses.

Results
All essential BNN features, such as PPF, plasticity, and associativity, have been evaluated in the avalanching-nanoparticlesbased artificial photon-avalanching synapse (PAS) by both, phenomenological modeling (Figure 2) and experimentally (Figure 3).The model was based on a phenomenological differential rate equations (Equations ( S3)-(S5) and Figures S5 and S6, Supporting Information).Highly nonlinear dependence of PA emission intensity on pump intensity (Figure 2a) together with relatively long living intermediate level, in combination of ESA excitation and inefficient GSA, results in unique (as compared to conventional fluorescent species) behavior.Using appropriately adjusted input pulses intensities, input information may selectively trigger PA emission similarly to digital AND gate (Figure 2b).Interestingly, more than two inputs and analog input weighting may be applied, as means to enable associativity and spatiotemporal behavior of this artificial PA synapse, which are originally known in biological and artificial neural networks.These results evidence the PA synapse may be suitable to detect temporal coincidence between input signals, which is discussed in detail later.Moreover, PA emission responds to rising number of identical pulses (Figure 2c) in a nonlinear way, what enables triggering the output not only above given stimulus intensity, but also above its defined frequency.Studying this behavior for constant number of pulses (here N = 9) within a package exhibiting decreasing frequency shows nonmonotonic behavior (Figure 2d).For high frequencies (thus short gap times between the pulses), system is illuminated shortly by quasicontinuous wave.Therefore, due to relatively short exposition to the excitation light (compared to the emission intensity rise time), resulting integral intensity of emitted light is low.On the other side, when frequency between the excitation pulses is low (thus the delays between the consecutive pulses from the excitation set are long), emission is also weak.Optimal conditions are found, when the delays between the pulses are comparable to the lifetime of the intermediate energy level, so the energy delivered with each of the pulses, after energy looping, is stored in the form of the increased population of this level, enabling to enhance the result of the consecutive excitation pulse (like in the paired pulse facilitation effect).This nonmonotonic relation resembles positive and negative potentiation effects, respectively, which exist in various types of synapses.Moreover, the order of weighted pulses (Figure 2e), the pulse widths (Figure 2f), and package frequency (that typically codes intensity of stimulus in artificial neural network (ANN), Figure 2g) modifies the response of the ANP synapse.Such a behavior is the evidence of plasticity of the avalanche synapse that is another important feature found in ANN, i.e., the luminescence response of PAS may be modulated in a dynamic way in situ.The PAS responds to various pulse types, mimicking tonic, bursting, irregular, and adapting ways of response (Figures S7 and S8, Supporting Information), typically found in neurons.The PAS plasticity is further evidenced by experimentally studying the role of pulse number and pulse width, as well as the pulse width modulation (Figure S9, Supporting Information).These results indicate the possibility of PAS to act as all-optical filter and summator, as the output signal depends on number, frequency, amplitude, and sequence of the photoexcitation pulses.Thus, photonic circuits employing PAS units can perform input signal translation and interpretation, aiming to distinguish the history of photoexcitation.
These unusual properties arising from differential rate equations modeling (Figures 1 and 2) were confirmed experimentally using 8% Tm-doped NaYF 4 nanoparticles and bulk 3% Tmdoped LiYF 4 crystals that both exhibit photon-avalanche properties (Figure 3a and Figure S3 (Supporting Information)).The PA synapse shows PPF effect (Figure 1e) exemplified by the fact that the PA emission intensity depends on the time interval between subsequent pulses (Figure 3b-;  = 10-100 ms, single pulse duration-10 ms, I P = 250 kW cm −2 for nanoparticles, and Figure S9 (Supporting Information) for bulk PA crystals), with PPF index >1200% (Figure 3e-PFF).Moreover, predicted nonlinear all-optical pulse counting (Figure 2c), as well as the plasticity of this counting behavior in response to pump intensity or pulse width (Figure 2e,f), is evidenced experimentally (Figure 3c,e-N, in bulk crystal Figure S9 (Supporting Information)).Furthermore, the recovery of the PA emission strongly depends on the temporal gap interval between subsequent pulses (Figure 3d,e-Δ) or the oddment population of the intermediate level (Figure 3f,e-Φ), which are key elements of short-memorylike effect of the PAS synapse.Longer time gaps (Figure 3e-Δ) or smaller residual pump intensities, i.e., stronger emptying the looping level (Figure 3e-ϕ), slow down the recovery time (quantified by the 50% risetimes parameter (Equation (S2), Supporting Information)).Finally, cooperativity of PAS was demonstrated, exemplified by temporal coincidence of two input pulses giving prospect for phase-matching-like optical detection (Figure 3g and Figure S10 (Supporting Information)).
The proposed here all-optical data processing using luminescent, lanthanide-doped inorganic (nano)materials can be considered as a generic approach, suitable for any fluorescent moieties, such as organic dyes, fluorescent proteins, quantum dots, or conventional lanthanide-doped down-or upconverting materials, as all of them are showing characteristic luminescence lifetimes in pico-, nano-, and micro-to-millisecond range, respectively, and may resemble short memory characteristics.However, not only the former luminescent species show no sufficient nonlinear behavior which hinders some of the features presented here (e.g., coincidence detection), but also the short memory effect would not be so spectacular, as the excitation wavelength in conventional fluorophores or upconverting materials is resonant to the ground state absorption.The ground states are basically highly occupied under modest photoexcitation intensities, therefore, the kinetics of the emission intensity rise do not depend in principle on the oddment population of the starting level in absorption transition.In consequence, this unconventional composition of features in PA phenomenon, i.e., the long luminescence lifetime of the intermediate level, strong ESA excitation, weak GSA excitation, and efficient energy looping, are eventually leading to extreme nonlinearities and thus enable unprecedented possibilities.
In order to demonstrate the analogy between temporal behavior of PA phenomenon and working principle of BNN, we  focused on avian nucleus laminaris (NL)-an auditory brainstem structure responsible for sound localization. [46,46]The NL encodes the interaural time difference (ITD) enabling to augment the neuronal response when the sound waves arrive simultaneously to a pair of ears (Figure 4a).Thus, individual neurons in NL behave as coincidence detectors, which are astonishingly insensitive to sound intensity variation. [47]A few explanations of this insensitivity have been postulated and validated experimentally as well as by numerical simulations. [46]For example, a phase locking between sinusoidal sound wave and neuronal firing was modeled using the stimulus model mimicking nucleus magnocellularis neuron firing patters that is known to be a division of the avian cochlear nucleus, which extracts the timing of auditory signals.The model coded ITD by a phase shift (from ϕ = 0°-180°) between the sound waves arriving to both ears (Figure 4b), while the sound intensity was coded as average spiking rates.Such an approach demonstrated phase-shiftand sound-frequency-dependent sound source localization, firing rate saturation for loud sounds, and synaptic plasticity to accommodate responsiveness to both silent and loud sounds. [47]imilarly to the model developed to explain sound localization in NL, the inherent properties of PA luminescence show analogous behavior in response to similar stimulation patterns of optical photoexcitation (Figure 4).These properties are displaying plasticity, i.e., dependence on pump pulse width (Figure 4c), frequency (Figure 4d), and pump intensity (Figure S11, Supporting Information).The analogy between experimental NL behavior [47] (gray curves in Figure 4c) and experimental PAS behavior is presented in Figure 4c ("exp," based on Figure S10 in the Supporting Information).The presented coincidence detection in PAS nanomaterials originates from: 1) remarkable nonlinear luminescence intensity response to photostimulation (Figures 2a and 3a), 2) related critical slowing down of the luminescence risetimes at PA threshold (Figure 1d and Figure S3 (Supporting Information)), as well as 3) sensitivity to oddment population of intermediate level of PA lanthanide ions (Figures 1e, 2c-g, and 3b-g and Figure S10 (Supporting Information)).The studies demonstrate not only the potential to use time-resolved PA emission as coincidence detection mechanism and neuromorphic data processing (Figures 3g  and 4), but also exhibit remarkable susceptibility of the PA emission intensity to photoexcitation modulation signal frequency (f s ) (Figure 4d), "spike" optical excitation intensity (I o ) (Figure S11, Supporting Information), and "spike" pump pulse width () (Figure 4c).Such in situ susceptibility of the response to external parameters of the stimulator is known in neuroscience as plasticity [5] and is considered as the major source of adaptations to sensory inputs, transient changes in behavioral states, and short-term lasting memory (for short-term plasticity), information storage, and neural circuit reprogramming (for longterm plasticity).In BNN, simultaneous activation of PA artificial "neuronal cell" leads to enhanced "synaptic strength" (Hebbian rule "neurons wire together when fire together") and trained response.In ANN, this spontaneous adaptation and true learning is not feasible, but the reprogramming of responsivity of ANN may be achieved in situ.
The short-memory-like behavior and its plasticity can be further evidenced by the fact, the oddment population of the first excited level in PA synapse (n 2 on Figure 1b) critically affects the recovery time (Figures 1d and 3b-f), which means a sequence of pulses produces response stronger than the sum of responses to individual pulses (Figures S7 and S8, Supporting Information), and thus the output PA luminescence intensity depends on the "history" of photoexcitation (Figure 2c).Traditionally, multi-input multilayer trained artificial neural networks are used for optical character recognition, but here we demonstrate that the digits carry enough specific information to be compartmentalized by exploiting photoexcitation-history-dependent photon-avalanche emission intensity.Importantly, by selecting optimized excitation pulse width and delay between pulses, one may in situ control the PA luminescence response and sensitivity to the number of incoming pulses, which ultimately mimic shortor long-memory-like response.Such memory-like behavior is reflected by a nontrivial response to a sequence of pulses, which depends on the number of pulses (Figure 2c), pulse frequency (Figure 2d), weights (Figure 2e), and pulse widths (Figure 2f).In other words, the final PA emission intensity depends on the history of photoexcitation (Figure 5a) and preliminary population of the intermediate level.Because Arabic digits are in principle built from straight or tilted, shorter or longer lines (Figure 5b, top), we raster scanned (vertically (V) and horizontally (H)) various digit signs.Raster V and H scanned images yield unique pixel sequences (patterns), which can be treated as a sequence of optical pulses with pumping power (Figure 2a) corresponding to the intensity of pixels (Figure 5a,b).As a consequence of the short memory performance existing in PA materials, longer sequence of bright pixels (Figure 5a, left) produced disproportionally larger signal on a cumulative integral plot of PA luminescence (Figure 5a, middle) for the vertical (I V ) and horizontal (I H ) scanning (Figure 5a, right).By presenting I V vs I H (Figure 5b), we clearly demonstrate the possibility to distinguish between up to 10 unique signs (digits), even if the signs are noisy (signal to noise S/N = 4).Importantly, despite the conditions, these results were obtained on single PA synapse without using any multilayer neural network or ANN training, and the quality of feature extraction is remarkably high.Obviously, more complicated patterns and larger number of patterns, as well as decreasing signal to noise ratio will make the feature extraction more difficult and less specific.The outcoming PA luminescence intensities after vertical and horizontal scanning (200 repetitions with randomly added noise S/N = 4) have been further delivered as input data to a simple ANN.The ANN was composed of only 2 neuron input layer (I V and I H ), 64 neuron intermediate hidden layer, and 10 neuron output classification layer, featuring reasonable accuracy of 93% (Figure 5c,d).Such an approach, although not as accurate as more complex ANN, provides important advantages, including significantly reduced input dataset, which in turn translates to efficiency of computation and training.
Importantly, the same physical PAS unit may demonstrate plasticity, i.e., by selecting optimized excitation pulse intensity, width, and delay between pulses, one may enable in situ control of the PA luminescence response and sensitivity of compartmentalization to the number of incoming pulses, which ultimately mimics spike-timing-dependent plasticity.
All-optical computing using conventional logic operations is promising [48] but still debatable. [49]Although optical processing may deliver input data remotely by light modulator with no electrical wiring, reduce heat generation, provide lowloss signal transmission over long distances at very high data rates, perform various vector-by-matrix or matrix-by-matrix multiplications, convolution, correlation, or fast Fourier transform (FFT) operation in highly parallel fashion with passive optical elements, [48] there are still unsolved issues of optical-transistorbased operations, such as cascadability of information, fan out, isolation of input from output, presence of critical biasing, independence of logic level from loss or degradation. [49]PA materials studied here may contribute to optical computing on several key levels as soon as we adopt data processing in frequency (spike coding) domain rather than replacing electronic transistors by the optical ones.Such PA materials and approach preserve capabilities of remote, multiple, and weighted input data delivery, nanoscale operation node (synapse) size, optical isolation, analog and in-memory computing, and massive parallelism.While the PA processes are relatively slow (they occur in micro-millisecond scale), they feature unique properties required to compensate slow kinetics by the parallel operation.As PA materials are in principle transparent for the excitation beam (for I P < I Th ) and the PA luminescence intensity depends on pump power density in highly nonlinear manner, such active optical elements can be confined to subdiffraction limited (spatially addressable) volumes (≈70 nm [34,37] ) and in principle could be performed volumetrically [25] with little or no cross talk between synapses.This paves a way to performing optical computing in parallel and simultaneously, with multiple weighted input signals as it actually happens in biological neural networks.Moreover, under specific conditions, photonavalanche emission in nanoparticles demonstrates reversible photodarkening, [34] which enables to all-optically latch a single bit of information for long period of time-a principle standing behind random memory access in modern computers.Absence of electrical or magnetic interferences, no need for electrical interconnection or power supply, no electrical power losses owing to heat deposition, and virtually unlimited bandwidth are not less important advantages. [9]Moreover, photonic synapses promise continuously variable synaptic plasticity that is mimicking true analog behavior of natural processes.Taking into account that lanthanide-doped materials exhibit narrow-and multiband anti-Stokes emission of extreme photostability under single excitation wavelength, these luminescent species and UC processes create a promising toolbox for many photonic applications.Finally, besides in situ plasticity, some features (PA slopes, thresholds, excitation wavelength, and emission color) may be predefined and controlled at synthesis stage of PA nanoparticles, ex-actly as there are known several types of distinguishable neuronal cells.
It is also reasonable to consider the power requirement of the proposed PAS device.The presented studies employ for operation the kinetics of the photoactivated avalanche emission from the Tm-doped NaYF 4 inorganic nanocrystals.The power density required to observe the presented effects was ≈250 kW cm −2 in the focused (diffraction-limited) laser spot (≈350 nm diameter), however, the actual pump power to obtain that high power densities was 2.8 mW.Thus, one 10 ms long excitation pulse, like in most of the presented experimental results, consumes 28 μJ per unit operation, which exceeds current requirements of single bit operation (Table S1, Supporting Information).Nonetheless, this value estimated for the presented proof of principle research is expected to be considerably reduced with using materials of lower PA threshold power-for example, materials with PA threshold I th of the order of tens of kW cm −2 [36] or even single kW cm −2 [50] have been reported.In fact, such energy consumption is comparable to energy required to perform one bit operation in traditional all-electronic devices, where it can reach the values in the range of microjoule to picojoule, typically, however, elementary operations of complementary metal-oxide-semiconductor (CMOS) gate switching requires only 50 aJ-3 fJ of energy. [51,52]In general, developed devices based on optical signal computing are expected to require even lower [51,53,54] or at least comparable [55] energy supply as compared to the electronic ones.It should be also mentioned that it may not be straightforward to quantitatively compare various methods-for example, in the case of the sound localization experiment, avalanching nanoparticles perform relatively complicated task in all-optical manner, while such operation using other approaches would require employment of multiple components and multiple operations.
Nonetheless, energy saving is not the only and major motivation for developing of such optical computing devices.Optical computing devices offer improvement of the bandwidth density and massively parallel space/volume-efficient processing, well fitted into the neural network operation. [53,54]It owes to the limitations of the traditional designs of the electronic devices, as size of the transistors on the chip is approaching the physical limits, and denser packing of the active elements or boosting up the operation frequency is challenging due to the heat dissipation issue. [51,56]Moreover, all-optical devices enable to disregard electric interference with environment, are free from the separate communication and power-supplying wires, and possibly offer fast and efficient information transfer.Moreover, the presented approach offers also further advantages over electronic devices.
1) Each of the ≈20-40 nm dimeter nanocrystals (of size comparable to single transistor in modern electronics) can be in principle considered as independent computing unit able to perform described operations-simple ones, such as logical AND gate, or relatively complex, such as paired pulse facilitation, pulse counting, phase locking, pattern recognition, or coincidence detection.2) As strict excitation power is required to trigger the avalanche features, which are produced exclusively in the diffractionlimited laser spot, volumetric addressing of individual NCs may be obtained. [25]Such space-efficient design combined with multibeam 3D confocal addressing system may bring unprecedented parallel operation.
3) The energy provided to the avalanching NCs fulfills two roles-it is the information input and power supply.The energy pumped into the system is in the dominant part emitted (optical output), limiting the problem of excessive system heating and thus further heat dissipation issues.4) All-optical operation offers integration of proposed system with other optical computing devices.In particular, with the avalanching nanomaterials reported to feature photoswitching effect at single bare core NP level (due to surface defects and energy trapping), which are promising for application as optical memory units. [50]) Avalanching nanocrystals are multifunctional materials, and besides the computing potential, they feature also great and still explored potential for utilization in super-resolution imaging [37] and sensing. [43,57,58]Furthermore, their properties might be tuned and tailored for given application during the synthesis (by adjusting architecture, dopant, or host compositions) or in situ, by proper stimulation (e.g., through variation of excitation powers, length or frequency of the excitation pulses, sequence of the pulses), as we demonstrated in this work.6) The current solution exploits simple, cheap, and widely available semiconductor, fiber pigtailed telecommunication laser diodes from near infrared spectral region at 1064 nm, while the emission from Tm 3+ ions occur at 800 nm falling into visible/NIR spectral region and sensitivity of fast and cheap photodetectors.Both wavelengths fall into transmission windows of conventional optical components.Simultaneously, autofluorescence from the substrate or other components of the setup is avoided and light scattering is also minimal for these two wavelengths, which enables volumetric access for input data delivery and output data readout.

Conclusion
Although a very simple, singular operational PAS unit has been studied here, further developments toward designing more complex, hybrid, and parallel optical processors seem feasible, as light confinement and guiding by optical fibers, waveguides, [59] nanoresonators, [13,26,60] metamaterials, [61] or plasmonic nanoantenna [62,63] materials has been intensively studied and may enable input data delivery, reprogramming, input/output data transfer, and storage.A concern related to PAS could be the stimulation source, but robust, cheap, and fast, electronically controlled single mode semiconductor lasers in the NIR spectral region exist (e.g., at 1064 or 852 nm).The pump power level required to perform single operations with current PAS unit requires kilo-mega W cm −2 densities, but with diffraction limited excitation beams, this actually translates to optical powers in the rather feasible range of single milliwatt.It has been also predicted that PA materials are highly quantum efficient (>50%) light sources, [34] thus, besides little heat overload is expected, heat sinking of bulk PA materials is technically simple and feasible.Moreover, further paths to optimize the PA materials (i.e., reduce PA threshold and enhance their gain) have been defined.[69] Furthermore, the properties of PAS can be either predefined at synthesis stage (by activator or host composition [67] as well as by core-shell architecture [69] ) or alternatively may be in situ controlled by pump intensity, pump pulse width, and gap between pulses.Finally, no requirement for (ultra)low temperatures for PA operation or ultrastable and ultranarrow spectral lines of excitation sources are another great advantage of the proposed materials as compared to other approaches.Although any fluorophore, which features a nonzero luminescence decay time and thus may support PPF in the range of nano [70] to tens of seconds, [23] the highly nonlinear response of PAS synapse is the key advantage of avalanching materials as it enables associativity, cooperativity, and NIR-to-NIR operation unlike the conventional luminescent materials.Moreover, the millisecond operational range is coherent with BNN, and thus may enable fabrication of optoneural interfaces.Moreover, such a time scale balances rather well the technical simplicity to intentionally generate pulse trains and control multiple inputs on the fly.Moreover, it has been proven that the luminescence of lanthanide-doped nanoparticles can be efficiently quenched on-demand by light, [72] which may mimic inhibitory functionalities that are found in some types of biological neurons.These considerations led us to develop and study a timedomain photon-avalanche phenomenon, whose features offer new possibilities such as reliance on multiple input signals, susceptibility to "history" of photoexcitation, and in situ controllable response.The observed performance of the nonlinear and threshold dependence of PA luminescence intensity on the number of identical pump pulses resembles the behavior of neuron response to stimulus.These features enable combined neuromorphic photoactivation, sensing, and computation applications on photonic circuits.For example, the PAmemory-like behavior was illustrated to be suitable for simple, learning-free feature extraction and further classification of digits signs with much simplified 2-input ANN.Additionally coincidence detection was found to be coherent in nature with the way a sound source is localized in animals' neuronal circuits.The obtained all-optical "phase-matching" was susceptible in situ to pump intensity, pulse widths, and pulsed stimulation frequency, which enables in situ plasticity of such optical detection.In summary, not only unexplored fundamental properties of photon-avalanche luminescence kinetics were explored, but combination of our approach with current achievements in photonics, light confinement and guiding, promises all-optical data processing and storage.Therefore, upon further developments, all-optical, multiple, and parallel operation, data transfer, and storage within single photonic processors will become possible.

Experimental Section
NaYF 4 Synthesis and Characterization: Hexagonal NaYF 4 -8% Tm 3 @NaYF 4 core-shell nanocrystals were synthesized by previously reported method with some modifications. [72]Nanocrystals with undoped shell were obtained by methods elaborated in the Supporting Information and characterized structurally by X-ray powder diffraction analysis and morphologically by transmission electron microscopy measurements (Figure S1, Supporting Information).

Microscopy Setup for Power Dependence Measurements and Time-Resolved Experiments:
The optical properties of the nanocrystalline avalanching materials were investigated using home-built setup based on Nikon Ti2 Eclipse inverted microscope with the single mode fiber excitation laser line at 1059 nm, two cooled Vis PMT (Hammamatsu) with quTAG (quTOOLS) photon counter and homemade software, providing the possibility of automatized emission intensity power dependence measurements, as well as arbitrary-waveform-generator (AWG Handyscope5, TiePie and DDS JDS6600)-based time-gated experiments (Figure S2, Supporting Information).The detailed setup description can be found in the Supporting Information.
Data Modeling: All simulations were performed in Matlab custom codes.Reproducing nonlinear behavior of photon-avalanche materials in simulations was achieved by using phenomenological PA differential rate equations for Tm 3+ :NaYF 4 nanoparticles (Equations (S3)-(S5), Supporting Information).Methodology of sound localization simulations and experiments, as well as digit feature extraction and classification are explained in the Supporting Information.

Figure 1 .
Figure 1.a,b) Analogies between signal processing in biological (a) and artificial (b) photon-avalanching synapse (PAS).a) Neurons communicate in spatiotemporal domain by transmitting electrical spike trains from presynaptic neurons, over axons to dendrites of the postsynaptic neurons.Communication occurs in synapses through neurotransmitters modulated Na + influx.b) 8% Tm 3+ -doped NaYF 4 core passivated with undoped NaYF 4 shellavalanching nanoparticles (ANPs) as photon-avalanching synapse accepts ≈1060 nm photon pulses as input signal and emits ≈800 nm photons as output; energy scheme of Tm 3+ ion pair is presented in the inset with the processes k, W, s referring to Equations (S3)-(S5) (Supporting Information).c) ESA@1060 nm and energy looping through cross-relaxation (CR) processes lead to 800 nmphoton-avalanching phenomenon, i.e., very steep relationship between luminescence intensity I L at 800 nm (inset) under 1060 nm photoexcitation pump intensity I P above the pump threshold (I TH ).d) The pump-power-dependent energy looping leads to slow population buildup time of the looping (n 2 ) and emitting (n 3 ) levels (pulse width Δ = 200 ms), these two levels show slow luminescence decays under pulsed excitation.e) The oddment population of level 2 is responsible for significant paired-pulsed facilitation (PPF) index (pulse width Δ = 0.2 ms, time gap  = 2 ms) as compared to individual, displaced pulses ( = 20 ms); data in (d,e) were simulated for I P = 10 kW cm −2 .The colors in (b-e) correspond to each other.

Figure 2 .
Figure 2. The impact of different temporal and pump photoexcitation schemes on PAS luminescence intensity performance derived from differential rate equation model of PA phenomenon in Tm 3+ -doped nanoparticles.a) The pump-power-dependent PAS emission intensity showing PA threshold (I th ) and the maximum pump power below saturation (I o ), b) realization of 3 input AND gate with three optical binary coded input signals of pump intensityI 1 = I 2 = I 3 = 1/3I o .Only simultaneous presence of all inputs in high state triggers high output value, mimicking spatial summation performance of BNN; c) PAS emission intensity for pulse counting-the avalanche system response for the sets of the I P excitation pulses (with power density close to I th ) which are shown schematically above the emission kinetics chart.d) Frequency-dependent PA emission intensity kinetics I PA (t)-the avalanche system response for the sets of N = 9 excitation pulses, fixed pump power I P = 250 kW cm −2 , fixed pulse width Δ = 50 s, and various frequencies of the pulses in the given set (i.e., for various time gaps between the pulses from one set).e-g) PAS emission intensity dependence on input pump power and temporal excitation pattern variations by intensity weighted pulses order (e), pulse width (f), and excitation history (g).

Figure 3 .
Figure 3. Experimental demonstration of PA luminescence in various configurations of temporal photoexcitation.a) Experimental PA photoluminescence intensity vs pumping power for Tm 3+ -doped NaYF 4 nanoparticles.b) Paired pulse index as delay ( = 10-100 ms) between subsequent 10 ms pulses.c,d,f) PA intensity kinetics in response to: c) 2, 5, 10, 15, 20 pulses, d) two pulses separated by gap time Δ = 1-100 ms, and f) two pulses separated by gap time Δ = 100 ms with various residual pump intensity ratios (Φ = 100% a/A) applied between the pulses.e) PPF index vs gap (PPF), PA intensity vs number of pulses (N), t 50% risetimes vs gap (Δ) and vs modulation depth (Φ) corresponding to (b-d) and (f), respectively.g) Impact of temporal coincidence of two identical pump pulses (transistor-transistor logic (TTL) standard pulses at two input channels (black and gray) and resulting temporal changes of the excitation power-top; the corresponding PA luminescence-bottom).The demonstrations presented in panels (b-g) were obtained using pumping pulses with the power density of 250 kW cm −2 .

Figure 4 .
Figure 4. Demonstration of optical short-term potentiation and short photonic memory.The analogy of PA phenomenon behavior to sound localization in avian nucleus laminaris (NL); a,b) 0.5 s lasting sound waves reach left (L) and right (R) ear with some phase delay depending on the source localization.b) The top of the phase-shifted sinusoidal waveforms reaching artificial NL, triggers (trig and pulse) photon excitation pulses (pulse width  = 0.4 ms) to independently from laser diodes, which in a spatially colocalized manner stimulate "optical PAS synapse" by an effective pump intensity kinetic profiles (I P = (T L + T R )I o ) and lead to appropriate (experimentally measured) PA kinetic emission intensity responses (I PA (t)), depending on the phase difference (ϕ) between L and R "ears;" c,d) 2 pulse phase-difference (ϕ = 0°-180°)-dependent PA optical synapse integral luminescence intensity shows plasticity in response to (c) pulse width ( = 0.1-50 ms, f = 10 Hz, I P = 10 3 W cm −2 ) and d) sound frequency ( = 10 ms, f = 10-40 Hz, I P = 10 3 W cm −2 ), which enables to detect coincidence of two excitation pulses aiming to control the sensitivity and directionality of detection.The gray empty and solid rectangles denote experimental NL data for low mean input rate (MIR) 150 spikes s −1 and strong MIR 400 spikes s −1 sound wave intensities (data digitized from ref.[46]).The localization relative sensitivity is up to 0.61 for neurons and up to 3.2% deg −1 for PAS (Figure S12, Supporting Information).

Figure 5 .
Figure 5. Direct, ANN-free feature extraction and simple 2 input ANN classification of images.a) Explanation of vertical and horizontal raster scanning, where the pixel intensity (left column) modulates the intensity of identical ( = 1 ms, f = 667 Hz) pulses (middle column) pumping PA synapse and induce time-and excitation-history-dependent PA luminescence intensity (right column); this PA emission intensity kinetics at the end of scanning (red dots) differs between vertically (I V ) and horizontally (I H ) scanned pattern.b) 2D histograms of number of occurrences of the horizontal (I H ) vs vertical (I V ) PA intensities enable simple, feature extraction of noisy (S/N = 4) "0"-"9" digit signs.Probability maps were obtained for N = 200 experiments and signal to noise ratio = 4. c) These I V and I H features for individual experiments became input data for simple ANN composed of 2 neuron input layer, 64 neuron intermediate layer, and 10 neuron output layer, which enabled to classify the digits signs.d) Confusion matrix, showing reasonable classification and 93% accuracy.