Toward a Lossless Conversion for Spiking Neural Networks with Negative‐Spike Dynamics

Spiking neural networks (SNNs) become popular choices for processing spatiotemporal input data and enabling low‐power event‐driven spike computation on neuromorphic processors. However, direct SNN training algorithms are not well compatible with error back‐propagation process, while indirect conversion algorithms based on artificial neural networks (ANNs) are usually accuracy–lossy due to various approximation errors. Both of them suffer from lower accuracies compared with their reference ANNs and need lots of time steps to achieve stable performance in deep architectures. In this article, a novel conversion framework is presented for deep SNNs with negative‐spike dynamics, which takes a quantization constraint and spike compensation technique into consideration during ANN‐to‐SNN conversion, and a truly lossless accuracy performance with their ANN counterparts is obtained. The converted SNNs can retain full advantages of simple leaky‐integrate‐and‐fire spiking neurons and are very suited for hardware implementation. In the experimental results, it is shown that converted spiking LeNet on MNIST/FashionMNIST and VGG‐Net on CIFAR‐10 dataset yield the state‐of‐the‐art classification accuracies with quite shortened computing time steps and much fewer synaptic operations.


Introduction
Artificial neural networks (ANNs) such as convolutional neural networks (CNNs) [1] and recurrent neural networks (RNNs) [2] have gained great success in dealing with computer vision, speech and natural language tasks.Even though they are coarsely inspired from the architectures and functions of human brain, there are still two critical issues on huge memory budget and computational power for intensive expensive multiply-accumulation (MAC) operations.This is inconsistent with the fact that biological neurons in the brain communicate with binary spike-based information and compute when necessary. [3]Driven by the evidence of efficient working mechanism in brain, spiking neural networks (SNNs) are designed with a closer biological style than ANNs and attract interest progressively, which are regarded as the third generation of neural networks. [4]Despite contemporary incomplete understanding of this emerging modeling paradigm, the spike-based computations of SNNs have shown powerful potential in spatiotemporal information extraction and cognitive learning field, [5] and can easily support a seamless mapping on some energy-efficient large-scale neuromorphic hardware such as IBM's TrueNorth, [6] Intel's Loihi, [7] and recent Tianjic chip. [8]owever, training a deep SNN to deal with typical real-world tasks such as image classification on CIFAR-10 [9] and achieve a comparable accuracy performance with its reference ANN is still a big challenge.Inspired from Hebbian learning rules [10] in neuroscience, early unsupervised spike-timing-dependent plasticity [11][12][13] rules and supervised learning algorithms like Tempotron [14] and SpikeProp [15] have been tried for shallow SNN training, but couldn't generalize to deeper architectures or more complicated tasks. [16]In the last decade, thanks to the success of back-propagation (BP) [17] training algorithm in deep learning, various BP-based SNNs [18][19][20][21][22][23][24] have been proposed and attain more competitive performances.Although many presented results have achieved much higher accuracy compared with earlier works, [25] there is still no a unified and effective method to train many modern SNN models. [26]The discontinuous nature of spike activities and large simulation latency make multilayer SNNs incompatible with traditional error calculation process in BP algorithm.
Considering another pathway to obtain a high-accuracy SNN, we can first train a decent static ANN with standard BP algorithm, and then perform an as lossless as possible conversion to bypass the difficulties of direct training.A lot of works [27][28][29][30][31][32][33][34][35][36][37] have contributed to this research topic.In these works, one common way is to use the firing rate of spiking neurons to approximate the rectified linear unit (ReLU) activation [38] of ANN neurons.Nevertheless, such conversion would inevitably introduce approximation errors and usually need lots of time steps (hundreds even thousands) [39] to get stable prediction outputs.In this article, we propose a novel ANN-to-SNN conversion framework, which can minimize the approximation errors by the means of a kind of median quantization [40] constraint in ANN training and produce the ANN activation which is exactly representative of SNN firing rate.More specially, we develop a negative-spike compensation technique originally outlined in ref. [38] into much deeper architectures with batch normalization (BN) [41] layer.Compared with previous works, our proposed ANN-to-SNN conversion algorithm achieves a truly lossless accuracy performance, which is even better than its ANN counterpart at some time steps.In addition, experimental results show that our spiking LeNet [42] on MNIST/FashionMNIST [43] and VGG-Net [1] on CIFAR-10 [9] need quite shortened computing time steps to get stable among other works and much fewer synaptic operations (SOPs) than their ANN counterparts.These spiking models consist of primary leaky-integrate-and-fire (LIF) [44] neurons, which are very suited for neuromorphic hardware implementation.
The rest of this article is organized as follows.Section 2 introduces the background and challenge of ANN-to-SNN conversion and Section 3 describes our proposed solutions including a quantization constraint and negative-spike scheme.Experimental results including the network accuracy, response speed, spiking sparsity, computation complexity, and effect of negative-spiking scheme are presented in Section 4. Section 5 discusses the unique characteristics of our SNNs and finally, Section 6 concludes this article.

Spiking Neuron Models
Main operations of an ANN neuron consist of value-based MAC or dot product of input X and weight W as in the left of (1).Then, if there is a subsequent BN [41] layer, the dot product result s needs be normalized by a mean μ and standard deviation σ as in the middle of (1).In addition, γ and β are the trainable scaling parameter and shift parameter, respectively, and ε usually equals to 10 À6 , which keeps the divisor from zero.
Finally, a widely used ReLU function [38] is applied on this preactivation r to give the final output y as in the right of (1).This process is usually static and has no timing concept except for RNNs. [2]In contrast, SNNs usually employ much closer biological neuron models [45] than traditional ANNs, such as Hodgkin-Huxley, [46] Izhikevich, [47] and LIF [44] model.The first two spiking neurons can imitate more biologically plausible schemes in real brain but with higher computational complexity and implementation difficulties, so most of works on SNN training and conversion concentrate on simple LIF neuron model and its variants.A primary discrete-time version of LIF neuron behavior can be described in (2).
Each neuron keeps a neuronal state (membrane potential) V, which is updated at each time step t according to the input integration (dot product) of spike trains X t from its presynaptic neurons and connected synapse strength W, and then leaks by adding a constant L. If the membrane potential V exceeds a predefined threshold θ, the LIF neuron will generate a spike event and transmit it to the subsequent postsynaptic neurons.In this work, we adopt a variant LIF prototype first outlined in ref. [37], which adopts an extra shadow membrane potential and negativespike scheme.More importantly, we enhance it to adapt to a dedicated quantization constraint, which will be described in Section 3 and generalize it to more challenging VGG-Net with BN [41] layer.

Existing Conversion Methods for SNNs
Broadly, there are two main methods used for ANN-to-SNN conversion: weight based and threshold based.The works [29,32,36]  try an original mapping from frame-driven to an event-driven multilayer perceptron (MLP) [42] or CNN by rate coding of spikes.
Based on the high correlation between non-negative characteristic of ReLU function in ANNs and firing rates of spiking neurons, i.e., firing rate ∝ ReLUðvalueÞ, they adopt various layer-by-layer weight scaling techniques to match the magnitudes of two kinds of output information.However, only some small datasets like MNIST [42] and shallow network architectures have been tested.The works [27,39] improve aforementioned methods using more empirical and robust scaling skills and generalize it to some more modern elements like BN, pooling, and SoftMax in conventional neural networks.Their developing open simulation tool (https://snntoolbox.readthedocs.io/en/latest/index.html) can automatically conduct many kinds of ANN-to-SNN conversion.However, it still brings a great drop in accuracy on some large-scale datasets like CIFAR-10, and usually needs many (hundreds or thousands, even more on ImageNet [48] ) time steps to achieve satisfactory performances.On the other way, the works [28,30,31,33-35,37,40] choose to optimize firing threshold while keeping the weight parameters of networks unchanged, to strive for the same correlation effect.An appropriate threshold can be obtained by similar activation normalization methods [27,29] or directly computed from the related network variables of target ANN counterparts. [30,35,37,40]However, this layer-bylayer threshold scaling is more coarse grained and couldn't be well compatible with widely used elements like BN [41] or deeper architectures. [1,39,49]Based on previous works, [28,34] different quantization methods are adopted to optimize both of the weights and thresholds and achieve more faster and highaccuracy performances.
Unfortunately, both of the two ANN-to-SNN conversion strategies inevitably cause accuracy degradation because of the essence of spike approximation (amount of spikes versus ReLU-based values), which we call as over-spiking (also called "unevenness error" in ref. [34]) and under-spiking phenomena (also called "clipping error" in refs.[28,34]).More specifically, we can consider eight simple cases illustrated in Figure 1, where a converted spiking neuron is driven by three input axon branches with connected synapse weights (blue values in oval frame) ∈ {À1,1}.We hypothesize that all of the input and output signals are represented as spike sequences (red vertical lines) ∈ {0, 1} and all of the spiking neurons are set as θ ¼ 1, leakage L ¼ 0, and time window ¼ 4 and use a threshold-subtraction resetting mechanism.Then there will be four unmatched spiking phenomena compared with their reference ANN neurons, according to the description of LIF behaviors in (2).For (A) and (D) in Figure 1, unexpected spikes will be generated during the first few time steps, which cause unmatched output responses with that of ReLU-based ANN neurons.On the contrary, for (E) and (F), a duration of four time steps will not guarantee that all spikes are generated in time (time window ¼ 4).In other words, SNN outputs are less than their ANN counterparts.In other cases ((B, C,G,H)), their (ANN and SNN) outputs are exactly matched.The under-spiking drawback can be resolved by increasing more simulation time steps, but the over-spiking error is irreversible once the error spikes are transmitted to subsequent layers.More generally, spiking latency and errors will be accumulated in higher layers, which must cause the conversion performance unpredictable.In this case, if there is a compensation scheme which can resolve these unexpected spikes, their performance will be significantly improved.
In this work, we first impose a median quantization constraint into ANN training process, which is adjustable for different quantization precisions, and achieve a perfect correlation with SNN firing rate.Then, we adopt a negative-spiking scheme, which can absorb modern BN layer and generalize it to deep VGG-Net networks, while achieving faster response speed and lossless accuracy performance.

Training with Quantization
To deal with spike approximation, i.e., unmatched over-spiking and under-spiking phenomena from the source, we would like to introduce a delicate quantization constraint called median quantization, which was recently detailed in this work. [40]This quantization module is formulated as in (3) where k is an adjustable hyper-parameter representing actual quantization level of ANN ReLU activations.This quantization process is illustrated in Figure 2, where the quantization level k was set to 1 and the continuous ANN activations were discretized to the median values of multiple quantization intervals.
Compared with the original median quantization algorithm, [40] we modified it by removing upper bound restrictions, which allowed us to quantize wider range of values and helped obtain a more accurate reference ANN than theirs.This improvement will be discussed in the next section.
: : : , : : : Without loss of generality, we could perform an equivalent transformation as in ( 4) by integrating this quantization (3) into ReLU activation (1), and then scaled it up for 2 k times to get (5).
: : : , : : : In these two new formulas ( 4) and ( 5), we denote a quantization interval as bin¼ 1=2 kþ1 and variables without the superscript (0) represent the same meanings as in (1), while s 0 is the scaled inner product, y 0 is the scaled ReLU activation (equivalent to the firing rate of SNN neuron), k is the quantization level, and other variables are the scaled BN terms.In form, we completed a perfect fusion of BN operation and quantized ReLU, so that the magnitude of a scaled activation could exactly represent the firing rate of a spiking neuron without any approximation error.It should be noted this fusion method together with quantization was able to support various deep or shallow architectures with or without BN (seen as layer while no extra hardware implementation budget.Intuitively, higher quantization level k would bring better accuracy performance, but more spike activities and larger simulation latency.

Neural Dynamics Behavior
As presented in (5), the quantized ANN activations built a perfectly consistent correlation with SNN firing rate indeed; however, this process was just static with no timing information.In a typical SNN, not all of spike events were arriving synchronously and the spike accumulation was not accomplished instantaneously.Even in a period of discrete duration, there was a fundamental concept called time step, i.e., a particular spiking time interval.As a result, if we directly mapped aforementioned parameters of an ANN to a standard SNN and simulated it with timing spike trains, there would be serious over-spiking and under-spiking phenomena discussed in Figure 1, which caused inevitable accuracy loss like. [29,35,39]These predicaments were slightly improved in ref. [27] by using a threshold-subtraction resetting mechanism and well resolved in ref. [37] by introducing two auxiliary variable, which were called shadow membrane potential and negative-spike scheme.However, in their works, [27,29,35,37,39] they still couldn't build perfectly representative correlation between ANNs and SNNs due to the lack of more suitable quantization constraint in training and didn't consider how to dissolve the modern BN layer, [41] thus the converted models only concentrated on some relatively limited tasks.Another compromised option was to increase simulation time steps like [39] which could produce more expected spikes and cover error spikes to some extent.This kind of method improved classification accuracy slightly while increasing the latency and computation complexity significantly.Our proposed approach was better designed to solve the aforementioned problems.If there was an error spike generated, we only needed to produce a negative spike to compensate it at once.This specific approach was as follows: for each spiking neuron, there was a normal membrane potential V used for common spike accumulation along time steps as in (2), and a shadow membrane potential V s with an initialized value according to (6) used for tracking membrane potential V.
To enable expected positive-and negative-spiking scheme, there were two thresholds θ þ (positive) and θ À (negative) for each SNN neuron, which were computed from BN terms and ReLU function as in (7).
In addition, according to the non-negative characteristic of ReLU function, there was a lower bound, i.e., R applied to the neural membrane potential V, which is given in (9).This bound would avoid producing unnecessary negative spike when the membrane potential V went below ReLU tipping point.At every time step, if the difference, Diff , in (8) between V and V s exceeded the θ þ or θ À , a positive or a negative spike would be generated.
We call this unique negative-spiking scheme as a spike compensation technique which could well solve the timing synchronization issue.In Equation ( 6)-( 9), we omit the corner marks (i.e., "n") of neurons as in Algorithm 1 and 2 for simplification.In fact, all variables including (V, V s , θ þ , θ À , and Diff ) must change from neuron to neuron, because the BN parameters, i.e., (μ, σ, γ and β) varied from different feature channels and layers in a neural network.Finally, there was no need for resetting membrane potential V after a neuron emitted a positive or negative spike and no leakage, i.e., L ¼ 0 at every time step.The complete spiking dynamics behavior with negative-spike compensation is described in Algorithm 1.More specially, because the last layer was usually a fully connected layer with a SoftMax activation function for loss calculation in training and gave a prediction for object recognition, we chose not to perform quantization and conversion operation on it, but to output its maximum membrane potential, which indexed the final category.

Input Spike Encoding
Despite the differences of static image datasets like MNIST, [42] CIFAR-10, [9] and dynamic ones like N-MNIST [43] from some event-based cameras called dynamic vision sensors (ATIS, DVS, and etc.), [50,51] training a standard CNN required to be fed with static frame-by-frame input images.So after the aforementioned training, quantization, and conversion processes, all of pixel-based samples must be transferred to spike-based event sequences, to adapt to the dynamic SNN simulation during discrete time steps.
In this work, we proposed a time-insensitive input-spikeencoding scheme by introducing a pixel-to-spike transfer layer, which is similar to aforementioned ANN-to-SNN conversion Algorithm 1.This module is described in Algorithm 2, where we replaced the input spikes of neurons by original image pixels and the output results were uniform constant spike trains.It should be noted that for this layer, the pixel-based input process would be performed only once during the whole simulation duration for an image instance, there was no need for any other inputs fed to the network after the first time step.The subsequent dynamic behavior would completely depend on network's state itself, as in Algorithm 1.In addition, after a simulation of an image instance was over, all of neuron states including the transfer layer would be reset and prepared for the next input image.

Experiments and Results
We test the proposed ANN-to-SNN conversion method on two gray scale image datasets including MNIST, [42] Algorithm 1. Spiking dynamics behavior with negative-spike modulation.
Input: input spike trains S in , spiking threshold (θ þ , θ À ), membrane potential bound (reset) R, membrane potential leakage L Output: output spike trains S out , spike collector S c Initialization for shadow membrane potential V s , spike collector S c ; for (t = 0; t<Timesteps; tþþ) do for (l = 0; l < Layers; lþþ) do for (n = 0; n < Neurons; nþþ) do Input: input pixels P in , spiking threshold (θ þ ), membrane potential leakage L Output: event-based spike trains S out , spike collector S c Initialization for shadow membrane potential V s ; for (t = 0; t<Timesteps; tþþ) do for (n = 0; n < Neurons; nþþ) do if t == 0 then else No need for repeated image input, pass; FashionMNIST, [52] and a cultured CIFAR-10 [9] dataset, which is a quite challenging classification task for traditional SNNs.We compare the converted spiking LeNet and VGG-Net with several state-of-the-art works of similar network sizes including various BP-based SNNs, and threshold-based or weight-based converted SNNs and their ANN counterparts.We have made a Python implementation available online (https://github.com/edwardzcl/Spiking_Networks_with_Negative_Spike_Dynamics) based on a customized deep-learning library TensorLayer, [53] which can efficiently complete network training, quantization, and conversion process, described in Section 3. We don't use any data augmentation other than a standard random image flipping and cropping for CIFAR-10 training, and it should be noted that our ANN-to-SNN conversion framework has the potential to be compatible with many optimization techniques such as model compression and regularization [54] or other architectures like ResNet [49] for better performances.

CIFAR-10
To verify the effectiveness of proposed method on deeper architectures, we test two kinds of VGG-Net variant (CNN1 and CNN2 as in ref. [24]) both with 10 layers on a more challenging CIFAR-10 dataset. [9]As shown in Table 3, our spiking VGG-Nets with quantization level k ¼ 1 achieve up to 92.77% and 93.35% accuracy on this dataset, which are very competitive among previously reported results.More importantly, these two SNN models are still powerful enough to generalize lossless accuracies by way of the proposed ANN-to-SNN conversion process.In addition, compared with refs.[27,31,33,35,37,39], our spiking networks not only obtain higher accuracy without approximation errors from under-spiking and over-spiking phenomena, but also consume much fewer time steps.
To further explore the robustness of designed SNNs, we impose three ratios (10%, 20%, and 30%) of noises to input data and the test results on MNIST and CIFAR-10 are recorded in Table 4 and 5.It shows that accuracies of our proposed SNNs degrade minor when noise ratio is 10%, and spiking LeNet with k ¼ 1 even gets a slightly better accuracy.In addition, bigger input variations (20% and 30%) cause bigger accuracy loss.In general, spiking models with higher quantization levels show slightly better robustness.Furthermore, we compare these results with other reproducible works, where input signal to noise ratio of SNNs was rarely discussed.It can be seen that our proposed SNNs seem to be more sensitive to input noises compared with refs.[27,29] (running more time steps) and refs.[21,22,24,56] (directly training with BP algorithm [17] ), but superior to ref. [40] (conversion with firing-rate restrictions).

Firing Sparsity and Computational Complexity
We count the average amount of spikes for one sample simulation of spiking LeNet on MNIST/Fashion and VGG-Net (CNN1) on CIFAR-10 except for the last classification layer and the proportion of positive and negative spikes, as shown in Figure 4A-C, Network: 20C5-P2-50C5-P2-500 in Table 1.Network: CNN1 in Table 3.
A) The amount of spikes for LeNet (Table 1) on MNIST.B) The amount of spikes for LeNet (Table 2) on FashionMNIST.C) The amount of spikes for VGG-Net (Table 3: CNN1) on CIFAR-10.The red bars denote the corresponding amount of spiking neurons.
respectively.For networks with higher precisions k (vary from 0 to 2), more spikes are generated but their negative/positive ratios slightly increase from about 1=7-1=5 for LeNet and 1=3-1=2 for VGG-Net.For a comparison between simple LeNet on MNIST/ FashionMNIST and large-scale VGG-Net on CIFAR-10 with the same quantization precision k, negative/positive ratios of the latter are significantly higher, such as from 1=7 of LeNet to 1=3 of VGG-Net subject to k ¼ 1.More specially, we give the amount of spiking neurons in corresponding LeNet and VGG-Net (refer to the red bars in Figure 4).It shows that there are only about 0.5, 1, and 2 spikes per neuron with respective k ∈ {0,1,2}, which suggests an excellent firing sparsity.Furthermore, we compare the amount of computing operations consumed in aforementioned spiking networks and their ANN counterparts in Figure 5.For a typical neuromorphic platform such as TrueNorth [6] and Loihi, [7] SNNs need no high-precision multiplication, only a simple SOP, i.e., addition is required when there is a presynaptic spike coming.In contrast with ANNs on CPU or GPUs, massive general matrix multiplication computations comprising MAC are performed.We hypothesize that an MAC is equivalent to three SOPs.In fact, the power and area cost of a multiplication are usually much more expensive than that of several additions in most of hardware designs.The results show that our converted SNNs with different quantization precisions consume nearly 3, 1, and 0.1 times fewer computing operations for LeNet and 3.5, 2, and 0.6 times fewer for VGG-Net compared to their ANN counterparts, respectively.

Effect of Negative Spikes
As described in Section 2, negative spikes can theoretically compensate the spiking errors when a membrane potential fluctuates, and keep the shadow membrane potential tracking this unexpected change.To verify its effect, we conduct two comparative experiments where two identical spiking networks except for enabling or no negative-spike scheme based on MNIST and CIFAR-10 dataset.
Figure 6 shows that there is a distinct accuracy gap between spiking models with or without negative-spike scheme, especially for spiking models with higher quantization levels or deeper networks like VGG-Net.This result indicates the negative-spike scheme is really important for classification accuracy, even though the proportion of negative spikes is very low (nearly one eighth for LeNet and one fourth for VGG-Net when quantization level k equals to 0, as shown in Figure 4).This effect is consistent with the derivation in ref. [27] where spiking errors are heavily accumulated in deeper layer and cause prediction failure. [39]tried to increase simulation time steps up to 2000, which could produce more expected spikes to overcome this dilemma.However, this method is not cost-effective in terms of inference time and energy.Interestingly, the number of time steps consumed in our SNNs with or without negative spikes are almost the same, both for the slope of the curve in the intermediate state or final stable state in Figure 6.

Discussion
In most ANN-to-SNN conversion works, [27,29,32,[34][35][36]39] only positive-spiking scheme is enabled. Thouh magnitudes of ANN activations are highly relevant to the SNN firing rates, their spatiotemporal dynamics behaviors are very different.For general simulation with ANNs, a group of frame-based feature maps will pass through the whole network only once.However, SNNs need to perform multiple forward processes during a specific duration of simulation and which neuron fires at which time step is indeterminate.Hence, there are more or less inconsistencies, i.e., over-spiking and under-spiking phenomena introduced in Figure 1.
Most previous methods couldn't achieve truly lossless conversion compared with their ANN counterparts.In contrast, for the transfer layer in Algorithm 2, our proposed pixel-to-spike conversion process will be performed only once during the whole simulation duration, and the subsequent dynamic behavior will completely depend on network's state itself.For the middle layers in Algorithm 1, the negative-spike scheme will solve the overspiking and under-spiking phenomena tactfully.To help capture this point, we provide a raster plot of spikes versus time steps as shown in Figure 7.It can be found there will be no any spiking activities when the time step is more than a specific number.In other words, the spiking models will be stable naturally, when all spiking information flow from the transfer layer goes completely through the network.This characteristic is the source of faster speed and lossless accuracies of our SNNs.
Compared with ref. [40], they also propose a dedicated spatiotemporal conversion algorithm but there are at least three important disadvantages.Spatially, an ANN neuron is replaced by multiple SNN neurons, which must increase redundant  computation budget and memory overheads on hardware.Temporally, they couldn't achieve so-called event-driven computation, because their proposed temporally converted neurons in each layer need accumulate spike inputs over all of time steps and then fire spikes over the same amount of time steps.In other words, their models are still frame-based and only achieve spikebased computation layer by layer.Hence, their total simulation time is related to the quantization precision and network depth B  (number of layers).In contrast, our proposed SNNs with negative-spike dynamics enable pipeline-style simulation just like the standard SNNs. [44]Moreover, as shown in Table 3, our SNNs successfully remove the upper bound restriction in the original median quantization algorithm [40] and help to obtain further accuracy improvements than theirs.

Conclusion
We propose a novel ANN-to-SNN conversion framework to obtain high-accuracy and low-latency SNNs.With presented quantization constraint and negative-spike scheme, our spiking networks can achieve truly lossless performance compared with their ANN counterparts, even equipped with BN layer or deep architectures like VGG-Net.Higher quantization precisions can bring slight higher accuracy improvements but produce more spikes and larger simulation latency.Even so, the presented spiking networks still require quite shortened time steps to get stable accuracy in contrast to previous SNN works.More specially, with quantization level k ¼ 1, our spiking LeNet and VGG-Net achieve excellent accuracies comparable to that of full-precision ANNs but consume much fewer (nearly one third) computing operations when deployed on a general neuromorphic hardware.

Figure 2 .
Figure 2. Median quantization on ReLU function.The blue line shows original ReLU function and the red line for the quantized ReLU.

Figure 3 .
Figure 3. A) Accuracy versus speed for spiking LeNet (Table 1) on MNIST.B) Accuracy versus speed for spiking LeNet (Table 2) on FashionMNIST.C) Accuracy versus speed for spiking VGG-Net (Table 3: CNN1) on CIFAR-10.The final stable accuracy and required time steps are given in the lower right corner of the figure.

Table 1 .
Performances of SNNs on MNIST.Bold data indicate the best SNN performance (the highest accuracy or the fewest time steps).

Table 2 .
Performances of SNNs on FashionMNIST.Bold data indicate the best SNN performance (the highest accuracy or the fewest time steps).

Table 3 .
Performances of SNNs on CIFAR-10.Bold data indicate the best SNN performance (the highest accuracy or the fewest time steps).

Table 4 .
Robustness to input noise on MNIST.Bold data indicate the best SNN performance (the highest accuracy or the fewest time steps).

Table 5 .
Robustness to input noise on CIFAR-10.Bold data indicate the best SNN performance (the highest accuracy or the fewest time steps).