Analog Sensing and Computing Systems with Low Power Consumption for Gesture Recognition

Traditional general architecture chips have shown excessive power consumption and insufficient functional redundancy in some customized applications. Flexible electronics also call for customized chips in smart wearable devices. Common chips in intelligent systems process digital signals, and continuous operation of the system clock brings great power consumption. Herein, an intelligent gesture recognition system is developed by combining the flexible 2D materials and an analog computing chip. The pressure‐sensitive sponge with graphene fillings is proposed as a piezoresistive sensor. A nine‐sensor array is used to detect the pressure field distribution caused by hand movement. To get rid of the power consumption caused by the system clock, a novel analog‐computing customized chip which adopts a near‐sensor processing architecture is proposed. It implements the binary neural network algorithm with an analog circuit and completes the recognition of the transmission signal at the hardware level. The chip possesses a low power consumption which is less than 1.8 mW. Moreover, a glove assembled by highly pressure‐sensitive material recognizes mute gestures including Arabic numerals 0–9, with a recognition rate as high as 98.5%. Herein, the prospects for the application of customized smart chips in the field of smart wearable electronics are illuminated.


Introduction
In the 21st century, mankind entered an era of intelligence with the continuous development of basic research fields such as materials and devices. The intelligent system brings a superior user experience due to its high integration and powerful information-processing capabilities. In recent years, the research and exploration of intelligent systems have attracted the attention of a large number of scientific researchers. [1][2][3] In a wide range of smart system applications, smart wearable sensing systems have excellent development prospects in both civilian and military fields. Different from traditional discrete sensing devices, smart wearable systems can perform specific processing on the signals while completing the collection of various sensing signals to obtain the results required by users. [4] Wearable devices usually have to meet the needs of users with small size and strong endurance. [5][6][7][8][9] The integration of the sensing and processing modules poses a greater challenge to the power consumption of the entire system. At the same time, as the demand for intelligence increases, the amount of data that the system needs to process also increases. Controlling the power consumption of the processing module makes this challenge even more severe.
To meet the needs of the earliermentioned smart system, the concept of application-specific integrated circuits (ASICs) is introduced to replace the traditional general integrated circuits. [10][11][12] Compared with general integrated circuits in mass production, ASICs have the advantages of smaller size, lower power consumption, improved reliability, improved performance, enhanced confidentiality, and cost reduction. [13,14] In the design of ASICs, there are two types, digital signal processing (DSP) and analog signal processing (ASP). For DSP, neural network-based processing unit (NPU) digital accelerators is a common option to improve processing efficiency. [15][16][17][18] However, as sensor data throughput increases, digital NPU-based accelerators still suffer from high power consumption consulted by the system clock while computing, which cannot balance the system energy efficiency and whole power Traditional general architecture chips have shown excessive power consumption and insufficient functional redundancy in some customized applications. Flexible electronics also call for customized chips in smart wearable devices. Common chips in intelligent systems process digital signals, and continuous operation of the system clock brings great power consumption. Herein, an intelligent gesture recognition system is developed by combining the flexible 2D materials and an analog computing chip. The pressure-sensitive sponge with graphene fillings is proposed as a piezoresistive sensor. A nine-sensor array is used to detect the pressure field distribution caused by hand movement. To get rid of the power consumption caused by the system clock, a novel analog-computing customized chip which adopts a near-sensor processing architecture is proposed. It implements the binary neural network algorithm with an analog circuit and completes the recognition of the transmission signal at the hardware level. The chip possesses a low power consumption which is less than 1.8 mW. Moreover, a glove assembled by highly pressure-sensitive material recognizes mute gestures including Arabic numerals 0-9, with a recognition rate as high as 98.5%. Herein, the prospects for the application of customized smart chips in the field of smart wearable electronics are illuminated.
consumption. Compared with DSP, ASP is more energy efficient. ASP does not rely on the rising or falling edge of the system clock to trigger and can process incoming perceptual signals at any time. [19] It is an optional solution to process some perceptual signals directly in the analog domain. Such processing includes directly multiplying and adding the original signal at the analog end and passing analog feature values through the analog-todigital converter (ADC). [20,21] In this work, an intelligent low-power-consumption gesture recognition system is fabricated by integrating an analog computing chip and graphene-based piezoresistive sensors. The chip is fabricated by a 180 nm standard CMOS mixed-signal process with a near-sensor processing architecture (NSPA). [22][23][24] With NSPA, this chip can directly process analog signals before the ADC part. Through the direct connection with the sensor, the analog computing chip embedding a binary neural network (BNN) algorithm can preprocess the analog voltage signal composed of the pixel array before the ADC module. The measurement results show that energy efficiency is as high as 545.4 GOPS/W, and the total power consumption is less than 1.8 mW. [25] Mute gesture signals are sent to the chip for processing and recognition in radian diagrams, and the recognition rate is up to 98.5%.

Sensor Design and Manufacturing
To reduce the computing power requirements of gesture recognition systems, it is necessary to obtain more accurate signals, and a more sensitive sensing material is an important prerequisite. Here, a pressure-sensitive sponge with Ecoflex as the skeleton is constructed as a piezoresistive sensor, as shown in Figure 1a. It is similar in size to a commercially available sugar cube. Notably, pure Ecoflex is an insulator. Graphene www.advancedsciencenews.com www.advintellsyst.com was filled into Ecoflex sponge to obtain piezoresistive properties. Graphene is adsorbed on the inner wall of the hole. In the initial state, there are fewer conductive paths inside the sponge, so the resistance is high. When being pressed, the internal graphene contacts form more conductive paths, so the resistance decreases. This is the conduction mechanism of the materials. The pure Ecoflex sponge was placed in a graphene-alcohol mixture to obtain a black pressure-sensitive sponge, shown in Figure 1b. Because of the unique porous structure of this pressure-sensitive sponge, as shown in Figure 1c, it has a maximum deformation of up to 85%. The deformation of the pressure-sensitive sponge with pressure is shown in Figure 1d. When the pressure is higher than 20 kPa, the curve is generally linear. To supplement the piezoresistive characteristics of the pressure-sensitive sponge, the change in current flowing through the sponge with increasing pressure is shown in Figure 1e under a DC voltage of 5 V. The piezoresistive stability of the sponge is shown in Supporting Information, S1. Finally, A glove is assembled to collect the data of different gestures with nine graphene-based pressure sensors, as shown in Figure 1f.
To obtain a significant response, the sensor is attached to the knuckle, and each sensor is connected to a fixed value resistor.

Gesture Recognition Process
The artificial intelligence gesture system consists of three parts. The glove assembled with pressure sensors collects the original gesture signal, and a voltage follower converts the signal. A chip embedded with a neural network algorithm directly processes the collected analog signals. Finally, the display terminal outputs the result. The flow of the gesture recognition process is shown in Figure 2a. As mentioned earlier, the sensing part passes different voltage signals. For each circuit, a voltage follower is responsible for reading the voltage value of the sensor. The voltage follower is used to maintain the voltage signal unchanged while greatly reducing the output impedance. Notably, the signal given by the sensing part remains the analog signal. A chip specifically designed and manufactured for neural network algorithms is adopted to further explore the implementation of wearable smart devices. The chip is fabricated by a 180 nm standard process. The intelligent system supporting the chip preprocesses the signals collected by the sensors and sends them to the chip for edge calculation and obtains the final recognition result. This chip applies the BNN algorithm to analog circuits, directly processes the analog signal obtained by the sensor, and completes the multiply-accumulate calculation of the signal through an analog www.advancedsciencenews.com www.advintellsyst.com circuit such as multipliers inside the chip. The BNN algorithm is embedded in the chip. In fact, many neural network algorithms whose main operation is accumulation, multiplication, and addition can be adopted in the chip.

Chip Architecture
In general, the chip in this work is composed of analog processing elements (A-PEs), which include certain functional blocks such as multiply-accumulate unit (MAU), nonlinear activation function (NL), and pooling layers. As a neural network customized chip, four MAUs inside work in parallel to accelerate the computing of data from the sensor part. As shown in Figure 2b, both the raw data and the weights are fed into the A-PE matrix under the control of the hierarchical finite-state machine (HFSM). The HFSM controls the MAU in the PE to conduct convolution. In continuous time, the intermediate data go through the NL activation function and pooling, which are stored in the output buffer. The output data only contain information and features. Convolutional (C) and pooling (P) layers appear as (nC þ P, n > ¼ 1) groups, the A-PE is designed as the C þ P layout, where C could be reused n times. In the configuration for a 3 Â 3 convolution kernel, a pooling operation requires four convolutional results. Each A-PE contains four parallel MAUs and one pooling unit. In addition, the NL activation function is needed after each convolutional and pooling operation. The NL is also involved in the A-PE structure. MAU is an important operation in each convolutional neural networks(CNN)/deep neural networks(DNN) algorithm and it takes the major computation resources. MAU determines the runtime of the algorithms. In this work, a capacitor array and switched capacitor integrator are adopted to achieve the function of MAU. The capacitor array is subdivided into two parts, the least significant bit (LSB) and a most significant bit (MSB). This split-array architecture can balance the charge between the arrays and reduce the passive influence caused by the huge area of the capacitance array. The split array is ratioed to achieve a configurable weight network, which ranges from 0 to 127. The strategy of the configuration can be illustrated by the following equations.  Figure 2c, the multiply-accumulate operation is decomposed into two steps. First, SP1 is switched to enable analog differential pairs to have access into the split array whereas SP3 is turned on. The closed direction (to V REF or GND) of all the single-pole double-throw switches (SPDT) is dependent on the fed weight. The charge is induced on the other plane of capacitors. Second, SP2 and SP4 are turned on whereas SP1 and SP3 are turned off, and the induced charge is transferred from the node V OUT to the capacitor in the integrator block. The node OUT_P obtains a charge so that the multiply-accumulate operation is completed. The split array is shown in Figure 2d. NL activation is also an essential component of the BNN. NL further reduces the data amounts delivered to the ADC part in the subsequent circuit. In this work, a current mode winner-take-all (WTA) circuit implements the function of the Rectified Linear Unit (ReLU). As shown in Figure 2e, the maximum pooling is achieved by the WTA circuit, where a set of output currents from the NL circuit is delivered to each input port of the WTA circuit. The pooling output is equal to the maximum value among all input currents.

Algorithm
Based on the architecture of the chip above, the chip can run a BNN algorithm. The customized chip processing of the neural network is shown in Figure 3a. There are three layers, input layer, hidden layer, and output layer in the neural network model. The input layer has 9 neurons, the hidden layer has 16 neurons, and the output layer has 10 neurons. The activation function of the output layer is softmax. The neural network model is trained by weight binarization in this paper. Binarization operation would be based on the sign function where w b is the binarized weight and w is the real-valued weight. In this work, the training process of the neural networks contains four steps. 1) Convent the real-valued weights of the neural network w tÀ1 to w b by sign function, and given the BNN input, compute the output of the BNN. 2) Given the BNN target, compute the training objective's gradient η ∂c ∂w b . 3) Update weights of BNN by w t ¼ w tÀ1 þ η ∂c w b . 4) Repeat steps 1, 2, and 3 until the acceptable accuracy rate is reached or the max train step is reached.
w tÀ1 is real-valued weight of previous parameters; η is the learning rate; c is the loss function; and w b is the binarized weight. A radar graph in Figure 3b shows some major indexes of the chip adopted with that in other works. The whole performance comparison of this chip with other works is shown in Figure 3c, which shows an extremely high energy efficiency.

Simulation
To demonstrate the function of the chip designed for neural network computing, a gesture recognition system was constructed based on the above sensors and chip. The system completed gesture recognition including 0-9 Arabic numerals, and its www.advancedsciencenews.com www.advintellsyst.com   [26][27][28]  recognition rate reached 98.5%. The gesture used in the experiment is a series of military gestures corresponding to the Arabic numeral, and the number is not judged based on the number of fingers extended. In Supporting Information, a real-time video of the gesture recognition process was added to better demonstrate the system (Video S1, Supporting Information). Each gesture and its corresponding voltage distribution are shown in Figure 4a. To compare the accuracy of gesture recognition on the digital side, the following simulation was conducted in this work. The glove collects gesture data from Arabian number 0-9 and sends the data to the receiving end. For each gesture, 200 sets of data were collected. The results of the binarization weights are shown in Figure 4b. Normalized data distribution of ten gesture data from À5 to 5 for each sensor is shown in Figure 4c. Figure 4d shows the simulation results of various neural network algorithms on the collected data. Among all the algorithms adopted, the bagged tree gave the highest recognition rate whereas weighted K-nearest neighbor (KNN) possessed the best energy consumption ratio. The confusion graphs of these two algorithms are shown in Figure S2, Supporting Information. Notably, the recognition rate using the neural customized chip is generally the same as those running algorithms on computers.

Conclusion
A neural network customized chip is used as a processing module for intelligent wearable devices. The chip preprocesses the sensing signals at the analog end, which greatly reduces the power consumption requirements of the ADC and even the entire system. The pressure-sensitive sponge prepared by the combination of graphene and Ecoflex is used as the sensing material of a gesture recognition system. Its excellent resistance to change provides convenience and more possibilities for subsequent algorithm identification. Hundreds of sensing points are no longer needed, but a sensing array of less than 10 can complete the recognition of Arabic numbers, thereby reducing the computing power requirements of the intelligent system. This work proposes a solution for the more highly intelligent AIoT equipment.

Experimental Section
The Ecoflex prepolymer solution was prepared by mixing a base and curing agent at a weight ratio of 1:1. A vacuum pump was used to isolate the air in the solution and ensure the solution was well mixed. A commercially sold sugar cube was immersed in the Ecoflex prepolymer solution for 1 h. A glass container with several pieces of resolvable paper was used to hold the sugar cubes taken out of the solution. Resolvable paper protected the bottom of the sugar cube covered by the Ecoflex from sticking to the petri dish, causing the sponge to damage. The glass container was put onto a heating platform at 80 C for an hour to solidify the Ecoflex. Then the sugar cubes and the resolvable paper were dissolved by water and the Ecoflex sponge was dried in an oven until it demonstrated its flexibility. The Ecoflex sponge was soaked in a graphene-alcohol solution for 1 h using ultrasonic dispersion to obtain a graphene piezoresistive sponge. The mass-to-volume ratio between graphene and alcohol was 0.6 g L À1 . Finally, the graphene sponge was heated to dry.

Supporting Information
Supporting Information is available from the Wiley Online Library or from the author.