Flexible Strain Sensors for Wearable Hand Gesture Recognition: From Devices to Systems

Hand gesture recognition has attracted extensive research interest, as an essential approach for human−machine interaction. The development of flexible strain sensors offers the possibility of directly measuring the finger motion behaviors for accurate and cost‐effective hand gesture recognition, by placing the sensors on the fingers or integrating them on data gloves. Herein, the operation mechanisms of various flexible strain sensor designs are introduced first. Progresses on related materials and device structures to achieve the demanded figure of merits for flexible strain sensors are then reviewed. In addition to continuous performance optimization of the flexible strain sensor devices, the development of system‐level algorithms is shown to be important for not only recognition functions but also for suppression of nonidealities of the sensor devices, which are difficult to be addressed at device and material level.


Introduction
Developing intelligent approaches for more friendly humanÀ machine interaction (HMI) has been continuously pursued in past decades. [1] One of the most important progresses is integration of touch sensors with displays, which facilitates wide use of finger touch as a main HMI approach in our daily lives. [2][3][4] In addition to simple finger touch, the full set of static and dynamic hand gestures can convey much richer information for HMI. [5][6][7] Various approaches have thus been developed to utilize hand gesture recognition for developing more natural, intuitive, and intelligent HMIs in a nontouch way ( Figure 1). [8][9][10][11][12][13][14] The visual-based techniques, using digital video cameras to capture the finger motion behaviors for further interpretation, are popularly studied and commercialized in many application scenarios. [8] However, these approaches are not cost and energy efficient to achieve continuous real-time hand gesture recognition and are difficult to be implemented in the dark or in the environment with unexpected ambient optical noise. [11] Radar technologies based on radio frequency (RF) sensing have also been adopted to identify the signatures of hand gestures through obtaining the range profile, Doppler, or the angle of arrival features from the received signals. [12] However, the range resolution is determined by the RF wave bandwidth, and it is of high cost to build hardware for generating RF waves with large enough bandwidths to meet the required resolution for hand gesture recognition. Another approach is based ultrasonic signals, for which the hardware can be integrated and miniaturized with relatively lower system complexity. [13] The gestures are recognized via calculations on propagation, reflection, time velocity, and triangulation, respectively. Although with low resolution, this approach is suitable for the environment's lack of light and having magnetic obstacles or noise.
To minimize the influence of unexpected ambient noises, wearable hand gesture recognition systems using micro-electromechanical system accelerometers and/or gyroscope sensors are developed. [10,14] However, due to indirect measurements, the hand gestures that can be recognized are very limited. The development of flexible strain sensors offers the possibility of directly measuring the finger motion behaviors, by placing the sensors on the fingers or integrating them on the data gloves. [15][16][17] The resultant hand gesture systems can have advantages of high accuracy, low cost, and insensitivity to environmental noise. There have been tremendous efforts on developing flexible strain sensors based on different materials and processing techniques. [18][19][20][21][22][23][24][25][26][27][28] The general performance figure of merits to be met for hand gesture recognition include stretchability, response time, and reliability. The finger bending motion can produce strain up to 100% within a very short time (hundreds of microseconds). [29] For practical applications, the sensors need to sustain long-term operation without obvious drift of the sensing performance. The undesired dynamic characteristics, such as overshoot and hysteresis, during fast stretchingÀreleasing cycles should also be suppressed. [24,30,31] Various materials and structures have tried to address these performance issues. [18][19][20][21] However, it is still hard to obtain sensor devices of ideal characteristics. Moreover, there are also always certain performance fluctuations among devices fabricated in the same batch and among different batches. [32] Therefore, various compensation and recognition algorithms were developed to reduce the impact of these factors for accurate hand gesture recognition, so a few hand gesture recognition systems DOI: 10.1002/aisy.202100046 Hand gesture recognition has attracted extensive research interest, as an essential approach for humanÀmachine interaction. The development of flexible strain sensors offers the possibility of directly measuring the finger motion behaviors for accurate and cost-effective hand gesture recognition, by placing the sensors on the fingers or integrating them on data gloves. Herein, the operation mechanisms of various flexible strain sensor designs are introduced first. Progresses on related materials and device structures to achieve the demanded figure of merits for flexible strain sensors are then reviewed. In addition to continuous performance optimization of the flexible strain sensor devices, the development of system-level algorithms is shown to be important for not only recognition functions but also for suppression of nonidealities of the sensor devices, which are difficult to be addressed at device and material level.
were able to be developed based on these "nonideal" flexible strain sensors. [33,34] The previous review focused on the discussion of various material technologies for strain sensors. [35] Different from these reviews, this article will focus on the related materials, device structures, and system algorithms for developing wearable hand gesture recognition systems-based flexible strain sensors.

Device Design
Many kinds of electrical and optical transduction mechanisms, such as fiber Bragg grating (FBG), [36,37] Raman shift, [38,39] and triboelectricity, [40,41] have been adopted to implement strain sensors but require sophisticated measurement equipment and are difficult to be used for wearable hand gesture recognition systems. Many of the developed flexible strain sensors are based on stretchable resistive or capacitive structures. Through material and structure engineering, the performance of the fabricated strain sensors in terms of stretchability and sensitivity can be much improved. The stretchability defines the maximum strain that can be measured and is the first prerequisite to be met for measuring hand motions. To detect the finger bending behaviors, the stretchability needs to be over 100%. Sensitivity is often characterized by the gauge factor (GF), which is defined as the ratio of the measured relative change of electrical signals versus the applied strain. In the following sections, the mechanisms of resistive and capacitive structure strain sensors are introduced first. Then, the strategies in materials and structures for improving the stretchability and sensitivity of the resistive and capacitive strain sensors are reviewed.

Strain Sensor Structure
The typical resistive and capacitive structures used for making stretchable strain sensors are depicted in Figure 2. The resistive structure strain sensor is composed of the strain sensitive layer and two contact electrodes. The strain sensitive layer is normally  www.advancedsciencenews.com www.advintellsyst.com made of the elastomer thin film with a conductive filler, which could be various nano-or microstructure materials such as carbon nanotubes (CNTs), [18][19][20] graphene, [21,22] nanoparticles, [23,24] metal nanowires, [25] conductive polymers, [26] or the fluid liquid metal and ion gel. [27,28] The resistance of the structure can be expressed as where ρ is the resistivity of the material, and L and A represent the length and the cross-sectional area of the sensitive layer, respectively. When a larger strain stimulus is applied, the resulting deformation of the sensitive layer may cause lower ρ, longer L, and smaller A and in turn increase of R. The measured R value can then be used to reflect the applied strain. The stretchability and sensitivity of the devices are dependent on the mechanical properties of the elastomer material, the electrical properties of the filler material, and its dispersion condition in the elastomer film.
The capacitive structure strain sensors are made by sandwiching an elastomer dielectric layer between two elastic conductive electrodes. [42,43] The capacitance of the sensor is expressed as where ε denotes the dielectric constant, and A and d denote the overlap area of two electrodes and the distance between the two electrodes, respectively. When a strain is applied, depending on the material structure design, the deformation of both the dielectric layer and the electrodes might cause changes of the earlier three parameters' values, resulting in the overall capacitance change for strain measurement. As the measured capacitance change is directly related to the mechanical structure deformation, the capacitive strain sensors own advantages of less sensitivity to environment temperature, smaller hysteresis, and better long-term operation reliability compared with the resistive structure devices. Moreover, without DC current paths, the power consumption would also be lower. The main drawbacks of the capacitive structure strain sensors are difficulty of achieving high sensitivity (i.e., large GF) and poor immunity to environmental electrical noise.

Material Engineering
Stretchable sensors that can measure large strain are important for detection of finger bending behaviors. For both resistive and capacitive structure sensors, as depicted in Figure 2, in addition to using host elastomer dielectric material, which can sustain the large strain, the distribution of the conductive materials in the elastic-sensitive film or electrode is vital to maintaining the sensitivity at large strain. [44] A type of effective approach is to use the nanocomposite materials to form the conductive mesh structure to maintain electrical conduction paths in the elastic composite at large strain. [15,25,[45][46][47][48][49][50][51][52][53][54][55][56][57][58][59] Huang et al. used the CNT network as the mesh structure, which bridged dispersed silver nanoparticles (AgNPs) in thermoplastic polyurethane electrospun membrane (TPUEM) (Figure 3a). [45] Compared with the sensor device without CNTs, the stretchability Figure 3. a) Fabrication processes of the composite composed of TPUEM, CNTs, and AgNPs. b) The measured relative resistance changes versus applied strain for TPUEM/AgNPS, TPUEM/CNTs, and TPUEM/CNTs/AgNPs. c) The repeatability of the strain sensor under strains ranging from 120 to 400%. Reproduced with permission. [45] Copyright 2019, Wiley-VCH. of the sensor was improved to 550%, and a high GF of 7066 at 500% strain was achieved (Figure 3b). The sensor also showed good durability under the strain up to 400% ( Figure 3c) and was thus applicable for measuring both subtle and large-scale human motion. Similarly, Liu et al. reported a stretchable strain sensor by sandwiching reduced graphene oxide (RGO)-coated polystyrene microspheres (PS@RGO) and silver nanowires (AgNWs) in an elastic polydimethylsiloxane (PDMS) matrix. [46] High-aspect-ratio AgNWs were used to connect the adjacent PS@RGO microspheres to prevent the electrical failure of the sensor at large strain. By adjusting the AgNW concentration, the sensor exhibited capability of measuring a wide range strain up to 230% with a large GF of 11 at 0À60% of strain and 47 at 100À230% of strain. However, a certain signal drift was observed over 1000 stretching/releasing cycles under 50% strain, which might come from the change of the relative locations between the AgNWs and the conductive microspheres. Duan et al. introduced a highly continuous gold nanowire (AuNWs) network as the mesh structure backbone into the small-sized AgNW network, which not only improved the stretchability from 35% to 90% by bridging the detached AgNW regions, but also helped to achieve high sensitivity in the large strain range (GF ¼ 12 at 5% and 2370 at 70%). [25] However, long-term stretching might impose gradual and cumulative damage to the Au layer, resulting in a certain signal drift being observed over 1000 stretching/releasing cycles under 40% strain.
In addition to using the nanocomposite materials, the mesh structure can also be formed with aligned nanotubes or conductive fibers. Yamada et al. developed a strain sensor using the aligned single-walled CNT (SWCNT) network as the sensitive film. When the aligned nanotube films are stretched, the film can be fractured into gaps and islands and bundles where the bundles bridge the gaps. This allows the strain sensors capable of measuring strain larger than 200% with good long-term stability. [15] Wang et al. fabricated strain sensors by encapsulating a carbonized silk fabric (CSF) sheet within an Ecoflex thin film ( Figure 4a). [47] The formed fabric structure mesh network helped to maintain conductive pathways under extremely large strain (Figure 4b). The sensor was able to be stretched to 500% with a GF of 9.6 at 0À250% strain and 37.5 at 250À500% strain ( Figure 4c). Although overshoots were found when it was stretched up to 100%, 200%, and 300% at a stretching rate of 40% s À1 (Figure 4d), the device exhibited excellent stability after 10 000 cyclic stretchingÀreleasing cycles at 300% strain ( Figure 4e). Based on the similar mechanism, Zhang et al. used carbonized cotton fabric as the mesh network to fabricate the strain sensor, which presented improved sensitivity with a GF of 25 at 0À80% strain and 64 at 80À140% strain. [48] Seyedin et al. fabricated a weft-knitted textile composed of stretchable PU/PEDOT:PSS conductive fibers using a wet-spinning method for strain sensors, and the sensor exhibited a stable response at strain up to 160%. [49] Foroughi et al. knitted highly stretchable conductive textiles using Spandex (SPX)/CNT composite yarns. [50] The textiles exhibited a near-linear response to strain up to %80%, but the GF was much lower than the sensors described earlier.
Under cyclic stretching/releasing, the earlier mesh structures made of solid-state nanostructure materials are difficult to fully recover to their original status, especially at large strain and fast stretching/releasing speed, leading to signal drift and hysteresis issues. Embedding fluidor gel-based conductive channels in a host elastomer film to make the sensitive layer might help to address this issue. [27,28,[60][61][62][63][64][65][66][67] Yoon et al. reported a strain sensor made by filling ion liquid in a linear-structure channel in the elastomer film, showing stable performance under different stretching/releasing speeds and a low degree of hysteresis (DH) of 2.41% at large strain of 200%. [27] DH was defined as (A S ÀA R )/A S Â 100%, where A S and A R are the area of the stretching and releasing state curve, respectively. The refractive index-matched ionic liquid in the microchannels also enabled to achieve high transparency. Choi et al. used a serpentine structure channel filled with ionic liquid to further reduce the hysteresis (Figure 5a). [28] The DH value was reduced to 0.15% at 250% strain ( Figure 5b). The main mechanism was that the serpentine structure channel, leading to uneven strain/stress distribution, which hindered the viscoelastic relaxation of the elastomer sensitive layer (Figure 5c). Besides that, the sensor also exhibited low overshoot of 1.7% at 150% strain and outstanding durability over 3000 stretching/releasing cycles at 300% strain ( Figure 5d).
Mechanical damages resulting from excessive deformation and accidental fracture in stretchable strain sensors are inevitable during practical applications. [68,69] Self-healing ability with the sensitive layer is vital to address this problem. [70] Zhang et al. reported a strain sensor using the cross-linked network of ion gel nanocomposite doped with Fe 3 O 4 nanoparticles as the sensitive layer. [71] Due to the reversible bonding feature of the Fe(III)-O carboxyl bond to reorganize the polymer network at the interface of fractured parts, the sensitive layer was able to self-heal to its original state after being cut (Figure 6a,b). The fabricated sensor presented fast self-healing properties (>95% in 1 s) and strong adhesion (347.3 N m À1 ) (Figure 6c). By tuning the composition of ion gel nanocomposites, the strain sensor showed good reliability and a large GF of 3.94 under a strain up to 800% (Figure 6d,e). Wu et al. fabricated a strain sensor using ethylene glycol (Eg)/glycerol (Gl)Àwater binary hydrogels as the sensitive layers. [72] The strain sensor exhibited a GF of 1.9 under a strain up to 250% and was able to maintain the sensing performance after 9 months. Miao et al. reported a strain sensor based on uniform self-healing liquid films formed on biomimetic PDMS microvilli. [73] The strain sensor exhibited high durability over 22 500 stretching/releasing cycles because the cracks of the microvilli were able to recover completely after releasing due to the capillary force-induced self-healing capability. The sensor showed high linearity in a wide strain range of 200% with a GF of 3. Cai et al. fabricated strain sensors by sandwiching a composite of hydrogel and SWCNTs between two very high bond (VHB) films. [61] With SWCNTs, the devices presented higher sensitivity with a GF of 1.51 and large stretchability of 1000%.
Although these kinds of strain sensors showed promising characteristics of improved stretchability and reduced hysteresis, volatilization of the solvent in the gel might cause performance degradation over time.

Mechanical Structure Design
Besides adopting various material composites, mechanical structures including wrinkle, helical, and porous ones were also implemented to improve the stretchability of the strain sensors. The recently reported highly stretchable and sensitive strain sensors are summarized in Table 1.

Wrinkle Structure
By depositing a conductive thin layer on top of a prestretched elastomer film and releasing, a wrinkle structure of the conductive thin layer on the elastomer film can be formed. Based on this method, Nur et al. made a wrinkle structured gold thin layer on top of the elastomer thin film as the stretchable electrode to construct capacitive-structure strain sensors. [43] The sensitivity of the sensor was improved more than three times compared with the tradition parallel-plate structure. The sensor also exhibited high linearity with negligible hysteresis over a strain range up to 140%. For resistive structure strain sensors, the formation of such a wrinkle structures can help to sustain large strain with less-elastic conductor material. [74][75][76][77] Li et al. reported a buckled sheath-core fiber-based strain sensor by wrapping a prestretched composite film of CNT and thermal plastic elastomer (TPE) onto a prestretched TPE fiber and then releasing (Figure 7a). [74] The sensor exhibited large stretchability c) The measured relative change in resistance of the strain sensor versus the applied strain in the y-direction (strain rate: 30% min À1 ). d) The measured transient response of the relative change in resistance at different strains (strain rate: 40% s À1 ). e) The measured relative change in resistance under repeated loading and unloading of 100% strain for 6000 cycles. Reproduced with permission. [47] Copyright 2016, Wiley-VCH.  (Figure 7b,c). There was very small signal drift over 20 000 stretching/releasing cycles under 2% strain (Figure 7d).

Helical Structures
Forming a helical structure of conductor material is commonly used to fabricate stretchable conductive fibers, which are able to sustain very large strain while using less elastic conductor material. [78][79][80][81][82][83][84] It is relatively easier to be fabricated compared with the wrinkle structure, as no prestretching is needed. Zhu et al. developed a strain sensor by twining the metal-wrapped nylon yarns on the PU core fiber to form the helical structure ( Figure 8a). [78] When the sensor is stretched, detachment of the adjacent metal-deposited nylon yarns occurs, leading to an increase in resistance (Figure 8b). The sensor exhibited large stretchability of more than 300% (Figure 8c) and excellent reliability after 5000 stretchingÀreleasing cycles at 50% strain . c) Illustration of the mechanism where the serpentine structure channel led to uneven strain/stress distribution, which hindered the viscoelastic relaxation of the elastomer sensitive layer. d) The measured relative change in resistance versus the strain for two sensors (3000 loading/unloading cycles at strain of 0À300% for sensor 1 and 5000 cycles at strain of 0À200% for sensor 2). Reproduced with permission. [28] Copyright 2016, American Chemical Society.
www.advancedsciencenews.com www.advintellsyst.com (Figure 8d). Hui et al fabricated a textile strain sensor with a helical structure via a solution-based process, which presented good linearity in a large strain range up to 200% and a high GF of 38.9. [79] After initial hundreds of stretching and releasing cycles, the device showed good long-term reliability for 2000 cycles at 50% strain. Cai et al. used the conductive material composed of CNT and polypyrrole (PPy) to form the helical structure for strain sensors, which had large stretchability of 350% with high linearity, excellent stability, and low strain detection limit of 0.1%. [81]   www.advancedsciencenews.com www.advintellsyst.com

Porous Structure
Forming the porous structure in conductive films can extend the stretchability with less conductive materials via structural deformation. [85][86][87][88][89][90] Cho et al. developed a stretchable strain sensor by forming a SWCNT 3D percolation network inside a nanostructured porous PDMS elastomer film (Figure 9a). [85] The sensor exhibited a GF of 24 with a maximum working range of 160% by tuning the infiltration of the SWCNT solution in the porous PDMS film (Figure 9b). The sensor also showed excellent stability over 1000 stretching cycles ( Figure 9c) and negligible hysteresis under a tensile strain up to 40% (Figure 9d)

System Implementation
To perform hand gesture recognition, the systems need to integrate several strain sensors to detect motion behaviors at different locations. Due to the presence of some nonideal characteristics with the sensors, compensation methods are normally needed. To convert the sensed signals to gesture recognition results, algorithms for recognition also need to be implemented in the systems.

Multisensor Integration
Generally, incorporation of more sensors to collect motion behaviors at various joints of the hands would help to improve the gesture recognition capability or accuracy. To realize highresolution sensor matrix, stretchable thin-film transistor (TFT) pixel arrays with the strain sensor should be developed for active c) The measured real-time response of a sensor upon application of a quasitransient step strain from 0% to 5%. d) The measured relative resistance change over more than 20 000 stretching/releasing cycles at 0.6 s for each cycle with an applied strain of 2%. Reproduced with permission. [74] Copyright 2018, Wiley-VCH.
www.advancedsciencenews.com www.advintellsyst.com matrix addressing to reduce the signal crosstalk and complexity of the layout of electrical interconnects. [91] However, integration of more sensors will increase the complexity of the fabrication process and also the data to be collected. Therefore, there is trade-off between the recognition capability or accuracy and the deployment cost to be considered.
Shintake et al. developed a data glove with five strain sensors, but in the capacitive structure, to measure the bending behaviors of each finger. [66] With the capacitive structure, the two electrode interconnects were distributed in different layers, making the interconnect wiring easier to be implemented, but the achieved detection accuracy of the system was relatively Figure 8. a) Illustration of the fabrication process of the helical structure elastic yarn strain sensor. b) Scanning electron microscope images of the nickelcoated yarns at 0% strain and 50% strain. c) The measured resistance change of as-prepared nickel-based yarns upon a strain to 300%. d) The measured relative resistance changes over 5000 cycles of repetitive stretching/relaxing at strain of 50%. Reproduced with permission. [78] Copyright 2020, Wiley-VCH.
www.advancedsciencenews.com www.advintellsyst.com Figure 9. a) Illustration of the fabrication procedure of the strain sensor by forming a 3D SWCNT percolation network inside a nanostructured porous PDMS elastomer film. b) The measured relative resistance changes upon the applied strain for the sensors made of SWCNTs with different infiltration cycles. c) StretchingÀreleasing cycles of the sensors at strains of 20%, 50%, and 90%. d) The measured hysteresis loops for the strain sensors with different SWCNT infiltration cycles. Reproduced with permission. [85] Copyright 2017, American Chemical Society. Figure 10. a) Photograph of the data glove system based on CNT strain sensors and the measured relative changes in resistance versus time. Reproduced with permission. [15] Copyright 2011, Springer Nature. b) Photos of the data glove system made using ten AgNW-coated P(VDF-TrFE) fiber sensors and the measured signals associated with different hand gestures. Reproduced with permission. [17] Copyright 2016, Wiley-VCH. c) Schematic of the data glove based on the textileÀsilicone composite capacitive strain sensor and the measured capacitance of all the fingers during hand motion. Reproduced with permission. [16] Copyright 2017, Wiley-VCH.
www.advancedsciencenews.com www.advintellsyst.com lower. Yamada et al. integrated ten SWCNT-based strain sensors on a glove to detect the behaviors of five fingers individually for hand gesture recognition (Figure 10a). [15] Chen et al. developed a data glove system with ten fiber-based strain sensors being placed at metacarpophalangeal (MCP) and proximalinterphalangeal (IP) joints, showing the capability of recognizing different hand gestures (Figure 10b). [17] Atalay et al integrated ten capacitive structure strain sensors on the fabric glove with microcoaxial cables for robust electrical connections, and local mechanical stress management structure was designed to maximize the strain-induced signal change from the sensors (Figure 10c). [16] Gu et al developed an e-skin system with ten strain sensors being integrated in a hand-shaped PDMS thin film for identifying various hand gestures, and serpentine copper wires were used to form stretchable and reliable electrical interconnects. [67] Suh et al. designed a finger motion detection glove, which used a current source method for resistance-to-voltage conversion in the signal-conditioning circuit to achieve faster capture speed. [92] 3.

Compensation of Nonidealities
Despite tremendous efforts on materials and structures for improving the sensing performance, hysteresis and signal drift over time and environmental parameters are still two common nonideal characteristics with the resistive structure flexible strain sensors, especially at large strain and fast stretching/ releasing rate. To suppress the influence of these device nonidealities on system performance, compensation methods were developed. Sunny et al. realized the rate-dependent dynamic hysteresis compensation by adding dynamic operators and modifying the hysteresis algorithm in the classic Preisach model. [33] However, as the model was based on complex data analysis, it might be difficult to be obtained in practice. Oliveri et al. proposed a compensation method based on the asymmetric power law (APL) model (Figure 11a), suitable for compensating the hysteresis of the loop rotating around the lower left corner as the stretching rate increased. [34] This model showed improvement in compensation performance compared with the modified Preisach model but was only applicable for a certain hysteresis characteristics. Kim et al. used the optimal transmission theory to compensate the signal drift of strain sensors due to long-term operation. [32] The initial sensor data (source domain) was used to train the neural network, and then the drift data (target domain) was mapped to the source domain through the optimal transmission theory, as depicted in Figure 11b. This mapping relationship was able to be updated in real time based on the drift data of the previous period. This method was also applicable to calibration of massproduced sensors (Figure 11c). First, the neural network was used to extract features from the data set of the strain sensors (source domain). Then, a small amount of data from other Figure 11. a) The method for compensating the hysteresis of strain sensors based on the APL model. Reproduced with permission. [34] Copyright 2018, IEEE. b) The method based on optimal transmission theory for compensating the signal drift of the strain sensors. c) The method based on optimal transmission theory for calibration of mass-produced strain sensors. Reproduced under the terms of the CC-BY license. [32] Copyright 2020, The Authors. Published by Wiley-VCH.
www.advancedsciencenews.com www.advintellsyst.com sensors in the same batch was collected as the target domain. Finally, the data of the target domain was mapped to the source domain through the optimal transmission theory, avoiding training multiple neural networks. Moreover, changes of environmental conditions, including temperature and humidity, might also lead to performance variations of the strain sensors. [93] Although there have been efforts on improving the robustness of strain sensors to varying environmental conditions through material engineering, full suppression of the influence is challenging. [94][95][96] Therefore, system-level compensation is also required to address the environmental influence issues. Figure 12. a) Manual approach to set the threshold for static gesture recognition. Reproduced with permission. [67] Copyright 2019, Mary Ann Liebert. b) Automatic data feature extraction through SVM for static gesture recognition. Reproduced with permission. [100] Copyright 2017, IOP Publishing Ltd. c) RBFNN model for static gesture recognition. Reproduced with permission. [101] Copyright 2020, IEEE d) Polynomial regression for dynamic gesture recognition. Reproduced with permission. [108] Copyright 2016, IEEE. e) U-net network architecture for dynamic gesture recognition. Reproduced with permission. [113] Copyright 2019, ACM.

Gesture Recognition
After the sensors converted the hand gesture-related mechanical deformation into electrical signals, recognition algorithms need to be developed to reconstruct the hand gestures based on the measured electrical signals. Static gesture recognition is a classification process, which divides different sensor signals into several categories and maps them to different gestures. A direct approach for static gesture recognition is through setting threshold for each sensor to distinguish the bending or straightening states of the fingers. [15][16][17]67,[97][98][99] For example, in the e-skin hand gesture recognition system designed by Gu et al., by setting a threshold to the sensed signal from each finger, the hand gesture was able to be recognized to represent a word (Figure 12a). [67] As this method needs to manually set a threshold for each sensor, the processing complexity will be much increased as the number of sensors increases.
To achieve more efficient recognition, many recent studies adopted machine learning methods to automatically extract the sensed data features under different gestures. Based on a data glove system using five resistive strain sensors, Guo et al. used a support vector machine (SVM) classification method to extract the features of the data for recognition of five gestures (Figure 12b). [100] Zhang et al. adopted the radial basis function neural network (RBFNN) and dynamic time warping (DWT) methods (Figure 12c) to recognize static and dynamic gestures captured by a data glove system. [101] The RBFNN has better generalization ability and is more suitable for analyzing time series data than other neural networks. The combination of the RBFNN and the DWT can better deal with the problem of inconsistent time series in dynamic gesture recognition.
Dynamic gesture recognition is a regression process, which converts the sensed temporally varied signals into dynamic gestures. The current methods for recognizing dynamic gestures are mainly divided into two categories: curve fitting and neural networks.
The first type of methods used curve fitting to obtain the relationship between the measured electrical signal (i.e., resistance or capacitance value) and the joint angle for regression. Linear mapping of the measured signal to the angle of the finger joint was used in several studies. [102][103][104][105][106][107] Although such a linear regression is simple to be implemented, it is not applicable to many practical applications,where the strain sensors have nonlinear characteristics. As shown in Figure 12d, Michaud et al. used the least squares method to determine the coefficients of the second-order polynomial regression model for reconstruction of the angles from the measured resistances. [108] Shen et al. regarded the sensor as an arc, and the relationship between the measured resistance value and the bending angle of the sensor was obtained based on the nature of the isosceles triangle. [109] Figure 13. a) The gestures capturing and recognition method for static alphabetic gestures (from A to Z) of sign language recognition. Reproduced with permission. [101] Copyright 2019, IEEE. b) Photo image of wearing the ultrathin strain sensors and the capacitive pressure sensors on the thumb, index, and middle fingers with a data acquisition (DAQ) circuit board for controlling a shooting computer game. Reproduced with permission. [115] Copyright 2019, Wiley-VCH. c) Photo of a wearable wireless music instrument made of graphene woven fabric/PDMS sensors. Reproduced with permission. [116] Copyright 2017 Royal Society of Chemistry. d) Photographs of a hand robot-catching process controlled by the smart glove. Reproduced with permission. [103] Copyright 2020, The Royal Society of Chemistry.
www.advancedsciencenews.com www.advintellsyst.com However, due to differences in finger joints, the polynomial regression is not able to represent the features of all joints. Lu et al. proposed using two regression methods for different types of finger joints: polynomial regression for the IP joints and MCP joints and a spatial analysis method for the carpometacarpal (CMC) joint of the thumb. [110] The second type of methods used neural networks to automatically extract data features. Compared with the curve fitting method, the neural network usually needs more data for calibration, and the final regression result would thus be more accurate. Zhang et al. used five regression models for mapping the resistance values of 10 to 14 joint angles, including linear regression, quadratic regression, SVM, random forest, and neural network, showing that the neural network regression had the highest accuracy. [111] Kim et al. implemented a long short-term memory (LSTM) network with a strain sensor data glove for hand motion tracking, showing the improved similarity to the real results compared with other traditional methods. [112] Glauser et al. designed a data glove consisting of 44 capacitive strain sensors and proposed a U-net network architecture composed of two 5 Â 5 matrices, transforming the input data map matrix and the hand posture, as shown in Figure 12e. [113]

System Applications
Wearable hand gesture recognition systems based on flexible strain sensors have been developed for various HMI applications, including sign language recognition, humanÀcomputer interaction, and robot control, as shown in Figure 13. The systems are generally implemented based on a glove that integrates several stretchable strain sensors and a readout circuit with wireless transmission module. The static or dynamic hand and finger gestures can then be recognized through reading out and processing the multisensor signals for those applications.
Eom et al. developed a data glove system for detecting the bending/straightening behaviors of each finger, which was able to identify the hand gestures for the American sign language (ASL) letters such as ''S,'' ''K,'' ''U,'' ''C,'' and ''A.'' [114] Gu et al. used an e-skin recognition system to translate hand gesture into words of ''HELLO SJTUER.'' [67] Zhang et al. also developed a data glove system for signal language recognition, which used a RBFNN algorithm to map 26 categories of static gestures into the alphabet from "A" to "Z" with a recognition accuracy of 93.33% (Figure 13a). [101] Chen et al. implemented a wearable humanÀcomputer interaction system composed of two strain sensors and one pressure sensor to control a shooting game in the computer (Figure 13b). [115] Liu et al. made a wearable system based on strain sensors for recognizing the gestures to play music (Figure 13c). [116] Wen et al. designed a glove-based humanÀcomputer interaction system to realize the control of virtual reality (VR) shooting games. In addition, by recognizing different gestures, real-time augmented reality flower arrangements were demonstrated. [117] Wan et al. used the commercial make-up accessory as a base material to make a stretchable strain sensor for finger control of virtual objects. [118] Moreover, gesture recognition is also conducive to the realization of VR-based surgical training. By recognizing different hand gripping actions, the types of surgical tools used by the trainer can be determined and displayed in the VR space. After that, the trainer can also control these tools through gestures to achieve the purpose of surgical training. [119] Gesture recognition can also be used to control robots. The electrical signal of each strain sensor in the glove is transmitted to the application program through the wireless transmission module, and the application program converts the electrical signal to control the movement of the robot. Li et al. developed a smart data glove system to remotely control a robotic hand in real time for different operations, including catching a thrown fast-moving copper foil tape in blink rate, holding a still object, and grasping a moving toy (Figure 13d). [103] The glove system designed by Gong et al. is used to remotely control a robotic arm to perform different tasks. [44] In addition, the combination of robot and gesture recognition technology will also be applied in more fields. The elderly and the disabled can control the direction and speed of the wheelchair robot through gesture recognition. Doctors can perform more delicate surgical operations by controlling the robot.

Conclusion
Flexible strain sensors show great promises for developing wearable hand gesture recognition systems in various HMI applications. Based on either the resistive or the capacitive structure, many material and device solutions have been developed to fabricate strain sensors of improved stretchability, sensitivity, and reliability. However, nonideal characteristics, such as hysteresis, signal drift, and sensitivity to environmental influence, still remain to be the common issues with strain sensors. Therefore, system-level algorithms need to be developed for not only converting the sensed signals to gesture results, but also compensation of those device-level nonidealities. The neural network-based automatic feature extraction approaches present improved accuracy for both static or dynamic gesture recognition and can also compensate the influence from the hysteresis and nonlinearity of the strain sensors to a certain extent.
Despite those progresses, several challenges have to be addressed for reliable detection of complex dynamic hand gestures, especially in fast motion. First, innovative material or structure designs are still needed to obtain highly stretchable sensitive layers, which can accommodate repeated cycles of stretching and releasing with fast response and recovery in long-term operation. Second, to recognize complex hand gestures, multisensor integration is required, and strategies for mechanical and electrical co-optimization of the sensitive regions, the interconnects, and the interfaces as a whole system need to be developed. Third, for convenience of wide applications, customizable multisensor topology designs, which can be manufactured to fit different users' hands and cover all key joints for hands, are of importance. Finally, system-level compensation and recognition algorithms need to be developed to be intelligently adaptive to variations of the sensor characteristics and the user scenarios.