Speech Recognition Using Intelligent Piezoresistive Sensor Based on Polystyrene Sphere Microstructures

Rapid advances in wearable sensing technology have demonstrated unprecedented opportunities for artificial intelligence. In comparison with the traditional hand‐held electrolarynx, a wearable and intelligent artificial throat with sound‐sensing ability is a more comfortable and versatile method to assist disabled people with communication. Herein, a piezoresistive sensor with a novel configuration is demonstrated, which consists of polystyrene (PS) spheres as microstructures sandwiched between silver nanowires and reduced graphene oxide layers. In fact, changes in the device's conducting patterns are obtained by spay‐coating the various weight ratios and sizes of the PS microspheres, which is a fast and convenient way to establish microstructures for improving sensitivity. The wearable artificial throat device also exhibits high sensitivity, fast response time, and ultralow intensity level detection. Moreover, the device's excellent mechanical–electrical performance allows it to detect subtle throat vibrations that can be converted into controllable sounds. In this case, an intelligent artificial throat is achieved by combining a deep learning algorithm with a highly flexible piezoresistive sensor to successfully recognize five different words (help, sick, patient, doctor, and COVID) with an accuracy exceeding 96%. Herein, new opportunities in voice control as well as other human‐machine interface applications are opened.

DOI: 10.1002/aisy.202200427 Rapid advances in wearable sensing technology have demonstrated unprecedented opportunities for artificial intelligence. In comparison with the traditional hand-held electrolarynx, a wearable and intelligent artificial throat with sound-sensing ability is a more comfortable and versatile method to assist disabled people with communication. Herein, a piezoresistive sensor with a novel configuration is demonstrated, which consists of polystyrene (PS) spheres as microstructures sandwiched between silver nanowires and reduced graphene oxide layers. In fact, changes in the device's conducting patterns are obtained by spay-coating the various weight ratios and sizes of the PS microspheres, which is a fast and convenient way to establish microstructures for improving sensitivity. The wearable artificial throat device also exhibits high sensitivity, fast response time, and ultralow intensity level detection. Moreover, the device's excellent mechanicalelectrical performance allows it to detect subtle throat vibrations that can be converted into controllable sounds. In this case, an intelligent artificial throat is achieved by combining a deep learning algorithm with a highly flexible piezoresistive sensor to successfully recognize five different words (help, sick, patient, doctor, and COVID) with an accuracy exceeding 96%. Herein, new opportunities in voice control as well as other human-machine interface applications are opened. carbon nanotubes, [30] carbon black, [31] metal nanowires, [32] and graphene, [33] have been used in piezoresistive sensors. Generally, charge transport is easier and quicker in highly conductive materials such as silver nanowire (AgNW), while it is slower and less efficient in materials such as reduced graphene oxide (rGO), which has defects and oxygen-containing groups on the surfaces and edges. [34,35] With the two materials combined, their electrical properties can be enhanced due to synergistic effects. Therefore, the variation in conductive pathways is crucial during our design of piezoresistive sensors.
In addition, a significant body of research has been devoted to using flexible sensors to detect human neck muscle movements in recent years. For example, an intelligent artificial throat based on laser-induced graphene was able to generate sound and detect simple throat vibrations with different intensities or frequencies. [36] Another artificial graphene throat was proposed that also integrated both sound detection and emission in a single device, which can detect simple pronunciations when attaching the device to the human neck. [37] However, these devices only initially detected throat movements without further exploring the feasibility of implementing artificial intelligence for speech recognition. In this case, an MXene-based artificial throat device was developed to detect different pronunciations of several different words, and it can also recognize long vowels and short vowels of the syllable "a" with a deep-learning algorithm when the device is attached to the vibrating membrane of a loudspeaker. [38] However, this research did not attach the sensor to human throat for experimental data collection. More recently, a graphene-based wearable sensor was attached to human throat to classify the words by using the neural network with an accuracy of 55%. [39] In this case, a more accurate system should be developed for speech recognition by wearable sensors.
Herein, we proposed a high-performance flexible piezoresistive sensor with a simple fabrication process that acts as an intelligent throat to recognize different throat pronunciations. The sensor was fabricated using two polydimethylsiloxane (PDMS) layers that were coated with AgNW and rGO films, whereas the PS microstructures spray coated between these layers. The as-prepared piezoresistive sensor demonstrates excellent sensing performance under the subtle pressure range: high sensitivity (21.8 kPa À1 ), ultralow detection limit (2.1 Pa), quick response time (162 ms), and cycle stability (>5000 times). The excellent properties make the sensor able to detect small physiological signals such as finger bending, eye blinking, and throat movements. Moreover, an intelligent artificial throat has been realized by combining the flexible piezoresistive sensor with an artificial neural network (ANN) algorithm, which not only detects sounds but also classifies the pronunciation of different words. When attaching the sensor to the human throat, it can distinguish the throat vibrations of five different words with a high accuracy of 96%.

Fabrication of Flexible Piezoresistive Sensor
First, two PDMS substrates were prepared by mixing PDMS monomer and curing agent (Sylgard 184) in a weight ratio of 10:1 for 15 min. The mixture was then poured into two plastic Petri dishes. The Petri dishes were placed in a vacuum chamber to remove air bubbles at %1 Â 10 À2 MPa for 30 min. After curing at 70°C for 1 h, the solid PDMS films were peeled off and placed on glass plates.
The second step involved preparing the AgNW layer and two AgNW flexible electrodes. A solution of 2 mg mL À1 AgNW was prepared by adding 1 mL of 20 mg mL À1 AgNW stock solution (Nanjing XFNANO Materials Tech Co., Ltd. Diameter: 50 nm, length: 20-60 μm) to 19 mL of absolute ethanol. The diluted solution was stirred for 30 min to obtain a homogeneous solution. A rectangular 3D-printed mask (15 mm Â 8 mm) was placed on one solid PDMS film, and a 3D-printed mask with two parallel slits was placed on the other PDMS film. The glass plates were placed on a hot plate to reduce the drying time of AgNW solution during spraying, and the temperature was set to 50°C to avoid deformation of the 3D-printed masks. The exposed PDMS areas were sprayed with AgNW solution using an airbrush with a 0.3 mm nozzle. The spraying was repeated every 30 s for a total of 20 times to obtain a uniform AgNW layer. The AgNW/PDMS was then thermally annealed in an oven at 150°C for 30 min.
The AgNW/PMDS film with a rectangular shape worked as the functional layer. Different weight ratios (0.25, 0.025, and 0.0025 wt%) of the PS spheres were prepared by diluting the original 2.5 wt% PS spheres (Tianjin BaseLine ChromTech Research Centre. Diameter: 2, 20, and 40 μm) with absolute ethanol, and magnetic stirred for 30 min. The PS spheres were sprayed on the AgNW layer two times with an interval of 1 min for complete alcohol evaporation.
Meanwhile, a 2 mg mL À1 graphene oxide (GO) solution was prepared by adding 2 mg GO powder into 8 mL of deionized water. The prepared solution was exposed to a 50 W ultrasonic transducer (bisafer250up, 19-25 kHz) for 15 min to obtain a highly dispersed GO solution. The PMDS film with two AgNW electrodes was covered by another rectangular 3D printed mask (15 mm Â 8 mm), and 300 μL GO solution was dropped on the exposed area. This assembly was then placed in an oven at 40°C for 3 h to get a dense film. Next, the GO film was reduced using the laser engraving machine to obtain the rGO film. Finally, we placed the AgNW/PS film on top of the rGO functional layer, and the whole sensor was encapsulated by the polyurethane film.

Characterization
The surface morphology of the sensor was characterized by field emission scanning electron microscopy (SEM) (S-4800 FESEM, Hitachi and JSM-6490LV, JEOL Ltd.). Compression properties were tested under a push and pull tester (HLB, HANPI), and the electrical properties were measured using an LCR meter (KEYSIGHT E4980AL) to trace the real-time resistance value at room temperature. A pressure platform consisting of a signal generating system (33250A, Agilent), a power amplifier (YE5872A, Sinocera Piezotronics, Inc.), and a shaker (YE5872A, Sinocera Piezotronics, Inc.) was used to apply a controllable pressure on the sensor, and a commercial pressure sensor (HP-160, HANDPI) was placed under the sensor to obtain the real-time force.

Sensing Mechanism and Performance Testing
A schematic diagram of the sensor is illustrated in Figure 1a. Briefly, the sensor consists of two conductive layers of AgNW and rGO, with the PS spheres sandwiched between them. The minimalist device structure allows the proposed sensor to have an easy, convenient, and low-cost manufacturing process, including spray coating technology for AgNW and PS spheres (Figure 1c,d) and laser reduction technology for rGO preparation (Figure 1e). The laminar-structured device also demonstrated remarkable flexibility, as shown in Figure 1b, where the sensor was folded at 180°in a bending state without any damage. Young's modulus of the sensor was approximately 1.25 MPa. The fabricated piezoresistive sensor's dimensions were 25 mm Â 15 mm, where the area of the function layer was 15 mm Â 8 mm. The thickness of the sensor was 0.65 mm.
To further investigate the morphology of the fabricated sensor, SEM images were taken after each step of the fabrication process. Figure 1f shows the surface morphology of the AgNW on the PDMS substrate after spray coating 20 times until a dense network of AgNW was observed. The average diameter of the randomly oriented AgNW was about 100 nm. The dense AgNW film demonstrates high conductivity, and the resistivity reached 3 Ω cm À1 . Figure 1g shows a top view of the PS spheres on the AgNW film, where the microspheres were randomly scattered on the surface. Figure 1h shows a cross-sectional view of the rGO film, where the stacked segments of rGO can be seen clearly. In comparison to the GO film, the rGO becomes more porous after laser reduction, and the resistivity is about 400 Ω cm À1 .
To understand the basic working principle of the device, we first prepared the AgNW/rGO device without PS spheres. Figure 2a depicts the sensing mechanism of the piezoresistive sensor without PS spheres. When an external force is applied to the sensor, the AgNW and rGO layers will naturally be in close contact and generate conduction channels. It is worth noting that the resistance of the AgNW network is much smaller than that of rGO, so the current tends to flow through AgNWs, resulting in a significant reduction in the total resistance of the device. However, from the cross-sectional SEM image of the device (Figure 2b), there are some contact points between the AgNW layer and the rGO layer without external pressure. The SEM image of AgNW layer in larger magnification can be found in Figure S1, Supporting Information. These existing conduction points lead to poor response under pressure. To evaluate the sensing performance of the device, an LCR meter was used to record the real-time resistance when an external mechanical force (0-10 kPa) was applied to the sensor. Figure 3b indicates www.advancedsciencenews.com www.advintellsyst.com the resistance change of AgNW/rGO device on various pressures. The relative resistance change of the device is defined as ΔR/R 0 , where R 0 is the resistance of the sensor under no pressure, and ΔR equals the resistance under pressure minus R 0 . In addition, the sensitivity of a pressure sensor is the key parameter to evaluate the performance of a pressure sensor and is described by S = (ΔR/R 0 )/ΔP. In this case, the resistance change of the AgNW/rGO device is about 0.28 under 10 kPa, and the sensitivity is 0.161 kPa À1 in the low-pressure range. The minimum detection limit of the device was also tested by placing different masses. From Figure S2, Supporting Information, the sensor was still sensitive under 20.6 Pa pressure, with a 0.04 change in resistance.
To improve device performance, PS spheres were inserted between the AgNW and rGO layers. The working principle is illustrated in Figure 2c, where the PS spheres act as insulators  www.advancedsciencenews.com www.advintellsyst.com to separate the AgNW layer from the rGO layer in a stress-free state, thus increasing the resistance variation of the device under pressure. This is also verified by SEM images of the piezoresistive sensor with 40 μm PS spheres (Figure 2d). To further investigate the role of the PS spheres, the pressure-response behavior of the piezoresistive sensor with 2, 20, and 40 μm diameters of PS spheres under 10 kPa pressure were investigated by varying the weight ratios, as shown in Figure 3a. At a small weight ratio of 0.0025 wt%, the device with 2 μm PS spheres demonstrated a high resistance variation of more than 0.5, while devices doped with larger spheres (20 and 40 μm) showed a poorer response. This is mainly attributed to the number of 20 and 40 μm spheres being much less than the same weight ratio of 2 μm spheres. Therefore, a very small amount of spheres was sprayed on the AgNW layer at a small weight ratio. The variation in the resistance of the device increases with an increase in the weight ratio. At 0.025 wt%, the resistance change was 0.78 for the device with 40 μm spheres, 0.57 for the device with 20 μm spheres, and 0.7 for the device with 2 μm spheres, which is the best performance for doping the 2 μm spheres. The device with 40 μm spheres at 0.25 wt% achieved the best performance with an average resistance change of around 0.95. Further increasing the weight ratio resulted in a decrease in the resistance change since there were too many spheres between the AgNW and rGO layers, thereby covering almost the entire bottom layer. Therefore, fewer contact points were created between the two conductive layers under external forces. As a result, we observed an optimal doping ratio for different sizes of PS spheres.
To further investigate the pressure sensing performances of the device, Figure 3b shows the dependence of the resistance change of the device with 40 μm spheres at 0.25 wt% on various pressures. The pressure sensitivities were calculated by dividing the resistance response-pressure curves into three linear parts. The AgNW/PS/rGO device was sensitive to a low-pressure range, where the device presented a high sensitivity of 21.8 kPa À1 in the range of 0-33 Pa, which is more than 135 times higher than AgNW/rGO sensor. The pressure sensitivities of the device were 0.2 kPa À1 in the range of 33-833 Pa and 0.007 kPa À1 in the range of 833-10 kPa. The reduced sensitivity can be attributed to the contact between the top layer and the bottom layer gradually approaching saturation in the high-pressure range.
To further investigate the pressure response of the device in the low-pressure range, different pressure was applied on the device. In Figure 3c, the resistance change increased in accordance with an increase in applied pressure, indicating the device is capable of distinguishing between different levels of pressure. It is worth mentioning that the device can also detect ultralight pressure. The resistance dropped by 0.017 when 2.1 Pa pressure was applied on it. Moreover, the device demonstrated a rapid response and recovery time of 162 ms, as shown in Figure 3d. The device also shows excellent durability and stability during long-term operation. As shown in Figure 3e, the change in resistance of the fabricated device remained fairly stable (only dropped by 0.0138) after 5000 cycles of continuous press-release process. More details about the performance comparison of microstructure piezoresistive sensors were given in Table S1, Supporting Information. The distribution of the PS spheres was also observed before and after the test, as shown in Figure S3, Supporting Information. The density of PS spheres on the AgNW layer decreased after the test, while some PS www.advancedsciencenews.com www.advintellsyst.com spheres were observed on the rGO layer. Thus, the PS spheres were not fixed on the AgNW layer. Indeed, they were also attached to the rGO layer when two layers connected by applying pressure.
The hysteresis response of the sensor under the loading and unloading cycle was tested. From Figure S4, Supporting Information, the resistance of the sensor did not change significantly during the press and release cycle, indicating that the piezoresistive sensor demonstrates good stability and reliability. Furthermore, the sensor's performance for changes in ambient temperature and humidity was studied. As shown in Figure S5a, Supporting Information, the resistance change decreased by 0.04 when the temperature increased from 20 to 50°C. As shown in Figure S5b, Supporting Information, when the relative humidity changed by about 30%, the resistance of the developed sensor remained stable. Thus, the temperature and humidity have little influence on the response of the sensor in comparison to the effects of pressure.
The device can also respond to different mechanical forces, such as compression and bending, as illustrated in Figure 4a, b, indicating the multifunctionalities and versatilities of the device. Figure S6, Supporting Information, shows the resistance response of the fabricated sensor after multiple bending. Meantime, benefiting from the remarkable sensitivity and stability of the fabricated AgNW/PS/rGO sensor, it can be used as a wearable device for human motion detection. For example, the device can be easily mounted on different body parts due to its small size, lightweight and excellent flexibility. As shown in Figure 4c, the pressure sensor was placed on the finger to detect various finger-bending movements. Figure 4d also shows the resistance response of the eye blinking. Moreover, different wrist bending angles of 30°, 60°, and 90°were detected, and the resistance change was 0.38, 0.5, and 0.88, respectively. These results further illustrate that the detected waveform positively correlates with muscle contraction intensity. When attaching the device to the human throat (Figure 4f ), different throat movements were distinguishable based on the unique characteristic waveforms. Consequentially, the device shows attractive potential in the field of wearable applications.

Speech Detection and Recognition
In this case, a speech recognition system was developed to distinguish different word pronunciations to help patients with communication. The system contains an ultrasensitive flexible piezoresistive device, a microcontroller (Arduino Nano) with a Bluetooth module and a computer terminal. The device was attached to the throat of a participant to detect muscle movements during different pronunciations. The detected resistance data were transferred to the corresponding voltage readings through a voltage divider circuit. Next, these readings were wirelessly transmitted to the computer terminal for postprocessing.
A total of five words were collected, which were frequently used in health monitoring situations, including two monosyllabic words: "sick" and "help" as well as three disyllabic words "doctor," "patient," and "covid." The detection waveforms of the five words are shown in Figure 5a, as well as the corresponding syllabic peaks of each word. For both monosyllabic words, the valley value of "help" is lower than that of "sick," which is probably attributed to the wider mouth opening when this word is pronounced, resulting in stronger laryngeal muscle stretching and contraction. Meanwhile, for disyllabic words, the waveforms contain two valleys. The first valley of the "covid" and "patient" words www.advancedsciencenews.com www.advintellsyst.com is higher than the second one, while both valleys of "doctor" are almost at the same level. This is because the shape of the mouth changes obviously for both syllables of "doctor" when pronounced. Thus, the associated laryngeal muscle movement becomes more pronounced. For post-data-processing, a large amount of data were required. Therefore, the participant repeated each word 100 times to demonstrate reliability for a total of 500 times. Moreover, 80% of the collected data were used for training, whereas the remaining 20% were used for testing.
To classify the throat movements more precisely, an ANN deep learning algorithm was used and demonstrated good performance. The ANN network was constructed with three hidden layers, and each hidden layer had 10 neurons. More parameter settings of ANN can be found in Table S2, Supporting Information, and the performance plot of the ANN mode is shown in Figure S7, Supporting Information. In addition to the collected signals, features, including mean, minimum, peak-to-peak, variance, percentiles, standard deviation, as well as kurtosis, width, and prominence of each curve, were extracted from the time domain and also supplied to the classifier. To better understand the clustering performance of the dataset, t-distributed stochastic neighbor embedding was used to reduce dimensionality. The visualization results are shown in Figure 5b. Our device demonstrated a classification accuracy reaching 96% for speech recognition. Furthermore, the confusion matrix of the prediction results is shown in Figure 5c. The classifiers precisely separate two monosyllabic words from the others but are sometimes confused by the words "patient" and "covid."

Conclusions
In summary, we proposed a new type of piezoresistive sensor that was fabricated using PS spheres as microstructures that were sandwiched between asymmetric layers of AgNW and rGO. An optimization of the microsphere diameter and weight ratio resulted in an ultrasensitive device in the low-pressure range (21.8 kPa À1 ), which is more than 135 times better than the same device without PS spheres. Moreover, the prepared sensor exhibits fast response and recovery time properties (162 ms), ultralow detection limit (2.1 Pa) and excellent stability (>5000 loading/unloading cycles). These promising performance characteristics enable the sensor to monitor subtle human body activities and work as a wearable intelligent artificial throat to detect throat vibrations during speaking. The device successfully recognizes five different pronunciations by incorporating the deep learning algorithm with an accuracy of 96%, which further demonstrates that it can be a valuable tool that facilitates communication for mute people.

Supporting Information
Supporting Information is available from the Wiley Online Library or from the author.