Robust and Flexible Sliding Tactile Sensor for Surface Pattern Perception and Recognition

Perceiving surface characteristics through tactile interaction typically requires high‐resolution devices or precise spatial scanning to record and analyze a significant amount of information. However, most available tactile sensors require complicated technological processes, redundant layouts, and data acquisition circuits, which limits their ability to provide a real‐time static perception and feedback for potential applications such as robotic manipulation. Drawing inspiration from the sliding tactile (ST) perception mode of the human fingertip, a robust and flexible ST sensor with a low array density of 2.7 cells cm−2 is reported. This innovative sensor has a soft and cambered configuration that allows it to rapidly and accurately recognize the 3D surface features of objects, including grooves as small as 500 μm. Benefiting from the strong correlation between collected electronic responding and local deformation of sensing cell, the ST sensor can adaptively reconstruct surface patterns with the assistance of deep learning, even on unstructured objects. The pattern recognition system based on the robot is demonstrated by accurately classifying a set of mahjong tiles with nearly 100% accuracy, surpassing human tactile perception capabilities in the same task.


Introduction
[7][8][9] By utilizing tactile receptors to perceive the surface features of objects, a human can accurately identify surface defects and differentiate between various types of objects.This enhances their ability to gain a comprehensive understanding of their environment, facilitating decision-making and adaptability.12] Therefore, it is significant to promote a tactile sensor capable of perception and recognition of the surface features of objects.
[15][16] That is, Oh et al. reported a flexible thin-film transistor array with a 100 μm pitch, which achieves pressure distribution images with a high spatiotemporal resolution of 500 μm. [17]However, using sensor arrays to detect edges usually requires a large number of tiny-sized sensor cells to enhance resolution, which leads to the creation of complex acquisition circuits to mitigate potential signal cross-talk issues.These limitations impede the tactile perception of intricate surface patterns by sensor arrays that rely on static force sensing.
Drawing inspiration from the exploratory movements of humans, sliding perception methods offer an alternative approach to perceiving object features.One such approach involves mimicking the sliding motion of a finger across an object's surface using a dynamic tactile sensor.That is, Song et al. developed a flexible triboelectric sensor that utilizes sliding signals to classify textures. [18]Park et al., created an electronic skin based on polyvinylidene fluoride piezoelectric film, which can distinguish surface textures with varying roughness levels through sliding. [19]The dynamic tactile sensor is similar to the fast-adapting mechanoreceptors in human skin, which are particularly responsive to interactive vibrations caused by surface DOI: 10.1002/aisy.202300225 Perceiving surface characteristics through tactile interaction typically requires high-resolution devices or precise spatial scanning to record and analyze a significant amount of information.However, most available tactile sensors require complicated technological processes, redundant layouts, and data acquisition circuits, which limits their ability to provide a real-time static perception and feedback for potential applications such as robotic manipulation.Drawing inspiration from the sliding tactile (ST) perception mode of the human fingertip, a robust and flexible ST sensor with a low array density of 2.7 cells cm À2 is reported.This innovative sensor has a soft and cambered configuration that allows it to rapidly and accurately recognize the 3D surface features of objects, including grooves as small as 500 μm.Benefiting from the strong correlation between collected electronic responding and local deformation of sensing cell, the ST sensor can adaptively reconstruct surface patterns with the assistance of deep learning, even on unstructured objects.The pattern recognition system based on the robot is demonstrated by accurately classifying a set of mahjong tiles with nearly 100% accuracy, surpassing human tactile perception capabilities in the same task.
[21] These sensors are typically designed for high-frequency sensing, but they are susceptible to interference from the unstructured patterns of real-world objects due to their limited static-force-sensing capabilities.
Slow-adapting (SA) mechanoreceptors play a crucial role in object feature sensing during the human perceptual process, especially for surface pattern perception and recognition.Sub-millimeter and millimeter-scale patterns often determine the significative representation of objects.By effectively sliding the fingertips across a surface, the pattern characteristics corresponding to the continuous contact stimuli can be extracted from the neural information of SA mechanoreceptors. [22]Li et al. developed a bioinspired R-skin with a lateral gating-sensing mechanism that demonstrated the feasibility of a sensor slide-sensing strategy. [23]However, it still lacks a self-adaptive recognition process for unstructured patterns and it is difficult to accurately reconstruct 3D geometrical features of objects.
In this study, we propose a robust and flexible sliding tactile (ST) sensor consisting of only a few cells (2.7 cells cm À2 ).The soft and resilient contact medium of the sensor allows it to conform to the surface features of objects, enabling the haptic sensing of sub-millimeter and millimeter-scale patterns.The deformation-induced tactile signal can be used to reconstruct high-definition tactile images in a self-adaptive sliding process of just a few seconds (with a structural similarity index method [SSIM] value of 0.39), mimicking the process of converting temporal scan signals into spatial perception during human touch.
The key geometrical information (such as depth and width) of the surface pattern can be quantitatively predicted by using a convolutional neural network (CNN), even during unstructured sliding on a curved surface.It allows the ST sensor to overcome the limitations of objects or scenes.We believe that the development of ST sensors brings forth substantial cost reduction in E-skin for robots and smart prostheses while simultaneously introducing a more natural and dynamic approach to perception.This simple yet efficient method of object pattern recognition not only advances autonomous decision-making and recognition in robots, but also fosters their continued evolution.

Results and Discussion
A representative ST sensor on a robotic fingertip comprises eight soft-arched cells, as shown in Figure 1a.The porous piezoresistive sensing layer is firmly attached to the interdigitated electrodes of the polyimide (PI) substrate by room-temperature vulcanization (RTV) adhesive.The overall thickness of the sensor is only 2 mm, which is significantly thinner than that of a typical vision-based tactile sensor. [24]The preparation process for the ST sensor is depicted in Figure S1, Supporting Information.Laser ablation and cutting ensure the structural consistency of each cell.The cross-section microstructure of the porous piezoresistive layer was observed using scanning electron microscopy (SEM), as shown in Figure S2, Supporting Information.The soft and resilient texture of the ST sensor is mainly attributed to the various porous architectures ranging from nanometers to microns, allowing it to conform flexibly to objects' surfaces.The finite-element simulation of the sliding perception process of the ST sensor reveals that the sensor's compression and rebound cause voltage changes that directly reflect the surface patterns of objects (Figure 1b).Upon contact with a plane, the sensor compresses, leading to a decrease in resistance.This change in resistance results in an increase of the voltage signal acquired from a typical voltage divider circuit (Figure 1b(i) and S3, Supporting Information).When encountering a groove, the conductive piezoresistive elastomer undergoes a dynamic transition between its compressed and relaxed states, resulting in a dramatic decrease and subsequent increase in the voltage (Figure 1b(ii)).When sliding on a bump, the sensor undergoes further compression, causing a rising voltage signal compared to the initial compressive state (Figure 1b(iii)).
To achieve force and pattern perception precisely, Figure 2a displays the typical resistance-pressure response curve of the ST sensor during the compression test.The piezoresistive sensing layer has a unique hierarchical deformation response with a large sensing range, which has been investigated in our previous work. [25,26]The fitted sensitivity is about 0.44 kPa À1 for the applied sensing range of this study (0-160 kPa).Figure 2b and Note S1, Supporting Information, depict the voltage versus pressure fitting function of eight ST cells, which shows a nearly linear relationship and consistency with respect to the loading pressure (R 2 > 0.97).In dynamic testing, the ST sensor can steadily respond to the external pressure during the loading and unloading cycle with low hysteresis (Figure 2c and S4, Supporting Information).The response and relaxation times under high pressure (670 kPa) are about 38 and 74 ms, respectively, indicating that the sensor can quickly return to its initial state once the external pressure changes.In durability tests, the performance of the ST sensor did not deteriorate even after 20 000 cycles at a pressure of 350 kPa and maintained significant consistency (Figure 2d).In summary, the ST sensor has been demonstrated to be linearly responsive to pressure, resilient, and robust in potential applications requiring contact and sliding perception.
The typical sliding perception process involves the "contacting-scanning-separating" procedure, and its response signal is shown in Figure 2e.When ST sensor makes contact with the object surface, the voltage signal increases to a reference value (V ref ) that is proportional to the magnitude of the contact force.As the sensor slides over a groove pattern, the voltage signal drops abruptly (V drop ), creating a signal trough with a time interval of ΔT.The V drop and ΔT are determined by the depth and width of the groove, as well as the contact force.Meanwhile, we evaluated the shape effect on the sliding suitability of the ST sensor.Figure 2f demonstrates that the arched cell exhibits a higher voltage drop rate (V ref /V drop ) than the planar cell when it slides on concave surfaces, indicating its enhanced sensitivity.Remarkably, the arched cell has a detection limit of 500 μm, demonstrating its ability to accurately sense tiny grooves during sliding.Moreover, Figure 2g shows a minimal detectable sliding distance of 30 μm for the ST sensor, indicating the sensor's superior sensitivity and potential for applications in robotic gripping and slide detection.
Accurately estimating geometrical parameters, such as the width and depth of a pattern, is critical for object perception and recognition.In sliding experiments, a 3D displacement platform was used to exert a sliding motion at a speed of 10 mm s À1 , comparable to the exploring speed of the human finger (Figure S5, Supporting Information).A set of grooves with different widths ranging from 0 to 4.0 mm are employed to calibrate dynamic resolution, as shown in Figure 2h.The plot analysis revealed that the groove width can be estimated by the time interval (ΔT ) when the width exceeds 1 mm.However, the estimation accuracy is constrained by the compression-rebound response time of the porous piezoresistive elastomer.In contrast, the V drop values vary, indicating the varying degrees of rebound of the compressed ST sensor as it slides over the groove.Overall, the ST sensor achieves a width resolution of up to 500 μm, enabling it to extract most details of an object's surface characteristics at a scale similar to human tactile perception. [27]igure 2i shows the relationship between the voltage drop rate and the width and depth of the grooves with a maintained pressure of 160 kPa.The results suggest that the voltage drop rate is proportional to the groove width when groove depth is constant because a wider groove offers more bounce space for the ST sensor.Similarly, when the groove width is held constant, the voltage drop rate is positively correlated with the groove depth.However, when the groove depth exceeds 0.6 mm, there is little change in the voltage drop rate because the groove depth has reached the maximum rebound height of the ST sensor (Figure S6, Supporting Information).
In a real scenario, the signal factors (V ref , V drop , and ΔT ) of the deformation-induced ST sensor are not only responsive to the width and depth of the pattern, but also determined by the initial contact pressure.To reconstruct the essence of the surface pattern, the output voltage signals during sliding can be represented as spatiotemporal grayscale images by mapping.As shown in Figure 3a,b, the tactile grayscale images show the results for different cases.When the groove depth and contact pressure are kept constant, an increase in the groove width leads to wider and darker dark stripes in the tactile image.Similarly, maintaining a constant groove width and contact pressure while increasing the groove depth results in darker streaks in the image.Furthermore, if both the groove width and depth remain fixed, an increase in contact pressure produces a brighter image.A regression model based on a nine-layer CNN was developed to predict three different classes from the tactile images (with separate training for the different classes): the width and depth of the groove, and the pressure applied to the sensor, as seen in Figure 3c.The training dataset comprised the results of 75 crossover experiments, where the ST sensor was slid on each measured surface 20 times, producing a total of 1500 tactile images.Among them, 1200 data were randomly selected as the training set, 200 data as the validation set, and the remaining 100 data as the test set.Figure 3d shows the loss curves of the results of numerous learning epochs.After 20 epochs of learning, the loss curve flattens out.The outcomes of the CNN model for extracting information from the tactile images in the validation set are shown in Figure 3e, indicating that the root-mean-square error between the predicted and actual values is considerably less than 0.15, suggesting that the model is stable in perceiving the features of the input tactile images.To demonstrate this, an ST cell was used to perform a column-by-column single-point scan of the university logo with a radius of 20 mm.The obtained data were then stitched together to obtain a high-density tactile image that recovered the pattern contained on the surface of the object under test with rich texture details, as shown in Figure S7, Supporting Information.The experimental results suggest that the tactile images produced by the proposed sliding ST sensor have the ability for quantitative recognition and reconstruction of the surface pattern features.
Concave patterns on carvings are usually more intricate than periodic textures and often convey meaningful information.In this study, we utilized a representative ST sensor with eight cells to reconstruct the surface pattern of Mahjong tiles, as depicted in Figure 4a.To capture and extract feature information, we propose a dynamic window method.This approach employs a sliding window that sweeps through the data of each sensor channel, as shown in Figure S8, Supporting Information, to capture segments that contain information.The window has a discriminating condition at the beginning and end, respectively, and the data points between them are picked only when both threshold conditions are satisfied.The threshold (θ) of each channel is selected as follow where S i is the signal value of the cell i, and δ is the trigger factor (0-0.2). Figure 4b illustrates the dynamic calibration processing and the results of each step.First, the voltage signal of each cell (Figure 4b(i)) is converted to a more homogenized pressure variation via the pressure-voltage fitting function (Figure 4b(ii)).To reduce the pixel value difference between the row (3000 pixels) and column (8 pixels) directions of the tactile image, a nearest-neighbors method is introduced for 2D interpolation to enhance the column direction from 8 to 400 pixels (Figure 4b(iii)).The resulting tactile images are finally downsampled to 106 Â 217 pixel points, smoothed without compromising data quality, thus reduced in storage requirements.To showcase the efficiency of the ST sensor (2.7 cells cm À2 ), a high-density 8 Â 10 sensor array (6.7 cells cm À2 ) with the same sensing material was used for comparison.In Figure 4c, the tactile images mapped by the 8 Â 10 array and the ST sensor array were compared with the camera image of the mahjong tile.30] The ST sensor performs with a better similarity than the 8 Â 10 array under both evaluation methods (Table S1, Supporting Information).Conventional pixel-based sensor arrays require high array density to achieve high resolution of mapped haptic images, leading to a large number of sensing cells.While some previous studies have reported achieving higher-texture resolution by using a single-micro-sensing unit, these methods are time-consuming when reconstructing tactile images.
33][34][35][36][37][38][39][40] To demonstrate the ST sensor's perception and recognition capabilities for complex patterns, we conducted a classification task using mahjong tiles.We mounted the ST sensor onto the UR5 robot to simulate the sliding process of a human finger, as shown in Figure 5a.The blank experiment showed a high consistency between the sensing signals for the eight cells (Figure S9, Supporting Information).The robotic finger performed 350 scans on each mahjong tile with random speeds, pressures, and starting positions.These data were organized into a dataset of 11 900 grayscale images, with each image labeled as one of 34 mahjong tiles (Figure 5b).The support vector machine (SVM) classifier with the radial basis function kernel was used to classify the 34 types of Mahjong tiles (Figure 5c).The training set consisted of 60% of tactile images, while the remaining 40% were used for validation.The overall classification accuracy was 99.0%, with the lowest accuracy of 98% for Mahjong tiles 7B and C (Figure S10, Supporting Information).These results demonstrate that the reconstructed tactile images possess a high level of information richness and can be precisely classified by the SVM model.To compare the performance of human hands and the robot, we conducted a small competition.The confusion matrix results in Figure 5d,e indicate that the ST sensor has a superior ability to perceive and recognize Mahjong tiles, achieving an accuracy of 97.0% in an average of 6.08 s, which is significantly better than human testers (with an average accuracy of 44.7% in 22.4 s).
The performance of a haptic glove was further tested in challenging scenarios to confirm the self-adaptive ability of the ST sensor for complex objects.As a proof of concept, as shown in Figure 6a, an ST sensor with four channels was mounted on a commercial glove.In contrast to the stability of the robot, the human fingertip is subject to constant variation when sliding, with both the loading force and sliding speed fluctuating.The planar patterns consisted of monograms, cracks, and stripes (containing grooves and bumps), as shown in Figure 6b.By capturing the signal of each channel, four columns of characteristic signals were extracted and transformed into a tactile image with depth information (Figure 6c).The reconstructed tactile images clearly display the contour and gradient features of different patterns, including characters, irregular shapes, and concaveconvex stripes.We even tested the feasibility of ST sensor on the curved patterns (Figure 6d).The tactile images of the surface patterns generated by manual sliding can clearly reflect the texture and edge features on the curved surface, and restructure them accurately compared the actual objects (Figure 6e).

Conclusion
In summary, we have developed a flexible tactile sensor inspired by human-like exploratory movements that can reconstruct and recognize sub-millimeter and millimeter-sized patterns.The contact medium of the sensor is highly conformable and undergoes deformation during the sliding perception process, resulting in piezoresistive sensing signals.These changes in signal intensity, time interval, and voltage drop rate have been shown to be strongly correlated with the 3D geometrical features of unstructured surface patterns.Compared to conventional tactile sensor arrays, our ST sensor, with a low array density, can provide abundant information for reconstructing tactile images with high resolution and low processing time.To demonstrate the superiority of our sensor and image processing approach, we conducted an experiment to recognize and classify 34 Mahjong tiles using deep learning.The results showed almost 100% accuracy, highlighting the effectiveness of our approach.Furthermore, the adaptive capability of the ST sensor enables it to produce effective tactile images even during unstable manual sliding.These results indicate that the ST sensor proposed in this study has significant implications for autonomous tactile identification in smart robots and prostheses.

Experimental Section
Finite-Element Analysis: Finite-element analysis (FEA) was performed using the commercial software Abaqus/Standard 2019.A hyperelastic-Yeoh model is used to represent the nonlinear deformation performance of the ST sensor.The material constants C10, C20, and C30 in the Yeoh model are 0.5738, 0.0747 and 0.0113, respectively, and the incompressibility parameters D1, D2, and D3 are 0.01, 0.1, and 0.5, respectively.
Preparation of ST Sensor: The porous piezoresistive slurry for the ST sensor was prepared by employing NaCl particles as a sacrificial template.All chemicals were used as received without purification.First, thermoplastic polyurethanes (TPU, BASF Co. Ltd.) were dissolved in N, N-Dimethylformamide (Aladdin), and mixed with carbon black (CB, Timcal Inc.) and NaCl (AR, Aladdin) at a mass ratio of 4: 1: 20 (TPU: CB: NaCl) using a planetary vacuum mixer (HM800, HASAI).The slurry was optimized in our previous works to obtain appropriate electrical and mechanical properties. [25,41]The mixture was then spread into a film using blade coating and dried in an oven at 80 °C for 4 h.After curing, the piezoresistive film was immersed in water for 24 h to remove the NaCl particles, leaving a porous structure.The dried porous material was then cut into the desired shape using a fiber laser engraver (Shanghai Diaotu Industrial).For a representative ST sensor, eight individual cells were fixed onto the electrodes of a flexible printed circuit board.RTV adhesive (K-704, Guangdong Hengda New Technology Co., Ltd.) was applied around the electrodes, and the cells were securely pressed into place within the adhesive.
Characterization: The surface micro-morphology of the piezoresistive sensing layer was observed by the field emission SEM (FESEM, TESCAN MAIA3).Mechanical tests were implemented using a universal testing machine (Instron E1000), while resistances were measured by a multiplex digital multimeter (Keithley DMM6500) equipped with a plugin scanner card.A programmable 3D-printing platform with three degrees of freedom was used to realize the sliding perception process.The sensing Convolutional Neural Network Model: The 2D CNN model is composed of three convolutional layers that use forward propagation paths, three pooling layers, and a fully connected layer that functions as a regression.Each convolutional layer is followed by a 2D batch normalization function and a rectified linear unit activation function.The output of the final convolutional layer is flattened into a 1D vector and then passed into a sequence of fully connected layers.To train the entire model, an adaptive moment estimation (Adam) optimizer and mean-squared error loss are employed, with a learning rate of 0.0001 across 100 epochs.
Robot Demonstration: A sliding perception system was performed using a collaborative robot (UR5, Universal Robots) and a 3D-printed finger.Tactile signals were measured and calculated with the use of an national instruments (NI) acquisition card and a personal computer (PC).The position and direction of robot are controlled by the robot-operating system.
Description of the Human Haptic Test: Forty individuals, both male and female, between the ages of 19 and 50, took part in a human haptic test to evaluate their ability to distinguish between patterns on the surface of Mahjong tiles based on their sense of touch.Prior to the experiment, all participants provided informed consent.During the test, the participants were presented with eight Mahjong tile samples randomly, using a blind testing procedure.They were instructed to identify each tile's pattern by running their fingers over its surface and recognizing it by touch.

Figure 1 .
Figure 1.a) Schematic diagram of the human-tactile-sensing mechanism and the sliding sensing process using a representative sliding tactile (ST) sensor.The sensor consists of a curved piezoresistive sensing layer, a room-temperature vulcanization (RTV) adhesive, and a polyimide (PI) substrate with interdigitated electrodes.b) The finite-element simulation and actual voltage signal changes of the ST cell i) when the sensor makes contact with a plane surface; ii) when the sensor slides across a groove pattern; iii) when the sensor slides over a bumpy pattern.

Figure 2 .
Figure 2. Electromechanical performance evaluation of the ST sensor.a) Resistive response of the ST sensor to loading pressure.b) The voltage signals of the eight cells in relation to the loading pressure.c) The ten consecutive cycles of compression loading and unloading curves of the ST sensor at 670 kPa.d) Stability and durability tests of the ST sensor for over 20 000 load-unload cycles at a frequency of 3 Hz.e) Typical signal when the ST sensor is sliding on the groove.f ) Shape effect of the ST sensor on the voltage drop rate and minimum identifiable width.g) The minimum detectable sliding distance of the ST sensor.h) Voltage signal and reconstructed grayscale image of the ST sensor when sliding on a test board.i) The relationship between the voltage drop rate of the ST sensor and the width and depth of the groove.

Figure 3 .
Figure 3.The predicting process and results of the width and depth of the groove as well as the applied pressure.a) Voltage signal plots of the ST sensor sliding on the grooves with different widths and depths.b) The restructured tactile images at a groove depth of 0.6 mm and applied pressure of 121.1 kPa, respectively.c) The convolutional neural network (CNN) regression model built in this study.d) The loss functions of three prediction results.e) The prediction results of the width and depth of the groove as well as the magnitude of the contact pressure.

Figure 4 .
Figure 4.The image processing of pattern recognition by using the ST sensor.Take the "Red Dragon" tile as an example.a) The description of pattern features of the Mahjong tiles.b) The voltage signals and grayscale images of the ST sensor in the image processing of i) sliding perception; ii) dynamic calibration; iii) pixels enhancement and downsampling.c) i) The original 2D grayscale image of the Mahjong tiles and the reconstructed tactile images by using ii) 8 Â 10 sensor array and iii) ST sensor.d) The comparison of the array density and the spatial resolution between the ST sensor and previously reported sensors. θ

Figure 5 .
Figure 5.The recognition and classification of Mahjong tiles.a) The ST sensor is installed at the end of the robot arm to perform a sliding perception.b) The restructured tactile images of 34 Mahjong tiles.c) Schematic diagram of the classification of tactile images.Results of the confusion matrix based on d) the human classification and e) the robot classification.

Figure 6 .
Figure 6.The reconstruction of actual objects by the haptic glove.a) A haptic glove equipped with four ST cells for executing a manual sliding process.b) Planar surface patterns, including monograms, cracks, and stripes.c) The reconstructed tactile images on the planar surface.d) Curved surface patterns, including waves and angular edges on the glass bottles.e) The reconstructed tactile images on the curved surface.