Intelligent Gripper Systems Using Air Gap‐Controlled Bimodal Tactile Sensors for Deformable Object Classification

Multimodal tactile sensors have played an important role in enhancing robot intelligence by providing reliable datasets originating from their high accuracy and durability characteristics. Herein, bimodal tactile sensors capable of simultaneously recognizing the size and stiffness of grasped objects, even deformable ones, are produced. The bimodal tactile sensors are fabricated using the identical process of controlling the air gap between a flexible substrate and the sensing layer, allowing the sensorized gripper to measure pressure and bending characteristics with high accuracy and durability. The pressure sensor yields an excellent durability performance with a negligible change of ΔI/I o < 4.01% even after more than 104 pressing–releasing cycles and broad detection range (5–360 kPa) characteristics. The bending strain sensor also exhibits high sensitivity in a broad bending strain range (0–2.3%) and high durability with a change of ΔI/I o < 4.48% for 104 bending cycles. Using these devices, the sensorized gripper demonstrates that seven tomatoes with different sizes and ripeness states can be classified with high accuracy of 98.78% using an artificial neural network. Finally, the tactile feedback system is expected to be utilized in smart factories, automation systems, and humanoid robots in the near future.


Introduction
Recently, artificial tactile systems (ATS) have emerged in the field of intelligent robotics, [1][2][3][4][5] robotic prosthetics, [6,7] automation systems, [8] wearable devices, [9][10][11] and human-machine interfaces (HMI), [12][13][14] imitating the cognitive process of human tactile reactions which poke, rub, or grasp objects.In the absence of visual information, humans can identify invisible objects only by the movement of their fingers and the tactile sensation of their skin.This has been described as human tactile functions and brain/neurological cognitive learning models that can elaborately recognize the complex features of objects such as size, shape, and stiffness. [15]In particular, when grasping an object, it is necessary to simultaneously detect the tensile sense of the bent fingers and the pressure stimulated by the object.Therefore, ATS requires the development of both multimodal tactile sensors with higher sensitivity and wider detection range and robust artificial neural networks (ANNs) with high accuracy. [5,16,17]any researchers have explored various types of tactile sensors to recognize objects sensitively and precisely, such as piezoresistive, [18][19][20] capacitive, [21,22] triboelectric, [23] and optical [24] sensing methods.Among these sensors, piezoresistive-type sensors have attracted more attention owing to their high sensitivities and long-term stabilities.To achieve extremely high sensitivity, several sensing materials have been developed using micropyramids, [25] micropillars, [26] and microdomes [27] for bending and pressure-sensing applications.However, the pressure detection ranges of these sensors are too narrow to be applied to the robot grippers.For example, a microstructured polydimethylsiloxane-based tactile sensor has been developed with a high sensitivity equal to 19.8 kPa À1 , but can only operate within a small pressure range of 0-0.3 kPa. [28]Since manipulating objects using robot grippers requires a wide detection range of at least 300 kPa, [29,30] simultaneously increasing the sensitivity and detection range of tactile sensors still remains a challenging task.
Furthermore, multimodal tactile sensors have been developed by integrating different types of sensors to recognize and distinguish objects more accurately, simultaneously providing various and complex tactile information.To integrate them to robots' grippers (fingers or hands), tactile sensors should possess multifunctional selectivity.Although several efforts have been made toward the structural designs of sensors to selectively collect various tactile information, [31,32] it has still been necessary to use two or more types of sensors or change the structure using external devices, which leads to complex structures and fabrication processes of the sensors.Therefore, we propose a bimodal tactile sensor that can not only be fabricated with a cost-effective and simple process but can also have high selectivity for pressure and bending strain just by adjusting the air gap structure.
To build an intelligent gripper system using the bimodal sensor, it is important to design a new machine learning (ML) algorithm for classifying objects with high accuracy based on complex sensing information.[38] Among them, ANN is more advantageous for solving complex classification problems with better accuracy and robustness.Recently, several ANN algorithms have shown powerful ability to classify objects or recognize textures by combining various tactile features. [39,40]Despite these advances, a fine-tuned ML model is required to achieve a more accurate and robust ANN.
In this study, we propose an intelligent gripper integrated with bimodal tactile sensors (pressure and bending strain) that can recognize the size and elastic stiffness of objects very accurately, even deformable ones, emulating the cognitive process associated with human fingers or hands.The bimodal tactile sensors were fabricated with cost-effective and simple processes to selectively detect pressure and bending strain by adjusting the air gap structure.As shown in Figure 1a, the intelligent gripper equipped with air gap-controlled (AGC) bimodal sensors recognizes the objects through the pressure stress and bending strain features acquired when grasping the objects.Eventually, the robots can classify tomatoes by their sizes and ripeness levels using bimodal sensing information and ANN algorithm.When the task of choosing high-quality tomatoes is assigned, the sensorized grippers continue to collect time-dependent tactile sensing characteristics and determine their quality according to the pretrained model.Therefore, the intelligent sensorized grippers have great potential in areas such as warehouse management and automated sorting and picking systems, wherein the identification of complicated objects plays a vital role in industrial productivity and quality assurance.

Design of AGC Bimodal Sensors
Figure 1b shows the schematic structure of the proposed AGC bimodal tactile sensor.The device consists of four main layers: a bottom substrate (thickness: 100 μm), interdigitated electrode (IDE) (thickness: 10 μm), an adhesive spacer, and a top sensing layer (thickness: 100 μm).We used polyimide (PI) film as substrate due to its good flexibility.Au/Cu electrodes are printed on the flexible substrate.As shown in Figure S1, Supporting Information, two types of sensors are designed for bimodal detection such as pressure and bending, respectively.As the top sensing layer, a piezoresistive-type Velostat film in which conductive nanoparticles are dispersed inside a porous polymeric matrix was used (Figure S2, Supporting Information).The areas of the active sensing layer are 10 Â 10 mm 2 for pressure and 10 Â 20 mm 2 for bending sensor, respectively.To assemble the entire sensor, the PI adhesive spacer was placed on the edge of the bottom substrate.Then, the top sensing layer was integrated on the spacer, thereby forming an air gap corresponding to the thickness of the spacer between the sensing layer and the substrate.Three different sample cases were prepared with air gaps of 45, 90, and 135 μm to compare the effect of the air gap on the sensing performance.Each case was marked as Sp1, Sp2, and Sp3 by the spacer thickness (Figure S3, Supporting Information) and was controlled by the number of PI spacer layers.

Working Principles and FEA Simulation of AGC Bimodal Sensors
The AGC bimodal sensor causes a change in electrical resistance by the contact of two conductive materials when external stimuli such as pressure or bending are applied. [41,42]Here, since the sensing layer of our AGC sensor is not significantly elongated depending on external deformation, it is reasonable to explain that the electrical resistance changes mainly due to the change in the contact area between the conductive sensing layer and the electrode.The detailed working principles of the AGC sensor can be interpreted as a beam deflection model, [43,44] in which the top sensing layer plays a role as the beam.First, when the AGC sensor is used as a pressure sensor, it can be interpreted by analyzing the change in the contact area between the sensing layer and the electrode according to the external pressure as shown in Figure 1c.When no pressure is applied to the sensor, the sensing layer and electrodes are not in contact due to the naturally formed air gap between them.When the threshold pressure (P th ) is applied, the sensing layer begins to contact the bottom electrodes.P th is the pressure when the maximum deflection of the sensing layer is the same as the air gap (spacer thickness).Then, when a pressure exceeding P th is applied, the contact area further increases, and eventually the resistance decreases due to the expansion of the conductive pathways.As shown in the crosssectional image of the sensing layer in Figure 1c, the contact area is proportional to the contact length (l), and thus the deflection of the sensing layer can be expressed theoretically as follows.
where P is the applied pressure, x is the geometrical position, L is the length, E is Young's modulus, and I is the moment of inertia of the beam (sensing layer), respectively.In this case, we assume that both ends of the beam are simply supported and uniformly distributed pressure is applied (Note S1, Supporting Information).Equation (1) indicates that the deflection of the beam, w sens , is a function of its position x and is directly proportional to the applied pressure level (P).In other words, the deflection profile of the sensing layer increases linearly as the pressure level increases, as illustrated in Figure S4a, Supporting Information.Moreover, as shown in Figure S4b, Supporting Information, the theoretically calculated deflection results (line) were almost identical compared to the deflection results (dots) obtained by finite-element analysis (FEA) at the same pressure (50 kPa).
To further understand the correlation between air gap structure and sensitivity of our AGC pressure sensor, the change in the contact area of the sensor under normal pressure was simulated through the FEA method.Figure 2a shows the deflection of each top sensing layer for three types of AGC sensors with different air gaps (Sp1: 45 μm, Sp2: 90 μm, Sp3: 135 μm).At a weak pressure (50 kPa), the contact areas between the sensing layer and the substrate vary depending on each air gap.Here, the contact length (l) can be approximately calculated by the intercept length between the deflected sensing layer and the substrate for each sample (Figure 2b).The sensor with the smallest air gap (Sp1) showed the longest contact length (l Sp1 ) among the three sensors (l Sp1 > l Sp2 > l Sp3 ) because the sensing layer was located too close to the substrate.Subsequently, a change in the contact area of each sample was simulated according to the applied pressure (Figure 2c,d).As the pressure increases, the contact area increases linearly, and each air gap sample shows different linearity characteristics.As depicted in

Figure 2e
, in the low-pressure range up to 100 kPa, the normalized slope of the contact area was 1.8, 1.3, and 1 for each sensor with Sp1, Sp2, and Sp3, respectively, and the P th values were 5, 20, and 30 kPa.The Sp1 sensor has the highest slope in contact area (Slope Sp1 ) according to the applied pressure and the lowest threshold pressure (P th ), which means that it has the highest sensitivity characteristics as well as the lowest detection limit.However, when a high pressure exceeding 100 kPa is applied, it shows a different tendency.The contact area of the Sp1 sensor already exceeds the electrode area, resulting in electrical calculated through the air gap size, l Sp1 , l Sp2 , and l Sp3 , respectively.c) Contact areas (A) between the sensing layer and electrodes under pressures of 0, 50, 100, and 300 kPa, respectively.d) Contact area variation as a function of applied pressure (0-400 kPa).e) Comparison of normalized slopes of contact areas and detection ranges of the pressure sensor.f ) Comparison of deflection states with Sp1, Sp2, and Sp3 bending sensors while bending strain(ε = 0.5%) is applied by FEA.g) Graphs of w sens of bending sensors with various air gap sizes, representing changes in contact length.h) Contact area between the sensing layer and electrodes at the bending strain of 0%, 0.1%, 0.5%, and 1.5%, respectively.i) Contact area variation as a function of applied bending strain (0-2.3%).j) Comparison of normalized slopes of A and detection ranges of the bending strain sensor.
saturation, whereas the Sp3 sensor shows linearity even at relatively higher pressures.In other words, the Sp3 sensor has relatively lower sensitivity with the normalized slope of 0.1 and higher P th compared to the Sp1 sensor but can operate in a wider detection range up to 360 kPa.As a result, increasing the air gap size changes the contact area of the sensing layer more gradually, allowing the AGC sensor to be designed to detect over a wider pressure range depending on the tasks performed by the gripper.
When the AGC sensor is used as a bending sensor, the sensing layer deforms with the substrate, unlike the pressure sensor.The bending strain sensor undergoes physical deformation of both the sensing layer and the substrate due to its elastic characteristics.Figure 2f,g shows the deflection state and l under bending deformation of the AGC sensor.The bending strain (ε) can be defined as h/r, where r is the bending radius, and h is the distance from the top surface to the neutral plane which is located close to the substrate. [45,46]Under the weak bending strain (0.5%), the value of l Sp1 was the highest, followed by the value of l Sp2 .In addition, the sensing layer of Sp3 did not contact the substrate, so value of l Sp3 could not be measured.Figure 2h,i also shows the change in the contact area of the AGC sensor as the bending strain increases.The contact area increases linearly for all sensors, which tends to be similar to pressure sensors in low-pressure ranges.In the case of the Sp1 sensor, the contact of the sensing layer occurs for a very low bending strain of 0.1%, and the contact area tends to increase rapidly.On the other hand, the sensing layer of the Sp3 sensor does not contact the substrate until the strain reaches 1.5%.Figure 2j shows that the normalized slope of the contact area is 6 (0.05-2%), 2 (0.5-2%), and 1 (1-2%), and threshold strain (ε th ) is 0.05%, 0.5%, and 1% for the bending strain sensor with Sp1, Sp2, and Sp3, respectively.For the bending deformation, it can be seen that the smaller the air gap, the lower ε th , which enables a wider detection range with higher sensitivity.Therefore, as previously mentioned in the AGC pressure sensor, manufacturing the bending sensor with an air gap structure is an effective way to control the desired sensing performance and working range.

Characterization of AGC Bimodal Sensors
After theoretical and simulative analysis, the mechanical properties and electrical responses of these sensors were verified by experiments.Figure 3a depicts a sensing layer that is deformed into a concave shape by the air gap under pressure load.When a force is applied to the surface of the sensing layer, it is deflected downward and comes into contact with the electrodes through the open area.During the pressure test process, the contact surface is limited to a specific area (consistent with the open area) of the sensor.However, since the open area is wider than the electrode area, the higher the pressure, the more contact area covers the entire electrode area.The open area of the pressure sensor is 12 Â 12 mm 2 , as shown in Figure S5, Supporting Information.
In order to accurately analyze the sensitivity characteristics of our AGC pressure sensor, it is necessary to analyze the effect of the resistance change of the conductive Velostat film on the performance of the pressure sensor.We measured the resistance of the Velostat film itself according to the pressure using the top-bottom type measurement method in Figure S6a, Supporting Information and compared it with the total resistance value that changes due to the increase in the contact area with the electrode under the same pressure.As shown in Figure S6b, Supporting Information, the resistance change of the Velostat film was negligible when compared to the total resistance value shown by the Sp3 pressure sensor at a pressure load of 80 kPa, meaning that the change in resistance of the conductive film had little effect on the sensor performance.In addition, the capacitance effect between the conductive Velostat film and the electrodes should also be considered.The initial capacitances (C 0 ) of our AGC sensors are determined by the initial resistances (R 0 ) in the unloading state.This effect caused by the air gap structure is a crucial factor in determining the range of resistance changes, as depicted in Figure S6c, Supporting Information.As the contact area is widened by the pressure (80 kPa), the resistance decreases and the capacitance increases.This result demonstrates that the initial air gap structure affects C 0 and also affects the sensitivity of the sensor, such as the change in resistance with pressure.
Figure 3b shows the relative current change (ΔI=I 0 ) response when applying a pressure range from 0 to 360 kPa, where ΔI and I 0 represent the current change and the initial current in the absence of an external load, respectively.Further, we define the sensitivity of the pressure sensor as S ¼ ðΔI=I 0 Þ=ΔP), where ΔP denotes the pressure change.In the low-pressure region (<100 kPa), the sensitivity of the Sp1, Sp2, and Sp3 sensors was 0.056, 0.04, and 0.03 kPa À1 , respectively, and the P th value was 1, 3.5, and 5.2 kPa, respectively.Here, the P th values are smaller compared to the simulated results, which seems to be due to the gravity that brings the center of the sensing layer closer to the electrode.In addition, these results show that the sensitivity of the Sp1 and Sp2 pressure sensors increased 1.8 and 1.3 times linearly compared to the Sp3 sensor (Figure 3c), respectively, which corresponds well to the simulation results shown in Figure 2e.The Sp3 sensor has a relatively lower sensitivity in the low-pressure region but provides a significantly extended detection range.It indicates that the effective contact area of the sensing layer by pressure gradually increases as the air gap increases, allowing the sensor to detect higher pressures and eventually helping the robot gripper operate over a wide range of detection.
Figure 3d shows the response time characteristics of the Sp3 sensor under pressure.At 80 kPa, the rise and recovery time were about 88 and 12 ms, respectively.However, the 88 ms delay observed when the anvil is pressed, compared to when the anvil is released (12 ms), seems to be mainly due to the systematic slow motion of the pressure sensor evaluation equipment.The pressure load equipment causes a constant response time delay of 85 ms when pressed and 10.5 ms when released, as shown in Figure S7, Supporting Information.Therefore, the correct response time and recovery time for our pressure sensor are 3 and 1.5 ms, respectively, indicating that our pressure sensor can respond quickly enough to be applied to the gripper.These response time delay characteristics of the sensor are due to the elastic properties of the polymer matrix in the Velostat film.In other words, the slow response of the elastic fabric according to the pressure causes a delay in the response time of the pressure sensor.
The mechanical durability of the sensor against repeated pressures is also critical for the gripper to operate reliably for a long time in detecting objects and then providing appropriate grasping feedback.As shown in Figure 3e, the high durability of the AGC pressure sensor was also confirmed after loading >10 000 cycles with a pressure of 200 kPa.The magnified graphs of the beginning and end show no significant decrease in the current output value for the repeated pressure loads, and the change rate in the ΔI=I 0 value of the AGC pressure sensor during 10 000 pressure loads was only 4.01%.As a result, the AGC pressure sensor with such a wide detection range and high durability is suitable for application to intelligent grippers that can provide precise feedback based on sensing information and intelligent algorithms.
Figure 3f shows a schematic structural diagram of the AGC bending strain sensor when it is bent.The open area of the bending strain sensor was 12 Â 24 mm 2 , as shown in Figure S5, Supporting Information.To investigate the effect of the air gap of the sensor on the bending detection performance, each gauge factor (GF = ðΔI=I 0 Þ=εÞ is presented in Figure 3g, where the bending strain (ε) values are calculated, as shown in Figure S8, Supporting Information.The bending radius (r) is calculated using the equation l = 2rsin(l 0 /2r), where l 0 (35 mm) and l are the chord lengths of the sensor before and after bending, respectively.Subsequently, the bending strain (ε) was calculated using the equation ε = h/r, where h denotes the measured value obtained from the bending images.Here, it can be assumed that the neutral plane is located close to the middle of the fabricated sensor structure (in the air gap of the sensor), and we applied this assumption in the equation.As a result, the Sp1 bending sensor has the highest sensitivity (GF = 2.08) and responds in a wide detection range (bending strain < 2.3%), while the Sp3 bending sensor is unresponsive up to 0.3% bending strain and then begins to respond slightly with the lowest sensitivity (GF = 0.15).
In addition, Figure 3h shows that the minimum threshold values are %0.02%,0.25%, and 0.5% for the Sp1, Sp2, and Sp3 bending sensors, respectively.Although the measured ε th values show a slight difference from the simulation values due to the spacer thickness error of the fabricated sensors, it is confirmed that the Sp1 sensor has the lowest detection limit, which is very consistent with the tendency shown in the simulation.This result means that the Sp1 sensor is very sensitive to small bending strains and has a wide detection range.The durability of the bending strain sensor was also investigated through repeated bending tests on the Sp1 sensor (Figure 3i).The change rate in ΔI=I 0 was only 4.48% during 10 000 repetitive cycles with ε of 1.5%, indicating that the AGC bending strain sensor can operate very reliably when used for a long time.
Based on the sensitivity and detection range characteristics of the AGC bimodal sensor as described above, we eventually adopted the Sp3 and Sp1 sensor structures, respectively, as optimized structures for pressure and bending strain sensors applied to intelligent grippers.The AGC bimodal tactile sensor proposed in this article can be optimized by adjusting the air gap to tune the sensor's performance parameters, including sensitivity and detection range, to suit the application of the robot gripper equipped with it.Specifically, for the application of warehouse management and automatic classification (including pick-up systems) using our robot gripper, it is essential that the reliability of the developed sensors be ensured in high-and low-temperature environments.Therefore, we conducted reliability tests, as shown in Figure S9, Supporting Information.The high-humidity storage test was conducted after maintaining the sensor in a chamber at 25 °C with 95% relative humidity (RH) for 24 h, and the low-temperature storage test evaluation was measured after the sensor was left in a À40 °C chamber for 24 h.Then, a pressure load test was conducted using the pressure sensors exposed to the reliability evaluation environment (applied load: 100 kPa).These results show no significant difference in sensor characteristics before and after the reliability test, indicating that our sensor can operate while maintaining its function under challenging environmental conditions such as low temperature and high humidity.

Demonstration of Intelligent Gripper System
To verify that an object can be accurately recognized using the AGC bimodal tactile sensor, a pressure sensor (Sp3 structure) and a bending sensor (Sp1 structure) were mounted on a gripper, as shown in Figure 4a.Given the complex properties of objects, each sensor must extract multiple features to achieve successful grasping-based object cognition.When a moving plate moves to grasp an object by force, the bending sensor records the gap between the two gripper plates, and at the same time, the pressure sensor attached to the fixed plate records the pressure signals generated by the objects.The sensing capability of the sensorized gripper was tested using nine sets of specimens with different sizes (or length, D) and stiffness properties, which were prepared as custom-built cubes.The specimens were produced to have different Young's modulus (E) using PDMS, Ecoflex, and a mixture of the two.We marked the specimens as L, M, and S for size, and as pdms, mixture, ecoflex for stiffness, as shown in Figure S10a, Supporting Information.More detailed information of nine specimens is given in Table S1, Supporting Information, and the deformation of each material under the same pressure is compared in Figure S10b, Supporting Information.The strain-stress data of each material are shown in Figure S10c, Supporting Information.
Figure 4b shows the signals obtained from the pressure (red line) and bending (blue line) sensors when a gripper equipped with the AGC bimodal sensors grasps nine specimens of different lengths and stiffness.The gripper moves to grab the object from its initial state, and after contact with the object, it grasps the object tightly until ΔD is 1 mm, and then slowly returns to its original state.The gripper moved at a constant speed of 1.5 mm s À1 and the initial gap (G o ) between its two plates was 52 mm.It took about 0.66 s for the gripper to move to a position where ΔD was 1 mm after contacting the object.When the gripper approached to grasp an object, the signal of the bending sensor continued to increase, but the pressure sensor did not respond.However, the signals from the pressure and the bending sensors both increase (highlighted by the green region), while the gripper contacts the object and grasps it to ΔD of 1 mm, and when the gripper releases the force to grasp the object, the signals from both sensors begin to decrease.
Comparing three samples (L e , L m , L p ) of the same length as L and different Young's modulus values, the bending sensor responds similarly to the gripper's movement, while the pressure sensor shows remarkably different results depending on each stiffness.As the stiffness of the object increased, the maximum outputs obtained from the pressure sensor gradually increased to 0.28, 1, and 2.45 V, respectively.This tendency of the pressure sensing characteristics is quite similar in other specimens of different sizes.As the stiffness of the object increases, the stress applied to the pressure sensor of the gripper also increases.Therefore, a larger output signal can be obtained from the more rigid object, when the gripper grasps the elastic objects and contracts them by the same length.For the same contraction length, the smaller the specimen size, the greater the output value, which is due to the larger relative contraction rate of the small elastic object, resulting in greater stress.
Meanwhile, as the sizes of the specimens decrease, the maximum output of the bending sensor also gradually increases.While the gripper approaches the object, the bending sensor operates up to a smaller bending radius (or a larger bending strain), resulting in greater linear output characteristics over a longer period of time.Therefore, the AGC bimodal sensor mounted on the gripper can recognize the size of the object through the output signal of the bending sensor while approaching and releasing the object and can also determine the stiffness of the object by detecting the maximum output of the pressure sensor when grasping the object.In other words, the sensorized gripper can provide the capability to distinguish, recognize, and more precisely grasp elastic objects with the help of the AGC bimodal tactile sensors with high sensitivity and a wide detection range.
Based on the experimental results earlier, the ANN model architecture is designed for accurate classification, as shown in Figure 4c.In particular, we derived two main principal components from the descriptions of the waveforms to extract where p(x) is the true distribution and q(x) is the predicted solution.The cross-entropy is minimized as q(x) approaches p(x).We used the Adam optimization algorithm to train the ANN architecture with over 100 epochs and a learning rate (LR) of 0.05.The ANN datasets were trained five times using random 5:5 splits between the training and testing datasets.Figure 4d(i-iii) shows the combination of bending, pressure, and bimodal features based on the PC1 and PC2 results obtained after grasping each of the nine specimens with a gripper for five cycles.The PC1 and PC2 values of the bending sensor illustrated in Figure 4d(i) do not show a significant change by Young's modulus of objects, but vary significantly depending on their size, which corresponds to the results in Figure 4b.On the other hand, according to the results of the pressure sensor shown in Figure 4d(ii), the PC1 and PC2 values change with both the size and stiffness of the object.This can be interpreted through the stress-strain characteristics of the objects shown in Figure S10c and Table S2, Supporting Information.As the object becomes stiffer, the stress applied to the pressure sensor increases, resulting in a gradual increase in the PC1 over time.In addition, as the size of the object decreases, the relative contraction rate increases when grasping an object under the same conditions (L: 2%, M: 2.5%, S: 5%), resulting in greater stress, and thus the PC1 value also gradually increases.Finally, as shown in Figure 4d(iii), the bimodal sensors that acquire signals from both pressure and bending sensors show more clearly significant features of objects recognized by the gripper based on the results in Figure 4d(i-ii).Each PC1 value of the pressure and bending sensor clearly shows features that distinguish the size and stiffness of objects, which show that grippers equipped with the AGC bimodal sensors can recognize objects more accurately than when only pressure or bending sensors are applied.
Based on the designed ANN model, Figure 4e(i-iii) shows the confusion matrices after training with the PC values of the pressure, bending, and bimodal sensors, respectively.Each of these represents a comparison between the predicted value and the true target class, where the cognition rate is calculated as a ratio of correct to incorrect identification.As a result of analyzing the classification accuracy using about 90 sets of sensing data from the AGC bimodal sensors, it was confirmed that the accuracy using bending-only, pressure-only, and bimodal results improved significantly to 51%, 85%, and 100%, respectively (Figure 4f ).
Here, it is necessary to examine in more detail the classification accuracy of the bimodal sensors compared to single sensors.Since the gripper grasps the object by pressing the same distance, the gripper can reasonably classify the stiffness of the objects even with only a single-pressure sensor.However, the sizes of objects cannot be distinguished only by the pressure sensor.In particular, as shown in Table S2, Supporting Information, ecoflex samples with low stiffness show similar stress values regardless of size, and output results of the pressure sensor in Figure 4b completely prove our interpretation.In other words, it is difficult to distinguish whether the increase in the output of the pressure sensor is due to a smaller size or greater stiffness, because the relative contraction rate depends on the size of the object when our sensorized gripper grasps deformable objects.Therefore, both the size and stiffness information of the objects are required to improve the accuracy of object classification, and as shown in Figure 4f, our bimodal sensor demonstrates much higher classification accuracy than just using the pressure sensor.The loss profiles by performing three learning trials for verification are compared as shown in Figure 4g.Combining both pressure and bending features increases the average accuracy and results in a quick zero loss These results highlight the importance of having complementary multifeatured and reliable sensing datasets for accurate tactile learning of object grasping.

Application 1: Tomato Classification Using Intelligent Gripper System
When we choose fruits and vegetables such as tomatoes, we can choose ripe ones by grabbing, touching, and pressing them.Perceiving compliance between size and ripeness is essential for choosing the best quality tomato.To demonstrate the classification of tomatoes more practically, we conducted tests to grasp tomatoes with various properties using the intelligent gripper with the AGC bimodal tactile sensors.Seven tomatoes with different diameters and ripeness states were prepared, as shown in Figure 5a, of which 'T7' was observed to change in five stages (mature green, turning, pink, light red, red) depending on the degree of ripening.The output characteristics of the pressure and bending sensors for each tomato are illustrated in Figure S11, Supporting Information, which are in good agreement with the results shown in Figure 4b.Here, the output values of the pressure sensor for "T1" and "T2" are quite different, because the two are similar in size but different in the degree of ripening.In addition, in the case of "T7", the signals of the bending sensor are similar without significant changes over time (no change in size), while the outputs of the pressure sensor decrease as it ripens.
The 220 datasets were acquired from the AGC bimodal sensors when grasping each tomato ten times using the gripper.Subsequently, each tomato was mapped into 11 different categories by PC features based on its physical size and ripeness (estimated maturity stages), as shown in Figure 5b.Based on the features extracted from the bending strain and pressure information, the confusion matrix yields a high accuracy of 98.78% for tomato classification, as shown in Figure 5c.Compared to the classification results using only bending and pressure shown in Figure S12, Supporting Information, the classification result shows that tomatoes with different sizes and elasticity can still be distinguished with very high accuracy, although it was slightly smaller than the result of the previous specimens due to their spherical shape.To optimize the ANN training model, we adjusted and compared LRs as configurable hyperparameters, which were trained correctly until the proposed model converged.The effects of LR changes are shown in the comparative accuracy and loss results (see Figure 5d,e).To verify the overall availability, we checked the loss profiles depending on the different LRs.As in the conventional appearance of accuracy profiles, [47] it is indicated that an excessively large LR value of 0.1 causes overfitting and an excessively small LR value of 0.001 causes underfitting.As a result, the cross-entropy loss results confirmed that the optimal step size in the proposed ANN architecture was LR of 0.05.In addition, it is also verified that the intelligent gripper with the AGC bimodal sensors and the optimized ANN learning model can distinguish tomatoes of various sizes and ripeness with a high accuracy of 98.78%.
Additionally, it has been verified that our bimodal sensors can also distinguish objects with a wider range of stiffness than tomatoes.In particular, it is necessary to determine whether the Sp3 pressure sensor has sufficient sensitivity to recognize the stiffness of objects over a range of detection.Generally, Young's modulus of tomatoes has quite high-pressure levels ranging from 0.2 to 2 MPa, depending on ripening.For further gripper experiments, we selected objects with lower or higher stiffness than tomatoes, such as a soft sponge ball (%2.5 kPa) and a hard wood ball (% 18 MPa) with the same diameter of 40 mm. Figure S13, Supporting Information, shows that our gripper system with the Sp3 pressure sensor can grasp both more rigid objects and highly deformable objects.Specifically, it is confirmed that the output signal of the Sp3 pressure sensor for the sponge ball was relatively smaller than that for the wood ball.This is because the pressure applied to the sensor is relatively greater when the gripper grasps the hard wood ball.On the other hand, the Sp1 bending sensor shows that the output signal of the soft sponge ball is larger than that of the hard wood ball, despite the same size of the two balls.This is because when the gripper grasps a soft object, it deforms it by a certain stroke (l = 36 mm for the sponge ball).These results indicate that our sensors are not limited to detecting objects with a narrow range of stiffness, but can distinguish objects with significant sufficient sensitivity characteristics over a wide range of detection, from highly deformable to highly rigid objects.

Application 2: Artificial Tactile Feedback System
When using a robot gripper, it is essential to have an optimal grasping function that does not apply less or excessive force to an object without human intervention.To further verify the application of the intelligent gripper developed in this study, a feedback control system was demonstrated to properly grasp objects through real-time tactile information and cognitive learning based on the AGC bimodal sensors.Figure 6a depicts the configuration of the robot gripper where the pressure sensor was mounted on one fingertip and the bending sensor was placed on the TPU connector between the two fingers.The detailed operation principle of the control system using tactile feedback from the AGC bimodal sensors is shown in Figure 6b.
To safely grab and lift up the deformable tomato (D: 31 mm), we performed grasping experiments by adjusting the gap (G) between both gripper fingers, as depicted in Figure 6c. Figure 6d shows the output signals obtained from the bimodal sensors during the gripper's grasping motion for three cases in which weak (G w > 30 mm), proper (G p > 27 mm), and strong (G s > 25 mm) forces are applied respectively.First, the initial gap (G o ) between the two fingers of the gripper was 85 mm (I).Then, while the two fingers approach the object (II), the pressure sensor does not respond, but the output of each bending sensor increases equally in all three cases (the size of tomatoes used was almost the same).Subsequently, when the gripper grasps the object up to the set G values after contacting the object (III), the force applied to each pressure sensor varies, resulting in a significant difference in each signal.The smaller the set G value, the greater the force applied to the pressure sensor, and therefore a larger output signal was observed.In addition, while the gripper tightens the tomato, the output values of the bending sensor also increase, due to the elastic characteristics of the object.The larger the set G value, the smaller the bending strain of the bending sensor and thus the smaller the output signal.Here, we can obtain the thresholds calculated in real time from output signals of the bimodal sensors, which allow us to determine when to lift the tomato using the robot gripper.The combined values (B þ P) obtained from the bimodal sensors when the gripper moves to each set G value were determined as thresholds W, P, and S. As a result, when the combined values of the bimodal sensors are higher than the thresholds (W, P, and S), the gripper fingers stop tightening the object and start lifting it (IV).
The detailed demonstration process of the gripper-based feedback control system is provided in Video S1, Supporting Information.When the gripper lifts the tomato with a too weak grip, it immediately drops it to the floor.On the other hand, when the gripper lifts the tomato with a too strong grip, it creates cracks in the object.Therefore, it is confirmed that the comprehensive tactile information of the AGC bimodal sensors mounted on the gripper plays an important role not only in recognizing the size or stiffness of the tomato but also in providing feedback to grasp it with optimal force depending on the degree of tomato ripening.The sensorized intelligent gripper developed in this study is expected to contribute to increasing industrial production efficiency as well as classifying deformable objects by quality.

Conclusion
In this article, AGC bimodal tactile sensors have been developed that can selectively and sensitively detect pressure and bending by controlling the air gap, and an intelligent robot gripper system equipped with it has also been developed.The AGC bimodal sensors have achieved a wide detection range essential for the application of robot grippers and also verified performance stability in long-term use by showing high durability characteristics in pressure and bending repetitive tests over 10 000 cycles.Furthermore, we demonstrate that the gripper integrated with the AGC bimodal sensors can provide various and complementary tactile information and even classify the size and stiffness of the deformable object.In particular, by implementing ML technology in the sensorized intelligent gripper, it was confirmed that it can provide high-accuracy object recognition, classification, and optimal feedback such as human cognitive-control interaction based on the complementary characteristics of the combined pressure and bending.As a result, the application for classifying tomatoes of different sizes and ripeness showed a classification accuracy of 98.78% based on bimodal functionality.Our bimodal tactile feedback system has tremendous potential to assist robot intelligence, HMI, and automotive systems that make decisions such as fruit quality assessment.

Experimental Section
Preparation of AGC Sensor: First, we adopted PI film as a substrate owing to its high-intrinsic flexibility and straightforward fabrication.The Au/Cu electrodes were patterned on the PI substrate using UV photolithography.Second, double-sided PI tapes used as adhesive spacers were located on the edge of the substrate.The air gaps were formed by the thickness of the spacer.The thickness was controlled by stacking the layers of the membrane-shaped Kapton PI tape.For sensor assembly, a sheet-like piezoresistive material called Velostat (DESCO Industries) was attached to the adhesive spacer.The entire sizes of the AGC bimodal sensors are shown in Figure S1, Supporting Information, and the active areas of the pressure and bending sensors were 10 Â 10 mm 2 and 10 Â 20 mm 2 , respectively.
Preparation of Elastomer Specimens: The polydimethylsiloxane (PDMS) elastomer (Sylgard 184, Dow Corning, MI, USA) was prepared by mixing the base and curing agent with a ratio of 10:1.The ecoflex elastomer (Smooth-on 00-30) was prepared with a 1:1 ratio of part A to part B. The mixture elastomer consisting of a 5:5 ratio of PDMS to ecoflex prepolymer was poured into silicone mold.
Characterization: The 3D surface morphologies of the AGC sensors produced differently depending on the number of spacers were observed using a confocal and interferometry microscope at a magnification of 20Â, as shown in Figure S3, Supporting Information.The cross-sectional geometric images of the piezoresistive film in Figure S2, Supporting Information, were characterized by a field-emission scanning electron microscope (FESEM, S-4800, Hitachi).In the pressure sensing test, the pressure sensor was measured by load-cell pressure-applying equipment with a force gauge (PI V-275.431, with a force range of 0.01 N-10 N and a resolution of 0.001 N (0.01 gf )).In the bending strain sensing test, a step motor (PK-245-01B, MISUMI, Schaumburg, IL, USA) with 0.01 mm resolution and motion stage (TS-150-M, NAMIL Inc.) was used to measure the degree of bending.The degree of bending was determined by angle analysis from the real-time video.The stress-strain properties were measured at room temperature using a universal testing machine (UTM, WL2100, WITHLAB).A crosshead speed of 1.3 mm min À1 was used and ASTM D 5467 was utilized for standard specimen with the test coupon being the face sheet.
FEA Simulation: FEA simulations based on COMSOL Multiphysics (COMSOL Inc., Burlington, MA, USA) were performed to simulate the structural mechanics for the contact effect of the flexible AGC sensor designs using geometric nonlinearity models.
Data Acquisition: The resistance range of our sensors varied from %20 MΩ to 3 kΩ in the range from 0.1 to 10 N based on measurements recorded by digital phosphor oscilloscope (DPO 4054B, Tektronix, Pennsylvania, PA, USA).The output voltage signals were acquired using a low-noise current preamplifier (SR570, Stanford Research Systems).To guarantee the functionality of the data acquisition device, we evaluated the AGC sensors using a voltage divider.
Control of Robot Arm and Gripper: A robot arm and gripper (MANDRO Inc.) were used to verify the applicability of our sensors, as shown in Figure 6a and Video S1, Supporting Information.For an artificial tactile feedback system with AGC sensors and a robot gripper, a microcontroller-based Arduino Uno platform and a wireless communication-based-nRF24L01 transmitter module (2.4 GHz) were used for laptop-based real-time monitoring.The multichannel electrical signals from our sensors were acquired through an oscilloscope (Analog discovery2 USB, Digilent Inc.).
Reliability Test in High-Humidity and Low-Temperature Environments: High-humidity storage test was conducted at 25 °C with 95% RH.The test was carried out by a low-temperature freezer system (Operon Inc., DFU).Low-temperature storage test was performed at À40 °C for 24 h by the temperature and humidity chamber (ESPEC Inc.SU).

Figure 1 .
Figure 1.a) Concept of the sensorized gripper used to classify tomatoes on a conveyor belt.The robot recognizes the deformable objects by monitoring the stress and strain through the AGC bimodal sensor.Artificial neural network algorithms are used to classify precisely the properties of tomatoes, such as their sizes and stiffness.b) Layer-by-layer schematic design of the AGC sensor.c) The schematic cross-sectional images of the working mechanism of an AGC pressure sensor under unloading, threshold pressure loads, and high-pressure loads.The red line illustrates the contact length between the sensing layer and electrodes.

Figure 2 .
Figure 2. Configuration and FEA simulation of AGC sensors.a) Comparison of deflection states with Sp1(air gap of 45 μm), Sp2(air gap of 90 μm), and Sp3(air gap of 135 μm) pressure sensors while 50 kPa is applied.The deflection of the sensing layer (w sens ) is simulated by FEA.b) Contact lengths (l)calculated through the air gap size, l Sp1 , l Sp2 , and l Sp3 , respectively.c) Contact areas (A) between the sensing layer and electrodes under pressures of 0, 50, 100, and 300 kPa, respectively.d) Contact area variation as a function of applied pressure (0-400 kPa).e) Comparison of normalized slopes of contact areas and detection ranges of the pressure sensor.f ) Comparison of deflection states with Sp1, Sp2, and Sp3 bending sensors while bending strain(ε = 0.5%) is applied by FEA.g) Graphs of w sens of bending sensors with various air gap sizes, representing changes in contact length.h) Contact area between the sensing layer and electrodes at the bending strain of 0%, 0.1%, 0.5%, and 1.5%, respectively.i) Contact area variation as a function of applied bending strain (0-2.3%).j) Comparison of normalized slopes of A and detection ranges of the bending strain sensor.

Figure 3 .
Figure 3. Electromechanical characteristics.a) Schematic diagram of AGC sensor under pressure load.b) Relative current changes of Sp1, Sp2, and Sp3 pressure sensors in the range from 0 to 360 kPa.c) Comparison of pressure-sensing performance with respect to sensitivity and detection range.d) Enlarged curve of the response and recovery time with Sp3 pressure sensor at 80 kPa.e) 10 000 cycle durability result of the Sp3 pressure sensor under a pressure of 200 kPa.f ) Schematic diagram of AGC sensor under bending strain.g) Relative current changes of Sp1, Sp2, and Sp3 bending sensors in the bending strain range from 0% to 2.3%.h) Comparison of bending sensing performance with respect to GF and detection range.i) 10 000 cycle durability result of the Sp1 bending strain sensor at strain ranges from 0% to 1.5%.

Figure 4 .
Figure 4. Classification for multifeatured bimodal sensing system.a) Illustration of a gripper with the AGC bimodal sensors grasping an object.b) Experimental results of pressure and bending sensors for nine sets of specimens, respectively.c) Procedure of ML based on neural network model with preprocessed data (maximum and standard deviation values) as input.The model classifies one of the nine output labels as an output.d) Principal component analysis (PCA) results extracted by pressure and bending sensors: (i) bending, (ii) pressure, and (iii) bimodal features.Confusion matrices for nine sets of specimens using e) (i) only bending, (ii) only pressure, and (iii) bimodal (combining bending and pressure) sensor's output signals.f ) Average classification accuracy of the bending, pressure, and bimodal sensors using neural networks.Error bars denote the standard deviation.g) Comparison of training loss using pressure, bending, and bimodal sensing results.

Figure 5 .
Figure 5. Application of the tomato classification on sensorized gripper.a) Photographs of the tomatoes with different sizes and stiffness.b) PC mapping results extracted by the bimodal (pressure and bending) sensors for various tomatoes.c) Confusion matrix of tomatoes classification accuracy (98.78%) through ML and combined output signals from pressure and bending sensors.d) Average classification accuracy with the bimodal sensors depending on the learning rate.e) Comparison of training loss among neural network models.

Figure 6 .
Figure 6.Artificial tactile feedback system.a) Photographs of the fully assembled artificial tactile feedback system with AGC bimodal sensors (Sp3 pressure and Sp1 bending sensors).b) Algorithm diagrams of the tactile feedback closed-loop control system with bending (B), pressure (P), and both (B þ P) signals.c) Schematic illustration of the movement of the gripper fingers in four different states: initial (I), approach (II), grip (III), and lift up (IV).d) The real-time output signals in step-by-step mode when the robot fingers are controlled to grip the tomato up to different G values.The images in step IV represent the state of the tomato when the gripper grasped and lifted it weakly, properly, and strongly, respectively.