Robust triboelectric information-mat enhanced by multi-modality deep learning for smart home

In metaverse, a digital-twin smart home is a vital platform for immersive communication between the physical and virtual world. Triboelectric nanogenerators (TENGs) sensors contribute substantially to providing smart-home monitoring. However, TENG deployment is hindered by its unstable output under environment changes. Herein, we develop a digital-twin smart home using a robust all-TENG based information mat (InfoMat), which consists of an in-home mat array and an entry mat. The interdigital electrodes design allows environment-insensitive ratiometric readout from the mat array to cancel the commonly experienced environmental variations. Arbitrary position sensing is also


| INTRODUCTION
Denoted as the next-generation Internet, the metaverse is attracting enormous and unprecedented attention than ever before, especially after Mark Zuckerberg announced to rebrand Facebook as Meta. 1,2The concept of the metaverse is essentially a merging of virtual reality (VR), augmented reality (AR), and physical reality, and the metaverse blurs the boundary between the interactions online and in real life. 3,46][7][8] Meanwhile, digital twin is another advancing technology developed for the connection between physical and virtual space, where a virtual representation is created as the real-time digital counterpart of a physical object.3][14][15] When incorporating digital twins in the metaverse, the duplicated digital counterparts of the physical reality can be displayed in VR and AR.If further combined with the powerful artificial intelligence (AI) based cloud computing, more immersive, comprehensive, and intelligent interactions between metaverse users are about to be achieved.Moreover, the visualization of digital twin will be better realized, thereby strengthening the connection between the virtual world and the real world for more effective communication and optimization.It is of great potential that incorporating digital twin in metaverse will accelerate the pace for developing smart homes, smart industry, and smart cities in the era of the Internet of Things (IoT).
7][18][19] With the rapid advancement of artificial intelligence of things (AIoT) technology, analysis of the acquired sensing data helps enhance connections among sensing units and processing units. 20,21A more intelligent, precise, and efficient sensor/cloud communication system is enabled so as to accurately project the real room status into a digital world in real time.Sensors like temperature, humidity, optical, motion, gas sensors, and so on, are all included in the built-up of digital-twin smart homes for information collection.Currently, most of them are based on resistive and capacitive sensing mechanisms, which rely on constant power supplies. 22,23In the AIoT system, if billions of sensors are deployed, it will lead to a total sum of extremely high power consumption, and nontrivial effort will be spent in charging or replacing the batteries.In this regard, self-powered sensors that consume no external power are desired to bypass these issues. 244][45][46][47][48][49] Compared with the conventional camera surveillances that require video recording for monitoring, TENG sensors can obtain real-time interactive sensing information related to human behaviors without any privacy concern. 50,51][54][55][56][57][58][59] Toward this direction, Ma et al. have demonstrated a TENG-based self-powered smart floor with a minimalist design for human motion monitoring and walking trajectory tracking in 2017. 60Jumping, running, walking, and squatting can be distinguished from the signals generated by the proposed mat.The walking trajectory can be monitored via the output electrode terminals when six mats with six electrodes are connected to the readout terminals.Afterward, Shi et al. have introduced a TENG-based mat with reduced electrode readout terminals when parallelly connecting the mat pixels to form only one electrode terminal. 61The functionality of walking trajectory monitoring is maintained via the varying magnitudes of output in the parallelly connected mat pixels.Identity recognition is also realized by using deep learning (DL) to analyze the generated triboelectric signals that carry unique gait information during walking processes.Meanwhile, a smart home scenario is demonstrated where the light switch is controlled by the signals from a designated pixel from the mat, and the door access security is enhanced by implementing DL on mat signals to retrieve user identities.
Although the TENG-based mats show great potential as self-powered sensors for digital-twin smart home applications in identity recognition, walking trajectory monitoring, activity monitoring, and other IoT/AIoT related applications, there are limitations associated with the intrinsic property of sensors based on triboelectric mechanism.One major limitation is that the magnitudes of output voltages are affected by varying environmental conditions, such as the humidity changes, which may result in the inaccuracy of command for those applications relying on exact magnitude values. 62,63Currently, two main strategies are employed to solve the humidity influence.One is to fully-package TENG devices to avoid the entering of water vapor, 64,65 and the other is to utilize the hydrophobic or electron-trapping materials for fabrication, 66,67 whereas both may limit the material choices and cause fabrication complexity.To meet the demand, Shi et al. have proposed a new strategy for the readout of mat sensors that takes the ratio of the output through electrode designs to bypass the environmental influence. 68Since all the electrodes are exposed to the same environmental condition, the output from every electrode is varied to the same extent, resulting in the unchanged output ratios.Yet, the designed electrodes for each mat pixel only cover the middle part and the remaining part cannot realize the position sensing function accurately.During the walking trajectory monitoring, the mat can only work with the walking manner of one-step-one-pixel.While for normal walking, it is highly possible that the users walk on the mat arbitrarily and may cover two pixels in one step.In this regard, there is also much space for the improvement of the smart mat toward more user-friendly applications in smart homes.Besides, as the DL analysis is explored to be integrated with TENG monitoring system, 69 identity recognition and user authentication have been realized successfully by extracting the unique features from signals generated during humans walking on the smart mat. 61,68DL analysis has enabled the recognition beyond the conventional gait features like magnitude, duration, frequency, phase, and so on. 46,70,713][74][75] So far, in terms of the smart mat monitoring systems, the gait information is only generated from one sensory channel and the output abundance is not fully presented, which may lead to limited classification accuracies.
Herein, a robust triboelectric mat monitoring system endowed with abundant sensory information is proposed and termed as the information-mat (InfoMat).The InfoMat contains a large-scale in-home environmentinsensitive mat array (four sets of mats) for arbitrary position sensing, walking trajectory monitoring and sports/ activity monitoring, and a two-channel entry mat with multi-modality sensory information for DL analysis to enhance identity recognition accuracy.In the in-home mat array, each set consists of six parallelly-connected pixels with diverse interdigital electrodes (IDEs) patterns where the width ratio of their two finger electrodes varies.Taking the output ratio of two IDE for each pixel enables the monitoring to be independent of environmental influence or stepping manners, because the changes are experienced by all the electrode fingers and hence the resulted fluctuations are canceled out when the ratio is taken.Owing to this advantage, the accuracy and stability of position sensing or walking trajectory monitoring are highly improved for smart home applications.By introducing four sets of the mats into the system where adjacent pixels are from different mat sets, eight electrode terminals are created.Arbitrary stepping position sensing can be achieved by taking readout terminals and output ratios together into account, which implies that the stepping position sensing is developing from single-pixel sensing to multiple-pixel sensing.Furthermore, a hierarchical weight sensor with a large continuous linear sensing range for human weighing is introduced beneath a one-electrode mat to form the entry mat for identity recognition.Using DL-assisted analysis, the classification accuracy is enhanced by multi-modality analysis compared with the results from the two channels separately.
Benefiting from these advanced improvements of the InfoMat, a digital-twin smart home is constructed via projecting the real home space into the digital world and visualizing it in VR space.Interactive signals captured from the InfoMat are analyzed by time-domain analysis and multi-modality DL analysis.Two digital-twin smart home enabled applications are developed, including the accumulated and separate skipping counting of two users simultaneously, and the customized two-way interaction (VR space-real space) yoga guidance.Correspondingly, the indoor statuses of users like access authorizations, standing positions, walking trajectories as well as dynamic sports/activities are duplicated successfully into VR in real time.Overall, with the support of AIoT system, the developed digital-twin smart home anticipates the promising future of metaverse toward wider and more immersive communication between the world, including online shopping, remote meeting, and remote education, and so on (Figure 1).

| Design and characterization of the environment-insensitive mat array
The large-scale, environment-insensitive in-home mat array is constructed from four sets of mats and each set contains six pixels, resulting in a total of 24 (4 Â 6) pixels.Here we take one set of mat for illustration and characterization first.As shown in Figure 2A, the mat pixel is a sandwiched structure with poly(ethylene terephthalate) (PET) as the triboelectric material, Silver (Ag) paste as the electrode material and polyvinyl chlorids (PVC) as the substrate.However, the electrode designs of the six pixels have varying IDE ratios.The screen printing technique is employed here to guarantee the precision and uniformity of the six diverse IDE patterns as well as the scalability of the devices.Figures 2B and S1 show that the width ratio of the two IDE fingers varies from each pixel in one set, that is, 10:0, 8:2, 6:4, 4:6, 2:8, and 0:10, respectively.Notably, red color and gray color are utilized to differentiate two IDE fingers, but they both represent silver electrodes.Since the output of TENG is proportional to the electrode area under the same triboelectric area, the triboelectric output ratio of the two IDE fingers in each pixel is supposed to be consistent with the IDE width ratio, thus providing the way to distinguish the six pixels.
To characterize one set of mat, six pixels with varying IDE finger width ratios are connected in parallel where all the E1 from each pixel are connected and all the E2 are connected, thus only two electrode terminals are required in the end, as shown in Figure S1A.It should be noted that although the finger width ratios vary from each pixel, the sum of two IDE finger areas of each pixel keeps constant.The detailed size of each pixel can be found in Figure S1B. Figure 2C shows the triboelectric output voltages of the six pixels from two electrode terminals (E1 and E2) when the user wearing polytetrafluoroethylene (PTFE) sole shoes frequently steps on and off the six individual pixels, respectively.The stepping position is controlled almost in the middle of each pixel and the stepping force is controlled almost the same manually.Correspondingly, the output ratios of the E1/E2 for six pixels are calculated and presented in Figure 2D.Here we take the ratios of the negative peaks which are generated via the step-on motion.The six ratios are distinguishable, which are 85.3 for pixel-1, 4.02 for pixel-2, 1.49 for pixel-3, 0.679 for pixel-4, 0.268 for pixel-5, and 0.00535 for pixel-6.It is obvious that the calculated ratios of pixel-2, pixel-3, pixel-4, and pixel 5 are relatively consistent with the finger width ratios, that is, 4, 1.5, 0.66, and 0.25.But the ratios for pixel-1 and pixel-6 varies a lot from width ratios which should be infinity and 0, respectively.These variations can be attributed to the triboelectric output generated from edge electrodes for both pixels.Specifically, although the width of finger electrode is 0 and no output is generated, there are still triboelectric outputs from edge electrodes of these finger electrodes.Luckily, the variations caused by the edge electrodes show negligible impact for sensing results as the calculated ratios are distinguishable enough between six pixels.To verify the environment-insensitivity of this mat set, the triboelectric output ratios of the six pixels under different circumstances are investigated in Figure 2E-J, including various sole materials, ambient humidity, user weights, stepping areas, gait directions, and stepping positions.All the corresponding triboelectric outputs are provided in Figure S2.Sole materials including ethylenevinyl acetate (EVA), PTFE, and fluorinated ethylene propylene (FEP) are used for comparison as photographed in Figure S3A.With various sole materials, the output ratios of E1/E2 keep almost constant for each pixel and can be distinguished easily between pixels without any confusing concern (Figures 2E and S2A).Humidity is always a great concern to TENGs because the output varies a lot with humidity change.This has hindered their adaptability and commercialization.However, although the outputs decrease with the increasing humidity for all six pixels, the output ratios still keep constant in each pixel and are also distinguishable enough between pixels (Figures 2F and S2B).Besides, applied forces and contact areas also influence the output of TENG devices.The eliminations of both factors are explored.It can be seen that when the user (50 kg) holds objects with increasing weight, the triboelectric output increases accordingly while the ratios keep unchanged and are distinguishable for six pixels (Figures 2G and S2C).The same phenomenon appears when the stepping areas increase (Figures 2H and S2D).Photographs of the sole with three sizes are displayed in Figure S3B.Allowing for practical applications, users may step on a mat arbitrarily without paying attention to stepping in the middle of a pixel.In such case, different stepping directions and stepping positions on each pixel are investigated as shown in Figure S3C,D, respectively.The results show that the six pixels can be successfully distinguished from each other regardless of the stepping directions (Figures 2I and S2E) or stepping positions (Figures 2J and S2F).Therefore, the ratios of each pixel are extremely stable and distinguishable under changing environments and users' stepping manners, contributing to environment-insensitive and user-friendly An overview of a digital-twin smart home enabled by the robust triboelectric information-mat (InfoMat), including the twochannel entry mat (containing one-electrode mat and hierarchical mat), interdigital electrode (IDE) mat array in the device level, multimodality deep learning (DL), and time-domain analytics in data analytics level, as well as the applications in virtual reality (VR).The InfoMat is offering great potential to create a more intelligent, immersive and effective communication interface between the physical and virtual world for diverse applications in Metaverse, such as virtual shopping, virtual education, virtual meeting, and so on mats.Notably, pixel-1 and pixel-6 have slightly different output ratios from the width ratios, owing to the influence of the edge electrode areas.Also, the grounding connection may result in some variations of the ratio values.Taking these inevitable issues into account, we have set the threshold ratio values around these exact ratios in Figure 2K for more precise pixel differentiation and AIoT applications in the digital-twin smart home.

| Unique designs and mat arrangement schemes enabled arbitrary position sensing and walking trajectory monitoring
Taking advantage of the stability and environmentinsensitivity of these mats, we have explored the application for position sensing and walking trajectory monitoring.First, the user walks in a one-step-one-pixel manner along one mat set, which contains six different pixels connected in parallel with two output electrode terminals, as shown in Figure S4A.From the generated output in Figure S4B and the calculated ratios in Figure S4C, we can see that only the ratio of stepping on the first pixel complies with the ideal ratio as we defined (>6.5) while the other ratios resulted from the following steps deviate.The reason is that for normal walking, the step-on motion on the next pixel is supposed to happen simultaneously with a slight step-off motion on the current pixel, which implies when the right foot of the user steps on pixel-2, the left foot will step off pixel-1 at the same time.All the respective E1 and E2 of the six pixels are connected together, and the step-on motions lead to negative peaks on both E1 and E2 while the step-off motion leads to positive peaks.In this regard, as the two motions happen simultaneously, the generated negative and positive signals will overlap and the resultant final peaks from the two terminals will be different compared with pure stepon or step-off motions.Deviations of the output values consequently result in the deviations of the calculated ratios and the position sensing are not accurate, as illustrated in Figure S4C.To address this issue, an interval arrangement scheme is adopted in Figure 3A, where two sets of mats are employed (set A and Set B denoted as blue and pink, respectively).The adjacent pixels are from different sets.With such a connection scheme, there become four output electrode terminals in total, marked as E1A, E2A, and E1B, E2B. Figure 3B provides the generated signals of both sets when the user walks along the intervalarranged mat array.The resultant ratios in Figure 3C show consistent ratio values within the defined range for every pixel, which are more stable and accurate for walking trajectory monitoring.No overlapping of positive and negative signals exists in this mat arrangement scheme, because despite the step-on and step-off still happen at the same time (like stepping on B-2 while stepping off from A-1), they generate signals on different mat sets (like negative peaks on E1B and E2B while positive peaks on E1A and E2A, respectively) and the electrode terminals of Set A and Set B are separated without any influence to each other.Meanwhile, for one set of mat here, the step-off motion from the pixel can be completely carried out when the following step-on motion on the next pixel happens (like stepping off from A-1 completely finished and then stepping on A-3) so there is also no overlapping of positive and negative signals in the same mat set.Hence, the onestep-one-pixel walking trajectory monitoring can be accurately and stably achieved via this interval-arranged scheme.
Furthermore, it should be more convenient and acceptable if the users can walk arbitrarily on the mat array without sticking to the one-step-one-pixel manner, where one step may cover two pixels.To investigate the feasibility, we utilize the interval-arranged two sets of mats for characterization first.The user continuously steps on and off five positions separately where all steps cover two adjacent pixels from different mat sets, as shown in Figure 3D.The generated triboelectric outputs of the five stepping positions are exhibited in Figure 3E, following the stepping sequence of A-1 and B-2 together, B-2 and A-3 together, A-3 and B-4 together, B-4 and A-5 together, and A-5 and B-6 together.Then we take the ratios of the corresponding negative peaks as E1A/E2A and E1B/E2B, respectively.According to Figure 3F, the calculated ratios for the five positions are all within our defined ratio ranges, indicating the feasibility of pixel distinguishment and position sensing when covering two pixels in one step.In Figure S4D, we have applied this mat arrangement scheme for normal walking trajectory monitoring where we intend to make every step cover two pixels at the same time.Detailly speaking, the first step begins with the left foot stepping on A-6 and B-5 together, and the next step is the right foot on A-1 and B-2, followed by the next foot stepping on A-3 and B-4.It should be noted that the second step-on motion on A-1 and B-2 happens together with a step-off motion from A-6 and B-5.Similarly, the step-on motion on A-3 and B-4 comes with a step-off motion from A-1 and B-2.Thus, due to the simultaneous step-on and step-off motions in the same mat set, overlapping of negative peaks and positive peaks will derivate the output signals as well as the resultant ratios.The generated signals during this walking trajectory are provided in Figure S4E and the calculated ratios are shown in Figure S4F.There is no doubt that the ratios are mismatched with the defined ratio ranges.To overcome this problem, we need to ensure the pixels that covered by the consecutive two steps are from different sets of mats, so that four sets of mats with eight electrode terminals are introduced in Figure 3G (set A, B, C, and D are denoted as blue, pink, green, and yellow color, respectively).The user walks along the mat array following the sequence of stepping on A-2 and C-3 first, then stepping on B-2 and D-3 while slightly stepping off from A-2 and C-3 together, and finally stepping on A-5 and C-6 while slight stepping off from B-2 and D-3 together.Figure 3H illustrates the generated triboelectric output signals of the four sets of mats with eight output electrode terminals.Taking the ratios of the negative peaks of each mat set regarding the walking sequence, we can find that the calculated ratios in Figure 3I are all within the defined ranges of the ideal ratios, indicating the feasibility of this mat array arrangement for arbitrary walking trajectory monitoring.The reason is the same as the one-step-one-pixel walking in Figure 3A, where although the step-on and step-off motions happen at the same time, they generate signals in different mat sets that have no mutual influence.By considering the electrode terminals and output ratio values together, we have successfully proposed a mat array that can be applied in the arbitrary position and walking trajectory monitoring, enabling the user-friendly, and true-to-life applications.
Therefore, following the four mat sets arrangement, there are supposed to be 24 pixels in total (six pixels in one set) which can be scaled up into a large-scale mat array for smart home applications.Figure S5A shows the arrangement and electrode connections of the 24 pixels for four sets of mats.The screen-printed mat is of A4 size so that we arranged them in a 4 Â 6 array to obtain a near-square shape which is more similar to the floor in the real room.The photograph of the InfoMat with a 4 Â 6 array in the real space is shown in Figure S5B.The width of each pixel is shorter than the length and it is of higher possibility that walking in the horizontal direction may cover two pixels in one step.So we arrange every horizontal line with four sets of mats where any four consecutive pixels are from different sets.This allows the simultaneous step-on and step-off motion happens on different mat sets, and the generated signals are readout from corresponding electrode terminals without overlapping.For the vertical line, the pixels are arranged in an interval manner where two sets of mats are utilized, assuming one step only covers one pixel.This arrangement also enables inclined walking because every two adjacent diagonal pixels are from different sets.Figure 4A indicates the diagonal walking trajectory on the large-scale mat array from the bottom-left pixel (A-2) to the up-right pixel (D-2), during which one step covers either one pixel or two pixels, and the walking sequence is A-2, D-4 and C-3, A-1, D-1 and C-2, B-1 and A-3, and D-2. Figure 4B presents the signals generated from the eight electrode terminals of the four sets of mats.Observing the negative peaks which are generated from the step-on motions, the signals can be divided into six time domains representing the six step-on motions.Figure 4C shows the calculated ratios of the negative peaks for each mat set in the six time domains, and they all match well with the defined ratios.Likewise, five other inclined walking trajectories (i.e., from A-2 to B-5 [T1], A-2 to D-6 [T2], A-2 to A-6 [T3], A-2 to C-4 [T4], and A-2 to B-1 [T5]) are also explored, the generated signals and the calculated ratios are shown in Figure S6.
Based on the investigation of arbitrary walking on the large-scale mat array, here we have realized the walking trajectory monitoring in a real-time VR smart home scenario via the signals triggered from the InfoMat.The real-time control system involves the large-scale mat array as signal-generating node, the signal collection and processing node, the personal computer (PC), and the virtual space in the Unity software.In detail, as the user walks on the mat array, the self-generated signals will be collected and processed by the microcontroller unit (MCU) in real-time so that the avatar in VR will respond accordingly.We have programmed that only the negative peaks are considered, and the ratios are taken by E1/E2 for each set of mat.Then, by considering the sets of mats and the ratios, the avatar walks on the corresponding pixel in VR space.If only one set is detected, the avatar walks exactly on the pixel.Furthermore, if two sets are detected, the avatar walks between the two pixels from the two sets.As illustrated in Figure 4D, the user is walking on the mat arrays following the sequence of (i) A-2, (ii) B-3, (iii) A-4, (iv) B-5, (v) D-6 and A-6, (vi) C-4 and B-1, (vii) D-2, (viii) D-2, (ix) A-3, (x) C-2, and (xi) B-2.The real-time collected outputs are plotted in Figure 4E accordingly.Relying on calculated ratios and the corresponding mat sets in time-domain sequence, the avatar in VR walks exactly on the pixels corresponding to the pixels in real space as shown in Figure 4F.The related video is provided in Movie S1.

| Investigation of hierarchical weight sensor and multi-modality DL analysis for enhanced identity recognition
To enhance the functionality of the InfoMat, we have introduced the weight sensor for healthcare applications because people nowadays pay much attention to reaching and maintaining a healthy weight which can help lower the risk of heart disease, stroke, diabetes, high blood pressure, and so on.The TENG-based weight sensors have been explored extensively as self-powered sensors and the generated output is highly dependent on the applied force.However, the sensing range of those reported weight sensors is very limited, especially saturating below the common human weight range (40-80 kg).Herein, we have applied the contact and separation mode TENG with the elastomer and aluminium (Al) as the triboelectric materials, and Al as the electrode material.It is reported that triboelectric layers with patterned structures (such as pyramid, hemi-sphere, pillar, etc.) can increase triboelectric contacting areas when the device is fully compressed, thereby effectively enhance the output performance.7][78][79][80][81][82] In order to find the optimal structure and elastomer for negative triboelectric materials, we have investigated three hierarchical structures of the triboelectric layer, including pyramid, sphere, pyramid and sphere mixed structure as shown in Figure 5A, as well as three elastomers including Ecoflex-50, skin-like elastomer, and silicone rubber with decreasing softness, respectively.The sizes of the elastomer pieces are all 3 Â 4 cm 2 , the length/height of the pyramid is 1 cm/0.5 cm, and the diameter of the sphere is 1 cm. Figure 5B depicts the curves of the transferred charges under varying applied force from 0 to 750 N regarding different hierarchical structures and elastomers.Here transferred charge is chosen as the output parameter because the entire contact and separation process can be reflected in the curve of the transferred charge while it is hard to obtain the entire information using short-circuit current due to the transient charge flows.Although open-circuit voltage can maintain the full information as well but due to the testing limitations, some output voltages that are beyond the measuring range of the Keithley 6514 are unable to be recorded.It shows that the TENG with skinlike elastomer possesses the highest output compared to the Ecoflex 50 and silicone rubber-based TENGs, since the skin-like elastomer is the softest one and can has a denser contact with the aluminium foil.Meanwhile, the pyramid structure shows the highest output and sensitivity compared to the other two structures for all elastomers, because the pyramid structure helps increase the areas of the elastomers and it is easy to be pressed compared to the sphere structure.Due to the toughness in pressing, devices with sphere structures are endowed with a higher sensing range compared to the pyramid one.Thus, by combining pyramid and sphere together into one piece, a higher sensitivity and a larger continuous linear sensing range are supposed to be obtained at the same time despite some compromise.On the one hand, observing the three curves of soft skin-like elastomer-based TENG, the sensing ranges of the pyramid one are 0-30 N, 30-120 N, 120-180 N, and 180-450 N while the sphere one covers the force range up to 570 N. The mixed structure has a sensing range of up to 540 N, larger than that of pyramid one but the sensitivity is compromised.On the other hand, for the tougher elastomer (Ecoflex 50), the pyramid one shows the highest output and the continuous linear sensing ranges are around 0-30 N, 30-120 N, 120-270 N, 270-420 N followed by the saturation while the sphere one exhibits continuous linear sensing ranges of 0-30 N, 30-540 N. Apparently, the pyramid and sphere mixed TENG has a higher sensitivity than the sphere one, and the continuous sensing range is extended compared to the pyramid one with 0-120 N and 120-420 N, 420-540 N. Thus, both continuous linear range and sensing range are improved in the mixed structure.Notably, regarding the silicone rubber-based TENGs, all the three structures exhibit the most continuous linearity and cover the largest sensing range of up to 750 N even though the output is relatively low.
Furthermore, we have explored the output performances of connecting the above TENG units parallelly to obtain the most suitable device structure for human weight sensing to match the large-scale mat design and smart home applications.Considering the output magnitude, the sensitivity, and the linear sensing range, the TENG units including skin-pyramid TENG, skin-pyramid and sphere mixed TENG, and silicone rubber-pyramid TENG are investigated.Accordingly, Figure 5C-E depict their transferred charge curves with the increasing number of units from 1 to 4. Theoretically, the sensing range should be extended as well which is proportional to the number of units.Detailly, four units are supposed to have a sensing range that is four times the range of 1 unit.In general, the tested results are consistent with the theoretical value and the sensing ranges of all the three kinds of devices cover more than 750 N. Yet, the continuous linearity of silicone rubber-pyramid TENG is more superior to the other two.Hence, in the following experiment and application, the silicone rubber-pyramid TENG structure with four units is applied.Notably, the device with two units is capable of covering the human weight range, but in order to obtain a more steady and balanced weight sensing mat, four parallel-connected units are more suitable here when arranged on the four corners, as shown in Figure 5F.The weight sensing mat  Skin-pyramid and sphere a Tra Tra ra a a a a a a a a a a a Tra a a a a a a a a a a Tra a a ra a a a a a a a ra a a a a a a a a a a a a a a a a a a a a a a a a ra a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a ra a a a a ra a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a ra ra r r r r r r r r r r r r r r r r r r  Cha Cha a a a a a a Cha a a a a a a a a a a a a a a a a a Cha a Cha a a a a a a a a a Cha a a a C a a a a a a a a a a a a a a a a Cha a Cha Cha a a a Cha a a a a a a ha ha Cha a a a a Cha a a a Cha h ha a a a a Cha a a Cha Cha ha ha ha a a a Cha Cha Cha a a a a Cha a a a a a a a a a a a a a a Cha a a a a a a a a a a a a a a a a a Cha a a Cha Cha a a a a a a a Cha a ge e e e e e e e g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g g ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( d d d d d d d d d d d d d d d d d d d d d d d d d ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( C C  is designed to have a similar size to that of one pixel in the mat array.Figure 5G illustrates the weighing results of the user (50 kg) holding the increasing weight of objects by adding 3 kg per time.The transferred charge increases almost linearly with the increasing weight of objects.For more practical applications in the AIoT system, the weight sensor is required to be connected with certain data collection and processing units, thus here we connect the weight sensor to MCU.Although only shortcircuit voltage would be recorded when connecting to MCU, integration of corresponding current and time is taken which can be treated as the transferred charge.

d Ch Ch Ch Ch h h h h Ch h h h h h h h h h h h h h h h h C Ch
Figure 5H shows the obtained results of 10 users with different weights, indicating that the 10 weights can be distinguished easily from each other.The detailed weight information of the 10 users in relation to the corresponding output intensity is plotted in Figure S7, and it shows a very linear fitting curve with an R-square of over 0.99.The linear sensing range of this proposed weight sensor has the largest continuous range compared to the state-of-the-art weight sensors based on TENG mechanism.A comparison table is provided in Table S1.
Benefitting from the rapid advancement in AI technology, a more intelligent world is blooming.AI helps process large amounts of data much faster and make predictions more accurately than human beings because it gives enterprising insights into the details that may not be aware of previously.4][85] Herein, aiming at constructing a digital-twin smart home with novel functionality of identity recognition and access authorization, DL is integrated to analyze the gait information generated from a two-channel entry mat.This is because gait information is treated as an effective and unique identifying characteristic.Figure 6A shows a typical step of normal walking on the entry mat including stepping on, standing still and stepping off process.Here, the entry mat is composed of a top one-electrode mat attached to a bottom hierarchical weight sensing mat (Figure 6B) where the top one-electrode mat is fabricated with the same materials as the mat array.Thereby, the generated signals during one step are obtained from two output channels separately, that is, the top one-electrode mat and the bottom hierarchical mat.In this work, 10 users have participated in data collection, and they all follow the one-step process depending on their own manners.Both signals from the top one-electrode mat and the bottom hierarchical mat are collected by the two analog-todigital converter (ADC) channels from a MCU, and the generated signals in one step are recorded in real time in terms of two separate channels, as shown in Figure 6C.The plotted patterns of the 10 users are supposed to include important features of their gaits by analyzing the amplitude, the time interval, and so on.However, it is relatively puzzling for human beings to extract the unique features from some slight variance and make predictions let alone in high accuracy or efficiency.In order to build up the dataset for DL, the 10 users have all repeated the one-step walking motion for 50 cycles with 1600 data points collected from each channel per cycle.Finally, 50 samples have been collected for every user in each channel which are then randomly divided into training set (80%) and testing set (20%).The DL analysis model proposed here is based on a convolutional neuronetwork (CNN), consisting of an input layer, four convolutional layers, four max-pooling layers, and a fully connected layer for outputting the predicted identification result, as described in Figure 6D.In terms of the input layer, initially, the samples from channel 1 and channel 2 enter separately, and the classification accuracy is 93% and 94% of 10 users, respectively in identity recognition.Figure 6E,F shows the corresponding confusion maps of the two channels.Although they both possess relatively good accuracies, improved accuracy can be achieved when the multi-modality sensory information including both channels enter the input layer together.Generally, the classification accuracy of the trained model is supposed to be enhanced with the increasing number of samples (50 for each channel and 100 for the multi-modality situation).In our entry mat design, the output patterns from channel 1 are more dependent on the contact areas between the sole and the mat during one step while channel 2 depends more on how the stepping force is applied during this step.We fuse the two different kinds of sensor signals in data-level feature.Because the two channels from MCU are collect into same format, we directly combine the two-channel signal into the input data with size of 3200 (2 Â 1600) data points for each channel per cycle.By entering such multi-modality sensory information into the training model, relationships between the two different channels are supposed to be considered, and more features of gait information may be extracted as well, leading to higher accuracy for identity recognition.It is proven that the classification accuracy for multi-modality analysis reaches 99% as shown in Figure 6G, which is a 5% and 6% accuracy improvement compared with mere channel 1 and channel 2, respectively.We have further employed different DL approaches for outputting the predicted identification results, including k-nearest neighbors (kNN) (k = 1), kNN (k = 10), support machine vector (SVM), multilayer perceptron (MLP), recurrent neural network (RNN), and long short-term memory (LSTM), as shown in Table S2.Obviously, CNN get the highest test accuracy of 99%, thereby it is the most suitable DL approach for further demonstration of the Info-Mat.Although identity recognition has been extensively explored in smart homes either using triboelectric mats or triboelectric socks, a feasible strategy to further improve the classification accuracy has been effectively proposed here.

| The InfoMat enabled digital-twin smart home application
To better visualize the multi-functionality of the InfoMat, a digital-twin smart home is developed via the interactions between real space and VR space.Figure 7A describes the working schemes of this system which enables four functionalities, that is, identity recognition, weight sensing, position sensing, and activity monitoring.The first two functionalities are based on the entry mat that outputs multi-modality sensory information with two kinds of triboelectric signals.One is from the top one-electrode mat and the other is from the bottom hierarchical mat.The multi-modality signals are collected and fused together in real time, followed by being sent to the DL trained model for identity prediction.The signals from the bottom mat are processed for real-time weight sensing.Likewise, when the individual walks on the large-scale mat array, the generated triboelectric signals are collected and processed by the MCU in real time for position sensing as well as activity monitoring which can be reflected in VR space.In the application of skipping in a digital-twin smart home, the triboelectric signals generated during this whole process are plotted in Figure 7B with respect to the two output terminals from the entry mat and the eight output terminals for the four-set mat array (10 output channels in total).Initially, User 1 steps on the entry mat for identity recognition (signals shown in light blue frame in one-electrode mat and hierarchical mat), in which case the corresponding avatar appears in VR space indicating successful identity recognition (Figure 7C).Next, User 1 walks on the in-home mat array to the targeted positions for triggering the skipping command in VR space (Figure 7D,E for User 2) because the two feet of the user jump up and down at the same time when skipping.Here only positive peaks from one mat set are taken into account for accumulating although positive peaks are generated simultaneously in Set B and D (or Set A and C), respectively.Hence, in this digital-twin smart home application, it is obvious that User 1 skips five times and User 2 skips four times.The signals are shown in blue and red shadowed frames, and the final accumulated counting number in Figure 7I is consistent with signals for both users, showing the perfect real-time interaction between the real space and VR space.The corresponding video can be found in Movie S2.It should be mentioned that the avatars in VR space do not turn their bodies in the same direction as the users in real space until the skipping begins.This is because the last negative peaks during the walking trajectory are for skipping triggering command and no further peak is generated for the body-turning command.However, during the skipping progress, the VR avatars are designed to skip in the same direction as the users in real space.
Furthermore, in the digital-twin smart home, we have also developed a two-way interaction enabled customized yoga guidance between real and VR space based on the proposed InfoMat.Detailly speaking, "customized" means each individual is assigned with their own yoga poses through the result of the identity recognition from the entry mat. Figure 8A illustrates the working flow of the two-way interaction between real space and VR space.Initially, the signals generated from the entry mat in real space are used for identity recognition, followed by the determined customized yoga poses for this user shown in VR space.Then the user follows the yoga guidance in real space and the generated signals from the contact and separation processes are collected and processed, which are visualized on the marking of corresponding pixels in VR space to indicate the correct poses.Figure 8B-D exhibits the corresponding photographs in VR and real space during the two-way interaction yoga guidance process.Following the guidance in VR space "Please go to the red mat for identification," the user in real space steps on the entry mat and the generated signals can be processed and enter DL trained model, then the identification result shows in VR space (Hi, YYQ!Your weight is 40 kg.) together with the assigned yoga poses.Thereafter, the user moves to the central position as guided in VR space.Once she reaches B-4, the generated signals will trigger the command of yoga.The first yoga pose is displayed in VR space and the corresponding pixels which are contacted by the avatar during Pose 1 are marked in red.By doing the same yoga pose in real space, the user steps on the two pixels in sequence (D-1, A-4), then taps on the third pixel (C-2), and finally returns to the original position (B-4).The signals generated in doing Pose 1 are collected and processed, leading to light-on of the green color for the corresponding pixels in sequence in VR space.Notably, the green light only lights up when the correct signal is detected for the corresponding pixel, thus Pose 1 is successfully done here in real space.When Pose 1 is done, the avatar in the VR space continues with Pose 2 where three pixels are marked in red one by one.After watching the guidance in VR space, the user begins to do Pose 2 in real space and these motions generate signals on D-1, A-4, and B-4 in sequence, which cause the light-up of green light in VR space one by one.Afterward, Pose 3 contains two steps as shown in VR space with light-on of red color in sequence.Following Pose 3 in real space, signals are generated from D-1 and B-4 and the timedomain signals are used to control the light-up of green color in VR space, which indicates the correct yoga pose.Moreover, the customized yoga guidance is realized for another user who is assigned three different yoga poses via identification.The corresponding videos of these two users for customized yoga guidance in this digital-twin smart home can be found in Movie S3.

| CONCLUSION
In summary, we have proposed a robust TENG based InfoMat, consisting an environment-insensitive in-home mat array via unique IDE design and a two-channel entry mat with multi-modality sensory information in device level.Integrating time-domain analysis and multimodality DL in data analytics level, together with VR technology, a robust InfoMat has been developed and enabled a digital-twin smart home.Specifically, owing to the unique electrode design of IDE and the resulted ratiometric readout method, the in-home large-scale mat array can achieve accurate position sensing and walking trajectory monitoring.It has successfully overcome the intrinsic limitations of TENG devices by eliminating the influence of environment condition changes and stepping manners.Meanwhile, by introducing four sets of mats, arbitrary walking trajectories can be precisely detected even when two pixels are covered during one step.This is a great advance toward practical applications compared with the one-step-one-pixel manner in previous matrelated works.In terms of the entry mat, the bottom hierarchical weight sensing mat is optimized to possess a large continuous linear sensing range covering 0-75 kg.When integrated with DL for identity recognition, the multi-modality DL analysis of two channels (bottom hierarchical mat and upper one-electrode mat) has successfully enabled accuracy improvement from 93%/94% for one respective channel to 99% with multi-modality analysis.Hence, the proposed strategies in device and data analytics level for robustness enhancement can be strong references for the development of TENG devices and DL analysis.
Enabled by the InfoMat, a digital-twin smart home is developed via duplicating the real-time status of physical home space to digital world, regarding the user access authorization, position, walking trajectory, dynamic activities, and so on.The signals generated from the InfoMat are analyzed using DL or time-domain analysis for the construction of a digital counterpart that is visualized in VR simultaneously.On the one hand, the skipping application in digital-twin smart home shows the feasibility of monitoring two users on the InfoMat at the same time.On the other hand, the application of customized yoga guidance is enabled via the two-way interaction between the real space and VR space, which offers a private and engaging mode for fitness training.The developed digital-twin smart home is good evidence for the technical convergence of digital-twin, AI, and VR.Therefore, it is of great potential that a more comprehensive, efficient and immersive environment is going to be established in the Metaverse to benefit the whole society regarding living, working, and learning.

| Fabrication of the IDE-based mat
This mat is a single-electrode TENG composed of three layers: PET as the positive triboelectric layer, Ag paste as the electrode layer, and PVC as the supporting layer.Initially, the PET film with a thickness of 125 μm is cut into A4 size and then one side of the film is pretreated with primer to enhance the adhesion with later electrode F I G U R E 8 Application of the two-way interaction (real space and virtual space) enabled customized yoga training in a digital-twin smart home.(A) The schematic illustration of the information flow between real-space and virtual reality (VR) space.(B) The customized yoga guidance in VR space, where the "customized" is enabled by the deep learning (DL)-assisted identity recognition from the entry mat.And the three poses are indicated on pixels of the mat array with red color.(C) The following of yoga poses in real space and the generated signals from the mat array.(D) The real-time indication of correct poses in VR space with the corresponding pixels marked in green, which are controlled by the signals from real space printing.Subsequently, Ag paste is screen-printed on the pretreated side via the pre-designed masks, followed by curing at 130 C in the thermal oven.Three masks are utilized in this screen-printing process to achieve the six IDE designs where the pixel with 10:0 becomes 0:10 with 180 rotation, so as 8:2 and 2:8, 6:4 and 4:6.The Ag paster is obtained with a thickness of about 15 μm.Next, the PET-Ag paster film is cold-laminated with an 80-μmthick PVC film serving as the supporting substrate and the encapsulation layer, where Ag paste electrode is in between the PET and PVC films.Meanwhile, the copper wires are connected on the two edge electrodes of the IDE mats by peeling off a small corner of PVC and exposing the two electrodes, which are then be sealed with double-side adhesive Kapton tape.Finally, through batch fabrication, four sets of mats are obtained, where each set contains six pixels with six IDE designs for further mat array investigation.

| Fabrication of the hierarchical mat
This mat is a contact/separation mode TENG with silicone rubber as the negative triboelectric layer, Al as the positive triboelectric layer and at the same time Al serving as the electrode layer for both sides.First, the liquid silicone rubber is mixed with curing agent universally (100:2 in weight ratio) and then poured into the preprepared 3D mold which has the pyramid (sphere or pyramid-sphere) structures.After curing at room temperature for 5 h, the hierarchical piece is peeled off from the mold with the size of 4 Â 6 cm 2 .Subsequently, the Al film is cut into four pieces with the dimension of 4 Â 6 cm 2 .The four Al films are attached to the four corners of the wood substrate, followed by attaching the four hierarchical pieces onto each Al film.Similarly, another four Al films of 4 Â 6 cm 2 are attached to the wood plate and the two wood plates are placed up and down with the latter Al facing hierarchical piece.Finally, to stabilize the device during contact and separation process, four holes are drilled in the four corners with four cylinders fitted in.

| Characterization of the triboelectric output
The output voltage of IDE mat array is captured and recorded by an oscilloscope (Agilent DSO-X3034A) with the impedance of 1 MΩ.The transferred charge of hierarchical mat is measured by the Electrometer (Keithley 6514) and the signals are displayed and recorded by the oscilloscope (Agilent DSO-X3034A).The varying force applied on the hierarchical mat is conducted by using a force gauge (Mecmesin, Multi-Test 2.5-i).

| Data collection and processing of DL and VR interaction
The analog voltage signals generated from the IDE mat array and the two-channel hierarchical entry mat are collected and processed by the microcontroller (Arduino MEGA 2560) in real time.The CNN model is developed in Python with a Keras and TensorFlow backend.In terms of digital-twin smart home VR interaction, the processed signals are sent to the VR scenario which is developed based on 3D Unity for controlling.

F
I G U R E 2 Legend on next page.

F
I G U R E 2 The design and characterizations of one mat set to verify the property of environment-insensitivity. (A) The structural illustration of a single mat pixel.(B) Detailed diagrams of the six interdigital electrode (IDE) patterns with different width ratios of IDE fingers in one mat set: 10:0, 8:2, 6:4, 4:6, 2:8, 0:10.(C) Triboelectric outputs of six individual pixels regarding two output electrodes.(D) The corresponding output ratios of the six individual pixels (E1/E2).(E) The output ratios of the six individual pixels when contacted with different materials (ethylene-vinyl acetate [EVA], polytetrafluoroethylene [PTFE], and fluorinated ethylene propylene [FEP]).(F) The output ratios of the six individual pixels under varying humidity (58%, 70%, and 83%).(G) The output ratios of the six individual pixels under varying contact weight (50, 53, 56, and 59 kg).(H) The output ratios of the six individual pixels with varying contact areas (large, medium, and small).(I) The output ratios of the six individual pixels with different stepping gait directions (vertical, horizontal, and inclined).(J) The output ratios of the six individual pixels from different stepping positions (middle, upper left, upper right, bottom right, and bottom left).(K) The defined threshold ratio values for the six pixels F I G U R E 3 Investigation of mat arrangement to enable arbitrary position sensing and walking trajectory monitoring.(A) The schematic illustration of interval arrangement of two sets of mats where each set is in parallel connection for one-step-one pixel walking.(B) The triboelectric output and (C) the corresponding output ratios when the individual walks along the mat array with a manner of one step one pixel.(D) The schematic illustration of stepping in-between two pixels on the interval arranged mat array with one step covering two pixels.(E) The triboelectric output of the two sets (four electrodes) and (F) the corresponding output ratios when the individual steps on the five positions.(G) The schematic illustration of the interval arrangement of four sets of mats for arbitrary walking monitoring.(H) The triboelectric output of the four sets (eight electrodes) and (I) the corresponding output ratios when the individual walks along the mat array with a manner of one step two pixels

F
I G U R E 4 Four sets of mats enabled arbitrary walking trajectory monitoring.(A) The schematic illustration of the arrangement of the four sets of mats and the included walking trajectory from bottom-left pixel to upper-right pixel.(B) The triboelectric output and (C) the corresponding output ratios during this walking trajectory.Demonstration of arbitrary walking in virtual reality (VR) space with real-time signals generated from the real-space mat array.(D) Photographs of the walking process in real space.(E) The triboelectric output generated in four sets (eight electrodes) during this walking trajectory.(F) The corresponding walking status in VR space Tr Tr Tr r r Tr r r r r r r Tr Tr Tr Tr r Tr Tr Tr Tr r Tr Tr T T T nsf nsf f f s s ns s s s s s ns s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s ns s s s s s s s s s s s s ns s s s s s ns s s s s ns s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s s ns s n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n n err e r r r err r err err r err err rr rr rr rr rr rr rr rr rr err r r r r r r r r r r r err rr err rr rr err err rr err rr rr rr err e er r r r r r r r er er r r er r r r r r e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e ed d d ed d edd d d d d d d d d d ed d d d d d d d d d d d d d d d d ed d d d d d d d d d d d d d d d d d ed d d d d d d d ed d d d ed d d d d ed d d d ed ed d d d ed d d d d d d d d d d d d d d d ed d d d d d d d d d d d d d d d d d d d d d d d d d d d d ed d d d ed d d d d d d d d d dd ed d d d d e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e Cha a a a a a ha a Cha a ha h h C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C C rr r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r rge e ge e e e ge e e e e e e e e ge e ge e e e e e e e ge ge e ge ge e e e e e e e e e e e e ge e ge e e e ge ge e e e e e e e e ge e ge e e e e e e e ge e e e e ge ge e e ge e e e e e e ge e e e ge e e e ge e e e e e e e e e ge e e e e e ge e e e e e e e e ge ge ge ge ge e e e e e e e e e e e e ge ge ge e e e ge e e e e e

5
foil silicone rubber Transferred charge (nC) Investigation of triboelectric nanogenerator (TENG) mat with hierarchical structures for large-range weight sensing.(A) The schematic illustration of the three kinds of hierarchical structures including pyramid, sphere, and sphere and pyramid mixed.(B) The transferred charges of TENG with different hierarchical structures and elastomer materials under varying applied forces from 0 to 750 N. (C)-(E) The transferred charges for TENGs with increasing number of parallel-connected hierarchical units from one to four, regarding (C) skin-pyramid unit, (D) skin-sphere and pyramid mixed unit, (E) silicone rubber-pyramid unit.(F) The configuration of the weight sensing mat containing four silicone rubber-pyramid units.(G) The relationship between the obtained transferred charge and the individual (50 kg) holding varying weights of objects from 3 to 12 kg.(H) The measured result of 10 users with different weights when connecting the sensing mat to microcontroller unit (MCU)

F I G U R E 6
Deep learning assisted identity recognition via entry mat.(A) The schematic illustration of a normal walking step including stepping on the entry mat, standing still and stepping off.(B) The configuration of the entry mat and (C) the corresponding outputs from the two channels during the stepping on, standing still, and stepping off processes for 10 users.(D) The structural illustration of the convolutional neural network (CNN) training model.(E)-(G) The confusion maps of the prediction with signals from (E) channel 1 only, (F) channel 2 only, (G) multi-modality analysis of channel 1 and channel 2 F I G U R E 7 Application of the private skipping in digital-twin smart home based on the InfoMat in virtual reality (VR).(A) The overview of the proposed system involved the signals flow for identity recognition, weight sensing, arbitrary position monitoring and activity monitoring.(B) The generated triboelectric signals of two users when they enter the smart gym, walk to the skipping positions, stand still one by one and then skip together.(C) User 1 walks on the entry mat for identification and the appearance of a man avatar indicating successful recognition.(D) Walking trajectory of the man avatar in VR space which is controlled by the corresponding signals generated by User 1 in real space.(E) The skipping command triggered in VR space when D-2 and B-1 are detected in sequence.(F) User 2 walks on the entry mat for identification and the appearance of a woman avatar indicating successful recognition.(G) Walking trajectory of the woman avatar in VR space.(H) The skipping command triggered in VR space when C-2 and A-1 are detected in sequence.(I) Separate and one-byone skipping counting in VR space when the two users skipped on their triggered pixels.The insets are the photographs of the users in real space ).The signals generated during this walking trajectory are marked in the dark green frame, and the two inner light green frames indicate the triggered skipping command signals on D-2 and B-1.Notably, the skipping triggering command works only when the D-2 and B-1 are detected in sequence.Then, User 1 stands still and no triboelectric signals are generated for all the 10 channels until User 2 comes in.Similarly, User 2 steps on the entry mat in the real space first for identity recognition (signals shown in light red frame), and the successful recognition is displayed as a woman avatar appears in VR space (Figure7F).After that, User 2 walks to the other skipping command triggering positions, during which the generated signals are marked in the dark yellow frame with two inner light yellow frames indicating the command signals triggered from C-2 and A-1 in sequence.Figure7G,H display the corresponding walking trajectory and skipping triggered positions in VR space, respectively.After the two users stand still, they begin skipping together in the real space and the corresponding avatars respond in VR space with accumulated counting concerning each user.The two blue shadowed frames in Set B and D represent the skipping signals from User 1, and the red ones in Set A and C stand for User 2. The counting number is accumulated separately for the two users by considering the positive peaks in the corresponding mat sets following the time sequence.It should be noted that the signals are simultaneously generated in Set B and D for User 1 (Set A and C 85.Dong B, Zhang Z, Shi Q, et al.Biometrics-protected optical communication enabled by deep learning-enhanced triboelectric/photonic synergistic interface.Sci Adv.2022;8(3):1-14.