Virtual Texture Generated Using Elastomeric Conductive Block Copolymer in a Wireless Multimodal Haptic Glove

Haptic devices are in general more adept at mimicking the bulk properties of materials than they are at mimicking the surface properties. Herein, a haptic glove is described which is capable of producing sensations reminiscent of three types of near‐surface properties: hardness, temperature, and roughness. To accomplish this mixed mode of stimulation, three types of haptic actuators are combined: vibrotactile motors, thermoelectric devices, and electrotactile electrodes made from a stretchable conductive polymer synthesized in the laboratory. This polymer consists of a stretchable polyanion which serves as a scaffold for the polymerization of poly(3,4‐ethylenedioxythiophene). The scaffold is synthesized using controlled radical polymerization to afford material of low dispersity, relatively high conductivity, and low impedance relative to metals. The glove is equipped with flex sensors to make it possible to control a robotic hand and a hand in virtual reality (VR). In psychophysical experiments, human participants are able to discern combinations of electrotactile, vibrotactile, and thermal stimulation in VR. Participants trained to associate these sensations with roughness, hardness, and temperature have an overall accuracy of 98%, whereas untrained participants have an accuracy of 85%. Sensations can similarly be conveyed using a robotic hand equipped with sensors for pressure and temperature.

Haptic devices are in general more adept at mimicking the bulk properties of materials than they are at mimicking the surface properties. Herein, a haptic glove is described which is capable of producing sensations reminiscent of three types of near-surface properties: hardness, temperature, and roughness. To accomplish this mixed mode of stimulation, three types of haptic actuators are combined: vibrotactile motors, thermoelectric devices, and electrotactile electrodes made from a stretchable conductive polymer synthesized in the laboratory. This polymer consists of a stretchable polyanion which serves as a scaffold for the polymerization of poly (3,4-ethylenedioxythiophene). The scaffold is synthesized using controlled radical polymerization to afford material of low dispersity, relatively high conductivity, and low impedance relative to metals. The glove is equipped with flex sensors to make it possible to control a robotic hand and a hand in virtual reality (VR). In psychophysical experiments, human participants are able to discern combinations of electrotactile, vibrotactile, and thermal stimulation in VR. Participants trained to associate these sensations with roughness, hardness, and temperature have an overall accuracy of 98%, whereas untrained participants have an accuracy of 85%. Sensations can similarly be conveyed using a robotic hand equipped with sensors for pressure and temperature.
thermoelectric devices. The sensation of roughness, in contrast, is simulated by an electrotactile signal emanating from conductive, bioinspired π-conjugated elastomer synthesized in our laboratory. In particular, the electrical signal at the fingertips creates a sensation reminiscent of surface texture: a continuous signal is perceived as smooth, whereas an intermittent signal produces an effect perceived as rough or bumpy. In a series of human-subject experiments in virtual reality (VR), trained and untrained participants were able to distinguish the properties of "mystery" panels having 2 3 ¼ 8 permutations of the following pairs of binary sensations: rough versus smooth, hard versus soft, and warm versus cool. In addition to its application in VR, this method for the "transmission of touch" [16,17] can also be accomplished when the glove is used to control a robotic hand equipped with sensors for temperature and mechanical force. These demonstrations highlight a design strategy in which mechanical, thermal, and electrical devices-comprising both commercial and purposesynthesized materials-can be integrated into a single device. These results may interest researchers working in the areas of haptics, medical training, [18] physical therapy, [19] and gaming. [20] To mimic the sensation of hardness, we chose vibrotactile stimulation using commercial vibrotactile motors embedded in the fingertips of the haptic glove. Upon making contact with the surface of a virtual object in psychophysical experiments, lower amplitude vibrations were perceived as softer, and higher amplitude vibrations were perceived as harder. The surface temperature of virtual objects was generated using thermoelectric devices, where the magnitude and polarity of the applied voltage determined whether the participant felt a heating or cooling sensation. To mimic surface texture, we used the electrotactile effect. [21][22][23] Electrotactile stimulation is a form of sensory substitution in which an electrical potential is applied to the surface of the skin. This signal generates action potentials in the nerve endings in the skin that is perceived as tingling. Although the sensation can be easily distinguished from surface texture, it does evoke it. [24] Electrotactile stimulation has a long history in the field of "haptic displays." [25,26] The types of electrodes most commonly used for electrotactile stimulation in flexible devices are metallic thin films. [27] However, metals have relatively high impedance when they make contact with the skin. [28] Moreover, in the context of a wearable device, metallic films are inherently fragile, [29] although the use of metallic serpentine traces can circumvent this limitation to some extent. [21] The challenges presented by the use of metallic electrodes for biointerfaces can, in part, be circumvented by the use of π-conjugated (conducting and semiconducting) polymers. Conductive polymers used in biomedical applications-"organic bioelectronics"-are attractive because they allow facile chemical modification, low-temperature processing, oxide-free interfaces, and mixed modes of conductivity (electronic and ionic). [30] In particular, the π-conjugated polymer poly(3,4-ethylenedioxythiophene) (PEDOT), usually complexed with the polyanion poly(styrenesulfonate) (PEDOT:PSS), is attractive in bioelectronic applications because of its tolerance of aqueous environments and low electrical impedance compared with metals. [28] It has thus been used in a wide variety of neurological and other electrophysiological recordings. [31][32][33] However, commercial forms of PEDOT:PSS are mechanically brittle [34,35] and do not achieve their highest conductivities (or greatest mechanical deformabilities [36] ) unless doped with additives-some of which are toxic-that can leach into the surrounding environment. [37] Recently, Withana et al. constructed a "tacttoo" consisting of an electrotactile array on a highly flexible elastomeric sheet used for temporary tattoos. [38] In this device, the author used screen-printed conductive inks comprising PEDOT:PSS as the flexible interconnects, although Ag/AgCl electrodes were used Figure 1. Schematic drawings and photographs of the haptic system described in this article. A haptic glove is equipped with two types of sensors-flex sensors on each of the fingers and a commercial motion tracker (on the wrist, not shown)-and three types of actuators-vibrotactile, thermoelectric, and electrotactile. The electrotactile device is composed of a textile embedded with a conductive, elastomeric block copolymer, whose structure is shown in the box. The glove communicates wirelessly and bidirectionally with a physical complement (robotic hand) and virtual complement (VR hand).
www.advancedsciencenews.com www.advintellsyst.com for actual contact with the skin. [38] While the authors were not specific as to the composition of the PEDOT:PSS ink, it is known that commercial PEDOT:PSS does not achieve useful levels of conductivity, [37] elasticity, [36] and wettability (for solution processing), [39] unless it is "doped" using solvent additives, some of which are toxic (e.g., Zonyl or Capstone fluorosurfactant [37] ).
To address both the mechanical and electrical shortcomings of PEDOT:PSS, we designed a single-component, intrinsically stretchable conductive polymer (i.e., no additives necessary) based on PEDOT polymerized within a stretchable copolymer scaffold composed of PSS and an acrylic polymer, poly(polyethylene glycol methyl ether acrylate) (PPEGMEA; Figure 2). The PPEGMEA block has a hydrophilic bottlebrush structure inspired by the soft proteoglycans found in cartilage. The synthesis of PSS-b-PPEGMEA was performed using an aqueous reversible addition fragmentation transfer (RAFT) polymerization ( Figure 2a). The use of RAFT polymerization allowed for excellent control over the molecular weight and polydispersity of PSS (M n ¼ 27.9 kDa, M w ¼ 33.6 kDa, Ð ¼ 1.2) and chain extension to the final block copolymer while maintaining a narrow polymer distribution (M n ¼ 51.9 kDa, M w ¼ 71.7 kDa, Ð ¼ 1.4; Figure 2b). The oxidative polymerization of EDOT was performed in the presence of dissolved PSS-b-PPEGMEA to afford PEDOT dispersed in the elastomeric scaffold. The approach is similar to our previously reported synthesis, [40] except that in the previous case, we intended to make a triblock copolymer using a bifunctional RAFT agent that was found to hydrolyze into a diblock copolymer after the incorporation of PEDOT. Neither the stretchability nor the conductivity could be optimized deliberately in this case because of a limited control over the molecular weight and polydispersity. Here, we modified the synthesis to afford greater control over the dispersity, mechanical, and electrical properties of the polymer. Namely, using a monofunctional RAFT agent (the dithioester shown as the first reactant in Figure 2a) and thus generating the diblock copolymer directly (no in situ degradation).
The impedance is the equivalent of resistance but for AC circuits. Contrary to resistance-which is a magnitude measurement-impedance has both magnitude and phase. The lower the impedance, the lower the voltage expected to produce an electrotactile sensation, and thus the lower the risk for redox reactions at the skin-electrode interface. Figure 3a shows the magnitude of the impedance of PEDOT:PSS-b-PPEGMEA thin films sandwiched between two stainless steel electrodes at various frequencies. The multiple traces are the results from three separate samples. Each sample was measured multiple times and the similarity between each run suggests that PEDOT:PSS-b-PPEGMEA is stable within the frequency range. The difference in overall amplitude can be attributed to a difference in thickness between the samples. Although the thickness was initially identical for all three samples, the tightening of the electrodes around the thin and stretchable PEDOT:PSS-b-PPEGMEA resulted in thickness variations during the measurements. The tighter the electrodes, the lower the thickness of the sample and hence the lower the impedance magnitude.
The Nyquist plot of the imaginary versus the real impedance ( Figure 3b) shows a behavior consistent with the equivalent circuit model shown in the inset. This model contains a constant phase element (CPE) consistent with a previous report of PEDOT:PSS blended with poly(vinyl alcohol). [41] The conductivity was calculated from the total resistance (R 1 þ R 2 ) to be 1.13 Â 10 À7 S cm À1 (AE0.33 Â 10 À7 standard deviation from n ¼ 6). We also measured the DC conductivity of our samples using a two-wire method, in which wires were attached to the ends of films casted from rectangular molds. PEDOT:PSS-b-PPEGMEA exhibited a better DC conductivity (0.11 AE 0.04 S cm À1) than the similar material that we have previously reported (0.05 S cm À1 ). The mechanical behavior of the material is shown in Figure 3c in the form of plots of stress versus strain (tensile tests) for the sample exhibiting the minimum, median, and maximum values of stretchability (strain at failure) and toughness (energy density corresponding to the total area under the curves). The maximum stretchability obtained was 43% with an average of 37 AE 5%, an average Young's modulus of 32 AE 11 MPa and average toughness of 665 AE 280 kJ m À3 (standard deviation from n ¼ 5). We attribute the large range in measured moduli and toughness to defects in the samples, which were Nevertheless, based on these electrochemical and mechanical measurements, we concluded that this stretchable conductive polymer would have the desired properties for an electrotactile electrode when integrated into a textile. We then sought to integrate the conductive polymer electrodes into the glove, which also contained the vibrotactile motors and thermoelectric devices (Figure 4). The electrotactile pads were fabricated by drop-casting small amounts of conductive elastomer dispersed in water onto prestrained spandex and drying over a hot plate at 100 C (without direct contact with the hot plate). Electrical contact between the conductive textile was made using a commercial thread containing steel fiber and insulated with VHB tape on the backside of the electrotactile pads. These pieces of conductive spandex were sewn into the interior surface of the fingertips of a golf glove. The electrotactile devices were operated at a range of 20-200 V, peak to peak (i.e., À10 to þ10 V and À100 to þ100 V). The frequency of stimulation was 50-300 Hz.
To determine the voltage and frequency to use for each participant, the examiner ramped both parameters until the participant indicated that they could perceive the sensations with high confidence. To simulate sliding of the fingers on a smooth surface in VR, the electrotactile signal remained on at all times, whereas the participant engaged with the virtual surfaces. To simulate a rough or bumpy surface, an intermittent signal was used. In particular, we used a duty cycle of 25% at 30 Hz (e.g., the electrotactile signal was on for seven cycles and off for 21).
To simulate the hardness (or softness) of the surface of objects using the glove, we used six vibrotactile motors: one at each fingertip and one in the center of the palm. Each device was an 8 mm diameter vibration motor (Jinlong Machinery & Electronics, C0825B002F). It was operated at 60 Hz for both "hard" and "soft" sensations, although the amplitude of vibration was greater for the "hard" sensation (i.e., applied voltage of 0.625 V for "soft" vs 3.3 V for "hard"). To provide the sensation of temperature, we used thermoelectric devices at each fingertip. These devices (Marlow Industries, Inc., NL1025T) measured 11 mm Â 9 mm. The temperature gradient produced through the thickness of the device was dependent on the polarity of the voltage. The devices were operated at voltages of þ0.45 V for "warm" and À1.8 V for "cool." Users reported that the "cool" side started to become warm after about 5 s. Future designs will include a heat sink or other form of thermal management to maintain the gradient, as has been demonstrated in previous work on multimodal haptic devices by Gallo et al. [42] and Guiatni et al. [43] Each finger was instrumented with commercial flex sensors to monitor the degree of bending of the fingers. The position of the hand was monitored using a commercial motion tracker worn on the wrist (an accessory to the HTC Vive headset, shown in Figure 5a). The tactile devices and flex sensors were controlled with two printed circuit boards (PCBs) whose designs are shown in Figure 4 (right). The PCBs were also capable of wireless communication with the robotic hand and the VR environment. We designed the PCBs to have the capacity for expansion up to 64 electrotactile pixels, e.g., 16 each on four fingers, or 12 on four fingers plus the thumb with four unused. The use of electrotactile arrays on the fingertips might allow for the sensation of propagating signals. In this demonstration, however, we used only one electrotactile device per finger because of limitations in our ability to pattern and address the conductive polymer on fabric. Complete details of the fabrication of the glove, PCBs, robotic arm, and VR environment can be found in the Supporting Information.
We then tested the performance of the haptic glove in a VR environment. We were interested in whether the electrotactile  . We recruited ten participants and divided them into two groups of five participants: "trained" and "untrained." We programmed a VR environment consisting of a room with eight rectangular panels on a wall and sought to explore whether participants could differentiate them by touch ( Figure 5). Each panel represented one of 2 3 ¼ 8 permutations of the binary sensations rough/smooth, soft/hard, and warm/cool. For the sake of simplicity, we used binary gradations only. However, the hardware was designed for a continuous range of stimulation. In the present case, we were interested if participants could identify sensations in the presence of simultaneous stimulation from all three actuators, and also if the sensations were sufficiently realistic to allow untrained individuals to perform the task. The psychophysical test in VR for the participants in the "trained" group proceeded as follows: participants were directed to wear the glove and VR headset and asked to touch each panel and describe the sensations associated with each one. In this exploratory phase of the training, only one type of actuator was engaged (vibrotactile, electrotactile, or thermoelectric), and the other two were off. For example, we asked participants to first  . Psychophysical discrimination tasks in VR. a) Photograph of a user wearing the haptic glove, VR headset, and commercial motion tracker on the wrist. b) Test panels which appeared on the wall of a VR environment were encoded with the eight permutations of three types of sensations (rough vs smooth, soft vs hard, and warm vs cool). The participant uses the glove to control the virtual hand that interacts with the panels. c) Accuracy of ten participants (five trained and five untrained) in determining softness, temperature, and texture. Error bars are 95% Clopper-Pearson confidence intervals on the binomial proportion. [44] Data points in red signify that chance of 0.5 lies within the confidence interval.
www.advancedsciencenews.com www.advintellsyst.com classify each panel as rough or smooth based on the intermittency of the electrotactile stimulation, with the thermoelectric and vibrotactile devices in the off state. We repeated this procedure for the other two modes of stimulation. We then revealed to the participants which panels had which characteristics, and the participants were allowed to re-explore the panels. Following this "training" routine, the characteristics of the panels were shuffled, and actuators for all modalities were engaged simultaneously. For the five "untrained" participants, the stimuli were combined with no opportunity to learn how each type of stimulation should be perceived. Details of the task for both trained and untrained participants are as follows: participants were asked to identify the all three modalities of sensation for each panel at once (as opposed to running through all eight panels three times, once for each modality). Participants addressed the panels in any order they wished, and indicated their identification to the experimenter verbally. They were also permitted to change their answers. Each participant completed the task in less than 10 min. Although panels representing all eight permutations of sensations were present for each participant (one panel per permutation), participants were not told how many occurrences of each permutation would be present.
The results of the discrimination task are shown in Figure 5c. Blue markers signify that participants performed better than would be predicted by chance, whereas red signifies that the accuracy is within error of chance. A logistic mixed-effect regression model showed that trained participants (98.3% correct overall) performed significantly better than untrained participants (85% correct) in identifying the characteristics of the virtual panels (Wald Z ¼ 2.85, p ¼ 0.0043). The data reveal differences in the ability of participants to discriminate sensations based on the type of stimulation present. For example, participants showed 100% accuracy for temperature, 90% accuracy for softness, and 85% accuracy for texture (average of trained and untrained participants). The modality of stimulation was a significant predictor of the accuracy of the participants (likelihood ratio test, χ 2 ¼ 19.447, p < 0.001); this finding suggests that some discrimination of some stimuli are indeed easier than others. In sum, we conclude that the haptic glove produced sensations that trained individuals could reliably discern in the presence of one another. The task was significantly more difficult for untrained individuals, although the mean accuracy in discrimination was above the chance for most sensations for most individuals.
The haptic glove was also capable of transmitting tactile signals from a robotic hand (instrumented with commercial sensors for pressure and temperature), whose fingers were controllable by flex sensors on the haptic glove (Figure 1c). We performed a psychophysical experiment to determine if tactile signals registered by the robotic hand could be transmitted to the participant through the haptic glove. In this experiment, the electrotactile sensation (produced by the PEDOT:PSS-b-PPEGMEA in the haptic glove) was associated with pressure applied to the finger of the robotic hand. The thermoelectric and vibrotactile devices were turned off. Seated behind the participant, the examiner pressed one of the five fingertips of the robotic hand. The participant then flexed the finger of the robotic hand corresponding to the finger at which the sensation was felt. The examiner performed these actions at random time intervals and recorded the accuracy. For a total of three participants and 20 stimulation events each, the combined accuracy was 60/60 correct, with no reporting of a sensation if none was present. We performed a similar experiment but with the thermoelectric devices turned on and the electrotactile devices turned off. In this case, the examiner placed a cold pack in contact with a randomly selected fingertip of the robotic hand, and again the participant reported the location of the sensation of coolness by flexing the appropriate finger. In a variation of this experiment, the examiner again pressed a randomly selected fingertip of the robotic hand, and the participant indicated the location of the perception of warmth (whose signal was triggered by the warmth of the examiner's hand). In these experiments, the participants similarly exhibited 100% accuracy. Although these psychophysical tasks were simple for the participants to perform in comparison with the tasks in VR, they highlight the ability to convey signals from a robotic end effector in addition to a virtual hand.
This article described a multimodal haptic glove capable of controlling and receiving tactile cues from VR and a robotic hand. This work highlights an approach to haptics that combines commercial tactile actuators with purpose-synthesized elastomeric conductive block copolymers. These devices make it possible for human participants to perform complex tactile discrimination tasks in VR. In particular, we found that the use of electrotactile stimulation using a conductive polymer provided a sensation of roughness of objects for both trained and untrained individuals. Although trained participants were better able to identify the intended tactile effects, we believe it should be possible to increase the realism of these sensations to bring the accuracy of untrained individuals closer to that of trained ones. In particular, the electrotactile and vibrotactile effects are merely reminiscent of (as opposed to identical to) sensations of roughness and hardness. Even so, in an eventual application in remote procedures of all types (e.g., in medical and search-andrescue contexts), it is highly likely that the user would be trained, and thus the acceptable level of realism may exist on a continuum. In future designs, we will aim to combine these modalities of interaction, such that it is possible to interact with a robotic end effector in a virtual environment.
Significant challenges revealed by the approach described here thus include the increase in the realism of sensations that can be generated. It is possible that this challenge might only be met by reimagining the electrotactile modality because it is difficult or impossible with current methods to control the location of sensation and the type of afferent that is targeted. Moreover, in our psychophysical experiments, stimuli were presented in binary gradations: rough versus smooth, warm versus cool, and hard versus soft. Future work on the psychophysical aspects of the work includes the determination of perceptual thresholds and the extent to which sensations are confused. Moreover, advancements in approaches to fabrication-particularly in increasing the density of miniaturized actuators on flexible and stretchable substrates [45] -will go hand in hand with the development of haptic devices capable of increased realism. [13] Despite these challenges, we nevertheless believe that complete realism in haptic interfaces may require the development of new materials capable of sensations that cannot be generated using off-the-shelf actuators. In particular, molecularly engineered materials whose surface energy, oxidation state, phase, and electrical and thermal conductivity can be changed in real time might allow for a wider gamut of tactile sensations than is now available. Moreover, materials that allow controllable deformation on smaller scales than is possible with conventional pneumatics will also accelerate discovery in the field. [13] We stress, however, the importance of chemistry and the design of materials at the level of molecular structure. This article represents the first time the tools of synthetic organic chemistry (namely RAFT polymerization) have been brought to bear on a problem in haptics. This intermingling of oft-separated fields may provide new tools for haptic devices capable of generating realistic sensations.

Supporting Information
Supporting Information is available from the Wiley Online Library or from the author.