Interactive Force Control Based on Multimodal Robot Skin for Physical Human−Robot Collaboration

This work proposes and realizes a control architecture that can support the deployment of a large‐scale robot skin in a Human‐Robot Collaboration scenario. It is shown, how whole‐body tactile feedback can extend the capabilities of robots during dynamic interactions by providing information about multiple contacts across the robot's surface. Specifically, an uncalibrated skin system is used to implement stable force control while simultaneously handling the multi‐contact interactions of a user. The system formulates control tasks for force control, tactile guidance, collision avoidance, and compliance, and fuses them with a multi‐priority redundancy resolution strategy. The approach is evaluated on an omnidirectional mobile‐manipulator with dual arms covered with robot skin. Results are assessed under dynamic conditions, showing that multi‐modal tactile information enables robust force control while at the same time remaining responsive to a user's interactions.


Motivation
Significant progress has been made in the realization of physical collaboration between humans and new generations of lightweight compliant robots. [1,2] These successes were possible due to advancements in mechanics, [3] actuators, [4] sensing, and control modalities.
A key strategy of collaborative systems is to effectively exploit the significant capabilities of both, humans and robots. For instance, superior human abilities such as high-level cognition and conscious perception are combined with the precision and repeatability of robots.
Physical humanÀrobot collaboration (pHRC) scenarios [5] are more complicated than noninteractive or purely reactive ones. Not only do they have to solve a collaborative task but also comply with external forces exceeded by humans. For example, rehabilitation robots need to guide a patient's limb and at the same time be compliant to prevent any injuries. Similarly, industrial manipulators need to react to the forces of their human coworkers and the surroundings to conduct a collaborative assembly task.
When humans solve these problems they often make contact with different parts of their body. An important aspect of natural collaboration is, therefore, the ability of the robot to feel contact forces not just at its end-effector but over a large area of its body. For example, Figure 1 shows a robot that controls the force it applies with its hand to an unknown surface (in this case, keeping hold of a poster on a wall), while simultaneously reacting to the multicontact tactile interactions from a human collaborator and avoiding collisions with its surrounding. This is just one of many possible applications where multicontact interaction can enhance collaboration. Other scenarios are carrying a heavy payload, handing over objects, manipulating limp materials, and so on. In this article, we focus on how to integrate multimodal sensor information, such as the local force and proximity signals measured, at the contact points, and a global distance sensor in a hierarchical controlled manner to help facilitate these kinds of tasks.

Related Work
A vast amount of research in the robotics community is directed toward physical humanÀrobot interaction. In these scenarios, humans may physically touch the robot and engage in a collaboration phase with the exchange of contact forces. Human skin often serves as an inspiration for these kinds of problems as the sense of touch allows us to dramatically outperform current autonomous systems when interacting with unstructured and dynamic environments. In nature, the presence of whole-body tactile sensing is ubiquitous, ranging from small insects to all mammals. However, most robots only acquire feedback through a small number of force/torque sensors in the joints or at the end-effector.
Fortunately, many promising systems for whole-body tactile feedback have been developed within the past decade. [6][7][8] Their application ranges from manipulation, [9,10] locomotion on unknown terrain, [11] to the generation of reactive compliant motions. [12,13] Different from robotics skin, extensive efforts have also been devoted to stretchable epidermal electronics, which paves the way for tactile sensing in human-friendly soft robots. [14,15] Yet another approach to obtain information about physical interactions is based on the indirect estimation of interaction forces. One method uses model-based joint torque residuals and thereby removes the need for tactile sensors. [16] Multiple contacts can be handled as well but require additional information about the contact location, e.g., obtained by an external camera. [17] In contrast to skin-based solutions, this approach requires an accurate dynamic model of the robot, which is in general difficult to obtain. [18] Once information about the type, intensity, and location of physical contacts is available through the underlying perception mechanism, their effects in terms of robot behavior need to be designed. Approaches to interaction control can be divided into two broad categories, namely, "indirect force control," where interaction forces are controlled through an impedance control law, [19][20][21] and "direct force control," where the control loop is closed via explicit force feedback. Impedance control is suitable for providing compliant behaviors but is limited in its ability to track desired forces. Improvements have been made by the introduction of set point generation, which adapts the impedance set point to enhance task execution. [22,23] Other strategies accurately apply desired forces by combining impedance control and direct force control. Such a generalization is, for example, proposed in the study by Magrini et al. [24] and enables force control at any point on the robot. The authors estimate contact forces based on the known exact dynamic model of the robot. In a later work, they further extend this to hybrid force/velocity control for pHRC. [25] Another hybrid force/impedance controller is developed in the study by Schindlbeck and Haddadin [26] to conduct accurate force tracking while providing compliant robot behaviors. The method is based on the concept of energy tanks and can safely handle unexpected contact losses. It is showcased on a torque-controlled robot, conducted a polishing task. Authors in another work [27] achieved force control and Cartesian compliance within a position-controlled manipulator. Their approach does not require the dynamic model of the robot and instead emulates a user-defined dynamics through admittance control.
Most of the aforementioned work was conducted on fixed-base manipulators. However, humanÀrobot collaboration (HRC) can greatly benefit from a robot's mobility. A mobile-based assistive robot was introduced in the study by Wu et al. [28] and successfully used in various collaborative tasks. Their framework implements a whole-body impedance controller that receives commands from a teleoperation interface or a mechanical admittance interface, attached to a human. [29] They demonstrate their system on complicated mobile collaborative tasks such as object caring and palletizing. [30] However, interactions during task execution are usually restricted solely to the robot's end-effector. This motivates the integration of whole-body tactile feedback for HRC and enables robots to interact more intuitively with their coworkers.

Contributions and Technical Summary
This work integrates our large-scale robot skin system in a collaborative humanÀrobot interaction scenario, allowing contacts across the entire surface of the arm. Compared with our previous publications, [12,13] where we used the skin to realize whole-body compliance, we are focusing here on the integration of skinbased tasks that go beyond pure reaction such as tactile guidance and force control.
Specifically, we show that our uncalibrated skin system can be used to implement stable force control, a task that is traditionally solved with F/T sensors. Although the reference needs to be defined in the uncalibrated sensor space, we still argue that this feature is very useful for pHRC. Many applications do not require exact knowledge of the interaction forces but rather their resulting effects. Often it is instead more important to detect and handle multiple contacts simultaneously.
To empirically test the robustness of our skin-based force controller we compare its tracking performance under various disturbances, such as movements in the mobile base or compliant reactions to multiple contacts on the arm.
A set of four interaction experiments were designed to validate the contributions of this work. In their implementation, we formulate the control law in Cartesian space and stack tasks in a hierarchy. For force control, we use contact information from the skin on the robot's hand to continuously adapt the wrist Figure 1. A Force-based task under pHRC. In this example, the skin on the robot's hand is used to apply desired forces on the surface of the board, whereas the robot reacts to the tactile interactions from the human collaborator. The distance sensors in the mobile base avoid collisions with the human and the environment. orientation perpendicular to the touched surface, producing welldistributed contacts.
Furthermore, global distance information from a light detection and ranging (LIDAR) sensor is fused with the distributed tactile signals of the skin to enable collaborations, where the mobile base avoids collisions with the user.
The implemented robot tasks for simultaneous real-time HRC are as follows: 1) skin-based force/distance control at the surface of an object, 2) compliant wrist control that can maintain the surface contact even if the object is moved, 3) tactile guidance on the arm to position the hand before force control, 4) real-time collision avoidance of the mobile base, and 5) joint space compliance based on the tactile skin signals.
We show that the skin is suitable for the control of interaction forces in pHRC scenarios with potential future applications in assistive and autonomous robotics.
The article is organized as follows. Section 2 introduces the robot model, general formulation of robot tasks, and the robot skin. Section 3 elaborates on our hierarchical approach, which includes a description of the implemented control tasks and the fusion of sensor signals. The experimental evaluation of the control framework is presented and discussed in Section 4. Finally, Section 5 gives a brief conclusion.

Robot Dynamics
We describe our holonomic mobile base as three additional degrees of freedom (DOF) that are added to the upper-body kinematic chain. The equations of motions for the complete system can be derived with Euler-Lagrange's formalism.
Here q ¼ ½ q ⊤ a w x b ⊤ ⊤ ∈ ℝ n is the vector of generalized coordinates, with q a ∈ R n a representing the joint angles of the upper arm and w x b ∈ SOð2Þ the planar pose of the base frame {b} with respect to world frame {w}. MðqÞ ∈ R nÂn is the generalized inertia matrix, Cðq,qÞ ∈ R nÂn is the Coriolis and centripetal effects matrix, and gðqÞ ∈ R n is the vector of gravitationally induced torques. The actuator torques are given by τ ∈ R n , and τ ext denotes the interactive torques produced by humans or by environment contacts.

General Description of Robot Tasks
Robot tasks are typically represented by a task variable x i ∈ R m i such as position or orientation. As the control input in Equation (1) is assumed to be at the torque level τ, it can only instantaneously affect accelerationsq. To express the task in terms ofq, the second-order time derivative of x i is needed: where w J i ðqÞ ¼ ∂ w x i = ∂q ∈ ℝ m i Ân is the Jacobian matrix, representing the differential kinematics mapping from joint space to task space. A robot task is then designed by specifying a reference accelerationẍ ⋆ i ∈ R m i . For example, a compliant reactive behavior with respect to some external wrench w W i can be achieved with well-known impedance control. [19] In this case, x ⋆ i is derived from a damped mass-spring system, leading to where Λ i , D i , K i ∈ R m i Âm i are the desired inertia, damping, and stiffness matrix and Δx ¼ w x d,i À w x i the position error to be minimized. Similar to Equation (4), a wrench task for direct force control can also be expressed as a reference acceleration. [24] wẍ⋆ We will use these formulations to describe our Cartesian interaction tasks in Section 3 and fuse multiple of them with a redundancy resolution strategy to produce the desired control torque τ ⋆ of Equation (1).

Multimodal Surface Contacts
The compliant robot tasks of the previous section require information about contacts with the environment and the user. Such information can be obtained with our robot skin (see Section 4.1.2 for details), which provides us with feedback about the type, location, and intensity of physical interactions. We use this skin information to compute external wrenches that are then inserted into one of the compliant tasks mentioned in Equations (4) and (5).
Consider the manipulator shown in Figure 2. The surfaces of the links of the robot, including the hand, are covered with patches of robot skin. When there is a tactile interaction, a certain number of cells are active and send a stream of uncalibrated normal force f s i and proximity p s i information.
The proximity signal is normalized to a range of ½0, 1 and reaches its maximum when the sensor touches a surface. Similarly, the force output of the skins' pressure sensors is normalized to ½0, 1. Both sensors have a nonlinear inversely proportional property with a high sensitivity at short distances and low forces with decreasing sensitivity at higher magnitudes. [31] The weighted sum of these two signals gives a virtual force produced by each skin cell i in its associated cell frame fs i g.
where w is a weighting factor to fuse the force and proximity signals. This allows us to react to contacts in a preemptive manner by setting the weight to a full-proximity reaction (w % 0), which becomes useful if the robot is approaching an unknown surface. Another unique ability of the skin is to change the weights based on the number of active cells and thereby control the pressure the robot applies over some surface. The force distribution sensed by all active cells n a j within a patch p j is then expressed as an equivalent wrench in the corresponding patch frame fp j g.
where p j R s i is the rotation matrix of the cell frame fs i g with respect to patch frame fp j g and p j t s i its corresponding translation vector. This transformation can be automatically obtained with our "Robot Skin Framework", as explained in the study by Cheng et al. [32] The next section presents the implemented controllers that use the output wrench p j W generated by each skin patch as feedback for designing different robot behaviors.

Hierarchical Control Framework
In this section, the control framework for whole-body motions is derived. Starting with a short overview of the proposed structure, we proceed to the formulation of robot tasks. Then, we provide a redundancy resolution that fuses these tasks into torque commands. Finally, we show how these torque commands are transformed into nominal velocity trajectories that can be executed on our robot. Figure 2 shows the main components of the framework. A "simple" state machine (SM) (a) (Section 3.1) triggers different robot behaviors by changing the desired values, parameters, and arrangements of robot tasks in a task hierarchy (b) (Section 3.2). The states of the SM, including its desired values, are predefined and the parameters are heuristically obtained. Activated tasks are then fused by a hierarchical policy (c) (Section 3.3), and a torque resolver (d) (Section 3.4) converts the resultant control torques into nominal velocity trajectories for the arm and mobile base.

State Machine
Depending on whether the robot's hand is currently in contact with the environment or not, the SM changes the task hierarchy. In this work, we consider a set of four hierarchical tasks that are arranged, as shown in Table 1. During the precontact phase, the first two tasks align the robot's hand toward an object's surface and the third task enables tactile guidance to position the hand. The mobile base on the lowest level avoids collision. In the contact phase, the first two tasks ensure stable contact with the object, and the third level keeps the hand at its initial position to prevent sliding. Compliant interactions and collision avoidance are handled on the lowest level.

Hand Direct Force/Distance Control
The first task controls either the distance or applied force to the environment using the proximity and force information from the skin on the robot's hand. Noticeably, this task is set to the highest priority of the hierarchy as it is a physical constraint that cannot be violated.
A new frame {h} is assigned to the center of the hand, where the contact is made, with its z-axis pointing into the normal direction (see Table 1). Using Equation (7) and (8) we compute the wrench h W in this frame measured by the skin patch on the hand. The contact normal force f c ∈ R is then given by the z component of that force vector and the corresponding task Jacobian becomes the third row of the body Jacobian associated with the hand J 1 ðqÞ ¼ h J h,z ðqÞ ∈ R 1Ân . Direct force control is then achieved as in Equation (5) through the Cartesian reference acceleration.
Whole-body reactive hierarchical force control framework. An SM a) coordinates a set of robot tasks b) by changing their goals, parameters, and arrangements in a task hierarchy to produce different robot behaviors. c) The output of each task is fused at torque level, using a sequential nullspace projection technique. d) The torque resolver transforms the fused control joint torques into nominal velocity trajectories for the arm and mobile base.
www.advancedsciencenews.com www.advintellsyst.com with the contact force error Δf c ¼ f c,d À f c and Λ 1 , K 1 , D 1 ∈ R specifying the desired contact dynamics in the direction of the normal force.
Changing the weight in Equation (6) between proximity only (i.e., w ¼ 0) and force only (i.e., w ¼ 1) allows us to interpolate between controlling the distance to the contact surface and controlling the applied force.

Compliant Wrist
The secondary task aims to realize a complaint behavior of the hand's wrist. This adjusts the hand toward unknown surfaces without the need to estimate the contact task frame on the object's surface. The skin's pretouch modality becomes especially useful since the adaptation occurs before the actual physical contact, greatly reducing the impact wrenches.
The moments h Γ measured in the hand frame {h} become zero if the hand is perfectly perpendicular to a surface. Therefore, the objective of this task is to ensure convergence of h Γ to zero. The task Jacobian is given by the rotational part (lower three rows) of the body Jacobian associated with the hand J 2 ðqÞ ¼ h J h,rot ðqÞ ∈ R 3Ân . The task is then again expressed on acceleration level.
where Λ 2 , K 2 , D 2 ∈ R 3Â3 determines the adaptation behavior of the wrist.

Tactile Interaction
The two previous tasks determine the dynamic behavior of the hand contact. In contrast, the next level generates adequate reactions to the tactile interactions with humans. Two different cases of human interactions are realized and selected by the SM. First, tactile guidance of the hand to a desired position is selected and second, compliance of the whole body.
In both cases, the wrenches, p j W, produced by each skin patch j on the arm are computed using Equation (7) and (8).
Tactile Guidance: For tactile guidance, the force components of these wrenches are first transformed into the hand frame {h} with the rotation matrix h R p j ∈ SOð3Þ and then added within the new base.
The dynamic response of the hand to the tactile interactions on the arm is designed as an impedance where the task Jacobian J 3 ðqÞ ¼ h J h,pos ðqÞ ∈ R 3Ân is the linear part (upper three rows) of the hand Jacobian. Body Compliance: When the hand is in contact with a surface, its Cartesian position should no longer be changed to prevent sliding. In this case, tactile signals are instead used to provide joint compliance in the remaining nullspace of higher-priority tasks. To this end, we directly compute the joint torques produced by the tactile interaction.
Compliance of the body and collision avoidance is then achieved by fusing τ skin with torques computed from the distance sensors in the mobile base. This process is explained in Section 3.2.4.

Obstacle Collision Avoidance and Compliance
Mobile-base collision avoidance and body compliance are handled on the lowest-priority level of the task hierarchy, allowing the robot to react to its environment while still achieving the higher-priority tasks (keeping a stable contact with the hand). The collision avoidance is based on the gradient of a repulsive "artificial potential field," [33] constructed from the distance information of two onboard LIDARs. [34] For every obstacle in the vicinity of the robot, the pair of closest points, b x o i ∈ R 2 and b x o i ∈ R 2 , that respectively belong to a point on the surface of the obstacle, and the robot footprint, are computed (See Figure 3).
The repulsive force between these two points is proportional to their distance The torque τ oca ∈ R 3 for obstacle Table 1. Task arrangement of the hierarchy during contact ("contact" phase) and before contact ("precontact" phase).
where α is a decay factor that determines the influence of distant objects and f max is the maximum force for d o i ¼ 0. b J o i is the Jacobian matrix from jointspace to the collision point b x o i on the robot footprint. A compliant behavior of the robot joints is then achieved by the controller where τ s ¼ τ skin þ 0 ⊤ n a Â1 τ ⊤ oca Â Ã ⊤ is the vector of sensed torques produced by the skin, as shown in Equation (14), and LIDARs, as shown in Equation (15). The term Δq ¼ q À q d is a posture error that pulls the robot on a desired posture if no τ s is present. The gains K c , D c ∈ R nÂn determine the influence of the position controller and the jointspace damping.

Redundancy Resolution
The tasks described in the last section are arranged in a strict hierarchy according to the order shown in Table 1 and fused to a commanded control torque using nullspace projection techniques. [35] We use the multipriority control formulation presented in the study by Sadeghian et al. [36] which combines tasks at the torque level. Given the reference task space accelerationsẍ ⋆ k on each task level k ¼ f1, : : : , L ¼ 3g, as shown in Equation (9,10,12), and the torque τ ⋆ c for collision avoidance and compliance, as shown in Equation (18), the commanded torque is computed with Here τ ⋆ , τ k , and J k are the commanded torque, projected task torques, and projected task Jacobians, respectively. The term a k corrects for accelerations produced by all higher-priority tasks projected into the current task k. The matrix Λ k ∈ R m k Âm k is the projected task space inertia and N k ∈ R nÂn the augmented nullspace projector for torques with J #M k ∈ R nÂm k being the dynamically consistent generalized inverse of J k , given by [35]

Torque Resolver
Fused torques are then sent to an admittance interface for the arm and the mobile base to compute adequate joint velocity commands. This enables us to apply model-based force/torque control strategies and also execute them on velocity or positioncontrolled robots. The virtual admittance is defined as [37] where τ ⋆ a ∈ R n a and τ ⋆ b ∈ R 3 are the arm-and mobile-base-related components of the complete torque vector in Equation (19). Parameters M b , D b ∈ R 3Â3 and M a , C a ∈ R n a Ân a specify the virtual dynamics of the system and can be different from the real one. The accelerations are then integrated to velocities and sent to the velocity controller of the mobile base and the arm. An inner-velocity control loop avoids drifting between the real velocity and the virtual velocity. [38] One advantage of the torque resolver is that it can achieve manipulator agnostic behavior, as the underlying hardware always emulates the same virtual dynamics. It also allows us to easily move from a velocity-controlled robot to one that only offers a position interface by integrating the virtual dynamics twice.

Robot Hardware
The evaluation and validation of the framework proposed in this article have been conducted on the autonomous mobile robot platform, Tactile Omnidirectional Mobile Manipulator (TOMM). [34] TOMM is a dual-arm mobile manipulator, composed of two UR5 industrial robots with six DOF, each covered with robot skin, and mounted on a holonomic mobile base. For the experiments, an Allegro hand is attached to the end-effector, the surface of which is also covered with robot skin (see Figure 2). TOMM's arms and mobile base are velocity controlled with a fixed control loop frequency of 125 Hz for the arms and 1 kHz for the mobile base.

Robot Skin
The e-skin deployed on the robot is a self-organizing skin system that consists of hexagonal-shaped sensing modules, the "skin cells", [39][40][41] as shown in Figure 4. Assembled together, these skin cells form bendable "skin patches". Due to the hexagonal shape of the skin cells, skin patches easily conform to arbitrary 3D surfaces of robots. A skin cell is a rigid element roughly the size of a two-euro coin. Each skin cell embeds a microcontroller and a set of multimodal tactile sensors. It measures pressure and forces with three capacitive force sensors, light touch and distance with an optical proximity sensor, vibrations with a three-axis accelerometer, and temperature with temperature sensors-nine modalities in total. The microcontrollers of the skin cells locally filter the acquired tactile measurements, implement the selforganizing protocol, [40] and realize the efficient event-driven signaling mode. [42] Self-organization and self-configuration allow a network of skin cells to automatically establish connections to the computer. The multiredundant self-organizing meshed networks of skin cells compensate for connection failures, skin cell failures, and communication bottlenecks, [43] thereby significantly contributing to the configuration flexibility and robustness of the e-skin system. A skin cell samples its sensors with up to 250 Hz. Self-configuration and efficient event-driven communication and processing allow the e-skin system to scale to large-area systems, without the need to reduce the effective sample rate of the system. [42] The e-skin on the two arms of TOMM incorporates in total 700 skin cells with 6300 multimodal sensors. [41] The e-skin system can handle these 6300 sensors at 250 Hz. Effective integration of large-area e-skin in control demands knowledge about the exact position and orientation of the tactile sensors with respect to the limbs of the robot. The manual localization of thousands of sensors is not feasible. To overcome this challenge, the e-skin system provides mechanisms for self-localization. [41,44] The e-skin can automatically reconstruct the 3D surface it covers. [44] Combined with self-configuration (the automatic organization of skin cells in skin patches), the e-skin can automatically determine the positions and orientations of its sensors. [41] Self-configuration and selflocalization significantly reduce the complexity of the system with respect to wiring, reconfiguration, and deployment.

Summary
A set of four experiments were conducted to demonstrate the main features of the described system. The experiments are sorted based on their complexity, moving from purely reactive to interactive to collaborative ( Table 2). First, we study individual aspects of the system in more detail such as the force tracking performance under disturbances generated by mobile-base motion (Section 4.3.3); the tactile guidance of the robot's hand based on interaction on the arm (Section 4.3.2); and force tracing with simultaneous user interactions (Section 4.3.1). Finally, we showcase the system in a pHRC scenario in which the robot firmly holds a part while the partner is operating on it (Section 4.3.4). The volunteer consented to his photos being taken and published as part of the figures.
We obtained the parameters for these experiments heuristically but keep them the same throughout all tests. The tuning Figure 4. Top and back of one skin cell. [39,40] A skin cell connects to up to four neighbors via four connection ports. Together, the self-organizing skin cells form the multimodal e-skin.
www.advancedsciencenews.com www.advintellsyst.com process involves creating a mapping of the skin signals to achieve desired physical forces/torques that suit the application. We provide a video showcasing the experiments (see Section 6, Supporting Information).

Procedure
All experiments follow the same protocol and can be divided into two phases.
Precontact Phase: The robot is positioned in front of an object and starts approaching the surface with its right hand. The proximity feedback provided by the skin on the hand is used to keep a constant distance from the surface (Section 3.2.1) and adapt the wrist orientation (Section 3.2.2). During this phase, the user can guide the hand to the desired position by interacting with the robot's arm (Section 3.2.3).
Contact Phase: Once the hand is positioned, the contact phase is manually activated by the user. The proximity information in the primary task is then replaced with the skin force signals by shifting the weight in Equation (6) with a sigmoid function to force control. This causes the robot to make contact with the object. During the contact phase, the force controller tracks a desired force profile (Section 3.2.1) and the remaining nullspace is used to provide compliance and avoid collisions (Section 3.2.4).
In this case, force control only occurs at the robot's hand, but our future goal is to use any part of the robot's body covered in skin to make intentional contact with the world (further discussed in Section 4.4).

Experiment 1. Force Tracking with Mobile-Base Motion
The first experiment evaluates the robustness of the contact force controller under dynamic disturbances. The experiment itself consists of placing the robot in front of a flipped table, as shown in Figure 5. The hand makes contact with the surface and tracks a sinusoidal force signal, whereas the mobile base executes a circular trajectory.
Between 15 s < t < 50 s, the base is commanded to move on a circle with a radius of 0.2 m. At 35 s < t < 90 s, the motion is changed to lateral swings parallel to the table with an amplitude of 0.3 m, as shown in Figure 5d. During execution, the higherprioritized hand position remains at its initial position, as shown in Figure 5c. The contact moments in the hand frame are shown in Figure 5b and become largest during the lateral base motion to keep the surface contact distributed. Figure 5a shows the applied contact force. Even though the base motion produces large disturbances, the contact remains stable and force tracking is still possible with an root mean square error of 0.38 N (9.0% of the desired force).

Experiment 2. Tactile Interaction
The second experiment studies the skins' pretouch modality when interacting with an unknown environment. First, the contactless compliant wrist adapts toward the table surface. Then, the user can then guide the hand toward different locations along that surface by interacting with the arm.
The key frames and signals of this experiment are shown in Figure 6. In Figure 6a, the distance d p to the object's surface is shown, which decreases as the hand approaches and is kept at a desired value of d p,d ¼ 0.03 m afterward. As the hand gets sufficiently close at t ¼ 5 s in Figure 6b, the moments h Γ start converging to zero by adjusting the hand perpendicular to the surface. The axis of frame {h} on the hand is distributed, as shown in Table 1.
In this experiment, the user can trigger robot behaviors in two ways. The red marked periods show the activation of the mobilebase collision avoidance task. When active, this induces movement in the base bẋ b (Figure 6d) but keeps the higher-prioritized hand position w x h unaffected (Figure 6c, e.g., t ¼ 12 s). The second interaction comes from tactile guidance and is marked by the green periods. As shown in Figure 6c at 20 s < t < 40 s, the robot reacts to the tactile contacts along the arm, allowing the user to guide the hand.

Experiment 3. Force Tracking with Tactile Interactions
This following experiment combines the interactions of the previous experiments with the force control of the first one (Section 4.3.1), enabling interactive behaviors during task execution.
After the wrist orientation adapts to the object surface as before, this time, the hand comes into contact. The primary task raises the applied force, as shown in Figure 7a, and then tracks a sinus with an amplitude of 2 N and a period of 4 s. Even when there are human interactions, e.g., the user approaching the mobile base (red areas) or touching the robot arm (green areas), the contact remains stable and is never lost.
Furthermore, the yellow marked areas correspond to changes in the objects orientation. The hand is able to follow the rotating surface because the second priority task constantly regulates the moments h Γ, as shown in Figure 7b, to zero. This allows the controller to keep the contact distributed over the hand skin patches. The corresponding wrist rotations are shown in Figure 7c and occur manly around the hand's x-axis, which is perpendicular to the ground.
Even in this situation, which involves a moving object, mobilebase collision avoidance, and tactile interactions, the robot can sustain force tracking as shown, e.g., in Figure 7a at t % 75 s. Without interactions, t < 30 s, the force generation is relatively accurate with an RMS force error of around 0.23 N (4.7% of the desired force). With there are external disturbances from the users, the tracking error gets worse (0.58N, 11.6%), but the contact is not lost.  www.advancedsciencenews.com www.advintellsyst.com

Experiment 4. pHRC
In the final experiment, we showcase the proposed control framework in a pHRC scenario. The goal of the robot is to help a human collaborator hanging a poster to a tilted wall by holding it in place.
Initially, the robot approaches the wall and adapts its wrist orientation based on the proximity signals, see key frame in Figure 8a. The user then guides the hand to the desired position through tactile interactions with the arm, as shown in Figure 8b. Once the hand is correctly positioned, the force controller is activated and starts applying a constant force, as shown in Figure 8c. (c) (d) Figure 6. Experiment 2. Tactile interaction: the robot approaches the surface of an object with its hand. b) Proximity sensors in the skin adjust the wrist orientation perpendicular to that surface by controlling the skin moments to zero. a) After this initial phase, a constant distance is kept . c) The user can then guide the hand along the surface through tactile interactions on the arm. Green marked areas indicate phases of tactile interaction and red areas collision avoidance of the mobile base. Interactions are collision avoidance (red areas), tactile interactions (green areas), and object movement (yellow areas).The compliant wrist follows the object surface by b) controlling the skin moments to zero and c) adapting its orientation.
www.advancedsciencenews.com www.advintellsyst.com While the robot is holding the poster, its mobile base avoids collisions with the user and reacts compliantly to contacts on the arm. The plots in Figure 8 show the corresponding control signals. The red marked periods correspond to the activation of the mobile-base collision avoidance task and the green periods to the tactile interactions on the arm. In Figure 8d, at 10 s < t < 20 s, the user guides the hand to the desired position, whereas the distance to the surface in Figure 8a is kept constant. Then, at t ¼ 27 s the force control is activated by manually switching the SM and the robot applies a contact force, as shown in Figure 8b. When the user approaches the robot and touches its arms at 40 s < t < 60 s, as shown in Figure 8e, the mobile base moves away, clearing space for the human to work. Throughout this collaboration phase, the hand remains at its assigned Cartesian position, as shown in Figure 8d.
This experiment demonstrates the dynamic interaction of our robot with a human while jointly collaborating on an exemplary task. The RMS force error throughout the experiment is around 0.8 N (12.9% of the desired force).
An overview of the tracking error of the previous three force experiments is shown in Table 3.

Discussion and Future Work
In the previous section we empirically demonstrated the feasibility of our skin system for interactive/collaborative force control. This section focuses on the current limitation, future work, and potentials of this research.

Parameters Estimation
Deployment of large-area sensory skin is still not common in robotics; the parameterization of these sensory systems is yet to Figure 8. Experiment 1. pHRC: the aim of this experiment is to attach a poster to a tilted wall. a) The robot approaches the wall and adapts its wrist orientation, 0 s < t < 10 s. d) The user then guides the hand to some position before it makes contact, 10 s < t < 25 s. b) The robot starts applying a constant force, holding the poster steady, 30 s < t < 80 s. e) When the user gets close to attach the poster, the collision avoidance of mobile base is triggered, creating more workspace for the user, 40 s < t < 60 s. An open question is how to automatically initialize and adapt the various control parameters based on a given robot, an environment, and a task. Data-driven approaches in which the robot first explores the contract with materials of different stiffness might be helpful.

Multicontact Force Control
In the earlier-mentioned experiments, contact force control only occurred at the robot's hand. In the future, we can extend the controller to do force control at any point of the robot or even at multiple points simultaneously, enabling the robot to push doors open with its elbow while carrying an object in its hands. This is just one example showing how whole-body tactile sensing unlocks many new abilities for robots to interact with their environment. The redundancy of a whole-body skin system can be carried over into the control framework and task planner, thereby allowing the robot to solve physical problems in a variety of ways.

Calibration
As mentioned earlier, the skin uses uncalibrated proximity and pressure sensors to measure distances and forces. The experiments show that this is not a problem for the design of compliant behaviors. However, it means that force control occurs in the uncalibrated (most likely nonlinear) sensor space. As the focus of this article was primarily on the interactions, a quantitative comparison between skin signals and physical forces measured by an Force Torque sensor was not discussed. This, however, could be an interesting comparison in future work. Nevertheless, we think that even without calibration the skinbased force control is valuable for pHRI. Very often we are not interested in the exact absolute value of the applied force but rather in the effects. For example in the last experiment, every force that prevents the poster from sliding is fine. Similarly in rehabilitation robots, we manly need to assure that the applied pressure to the human body never exceeds a given safety threshold.

Soft Robotics
This study might have some applicability in soft robotics, especially in the context of full physical interaction of humans and soft exosuits. [45] However, this requires future work. Besides bendability, the skin would additionally need to be stretchable and control algorithms would need to be extended from rigid body control to soft body control.

Conclusion
In this work, we presented a control approach capable of integrating the multimodal tactile signals of a robot skin and a distance sensor with a set of controllers to produce robot behaviors suitable for mobile pHRC. Specifically, we demonstrated the benefits of the robot skin for interactive force control tasks in unknown environments. To this aim, we formulated a hierarchy of compliant control tasks, enabling the robot to apply desired forces, adapt to unknown object surfaces, and react to interactions from a human, simultaneously. The robot behaviors are assessed with a set of experiments conducted on a mobile manipulator to show the applicability of this approach in real-world dynamic scenarios.
These experiments demonstrate that the system achieves stable force control with an uncalibrated large-scale skin system and can handle user interactions simultaneously.
Future work in this direction can extend the controller to enable contact force control at any point of the robot's body. This has potential application in assistive robots, which have to make contact at multiple points to provide physical assistance. Autonomous robotics can also benefit from this as robots can potentially manipulate their environment with other parts than just their end-effector.

Supporting Information
Supporting Information is available from the Wiley Online Library or from the author.