Tactile‐Based Negotiation of Unknown Objects during Navigation in Unstructured Environments with Movable Obstacles

Traditional robot navigation passively plans/replans to avoid any contact with obstacles in the scene. This limits the obtained solutions to the collision‐free space and leads to failures if the path to the goal is obstructed. In contrast, humans actively modify their environment by repositioning objects if it assists locomotion. This article aims to bring robots closer to such abilities by providing a framework to detect and clear movable obstacles to continue navigation. The approach leverages a multimodal robot skin that provides both local proximity and tactile feedback regarding physical interactions with the surroundings. This multimodal contact feedback is employed to adapt the robot's behavior when interacting with object surfaces and regulating applied forces. This enables the robot to remove bulky obstacles from its path and solves otherwise infeasible navigation problems. The system's ability is demonstrated in simulation and real‐world scenarios involving movable and nonmovable obstacles.


Introduction
Seamless navigation of robots within human environments, encompassing homes and offices, is essential for their successful integration into our society.Nevertheless, a significant challenge emerges when robots encounter unfamiliar objects in these environments.Traditional robot navigation techniques primarily rely on a limited search for collision-free paths, [1] following a strategy that prioritizes obstacle avoidance at all costs.Consequently, these algorithms often fail in cluttered situations as they treat all objects as obstacles, neglecting the manipulation capabilities of robotic systems, such as humanoid robots [2,3] or mobile manipulators. [4,5]This approach starkly contrasts with the natural behavior of humans, who leverage their ability to modify the environment in a way that assists locomotion.If necessary, even a child does not hesitate to move an object to reach an obstructed goal.
Inspired by this behavior, our objective is to equip robots with the capacity to analyze the environment and choose when to manipulate an object.Robot movements are planned by considering moving objects when the goal cannot be reached or the path to the goal can be significantly shortened.
Figure 1 shows an example of the type of problem we address in this article.The robot is confronted with reaching a desired goal, obstructed by unknown objects.In such a scenario, a conventional path planner would fail to provide a feasible solution since there is insufficient space to circumvent obstacles.Therefore, the system needs to identify objects, determine whether or not they can be moved, and clear them from the path to create sufficient space for continued navigation.Feedback is provided by the robot's onboard camera, LIDAR sensors on the mobile base, and a robot skin covering its arms and hands.

State of the Art
Taking into account the possibility of reconfiguring the environment by moving objects is known in the literature as navigation among movable obstacles (NAMO).The first planner that solved NAMO within a practical amount of time was presented in ref. [6].In this work, the author reduced the search space by assuming that disconnected free-space components can be connected independently by moving only a single obstacle.The strategy was successfully realized on the humanoid Robot HRP-2. [7]owever, the authors assumed full knowledge of the environment.Sensor data were acquired through external cameras and markers to position priory known models of movable obstacles.Beyond this, they further solved the domain where maximally k obstacles have to be moved to restore connectivity in the robot's free space, and each object needs to only be moved once. [8]ll the previously mentioned approaches rely on a complete knowledge of the environment and perfect data.The first extensions of NAMO to unknown environments are presented in refs.[9,10].The solution in ref. [9] was implemented on the humanoid Robot HRP-2 using onboard sensors only.Movable obstacles are detected by active sensing with a force-torque sensor.Their planning algorithm was a local approach only, where the robot reacts to a specific movable obstacle without considering other obstacles in the computation of a new plan.A planner that achieves local optimality was formulated in ref. [10].The authors discretize the world into a planar grid and employ A*-Search.The solution is then improved by considering the costs of manipulating or avoiding obstacles.
][13] The main challenge is the uncertainty in the perceived world state and the action outcome of a real robotics system.This is why more recent work tries to incorporate these uncertainties directly into the planning algorithm.In refs.[12,14], the authors leveraged ideas from probability theory to formulate the NAMO problem as a Markov Decision Process called NAMO-MDP.In ref. [12], the approach is further extended to incorporate the learned dynamics of movable obstacles.The robot gathers data by applying manipulation forces to one or more points on an object and records the forcetorque responses and resulting object trajectory.These data are then used to discover constraints in the object's motion.Scholz et al. [12] evaluated their ideas on a real-world robot but depended on external cameras.They test the algorithm in an environment with two constrained obstacles.In ref. [13], the semantics of movable obstacles are obtained through image scene classification.The system infers movability by assuming that specific object categories are movable.Tests were performed on a mobile manipulator concerning three different kinds of movable obstacles.Wang et al. [15] solved NAMO by extracting object affordances such as pushability and liftability from sensory data.They used trajectory optimization to plan robot movements.The method is showcased in a home environment with a moveable chair and a liftable water bottle.NAMO has also been approached with deep learning techniques.Zeng et al. [16] trained a forward model of object interactions that predicts the outcome of push and pull actions from vision.The method is tested in simulations that usually require removing a single obstacle to reach the goal.Wang et al. [17] trained a reinforcement learning agent and successfully deployed it on a real robot.The method does not include force sensing on the end-effector.
In the literature, knowledge about movability is usually given as input.However, active sensing has also been used based on visual feedback [18] and F/T-sensors. [9,12]Recently, another sensor for physical interactions has become available as whole-body tactile feedback.Skin systems such as refs.[19-21] are inspired by the human sense of touch and have proven to be very useful when acting in unstructured and dynamic environments. [22]f particular interest is the system developed in ref. [23] that offers multiple modalities such as force, proximity, acceleration, and temperature.

Our Approach and Contributions
In this article, we address the problem of navigation in a cluttered environment containing reconfigurable objects.Our approach is based on the tactile and proximity feedback measured by a robotic skin system.This sensor allows us to remove the common assumptions about an object's movability and replace them with real-time measurements.For planning object movements, we approximate their geometry with simple boxes and only require some rough information about their position and footprint.
The robot assesses the movability of unknown obstacles by approaching them with its hand and applying a test force.Once it is determined whether or not these obstacles can be moved, a planner decides how to clear the obstacle from the path or, instead, how to navigate around it (Section 2).To address the clearance of large and bulky objects, we employ a compliant pushing strategy that can execute previously planned actions (Section 3).This process repeats iteratively whenever a new obstacle is detected until the robot reaches its goal.Our contribution is a fully functional framework that can solve the navigation among movable obstacles problem in simulation and on a real robot.We validate our approach in cluttered simulated environments containing up to 30 obstacles (Section 4.1) and on a real robot operating in an unknown environment with six different obstacles (Section 4.2).
Overall, our experiments show that the skin on the robot's hand helps to relax the requirements in the perception and planning pipeline.Local information such as object distance, surface shape, and interaction forces can be handled through reactive compliant behaviors rather than exact calibration and planning.

Overview of Our Approach
The essential steps required by any solution to NAMO problems are: 1) identify the obstacle that requires relocation, 2) maneuver the robot to a suitable position for reaching it with the arm, 3) assess whether the obstacle is movable or static, and 4) negotiate the obstacle by moving it from the path or navigating around it.
These steps have to be solved iteratively until the robot reaches the goal position.The overview in Figure 2 depicts our approach to the problem.
The information flow begins with an object detector that identifies obstacles in front of the robot.Their position and geometry information is stored in a movability map.Initially, new obstacles are unknown and assumed to be movable.A NAMO planner (Section 2.2) with access to the map simulates possible pushes on detected obstacles to find the one that leads to enough clearance for further navigation.Once completed, the planner returns the next obstacle to move and the desired direction to push.At the next stage, movability detector (Section 3.1.3),the robot approaches the selected obstacle and applies a small test force to determine its movability.The map is updated with this new information.If the obstacle can be moved, the robot initiates a pushing strategy (Section 3.2) to clear the object from its path.Conversely, if the object is deemed static, the planner is reactivated, and with the updated movability map, it has to find another solution.Whenever the robot creates enough free space, the navigation process continues toward the goal.The planner aborts when all known obstacles have been tested, or they are inaccessible, and no viable path can be found.

NAMO Planner
A plan consists of a navigation sequence and compliant actions designed to bring the robot closer to the goal by systematically removing one movable obstacle at a time from the path.We build upon the planner that was first developed in ref. [10] since it is designed to operate in unknown environments.The algorithm has access to a static 2D map of the environment but no knowledge about obstacles.We make the following assumptions: 1) Connectivity through object manipulation: the planner assumes that two disjoint free space regions can be connected by moving a single object, as illustrated in Figure 3. 2) Planar projection of the environment: the planning domain is restricted to a planar projection of the three-dimensional environment.and 3) Obstacle detection and movability assessment: the robot can detect unknown obstacles and actively sense their movability.
The algorithm generates two types of plans: a direct path plan if a clear path to the goal exists without requiring obstacle interaction or a three-step plan if obstacles obstruct the goal.In the latter case, the procedure involves: first, reaching the obstacle (c 1 ), second, clearing the obstacle (c 2 ), and finally navigating from the cleared area to the goal (c 3 ).A visualization of this procedure is shown in Figure 3. Overview: the goal is to find a sequence of compliant actions that activity creates clearances for navigation.An onboard camera detects new obstacles and adds them to a movability map.Subsequently, a NAMO planner decides which obstacle to move next and in which direction.The robot approaches the selected object and assesses its movability.If it can be moved, a pushing strategy clears it from the path, thereby allowing the robot to proceed further toward the goal.This process repeats iteratively until the goal is reached, or all known obstacles have been tested, and still no path is found.
The overall cost of a particular path C path is formulated as a combination of these three segments and based on their Euclidean length: here, C nav is the step cost associated with moving the robot from one location to another.C act (o, p, a) denotes the step-cost required to execute clearing action a on object o when making contact at point p on its surface.The notation jc Ã j indicates the Euclidean length of a path segment.
The planning procedure outlined in Algorithm 1, finds the best plan according to C path , and can be separated into two steps.First, namoPlan that computes the plan from start to goal f w x i , w x g g and second obstaclePlan that computes plans for individual obstacles.

NAMO-Plan
Starts by evaluating the direct path between the start w x i ∈ SE 2 ð Þ and the goal w x g ∈ SE 2 ð Þ pose, using an A*-Search. [24]This initial path is the current optimal plan, denoted as p*.The manipulation of objects is only considered as long as it decreases this initial cost.The routine first sorts obstacles based on their distance to the goal and then starts calling obstaclePlan on the current object to see if it can reduce the cost (Line 8 in Algorithm 1).

Obstacle-Plan
Possible plans for a single object are evaluated.This involves simulating the outcome of different clearance actions a on object o and checking for new paths to the goal.The evaluation is performed iteratively, considering different contact points p ∈ ℝ 2 on the faces of the object.For each contact point p, a subroutine computes a suitable planar mobile base pose (Section 2.3).If a base pose w x b ∈ SE 2 ð Þ can be found, the algorithm plans the first path segment c 1 to reach that pose (Line 4).Next, the algorithm iterates over possible clearance actions, applying them to o, and simulating the movements of both the object and the robot.This simulation is performed for a maximum of n max steps.The effort associated with moving the object is determined based on the displacement of the mobile base robot.xthroughout the Input: w x i , w x g , robot, env x g , env:mapÞ x, w x g , env:mapÞ interaction and stored as the cost of the second plan component c 2 (Line 13).In each iteration, the algorithm checks for the existence of a new path between the updated robot position and the goal (Line 14).If a valid path component c 3 is found, a complete object plan p o is formed.If this plan has a lower cost than the current optimal plan p*, it is updated accordingly (Line 18).There are two termination criteria within the evaluation process.First, if the object o is static or becomes static because it collides with a wall or another static object, the evaluation is stopped by setting o.movable to false (Line 11).Second, a termination criterion based on the heuristic ĉ, which considers the moving costs required to reach the object (c 1 .cost)and the estimated remaining distance to the goal k w x g À robot.xk(Line 19).This heuristic stops action evaluations as soon as the costs of moving alone become higher than the current optimal p*, since adding segment c 2 would only increase costs. [10]fter running the planner, it returns the optimal plan, which contains the next navigation goal for the mobile base w x b , the object to manipulate o, the contact point p on one of its faces, and the action a to execute.

Suitable Mobile Base Placement
A mobile-based positioning strategy is necessary to plan path segments for reaching an object before manipulation.Since we are interested in pushing bulky obstacles that might exceed the available workspace of the robot, we have to consider the robot's manipulability.
Different base poses are evaluated against each other by assigning them a performance measure p q ð Þ. [25] We consider two factors: arm manipulability m q ð Þ and the distance d q ð Þ of joint angles q to a preselected arm posture q post which is suitable for pushing.
here 0 J h q ð Þ is the Jacobian of the hand {h} w.r.t. the robot arm mounting frame {0} and the scalar w ∈ 0, 1 ½ a weight factor, adjusting the influence of the preselected arm posture on the performance index.The goal is to maximize this performance index by selecting a suitable placement for the mobile base relative to a detected object.
If the robot's hand {h} has reached a desired contact point {obs} on one of the object's surfaces, such that w T h ðqÞ ¼ w T obs .We can re-express this relation using the kinematic chain of a mobile manipulator as: here w T b ∈ SE 3 ð Þ represents the transformation of the mobile base with respect to the world, b T 0 ∈ SE 3 ð Þ, the fixed transformation between the arm mounting and the mobile base, and 0 T h q ð Þ ∈ SE 3 ð Þ, the forward kinematics of the arm.
Rearranging for the kinematics of the arm 0 T h ðqÞ allows us to determine the joint angles required to reach for an object given the south-after base transformation obs T b ∈ SE 3 ð Þ.If we assume that the object's geometry can be represented with a simple box during the planning phase, the transformation obs T b only depends on rotations around the world z axis, and can be specified with four degrees of freedom: where h is the height of the object and obs x b ¼ x y θ ½ T is the planar pose of the base.By sampling these four parameters within a predefined range and resolution, the corresponding joint solutions and maximum performance index can be obtained through inverse kinematics.Finally, these values are stored in a data structure, indexed by fh, x, y, θg.Building up this data structure is done once in an offline step.During execution, a map M of possible base positions w x b ∈ SE 2 ð Þ for a given w T obs is found by slicing the data structure based on the object's height h.The base configuration is then selected by optimizing the performance index within that slice.
where D ⊂ W is a restricted workspace domain that originates from the required minimum clearances to other obstacles.Examples of generated base poses for pushing are shown in Figure 4.

Internal Simulation of Obstacle Clearance
The planning procedure, described in Algorithm 1, requires the definition of clearance actions capable of removing obstacles from the robot's path.In this work, we consider a restricted set of planar pushes, executed with a constant force, in the directions A ¼ f0, AE π=8, AE π=4, AE π=2grad.
Of course, a broader range of options is available, but we opt for pushing actions as they align closely with the physical capabilities of our real robot.We define possible contact points for each obstacle to apply pushes at the centers of its vertical surfaces.
We leverage a physics engine to simulate the effects of pushes and assess the feasibility of plans before executing them on the physical hardware.The physics engine is called in every iteration of the planner (Line 20) and simulates obstacles' rotation, translation, and collisions when forces are applied.It also takes into account the cascading effect when multiple obstacles collide with each other.Given that pushes are inherently planar actions, it is sufficient to simulate movements in two dimensions and exclusively consider the footprint of obstacles.This simplifying assumption increases the efficiency of the simulation step a lot.
An illustrative example of how the planner, the mobile base positioning, and the simulation work together is shown in Figure 5.The example considers a hallway scenario with three obstacles, two movable and one declared static.Whenever a new obstacle enters the robot's field of view or an object changes its movability, Algorithm 1 is restarted to adapt to the new circumstances.
Of course, such simulations can deviate from reality.Therefore, the following section connects the planner with real-time measurements from the robot and includes strategies for failure detection.

Perception
The essential information required to solve NAMO problems is the movability property of obstacles in the environment.Since we www.advancedsciencenews.com www.advintellsyst.comhave no prior knowledge about them, the robot needs to employ active sensing.

Robot Skin
One way to gather the information is through a distributed sensing network called robot skin that can cover the surfaces of a robot, see, e.g., Figure 1.Robot skin obtains real-time measurements about the type, location, and intensity of multiple physical interactions. [23]It is multimodal and senses nearby obstacles through proximity sensors prior to a contact.Forces during physical contact are measured with pressure sensors.Both proximity and force feedback are valuable to realize the required interactions between an obstacle and the real robot: first, to determine object movability, and second, to realize a robust path clearance strategy.
To use the skin's information, its raw cell-wise measurements must be converted into 6D wrenches.Each skin cell inside the sensing network delivers a stream of force f i ∈ ℝ and proximity p i ∈ ℝ information.They are combined into a virtual force f i ¼ ωf i þ ð1 À ωÞp i through a weighing factor ω that allows shifting between force and proximity-based responses.Since the location of every cell is known within the robot's kinematic chain, we can compute the resultant wrench of active cells inside any reference frame, e.g., the hand {h}.
here h n i and h t i are the normal and translational vector of the i-th cell, and γ scales the signals to treat them as wrenches.Subsequent sections use the obtained skin wrench to implement the robot-obstacle interactions.

Visual Perception for Object Detection
Besides the movability property, the planner requires geometric information about an object.This includes an estimation of its 2D footprint and height.In this work, the primary focus is not on advanced visual perception.Consequently, we achieve object detection through ArUco markers. [26]More advanced object classifiers can replace the simplified approach, as demonstrated in ref. [13].

Tactile Perception for Object Movability
The first interaction between the robot and an obstacle is actively determining its movability.This process involves establishing a light contact between the robot's hand and the object's surface.
A spline then gradually increases the applied normal force to a predefined maximum, denoted as f max .This maximum force is carefully set to ensure the safety of the robot arm while engaging with the object.Simultaneously, the contact velocity ẋc ¼ n T c J c q is integrated to calculate the displacement Δx c in the object's position.Here, J c denotes the contact's Jacobian matrix, n c the contact's normal vector, and q the robot's joint velocity vector.If the displacement exceeds a defined threshold distance, denoted by Δx th , the object is classified as movable, and the corresponding maximum force required for pushing f push is recorded.Figure 6 illustrates an example of the discrimination process for a movable and static object.Note that this method can fail when dealing with soft objects that deform instead of moving linearly.However, more advanced methods might be able to detect soft deformable objects from tactile feedback as well. [27]

Pushing Things Forward
Once the robot classifies an object as movable, an action is required to clear it from the path.We adopt a pushing strategy to deal with large furniture-like objects that can not be grasped.

Pushing Strategy
Our goal is not to precisely position objects in the world but to create enough space for continued navigation.Furthermore, we assume objects behave like holonomic entities with roughly uniform mass distribution.
This allows us to break down the pushing problem into two steps: first, rotating the object toward a desired angle, denoted as θ d and planned by Algorithm 1, followed by a forward push through the object's origin until the path is cleared.Figure 7a, b shows a visualization of this procedure.Here, we assign a contact frame {c} to the contact point with its x-axis pointing toward the object's origin {o}.In this new frame, the steps from above can be described as follows: first, a tangential force f t is applied in the direction 0 1 ½ T primarily causing the object to rotate.Second, a normal force f n in the direction 1 0 ½ T is applied, which acts through the center of rotation, causing a forward motion.
A proportional controller based on the current orientation error δ ¼ θ d À θ fuses these two directions and gives the overall pushing direction in the contact frame.
The scalar k δ ∈ ℝ is a gain parameter used to tune the influence of the orientation error.This strategy leads to trajectories, as visualized in Figure 7c, showing how the object (green) first rotates and then translates while the mobile base (red) follows to maintain contact.
The applied forces have to remain within the object's friction cone to prevent slippage during the pushing.However, determining the exact limits of the cone is difficult.Here, we limit the pushing angle at the object's surface, denoted as ∠ o n, to a fixed range of AE0.9 rad.
By further transforming this direction into the robot's hand frame {h} and multiplying with the desired object speed v d , we get the hand velocity We opt for a compliant approach rather than directly sending this command to a velocity controller.Instead, we convert the current velocity error into a desired force that a compliant force controller tracks.where K v ∈ ℝ 3Â3 is a velocity gain factor and h J h;p q ð Þ ∈ ℝ 3Ân is the position part of the hand's Jacobian.
In an unforeseen collision, the controller should not apply more force than a given maximum denoted as f max .We set f max slightly higher than the required force for pushing, determined before in Section 3.1.3when detecting movability.
here h f is the actual external force sensed at the robot's hand and γ is a factor that limits the applied interaction force to a magnitude of f max .
The final component of the pushing action involves a strategy to maintain alignment between the hand orientation and the object's surface.This alignment is achieved by following the object orientation reactivity as it turns.The objective can be formulated by ensuring convergence of moments h Γ, measured in the hand frame, to zero The complete wrench error Δw ¼ ½ Δf T ΔΓ T T at the hand is then minimized by the whole-body controller in Section 3.3.

Selecting the Contact Point
A remaining question is where to make contact on the object's surface before applying the pushing strategy.As evident from Figure 7a, objects can be turned most easily by exerting forces near the corners.If the goal is to push to the right, the contact point should be located at the left corner and vice versa.The most suitable contact point is in the middle when the objective is to push forward.The proximity sensing embedded in the skin on the robot's hand offers an elegant solution.By setting the skin wrench in Equation ( 5) to pure proximity (w ¼ 0), the robot can keep a fixed distance to the surface f d ¼ 0 0 p z ½ T and scan horizontally along the contour of an object until a jump in the proximity signal indicates the presence of a corner.While searching for the corner, the hand aligns its orientation reactively in the same way as during pushing.Reliably finding the corner of an object through other sensors, such as a camera or force-torque sensor, would be more complex.

Controller
In the end, all robot actions required in Figure 2, such as reaching for an object, detecting its movability, pushing it, and avoiding collision when the mobile base is in motion, must be converted into torque commands for the real robot.
We make use of our whole-body impedance controller, developed in ref. [28], which combines the holonomic mobile base and the arm into a redundant system Every action is a combination of low-level compliant controllers, formulated as tasks in Cartesian space [29] for the robot's hand and joint space else.Multiple active tasks are fused in a hierarchical manner using null space projection. [30]ll tasks are expressed in the form of impedance controllers where Λ, D, K define the desired inertia, damping, and stiffness matrix.The task error to be minimized is either measured in Cartesian space by Δx ¼ w x d;h À w x h , in wrench space by Δw ¼ h w d À h w or in joint space with Δq ¼ q d À q.

Hand Motion Task
This task tracks a desired trajectory f w x d , w ẋd , w ẍd g and is used for reaching contact points on an object's surface with the robot hand.
here w w h is the external wrench measured at the hand to render the motion compliant, especially when the object is detected incorrectly and appears closer than initially anticipated.

Hand Wrench Task
The wrench task controls the force and moment the robot's hand applies to an object.It is used in three ways.First, to determine the movability in Section 3.1.3.Second, to find the corner of an object with the proximity signals.And third, to realize the pushing strategy outlined in Section 3.2.

Mobile-Base Collision Avoidance Task
This task prevents the mobile base from colliding with the environment while the hand pushes an object.The method is based on planar artificial potential fields Ψ w x b , fog ð Þto guide the mobile base away from all close by objects {o}. [31]The task has lower priority than the other two to not interfere with the hand motions.

Arm Posture Task
The lowest priority task is a joint posture task that keeps the arm close to a desired configuration q d;arm .
Finally, in every control cycle, active tasks are evaluated and fused into the next torque command (see ref. [28] for more details).

Error Handling
Since the real dynamics of an object is unknown, the pushing action is susceptible to errors and requires a recovery strategy in the case of failures.Errors can occur when the object turns unexpectedly, loses contact with the hand, or collides with another static item in the environment.These cases can be detected by monitoring the contact state at the hand to trigger a replaning.

Contact-Loss
A loss of contact is detected in two ways.First, we monitor the sensed normal force component, and if it falls below a predefined minimum threshold, i.e., f c < f min , this signals a loss of contact.Second, we examine the magnitude of hand moments.If it surpasses a specified maximum threshold, i.e., k h Γk > μ max , there is a misalignment in the hand orientation and the object surface, as per Equation (11).In either case, the pushing action is aborted, and we initiate a second attempt by approaching the object once more.

Collision
Collision scenarios are identified by examining whether the sensed normal force, f c , exceeds a maximum threshold, f max .In this case, the object is marked as static and no longer considered by the planner in a subsequent replanning step.Figure 8 shows a collision situation involving a pushed chair and a wall.The objects collide at t = 8 s.The compliant pushing controller of Section 3.2 successfully limits the maximum force magnitude to f max = 2f push ≈ 9 N.

Experimental Section
We evaluate our approach using two testbeds: 1) a software simulation that can spawn worlds with many obstacles and 2) a real mobile manipulator with a 6 degree of freedom arm.

Simulation
For the simulation, we use the open-source physics engine Box2d [32] and the ROS Navigation Stack. [33]Box2d efficiently simulates the motion and collision events between many objects.
In contrast, the ROS Navigation Stack is a collection of software packages that can guide a robot to a specified goal location while avoiding obstacles.
In our simulation environment, we approximate obstacles with simple boxes of varying sizes.To apply pushing forces, we add potential contact points on each of the four vertical surfaces of these obstacles.
Initially, the robot is supplied with a static map of the environment but does not know the location of obstacles.We assume that new obstacles can be detected once they are within 2 m range of the robot's camera.We set the stepping cost for obstacle clearing in Equation ( 1) twice as high as the stepping cost for navigation: C act = 2.0 and C nav = 1.0.
To validate the effectiveness of the NAMO planner detailed in Section 2.2, the mobile-base placement in Section 2.3, and the pushing strategy in Section 3.2, we conduct experiments within this simplified setup.As a reference baseline, we use the standard ROS Navigation Stack planner, referred to as Move_base.Move_base follows the "avoid at all cost" strategy and, therefore, needs to navigate around every single obstacle in the environment.

Random Obstacles
The first experiment aims to address the scalability of the NAMO planner with respect to the number of obstacles in the environment.We configure a 12 m Â 12 m maze environment, and the robot's objective is to navigate from a predefined starting pose to a fixed goal pose.Obstacles are added to the map by randomly sampling their positions, orientations, and sizes.Note that there is no guarantee that a direct path to the goal exists.
In each iteration of the experiment, we increase the number of obstacles in steps of 5 s, starting with an empty map and gradually reaching up to 30 obstacles.
We conduct each experiment 10 times, employing both the standard Move_base planner and the NAMO planner to collect statistical data.Figure 9 shows four successive key-frames of the navigation process when the NAMO planner is active, and 25 obstacles are distributed over the map.The experiment's results are depicted in Figure 10.In the first plot, we show the total success rate of reaching the goal relative to the number of obstacles in the environment.The Move_base planner frequently returns infeasible from 15 obstacles onward when the path to the goal becomes too obscured.The NAMO planner can still reach the target up to the maximum number of obstacles.However, its performance drops when the number of obstacles becomes large because there is often not enough space to position the mobile base when reaching for an obstacle that needs to be moved.
Figure 10b compares the total path length required to reach the goal.In the case of 0-15 obstacles when both planners still manage to complete the task, NAMO consistently produces shorter paths than Move_base.Move_base often has to take large detours when a new obstacle comes into view and blocks the way.
Quantitative time measurements from the experiment can be found in Table 1.It lists the total execution time for NAMO and the time spent in different plan stages relative to the number of obstacles.The occupancy field measures the ratio of occupied map area to total map area after an inflation radius of 0.3 m is applied.The time spent on planning suggests a linear relation between planning time and the number of obstacles.
These simulations show that even if the environment becomes quite cluttered with 30 obstacles (75% of the gird map is occupied space), the common assumption in NAMO [10] of only having to Figure 10.Simulation: comparison between Move_base (green) and NAMO (red) in a simulated maze environment.Left: the success rate for reaching the goal.Move_base fails to reach the goal when the map contains more than 20 obstacles.Right: the total path length required to reach the goal.Without the ability to move obstacles, the path length is higher.move a single object to connect two regions of free space, works well in practice and could potentially scale to even larger maps.

Experimental Platform
We perform the real experiments on our robotics platform tactile omnidirectional mobile manipulator (TOMM), [4] see Figure 1.TOMM is a dual-arm mobile manipulator composed of two UR5 industrial robots with six degrees of freedom mounted on a holonomic mobile base.In the experiments, we use the right arm on which an Allegro hand is mounted.The robot is equipped with an RGB-D camera in the front and two LIDARs on its base.TOMM's arms and mobile base are velocity-controlled with a fixed control loop frequency of 125 Hz for the arms and 1 kHz for the mobile base.Control algorithms are implemented in Cþþ using ROS.

Electronic Skin
For the experiments, we use an electronic skin system (e-skin) capable of large-area tactile sensing. [23]It consists of hexagonal-shaped sensing modules, the skin cells. [20]Each skin cell embeds a microcontroller and a set of multimodal tactile sensors.It measures normal forces with three capacitive pressure sensors, light touch and distance with an optical proximity sensor, vibrations with a three-axis accelerometer, and temperature with temperature sensors.Assembled together, these skin cells form bendable skin patches mounted on our robot's arms and hands.
Self-configuration and efficient event-driven communication allow the e-skin system to scale to large areas without the need to reduce the effective sample rate of the system. [34]This enables us to deploy 700 cells on the robot that provide information at a frequency of 250 Hz.Manually locating every single cell within the kinematic tree of the robot is impractical.To overcome this challenge, the e-skin can automatically reconstruct its 3D surface coverage and determine the positions and orientations of its sensors.When the proximity signals are converted into distance measures, they can reliably detect objects in the range of 0-0.06 m.

Obstacle Clearance
The initial real-world experiment evaluates the pushing controller as outlined in Section 3.2.In this scenario, the robot is positioned in front of a chair, and the objective is to execute pushes in various directions as specified by the planner in Section 2.4.The chair is marked with an AruCo marker to get a rough estimation of its location through the camera system.Subsequently, the robot approaches this position and comes to rest if the skin proximity sensors on the hand detect the chair's surface.Before pushing can start, the robot needs to locate the object's corner.This is achieved by moving the hand along the surface while maintaining a constant distance with the proximity sensors.The motion  stops if skin cells detect a sudden increase in the distance, indicating the corner location.Finally, the hand makes contact and applies the pushing force based on Section 3.2 to move the chair.We repeat this procedure for the desired pushing directions of f0, AE π=4, AE π=2grad, with a path length of 1.4 m.Each direction is repeated five times, and the trajectories are recorded.
Figure 11 shows the resulting motions.The robot can reliably push the obstacle within an area of around 15 cm radius around the goal location.For our intended application, this accuracy is sufficient.

Navigation Among Movable Obstacles
In the final experiment, we tested the entire system within a realworld scenario.The objective is to navigate a distance of approximately 12 m from one end of a hallway to the other.Objects are randomly placed in the environment, and no direct path to the goal position exists, as shown in Figure 1.This scenario highlights the limitations of a standard planner, which would not be able to plot a path.
Like in simulation, we record a static map of the environment without any objects and use it for localization purposes.We selected two chairs, one table, and multiple boxes as obstacles.All of them fulfill the assumptions made earlier in Section 3.2.1 (holonomic entities with roughly isotropic friction).Some boxes are not labeled with ArUco tags and, therefore, are not visible to the camera system.Nevertheless, they are avoided by the mobile base during navigation and the object-pushing process.
To introduce a static object into the environment, we have increased the weight of the largest box to a level that exceeds the maximum pushing force f max = 18 N of the robot.The system must identify this box as static.
Key-frames of the experiment are shown in Figure 12.Planning and execution proceed as follows: the first obstacle that obstructs the path is a chair, as seen in Figure 12a.Once the camera detects it, it triggers a search by the NAMO planner.The planner simulates various pushing actions on the object and selects a leftward push as the most cost-effective action to create free space.The robot then approaches the chair with its hand and applies a preliminary test force to assess whether it can be moved.The relatively lightweight chair starts moving at a force of 4.6 N.With this confirmation, the robot executes the push and can continue navigation.
The next visible obstacle blocking the path is the table shown in Figure 12b.Following the same procedure, the planner opts for a push to the right.After checking that it can indeed be moved and locating the table's corner, the robot clears the path.
Then, two new objects appear in the field of view, Figure 12c.The planner attempts to first manipulate the large box since the goal is behind it.However, when applying a test force to the box, it will not move and is therefore classified as static.This triggers a replanning process, leading the robot to consider the chair as the next viable option, Figure 12d.Ultimately, the robot successfully reaches the goal by relocating the chair to the right.
Quantitative results from this experiment are listed in Table 2.As before, we show the total time spent in different planning phases and the real force required by the robot to remove obstacles.For comparison, we also replicated the corridor environment in simulation and rerun the experiment.Reality and simulation match the time taken for planning and navigation but differ greatly in accessing movability and pushing.This discrepancy arises because the movability attribute in simulation is determined instantaneously and because the real robot needs to perform additional actions, such as folding and unfolding its arms.

Conclusion
In this work, we tackled the challenge of enabling robots to navigate common human environments filled with unknown and potentially movable obstacles.The framework we developed effectively addressed this problem and allowed our robot to reach desired goals even if the path was obscured.
Instead of relying on given knowledge about an obstacle's manipulability, we use tactile signals from a robot skin system to actively sense movability.
Object movements are planned in a simulator before executing them on the real robot and only require approximate information about the object's location and a 2D footprint.
To remove blocking obstacles from the robot's path, we developed a pushing strategy that can realize the planner's action on a real robot.The method employs proximity information to reach for unknown obstacles and detect their corner location.Then, a feedback controller based on tactile information pushes the obstacle aside, creating space for further navigation.During pushing, the system can identify collisions with other static items of the environment and unexpected object motions to trigger a second attempt.Our simulation experiments show that the NAMO approach substantially increases the chance of success over standard "avoid at all costs" navigation.In simulated environments where ≈75% of the map is occupied, NAMO was still able to reach the goal in 80% of the cases.
Table 2. Timings real experiment: planning and execution times of the NAMO experiment on the real robot in a 12 m Â 3 m corridor.Note that the time for assessing and pushing during real execution also includes the time for unfolding and homing of the robotic arm and is, therefore, much longer than simulation.

Figure 1 .
Figure 1.Navigation among movable obstacles: the robot autonomously navigates from the initial pose to the goal pose, negotiating movable and nonmovable obstacles along the way.

Figure 2 .
Figure2.Overview: the goal is to find a sequence of compliant actions that activity creates clearances for navigation.An onboard camera detects new obstacles and adds them to a movability map.Subsequently, a NAMO planner decides which obstacle to move next and in which direction.The robot approaches the selected object and assesses its movability.If it can be moved, a pushing strategy clears it from the path, thereby allowing the robot to proceed further toward the goal.This process repeats iteratively until the goal is reached, or all known obstacles have been tested, and still no path is found.

Figure 3 .
Figure 3. Obstacle plan: a plan between the initial robot pose and the goal pose w x g involves three segments.Reaching the blocking object (c 1 ), clearing it (c 2 ), and finally moving further toward the goal (c 3 ).

Figure 4 .Figure 5 .
Figure 4. Mobile-base placement: the distribution of possible base poses is visualized as colored spheres on the ground.The blue color indicates a higher performance index.

Figure 6 .
Figure 6.Detection process of movability attribute: object 1 in a), b) is classified as movable after a displacement of Δx o ¼ 0.05 m is detected in b).The maximum recorded force before the object starts moving is f push = 7 N. Object 2 in c) d) is classified as static after the maximum effort of f max = 18 N is reached in c).The slight displacement in d) is due to a deformation of the object.

Figure 7 .
Figure 7. Obstacle pushing: clearing the path is achieved by first rotating a) and then pushing an object forward b) until the path is cleared.This approach leads to the trajectories depicted in c) when pushing at 45°to the left.The mobile base is colored red, and the object is green.

Figure 8 .
Figure 8. Collision situation: a pushed object accidentally collides with the static environment at 8 s.The compliant pushing controller limits the exerted force f c to a maximum of f max ≈9 N a).The corresponding object speed is shown in b).

Figure 9 .
Figure 9. Simulation: four successive key-frames of the random obstacle simulation.The robot discovers objects within its vicinity and pushes them away to reach the goal.Objects that the planner has moved are displayed in blue.

Figure 11 .
Figure 11.Obstacle clearance: resulting trajectories when clearing obstacles in front of the robot.In this experiment, pushes are executed five times in the directions {0, AEπ/4, AEπ/2}rad.

Figure 12 .
Figure 12.Experiment: key-frames of the real execution.The task is to autonomously reach the opposite side of the hallway.

Table 1 .
Simulation timings: total execution times for NAMO planner when executed in environments with increasing the number of objects.The planning time scales linearly with the number of obstacles, suggesting that the strategy could be used for even larger maps with more obstacles.