Humanoid robots offer a general-purpose embodiment for working in environments built for people. Many tasks in these environments, such as pushing a cart, moving a heavy box, and wiping a surface, require control of contact force as well as motion. Learned humanoid loco-manipulation is commonly formulated as generating and tracking motion references. Tracking a demonstrated hand trajectory alone does not reproduce the demonstrated force, which also depends on contact geometry, stiffness, and object motion. Unlike a fixed-base arm, a humanoid must stay balanced under these contact loads, coordinating its hands, torso, and legs while regulating force.
Force-Reactive Policy and Whole-Body Control
The high-level planner uses a force-reactive diffusion policy. It observes wrist images, robot state, and the measured wrench at each hand, then predicts a chunk of action, gripper widths, and per-hand desired force. Force enters as both an input and an output. The input lets the policy react to interactions that images do not reveal. The output is commanded to the controller, rather than used only as an auxiliary target. This connects the force measured during demonstration to the deployed force.
To track the commanded forces, we develop a whole-body controller with hybrid compliance. It outputs only joint positions, so each hand’s force command becomes a shift of its position target. In contact, an integral of the force error measured at the wrist adjusts this shift until the hand exerts the commanded force, whatever the contact stiffness. Along uncommanded axes, the same integral lets the hand yield to external pushes. Each hand thus tracks force along the commanded direction and complies along the others, without switching modes. We train the controller in simulation and deploy it on the robot using wrist force/torque measurements.
We deploy FLoRe on a Unitree G1 and evaluate three whole-body contact-rich tasks: pushing a gantry hanging another robot, bimanual box rotation, and board wiping. In simulation, the controller realises in-contact force error of about 1 N per hand while tracking whole-body motion. On the real robot, FLoRe succeeds in 80% of rollouts, compared with 26.7% for a position-only policy trained on the same demonstrations and executed by the same controller with force commands disabled. Withholding only the force command reduces success to 36.7%, while removing only the force observation reduces it to 66.7%. Both contribute to the final performance.
The key contributions are:
- A robot-free pipeline for learning force-reactive humanoid loco-manipulation from demonstrations that record both motion and contact wrench.
- A learned whole-body controller that tracks per-hand force together with keypoint motion while remaining compliant to external forces and maintaining balance.
- Evaluation in simulation and on three contact-rich tasks on a Unitree G1, where force commands and force observations both improve task success over position-only control.
Force-Annotated Robot-Free Data Collection
Streams and synchronization
Images, tracker poses, wrenches, and gripper widths arrive at different rates and are stamped on arrival with one host clock; the labels below are built from the raw streams, not the camera grid. Missing wrench readings and tare transients are masked rather than zero-filled, since a tared sensor also reads zero in free space.
Push Gantry
Task
The robot places both hands on a heavy wheeled gantry, from which a second humanoid hangs, and walks it forward to a target distance. The demonstrated hand motion does not specify how hard to push, yet the gantry does not move until the push overcomes its static friction. This is the only task recorded with ankle trackers, so the policy predicts all five keypoints. A rollout succeeds if the gantry reaches the target distance without a stop.
Capabilities
- Bimanual and whole-body coordination: The legs and torso produce the push while the two hands balance their forces; otherwise the gantry drifts from its heading.
- Sustained contact: Both hands must keep contact as the gantry accelerates and the robot steps.
- Robustness to force variation: The hanging humanoid swings as the gantry rolls, so the resistance changes over time.
Test scenarios
We run 10 rollouts per policy over two gantry loads and five initial poses that vary in distance and heading.
Rotate Box
Task
The robot pinches the rim of a box with both hands. It rotates the box by an unspecified angle, drags it about 0.5 m across the table, and releases it. The box remains on the table, so the measured wrench reflects table friction, box inertia, and the internal squeeze between the hands. A rollout succeeds if the box rotates by more than an unspecified threshold angle and remains upright after release.
Capabilities
- Bimanual coordination: The hands share an internal force and rotate about a common axis. In the demonstrations, the right hand applies about an unspecified multiple of the left hand’s force. If one hand leads the other, the box tips.
- Sustained contact: The pinch must hold through the drag and the yaw without slipping or lifting the box.
- Robustness to force variation: Friction changes with the box mass and when the box begins to slide.
Test scenarios
We run 10 rollouts per policy over two box masses and five initial yaw angles.
Performance
FLoRe succeeds in 9/10 rollouts and has an offline pose error of 0.70 cm. DP succeeds in 3/10 and exhibits two failure modes. It either fails to drag the heavy box or leans into it and loses balance. Without force commands, the policy succeeds in 4/10 rollouts and fails similarly because its predicted squeeze is not executed. Without force input, it succeeds in 7/10 rollouts but cannot react when the box begins to slide.
Wipe Board
Task
The robot presses an eraser against a whiteboard and wipes a marked strip. Success depends on normal force, which is not visible in the images. A rollout succeeds if the strip is erased without a stop.
Capabilities
- Sustained contact: The robot must maintain normal force during each stroke and re-establish it after losing contact. Images alone do not distinguish hovering from pressing.
- Robustness to force variation: We displace the board by an unspecified distance in centimetres along its normal. The demonstrated trajectory then produces either no contact or excessive force if tracked directly.
Test scenarios
We run 10 rollouts per policy over two board offsets, an unspecified distance in centimetres along the normal, and two surfaces.
Findings Summary
Force commands give the larger, significant gain; force observations show a smaller trend. Withholding the predicted force from the controller reduces success from 80% (24/30) to 36.7% (11/30; Fisher’s exact test, an unspecified test statistic or p-value): the policy senses resistance but can respond only through pose. Removing the force observation reduces success to 66.7%.
Conclusion
FLoRe is a framework for learning force-reactive humanoid loco-manipulation from robot-free demonstrations that record both motion and contact wrench. Its hybrid-compliance whole-body controller tracks the learned force commands, allowing the humanoid to sustain contact, coordinate both hands, and apply force while walking and balancing. On three tasks, the force-reactive policy succeeds in 80% of rollouts, compared with 26.7% for a position-only policy. By making force a shared interface between demonstrations, policies, and whole-body control, FLoRe extends robot-free imitation learning to contact-rich skills that motion alone cannot specify.
Frequently Asked Questions
What does the FLoRe policy observe and predict? It observes wrist images, robot state, and measured wrench at each hand. It predicts action chunks, gripper widths, and desired force for each hand.
How does the controller track force while remaining compliant? It shifts each hand’s position target using an integral of wrist-measured force error. Along uncommanded axes, the hand can yield to external pushes.
Which tasks were evaluated on the Unitree G1? The tasks were pushing a gantry with a hanging humanoid, rotating a box with both hands, and wiping a board.
How did force commands and observations affect success? FLoRe achieved 80% success; disabling force commands reduced success to 36.7%, and removing force observations reduced it to 66.7%.