We present a hierarchical reinforcement learning framework that enables quadruped robots to manipulate objects using their feet — a capability known as pedipulation. By using affordances as mid-level representations that identify where and how to interact with an object, our method reduces the need for expensive full-body motion planning while improving generalization across different objects and terrains.
Introduction
Legged robots have traditionally focused on locomotion, leaving manipulation tasks to fixed-base arms. Pedipulation — manipulating objects with the feet — offers a natural way for quadrupeds to interact with their environment without sacrificing mobility. However, existing approaches require precise modeling of the robot-object dynamics and often fail to transfer from simulation to the real world.
The Affordance-Based Approach
We introduce a hierarchical reinforcement learning architecture that operates at two levels. A high-level policy selects which affordance to execute — such as pushing left, pushing right, or pulling — while a low-level policy learns the precise foot trajectory for that interaction. Affordances are defined as regions on the object surface that afford a specific action, identified using a deep neural network trained on interaction data.
Learning Framework
The high-level policy receives the robot's state and object pose, and outputs a categorical action corresponding to one of five interaction modes: idle, push-left, push-right, pull-left, or pull-right. The low-level policy then generates joint-level motor commands to execute that mode. This two-level structure allows the system to reuse learned low-level skills across different objects and environmental conditions.
Simulation Results
We evaluated our method in simulation across multiple object types (boxes, cylinders, and irregular shapes) and surface friction conditions. The affordance-based approach achieved 87% task success rate compared to 62% for a baseline that used fixed interaction points, while requiring 40% fewer training steps to converge.
Real-World Transfer
To validate sim-to-real transfer, we deployed the learned policies on a Unitree Go1 quadruped robot. The robot successfully displaced objects weighing up to 1.5 kg across various floor surfaces including carpet, tile, and concrete. We observed a sim-to-real success rate of 73%, with most failures attributable to unmodeled friction dynamics and ground irregularities.
Comparison with Alternatives
Our hierarchical approach outperformed two baselines: a non-hierarchical reinforcement learning method that struggled to discover effective manipulation strategies, and a model-based controller that required precise object tracking and failed under occlusions. The affordance-based method maintained robust performance even when object pose estimates were corrupted by up to 5 cm of noise.
Experiments
Since existing pedipulation studies are not publicly available, we benchmark our approach under a variety of conditions to evaluate its effectiveness. We first compare the displacement achieved per physical interaction across different interaction directions and demonstrate, in simulation, the advantages of selecting an appropriate contact point on the obstacle surface. We then extend our evaluation to real-world experiments to assess both sim-to-real transfer performance and the effectiveness of the complete pipeline under non-ideal operating conditions.
Conclusions
For future work, we aim to leverage the collected interaction data to learn object-centric representations that can be used to improve downstream manipulation and in-situ utilization of objects.
Frequently Asked Questions
How does the affordance-based approach differ from standard reinforcement learning for pedipulation? Instead of learning a single monolithic policy, our method uses a hierarchical structure where a high-level policy selects interaction modes and a low-level policy executes foot trajectories, making learning faster and more transferable across objects.
What types of objects can the robot manipulate? The system was tested on boxes, cylinders, and irregularly shaped objects weighing up to 1.5 kg across various floor surfaces including carpet, tile, and concrete.
How well does the method transfer from simulation to the real world? We achieved a 73% sim-to-real success rate, with most failures caused by unmodeled friction dynamics and ground irregularities rather than fundamental policy errors.
Why use affordances rather than precomputed interaction points? Affordances allow the robot to dynamically select optimal contact points on the object surface, achieving 87% task success compared to 62% for fixed interaction points, while using fewer training steps.