Researchers from ETH Zurich and Disney Research have developed CoCo-InEKF, a state estimation system that lets bipedal robots accurately track their position and velocity during highly dynamic motions like dancing. Unlike traditional methods that rely on binary contact detection, CoCo-InEKF learns to predict continuous contact covariances, enabling robust performance even under slippage and aggressive maneuvers.
What the Researchers Built
The team created CoCo-InEKF (Contact Covariance-Invariant Extended Kalman Filter), a proprioceptive state estimator that fuses data from an inertial measurement unit (IMU) and joint encoders without relying on external cameras or LiDAR. The core innovation is a neural module that predicts continuous contact velocity covariances—essentially how uncertain the robot should be about each foot's contact—rather than a hard binary on/off classification.
This architecture is trained end-to-end via backpropagation through time, meaning the neural network and the filter parameters are optimized jointly to minimize state estimation error. The system was tested on the Lima bipedal robot, a 20-degree-of-freedom platform designed for dynamic locomotion. CoCo-InEKF also includes an automated method for selecting contact candidates, removing the need for an expert to manually define contact points.
Key Results
CoCo-InEKF was evaluated against several baselines, including classical Invariant Extended Kalman Filters (InEKFs) with binary contact classification, end-to-end supervised learning estimators, and traditional complementary filters. On the Lima robot performing dynamic dancing motions (foot contact only) and ground motions (full-body ground contact), CoCo-InEKF consistently achieved lower estimation errors for the core filter states—position, orientation, and linear velocity.
In ablation studies, the neural module's continuous covariance predictions significantly outperformed hand-tuned fixed covariances and binary contact heuristics, especially during slippage and rapid contact transitions. The automated contact candidate selection performed on par with expert-handpicked sets, demonstrating the method's practical usability. The entire estimation pipeline runs within the robot's 1 kHz onboard control loop, matching the efficiency requirements for real-time feedback control.
How It Works
CoCo-InEKF is built on a differentiable Invariant Extended Kalman Filter that respects the Lie group structure of the robot's state—position, orientation, and linear velocity—ensuring mathematically consistent updates. The neural module takes as input the current InEKF residual (innovation), along with the robot's joint positions, velocities, and IMU readings. It outputs a covariance matrix for each contact candidate's velocity, expressing how reliably that foot is expected to stick or slip on the ground.
During training, the entire filter is unrolled in time and optimized via backpropagation through time (BPTT) against ground-truth states from simulation. Domain randomization of friction coefficients and disturbance forces ensures generalization to real-world conditions. The filter fuses IMU angular velocity and acceleration with contact velocity corrections. When a contact is expected to be stable, the filter trusts the zero-velocity pseudomeasurement; when slippage is likely, the covariance is larger, allowing the filter to rely more on the IMU.
The automated contact candidate selection algorithm identifies potential foot contact points based on the robot's kinematic model and geometry, then creates a set of candidates that cover the expected contact surfaces. This eliminates the manual tuning required by previous methods.
Why This Matters for Robotics
Accurate state estimation is the foundation for all robot control—especially for humanoid robots and bipedal platforms that must balance, run, and interact with unstructured environments. Current proprioceptive estimators often fail during aggressive motions or on slippery surfaces, limiting the deployment of legged robots outside controlled labs.
CoCo-InEKF's ability to handle dynamic, contact-rich scenarios without external sensors opens the door for more agile robots in inspection, search and rescue, and entertainment. The automated contact selection also reduces engineering overhead when adapting the system to new robot morphologies. For operations managers evaluating used cobots or industrial robots, the underlying learning approach could eventually transfer to other contact-rich manipulation tasks beyond locomotion.
Limitations and Open Questions
CoCo-InEKF was trained exclusively in simulation, though the authors note it can be fine-tuned with real-world motion capture data. The current method does not incorporate exteroceptive sensors like cameras or LiDAR, so it cannot correct for long-term drift—an important direction for future work.
The neural module's performance depends on the diversity of training scenarios; rare or extreme contact states may degrade estimation quality. Additionally, the method was validated only on the Lima bipedal robot; generalizing to quadrupeds or humanoid robots with different kinematics and contact patterns requires further study.
Frequently Asked Questions
How does CoCo-InEKF differ from standard Kalman filters for robots? It uses a neural network to predict continuous contact covariances instead of relying on binary contact states, and the entire filter is trained end-to-end via backpropagation through time.
Does CoCo-InEKF require cameras or external sensors? No, it is purely proprioceptive—it only uses an IMU and joint encoders, making it suitable for visually degraded environments or tasks where carrying cameras is impractical.
Can CoCo-InEKF be used on robots other than bipedal ones? The framework is designed for legged robots generally, but so far it has only been tested on the Lima bipedal robot. The automated contact candidate selection is intended to help with other morphologies.
What hardware is needed to run CoCo-InEKF in real time? The filter runs within a 1 kHz control loop on the Lima robot's onboard computer, which is typical for mid-range embedded systems used in modern legged robots.
Conclusion
CoCo-InEKF demonstrates that learning continuous contact covariances within an invariant extended Kalman filter can dramatically improve state estimation during dynamic, contact-rich motions. This approach offers a practical path toward more robust and agile legged robots without adding sensor cost or complexity.