Hardware: A Shared Exoskeleton
The hardware of SEED-UMI matches the contact-facing exoskeleton geometry and encoder frame on the human and robot sides, while adapting cuffs and mounts to each embodiment. This makes encoder readings a shared cross-embodiment measurement: discrepancies under the same command supervise the residual command-to-joint mapping.
Exoskeleton Mechanism Design
The exoskeleton is co-designed with a single fully actuated dexterous hand as its sole target platform. Because cross-platform transferability is not pursued, link lengths, joint axes, and range-of-motion limits can be fixed to directly replicate the robot’s kinematic tree. This enables two stronger properties: one-to-one joint correspondence on both sides of the device and contact geometry that is identical between the human-worn and robot-worn instances.
The skeleton has one revolute joint per active degree of freedom of the target hand, with link lengths matched to the robot’s phalanx lengths and joint axes oriented identically. Hard stops mirror the robot’s range-of-motion limits, so configurations the operator can reach are exactly those the robot can reach. The exoskeleton instantiates 20 independently measured joints across five fingers.
Sensor Integration
All sensing is mounted on the exoskeleton frame so that every signal is recorded in the same physical frame regardless of which embodiment wears the device. Two design objectives drive the sensor layout:
- Capture the action and observation channels needed for downstream policy learning.
- Minimize the distribution shift of those channels between the human-side and robot-side.
Experiment Setup
The platform consists of a RealMan RX75 robot arm and a Wuji dexterous hand with four actuated degrees of freedom per finger, 20 in total. The human and robot wear the shared exoskeleton, preserving the outer contact geometry and camera placement across embodiments.
A dorsal Intel RealSense T265 provides six-degree-of-freedom wrist pose, while a ventral fisheye camera with a wide field of view supplies policy observations. Wrist trajectories use a fixed T265-to-tool-center-point alignment without trajectory correction.
Limitations
Although SEED-UMI improves the efficiency of dexterous demonstration collection, several limitations remain. The current system is not yet fully automated across dexterous hands: adapting to different link layouts, joint limits, and actuation ranges still requires engineering choices in exoskeleton geometry, calibration motions, and paired regression targets.
Because operators wear the exoskeleton during collection, wearability, donning time, and task convenience also affect long-horizon and delicate manipulation. Future work will automate cross-hand design and mapping, improve the wearing experience, and incorporate tactile sensing for more accurate contact estimation.
Frequently Asked Questions
What is SEED-UMI designed to do? SEED-UMI shares an exoskeleton between a human and a robot to support one-to-one dexterous demonstrations.
How many joints does the exoskeleton measure? It measures 20 independently measured joints across five fingers.
Why are the human-side and robot-side exoskeletons mechanically matched? Matching their contact geometry, encoder frame, joint correspondence, and camera placement minimizes cross-embodiment distribution shift.
What are the main limitations of the current system? Different dexterous hands still require engineering for exoskeleton geometry, calibration, and regression targets, while wearability affects long-horizon and delicate manipulation.