Tasks
Peg insertion
The pegs are prisms with varying cross-section, size, aspect ratio, mass, and length, with 1.0–1.5 mm of clearance per side. Pegs typically start around 2 cm from the hole, tilted far more than the clearance admits. The task succeeds when the peg is seated in the hole.
Gear meshing
Spur gears of varying size and tooth count must slide down a shaft into mesh with one or two anchor gears that turn on their own axles, with 0.2–0.6 mm of backlash. Episodes start with the gear in contact with the shaft tip, but misaligned and with its teeth out of phase with the anchors’. The task succeeds when the gear is seated on the shaft with its teeth meshed.
Nut threading
Metric nuts from M24 to M42 must thread onto a bolt with 0.05–0.2 mm of thread clearance. Episodes start with the nut resting on the bolt’s tip, off its axis and tilted. The task succeeds a quarter turn after the thread catches.
Simulating Tight Insertion at Scale
Training on this distribution means simulating thousands of parallel worlds, each containing a different, randomly generated assembly, with per-side clearances down to 0.5 mm. MuJoCo collides meshes through their convex hulls, so a concave hole must be given an SDF or built from convex pieces. Simulation benchmarks assemble these from boxes by hand.
Results
The experiments are designed to answer the following questions:
- Can a policy that senses only its own joints learn precise assembly tasks, including peg insertion, gear meshing, and nut threading?
- Do blind policies learn intelligent, contact-reactive search, or do they learn scripted patterns?
- Do blind policies generalize better than visual policies to part geometries outside the training distribution?
- Can blind policies that rely on contact-rich search be transferred zero-shot with high fidelity to the real world?
Proprioception Is Sufficient
All three tasks achieve near-perfect success on in-distribution geometries. The joint history alone therefore carries enough information to find an opening, bring teeth into mesh, and engage a thread.
Compared with visual policies, which can typically observe the poses of the grasped and mating parts directly, blind policies may be slower. On peg insertion, the blind policy has to search for the opening through contact, so it takes a less direct path and seats later than a visual policy. The gap is smaller on the other two tasks, plausibly because the grasped part occludes more of the mating part from the camera’s view, meaning the visual policy also has to search to some extent.
Generalization to Unseen Parts
On all three tasks, both blind policies and visual policies are reliable on the parts they were trained on. However, the blind policy is more reliable on assemblies sampled from outside the training distribution.
The visual policy is particularly vulnerable to geometry changes that significantly change how the parts look. It consistently underperforms the blind policy across tasks when evaluated on parts that have a different shape or are extremely different in size from the training distribution. The visual policy learns to align visual features that may not appear in held-out assemblies. The blind policy, on the other hand, learns a searching strategy that is much more transferable.
Limitations and Conclusion
This research was financially supported by the Ministry of Trade, Industry, and Energy (MOTIE), Korea, under the “Global Industrial Technology Cooperation Center program” supervised by the Korea Institute for Advancement of Technology (KIAT). (Grant No. P0028435)
Frequently Asked Questions
What sensing information does the blind policy use? It senses its own joints; the paper reports that joint history alone carries enough information for the three assembly tasks.
Which assembly tasks are studied? The tasks are peg insertion, gear meshing, and nut threading.
How does the blind policy perform on parts outside the training distribution? It is more reliable than the visual policy on assemblies sampled from outside the training distribution.
Why might visual policies struggle with unseen parts? They learn to align visual features that may not appear in held-out assemblies.