An imitation-learned dexterous robot policy matched its scripted expert at normal speed but fell to 53% success at the fastest demonstrated speed, while the expert held 84%. The ParcelStow study shows that nominal success can hide a serious timing problem: demonstrations can teach what to do without preserving how reliably the task works when execution accelerates.
What Did the Researchers Build and Test?
The researchers created a controlled comparison between a scripted robot expert and an imitation-learning policy on ParcelStow, a contact-rich dexterous manipulation task. The robot must acquire a parcel, reorient it, transfer it through free space, and insert it into a target. Each stage combines arm motion, object handling, timing, and contact with the environment.
The learner used Action Chunking with Transformers, or ACT. Instead of predicting only the next motor command, ACT predicts short sequences of actions, called chunks, from demonstrations. This approach can produce smoother behavior and is widely used for learning manipulation skills from example trajectories.
The expert generated the demonstrations used to train ACT. The evaluation then applied the same task conditions, initial-condition draws, and speedup factors to both systems. The manipulation phases after parcel acquisition were performed at different speeds, including speeds represented in the demonstrations.
This design isolates a practical question: does copying successful behavior at one speed also preserve reliability at other speeds? The comparison avoids treating the expert and learner as separate systems tested under unrelated conditions. It instead measures how much performance changes when both face the same timing challenge.
What Were the Key Results?
Both the scripted expert and the ACT learner achieved 100% task success at nominal speed. That result alone made the two systems appear equivalent, but the gap widened as execution accelerated. At the maximum demonstrated speed, expert success was 84%, while the reported ACT success rate was 53%.
Two ACT policies trained with different parameter initializations showed similar degradation. Their success rates dropped by 34 and 48 percentage points from nominal speed to the maximum demonstrated speed, compared with a 16-point decline for the expert.
The failures were concentrated in a specific part of the task rather than distributed evenly across all phases. At the maximum demonstrated speed, 35 of the ACT policy’s 47 failures involved insertion misalignment. The learner often handled the parcel through earlier motion but failed to place it accurately enough for the final contact-rich insertion.
The acquisition analysis exposed another important dependency. Under the relative-motion handoff, every ACT acquisition retained the parcel through reorientation and free-space transfer, yet only 64% completed the full task. After expert acquisition, 95% completed the task. Across every evaluated policy and speed, all 414 acquisitions without force closure failed to complete the task.
| Evaluation measure | Scripted expert | ACT learner |
|---|---|---|
| Nominal-speed task success | 100% | 100% |
| Maximum demonstrated-speed task success | 84% | 53% |
| Success decline from nominal to maximum speed | 16 percentage points | 34 and 48 percentage points across two initializations |
| Completion after successful acquisition under relative-motion handoff | 95% | 64% |
How Did the Expert and Learner Work?
ParcelStow separates manipulation into phases with different control demands. Acquisition requires the robot to establish a reliable grasp. Reorientation changes the parcel’s pose, transfer moves it through free space, and insertion brings the object into contact with a target where small position or orientation errors can cause failure.
The evaluation changes the timing of the phases after acquisition by applying speedup factors. This tests temporal robustness: the ability to maintain task performance when the same movement is executed faster or slower than the nominal behavior. The comparison is especially relevant to learned policies because their outputs depend on observed motion histories and the timing of state changes.
The scripted expert follows predefined behavior rather than learning from demonstrations. Its 16-point decline at the fastest demonstrated speed provides a baseline for how the task itself responds to acceleration, including physical effects such as contact timing and insertion tolerance.
ACT learns from the expert’s demonstrations. Its transformer-based policy observes the task and produces action chunks, giving the robot a short-horizon sequence of commands. Action chunking reduces the need to generate every command independently, but it does not automatically guarantee that the chunks remain synchronized with faster object motion or contact events.
The stage-level results suggest that acquisition quality is a major predictor of final success. Force closure means the grasp constrains the object securely enough to resist slipping or unwanted motion. Every one of the 414 acquisitions without force closure failed later, showing that an apparently completed pickup is not equivalent to a mechanically stable grasp.
Even when the learner retained the parcel during reorientation and transfer, insertion remained vulnerable. The handoff results separate “the object stayed in the gripper” from “the entire task succeeded,” which is essential for evaluating real manipulation systems.
Why Does Temporal Robustness Matter for Robotics?
Robot buyers and deployment teams often rely on nominal task-success numbers. A policy that completes every test at one speed can look production-ready, yet real operations rarely run at a single fixed tempo. Conveyor variation, changing cycle-time targets, sensor delays, operator interventions, and hardware wear all alter the timing relationship between the robot and its environment.
The ParcelStow results show why speed testing should be part of acceptance testing for learned manipulation. A learner can reproduce the broad shape of an expert trajectory while losing accuracy at the moment of contact. For parcel handling, bin picking, kitting, and assembly, that can create jams, dropped objects, retries, or downstream quality problems.
The study also points toward better diagnostics. Overall success should be paired with stage-level metrics for grasp stability, object retention, reorientation, transfer, and insertion. Force-closure checks could identify weak acquisitions before the robot commits to the next stage.
For teams evaluating used industrial robots, the lesson applies beyond the arm itself: the robot, gripper, controller, and learned policy must be assessed as one timed system. In warehouse settings, warehouse robots also need tests that cover changes in throughput and motion timing, not only nominal demonstrations.
What Are the Limitations and Open Questions?
The comparison covers one contact-rich task, one scripted expert, and ACT policies trained from that expert’s demonstrations. The findings establish a clear speed-related gap in ParcelStow, but they do not show whether the same pattern appears across different objects, grippers, insertion geometries, robots, or imitation-learning algorithms.
The abstract reports success rates and stage-level failure counts but does not provide detailed trajectory, controller-frequency, latency, force, or object-geometry analyses. Those measurements could distinguish whether failures come from action-chunk timing, perception delay, gripper mechanics, contact dynamics, or insufficient demonstration diversity.
The demonstrated speed range also matters. A learner could be robust within one range and fail outside it, while a policy trained with broader timing variation might behave differently. Future evaluations should compare speed augmentation, temporal resampling, force feedback, closed-loop replanning, and alternative policy initializations under the same controlled protocol.
Frequently Asked Questions
What is temporal robustness in robot manipulation?
Temporal robustness is the ability to maintain task performance when execution speed changes. It matters when robots must adapt to throughput targets, timing delays, or changing environmental motion.
What is ParcelStow?
ParcelStow is a manipulation task requiring a robot to acquire, reorient, and insert a parcel. It is contact-rich because the final insertion depends on accurate object pose and physical interaction with the target.
Why did the ACT learner fail more often at higher speeds?
The reported failures were dominated by insertion misalignments, with 35 of 47 failures at the maximum demonstrated speed occurring during insertion. The results indicate that the learner did not preserve the expert’s timing and placement reliability as execution accelerated.
Does 100% nominal success prove that an imitation policy is reliable?
No. Both systems reached 100% at nominal speed, but the learner dropped to 53% at the fastest reported speed, showing that nominal success does not establish robustness across execution conditions.
What Is the Conclusion?
The expert–learner comparison shows that imitation can preserve nominal task success without preserving temporal robustness. Dexterous manipulation policies therefore need evaluation across execution speeds, with stage-level checks that reveal grasp and insertion failures rather than relying on a single overall success rate.