Scaling robots to a wider range of real-world tasks requires a way for their policies to keep improving after deployment, without relying on continual human guidance. New objects, changes in the scene, and small execution errors can all expose weaknesses in a task policy. Simply making more attempts does not provide the experience needed for improvement, as a robot can repeat the same mistake without discovering the right fix. While humans can supply such fixes, asking for a new demonstration or correction whenever the robot encounters a failure bottlenecks progress to supervision. We therefore seek an improvement process that generates its own corrective experience and makes initial teaching useful beyond a single task. Such a process would reduce both the supervision needed to improve a task policy and the additional human effort needed for generalization to new tasks.
Related Work
We contextualize skill-space shooting against three main bodies of work, ranging from skill composition, to learning from corrections, and finally to autonomous improvement from experience.
Method
Skill-space shooting uses learned skills to try corrections where a task policy is likely to fail, then trains the policy on successful repairs. A skill can supply a useful local behavior even when the policy cannot yet complete the whole task. To turn that capability into supervision, the loop must recognize when help is needed, test a correction from the learner’s current state, and preserve the actions that resolve the difficulty. The rest of this section describes how the models identify failure to initiate these shooting trials, how repairs become supervision over successive iterations, and how a shared skill library supports improvement on new tasks.
Deciding When and How to Shoot
For simplicity, each task stage has a designated primitive. In practice, identifying the stage that needs recovery or correction also determines exactly which skill to shoot. For scalability, the task and skill descriptions are inputs to a fixed prompt, allowing the same selection procedure to apply across tasks.
Repairing Rollouts and Improving Policies
Shooting a skill
To test a correction behavior, the selected skill takes control where the task policy stopped. Earlier errors may have changed object positions or contacts, so the skill must respond to the state the task policy actually reached. This also makes the resulting actions useful supervision: a fresh demonstration from a clean starting state may never encounter the same difficult failure state. Once the stage is repaired, control returns to the task policy so it can attempt the remaining stages. Returning control lets the task policy use the behaviors it already performs well and encounter further difficulties that may need correction within the same rollout.
Learning from repairs
The repaired rollout contains actions for the task policy’s failure states, but also the mistakes that led there. Training on the full sequence would teach the policy to imitate both. We therefore retain only accepted, skill-executed repair segments from rollouts the judge confirms as complete. Each segment contains observations and executed actions from invocation to handback, relabeled with the task instruction. We exclude the policy’s actions and rejected trials.
The updated policy then collects the next round of experience. This iteration helps the task policy learn corrections and exposes a wider range of failures. For example, a policy that now completes an early grasp may reach a placement stage that previous attempts never reached. Iteratively collecting with the updated policy brings these newly encountered states, along with remaining failures, to the same skill library. The skills thus provide new supervision as the learner progresses, without needing another round of demonstrations.
This process remains autonomous: the models decide when to intervene, which corrections to shoot, and whether to accept them. It is also scalable and minimizes human involvement: skills and the judge prompt remain fixed, as does the value model after its initial calibration, so human input is limited to initial demonstrations and scene resets.
Reusing the Skill Library
Why skills?
For the improvement loop to work, corrections must be meaningful and identifiable choices for the judge, as well as executable behaviors for the robot. A short-horizon skill satisfies both conditions: its identity—visual and textual descriptions—describes a clear local correction, while its policy supplies the coordinated actions needed to attempt it. For example, selecting a grasp invokes an approach, gripper closure, and lift, with the skill responding to observations along the way. The foundation model can therefore reason about the behavior needed to repair a stage while the skill handles its physical execution.
Teaching each skill separately also lets it learn this shorter behavior without having to master the entire task, which is a far easier learning problem.
Sharing across tasks
Because these behaviors recur across tasks, their training data can help prepare corrections for a new task. For example, grasping a new object may require adapting to its shape while retaining the approach-and-lift behavior learned from earlier objects. To this end, we combine existing skill data with new demonstrations to train the adapted primitive, and show that this reduces the need to teach the entire motion primitive from scratch. The new task has its initial policy and value model, but uses the same procedure to invoke skills, check repairs, and learn from them.
Experiments
Our experiments answer three questions about skill-space shooting as a source of reusable corrective supervision:
- Do skill-based corrections improve the task policy over successive iterations?
- Can the system autonomously invoke skills and verify their repairs reliably?
- Can a shared skill library transfer across distinct tasks, reducing the human effort needed to prepare new skills?
Our four main tasks require a range of corrective behaviors. Stack-3 stacks three cubes one on top of another in sequence, while Drawer requires opening a drawer, placing a roll of tape inside, and closing it. Coffee pours milk, coffee, and ice, and adds a stirring stick to a cup. Finally, Sweeping retrieves a small handheld broom, sweeps six blocks into a target zone, and rehangs the broom. Together, these tasks test corrections within ordered sequences and during dynamic contact.
Sharing and Extending Skills for New Tasks
With only 10 new demonstrations, sharing raises grasp success from 5% to 70%. With 30 demonstrations, sharing matches the success obtained from nearly double the number of demonstrations without sharing. Prior skill learning therefore does more than improve performance at a fixed data budget: it reduces the new demonstrations needed to prepare the same skill for an unfamiliar object or motion primitive.
Limitations and Future Directions
Acquiring and expanding skills
Repeated improvement with a fixed library shows that skills can remain useful after the policy learns from earlier corrections, but specifying and teaching them still requires initial human effort. Foundation models will increasingly help decompose tasks and identify reusable behaviors. Persistent failures could guide which skills to acquire next, with each addition evaluated by how many tasks it benefits. Learning skills from human video could further reduce demonstration effort as the library expands.
Sustained autonomous operation
Although the improvement loop is autonomous, people still reset scenes between rollouts. Learned reset behaviors or task loops, where completing one task prepares the next, could enable unattended operation over days or weeks on open-ended tasks. Combining them with skill-space shooting would allow sustained collection and test whether corrective supervision remains useful as tasks and surroundings change.
Generalizing failure detection
The failure detector requires per-task training, though skill selection and repair verification share a common procedure. Foundation models already detect execution failures, track task progress, and locate events in robot video. As their temporal reasoning improves, extending these capabilities to anticipate failure from proposed motions could remove the need for a separate detector. A shared model would then decide both when to shoot and which skill to try, reducing preparation for new tasks.
Conclusion
We introduce skill-space shooting to make reusable short-horizon robot behaviors a renewable source of corrective supervision. Foundation-model guidance identifies meaningful behaviors to shoot at a task policy’s failures, and monitors their physical rollout to test whether they provide a correction. Training on accepted repair segments from completed rollouts then incorporates those corrections into the task policy.
Our experiments show repeated gains that persist without skill assistance, including improvement from a policy with zero observed initial successes. We also show that sharing skill data reduces the new teaching needed to prepare another task’s improvement loop, connecting improvement within a task to reuse across tasks. Skill-space shooting thus provides a foundation for autonomous improvement at scale: reusable skills let robots learn from their own failures and extend that learning across tasks, without requiring humans to teach every possible correction.
Frequently Asked Questions
What is skill-space shooting? It uses reusable short-horizon skills to repair task-policy failures, then trains the policy on accepted repairs.
How does the system decide which skill to use? Each task stage has a designated primitive, so identifying the stage needing correction determines which skill to shoot.
Which parts of a repaired rollout are used for training? Only accepted, skill-executed repair segments from rollouts confirmed as complete are retained; policy actions and rejected trials are excluded.
How does sharing skills reduce teaching effort for new tasks? Existing skill data is combined with new demonstrations to train an adapted primitive, reducing the number of new demonstrations needed.