RPG coordinates six specialized agents across three stages: Constructor, Runtime Agent, Privileged Agent, Video Analyzer, Implementor, and Merger. It first constructs simulation practice tasks from an offline dataset, improves a shared robot execution system by practicing these tasks, and ultimately deploys the resulting system on a physical robot. The execution system is formulated as Agent-as-Policy: a multimodal Runtime Agent follows a system prompt to complete each task by selecting and composing reusable skills from a shared skill library. These skills provide generalizable primitives for perception and robot control, and are constructed and refined through the Reconstruct and Practice stages.
Agent-as-Policy Setup and Initialization
RPG coordinates six specialized agents across its three stages: Constructor, Runtime Agent, Privileged Agent, Video Analyzer, Implementor, and Merger. It first constructs simulation practice tasks from an offline dataset, improves a shared robot execution system by practicing these tasks, and ultimately deploys the resulting system on a physical robot.
The execution system is formulated as Agent-as-Policy: a multimodal Runtime Agent follows a system prompt to complete each task by selecting and composing reusable skills from a shared skill library. The skills provide generalizable primitives for perception and robot control, and are constructed and refined through the Reconstruct and Practice stages described below.
Reconstruct: Build and Freeze the Practice Tasks
The Constructor first selects source tasks from an offline dataset to cover useful and diverse manipulation capabilities. For each selected source task, it chooses 3–8 trajectories and reviews their videos and textual descriptions. It identifies the manipulation capabilities represented by the source task and proposes a simulation task for practicing them. The agent has a 7.5-minute budget per source task.
Practice: Improve the System from Execution
Asymmetric actor-critic learning uses full simulator state to train a critic while restricting the actor to images. RPG instead uses privileged state in a separate execution to support failure diagnosis. With privileged state information, the Privileged Agent may complete some tasks more reliably than the Runtime Agent using the current skills. Its successful executions can guide improvements to the Runtime Agent, while failures shared by both agents can expose defects in the shared skills.
The Video Analyzer identifies the first observed failure and recommends a concrete change, supporting its diagnosis with frames or traces and describing the expected physical effect of the change. It does not modify the execution system. Its diagnosis and recommendation are passed to the Implementor for revision.
Go Real: Adapt and Freeze for Deployment
The final RPG system and CaP-Agent0 are adapted to the physical YAM robot before evaluation. Both receive the same coordinate calibration and hardware adaptation, including pad offsets and the empty-gripper aperture. No object- or task-specific tuning is applied.
Low reasoning effort is used for all physical evaluations to reduce inference cost and latency. Beyond these adaptations, no behavioral code changes are made based on physical feedback. The adapted systems remain fixed during evaluation. The Privileged Agent and Video Analyzer are not used during physical deployment.
Evaluation
The evaluation is organized around four questions:
- Q1: How does task success change with practice, and how does RPG compare with existing agents?
- Q2: How do the diagnostic inputs affect improvement?
- Q3: Can library changes improve task completion with the surrounding runtime fixed?
- Q4: Does the resulting system complete physical tasks after common adaptation?
Q1: Does RPG Improve with Practice?
Evaluation budgets
CaP-Agent0 receives up to 50 model turns per episode. RATs receives up to 50 model turns and a 1,200-second wall-clock time limit. RPG and ASPIRE receive up to 30 model turns, 12 model-based observation calls, and 1,200 seconds of wall-clock time.
Self-improvement models
RPG uses Fable 5.1 for offline code revision during self-improvement, including execution-video analysis and updates to the shared execution system between episodes. Gemini 3.8 Flash generates task-level programs during online execution.
Task completion
RPG achieves 209/220 successes (95.0%), outperforming all baselines. Among the four models evaluated with CaP-Agent0, GPT-6 Astra Pro performs best, with 132/220 successes (60.0%), below RPG’s performance with Gemini 3.8 Flash. Gemini CaP-Agent0 achieves 107/220 (48.6%), ASPIRE achieves 166/220 (75.5%), and RATs achieves 92/220 (41.8%).
Compared with Gemini CaP-Agent0, RPG improves on 19 tasks and ties on the remaining three. The improvement spans both task groups: success increases from 51.4% to 98.6% on CaP-X tasks and from 47.3% to 93.3% on ABC-derived or analogous tasks.
RPG succeeds in all ten trials on 17 of the 22 tasks. Compared with ASPIRE, its largest gains come from fixture insertion and bottle placement, which together account for 15 of its 43 additional successes. ASPIRE succeeds in one more trial than RPG on serving and standard cube sorting. Seven of RPG’s 11 failures occur in towel folding, making deformable manipulation the main remaining failure mode in this evaluation.
Improvement across rounds
RPG increases from 63/220 successes after round 1 (28.6%) to 209/220 after round 15 (95.0%). Round 4 exceeds all four CaP-Agent0 configurations, and round 5 reaches 78.2%, exceeding ASPIRE’s 75.5% endpoint.
After the system prompt is fixed, success rises by another 21.4 percentage points between rounds 4 and 15. During this period, the system makes 60 modifications to existing skills. In rounds 9, 11, 14, and 15, it modifies existing skills without adding new ones. RPG therefore improves both by expanding the library and by repairing shared skill implementations.
The evaluation curve is not strictly monotonic: the number of successes decreases by one at rounds 6 and 14. These revisions passed the five-seed development checks but did not improve every result on the ten held-out evaluation seeds.
Conclusion
RPG diagnoses failures using privileged execution and validates revisions across tasks to improve a persistent robot execution system without updating model weights. Success rises from 28.6% after the first practice round to 95.0% after 15 rounds on held-out initializations of 22 tasks, outperforming ASPIRE (75.5%) and CaP-Agent0 with GPT-6 Astra Pro (60.0%).
Ablations support combining the Video Analyzer and Privileged Agent, while a library-only comparison isolates gains from skill revisions and highlights the importance of task-level validation. Following the same calibration and hardware-adaptation procedure as the baselines, the frozen system succeeds in all 30 trials across three physical tasks. These results show that shared skill code and the system prompt can serve as practical targets for persistent robot improvement.
Discussion and Limitations
Two bottlenecks limit self-improvement. Each round collects Runtime Agent and Privileged Agent trajectories across all 22 tasks, but API rate limits constrain parallel rollouts. As the task set grows, overlapping revisions to shared skills and dependencies can also complicate integration. Future work could improve efficiency through quota-aware rollout scheduling and dependency-aware proposal integration.
Acknowledgments
The authors thank Zi Wang and Bike Zhang for helpful discussions and feedback. OpenAI ChatGPT and Anthropic Claude were used to assist with language editing and manuscript polishing. All technical content, analyses, and conclusions were developed and verified by the authors.
Self-Improvement Protocol
At the beginning of each round, the Runtime Agent attempts every practice task on five fixed development initializations. It selects and composes reusable skills using only observations available at deployment time and its execution history.
In parallel, a Privileged Agent runs separate episodes from matched initial states using the same runtime model and the same shared skill library. The Privileged Agent differs only in its access to simulator state, including object identities, poses, sizes, velocities, and cloth-corner positions. Both agents invoke the same deployment-compatible skills. The skills themselves do not receive privileged state.
Qualitative Examples of Skill Acquisition and Refinement
The authors present representative examples of how self-improvement changes the robot’s reusable manipulation behaviors. During self-improvement, per-task development success is measured on five fixed development initializations, so these in-loop success rates change in increments of 20 percentage points. Candidate revisions are subsequently evaluated across the full cross-task validation suite as described in Appendix A. These in-loop development measurements are distinct from the separate 10-seed held-out evaluations reported in the main experiments.
The Place bottles in bin task provides an example of acquiring a previously unreliable manipulation capability. In round 12, its in-loop success rate increased from 1/5 (20%) to 5/5 (100%).
Before this improvement, the agent often treated moving the gripper near the receptacle as sufficient for successful placement. It could commit to a placement without checking the complete grasp-to-placement trajectory, continue after an unreachable waypoint, or release a bottle near the bin without ensuring that the bottle itself had cleared the rim and entered the opening.
The improved behavior instead localizes the receptacle opening, checks the complete manipulation route, reasons about the geometry of the bottle rather than only the fingertip pose, clears the rim before lowering, and verifies containment after release. In short, the behavior changes from:
Pick → move near receptacle → release
This represents a transition from end-effector-centric manipulation to object-centric manipulation: reaching the commanded gripper pose is no longer treated as equivalent to achieving the desired physical state.
Simulation Task Suite
RPG is evaluated on a suite of 22 bimanual manipulation tasks on the YAM station. Each task specification fixes the simulation scene, language instruction, execution budget, and a binary evaluator that evaluates the final outcome from simulator state. The policy’s own success claims are recorded for diagnostics but never affect the success label.
The suite contains 17 development tasks, one validation task, and four feedback-held-out tasks. Development tasks provide task-specific failure feedback and may generate candidate revisions. The validation task, Restack cubes, does not generate candidate revisions and is used as a cross-task regression check. It is reported separately from the feedback-held-out tasks because only the latter are used in the transfer analysis of Appendix D.
The four feedback-held-out tasks are Serve onto plate, Two-arm handover, Transfer object, and Close sunglasses case. They are neither analyzed by the Video Analyzer nor used to generate task-specific candidate revisions. They remain in cross-task validation only as regression checks.
All tasks use the same observation interface: RGB images from the top and both wrist cameras, top-camera depth, and robot proprioception. Initial conditions randomize object poses and selected physical parameters, including friction, mass, table height, camera pose, and depth noise, followed by a fixed settling period before execution. Each trial consists of one episode. Task-specific limits on skill calls and robot execution time are part of the task definition.
Unless otherwise noted, task success is scored from the final simulator state. RATs follows its native any-time success criterion.
The benchmark uses nine evaluator types that capture different manipulation objectives. Most placement tasks use an all_in_targets evaluator, which checks that every required object lies within task-specific horizontal and vertical tolerances around its target. Other evaluators check lifting, articulated closure, extraction, stacking, restacking, level bimanual lifting, debris removal, and cloth folding.
Several evaluators include additional guard conditions to reject degenerate solutions. For example, moving an entire fixture does not satisfy extraction, and a pre-stacked scene does not satisfy restacking unless the base itself is relocated.
Frequently Asked Questions
How does RPG improve robot task performance? It diagnoses execution failures and revises a shared library of reusable skills, validating changes across tasks without updating model weights.
What roles do the Runtime Agent and Privileged Agent play? The Runtime Agent operates from deployment-time observations, while the Privileged Agent uses simulator state in separate executions to support failure diagnosis.
How well does RPG perform in simulation? It achieves 209/220 successes (95.0%) after 15 practice rounds across 22 tasks.
Does the adapted system work on a physical robot? The frozen system succeeds in all 30 trials across three physical tasks after common calibration and hardware adaptation.