A Robot That Writes Code to See, Search, and Manipulate Objects

AMP1 gives a general-purpose AI model an ongoing role in robot manipulation: it writes Python code, inspects camera images, and revises its plan as the task unfolds. In RoboLab-120, the strongest tested model reached 85.6% task success, while the system also showed behaviors such as searching under objects and calibrating a wrist camera—skills that were not preloaded into its harness.

What did the researchers build?

The researchers built AMP1, a software system that puts a general-purpose AI model in the robot’s control loop. Instead of choosing from a fixed menu of grasping or placing skills, the model writes Python code to inspect the scene and command the robot. It receives the results of that code—including images, printed information, and errors—and uses them to decide what to do next.

The system’s Python environment stays active throughout each task. That lets the model keep variables and functions between turns, reuse earlier calculations, and build up a sequence of actions. It can also request another image or query the robot’s state. If an action fails, the harness can report the error so the model can attempt a recovery rather than simply stopping.

The team evaluated AMP1 across several simulated manipulation testbeds. RoboLab-120 contains 120 language-specified pick-and-place tasks, each with default, vague, and specific instruction variants. The evaluation also covered tasks from LIBERO-PRO, robosuite, BEHAVIOR-1K, and RoboDojo, which includes bimanual tasks. The researchers also ran trials on a real Franka robot, adding functions for checking the arm and recovering from collision-reflex errors.

How well did AMP1 perform?

The strongest tested model, Gemini 4 Argon, achieved an 85.6% success rate in the model comparison. The researchers kept the harness fixed while testing 13 models at different reasoning settings, so the comparison focuses on how model choice and reasoning effort affect AMP1—not on differences between robot-control interfaces.

More reasoning improved success for every model tested, but it also raised inference costs. For Claude Opus 5.5 and Sonnet 5.5, the higher-reasoning setting added 14.6 and 17.2 percentage points, respectively, for about $0.90 more per episode. Gemini 4 Argon gained 2.3 points for about $1.30 more.

For a cost comparison, the paper reports an average of $1.39 per RoboLab-120 episode for Opus 5.5. That is about 26 times lower than the $36.40 reported for Astranomy using GPT-6 Astra; the reported success rates were 82.8% for AMP1 and 74.8% for Astranomy. The comparison points to an efficiency advantage from having AMP1 issue longer code-driven actions rather than asking the model to approve each small end-effector movement.

How does an agentic manipulation policy work?

AMP1 operates through a repeated observe–act–review cycle. The model receives a language instruction and access to camera views, then writes code in a persistent Python interpreter. That code can inspect images, perform calculations, call robot-control functions, and command a trajectory. After the code runs, the model sees the printed output, any error traceback, any displayed images, and information such as remaining control steps and wall-clock time. It then decides what to do on its next turn.

The important difference from a conventional end-to-end policy is that the model can explicitly construct a plan and use tools to carry it out. It is also different from a system that asks a model to write a complete program before deployment: AMP1 remains in the loop and can update its strategy when new observations arrive. The model can combine visual evidence, geometric reasoning, and physical interaction without relying only on a library of prewritten manipulation skills.

Some of the reported behaviors illustrate what this flexibility enables. When the target object was hidden from the cameras, AMP1 reasoned about which objects might be covering it, moved them one at a time, and checked again. In another case, it calibrated the wrist camera by collecting images from known gripper poses. That provided multiple views for estimating the camera’s internal parameters and its fixed position relative to the gripper. The calibration and 3D estimation took 16 of the episode’s 44 model calls, all before the first grasp.

The approach also has a clear timing boundary: the model cannot revise a trajectory while that trajectory is executing. It can react after the action finishes and the result is returned, but not continuously during the movement.

Why does this matter for robotics?

Robots often struggle when an instruction is ambiguous, an object is hidden, or the scene differs from the examples in their training data. AMP1 addresses these situations by letting a general-purpose model inspect the result of an action and choose a new one. The hidden-object search is a useful example: the robot moves objects not just to complete the requested pick, but to learn where the target is. This is a form of interactive perception—acting to improve what the robot knows about its surroundings.

The results also clarify a practical trade-off. Specialist manipulation policies can act quickly, but the paper describes them as unreliable; AMP1 can achieve stronger task success, but spends substantial time waiting for model reasoning. That makes the approach interesting for flexible, low-volume workflows where task variety matters more than cycle time, while leaving fast, repetitive production as a harder fit.

For buyers and operations teams exploring physical automation, the findings are a reminder that the robot arm is only part of the system: the model, sensing, control interface, and recovery process all shape performance. Teams assessing platforms can browse used robots for sale or explore warehouse robots as part of a broader automation search. The paper does not establish that AMP1 is ready for those deployments, but it offers a software direction for robots that need to handle varied instructions and unexpected scenes.

What are the limitations and open questions?

AMP1 is slow because the robot spends much of an episode waiting for the model to reason. It also cannot interrupt or rethink a trajectory while that movement is underway. The evaluations largely involve quasi-static scenes, so they do not establish how well the approach handles a moving target, an object slipping from a gripper, or another event that requires a fast mid-action response.

The paper’s results do not show that AMP1 is ready for mission-critical work. Although the system was tried on a real Franka robot and included recovery from collision reflexes, the provided results do not give an aggregate real-world success rate. Further questions include how reliably it transfers between robot platforms, how much performance depends on image quality and model choice, and whether faster inference can preserve the success rates. For now, the system is a promising foundation, not a deployment-ready substitute for reliable, fast control.

What should readers know about agentic manipulation?

What does “agentic manipulation” mean? It means a general-purpose model stays involved while the robot works, interpreting observations and choosing actions rather than only selecting a prebuilt skill.

Does AMP1 rely on preprogrammed manipulation skills? The highlighted strategies, including searching for a hidden object and calibrating a camera, were not specified in the prompt or supplied as predefined skills. The model generated them while solving tasks.

Which model performed best in the reported comparison? Gemini 4 Argon reached the highest reported success rate, 85.6%, with the AMP1 harness. That result describes this evaluation, not every robot or task.

Is AMP1 ready for factory or warehouse deployment? The paper identifies slow reasoning and limited response during executing trajectories as important constraints. It presents AMP1 as a promising foundation, not as a mission-critical deployment system.

What is the practical takeaway?

AMP1 shows how a robot can use a general-purpose model to plan actions, inspect outcomes, and adapt its approach without relying entirely on fixed skills. Its results are encouraging, but making that flexibility fast and dependable remains the central engineering challenge.