Variational Automation (VA) tasks that repeat over extended periods assume the workcell environment, robot, and sensors are known and fixed. Furthermore, the range of potential objects and the range of initial object poses are assumed known. These assumptions are part of the VA setting rather than oracle information: unlike generalist robotics, VA tasks are defined by a known workcell and bounded operating envelope, allowing systems to use object models, calibrated sensors, and reusable automation skills when available.
Problem Formulation
For Variational Automation tasks that will be repeated over extended periods, the workcell environment, robot, and sensors are known and fixed. The range of potential objects and the range of initial object poses are also known. These assumptions are part of the VA setting rather than oracle information: unlike generalist robotics, VA tasks are defined by a known workcell and bounded operating envelope, allowing systems to use object models, calibrated sensors, and reusable automation skills when available.
Simulation and Real-World Variational Automation Benchmarks
We present 8 VA benchmark tasks, including 4 in simulation and 4 in real-world. Six of these are inspired by LIBERO, the de facto standard for evaluating VLA policies in simulation.
Benchmark (II-a,b): Pack Grocery Items (Sim and Real)
We modify Benchmark I-a,b to create multi-object pick-and-place tasks, as in a grocery checkout line, where the robot is assigned to pack 6 objects into a container without needing to identify or select any specific items. We quantify the success rate as the number of items (out of 6) placed into the basket after 6 grasping attempts.
Benchmark (III-a,b): Make Popcorn (Sim and Real)
We use the LIBERO frypan, stove, and knob assets to create a manipulation task that requires the robot to sequentially turn on a stove, pick up the pan handle, place it on the stove, remove the pan, and turn off the stove. For the real version of this benchmark, we use the Franka Robot arm with wrist-camera, a portable stove from Amazon and Jiffy-Pop popcorn.
Benchmark (IV): Insert Cables (Real)
This benchmark task requires a UR5 robot arm with the ZED Mini wrist camera to execute a series of USB-C cable insertions and extractions in a bank of 6 sockets. The robot uses internal force torque feedback to probe the insertion when the target port goes beyond the camera's field of view. The benchmark includes two varying conditions. First, we vary the pose of the cable ports, testing three distances in 5 mm increments and three angles in 15° increments. Second, we vary the text prompt to specify five distinct goals: targeting individual ports, inserting in ascending order, descending order, odd-indexed ports, and even-indexed ports.
Baselines
For GaP and CaP-X, we use Gemini-3.1-Flash-Lite for LLM agents and VLM with a temperature of 0.1. For Benchmarks I and II, GaP does not perform self-learning because the first graph generated achieves high performance.
Benchmark III A, B: Make Popcorn in Sim and Real
The Make Popcorn task requires the robot to grasp the stove knob and rotate it to turn on the burner, then the robot must find and pick up the handle of the JiffyPop popcorn pan, place it on the stove burner, wait, and then turn off the stove.
GaP generates a graph using GraspGen, knob turning skills and other object localization and grasping skills from Benchmark 1 and 2. The initial graph is able to turn on and off the knob but unable to pick and place the pan properly – achieving only a 33% success rate.
The resulting policy achieves 94% in simulation with variations of pan position and orientation, and achieves a 90% (18/20) success rate in real physical trials. The two physical failures resulted from one inverse kinematics (IK) error during linear Cartesian motion, and one misgrasp caused by accumulated kinematic error.
Benchmark V: Wash Crates (Simulation)
We compare GaP with a hand-engineered execution graph authored by an expert, representing the conventional manual engineering effort required in such settings. We run both policies over 150 trials sampled from the pose distribution, and additionally measure sustained throughput over a 3-hour continuous execution window.
Conclusion
Agentic robotics has potential to build on rapidly-advancing LLM models to provide a bridge between Good Old Fashioned Engineering (GOFE) and model-free VLA policies. As shown in the Ablation experiments, GaP's graph-structured agentic robotics facilitates modularity, multi-agent integration, and self-learning. As robot applications become more complex and skill libraries grow, more sophisticated agent harnessing will be required.
Limitations. Although in experiments, GaP improves success rates significantly over baselines, execution reliability is not yet at industry levels and additional self-learning and parameter tuning is required. Similarly, GaP execution times are still well below industry standards of 500 units per hour (7 seconds per instance); more work is required to reduce VLM inference requests and IK motion planning time during execution. Also, the 8 VA benchmarks focus on quasi-static pick-and-place operations – only Cable Insertion (IV) requires force sensing. More work is required to apply GaP to tasks with deformable objects, dynamic forces, and moving targets.
Frequently Asked Questions
How does GaP handle tasks where the initial policy graph fails? GaP uses self-learning to iteratively improve the graph through rehearsal and retry, achieving significantly higher success rates than the initial policy.
What benchmarks were used to evaluate GaP performance? Eight VA benchmark tasks were used, including four in simulation and four in real-world, covering packing groceries, making popcorn, inserting cables, and washing crates.
What hardware was used for the real-world experiments? The real-world experiments used a Franka Robot arm with wrist-camera for the popcorn task and a UR5 robot arm with ZED Mini wrist camera for the cable insertion task.
What are the main limitations of GaP in its current form? Execution reliability is not yet at industry levels, execution times are below industrial throughput standards, and the system focuses primarily on quasi-static operations rather than deformable objects or dynamic forces.