Vision Language Action (VLA) models map visual observations and language instructions to robot actions, showing strong capabilities in robotic manipulation. Reliable execution also requires recognizing whether a task is ongoing, has succeeded, or has failed, so a system can decide when to continue, stop, or intervene. Existing methods predict progress, completion, or failure signals within VLA policies or through external monitoring models.
However, existing approaches face two limitations. Learning and integrating these signals can require fine-grained frame-level supervision, external monitoring models, or complex architectural designs. In addition, predicted signals are typically used only for monitoring, leaving their potential to improve action generation underexplored.
To learn these signals without frame-level progress annotations, Echo combines successful demonstrations, failed policy rollouts, pre-intervention failure segments, and recovery demonstrations. It uses subtask-level prompts together with success or failure labels. Failed rollouts and pre-intervention failure segments supervise terminal and recoverable failures, respectively, while their actions are excluded from imitation. Recovery demonstrations pair similar status supervision with corrective actions at corresponding recoverable failure states, linking failure recognition to recovery behavior.
To use learned status information during action generation, Echo controls interactions among stop signals (SS), failure signals (FS), and robot actions while preserving accurate, complementary signal predictions. It also introduces asynchronous denoising: signal tokens are denoised faster than action tokens, so later action-denoising steps condition on cleaner status estimates. When the policy recognizes a recoverable failure with low SS and elevated FS, those estimates, together with current observations, can guide action generation toward corrective behavior learned from recovery demonstrations.
Experiments on five real-world manipulation tasks demonstrate accurate task-status prediction and improved performance over action-only baselines, with larger out-of-distribution gains. Component ablations, signal interventions, and failure-data scaling analyses examine how status signals influence action generation and how different data sources contribute to prediction and execution.
The contributions are threefold: a compact task-status representation that incorporates stop and failure signals into VLA action chunks with little additional parameter overhead; a supervision scheme that learns these signals from successful, failed, and recovery data without frame-level progress annotations; and controlled interactions with asynchronous denoising to couple status prediction and action generation.
Preliminaries
A sampled chunk may extend beyond the current segment, entering a subsequent subtask or exceeding the episode length. Let the final valid action timestep in the segment denote the segment’s last valid action timestep. A padding mask identifies positions beyond this boundary.
At padded positions, the last valid action target is repeated. Besides excluding actions beyond the current segment, padding naturally provides boundary-based signal labels. For example, when a segment ends in task completion, its padding mask indicates where stopping should be predicted. Echo combines this mask with the segment type to construct task-status signal targets without additional frame-level progress annotations.
Experiments
The experiments report main results and comparisons with existing methods to evaluate the accuracy of Echo’s task-status signals and their benefit to task execution. They then analyze in-distribution (ID) and out-of-distribution (OOD) performance and ablate model components. Finally, the experiments intervene on the signals to verify their effect on action generation and vary the amount of failure data to examine the role of different data sources.
Experimental setup
Unless otherwise specified, evaluations use 30 trials per model and task, stop-signal and failure-signal thresholds, and a 15-frame decision window (3 seconds at a 0.2-second inference interval). Results are reported as the mean with standard error; F1 scores are given as percentages. Scoring, annotation, and outcome-decision rules are detailed in the evaluation protocol below.
Ablation and mechanism analysis
The analysis investigates why Echo improves execution and when its learned signals are most useful. It compares ID and OOD conditions, ablates the signal–action architecture, directly intervenes on the signals, and varies the amount of terminal-failure data.
For Insert Flower and Open Drawer, the main results are broken down into ID and OOD conditions, with 15 trials in each condition. Echo’s gains are larger under OOD conditions, while ID differences are modest and task-dependent. Status prediction remains broadly stable: frame-level performance changes modestly, and trajectory-level performance declines slightly under distribution shift.
Successful demonstrations emphasize nominal execution, whereas recovery demonstrations emphasize adjustment and correction. The authors hypothesize that status signals steer the policy toward the latter behaviors under distribution shift. Signal interventions directly test this hypothesis.
Conclusions
Echo is a compact framework that jointly denoises actions and task-status signals within a single VLA. It learns from successful, failed, and recovery data without frame-level progress annotations, using controlled interactions and asynchronous denoising to guide action generation. Experiments on five real-world tasks demonstrate accurate status prediction and improved task completion, supporting the use of task-status signals for both monitoring and action generation.
AI Use Statement
Generative AI tools were used to improve the clarity, grammar, and readability of the manuscript. All AI-assisted edits were reviewed by the authors, who take full responsibility for the final content.
Robot Platform and Task Details
Data collection and policy deployment use a wheeled humanoid robot with two seven-degree-of-freedom arms, gripper end effectors, and a waist–leg mechanism with six degrees of freedom. The mobile base remains stationary during task execution. The policy controls a total of 20 degrees of freedom through the two arm end effectors, the grippers, and the waist–leg mechanism. Visual observations come from three RGB cameras: one on the head and one on each wrist.
Evaluation Protocol and Task Scoring
Each episode has a maximum task score of 10. Successful episodes receive 10 points before task-specific deductions. Failed episodes receive 0 points, except in Make Tanghulu when insertion is achieved but the hawthorn subsequently falls off the skewer after release; these failed episodes receive 5 points as partial credit.
For Insert Flower and Insert Pen, each contact event incurs a one-point deduction, capped at three points. For Capsule Toy and Open Drawer, each empty grasp attempt incurs a one-point deduction, capped at five points.
Data Sampling Ratio
For every task, the training sampling ratio is the ratio assigned to Success, Failure, Pre-intervention failure, and Recovery data, respectively.
Implementation, Hyperparameters, and Inference Cost
For chunks containing padding, both signal-loss weights are set to 1, following the supervision rule in the main text.
Implementation of the Motivating Experiment
The motivating experiment examines how intermediate action and status estimates evolve during synchronous denoising. Its purpose is to determine when status information becomes available before introducing the asynchronous schedule.
The baseline denoises actions, SS, and FS synchronously, assigning the same flow time to all three groups.
Baseline Implementations and Performance Analysis
The paper compares action-only a baseline that predicts actions without stop or failure signals, CycleVLA, VLA-Corrector, Robometer, and Echo on Insert Flower. This section distinguishes original methods from adaptations in the experimental setup.
The baseline uses the same successful and recovery action-supervision data as Echo, with matched optimizer settings and training steps. It predicts actions without SS or FS, so status metrics are not assigned to it. During implementation, key settings—including training data, model learning rate, and optimizer—were kept consistent with Echo. The model was trained for the same number of steps until convergence.
Hyperparameter Sensitivity
Frame-level performance is generally more sensitive to the SS threshold than to the FS threshold over the tested ranges. For example, Capsule Toy changes from 98.1 at an SS threshold of 0.1 to 88.3 at 0.5, while remaining unchanged across the tested FS thresholds. The other tasks also show relatively modest changes along the FS-threshold axis, although the best SS threshold varies across tasks. Thus, a single threshold setting does not maximize frame-level F1 for every task.
Compatibility with External Monitoring and Planning
Echo improves the VLA policy itself by allowing learned task-status information to participate in action generation. This objective is compatible with external monitoring, recovery, and planning modules. SS and FS can also provide execution-status information to a higher-level controller using the signal semantics defined in the main text.
An external controller could combine these signals with broader temporal context or additional observations to verify completion, request intervention, or replan. Echo supplies policy-internal status estimates and corrective action generation, while the external module adds its own decision logic. This is architectural compatibility; improved performance from a particular combined system requires a separate evaluation.
Simulation Experiments
The real-world experiments are complemented by a three-type simulation study based on LIBERO and LIBERO-Recovery. This study evaluates both task completion from recovery states and task-status prediction using simulator success checks.
Frequently Asked Questions
What does Echo predict alongside robot actions? Echo predicts stop and failure signals that indicate task status.
What data does Echo use to learn task-status signals? It combines successful demonstrations, failed policy rollouts, pre-intervention failure segments, and recovery demonstrations.
How does Echo use task-status signals during action generation? It controls interactions between status signals and actions and denoises signal tokens faster, allowing later action-denoising steps to use cleaner status estimates.
What robot and tasks were used in the real-world experiments? Experiments used a stationary-base wheeled humanoid with two arms and three RGB cameras across five real-world manipulation tasks.