World Action Models (WAMs) have recently advanced robotic manipulation by jointly predicting robot actions and future observations. This joint modeling provides visual context for action generation, helping the policy anticipate how the scene will evolve. However, deployment in dynamic environments requires closed-loop control: the robot must repeatedly replan from the latest observations to correct execution errors and respond to environmental changes.
Frequent replanning exposes a computational bottleneck in standard WAMs. Joint denoising samplers process the entire prediction horizon from pure noise to clean data at each replanning cycle. Each denoising step processes the entire video-action sequence, making repeated model evaluations computationally expensive. The resulting inference latency delays action updates, limiting closed-loop responsiveness. Some methods achieve faster inference by omitting future-video generation at test time, at the cost of losing explicit visual predictions that could inform action generation.
We argue that denoising can be organized more efficiently across replanning cycles. Conventional chunk-based samplers fully denoise a new prediction horizon at each replan, even when receding-horizon control executes only an initial portion. The unexecuted tail can inform current actions but is discarded rather than reused across replans. Retaining and progressively refining future video-action chunks across cycles can preserve predictive context and reuse denoising computation, reducing the sequential steps needed to produce the next executable chunk.
We therefore introduce Rolling-WAM, a formulation that inherits this intuition by distributing denoising across successive replanning cycles. Building on the concept of rolling diffusion for sequence generation, Rolling-WAM maintains a sliding window of aligned video-action chunks with progressively higher noise levels toward the future. At each replanning cycle, only the imminent action chunk is fully denoised for execution, while farther-future chunks remain partially denoised. This resembles the intuition behind human planning: near-term actions are concrete, while more distant plans remain provisional and are refined as new observations arrive.
After the first action chunk is executed, the window advances, retaining the remaining predictions and appending a new chunk initialized with Gaussian noise. The retained predictions are then jointly refined with the new chunk, conditioned on the newly acquired camera observation. As chunks move through the window, their denoising steps are distributed across replanning cycles. Each steady-state cycle therefore requires only a fraction of the total steps to produce the next executable chunk. The model is trained on matching noise profiles, with action tokens attending to partially denoised visual futures across the window.
Rolling-WAM is evaluated on simulation benchmarks, including LIBERO and RoboTwin, as well as three real-world humanoid manipulation tasks on the Unitree G1. The experiments demonstrate that Rolling-WAM retains the benefits of visual imagination while significantly reducing computational overhead. It achieves a 98.1% average success rate on LIBERO and 93.3% on RoboTwin, remaining competitive with or exceeding state-of-the-art WAMs.
The primary contribution is a rolling formulation for joint video-action models that distributes denoising across successive replanning cycles. This formulation achieves a reported inference speedup over standard WAMs while maintaining competitive manipulation performance.
Rolling Diffusion for Sequence Generation
Rolling Diffusion introduces a sliding denoising window with progressively higher noise levels toward the future. This concept has been extended with independent token noise levels for flexible sampling schedules and bidirectional denoising with rollout-based distillation for autoregressive long-video generation. These works motivate the rolling formulation of Rolling-WAM.
In robotic manipulation, Streaming Diffusion Policy and RNR-DP maintain partially denoised action buffers to accelerate action generation. However, these methods focus exclusively on action-only denoising and evaluate task-specific policies in relatively small-scale settings. Rolling-WAM extends this rolling mechanism to couple world modeling with action generation, enabling action tokens to attend to partially denoised visual futures, and evaluates the approach across broader multitask settings.
Problem Formulation
This distribution can be modeled jointly or factorized with action generation conditioned on future visual prediction.
Deploying WAMs in a closed loop requires multiple denoising steps within each replanning cycle before an action chunk is executed. This iterative process incurs a high computational cost. It increases inference latency and limits the responsiveness of the controller to new observations.
Rolling-WAM addresses this bottleneck by distributing the denoising computation across successive replanning cycles. It maintains a sliding window that jointly refines the next action chunk and its future continuation. This approach completes near-term predictions while leaving longer-term chunks partially denoised at staggered noise levels. As the window advances with execution, the retained predictions continue to evolve under new observations. This mechanism carries predictive context across chunk boundaries, allowing successive action chunks to be generated with a shared, evolving visual-action future.
Sampling and Execution
At each denoising step, the model jointly updates the video and action predictions throughout the window.
The model applies an Euler integration update using the predicted flow velocity to advance the current denoising state of each video-action chunk.
Once the first chunk is clean, the robot executes its predicted actions. The window then slides over. It removes the executed chunk, retains the remaining predictions, and appends a new chunk initialized with Gaussian noise. The robot acquires the latest camera observation and state to condition the next replanning cycle.
Experiments
The experiments verify whether rolling denoising can accelerate joint video-action prediction while preserving manipulation performance. Task success is compared with a broad set of vision-language-action models and world action models on LIBERO, RoboTwin 2.0, and real-world Unitree G1 tasks. Controlled latency measurements quantify the computational savings of rolling denoising. Ablations examine the roles of window size, training noise schedules, and cross-chunk action attention.
Experimental Setup
Rolling-WAM is evaluated in the following settings. Task success rate, reported as a percentage, is the primary performance metric, and replanning latency is measured to assess efficiency.
The Spatial, Object, Goal, and Long suites of LIBERO are evaluated. Each suite contains 10 tasks and 500 demonstrations. Observations include external and wrist RGB views. One policy is trained across all four suites for 10 epochs with an effective batch size of 128, and each task is evaluated with 50 rollouts.
Performance on Simulation Benchmarks
The first evaluation examines whether rolling denoising can deliver competitive manipulation performance with fewer denoising steps per replanning cycle.
Rolling-WAM achieves an average success rate of 98.1% on LIBERO, within 0.4 percentage points of the joint leaders, LingBot-VA and Joint-WAM, both at 98.5%, and above Motus at 97.7% and Fast-WAM at 97.6%. Success ranges from 97.8% to 98.2% across the four suites, showing consistently strong performance with rolling inference.
RoboTwin 2.0 further tests bimanual manipulation under scene randomization. Rolling-WAM achieves success rates of 93.5% and 93.0% in clean and randomized settings, respectively. Its 93.3% average leads LingBot-VA at 92.2%, Fast-WAM at 91.8%, and Joint-WAM at 90.6%. The high success rate in both settings shows that rolling inference remains effective under the evaluated scene variations, without additional embodied pretraining. Per-task results are provided in the paper’s appendix.
Rolling-WAM achieves these results with only two denoising steps per replanning cycle, suggesting that co-refining current and future predictions can potentially combine faster replanning with improved manipulation performance.
The retained visual predictions are also inspected for alignment with observations as execution proceeds. Imagined robot motion and task progression closely follow the observations in an example involving placing an object in a cabinet in the RoboTwin 2.0 benchmark, providing a qualitative view of the visual context maintained during rolling execution.
Conclusion
Rolling-WAM is an efficient World Action Model that jointly refines near-term and future video-action predictions within a sliding window. By distributing denoising across replanning cycles, it reduces the sequential steps needed to produce the next executable action chunk, while using partially denoised visual futures to guide action generation.
Experiments show that Rolling-WAM achieves competitive performance on LIBERO and outperforms the compared methods on RoboTwin 2.0 and real-world humanoid manipulation tasks, while delivering substantial speedups in inference over standard joint WAMs. Combining rolling denoising with complementary model and system optimizations offers further opportunities for low-latency robot control.
Limitations and Future Work
Design choices such as window and chunk sizes warrant further exploration across task settings with different dynamics and control frequencies. Despite observation feedback, retained predictions may lag behind rapid scene changes and misguide action generation, particularly with long windows. Adaptive window management and asynchronous execution offer directions for improving closed-loop responsiveness.
Acknowledgments
This work was partially supported by the National Science Foundation through NSF CPS #2434460. The USC Physical Superintelligence Lab acknowledges generous support from Toyota Research Institute, Dolby, Google DeepMind, Capital One, Nvidia, Bosch, NSF, and Qualcomm. Yue Wang is also supported by a Powell Research Award.
Frequently Asked Questions
What problem does Rolling-WAM address? It addresses the computational cost and latency of repeatedly denoising complete video-action prediction horizons during closed-loop robot control.
How does Rolling-WAM reduce inference overhead? It retains partially denoised future video-action chunks and progressively refines them across replanning cycles instead of restarting the full denoising process each time.
What performance does Rolling-WAM achieve on simulation benchmarks? It achieves a 98.1% average success rate on LIBERO and a 93.3% average success rate on RoboTwin 2.0.
What are the main limitations identified by the authors? Retained predictions may lag behind rapid scene changes, especially with long windows, while window and chunk sizes require further study across different task settings.