Diagnostic experiments trace candidate robot-signal injections through video diffusion model (VDM) denoising, covering both single-signal injection and multi-signal packing. They measure how strongly a frozen denoiser corrects the injected latent across noise levels. The arrangement of robot signals in latent space measurably affects the denoiser: frequency-domain packing yields latents it corrects far less than spatial tiling or discontinuous canvas layouts. Compactness alone is therefore insufficient; an effective interface must be distribution-aware.
MetaWAM is instantiated on a Cosmos-Predict2 backbone and evaluated on RoboCasa, LIBERO, LIBERO-Plus, and three real-world Franka tabletop tasks. It exceeds the RoboCasa frame-injection baseline while collapsing multiple robot-signal frames into one. The work recasts robot-signal injection as a distribution-aware latent-interface problem for Joint World Action Models (WAMs), provides diagnostic evidence that frequency-domain packing aligns better with a pretrained VDM prior than spatial signal encodings, and introduces a unified frequency-domain meta-frame that carries all robot signals in a single decoder-free latent frame without architectural changes.
Distribution-Aware Spectral Lifting
Broadcasting a robot vector over the spatial grid yields constant patterns, while packing values into spatial regions creates discontinuities. These are the failure modes identified in the motivating analysis. MetaWAM instead encodes robot variables in the frequency domain and lifts them back to the spatial latent.
Channel-band layout
For each meta-frame, MetaWAM builds a sparse coefficient tensor with disjoint channel bands for distinct robot signals, one band per signal. Selecting the action, future-proprioception, and value slots restores their original shapes; task and current-proprioception bands serve as inputs. The selected coefficients directly represent the normalized signals, so no further inverse DCT is applied.
Slot-Aware Task Routing
Dual-granularity masking
Routing runs at two granularities. The slot-level mask carries all task semantics on the meta-frame. A single-channel coarse mask is deterministically derived from it for the original video-denoising path, so the six visual frames are conditioned through the interface expected by the pretrained VDM.
The fine mask carries all task information. The coarse mask carries none and exists only to preserve byte-level compatibility with the backbone.
Conclusion
MetaWAM treats robot-signal injection as a distribution-alignment problem. Frozen-denoiser diagnostics favor frequency-domain packing over spatial duplication and region-based alternatives. Fixed coefficient slots and analytical DCT readout provide a compact interface for joint policy, world-model, and value learning. Results on RoboCasa, LIBERO, LIBERO-Plus, and real-world Franka tasks demonstrate the interface’s efficiency and applicability.
Limitations
The fixed band layout and loss weights may require adaptation when task importance or signal scales vary substantially. Instruction following remains limited by the Cosmos-Predict2 backbone; stronger language grounding is a useful direction. LIBERO-Plus also exposes sensitivity to background and robot initial-state changes.
Acknowledgment
Guoqiang Liang and Yiqi Lin were supported by the NUS Guangzhou Research Translation and Innovation Institute (GRTII)-Affiliated PhD Scholarship Program.
Experiment Details
All motivating-analysis experiments use samples from the RoboCasa dataset released with Cosmos Policy. For the single-signal analysis, the authors randomly select 30 action chunks from distinct episodes and compare ways of injecting this low-dimensional signal into the latent space.
For the multi-signal packing analysis, they use the corresponding action chunks, proprioceptive states, and value labels to construct packed meta-frame variants. This allows them to evaluate the distributional compatibility of different multi-signal encoding schemes under the same pretrained video diffusion model.
The evaluation uses five diffusion noise levels. At each noise level, denoising is repeated five times with independently sampled noise, and results are averaged across trials.
Training Details and Real-Robot Setup
RoboCasa training
MetaWAM is trained on RoboCasa for 12K optimization steps using four NVIDIA H200 GPUs and a global batch size of 1024. Training takes 64 hours of wall-clock time. All parameters of the base Cosmos-Predict2 model are fully fine-tuned.
The policy predicts an action chunk of 32 timesteps, and 16 steps are executed before the policy is queried again.
Additional Visualization Results
Additional qualitative results for real-world out-of-distribution evaluation show rollouts with moderate scene shifts, including changed object locations, additional distractor objects, and background or layout variations. These examples complement the quantitative real-world evaluation by showing that MetaWAM can produce coherent action trajectories under visual and spatial changes in the workspace.
Analytical Readout and Band Supervision
In the unified policy, world-model, and value setting, a robot signal can serve as a clean condition or a prediction target, depending on the task. A learned interface therefore needs per-signal encoding and decoding together with mask-aware routing. Updating its encoders also changes the latent targets presented to the denoiser.
A zero-initialized projection initially conveys no signal. Fixed packing and DCT/iDCT instead provide a static analytical mapping and readout from the start of training. The observed optimization behavior supports this design for the tested setup, without implying that learned interfaces cannot work with other architectures or training procedures.
Frequently Asked Questions
What is MetaWAM designed to do? It provides a unified latent interface for robot signals in Joint World Action Models, packing them into a single frequency-domain meta-frame.
Why does MetaWAM use frequency-domain packing? The reported diagnostics show that a frozen denoiser corrects frequency-domain-packed latents less than spatially tiled or discontinuously arranged alternatives.
Which benchmarks and tasks were used for evaluation? The evaluation covers RoboCasa, LIBERO, LIBERO-Plus, and three real-world Franka tabletop tasks.
What limitations do the authors identify? The fixed band layout and loss weights may need adaptation, and performance is sensitive to language grounding, background changes, and robot initial-state changes.