A new diffusion policy architecture from Stanford researchers lets robot arms and mobile bases reason across multiple coordinate frames at once, achieving higher success rates than any single-frame policy — including an oracle that knows the best frame ahead of time. The Mixture of Frames (MoF) policy is the first visuomotor system that denoises action trajectories in parallel across different spatial reference frames, fusing their predictions to pick the easiest path for each movement.
What the Researchers Built
Most diffusion policies for robot manipulation define actions in a single coordinate frame — for instance, robot-base-relative coordinates or end-effector-relative coordinates. But different phases of a bimanual mobile manipulation task are easier to learn in different frames. Gripping an object is natural in wrist frame; moving the base is easier in world frame. MoF tackles this by instantiating one expert denoiser per candidate frame (e.g., base-relative, left gripper-relative, right gripper-relative, and a trajectory-relative frame) and running them in parallel.
At each denoising step, all experts receive the same canonical noisy action transformed into their respective frame, predict the noise in that frame, then convert their predictions back to the canonical frame. A learned router module weights these predictions — either through a soft mixture-of-experts combination or uniform ensembling — to produce the final denoised action. Crucially, MoF also introduces a column-vector action representation that makes the transformation of noisy actions exact and differentiable, a requirement for multi-frame denoising to work correctly.
Key Results
The team evaluated MoF on nine bimanual mobile manipulation tasks from two benchmarks: BiGym using the RB-Y1 robot and DexMimicGen using a dual-arm mobile platform. The tasks included pouring, serving, placing, and coordinated bimanual grasping. MoF was compared against:
- Single-frame policies (base-relative, left gripper-relative, right gripper-relative, trajectory-relative)
- Oracle best-frame (the best single-frame policy for each task, selected by hindsight)
- MoE-DP (a standard mixture-of-experts diffusion policy that only conditions on multiple frames but denoises in one frame)
MoF consistently outperformed all single-frame baselines across the nine tasks. More strikingly, MoF also surpassed the oracle best-frame policy — meaning that combining multiple frames yields better performance than even the optimal single frame. The improvements were largest on tasks with complex coordination, such as bimanual pouring where both arms and the base must move simultaneously. MoF also proved robust to artificially injected frame uncertainty, maintaining high success rates when frame transformations were noisy.
How It Works
MoF builds on the standard diffusion policy framework where a neural network iteratively denoises random noise into a clean action trajectory. The key innovation is the synchronized multi-frame denoising loop.
- Action representation: Actions are expressed as column vectors in a canonical frame (e.g., base frame). Critically, the representation supports exact linear transformation to any other frame — including for noisy actions — enabling consistent multi-frame processing without approximation errors.
- Multi-frame denoising step: Given the current noisy action in canonical frame, MoF transforms it into each expert’s frame using differentiable transformations. Each expert denoiser (a small neural network) predicts the noise in its own frame. These noise predictions are then transformed back to canonical frame and fused.
- Fusion: Two fusion methods are tested: a learned router that outputs soft weights per expert, and uniform averaging. The learned router is trained end-to-end with the denoising loss. To prevent unused experts from drifting, an auxiliary per-expert loss directly supervises each expert’s output against the ground-truth noise in its own frame.
The critical distinction from MoE-DP: In MoE-DP, mixture occurs in the conditioning pathway (features) and a single denoiser works in one frame. In MoF, the mixture happens at the denoising layer — each expert operates on a different parameterization of the same underlying action. This means MoF can exploit the natural structure of the action distribution across frames, rather than just increasing model capacity.
Why This Matters for Robotics
Bimanual mobile manipulation is one of the hardest challenges in robotics today. Coordinating two arms and a mobile base creates action spaces with high dimensionality and complex dependencies. Choosing the right action frame is task-dependent and even phase-dependent — a fixed choice leaves performance on the table. MoF automates this selection without requiring manual tuning or task-specific engineering.
Practical implications: - Reduced engineering burden: Operators no longer need to hand-select the best action frame for each new task. MoF adapts automatically. - Better coordination: By allowing the policy to reason in multiple frames simultaneously, MoF handles tasks where arms and base must collaborate (e.g., carrying a large object together). - Plug-and-play: MoF can be added to existing diffusion policy pipelines with minimal changes; the multi-frame experts share the same vision encoder.
For teams integrating bimanual humanoid robots or collaborative robots in warehouse or service settings, MoF offers a path to more robust, data-efficient manipulation without per-task frame engineering.
Limitations and Open Questions
The current study focuses on relatively small diffusion policies (under 100M parameters). Scaling MoF to large vision-language-action (VLA) models remains unexplored. The router is trained purely on denoising loss, which may not align perfectly with task success — a behavior-aware training objective could improve real-world performance. MoF also requires a predefined set of candidate frames and accurate proprioceptive transformations; automatic frame discovery and robustness to uncertain transformation estimates are open research directions. Finally, the computational cost of running multiple expert denoisers scales linearly with the number of frames, which could be a concern for real-time deployment on resource-constrained robots.
Frequently Asked Questions
What are coordinate frames and why do they matter? A coordinate frame defines the reference point for describing robot motion — e.g., relative to the robot’s base, its left gripper, or the object. Different tasks are easier to learn in different frames because the action distribution becomes simpler (e.g., less rotation or translation). MoF exploits this by using multiple frames simultaneously.
How does MoF differ from a standard mixture-of-experts policy? Standard MoE mixes features before a single denoiser, but MoF mixes denoising predictions themselves — each expert works in a different action parameterization. This allows the policy to leverage the natural simplicity of actions in different frames.
Does MoF work for single-arm or non-mobile robots? Yes. The concept applies to any robot where multiple coordinate frames are relevant, such as a single arm mounted on a mobile base or a fixed dual-arm robot. The candidate frames can be chosen accordingly.
What hardware was used in the experiments? The team used two simulated benchmarks: BiGym with the RB-Y1 humanoid-style robot and DexMimicGen with a dual-arm mobile platform. Real-world validation is mentioned as ongoing work.
Conclusion
Mixture of Frames is a simple yet powerful idea: instead of forcing a diffusion policy to learn in one coordinate frame, let it learn in many and combine the results. The experiments show consistent gains across a range of bimanual mobile manipulation tasks, even surpassing the best single-frame oracle. This work points toward a future where visuomotor policies automatically adapt their spatial reasoning to the structure of each task.