Researchers have built a single navigation model called WAM-Nav that works across multiple goal types—point, image, language—on a humanoid robot. By jointly predicting future visual observations and actions, the model unifies diverse navigation tasks into one framework without task-specific retraining.
What the Researchers Built
WAM-Nav (World-Action Modeling for Navigation) is a generative, transformer-based policy that maps camera observations and a user-specified goal directly to robot actions. The key innovation is its asymmetric latent architecture: the model learns a compressed representation of the world (what the robot expects to see next) and the actions (how to move there) simultaneously, but in separate latent spaces. This allows the model to reason both about visual foresight and motor commands within a single training process.
The system is tested on a Unitree G1 humanoid robot equipped with an Intel RealSense D455 depth camera and an onboard RTX 4060 GPU. It runs a receding-horizon control loop: at each step, the model samples 16 candidate trajectories and executes the first one, then repeats. The framework accepts three goal modalities—a 2D point on an image, a natural language instruction, or a goal image—without changing the underlying model architecture.
Key Results
In simulation benchmarks, WAM-Nav matches or exceeds the performance of prior single-task navigation policies—such as NavDP—while being the first model to handle all three goal types in one unified framework. The critical result is balanced performance across goal modalities: while previous methods often favor one goal type (e.g., image goals over language goals), WAM-Nav achieves consistent navigation success rates whether the goal is specified as a point, an image, or a sentence.
Real-world tests on the Unitree G1 humanoid show that the model runs at 1 Hz inference on the onboard GPU (occupying ~1.3 GB of memory) and outputs trajectories that are tracked by a 50 Hz Model Predictive Controller. The system successfully navigates indoor environments with dynamic obstacles using only a forward-facing depth camera mounted at a 20° downward pitch.
No single-task method in the literature simultaneously achieves competitive performance on all three goal types—WAM-Nav is the first to demonstrate this unification without per-task retraining.
How It Works
WAM-Nav is built on a flow-matching generative framework, similar to diffusion policies, but with an asymmetric design that separates world modeling from action modeling. The core pipeline has three stages:
- Goal Alignment Module – The user’s goal (point, image, or text) is fed through a modality-specific encoder to produce a base embedding. This embedding is then projected into two separate token spaces: one for conditioning the world model and one for conditioning the action model. This “asymmetric” design preserves information relevant to each sub-task while keeping a unified interface.
- Latent World-Action Model – A transformer processes the current observation (RGB-D image) along with the goal-conditioned tokens. It simultaneously predicts a latent future observation (the “world token”) and a latent action trajectory (the “action token”). The world and action latents are modeled independently but share the same transformer backbone. Training uses a flow-matching velocity regression loss, which learns the continuous transformation from a noise distribution to the target distribution of world and action latents.
- Receding-Horizon Control – At deployment, the model receives the current observation and goal, then samples 16 candidate trajectories from the learned distribution. The first trajectory is selected and passed to a Model Predictive Controller that converts the planned path into motor velocity commands at 50 Hz. The process repeats every 1 second (the model’s inference rate).
The system runs entirely onboard the robot’s RTX 4060 GPU with a memory footprint of 1.3 GB, enabling real-time closed-loop navigation without cloud connectivity.
Why This Matters for Robotics
WAM-Nav addresses a fundamental pain point in mobile robotics: the need for separate navigation stacks for different goal types. A warehouse robot, for instance, might need to navigate to a coordinate, follow a person in a photo, or respond to a verbal command—traditionally requiring three separate models. WAM-Nav collapses this into one.
For humanoid robots and other mobile platforms designed for human environments, the ability to accept goals in any format—especially natural language—dramatically lowers the barrier for non-expert operators. A factory manager could simply tell the robot “go to the loading dock” without programming coordinates or capturing a goal image.
The asymmetric latent design also points toward more sample-efficient training: by learning both world dynamics and action strategies together, the model may transfer better to new environments with fewer demonstrations, a critical requirement for practical deployments.
Limitations and Open Questions
Two failure modes emerged during real-world testing. First, the fixed forward-facing camera (20° downward pitch) does not capture obstacles directly at the robot’s feet, causing occasional delayed collision avoidance. Second, the model does not explicitly account for the robot’s full body geometry—it treats navigation as a camera-level problem, ignoring whether the torso or limbs would collide with narrow passages.
These limitations suggest two future directions: adaptive camera view controlled by the policy itself (active perception), and embodiment-aware training where the model also learns the robot’s shape and kinematics. Additionally, the 1 Hz inference rate may be too slow for highly dynamic environments—latency improvements or predictive tracking are open engineering challenges.
The paper does not address sim-to-real transfer systematically; all training is done in simulation, and deployment on real hardware requires careful camera calibration and MPC tuning.
Frequently Asked Questions
What makes WAM-Nav different from previous navigation policies? It is the first single model to handle point, image, and language goals with no task-specific retraining, thanks to its asymmetric latent world-action architecture.
What hardware do I need to run WAM-Nav? A depth camera (e.g., Intel RealSense D455) and a GPU with at least 1.3 GB of memory, such as an RTX 4060, are sufficient for real-time onboard inference.
Can WAM-Nav be used on wheeled robots or other morphologies? The current version does not model the robot’s embodiment, so transfer to different platforms would require re-tuning the controller and possibly retraining with embodiment-aware conditioning.
How fast does the model run on a robot? Inference runs at 1 Hz, with trajectory tracking at 50 Hz using a Model Predictive Controller, enabling smooth navigation in indoor environments.
Conclusion
WAM-Nav demonstrates that a single generative navigation policy can handle multiple goal modalities without sacrificing performance. By learning world and action representations jointly in asymmetric latent spaces, the model offers a practical path toward truly universal robot navigation.