Building on this interface, an end-to-end system uses only a compact vision-language model backbone and token-based outputs. An automatic dual-channel annotation pipeline projects navigation targets into a shared pointing space. Temporally aware visual history compression preserves both recent detail and long-horizon context within a bounded visual-token budget. An RVQ-based action tokenizer converts short-horizon trajectories into language-model tokens, which are subsequently mapped by an execution layer to platform-specific controls. Embodied-reasoning mid-training, supervised fine-tuning with DAgger, and online reinforcement learning progressively align perception, reasoning, and control.
LightNav-0 is evaluated across 10 public navigation simulation settings covering instruction-following visual-language navigation, object goal navigation, and embodied visual tracking. Complementary pointing and spatial-VQA evaluations test whether navigation adaptation preserves the VLM’s original grounding abilities. Real-world demonstrations further probe transfer across robot embodiments, outdoor scenes, and dynamic targets. Together, these experiments demonstrate a central hypothesis: a compact VLM backbone can provide a unified, transferable basis for cross-task and cross-embodiment navigation.
Linguistic and Visual Chain-of-Thought
LightNav-0 combines explicit reasoning with visual grounding in a fixed-length trace. Its dual-channel pointing prefix predicts an affordance point for feasible local motion and an object point for a target object or goal location. Both are represented with channel-specific image-grid tokens that leverage the backbone’s pretrained spatial grounding rather than replacing it. The same representation supports instruction-following visual-language navigation, open-vocabulary object navigation, and embodied visual tracking.
The prefix is supervised in the same token space as the action and costs a constant number of tokens per decision. It therefore keeps the interpretability of an explicit reasoning step without the variable decoding latency of free-form textual deliberation or a second planning system.
Reinforcement Learning Post-Training
Reinforcement learning has long complemented imitation in embodied navigation, from large-scale distributed on-policy training and its transformer-scale successors to imitation pretraining followed by reinforcement-learning fine-tuning. Those policies are task-specific and act over a handful of discrete primitives, so an action probability is immediate and a rollout is cheap. Neither property is automatic once the policy becomes a generalist vision-language-action model.
Policies that attach a continuous head recover precision and lose that quantity. SimpleVLA-RL therefore keeps an autoregressive policy and supplies outcome-only rewards, whereas flow-based policies must first recast their denoising process as a Markov decision process before policy gradients apply.
Unified Autoregressive Objective
Navigation and auxiliary VQA examples are trained with the same causal language-model objective.
The model minimizes the negative log-likelihood of the supervised output tokens conditioned on the input context.
Training and Inference Efficiency
The compact, purely autoregressive interface allows variable-length samples to be packed without padding each sample to a common maximum. Approximately 8.6 samples are dynamically packed into each 8,192-token training sequence.
Fused vision rotary-position-embedding operations and an LM-head dimension aligned to a multiple of 8 further reduce kernel and memory overhead. These optimizations apply uniformly because the model contains no task-specific action modules.
Embodied-Reasoning Data
Eleven single-image VQA sources provide indexed supervision. Three standalone collections are SenseNova-SI Spatial, CLEVR Spatial VQA, and SAT Spatial VQA. The remaining eight are VSTP-SI Depth Comparison, VSTP-SI Distance, VSTP-SI Scene Caption, VSTP-SI Measurement, VSTP-MI Correspondence, VSTP-MI Object–Object Relation, VSTP-MI Camera Motion, and VSTP-MI Scene Caption.
Their questions cover relative position, metric depth and distance, object–object relations, camera motion, physical measurement, scene description, and affordance-oriented reasoning. This mixture teaches the model to recover both qualitative relations, such as left/right and in front/behind, and quantitative cues needed to distinguish traversable free space from visually plausible but geometrically invalid targets.
Navigation Data
The instruction-following mixture combines randomized-camera expert trajectories from R2R and RxR with self-distilled routes and ScaleVLN data. The synthesized and self-distilled sources provide substantially broader scene and instruction diversity than the human-annotated corpora alone.
Together, expert, synthesized, and self-distilled routes expose the model to both fine-grained linguistic alignment and large-scale geometric variation.
Evaluation Benchmarks
Navigation evaluation comprises 10 simulation settings across three task families. Continuous instruction following is measured on the R2R and RxR validation-unseen splits. Closed-vocabulary ObjectNav is evaluated on MP3D and HM3D v1/v2, and open-vocabulary generalization is measured on HM3D-OVON. Embodied visual tracking is evaluated on the STT and DT settings of EVT-Bench.
All navigation benchmarks use the same checkpoint without benchmark-specific fine-tuning.
INSIGHT-Bench
INSIGHT-Bench extends the data distribution with heterogeneous mesh-based and 3D Gaussian-splatting environments. It contains 1,683 scenes and 53,090 training episodes, together with 210 scenes and 1,097 evaluation episodes. The source composition of both splits is detailed in the paper.
Embodied-Reasoning Mid-Training
Specialization alone can narrow the general visual-language competence inherited from the backbone. A specialize-then-retain curriculum is therefore adopted: during the subsequent supervised stage, a rebalanced embodied-reasoning mixture is rehearsed together with navigation data.
This second stage preserves the pointing, grounding, and spatial-reasoning skills acquired during embodied-reasoning mid-training while aligning them with embodied action generation. The resulting curriculum establishes a common perceptual and reasoning basis before cross-task navigation supervision is introduced.
Cross-Domain and Real-World Generalization
The unified spatial interface is qualitatively evaluated beyond the simulation domains used for training. These demonstrations use the same LightNav-0 checkpoint without domain-specific fine-tuning.
Because the virtual and physical environments do not share a standardized action space or success protocol, the rollouts assess the breadth of transfer rather than provide a quantitative benchmark comparison.
Scaling Analysis
Scaling behavior differs across model capacity, data volume, and environment coverage.
Increasing the backbone from 2B to 4B raises R2R success rate and success-weighted path length from 59.9/55.4 to 68.5/62.8, and RxR success rate and success-weighted path length from 64.0/57.9 to 73.6/64.5. These correspond to gains of 6.6–9.6 points across the four measures.
Scaling further to 8B is not consistently beneficial: R2R success rate and success-weighted path length decrease by 1.6/0.5 points, and RxR success rate decreases by 0.5 points, although RxR success-weighted path length increases by 2.0 points. Among the tested checkpoints, the 4B model therefore provides the strongest capacity–performance trade-off. The mixed 8B result indicates that additional parameters alone do not guarantee better navigation performance at this scale.
Data scaling produces monotonic but saturating improvements. Moving from 1/16 to the full training set raises R2R success rate and success-weighted path length from 53.0/47.1 to 68.5/62.8, and RxR success rate and success-weighted path length from 56.2/49.5 to 73.6/64.5. These results yield gains of 15.0–17.4 points.
Most of this improvement is obtained before the final doubling: increasing the data fraction from 1/2 to the full set adds only 0.8/0.9 points on R2R and 1.4/0.4 points on RxR. Thus, additional training data remain beneficial over the tested range, but their marginal return diminishes as the training set approaches full scale.
Conclusion
Several directions can extend this framework. First, the current model uses a single decision pathway and does not explicitly separate high-frequency local control from slower semantic deliberation. A dual system could pair a lightweight reactive policy for obstacle avoidance and frequent trajectory correction with a slower VLM planner for long-horizon reasoning.
Second, stronger pretraining on Internet-scale video could broaden open-world concept coverage and expose the model to rarer scenes, interactions, and motion patterns than curated embodied datasets alone.
Contributors and Acknowledgements
- Training Infrastructure: Qianli Ma, Ran Mei, and Jia Wei
- Data Infrastructure: Fei Huang, Shaoan Wang, Jingyi Xu, Yueyu Wang, and Aocheng Luo
- Mid-training: Shaoan Wang and Fan Yang
- Supervised Fine-tuning: Shaoan Wang and Aocheng Luo
- Post-training: Aocheng Luo and Shaoan Wang
- Benchmark: Fei Huang, Shaoan Wang, Jingyi Xu, and Yueyu Wang
- Real-world Deployment: Xiaoyang Wang, Jiangpeng Hu, Xuhao Liu, Hongming Chen, Yuanbin Shao, Yiyang Lin, and Ziliang Li
- Writing: Shaoan Wang, Tingxiang Fan, Aocheng Luo, Fei Huang, Liang Pan, Xinhang Liu, and Yuntao Ma
- Project Leads: Tingxiang Fan and Shaoan Wang
The authors thank Bo Liang, Yuxuan Xie, Jiaxin Li, and Tianwei Zhang for their contributions to the early-stage infrastructure and initial technical exploration. They also thank Shiyao Zhang and Shuqi Liao for filming and editing the real-world videos, and Kaisong Chen for designing the cover.
The authors thank their collaborators at LimX Dynamics and Manycore Tech for their support across physical deployment and simulation.
Frequently Asked Questions
What is LightNav-0 designed to do? LightNav-0 is a generalist navigation system for instruction following, object goal navigation, and embodied visual tracking across tasks and robot embodiments.
How does LightNav-0 represent navigation targets? It uses a dual-channel pointing prefix with separate affordance and object points represented as channel-specific image-grid tokens.
What benchmarks are used to evaluate the system? Evaluation covers 10 simulation settings across instruction-following navigation, closed- and open-vocabulary ObjectNav, and embodied visual tracking.
Which model size provides the strongest capacity–performance trade-off? Among the tested checkpoints, the 4B model provides the strongest capacity–performance trade-off.