SatNav defines three task families—Boundary, Landmark, and Route—to evaluate complementary aspects of memory-intensive, long-horizon navigation: looped-route progress tracking, landmark-based spatial grounding, and sequential route reasoning.
Task Definition
Boundary tasks require the agent to follow the perimeter of a lake, stadium, or building complex. Starting from a point on this closed boundary, the agent navigates along it and stops at an instruction-specified goal. Three variants are defined:
- Partial-arc: The agent stops before completing a full loop.
- Full-loop: The agent stops upon returning to the start after one complete traversal.
- Extended-loop: The agent passes the start and follows an additional segment before stopping.
These tasks evaluate long-horizon progress tracking and accurate stopping, requiring the agent to distinguish between intermediate visits and final arrival at a goal location.
Landmark tasks require the agent to follow a route to a specified destination, using visually distinctive landmarks to determine its heading at turns. At each turning point, the agent rotates until the referenced landmark appears in the specified region of its egocentric view. For example, “turn left until the stadium appears in the upper-right of the view, then continue forward” uses the stadium’s position to specify the desired heading after the turn. The agent follows the remaining instructions and stops at the destination. One-turn and two-turn variants evaluate landmark grounding and relative spatial reasoning.
Route tasks require the agent to navigate along roads, waterways, or combinations of both, following instructions that specify when to turn or stop. The agent identifies relevant intersections or junctions using cues such as “turn left at the first bridge” or “stop at the third crossroad.” According to route composition, tasks are categorized as road-only, waterway-only, and hybrid, with hybrid tasks combining road and waterway segments. These tasks evaluate route following, junction-level decision-making, and the ability to track and count successive decision points over long trajectories.
SwiftVLN Framework Overview
SwiftVLN refactors the training and evaluation pipeline of StreamVLN within ms-swift into a flexible and extensible vision-language navigation (VLN) structure. It enables controlled substitution and evaluation of diverse memory designs while facilitating the adaptation of different large vision-language models (LVLMs) to VLN.
Historical observations outside the short-term context window are processed by a long-term memory module to form a representation of the navigation history. To facilitate controlled comparisons, SwiftVLN implements the memory component as a suite of interchangeable mechanisms:
- History Frame Sampling: Extracts past observations using Uniform, Random, or Temporal-Biased strategies. Temporal-Biased sampling explicitly prioritizes recent frames over distant ones.
- Input Augmentation: Initial Frame Prompting adds the first-frame tokens to the system context as a stable visual anchor for the starting scene. Pose Encoding augments each visual frame with relative pose cues to provide spatial displacement and heading information.
- Memory Compression: Map Memory replaces raw history frames with constructed global and local maps encoded as a compact memory block. Global Token Clustering (GTC) aggregates tokens from all retained frames into a fixed-capacity global memory. Segment Token Clustering (STC) partitions history into a fixed number of temporal segments and clusters each segment independently to preserve coarse temporal structure.
Together, these plug-and-play variants establish a testbed for systematically ablating memory architectures.
Real-World Experiments
The flights demonstrate the feasibility of deploying a satellite-trained navigation model on a real UAV with lightweight visual adaptation. Real-world conditions remain challenging, particularly when local scenes are ambiguous or execution is uncertain, highlighting room for further improvement in satellite-to-UAV transfer.
Detailed Landmark Pipeline
Landmark episodes are designed to evaluate landmark-grounded turning and relative spatial reasoning. Candidate landmarks are detected using a coarse-to-fine localization process over satellite imagery. A coarse scene crop provides global context for identifying visually distinctive objects or regions; a finer grid then verifies the selected landmark and estimates its location more accurately. This hierarchy keeps the VLM input interpretable while allowing landmarks to be localized over long-range trajectories.
Trajectory waypoints are generated around retained landmarks under two executability constraints:
- Decision waypoints are placed within 30 m of their associated landmarks so that the cue remains visible in the local observation.
- Segment lengths and turning angles are quantized to the SatNav action space, where the agent moves forward by a fixed metric step and rotates by a fixed angular increment.
These constraints ensure that the generated demonstration can be exactly executed using the same action interface as model evaluation. Instructions are generated by ordering the landmarks along the trajectory and expressing each key decision through relative-view cues, such as turning until a landmark appears in a specified region of the observation.
Real-World Deployment
The system is deployed on a DJI Matrice 4D for real-world flight tests. During deployment, the onboard image stream and UAV state messages—including longitude, latitude, altitude, and heading—are transmitted to a DJI RC Plus 2 controller through the DJI OcuSync Enterprise video transmission system. The controller relays the data to a local workstation over Wi-Fi, using RTMP for video streaming and MQTT for state and control communication.
On the workstation, the satellite-to-UAV adapted SwiftVLN baseline, built on Qwen2.5-VL-3B, runs on a single RTX 4090 GPU for online inference.
Latency between the UAV and controller is nearly negligible. The main delay comes from forwarding the video stream from the controller to the workstation via RTMP over Wi-Fi, which typically introduces 3–5 seconds of latency due to buffering and network variability. By contrast, model inference on a single RTX 4090 takes about 250 ms on average, making the communication pipeline the primary runtime bottleneck.
Additional Real-World Flight Results
Additional real-world flight tests examine the behavior of the satellite-trained model under physical deployment. The tests cover all three SatNav task families: Boundary, Landmark, and Route. Examples for Landmark and Route tasks show the model executing landmark-guided turning and route-following in real-world scenes.
Acknowledgments and Funding
This work was supported in part by the Guangdong Basic and Applied Basic Research Foundation under Grant 2023A1515111151, the Guangzhou Municipal District Science and Technology Bureau under Grant 2025A03J3655, GDST under Grant C_2025_017, BYD under Grant CP2025O007, and research funding under Grant 2023QN10X121.
The authors thank the Low-Altitude Intelligent Integrated Test Base in Longgang, Shenzhen, China, for providing the site for the real-flight experiments. They also thank Ziyi Chen from Galbot for discussions during the early stages of the project, and Shuo Sun and Lei Zhang from the Low Altitude Space Economy Research Center (LASER), International Digital Economy Academy (IDEA), for help with figure design and real-flight experiments. Chao Wang and Jiayang Sun from LASER, IDEA, are thanked for maintaining computing resources and helping resolve issues during their use.
Frequently Asked Questions
What navigation tasks does SatNav benchmark? SatNav covers Boundary, Landmark, and Route tasks, evaluating progress tracking, landmark-based spatial grounding, and sequential route reasoning.
What are the three Boundary task variants? They are partial-arc, full-loop, and extended-loop tasks, distinguished by where the agent stops relative to completing and passing the starting point.
How does the Landmark pipeline ensure demonstrations can be executed? Decision waypoints are placed within 30 m of landmarks, and segment lengths and turning angles are quantized to the SatNav action space.
What is the main runtime bottleneck in the real-world deployment? Video forwarding from the controller to the workstation typically adds 3–5 seconds of latency. Model inference takes about 250 ms on average.