Real-World Dataset
To capture expert flight behaviors, the authors construct UAV-Flow by collecting and annotating real-world flight demonstrations.
Instruction Annotation
Language annotations are created through a quality-controlled process. Annotators first filter clips with ambiguous behaviors, unsafe execution, unclear scene relations, or inconsistent motion semantics. They then assign concise commands describing the intended motion and, when needed, the spatial relation to a visible object.
To support standardized annotation and efficient data processing, UAV-Flow adopts a fixed command set for canonical task labels. It also constructs an open-vocabulary command set by paraphrasing the canonical annotations with large language models. The resulting real-world dataset contains 30,692 trajectories across eight major motion categories, paired with egocentric videos, aligned UAV states, and instructions.
Data Analysis
UAV-Flow and UAV-Flow-Sim provide complementary coverage for the Flow formulation. The real-world dataset contains 30,692 expert-piloted trajectories across eight major motion categories, capturing diverse visual contexts, human control styles, and physically executed UAV dynamics. The simulation dataset contains 10,109 trajectories generated under matched instruction and state formats, enabling controlled testing and simulation-based policy optimization.
The authors also construct a simulation test set comprising 273 annotated trajectories, covering all major action types in the dataset to facilitate systematic evaluation on the Flow task. Each test episode specifies its task label, initial state, instruction, reference trajectory, and task-specific completion criteria, establishing a reproducible protocol for closed-loop evaluation.
Hybrid Imitation Pretraining
The policy is pretrained by imitation using a mixture of real-world flight demonstrations from UAV-Flow and simulated demonstrations from UAV-Flow-Sim. Both sources follow a unified episode format and are expanded into frame-level training samples. The pretraining dataset comprises approximately 20,000 demonstration trajectories.
Real-world demonstrations provide authentic outdoor visual diversity, sensor stochasticity, and physical motion dynamics. Simulated demonstrations offer controllable and scalable coverage of Flow maneuvers and task configurations. Joint training on these complementary data sources enables the policy to map egocentric visual observations, UAV states, and language instructions into future action chunks.
For each training sample, the expert future action chunk is converted into categorical labels using normalization and discretization, and action prediction is supervised with categorical cross-entropy. The loss is averaged over the number of future steps and four action dimensions:
The normalized expert action is mapped to its corresponding discrete action bin.
The pretrained policy predicts short-horizon UAV action chunks conditioned on language instructions. It serves as the policy for subsequent closed-loop evaluation and initializes reinforcement-learning post-training.
Real-World Deployment
To deploy the trained policy on a physical UAV, the authors use a ground-station and drone collaborative execution architecture. Because onboard computing resources are limited, VLA inference is performed on the ground station. The UAV transmits egocentric images and state estimates to the ground station over a wireless link. Based on the received observations and language instruction, the ground station generates action chunks, converts them into control targets, and transmits them back to the UAV for execution by its onboard flight controller.
The policy outputs position-yaw actions in a local coordinate frame. The execution layer decodes the discrete predictions into continuous actions and incrementally converts each action chunk into position-yaw waypoints using the UAV pose associated with the input observation.
Experiments
FlowVLA is assessed through closed-loop simulation, ablation studies, and physical UAV deployment. The experiments first compare the pretrained policy with adapted baselines and evaluate the effects of GRPO and TGRPO post-training on task completion and trajectory similarity. Ablation studies then compare alternative reinforcement-learning algorithms and examine the effects of training-data composition, action representation, and VLM feature selection. Finally, the paper presents flight execution examples in outdoor and indoor real-world scenes.
Baseline Models
The authors select five representative methods as baselines and adapt them to the Flow task. All baselines are trained by imitation learning on the same simulated demonstrations, with their VLM components fine-tuned using LoRA. To provide a common closed-loop execution interface, each model’s outputs are converted to local position-yaw commands and executed by the same simulator-side controller.
FlowVLA denotes the policy obtained by imitation pretraining on mixed real-world and simulated demonstrations. FlowVLA-GRPO and FlowVLA-TGRPO denote variants that undergo reinforcement-learning post-training from this policy using the corresponding algorithm.
To examine both optimization efficiency and performance on these tasks, all policies are evaluated on the full test set. The evaluation quantifies performance improvements on the four reinforcement-learning training tasks alongside capability retention on the remaining tasks.
Ablation Studies
Training curves further illustrate how the methods perform over the course of training. Overall success rate generally increases for FlowVLA-GRPO and FlowVLA-GDPO, while GSPO remains near 90% in later training and DAPO declines substantially. FlowVLA-GRPO maintains relatively high NDTW in later training, whereas FlowVLA-GDPO and FlowVLA-GSPO show declines.
The late-stage performance degradation of FlowVLA-DAPO and FlowVLA-GSPO may be partly related to their lack of reference-policy KL regularization, as their updates do not explicitly constrain cumulative deviation from the pretrained reference policy.
Effect of Mixed Training Data
Adding real-world flight demonstrations increases FlowVLA’s mean success rate from 73.14% to 75.58%, a gain of 2.44 percentage points. Mean NDTW also increases slightly, from 0.399 to 0.402. These findings indicate that, beyond facilitating direct real-world policy learning, physical flight demonstrations provide complementary training signals to simulated data, enhancing closed-loop task success in simulation.
Real-World Deployment Evaluation
Following the deployment scheme described above, the authors deploy a VLA policy on a physical UAV and evaluate its ability to execute language instructions in closed-loop flight in outdoor and indoor scenes.
During flight, the UAV continuously transmits egocentric images and state estimates to the ground station, which predicts action chunks. The execution layer converts the predictions into waypoint commands, selects look-ahead waypoints based on end-to-end delay, and dispatches them to the flight controller. This maintains continuous closed-loop flight despite communication and policy-inference delays.
Acknowledgments
This research is supported in part by the National Key R&D Program of China (Grant No. 2024YFF1401400), the Science Fund for Creative Research Groups (No. T2621006), the National Natural Science Foundation of China (Nos. U23B2010, 62461160308, 62576024), the Fundamental Research Funds for the Central Universities (No. 501RCQD2025141003), and BeiHang GanWei Project (No. 502GWXM2024141001).
Physical UAV Platforms
Physical execution is tested in indoor and outdoor environments using two UAV platforms: a P600 UAV platform and a DJI Tello.
Frequently Asked Questions
What is included in the UAV-Flow real-world dataset? It contains 30,692 trajectories across eight major motion categories, paired with egocentric videos, aligned UAV states, and instructions.
How many trajectories are in the simulation dataset and test set? UAV-Flow-Sim contains 10,109 trajectories, and the simulation test set contains 273 annotated trajectories.
What does hybrid imitation pretraining combine? It combines real-world flight demonstrations from UAV-Flow with simulated demonstrations from UAV-Flow-Sim, in a pretraining dataset of approximately 20,000 trajectories.
How does the physical UAV receive and execute policy actions? The UAV sends images and state estimates to a ground station for inference, then receives position-yaw waypoint commands for its onboard flight controller.