A common pretrained VLA instantiation couples a vision-language model (VLM) with a flow-matching-based action expert (AE). During denoising, the policy conditions on a noisy action chunk [a mathematical expression representing the denoising prediction]. Through large-scale pretraining, the VLM acquires general vision-language representations, while the AE learns reusable motor priors for action generation. These complementary pretrained capabilities form the basis for downstream adaptation.
Problem Formulation
PriorVLA addresses the challenge of adapting pretrained Vision-Language-Action (VLA) models to downstream tasks while preserving the general-purpose knowledge acquired during large-scale pretraining. When fine-tuned on narrow downstream datasets, VLA models risk losing their pretrained motor priors and visual grounding capabilities, which harms out-of-distribution (OOD) generalization.
Dual Action Experts
A pretrained AE contains reusable motor priors encoded in its denoising dynamics, but full fine-tuning can shift these priors toward narrow training-distribution action patterns. PriorVLA decouples prior preservation from downstream specialization by branching the pretrained AE into Dual Action Experts.
The Prior Expert serves as a read-only source of motor priors: its frozen weights preserve pretrained motor knowledge, while its internal denoising representations provide motor-prior features to the adapted policy.
Expert Queries
Structural separation preserves the pretrained AE, but does not by itself make its priors usable for downstream action generation. Since pretrained priors are distributed across layers and denoising steps, PriorVLA introduces Expert Queries as learnable token interfaces attached to different functional paths. They comprise three groups:
- Scene Queries: Learnable tokens inserted into the VLM input sequence that capture task-relevant scene priors from VLM representations. Their layer-wise key-value caches provide a compact scene-prior interface to the Adaptation Expert.
- Motor Queries: Learnable tokens inserted into the frozen Prior Expert that capture motor priors from its denoising representations.
- Action Queries: Learnable tokens inserted into the Adaptation Expert that integrate scene and motor priors for downstream action generation.
Attention-Mask Implementation
The attention mask is implemented with a block structure where tokens within the same block use bidirectional self-attention, while later blocks attend to earlier blocks but not vice versa. For the frozen Prior Expert, noisy action tokens follow the original pretrained action path and are not allowed to attend to Motor Queries. Motor Queries can attend to the Prior Expert's noisy action tokens but are blocked from attending to the VLM prefix or Scene Queries, ensuring they focus on motor-prior features rather than scene features.
For the trainable Adaptation Expert, Action Queries and noisy action tokens share bidirectional interaction. The Adaptation Expert attends to original VLM features, Scene Query features, and Motor Query features, but not to raw Prior Expert action states, which improves training stability.
Training
PriorVLA optimizes the full Adaptation Expert, the three groups of Expert Queries, and the VLM vision encoder, while keeping all other VLM parameters and the Prior Expert frozen. Overall, PriorVLA updates approximately 25% of the parameters updated by full fine-tuning. This preserves pretrained prior sources in the frozen VLM components and Prior Expert, while allowing the vision encoder and adaptation branch to specialize to downstream data.
Base VLA Backbone
PriorVLA is built on a pretrained VLA backbone that couples a VLM with a flow-matching action expert. The VLM consists of a SigLIP-style vision encoder and a Gemma-2B language backbone, encoding multi-view images, language prompts, and proprioceptive states. The action expert follows a Gemma-300M-style transformer architecture and denoises future action chunks for continuous action generation.
Dual Action Experts Implementation
The Prior Expert and Adaptation Expert are both initialized from the pretrained AE. The Prior Expert is frozen and serves as a read-only source of motor-prior representations, while the Adaptation Expert is trainable and drives the denoising trajectory that produces the final action chunk. At each denoising step, both experts receive the same current noisy action chunk. The Prior Expert's denoising output is discarded (not used in loss), but its internal representations are exposed through Motor Queries.
Expert Queries Implementation
PriorVLA uses three groups of learnable Expert Queries:
- Scene Queries are provided as additional input tokens to the VLM alongside multi-view image tokens, prompt tokens, and proprioceptive-state tokens. They capture scene-prior features.
- Motor Queries are provided as additional input tokens to the frozen Prior Expert at each denoising step. They read motor-prior features without updating the noisy action trajectory.
- Action Queries are provided as additional input tokens to the trainable Adaptation Expert at each denoising step. They integrate scene-prior and motor-prior features to guide the Adaptation Expert's denoising prediction.
Hyperparameters
Across all experiments, PriorVLA uses AdamW with global-norm gradient clipping at 1.0. Trainable parameters are optimized in float32, and frozen pretrained parameters are stored in bfloat16. Unless otherwise specified, all runs use random seed 42. PriorVLA applies grouped learning-rate multipliers to four parameter groups:
| Parameter Group | Learning Rate Multiplier |
|---|---|
| Scene Queries | 2.0 |
| Motor Queries | 4.0 |
| Action Queries | 4.0 |
| Remaining trainable parameters | 1.0 |
Experiments
To comprehensively evaluate PriorVLA, experiments were conducted on RoboTwin 2.0, LIBERO, and eight real-world tasks across two robot embodiments. Evaluation was organized around five questions:
- Can PriorVLA improve downstream adaptation over full fine-tuning and strong VLA baselines?
- Does preserving pretrained priors improve OOD generalization?
- Are the benefits stronger when downstream data coverage is limited?
- Do these gains transfer to real-world robots across embodiments and evaluation conditions?
- Are the gains explained by preserved pretrained priors and learnable query interfaces?
Training Data
For RoboTwin 2.0, one model was trained per task using the official setting. A representative 13-task subset was used from the full 50-task benchmark. The standard setting uses 50 clean demonstrations per task. Few-shot and large-data settings use 10 and 250 clean demonstrations per task, respectively.
RoboTwin 2.0 Evaluation
For RoboTwin 2.0, evaluation was performed on the same 13 tasks used for training under official Easy (in-distribution) and Hard (OOD) modes. The final 30k checkpoint was evaluated with 300 rollouts per task for each mode, using the official simulator success signal.
LIBERO Evaluation
For LIBERO, four suites were evaluated: Spatial, Object, Goal, and Long. Each suite contains 10 tasks with 50 rollouts each, totaling 500 rollouts per suite. Initial states from LIBERO were used, with success measured by the LIBERO environment signal. The policy replans every 10 environment steps, and the best checkpoint from 20k steps onward was reported.
Real-World Evaluation
Real-world evaluation tested the final 30k checkpoint on 20 ID trials and 20 OOD trials per task. The same policy was evaluated under both settings without retraining. For all real-world tasks, the policy predicts an action chunk of length [a specific chunk length], executes the first 15 actions, and re-queries the policy with a new observation.
Why Prior Preservation Helps OOD Generalization
Out-of-distribution generalization in downstream VLA adaptation is challenging because downstream demonstrations often cover only a limited range of visual conditions, object configurations, and workspace layouts. Full fine-tuning updates the pretrained action-generation pathway directly with this restricted supervision, which may shift pretrained representations and action priors toward narrow downstream patterns.
PriorVLA mitigates this by separating prior preservation from downstream specialization. The frozen Prior Expert preserves the pretrained action-generation pathway, while the Adaptation Expert specializes to downstream action generation. Expert Queries provide learnable interfaces for using preserved priors. This design enables reuse of priors learned during large-scale pretraining—such as object-centric visual grounding, affordance-related scene understanding, and general motor regularities—when evaluation scenes contain unseen conditions.
Why Prior Preservation Helps Few-Shot Adaptation
Few-shot adaptation amplifies the risk of over-specialization because a small number of downstream demonstrations covers only a narrow subset of initial states, visual appearances, and successful action trajectories. The frozen Prior Expert maintains a stable source of motor-prior features, while the Adaptation Expert learns the downstream action mapping. Expert Queries serve as learnable interfaces for selecting and integrating preserved priors. The model does not need to recover all relevant structure from few-shot data alone; it can reuse pretrained priors and learn how to integrate them for the downstream task.
Broader Impacts
PriorVLA aims to improve data-efficient adaptation of pretrained robot policies. Its potential positive impact is to reduce the amount of task-specific robot data needed for downstream adaptation, lowering development costs and improving robustness under moderate distribution shifts. Potential negative impacts may arise if stronger robot policies are deployed without sufficient validation, leading to failures under unseen scenes or unsafe environments. The method could also be misused in inappropriate automation settings. Real-world experiments are conducted in controlled lab settings with appropriate safety measures.
Ablation Study
Ablations were conducted on a representative subset of six RoboTwin 2.0 tasks, following the same training and evaluation protocol. Full task list and per-task results are provided in the appendix.
Conclusion and Limitations
While PriorVLA demonstrates strong adaptation across simulation and real-world settings, several aspects remain to be studied. The RoboTwin 2.0 results are reported on a representative 13-task subset rather than the full 50-task benchmark because each task requires a separate model. In real-world evaluation, OOD factors are applied jointly, which reflects deployment-like shifts but does not disentangle individual effects. PriorVLA introduces additional inference computation because the frozen Prior Expert is executed during denoising; though manageable in the chunked-control setting, more efficient implementations or cached prior readouts could further reduce cost. A finer-grained analysis of how scene and motor priors emerge, interact, and evolve across layers and denoising steps remains future work.
Frequently Asked Questions
Q1: What problem does PriorVLA solve? PriorVLA addresses the challenge of adapting pretrained Vision-Language-Action (VLA) models to downstream tasks while preserving the general-purpose knowledge acquired during large-scale pretraining, particularly for out-of-distribution generalization and few-shot adaptation.
Q2: How are the Dual Action Experts configured? The Prior Expert is frozen and serves as a read-only source of motor-prior representations, while the Adaptation Expert is trainable and drives the denoising trajectory that produces the final action chunk. Both are initialized from the same pretrained action expert.
Q3: What are the three types of Expert Queries? Scene Queries capture task-relevant scene priors from the VLM, Motor Queries capture motor priors from the frozen Prior Expert's denoising representations, and Action Queries integrate both priors inside the Adaptation Expert for action generation.
Q4: How much of the model is updated during training? PriorVLA updates approximately 25% of the parameters updated by full fine-tuning, keeping frozen all VLM parameters except the vision encoder, plus the entire Prior Expert.
Q5: What benchmarks were used for evaluation? Evaluation was conducted on RoboTwin 2.0 (13 tasks), LIBERO (four suites: Spatial, Object, Goal, Long), and eight real-world tasks across two robot embodiments.
Q6: How does PriorVLA improve OOD generalization? By preserving the frozen Prior Expert as a stable source of motor priors from large-scale pretraining, PriorVLA enables the model to reuse general-purpose motor and visual knowledge when evaluation conditions differ from the downstream training distribution.