We present an empirical study of experience-driven post-training in a real retail setting, identifying where self-generated experience improves robustness and where repeated refinement begins to degrade performance as self-generated rollouts dominate the training distribution, yielding practical guidance for deploying foundation-model-based humanoids in the real world.
Data-Efficient Recipe for VLA Post-Training
Finetuning a Vision-Language-Action (VLA) model for real-world deployment involves more than collecting demonstrations and training a model. This section presents the Data-Efficient (DE-) post-training recipe of DEED: the practical decisions and tricks we found necessary when finetuning a VLA (here GR00T N1.6) out of the box, which we believe help anyone in this position avoid common pitfalls.
Experience-Driven Learning Strategy
Adapting RECAP to GR00T poses an architectural challenge: unlike the tightly coupled VLA for which RECAP was proposed, GR00T separates perception and reasoning (a Vision-Language backbone) from control (a Flow-Matching action head). We address this by conditioning the policy through a text-based instruction prefix ("Advantage=True/False") and restricting the value function to camera observations and text instructions. The refinement procedure thus requires no architectural modification to the pretrained model and should be applicable to other VLA architectures.
Discussion
The two stages of DEED play complementary roles: the Data-Efficient stage turns an unusable checkpoint into a functioning policy from few demonstrations on a single GPU, while the Experience-Driven stage refines it from deployment experience, with one RECAP iteration yielding measurable if not statistically conclusive gains.
Unlike benchmarks with fixed poses and scene geometry, our evaluation involves unavoidable variability in robot initialization, object placement, and physical interaction, so success rates reflect end-to-end deployment robustness rather than performance under controlled assumptions. Within this context, the gains from experience-driven refinement do not compound across iterations: the second-iteration degradation is best explained as distributional drift, as self-generated rollouts come to dominate the training set, teleoperation demonstrations providing the broadest coverage of successful behavior lose relative influence, and advantage relabeling becomes increasingly tied to the policy's own sampling distribution, creating a feedback loop that progressively narrows the effective state-action distribution. Our in-distribution tool supports this, showing that the refined policies drift away from the demonstrated manifold even while remaining covered by the combined dataset.
A second, structural finding concerns reset behavior: refined policies tend to stop after a single placement rather than return to their start configuration, since the terminal completion state carries higher value than the intermediate poses of a reset, so reset transitions receive low or negative advantage and are suppressed during training. In practice we command the robot back to its starting pose rather than rely on the policy; addressing this within the learning framework would require changing the episode or reward definition, for instance by incorporating subtasks into the value function or explicitly assigning positive advantage to reset trajectories.
Finally, the in-distribution tool illustrates the value of treating monitoring as a first-class part of the framework. Because it measures shift in the policy's own latent representation and is agnostic to the specific VLA, it applies unchanged across both stages and across model versions. Here it served primarily as a diagnostic that turned an observed performance drop into a concrete, testable explanation, but the same signal could in principle be used online to detect when a deployed policy is leaving its training distribution and to trigger human intervention or data collection, closing the loop that DEED is built around.
Conclusions
We presented DEED, a systems-level framework for deploying VLA-based humanoids in real retail settings, demonstrated on a supermarket chip-restocking task with a Unitree G1-Edu and GR00T N1.6. Our main finding is that bridging the gap between pretrained VLAs and reliable deployment is primarily a systems integration challenge rather than an architectural one: without modifying the underlying model, careful data curation and targeted post-training turned a non-functional checkpoint into a capable policy on a single GPU, with data quality mattering more than data volume.
The two stages play complementary roles. The Data-Efficient stage does the bulk of the work (0% to 32% success), while a single Experience-Driven iteration raises success to 42%. Gains did not compound: a second iteration reduced robustness, likely from distributional drift as self-generated rollouts came to dominate the data, underscoring the need to maintain a balanced mixture of demonstrations and rollouts during continual learning, especially for high-variability humanoid tasks. Our in-distribution analysis provides a model-agnostic signal for diagnosing such degradation and could be extended to trigger intervention or data collection online.
These conclusions are limited by a single task, one platform, a binary success metric, few evaluation episodes, and two refinement iterations. Future work should evaluate DEED across more tasks and platforms, develop continual-learning strategies that preserve the demonstration-rollout balance, and couple deployment monitoring with automatic adaptation.
Frequently Asked Questions
How does DEED differ from standard VLA finetuning approaches? DEED separates post-training into two distinct stages: a data-efficient stage that turns a non-functional checkpoint into a usable policy with minimal demonstrations, and an experience-driven stage that refines the policy using self-generated deployment data.
Why does a second iteration of experience-driven refinement degrade performance? Self-generated rollouts begin to dominate the training set, causing distributional drift as teleoperation demonstrations lose relative influence and advantage relabeling becomes tied to the policy's own sampling distribution.
What causes refined policies to stop performing reset behaviors? The terminal completion state carries higher value than intermediate reset poses, so reset transitions receive low or negative advantage and are suppressed during training.
Can the in-distribution monitoring tool be used during live deployment? Yes, the tool measures shift in the policy's latent representation and is model-agnostic, making it suitable for online detection of distribution drift to trigger human intervention or data collection.