Ablation Study: Predicting and Conditioning on Subtasks
Next, we assess the importance of predicting the active subtask and conditioning value estimation on it for shirt-hang.
In the first variant, we remove subtask prediction and conditioning, training the model with the subtask-level temporal-difference loss alone. In the second variant, we retain the subtask prediction loss but do not condition the value head on the subtask. The subtask is therefore not decoded at inference time and is represented only implicitly in the model’s activations.
Discussion, Conclusion, and Future Work
Finally, our evaluation uses the Q-function to select among action chunks proposed by a fixed base policy, so steering is limited by the quality and diversity of these candidates.
Producing more exploratory base policies and using the learned Q-function to directly improve the policy is a natural next step. Our experiments also fine-tune the generalist value function on each target task; evaluating transfer with less downstream data, or without task-specific fine-tuning, would further clarify the scope of its generalization.
Additional Experimental Results
The ablation in Section 4.4 separates the benefit of the auxiliary next-token prediction (NTP) loss from that of explicitly conditioning values on the decoded subtask.
Skipping subtask decoding is an appealing way to reduce inference latency: joint training might allow latent activations to carry the relevant subtask information implicitly. A similar motivation appears in video–action models such as UVA and Fast-WAM, which retain video supervision while bypassing explicit video generation at action-inference time.
We compare SeeQ with the variant that retains the auxiliary NTP loss but does not condition its value head on subtask tokens. Both use subtask-level TD-BoN and the same NTP weight of [formula: the original text specifies the NTP loss weight symbolically], and are fine-tuned on shirt-hang with the same learning-rate schedule, starting from their corresponding RoboCOIN-pretrained checkpoints.
Frequently Asked Questions
What does the ablation study evaluate? It evaluates the importance of predicting the active subtask and conditioning value estimation on that subtask.
What happens when subtask prediction and conditioning are removed? The model is trained with the subtask-level temporal-difference loss alone.
Why might skipping subtask decoding reduce inference latency? Joint training might allow latent activations to carry relevant subtask information implicitly.
What is a proposed direction for future work? Future work could use more exploratory base policies and apply the learned Q-function to directly improve the policy.