Once trained with joint supervision, the model generates Chain-of-Dynamics (CoD) sequences and predicts future visual states conditioned on them. However, prediction under teacher-forced CoDs alone does not train the vision-language model (VLM) to plan the intermediate dynamics needed to reach a desired future state. Therefore, ThinkWorld rewards sampled CoDs according to the agreement between their imagined and observed futures, encouraging the VLM to reason about the world changes implied by its CoDs. Since this reward requires only observed transitions, it enables the VLM to explore alternative CoDs and learn from human and robot videos without action labels or executing candidate motions.
Scaling Chain-of-Dynamics Reasoning Through World Imagination
Reward Shaping from World Imagination
For each candidate CoD, ThinkWorld predicts its future visual representation and compares it with the observed future in latent space. The reward uses a motion-enhanced feature similarity score that gives greater weight to DINO feature agreement in patches with larger changes between current and future frames.
The score retains a uniform contribution from the full scene to preserve global consistency while preventing redundant background regions from dominating the reward. Motion weighting uses only observed frames, so scoring requires no action annotations.
Optimizing CoD Reasoning with World-Imagination Reinforcement Learning
ThinkWorld aims to learn CoDs that capture the intermediate dynamics leading from the current scene to a desired future state. To train this reasoning process, it samples alternative CoDs from the same visual context and rewards those whose imagined futures best match the observed future. These rewards let ThinkWorld explore alternative CoDs that could lead from the current scene to the observed future, allowing passive human and robot videos to guide action reasoning without action labels or online execution of candidate motions.
The VLM is updated with Group Relative Policy Optimization (GRPO) using normalized world-imagination rewards, while supervised replay preserves the motion grounding learned during supervised fine-tuning (SFT).
By comparing imagined and observed futures, this stage encourages the VLM to encode action-relevant dynamics in the representations later used for continuous control. All other components remain frozen during reinforcement learning, and neither CoD generation nor future prediction is required at deployment. Complete optimization details are provided in Appendix A.6.
Implementation Details
Real-Robot Experimental Setup
The real-robot evaluation uses a bimanual YAM system equipped with three cameras: one global top-view camera and one wrist-view camera for each arm. Only RGB observations are used for training and inference. The embodiment-adapted ThinkWorld-YAM model is then used as the initialization for efficient task-specific fine-tuning. Additional implementation and training details are provided in the appendix.
Real-World Generalization Under Distribution Shifts
Real-world generalization is evaluated under matched in-domain (ID) and out-of-distribution (OOD) settings across three manipulation tasks: placing cups on a hanger, color matching, and object sorting. OOD settings introduce changes in object color or category and, in some cases, require trajectories not observed during task-specific training.
For each task, demonstrations are collected across at least six object configurations, and multiple settings are evaluated under both ID and OOD conditions, with five trials per setting. These tasks require the policy to identify task-relevant object attributes, such as color and category, and adapt its motion accordingly.
Conclusion
ThinkWorld is a vision-language-action (VLA) learning framework that learns transferable Chain-of-Dynamics through reinforcement learning with world imagination. By grounding action thoughts in the world changes they induce, ThinkWorld learns a common representation across human and robot embodiments and extends action reasoning to videos without explicit action annotations.
Experiments demonstrate effective human-to-robot transfer and strong generalization to unseen tasks and environments, validating the effectiveness of learning transferable action reasoning from observed world dynamics and its potential for scaling robot learning with diverse human and robot experience.
Model Configuration
Continuous actions are produced by an action expert initialized from MolmoAct2. The expert contains 36 non-causal Transformer blocks with hidden width 768 and operates in FP32. It exposes a maximum action tensor size; a two-dimensional validity mask restricts each example to its dataset-native horizon and action channels.
The expert receives layer-wise Qwen key/value features from input tokens only; generated CoD tokens and future-query tokens are not part of its direct context.
Input and Output Configuration
All media precede the textual metadata in a single user turn. When temporal context is available, the order is main-view history, current main view, and the remaining current views in their dataset-defined order. History contains at most 16 main-view frames sampled at 2 Hz and ordered from oldest to newest. The current main image is not repeated in the history.
The complete VLA input/output template is shown in the paper. Bracketed lines are optional, and the textual portion is rendered as one normalized string rather than as separate lines.
Real-Robot Setup
The BimanualYAM robot system is used to collect the in-lab dataset and evaluate trained policies. It comprises two YAM arms and three RealSense D405 cameras: one providing a top view and one mounted on each wrist. Policies use RGB observations only.
Policy inference runs on a single NVIDIA RTX 3090. To account for differences in inference latency, each policy is queried and its actions executed at the same 15 Hz control rate, while the remaining system components run at 30 Hz. For each predicted action chunk, the first half is executed, then the policy replans using the latest observation.
Additional PCA and Attention Analysis
Additional qualitative visualizations extend the analysis in the main text. Future-representation PCA and attention visualizations use Qwen layers 24–27. PCA and attention are descriptive analyses and are not used as causal evidence.
The expanded PCA visualization of future-prediction representations compares five CoD conditions: the original CoD, shuffled CoD, empty CoD, a donor CoD from another episode, and no CoD. The comparison illustrates how predicted future representations vary with CoD conditioning.
The extended CoD-to-input-observation attention maps also show attention from future-prediction tokens to the input observation. CoD attention tends to concentrate on dynamic, task-relevant regions, whereas future-prediction attention is distributed more broadly across the scene. This pattern suggests that CoD emphasizes localized changes relevant to action, while future prediction incorporates wider scene context.
Inference Efficiency
Steady-state batch-1 policy latency was measured with CUDA graph capture disabled, using eager inference without static-shape graph setup. All models were measured on the same NVIDIA H100 80GB GPU, with five warm-up calls followed by 50 CUDA-synchronized timed calls. Each model uses its native deployed input contract, so the measurements reflect practical policy latency rather than an identical-input architecture ablation.
ThinkWorld uses the reasoning process only during training, where it encourages the VLM to learn more action-centric representations. The deployed policy does not execute this process at inference, so each policy call avoids its additional runtime computation. This separation offers an explanation for how training-time reasoning can support strong task performance while preserving low inference latency.
Frequently Asked Questions
What does ThinkWorld learn through world imagination? It learns transferable Chain-of-Dynamics that represent intermediate dynamics leading from a current scene to a desired future state.
Does ThinkWorld require action labels to learn from videos? No. Its world-imagination reward compares imagined and observed futures, allowing learning from human and robot videos without action annotations.
What robot system is used for real-world evaluation? The evaluation uses a bimanual YAM system with two arms and three RealSense D405 cameras.
Does the deployed policy run the world-imagination reasoning process? No. The reasoning process is used during training, but neither CoD generation nor future prediction is required at deployment.