World models that predict future video frames from observed images and intended actions are a central component of modern robot learning. We present iMaC, an embodied world model that translates future actions into two complementary image-based controls: motion images that visualize future robot joint configurations through forward kinematics rendering, and contact images that encode whether and where the robot will touch an object via scene-level distance field heatmaps. These action-conditioned images are injected into the video diffusion model as visual control signals, guiding the generation of accurate future frames. On eight real-world cloth manipulation tasks, iMaC achieves state-of-the-art video prediction quality and policy evaluation scores without relying on domain-specific action embeddings or low-dimensional action-image projections.
Experimental Setup
Baselines and metrics. We compare iMaC with action-conditioned world-model baselines that inject future actions through learned action embeddings or sparse action-image controls. The main video-prediction metrics are computed between generated and ground-truth future videos under the same initial observations and action sequences. For policy evaluation, we report the correlation between world-model and real-world scores for each task. We ablate URDF/FK-rendered motion images, two-stream contact images, and the source of depth used for contact-image construction.
Implementation details. We train iMaC in two stages. The first stage trains a shared model on data from all eight tasks, and the second stage finetunes a task-specific model on each individual task; the final evaluation therefore uses one world model per task. Because contact images depend on the quality of predicted depth used for pointcloud construction, the first stage uses only motion-image controls, and contact-image controls are introduced during the second-stage finetuning after the model produces clearer RGB-D predictions. Training-time rollout is also warmed up: for the first 40 epochs, chunks use clean reference observations before enabling one-step generated references.
Ablation Study
Motion and contact images. The first two ablation columns isolate the two action-derived controls. Without contact images, the world model lacks contact-aware guidance for action following: the gripper does not grasp the cloth, even though later frames still generate an interaction-like cloth motion. Without motion images, the model lacks direct guidance about the future robot configuration; the gripper repeatedly attempts the motion but cannot produce the precise grasp.
Depth source for contact images. Keeping motion and contact controls but comparing how the depth used for contact-image construction is obtained, using DA3 depth partially improves action following because both controls are present, but its contact geometry is less consistent than iMaC's RGB-D world-model state: compared with the ground truth, the gripper still misses the cloth corner.
Limitation
iMaC relies on accurate 3D information to train depth prediction and to construct pointcloud-based contact images. In the current system, depth supervision is estimated by Depth Anything 3 (DA3) from multi-view RGB observations and camera poses, which can introduce centimeter-level errors in manipulation scenes. The two-stream contact images remain useful because they are heatmaps over distance fields, so the model can exploit coarse approaching and separating trends rather than exact metric contact at every pixel. Higher-quality depth sensors or manipulation-adapted depth models should further improve contact timing, collision localization, and long-horizon rollout reliability.
Frequently Asked Questions
What makes iMaC different from other action-conditioned world models? iMaC translates future actions into visual control signals — motion and contact images — rather than relying on learned action embeddings or sparse action-image projections, enabling more precise and interpretable action following.
How does iMaC handle tasks where precise contact with objects is critical? It constructs two-stream contact images from scene-level distance field heatmaps that encode approaching and separating contact trends, allowing the model to generate accurate grasp interactions even without exact metric contact at every pixel.
What training strategy does iMaC use for depth-dependent contact images? The model is trained in two stages: first with motion images only, then contact-image controls are introduced during finetuning after the model produces clearer RGB-D predictions, avoiding noise from poor early depth estimates.
Where does iMaC currently face limitations in real-world deployment? Its depth supervision relies on multi-view RGB estimation which can introduce centimeter-level errors in manipulation scenes, affecting contact timing and long-horizon rollout reliability.