We present ContactGuard, a pre-contact grasp monitor that predicts whether an imminent grasp is likely to fail before the gripper closes. It combines a JEPA-style latent world model with a frozen linear failure probe. The world model is trained from unlabelled robot trajectories to predict action-conditioned future latents; the probe is trained from a smaller labelled grasp set to score the predicted post-contact latent. At deployment, the frozen model and probe run alongside a visuomotor policy and abort execution when the planned action chunk is predicted to fail.
We first define the world model and probe data, then describe the LeWM-style latent predictor, its multi-view architecture, the failure probe, and the online pre-contact monitor.
Problem Setting and Data
ContactGuard separates unlabelled latent dynamics learning from small-data outcome supervision. The world-model dataset contains robot trajectories, where synchronised observations from fixed cameras are paired with joint-space actions. No grasp-success labels are used to train the world model.
Background: LeWM-Style Latent Prediction
The predictor is a causal Transformer conditioned on actions through AdaLN-zero modulation. It is trained with next-embedding regression and the SIGReg anti-collapse regulariser, which encourages the latent distribution to match a standard Gaussian on random projections.
We reuse this encoder–predictor interface, but adapt it to multi-view observations, action-aligned prediction windows, and downstream pre-contact failure readout.
Experimental Results
We instantiate ContactGuard on grasp closure, a canonical short-horizon imminent-contact event with challenging visual geometry, occlusion, object variation, and a strict real-time pre-contact budget. We evaluate four grasp settings with ACT as the underlying chunked visuomotor policy. All metrics evaluate the predictor’s decision at the triggered pre-contact state, not the trigger itself or post-abort recovery.
Our experiments ask four questions:
Prediction Quality
Can predicted future latents anticipate execution outcomes better than single-view world models, current-state monitoring, and established runtime failure detectors?
Information Source
Does the signal arise from the imagined consequence of the specific proposed action, rather than from the current observation, visual shortcuts, or generic motion cues?
Robot-State Fusion
Does adding proprioceptive state improve grasp-relevant latent prediction?
Runtime
Is the forecast fast enough for the pre-contact window before gripper closure?
Closed-Loop Grasp-Outcome Prediction
Closed-loop metrics are collected from live rollouts; the world model, probe, and per-task threshold are frozen beforehand. All subsequent diagnostic tables use held-out offline replay splits.
Multi-View Versus Single-View
Multi-view fusion improves both balanced accuracy and AUC over the single-view LeWM baseline across all four tasks. The gain is largest on Towel, where the gripper–cloth contact region is frequently occluded from any single viewpoint, and smallest on Box, where the single-view baseline is already strong.
Pencil yields our model’s lowest absolute AUC, consistent with its training trajectories covering only a narrow band of grasp poses so that additional viewpoints add little headroom.
Conclusions, Limitations, and Future Work
We presented ContactGuard, a real-time, policy-decoupled predictive verifier for pre-contact manipulation. An independently trained action-conditioned latent world model evaluates the concrete action chunk proposed by an otherwise unchanged visuomotor policy and scores its predicted post-contact consequence with a lightweight failure probe.
Across four real-world grasp settings, ContactGuard outperforms matched current-state monitoring and the tested external runtime failure detectors. Holding the observation fixed while replacing only the proposed action also changes the predicted failure probability substantially, confirming that the monitor responds to the pending action rather than only to static visual risk.
These results support action-conditioned latent imagination as a practical substrate for vetoing likely failures before scene-changing contact.
Limitations
ContactGuard prevents failures by abstaining, but does not recover from them or complete the task after an abort. Post-abort task completion requires an external recovery module, which we leave for future work.
ContactGuard targets imminent contact events whose outcome is determined within the next action chunk; extending it to longer-horizon skills would require hierarchical or repeated event-level monitoring.
Future Work
Future work can close the loop after an abort by selecting a new action chunk, replanning from the preserved scene, or coupling the monitor with multi-sample policies such as Diffusion, Flow Matching, or Streaming policies.
Frequently Asked Questions
What does ContactGuard predict? It predicts whether an imminent grasp is likely to fail before the gripper closes.
How is the latent world model trained? It is trained from unlabelled robot trajectories to predict action-conditioned future latents without grasp-success labels.
What happens when a planned grasp is predicted to fail? The monitor aborts execution of the planned action chunk.
Can ContactGuard recover after an abort? No. Post-abort task completion requires an external recovery module.