An object-centric world model gives underwater remotely operated vehicles (ROVs) a way to predict salvage consequences before moving a gripper. C3-JEPA combines cross-view camera fusion, target-and-gripper identity binding, and control-conditioned latent prediction so planning can compare candidate actions while preserving the visual and pose information needed for manipulation.
What did the researchers build?
Researchers built C3-JEPA, a learned world model for underwater ROV salvage. A world model is a compact internal representation that lets a robot estimate how a scene will change after an action. Instead of predicting every future camera pixel, C3-JEPA predicts how important objects and their relationships evolve.
The system focuses on three kinds of visual information. Two dedicated task-object entries represent the salvage target and the ROV gripper. Three additional context entries represent other scene elements without forcing them into fixed identities. This structure prevents the target and gripper from disappearing into a single undifferentiated scene embedding.
C3-JEPA also combines views from multiple cameras. Underwater salvage scenes often contain occlusions, reflections, poor visibility, and viewpoints that reveal different parts of the same object. The model aligns the target and gripper across cameras while allowing context features to remain flexible.
The visual encoder follows the VideoSAUR approach, which produces object-like “slots” from real-world video without requiring motion or depth inputs. That choice keeps the perception pipeline practical for an ROV operating with ordinary camera observations.
The resulting representation supports model-predictive control (MPC), a planning method that evaluates possible short action sequences before selecting the next control command. For an ROV, those commands can describe vehicle motion or gripper-related movements during approach, alignment, and retrieval.
What were the key results?
C3-JEPA’s main result is that object-centric supervision preserves manipulation-relevant information better than a reconstruction-free latent objective used on its own. The comparison centers on SIGReg, a regularizer intended to shape learned representations. When SIGReg is used without reconstruction and object binding, the representation compresses visual information toward general dynamics statistics and loses details needed for pose and joint-level estimation.
The combined design performs better because reconstruction-guided slots retain visual evidence, weak binding keeps the target and gripper identifiable, and a small SIGReg contribution regularizes the latent space without dominating it. The same regularizer that harms downstream heads in reconstruction-free training improves those heads when paired with reconstruction and object binding.
The evaluation follows an evidence ladder rather than relying on one headline score. It checks object-mask Dice, held-out-view feature error, slot stability, latent prediction error, recursive rollout quality, observable state estimates, action ranking, and conservative closed-loop simulation.
The supplied results text does not report numerical benchmark values, so no accuracy or percentage improvement can be stated without inventing data. It does, however, establish a clear comparison: SIGReg alone is insufficient for manipulation, while the full object-centric design retains more information for downstream control and transfer.
How does the underwater world model work?
C3-JEPA has four important stages: object-centric encoding, cross-view fusion, action-conditioned prediction, and planning.
First, the encoder converts camera observations into a small collection of slots. Each slot is a learned feature vector representing an object or part of the scene. The first two slots are reserved for the target and gripper. Their identities are weakly bound across cameras, meaning the training process encourages the same physical entity to occupy the same slot without requiring perfect manual tracking. The remaining context slots are not forcibly aligned, which gives them freedom to represent changing background content.
Second, the model fuses evidence from different camera views. Cross-attention allows a bound object slot to inspect visual tokens from cameras that currently see the object. During training, one view is deliberately held out from the fusion input. A decoder must then predict the missing feature evidence for that view. This creates a direct test of whether the fused object representation contains information from complementary viewpoints rather than simply copying the visible image.
Third, the predictor receives the ROV’s normalized control signal. It uses that signal to estimate how object-centric latent features will change after the vehicle acts. This is a latent prediction task: the model predicts a future representation instead of generating a full future image. The approach reduces computational demand while keeping the state focused on objects relevant to manipulation.
Finally, MPC evaluates candidate action sequences through the predictor. The planner can compare possible future target and gripper states, reject actions that lead to poor alignment, and select the next executable command. Imagined rollouts extend this process beyond a single prediction step.
The evaluation separates perception from prediction and prediction from decision quality. Teacher-forced tests measure one-step latent accuracy. Recursive rollouts test whether errors compound over time. Frozen readouts then convert latent states into observable quantities such as position, relative pose, or image-space location.
Why does this matter for robotics?
Underwater salvage is difficult because the robot operates through cameras, often with limited visibility and delayed human feedback. A gripper that approaches from the wrong angle can disturb the target, lose contact, or turn a recoverable object into a more difficult retrieval. Planning against predicted consequences is therefore more useful than reacting only to the latest image.
C3-JEPA addresses a practical weakness in many learned robot representations: compression can remove exactly the details needed for manipulation. A compact state is valuable only if it still preserves target pose, gripper location, and the relationships between them. Separating those elements gives the planner a more actionable state than a generic scene embedding.
The cross-view design also fits how many inspection and salvage ROVs are equipped. Multiple cameras can provide overlapping evidence, but only if the software can combine those views without confusing the target with background objects. Held-out-view training directly tests that capability.
For operators, the work points toward ROVs that can assist with inspection, retrieval, cable handling, and underwater infrastructure maintenance. It is especially relevant when evaluating used industrial robots for adaptation to harsh or remote work environments, where perception software can determine whether an existing platform is useful beyond basic teleoperation.
What are the limitations and open questions?
The supplied results do not provide numerical scores, hardware demonstrations, deployment conditions, or comparisons against a broad set of underwater control baselines. That makes it difficult to judge how much the representation improves real salvage success, latency, or operator workload.
The held-out-view task also tests feature prediction rather than complete physical reconstruction. A model can retain useful visual evidence without understanding water currents, buoyancy, tether forces, contact dynamics, or deformable objects. Those factors can strongly affect ROV manipulation.
Another open question is long-horizon reliability. Latent prediction errors accumulate during recursive rollouts, and an incorrect object identity could cause the planner to optimize the wrong target. Closed-loop experiments with real hardware, changing visibility, moving debris, and failed grasps are needed to establish operational robustness.
The model also depends on task-object assignment and training data that expose enough variation in camera views and actions. Performance under unseen salvage objects and unfamiliar underwater environments remains unresolved.
Frequently Asked Questions
What is C3-JEPA? C3-JEPA is an object-centric world model that fuses multiple camera views and predicts how underwater scene objects change after ROV actions.
Why are object-centric representations useful for ROV salvage? They preserve separate information about the target, gripper, and surrounding context, giving a planner clearer variables for approach and manipulation.
Does the system generate future underwater images? No. It predicts future latent features, which are compact internal representations, instead of reconstructing every future camera pixel.
Was the system tested on a physical salvage ROV? The supplied text describes evaluation protocols and closed-loop simulation, but does not provide enough detail to confirm the scope of physical-robot testing.
Conclusion
C3-JEPA combines object-aware visual encoding, cross-view evidence, and action-conditioned latent prediction for underwater ROV planning. Its central finding is that reconstruction and object binding preserve manipulation information that a reconstruction-free regularized representation loses.