Attacks on embodied AI systems are particularly concerning because model failures can translate into physical consequences. Prior work has studied attacks on perception modules, reinforcement-learning agents, and embodied policies, including perturbations that alter observations, timing, or environmental cues. These attacks typically target reactive decision-making. For a reactive policy a = π(o), an observation attack can be written as δ = argmax L(a_clean, a_adv) or, in closed-loop evaluation, as minimizing task reward or success. BadWAM naturally generalizes this view to predictive-action models whose outputs include both actions and imagined futures. The relevant question is no longer only whether the action changes, but whether the action changes while the imagination remains plausible.
Closest to our setting is recent work on oracle-level integrity attacks against imagine-then-act world models. That work studies VLA-style and world-model-based systems, where an imagined future is consumed by downstream oracles such as safety gates, MPC planners, or imagine-then-check verifiers. It shows that corrupting the trusted imagination can compromise systems that rely on future predictions. BadWAM studies a complementary failure mode. Rather than attacking the imagined future as the primary object, we target the alignment between imagination and action in deployed WAM controllers. We show that action and imagination can be adversarially desynchronized: the predicted future may remain close to the clean prediction while the executed action changes enough to cause task failure.
This distinction is central to our threat model. If a safety monitor trusts plausible imagined futures, then a purely imagination-based check may miss an attack that preserves future predictions while hijacking actions. BadWAM therefore treats the alignment between action and imagination as the security-critical object. Our action-only adversarial attack prioritizes task disruption, while our imagination-preserving adversarial attack explicitly studies the trade-off between attack strength and stealthiness.
BadWAM Framework
BadWAM is a unified framework for modeling and evaluating world-action drift attacks. The framework treats a WAM as a queryable predictive-action system and optimizes bounded visual perturbations at each replanning step. Its two attack instantiations correspond to two points on the attack-strength/stealthiness spectrum: an action-only adversarial attack that prioritizes execution failure, and an imagination-preserving adversarial attack that additionally constrains predicted-future drift.
Design Motivation
We therefore formulate BadWAM around two distances. d_action measures the deviation between clean and attacked action chunks, and d_imagination measures the drift between clean and attacked imagination. These distances are interface-level quantities: they do not require a particular WAM architecture, only access to the corresponding outputs. Conceptually, a stealthy adversarial attack seeks large action deviation under bounded visual perturbations while keeping imagination drift below a monitorable threshold:
max_δ d_action(a_clean, a_adv) subject to d_imagination(i_clean, i_adv) ≤ τ and ||δ||_∞ ≤ ε
This defines the core BadWAM attack surface. The two attacks below instantiate different relaxations of this objective.
Action-Only Adversarial Attack
The action-only attack captures the high-strength end of BadWAM. It treats the WAM as a queryable controller and focuses purely on execution disruption. Rather than enforcing any constraint on the imagined future, it searches for a bounded visual perturbation that maximizes the deviation between clean and attacked action chunks:
max_δ d_action(a_clean, a_adv) subject to ||δ||_∞ ≤ ε
The clean action chunk is used only as a reference; the attacker does not need ground-truth expert actions or task success labels during optimization. This makes the attack applicable to deployed policy interfaces where the observable output is the action command sent to the robot controller.
Although simple, this objective is already highly disruptive. In our closed-loop experiments, the action-only attack reduces one WAM variant from 96.5% to 43.1% task success. We therefore use it as the high-strength endpoint of BadWAM. At the same time, the attack is not merely unstructured output corruption: the induced shifts are unevenly distributed across both action channels and temporal horizon segments. For example, perturbations concentrate on task-relevant channels such as translation and gripper commands, while their temporal concentration varies across WAM variants. This motivates logging channel-level and horizon-level action statistics in our evaluation.
Imagination-Preserving Adversarial Attack
The action-only objective provides no control over what happens to the imagined future. For WAM deployments that expose imagined futures to planners, monitors, or imagine-then-check safety gates, this is insufficient: a stealthier attack should degrade execution while preserving the model's clean imagination. BadWAM therefore instantiates an imagination-preserving attack by solving the Lagrangian relaxation:
max_δ d_action(a_clean, a_adv) - λ · d_imagination(i_clean, i_adv) subject to ||δ||_∞ ≤ ε
The coefficient λ controls the trade-off between attack-strength and stealthiness. When λ = 0, the objective reduces to pure action disruption. As λ increases, the optimizer is discouraged from changing the imagined future, making the attack more stealthy but potentially less damaging.
When decoded future videos are available, we instantiate the imagination distance as an average frame-level distance:
d_imagination = (1/T) Σ_t ||i_clean_t - i_adv_t||_2
The same objective can be applied to latent futures by replacing the frame-level distance with a latent-space distance. In either case, the attack targets a WAM-specific failure: the robot may appear to imagine a plausible future while executing a desynchronized action.
Query-Based Online Optimization
BadWAM does not backpropagate through the WAM or access model parameters. At each replanning step, the attacker treats the WAM as a queryable input-output system and optimizes the attack objective with zeroth-order finite-difference queries. Concretely, for a current perturbation δ, the attacker samples random directions u and evaluates the scalar objective under positively and negatively perturbed observations. This gives the estimator:
∇̂L(δ) = (L(δ + σu) - L(δ - σu)) / (2σ) · u
This optimization is zeroth-order because it only uses objective values computed from WAM outputs; it does not require gradients, model weights, or training data. The same optimizer supports both BadWAM instantiations by changing only the scalar objective L: the action-only attack scores action deviation, while the imagination-preserving attack scores action deviation together with future-prediction consistency.
For each replanning step, BadWAM keeps the best perturbation found within the query budget. The final adversarial observation is then used to produce the action chunk executed by the robot. This online design avoids training a separate attack model and naturally adapts to the current observation, instruction, and robot state.
Closed-Loop Execution and Measurements
BadWAM is evaluated under closed-loop execution. The attack is recomputed at each replanning step, the WAM outputs an attacked action chunk, and the robot executes the selected actions before the next replan. This setting is more faithful than single-step action comparison because small action shifts may compound over time and cause task failure only after multiple replans.
We report three groups of measurements. First, we measure task-level attack effectiveness using closed-loop success rate and induced failures relative to clean execution. Second, we measure action disruption using action distance, action-channel statistics, and horizon-level action shifts. Third, for imagination-preserving attacks, we measure stealthiness using predicted-future distance and adversarial score. Together, these metrics separate two questions: whether the robot fails, and whether the failure is visible through the WAM's imagined future.
Failures Are Systematic Across Suites and Trials
To test whether the attack is concentrated in a few task types or persists across the benchmark, we break success down by LIBERO suite. The largest drops occur on tasks requiring precise spatial control or long-horizon execution. For the action-only WAM, Spatial success falls from 96.5% to 16.0%, Goal success falls from 97.0% to 40.0%, and Long-horizon success falls from 92.5% to 23.5%. Object tasks remain comparatively robust at 93.0%. The joint and IDM WAMs show the same qualitative pattern, although the degradation is less extreme. For example, the joint WAM under action-only attacks reaches 52.0% success on Spatial and 42.0% on Long-horizon, but remains at 92.5% on Object.
This suite-level structure suggests that BadWAM is not simply destroying visual perception uniformly. It is most effective when small action deviations can compound through geometry, contact timing, or sequential dependencies. Object-centric tasks often tolerate moderate drift as long as the relevant affordance remains visible and reachable. Spatial and long-horizon tasks leave less margin for a wrong approach direction, grasp position, or gripper timing.
What Do These Results Imply for WAM Safety?
Across all research questions, the results point to a subtle failure mode for imagination-based safety checks. A natural WAM monitor might inspect predicted futures and accept an action if the future looks plausible. BadWAM shows that this criterion is incomplete. The relevant security property is not plausibility of the imagined future in isolation, but synchronization between the imagined future and the action that will actually be executed. In the matched-strength run, the imagination-preserving objective lowers predicted-future drift on 39 out of 40 LIBERO tasks while still inducing substantial closed-loop failures. A monitor that only scores visual future plausibility could therefore be satisfied even after the action channel has shifted toward failure.
This also explains why the attack is WAM-specific. For a conventional action-only policy, an adversarial perturbation can only be assessed through its effect on the action or final task outcome. For a WAM, the attacker can exploit a gap between two interfaces: the future-imagination interface and the action interface. The action-only BadWAM objective targets the high-strength endpoint of this spectrum; the imagination-preserving objective targets a stealthier endpoint. The latter is especially relevant for systems that expose predicted futures to planners, runtime monitors, or human supervisors. Even when the predicted future remains visually close to the clean prediction, the executed action chunk may no longer be the action that would realize that future.
The defense experiments reinforce the same lesson. Preprocessing can be a useful diagnostic, but it is not a principled solution to WAM safety. A practical defense must preserve clean control while checking whether action and imagination remain mutually consistent over the closed-loop trajectory. The low-recall detector in our experiments further shows that moderate AUROC is insufficient for robot safety: runtime monitors must operate at very low false-positive rates, where recall is hardest to maintain.
Finally, the transfer results suggest that the vulnerability is not merely an implementation artifact. Perturbations optimized on one WAM variant remain damaging on another, indicating shared observation-space directions that affect action generation. A WAM safety evaluation should therefore go beyond clean and attacked success rates. It should also report action distance, predicted-future distance, input perceptibility, horizon-level shifts, and failure distributions across task families. Together, these metrics distinguish overt action hijacking from stealthier imagination-preserving failures, and test whether a defense restores action-imagination synchronization rather than masking one perturbation pattern.
Summary of Findings
The evaluation supports six conclusions. First, BadWAM substantially lowers closed-loop success across WAM variants, with around 30%-50% drops. Second, the failures are systematic: they persist across repeated trials and are strongest on spatial and long-horizon tasks. Third, the attack induces structured closed-loop failures rather than immediate random behavior. Fourth, imagination preservation creates a measurable stealth trade-off: it keeps predicted futures closer to clean rollouts while retaining comparable perturbation budgets and substantial attack strength. Fifth, BadWAM exposes a tunable sensitivity frontier: attack strength depends strongly on perturbation size, future-preserving weight, and query budget. Sixth, the vulnerability is not isolated to one model: attacks transfer across WAM variants, while simple detection baselines miss most attacks at practical false-positive rates.
Together, these results show that WAM safety cannot be reduced to asking whether the imagined future looks plausible. The more security-relevant question is whether the model's imagined future and selected action remain synchronized under adversarial observation perturbations.
Frequently Asked Questions
What is BadWAM? BadWAM is a framework for attacking world-action models by desynchronizing their imagined future predictions from their executed actions, causing task failure even when the predicted future appears normal.
How does the imagination-preserving attack differ from the action-only attack? The imagination-preserving attack adds a regularization term that penalizes drift in the predicted future, making the attack stealthier while still causing substantial task failures.
What types of tasks are most vulnerable to BadWAM attacks? Spatial reasoning and long-horizon manipulation tasks are most vulnerable, while object-centric tasks remain comparatively robust due to their tolerance for moderate action drift.
Can BadWAM attacks transfer between different WAM models? Yes, perturbations optimized on one WAM variant remain damaging on others, indicating shared observation-space directions that affect action generation across models.