To our knowledge, T3R is the first training-free approach to prune 85% of scene-camera tokens before decoder entry while retaining success within 1 percentage point of the unpruned baseline on LIBERO-Spatial. External grounding and attention repair benefit every decoder layer without retraining.
Related Work
Positioning
To our knowledge, T3R is the first training-free approach to prune 85% of scene-camera tokens before decoder entry while retaining success within 1 percentage point of the unpruned baseline on LIBERO-Spatial. External grounding and attention repair benefit every decoder layer without retraining.
Method
T3R follows a two-stage pipeline: segmentation-guided token pruning removes spatially redundant tokens, then saliency-guided attention biasing sharpens attention on the most important tokens that remain.
Saliency-Guided Attention Biasing
Pruning inevitably removes scene information. To mitigate the loss of task-relevant signal, T3R steers attention toward salient instruction tokens and retained task-relevant patches within the reduced sequence.
Attention-bias injection. The attribution is added as a bias to the attention logits of the middle decoder layers.
Experiments
Three hypotheses are tested: (H1) segmentation-guided pruning substantially reduces scene-camera tokens while maintaining success; (H2) fewer tokens lower inference cost; and (H3) the full T3R pipeline improves robustness under out-of-distribution object-placement shifts. The experiments also examine cross-backbone generalization and sensor redundancy.
Most Visual Tokens Are Negligible
For each decoder layer, attention mass is summed from every action-token query to every visual key. Each token’s share is expressed as a percentage of the total, averaged over 32 layers, 10 LIBERO-Spatial tasks, and 3 timesteps per task.
A total of 93.9% of visual tokens receive less than 0.2% of action-query attention each. In late layers (24–31), 50–75% of tokens fall below 0.1%. This shows that most tokens are near-inert, but safe pre-decoder removal still depends on selecting the retained set well.
Generalization and Sensor Redundancy
To test whether T3R’s benefit transfers across backbones and sensor suites, it is ported to CogACT on SIMPLER, which uses a single third-person camera, and to a second model whose name is missing from the provided text on RoboTwin, which uses three cameras. Both evaluations are zero-shot, with no fine-tuned checkpoint, so absolute success is low; the focus is on relative behavior versus the unpruned base and competing pruners at matched budgets.
Single-camera (CogACT / SIMPLER). CogACT is evaluated on the SIMPLER Google-Robot suite (11 tasks, 189 episodes per method) and WidowX/Bridge suite (4 tasks, 96 episodes per method), each using one third-person view. With a 60% keep ratio, T3R matches the base on Bridge and trails by 5.5 percentage points on Google-Robot, while methods retaining more tokens (ADP, FastV) fare better. With one view, little cross-sensor redundancy remains, so aggressive pruning removes tokens the policy still needs.
Takeaway. Across the evaluated backbones and benchmarks, T3R shows greater pruning headroom in multi-camera settings. This cross-system trend suggests that overlapping visual evidence provides opportunities for token reduction.
Ablations
Each component is isolated on LIBERO-Spatial (10 tasks × 10 episodes). Prune-only reaches 90% at 85% scene-camera token reduction, bias-only reaches 98%, and full T3R reaches 96%, versus a 97% baseline. Biasing recovers 6 percentage points of pruning loss; bias-only exceeding the baseline indicates diffuse unpruned attention.
In the bias-only layer sweep, with no pruning, middle-layer biasing (layers 8–23) reaches 98%, versus 95% for early layers, 92% for late layers, and 93% for all layers. This is consistent with early feature distortion and late output disruption.
Why T3R outperforms prior pruners. T3R prunes before all 32 decoder layers, retaining approximately 37 scene tokens, while TeamVLA carries 512 tokens through layers 0–15, allowing irrelevant context into representations before merging. Segmentation preserves coherent object regions and RoPE geometry rather than disjoint similarity samples; IG-guided attention repair then offsets pruning loss.
Hyperparameter robustness. One hyperparameter set served all 10 LIBERO-Spatial tasks and all four suites, without per-task tuning. Sweeping bias strength, active layers, saliency top-k, and pruning threshold changed success rate by 1–6 percentage points. DBSCAN ε and SAM point counts follow the fixed patch-grid geometry.
Failure Analysis and Limitations
All evaluations use simulation; physical-robot control and asynchronous execution with real sensor and actuation delays remain unvalidated. Hardware profiling uses only an NVIDIA L40S, with no energy measurements or edge-device tests. Latency improvements alone therefore do not establish energy savings or edge performance.
Occlusion and lighting changes may degrade segmentation and discard task-relevant patches. These effects have not been isolated for their impact on task success, and alternative segmenters have not been compared.
Frequently Asked Questions
How much does T3R prune? It prunes 85% of scene-camera tokens before decoder entry while retaining success within 1 percentage point of the unpruned LIBERO-Spatial baseline.
What are the two stages of T3R? Segmentation-guided token pruning removes spatially redundant tokens, followed by saliency-guided attention biasing on the remaining tokens.
How does T3R perform with a single camera? At a 60% keep ratio, it matches the base on Bridge and trails by 5.5 percentage points on Google-Robot.
Has T3R been validated on physical robots? No. The evaluations use simulation, and physical-robot control and real sensor and actuation delays remain unvalidated.