Embodied Robots for Industrial Manipulation
Despite differences in task objectives, these applications share a common execution pattern: a robot operates at a fixed workstation and repeatedly performs similar tasks, with variations mainly in the target objects and their locations. Such settings also impose stringent requirements on the responsiveness and reliability of the policy.
Dual-Phase Inference in VLAs
We focus on VLA models that couple a vision-language model (VLM) with a diffusion action expert, an architecture adopted by the majority of recent VLA models.
The vision transformer (ViT) embedding stage converts the observations and language instruction into input embeddings. The visual observations typically consist of images captured by the robot’s cameras. Inference then proceeds through two core phases that account for most of the latency.
First, the VLM performs a prefill pass over the ViT embeddings to compute visual features for action generation. Second, the action expert starts from a noisy action sample and iteratively refines it through multiple denoising steps to produce an action chunk.
Denoising-Step Skipping and Its Limits in VLAs
However, these methods do not transfer well to embodied model inference, because both forms of reuse operate at the granularity of denoising steps. Whole-step reuse relies on consecutive steps producing nearly identical outputs, as is often the case for text-to-image and text-to-video models that typically run 20–50 steps.
VLA models use far fewer steps. For example, GR00T uses only four steps. Each step therefore makes a substantial change to the action chunk, so skipping any step discards refinement that subsequent steps do not recover.
Characterizing Embodied Workloads
To identify opportunities for accelerating embodied inference, we characterize VLA workloads from both the execution and model perspectives. We first locate the performance bottlenecks in VLM prefill and action denoising, then examine observation and trajectory similarity across executions. We next examine how this similarity extends to internal model states, specifically in VLM states and neuron contributions during denoising.
Trajectory Similarity Across Repeated Tasks
In industrial assembly-line settings, robots repeatedly perform the same task using a limited set of objects in a relatively stable workspace. However, variations in lighting, tabletop texture, and object pose can change both observations and required motions. We therefore examine whether action trajectories remain similar across executions despite these variations.
An action trajectory is the VLA’s observable output: a temporally ordered sequence of robot actions that traces a continuous path in 3D space. To test whether useful repetition survives environmental variation, we model these factors in simulation. In the representative task, a dual-arm robot lifts two beverage bottles and moves them to designated positions above the tabletop. We vary the initial bottle positions and lighting conditions across executions, then compare a current execution with a reference execution of the same task.
Both the current and reference runs finish in four inference steps and reach their target positions successfully and stably. Although the initial bottle positions and lighting differ, the end-effector trajectories and the camera views of both arms remain closely aligned, with path cosine similarities of 0.995 for the left arm and 0.994 for the right arm.
VLM-State Similarity and Visual-Token Coverage
Comparing MLP inputs and outputs from the current and reference executions at every VLM layer, we find cosine similarities as high as 98.5% for inputs and 81.0% for outputs. This similarity motivates examining how the remaining output changes are distributed across visual tokens. We further investigate whether MLP input differences can identify the tokens that account for most of these changes.
The selected regions for the bottle-moving task show that important patches concentrate around the bottles, grippers, and arms, with a smaller number on background regions that encode scene context. Their locations shift across action chunks as manipulation progresses, motivating selection for each policy call rather than a fixed spatial mask.
Denoising-Neuron Contribution and Overlap
The similar trajectories of current and reference executions motivate examining whether their action-denoising processes also exhibit similar internal activations. Following our analysis of VLM states, we first compare the inputs and outputs of action denoising. Although the initial noise is identical under the same random seed, the generated action chunks exhibit only 69% similarity.
This difference motivates exploring finer-grained reuse within denoising instead of directly reusing reference actions. We therefore examine how neuron contributions are distributed within each denoising FFN and whether these contribution patterns are similar between the current and reference executions.
For each denoising step t and layer l, we quantify each neuron’s contribution to the FFN output using a contribution score.
The contribution score measures how strongly each neuron influences the FFN output for a given denoising step and layer.
Retaining a selected fraction of neurons captures 84.4% of the total contribution score.
Design
rMuscle combines two complementary mechanisms: dual-phase caching for selective recomputation and cache management for low-overhead access and storage.
The dual-phase cache uses a Context Cache to accelerate compute-bound VLM prefill and an Action Cache for memory-bound action denoising. The Context Cache reuses FFN outputs for similar visual tokens and recomputes those whose inputs have changed substantially. During action denoising, similar executions tend to activate overlapping sets of important neurons. The Action Cache uses these reference activation patterns to select which neurons to evaluate and which weights to load.
Cache management supplies the dual-phase cache with reference FFN inputs and outputs and neuron masks from a shared library. To reduce memory overhead, the library retains compact reference inputs and neuron masks, and expanded FFN states are only reconstructed when a reference is likely to be retrieved. References are selected using visual context and robot-state history. A sliding GPU working set is further used to bound memory residency, combined with a reconstruction budget that limits reconstruction work.
Cache Management
Cache management supplies the dual-phase cache with relevant reference data while bounding GPU residency and reconstruction work. It combines asynchronous online reconstruction, context-aware retrieval, and sliding-window prefetching guided by temporal locality.
Expanded VLM states are too large to store for every reference across all tasks. For three-view inputs, each of the 17 FFN layers retains a pair of BF16 input and output tensors, approximately 107 MB per policy call. An episode of 40 policy calls therefore requires about 4.28 GB of intermediate states. Storing only compact inputs avoids this expansion, but reconstructing the states synchronously would put dense VLM computation back on the inference critical path.
rMuscle instead reconstructs references during physical action execution. The repository retains reference model inputs and offline neuron masks. During an execution window, the GPU fetches inputs for upcoming candidate positions and runs VLM prefill densely to prepare their FFN states. Since neuron masks are computed offline, reconstruction does not rerun action denoising.
Only the active working set holds expanded FFN states on the GPU. This organization reduces persistent host and disk storage, but does not reduce the GPU footprint of an individual reconstructed entry. The candidate positions are determined by the prefetching policy described below.
Implementation
This section describes how rMuscle builds the cache offline and prepares and retrieves cached references to accelerate online inference.
Evaluation
We evaluate rMuscle to answer three major questions:
- How much speedup does it provide, and how much does each optimization contribute?
- Does it preserve task success rates across diverse tasks, and how do cache configurations affect this?
- What additional memory overhead does it incur?
Inference Performance
Compared with denoising step-skipping methods, rMuscle also achieves inference speedup by accelerating both VLM prefill and action denoising. Although step-skipping methods can achieve greater speedup in the denoising stage by aggressively skipping denoising steps, this comes at the cost of reduced task success rates.
Memory Overhead
We quantify the CPU cache footprint of rMuscle and compare its GPU memory use with vanilla execution. To represent increasing reference requirements for more difficult tasks, we increase the per-task cache capacity from 200 entries across 8 reference episodes to 800 entries across 32 reference episodes, while expanding the candidate window from a smaller initial window to a larger final window.
On RTX 4090, the CPU cache accounts for 3.7–14.9% of 32 GB host memory. GPU memory increases from 8.4 GB for vanilla to 9.4–11.4 GB, raising occupancy by less than 13%.
On Jetson Thor, where CPU and GPU share 128 GB unified memory, occupancy increases from 6.6% to 8.3–12.6%. Overall, rMuscle supports up to 800 cached inference entries with less than 3 GB additional GPU memory on RTX 4090 and at most 6.0 percentage points additional unified-memory occupancy on Jetson Thor.
Ablation Studies
We evaluate cache contributions and quality–latency tradeoffs using the evaluated VLA model configuration on RoboTwin.
Cache Contributions
We compare FlashRT, Context Cache alone, Action Cache alone, and the full system to isolate the latency contribution of each cache on RTX 4090 and Jetson Thor.
Reference Repository
The rMuscle without Cache Repository variant uses the previous policy call within the same episode as its reference instead of retrieving one from the repository. The default configuration achieves a success rate of 35.2%, compared with 25.9% without the reference repository, at similar latency.
These results highlight the value of references from prior executions in preserving policy quality, while the similar latency suggests that cache management adds negligible overhead to the critical path.
Quality–Performance Tradeoff
We vary the visual-token recomputation ratio and the neuron fraction of rMuscle. The Fast, Default, and Slow configurations are compared on RTX 4090 alongside Vanilla, DP-Cache, NIRVANA, and the repository ablation discussed above.
Default increases success from 23.0% to 35.2% compared with Fast, at an additional latency of 4.9 ms. Default matches Vanilla’s success rate, while Slow achieves a slightly higher success rate. They reduce latency relative to Vanilla by 10.5 ms and 4.5 ms, respectively.
These results meet our design goal of preserving policy quality while reducing inference latency. Consistent with the profiling results, Default offers the lowest latency without sacrificing success.
Related Work
VLA-Perf analytically explores how architecture, hardware placement, and serving choices affect inference latency, while a cross-XPU study measures energy and latency across heterogeneous accelerators and validates DP-Cache on LIBERO and a physical Franka robot.
Discussion
Policy latency is the gap between the last action of one chunk and the first action of the next, while actions within a chunk are executed at 20–30 Hz. As policy-call frequency approaches the execution rate, chunk boundaries become smoother, and the robot can re-observe and react to environmental changes more frequently.
Lower inference latency therefore improves responsiveness, motion smoothness, and task throughput. Since each episode requires 4–40 policy calls, reducing per-call latency shortens the overall task cycle and enables a fixed-workstation robot to complete more tasks per hour on the same hardware.
Frequently Asked Questions
What is rMuscle? rMuscle is a caching system for accelerating vision-language-action model inference in repeated embodied-robot tasks.
What are the two main components of rMuscle? It uses a Context Cache for VLM prefill and an Action Cache for action denoising.
How does rMuscle preserve task quality? It selectively recomputes changed visual tokens and important denoising neurons while retrieving references from prior executions.
What memory overhead does rMuscle introduce? On RTX 4090, it uses less than 3 GB of additional GPU memory and 3.7–14.9% of 32 GB host memory for the evaluated cache capacities.