Chronos Policy makes observation and action timing explicit in the policy. Each observation is associated with when it was captured, and each predicted action with when it should execute. The policy uses a diffusion transformer with time-aware RoPE, which introduces time through attention while leaving token embeddings unchanged. During training, observation and action timing are also varied while their timestamps are preserved, so the policy learns to interpret timing changes it may encounter at deployment.
Method
Policy Formulation on Physical Time
Let observations be captured at their respective times. At a given request time, the policy queries an action chunk of a specified number of steps, beginning after a delay and continuing at a specified action interval.
This formulation makes timing information explicit. Observation spacing captures the observation rate; the gap between observation and action times captures measurement delay and execution latency; and the action interval specifies the desired execution frequency. For multimodal sensing, each modality keeps its own capture times rather than being aligned to a common sampling grid.
Encoding Physical Time in the Policy
Given explicit timestamps in the policy inputs, temporal information can enter the network in different ways. One option is to encode each timestamp and add it directly to the corresponding token embedding, allowing time to modify the token representation. Another is to keep the token embedding unchanged and use time only through the positional encoding in attention. The latter is consistently more robust to temporal variation, so Chronos Policy uses time-aware RoPE.
Every token is assigned a physical timestamp relative to the policy request time. Image patches inherit the capture time of their frame; proprioceptive and force/torque tokens use their measurement times; and action tokens use their queried execution times. Observation timestamps describe when measurements were captured relative to the request, while action timestamps lie in the future according to the requested execution schedule. Modality and spatial embeddings continue to represent token content, while physical time enters separately through rotary positional encoding.
All timestamps are expressed relative to the policy request time. For action tokens, this means inference latency shifts the queried action times into the future. The rotary coordinate is the timestamp relative to that request time, and the resulting query–key interaction depends only on elapsed physical time.
Thus, attention depends directly on elapsed physical time. Two observations separated by 20 ms and 200 ms receive different temporal relationships even if they occupy the same sequence positions. At the same time, shifting all timestamps by the same amount leaves the representation unchanged. Compared with adding time directly to token embeddings, time-aware RoPE keeps token content and timing separate while making pairwise temporal relationships explicit in attention.
Experiments
The experiments address three questions: when time information is useful, which parts of the time-aware policy matter, and whether adding time hurts tasks where timing is less important. The evaluation covers dynamic tasks, where actions depend on how the environment changes over time; multisensory tasks, where timing between observations matters; and standard quasi-static manipulation tasks, used to test whether Chronos Policy preserves nominal performance while improving robustness to timing changes at deployment.
Tasks and Evaluation Setup
The evaluation covers three types of tasks. Conveyor is a dynamic pick-and-place task with randomized belt speed, where the policy must account for object motion and the time between sensing and action execution. Pivoting is a contact-rich multisensory task combining vision, proprioception, and force/torque measurements acquired at different rates and times. Six standard quasi-static manipulation tasks test whether explicit time modeling preserves performance when timing is less critical.
Each policy is evaluated under different timing conditions. Nominal uses the timing seen during training. Observation timing is varied by changing sensing frequency and delay, and inference latency is varied by changing the delay before predicted actions are executed. The combined setting varies both observation timing and inference latency, providing a stronger test of robustness to temporal mismatch.
When Is Time Information Useful?
On Conveyor, Chronos Policy improves nominal success from 73.1% to 82.0%. The gap grows when deployment timing changes: with moderate observation variation, Diffusion Policy drops to 15.6% while Chronos Policy retains 77.6%; under inference delays up to 200 ms, Diffusion Policy falls to 1.8% while Chronos Policy achieves 76.4%. Even under the combined perturbation, Chronos Policy retains 32.7% success while Diffusion Policy fails entirely.
Sequence order alone is not enough to determine object motion when observation intervals vary, or to determine how far ahead to act when execution is delayed. Providing both observation and action times allows the policy to interpret these changes directly.
What Makes a Policy Time-Aware?
The experiments ablate two design choices in Chronos Policy: temporal variation during training and the representation of physical time in the transformer.
Temporal randomization exposes the policy to a wider range of sensing and execution schedules that may occur during deployment. It also allows training to use measurements that would otherwise be discarded when demonstrations are downsampled to a fixed temporal grid. This additional diversity can act as data augmentation. On Conveyor, temporal randomization alone increases Diffusion Policy’s average success across the five perturbed conditions from 6.3% to 26.9%. Even without timestamps, exposing the policy to more temporal variation can improve robustness when the relevant timing structure is relatively simple.
The same strategy does not generalize to multisensory Pivoting. There, temporal randomization reduces the perturbed-condition average from 48.4% to 21.6%. Without timestamps, the policy cannot tell what timing produced each sampled history: the same sequence positions may correspond to different physical intervals, and asynchronous measurements from different modalities become difficult to interpret.
Once physical time is explicit, temporal randomization becomes consistently useful. It increases the perturbed-condition average from 48.8% to 63.1% on Conveyor and from 33.3% to 55.3% on Pivoting. Temporal randomization broadens the training distribution, while timestamps make that variation interpretable.
Does Time Hurt Standard Quasi-Static Manipulation?
The six standard quasi-static manipulation tasks test settings where precise timing is less critical. Under nominal timing, Chronos Policy and Diffusion Policy achieve comparable performance, showing that explicitly modeling time does not sacrifice performance.
The advantage becomes clear when deployment timing differs from training. Averaged across the five perturbed conditions, success improves from 59.7% with Diffusion Policy to 73.4% with Chronos Policy, with the largest gains under wide sensing variation and combined perturbations.
Overall, Chronos Policy preserves nominal performance while providing substantially greater robustness to temporal mismatch. When timing is close to the training setup, explicit time does not sacrifice performance; when sensing or inference timing changes, it gives the policy information about observation and action timing that is absent from the baseline.
Does Time Awareness Improve Real-World Robustness?
Chronos Policy achieves the highest task score in every evaluated condition across all three real-world tasks. The advantage persists under observation perturbations, inference latency, and their combination, showing that the benefit of explicit time extends beyond controlled simulation. The gains span both moving-target and contact-rich manipulation, suggesting that the same physical-time interface can handle different sources of temporal variation.
RTC improves robustness to inference latency in several settings. However, it does not explicitly account for changes in observation timing, so its benefit does not consistently extend to sensing perturbations. Chronos Policy represents both when each observation was captured and when each predicted action should execute, allowing sensing and execution timing to be modeled within the same policy architecture. These results suggest that asynchronous execution and explicit time awareness address complementary aspects of temporal mismatch.
Conclusion and Future Work
Chronos Policy grounds robot policy learning in physical time by conditioning on when observations are captured and when actions should execute. This formulation provides a unified representation of sensing rate, measurement delay, and inference latency within a single policy. Across dynamic, multisensory, and standard quasi-static manipulation tasks in simulation and on real robots, Chronos Policy improves robustness to temporal mismatch while preserving nominal performance.
A direction for future work is to extend Chronos policies beyond fixed action rates to variable control frequencies and temporal scaling, allowing robots to execute the same behavior faster or slower than demonstration.
Acknowledgment
The authors thank all members of REALab at Stanford for helpful discussions and feedback. This work was supported in part by the Knight-Hennessy Fellowship and the NSF Graduate Research Fellowships Program (GRFP), NSF Awards #2143601, #2037101, and #2132519, Samsung, and Stanford HAI Center. The views and conclusions are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of the sponsors.
Appendix: Policy Architecture and Evaluation Details
Policy Architecture and Input Representations
Conveyor and the six standard tasks use two image observations per camera and two proprioceptive observations. Simulated Pivoting uses two images, three pose observations, and 32 six-dimensional force/torque samples. A pose is represented by its position and 6D rotation.
Pivoting predicts a 19-dimensional adaptive-compliance action containing a nominal pose, a virtual-target pose, and stiffness; the same controller and action representation are used for all methods. Other simulation tasks use absolute pose and gripper or hand commands. Every policy predicts 16 actions at 20 Hz.
Baselines and Time-Aware Variants
The transformer-based baseline uses sequence position without observation or action timestamps. Its temporal-randomization variant uses the same randomized observation schedules and action-start delays as Chronos Policy, but does not receive their physical times.
Checkpoint Selection and Evaluation Setup
Three seeds are trained for each policy. Checkpoints are evaluated on the nominal validation setting every five epochs, and the checkpoint with the highest nominal success rate is selected. For the DexMimicGen benchmark, 50 episodes are evaluated per training seed, following the original paper’s evaluation setup.
For the Conveyor and Pivoting ablations, each selected training checkpoint is evaluated with three evaluation seeds, with 50 episodes per seed and condition. Results are first averaged across the three evaluation seeds for each training seed, then reported as the mean and sample standard deviation across the three training seeds, for a total of 450 episodes per method and condition.
Temporal Settings
All tasks sample observation rates of 12.5–33.33 Hz and delivery delays of 0–50 ms independently across sensor streams, using a 100 Hz original dataset. For Pivoting, force/torque is sampled from 1–100 Hz with 0–100 ms delivery delay. Half of the action chunks start with zero delay; the rest sample a start delay up to 100 ms.
Dynamic and Multisensory Tasks: Full Ablations
On Conveyor, adding temporal randomization to Diffusion Policy substantially improves several perturbation settings. Success increases from 15.6% to 60.9% under small observation variation and from 14.2% to 47.3% under delays up to 100 ms. This suggests that exposing the policy to a broader range of observation and execution schedules can itself act as useful data augmentation.
The benefit does not extend cleanly to more severe shifts: under wide sensing, success improves only from 0.0% to 6.4%, and both variants remain near zero under the combined condition. Randomization also lowers nominal Conveyor performance from 73.1% to 53.6%, indicating a trade-off when temporal variation is introduced without telling the policy what timing generated each sample.
This limitation is more pronounced on Pivoting. Adding temporal randomization to Diffusion Policy reduces performance in every condition, including nominal success from 61.6% to 29.8%. Unlike Conveyor, Pivoting requires relating vision, proprioception, and force/torque measurements sampled at different times. When schedules are randomized but their timestamps are hidden, the same token positions can correspond to different cross-modal time offsets. The additional diversity therefore also introduces ambiguity that the policy cannot resolve.
Real-World Task Setup and Demonstrations
The real-world evaluation covers Dynamic Cup, Peg Insertion, and Pivoting. Dynamic Cup is evaluated across different saucer speeds, positions, and rotation directions. Peg Insertion is evaluated with five trials for each of two hole orientations, forward and backward. Pivoting is evaluated with five trials in each of four settings: a large eraser, a small eraser, a box, and a sideways configuration.
All tasks use two RGB observations together with histories of relative pose and measured gripper width at a nominal rate of 20 Hz. The provided paper text ends mid-sentence after stating that Peg Insertion and Pivoting additionally use “3”.
Frequently Asked Questions
What does Chronos Policy add to robot policy inputs? It associates observations with their capture times and predicted actions with their intended execution times.
How does Chronos Policy represent physical time? It uses time-aware RoPE in attention, keeping token embeddings unchanged while making elapsed physical time explicit.
Why are timestamps useful when observations are randomized in time? Timestamps let the policy interpret varied observation schedules and asynchronous measurements rather than treating sequence positions as fixed intervals.
Does explicit time modeling reduce nominal task performance? The reported quasi-static manipulation results show comparable nominal performance to Diffusion Policy, while Chronos Policy is more robust when deployment timing changes.