We introduce the Warp-Augmented Relative Progress Reward Model (WARP-RM), a vision-based model that estimates the dense, per-frame progress velocity of a task directly from visual observations. To train WARP-RM without human annotations, we propose the WARP algorithm, a fully self-supervised method that trains the model to predict the normalized time delta for each frame relative to a starting frame, using a dataset of successful human teleoperated demonstrations. Time-warping video playback speed—and its corresponding progress label—provides a strong self-supervised signal: subsampling frames at wider intervals simulates faster execution, while sampling at narrower intervals simulates slower execution. By varying this time-warping augmentation non-uniformly, a single demonstration provides dense supervision over a continuous range of progress velocities.
Overview and Notation
The Warp-Augmented Relative Progress Reward Model (WARP-RM) is a vision-based model that estimates dense, per-frame progress velocity of a task directly from visual observations. To train WARP-RM without human annotations, the WARP algorithm uses a fully self-supervised method that trains the model to predict the normalized time delta for each frame relative to a starting frame, using a dataset of successful human teleoperated demonstrations.
Progress Model Training
Relative cumulative progress targets are computed from sampled indices. The per-frame cumulative progress pseudo-labels are calculated as the signed displacement from the window start normalized by a constant T. This normalization creates synthetic progress targets across a range of -T to T, enabling the model to learn the relative pacing of task execution.
Limitations and Future Work
This research was performed at the AUTOLAB at UC Berkeley in affiliation with the Berkeley AI Research (BAIR) Lab. XDOF provided the teleoperated demonstration data, training compute, and evaluation hardware infrastructure used for this project. In their academic roles at UC Berkeley, Justin Yu, Andrew Goldberg, Kavish Kondap, Karim El-Refai, Ethan Ransing, and Ken Goldberg are supported in part by donations from Toyota Research Institute, Autodesk, Meta, Google, Siemens, and Bosch, and by equipment grants from NVIDIA, PhotoNeo, the NSF AI4OPT Center, and Intuitive Surgical. Justin Yu is supported by the National Science Foundation Graduate Research Fellowship Program under Grant No. DGE 2146752. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
Frequently Asked Questions
What is WARP-RM and what does it estimate? WARP-RM is a vision-based model that estimates the dense, per-frame progress velocity of a robot task directly from visual observations, without requiring human annotations.
How is WARP-RM trained without human labels? The WARP algorithm uses a self-supervised method that trains the model to predict normalized time deltas for each frame relative to a starting frame, using only successful human teleoperated demonstrations.
What is the time-warping augmentation technique? Time-warping varies video playback speed by subsampling frames at different intervals—wider intervals simulate faster execution, narrower intervals simulate slower execution—providing dense supervision over a continuous range of progress velocities.
What data and infrastructure were used for this research? The research used teleoperated demonstration data provided by XDOF, with training compute and evaluation hardware from XDOF, and was conducted at UC Berkeley's AUTOLAB in affiliation with BAIR.