For every method, we convert the reference to the same 29-DoF humanoid qpos representation and execute the policy through a common MuJoCo evaluation entry point. Each tracker retains its native policy observations and action decoder, and the evaluator instead standardizes the motion list, robot model, reference indexing, rollout accounting, and metric implementation.
Standardized Tracker Evaluation
The control trajectory is recorded at 50 Hz. At every step, the evaluator stores the simulated generalized position and velocity, policy action and motor target, foot contacts and contact forces, foot and pelvis velocities, and 14 keypoint poses and spatial velocities. The same state history is used for conventional diagnostics and HumanScore, which prevents differences in post-processing from being mistaken for differences between trackers.
We use the same tracking metric and success criterion with SONIC for each tracker. It measures vertical position error at the pelvis, both ankles, and both wrists, together with pelvis rotation error. The episode fails when any vertical error exceeds 0.25 m, the pelvis rotation error exceeds 1 rad, or the generalized position or velocity contains a nonfinite value.
We report Succ as the fraction of completed episodes and MPJPE as the mean absolute error over the 29 actuated joint angles, in radians, over the executed portion of each rollout. Additional diagnostics include joint-velocity error, keypoint-position error, foot-contact agreement, and finite-difference joint acceleration and jerk. All reported benchmark values use the HumanTracker test split.
Preference Data Construction
The preference pool is generated exclusively from motions in the HumanTracker training split, so no benchmark test motion is used to train HumanScore. For each source motion, GMT, Humanoid-GPT, SONIC, and TWIST2 produce aligned rollouts of the same robot-space reference.
At 50 Hz, every rollout is divided into consecutive 250-frame windows, each spanning 5 s. If a rollout ends with a shorter window, the final short window is retained.
HumanScore
The frame vector is linearly projected, normalized, and augmented with sinusoidal positional encoding before being processed by a Transformer encoder. A padding mask is applied in every attention layer. The elementwise mean of the valid output tokens then forms the trajectory representation, which an MLP maps to the scalar reward.
The same mask is used for the retained tail clips. Segments shorter than 250 frames are padded with zeros on the right, while the validity mask excludes padded positions from both attention and temporal pooling. Thus, full and truncated windows can be processed by the same model without introducing padding artifacts.
HumanScore Computation
We then average the window rewards according to the number of frames that they represent.
Human Preference Comparison
HumanScore agrees with human preferences more consistently than any individual analytic diagnostic. Conventional metrics isolate pose, velocity, or contact fidelity, whereas annotators also consider human-like motion, smoothness, stability, and how errors develop over time. Their lower agreement indicates that no single diagnostic captures the full trajectory quality reflected in human judgments.
The comparison also rules out several distributional shortcuts. Grouping the test set by source motion prevents adjacent clips from the same sequence from crossing partitions. Averaging agreement within each family prevents the more frequent Daily and Interaction samples from obscuring performance on Ground and Highly Dynamic. Tracker pairings are balanced by construction, so the result cannot be explained by a dominant family or tracker matchup.
Sensitivity Analysis
Removing measured contact features degrades performance most clearly on Ground. This indicates that contact information is important for motions with complex contact transitions. Adding future reference information performs slightly worse than the baseline, suggesting that this extra signal offers limited benefit and is difficult to exploit. It is sufficient to rely on the current and historical states to evaluate the quality of the action.
Longer context improves alignment by revealing sliding, jitter, drift, and recovery that isolated poses cannot capture. Alignment also improves steadily as the available context grows from one to five seconds.
Short segments capture instantaneous pose errors, whereas longer context reveals evolving artifacts such as foot sliding, repeated jitter, progressive drift, and recovery from instability. HumanScore therefore benefits from integrating complementary evidence over time, rather than evaluating motion frame by frame.
Conclusion
We introduce a large-scale benchmark and HumanScore, a human-aligned metric, for evaluating humanoid motion tracking. Built from optical motion recorded from 24 professional performers, the released benchmark contains approximately 153 hours across four motion families. It reveals persistent weaknesses in highly dynamic and ground-contact motions.
HumanTracker provides a reproducible framework for tracker comparison and failure analysis, with future work extending to cross-embodiment settings, real-world hardware, and reward optimization.
Segment Construction and Padding
The formula describes how valid frames are used to construct and weight trajectory segments.
Short terminal segments can therefore contribute without padded frames affecting attention or pooling. When window rewards are combined across a rollout, each window is weighted by its number of actual frames. Padded positions contribute neither to the window representation nor to the trajectory average.
Frequently Asked Questions
What is HumanTracker? HumanTracker is a large-scale benchmark and framework for evaluating humanoid motion tracking with human-aligned quality assessment.
What does HumanScore measure? HumanScore evaluates trajectory quality using temporally aggregated motion information intended to align with human preferences.
How are tracker rollouts evaluated consistently? References are converted to a shared 29-DoF humanoid representation and evaluated through a common MuJoCo entry point with standardized metrics and state recording.
Why does longer context improve HumanScore alignment? Longer context reveals evolving artifacts such as foot sliding, jitter, drift, and recovery from instability.