Humanoid Skills Scale with 50,000 Hours of Human Motion and Video

Humanoid Skills Scale with 50,000 Hours of Human Motion and Video

EngineAI

8 min read•Nov 22, 2026

Awaken-HI trains a humanoid robot on human videos and full-body motion, then adapts that experience with 150 hours of robot teleoperation. On an EngineAI T800, the resulting policy averaged 93.75% success across eight whole-body tasks, while larger human-data sets also improved household-task performance. That offers a practical route to teaching locomotion and manipulation together rather than collecting robot demonstrations for every movement.

What did the researchers build?

EngineAI’s Awaken-HI is a training system for humanoid robots that starts with demonstrations from people, then adapts what it learns to a specific robot. Its foundation is Awaken-Human, a dataset pairing 50,000 hours of first-person video with synchronized recordings of human body movement. The corpus contains about 17.02 million episodes, 5.41 billion pose frames, and 11.18 million language instructions.

People recorded demonstrations wearing a PICO headset and five body trackers. That setup captured stereo video—a view from two cameras that helps estimate distance—along with body motion. More than 100 skills appear across homes, workplaces, shops, and storage areas, including carrying objects, placing items on shelves, folding clothes, and wiping surfaces. The continuous recordings preserve how people walk, turn, crouch, and reposition while using their hands.

The team also added labels to make the recordings useful for training: descriptions of task segments, hand poses reconstructed from video, and estimated scene depth. Human reviewers checked the initial task labels, and low-quality hand observations were filtered out. This cleaned and aligned data lets the robot learn not just what a person did, but how movement, objects, and instructions fit together.

Examples of human demonstrations and objects in the Awaken-Human dataset

What results did Awaken-HI achieve?

The strongest reported result is a mean success rate of 93.75% across eight individual whole-body tasks after robot-specific training. Those tasks included walking to objects, picking them up, placing them on shelves, hanging a garment, and moving a laundry basket. The evaluations used ten trials per task, with shared starting configurations across methods.

More human pretraining data also improved performance on three household tasks. Increasing the human demonstration set from 5,000 to 50,000 hours raised mean success from 56.7% to 90.0%. In tests that changed which objects belonged in which destinations, Awaken-HI succeeded in all trials for two industrial instruction changes and in 80% of trials for two household changes. The paper reports better results than both tested vision-language-action baselines on all four tests.

Awaken-HI also handled two objects absent from robot post-training demonstrations: tissues and a medicine bottle. Its object-level completion rates were 50% and 30%, respectively, while both baselines recorded 0% for each object. These tests indicate better transfer, but the object results also show that unfamiliar items remain challenging.

EvaluationAwaken-HI result
Eight individual whole-body tasks93.75% mean success
Household tasks after 5,000 hours of human pretraining56.7% mean success
Household tasks after 50,000 hours of human pretraining90.0% mean success
Two industrial instruction-combination tests100% success on each
Two household instruction-combination tests80% success on each
Unseen tissues / medicine bottle50% / 30% object-level completion
Baselines on unseen tissues / medicine bottle0% / 0%
Humanoid task examples used to test whole-body walking and manipulation

How does Awaken-HI turn human experience into robot actions?

A key design choice is that Awaken-HI learns human movement and robot movement together. Human bodies and robot bodies have different proportions, so copying a person’s pose directly would not produce a workable robot action. The system therefore retargets human demonstrations into the action space of the EngineAI T800 while retaining the original human motion as a second training target.

The model combines a vision-language backbone with a shared action expert. The vision-language component processes camera images and instructions; the action expert predicts movement over a future time horizon. During pretraining, separate input and output branches represent T800 actions, original human motion, and wrist poses. The action expert is shared, so all three kinds of motion supervision shape the same central model. Additional depth prediction helps the system reason about distances and object locations, while language prediction identifies the current subtask.

Before use on the robot, the pretrained policy is fine-tuned on 150 hours of T800 teleoperation data. That data pairs robot observations and measured states with actions demonstrated by a remote operator. Fine-tuning grounds the learned human movement patterns in the robot’s own body and controls.

At deployment, the policy predicts an action sequence from current observations, passes the targets to the T800’s low-level controller, and repeats the process as new observations arrive. This repeated update is called receding-horizon control: the robot plans ahead, executes, then refreshes its plan instead of committing to one long sequence.

Awaken-HI training pipeline connecting human motion, robot actions, and visual inputs

Why does whole-body pretraining matter for robotics?

Many useful humanoid tasks are not just hand movements. A robot must approach an object, orient its body, reach or crouch, manipulate the object, and sometimes move away while carrying it. Training on continuous human demonstrations gives Awaken-HI examples of these coordinated transitions, rather than isolated snapshots of grasping or walking.

The reported scaling result is especially relevant to teams weighing the cost of collecting robot data. Awaken-HI’s pretraining uses human demonstrations at scale, then relies on a smaller robot-specific dataset to adapt the behavior. The improvement from 56.7% to 90.0% across the smallest and largest reported human-data sets suggests that human experience can contribute beyond an initial warm start.

The generalization tests are also useful operational signals. The robot adjusted placements when familiar objects were assigned to new destinations, and it completed some tasks with unfamiliar objects that defeated the baselines. That kind of flexibility matters in changing workspaces, although the results do not establish reliable performance across arbitrary objects or environments.

Teams evaluating platforms can browse humanoid robots on Robot Overflow. For warehouse and material-handling use cases, warehouse robots provide a related category to compare, even though Awaken-HI was tested on a humanoid platform.

What are the limitations and open questions?

The evaluation covers eight individual tasks and two long-horizon task sequences on one humanoid platform, the EngineAI T800. Each reported trial set is small, with ten trials per task or instruction test, so the results do not show how performance varies across larger populations of objects, environments, or operating conditions. The supplied results also do not give exact baseline success rates for every task, making some comparisons hard to quantify.

The unfamiliar-object results are promising but modest: completion reached 50% for tissues and 30% for a medicine bottle. The approach also depends on substantial data infrastructure, including wearable trackers, stereo video, annotation, and 150 hours of robot teleoperation. Further testing is needed to establish how well the learned skills transfer to other humanoids, withstand unexpected changes, and operate safely over long deployments.

What should teams know about Awaken-HI?

Does Awaken-HI train only on robot demonstrations? No. It first learns from human video and motion, then fine-tunes on 150 hours of teleoperation data from the T800.

What kinds of tasks did the robot perform? The tests included walking to objects, picking and placing, shelf stocking, garment hanging, and carrying a laundry basket.

Did more human data improve performance? Yes. On three household tasks, the reported mean success rate rose from 56.7% with 5,000 hours of human demonstrations to 90.0% with 50,000 hours.

Can the system handle objects it has never seen? It completed some trials with two objects absent from robot post-training, while both baselines scored zero on those objects. Its completion rates were still limited.

What is the takeaway?

Awaken-HI shows how large-scale human motion data can help a humanoid coordinate walking and manipulation, then adapt those skills through robot teleoperation. Its results are strong on the tested tasks, while broader transfer and reliable handling of unfamiliar objects remain open challenges.