Our contributions are threefold:
- A training collection of 3.7 million question–answer pairs and a data construction pipeline that derives execution-related supervision from human videos, robot episodes, and simulation, with egocentric video serving as a major data source.
- ACE-Brain-0.7, a 9B VLM that jointly supports spatial grounding, task understanding, and progress estimation, achieving leading overall performance across the corresponding evaluations.
- An empirical analysis of planning and execution judgments through capability-level comparisons, controlled visual inputs, temporal question pairs, and an ablation of egocentric supervision.
Data and model

Evaluation setup
Models
We compare ACE-Brain-0.7 with GPT-6 Astra and Claude Opus 5.5; Qwen3.5 at 9B (our base) and 27B parameters; and twelve open embodied models of up to 32B parameters.
Conclusion
Describing a task by its structure and an execution by its state turns five public benchmarks into one profile of seventeen sub-items and separates what a model takes from the task prior from what it reads from the execution: plans survive the removal of the images; state answers do not. Supervised fine-tuning on 3.7 million annotation-derived question–answer pairs makes answers follow the execution shown.
ACE-Brain-0.7, a 9B model obtained from Qwen3.5-9B by such fine-tuning on these pairs together with spatial-grounding and progress-estimation data, leads both closed models on the next sub-task of a robot episode, has the highest six-benchmark average of all compared models, and has the highest average of all compared models on seven public spatial-grounding benchmarks.
Contributors and affiliations
Contributors: Yinhe Wang, Shijun Shi, Feng Pan, Tao Tao, Wei Mao, Jiangning Zhang, Zhiyao Ren, Yihang Jiao, Haoming Gu, Yunpeng Qing, and Jiyu Guo.
Advisors: Zhi Hou, Dacheng Tao, and Xiaogang Wang.
Affiliations: ACE Robotics; Nanyang Technological University; Zhejiang University.
Evaluation protocol notes
Decoding and inputs
All open models, including ACE-Brain-0.7 and the base model, decode greedily; on Cosmos-Reason1 the generation budget is 4,096 tokens with a repetition penalty of 1.05. Cosmos-Reason1 and RoboBench images are undistorted for every model.
RoboBench is used in a natural-language version for every model: in Q1 and Q2 the skill library and worked examples are removed from the prompt, the model answers with numbered steps in plain language instead of skill-function calls, and the reference plans are converted from function calls to sentences by fixed templates; Q3 is unchanged. Navigation questions and 28 questions whose converted references could not be verified are excluded, leaving 3,979 items, and the reported score is the mean over these items.
Scoring
Multiple-choice and yes/no answers are extracted and scored by each benchmark’s native script. On EgoPlan-Bench the open models are scored from the option letter at the end of the response and the closed models with the extractor of the public evaluation toolkit.
A language-model judge (Qwen3.8-Flash) scores RoboBench Q1 and Q2 and RoboFAC task identification against the reference, with the same prompt for every model; RoboBench Q3 is scored as yes/no. LADDER is the mean of decomposition exact match, verification balanced accuracy, state accuracy and next-sub-goal validity; its state component counts a scene as correct only if every state query about it is correct. RoboFAC is the macro average of its four question types. The progress MAE counts an unparsable answer as an error of 1. Sub-item scores of the closed models are computed from their per-item predictions.
Frequently Asked Questions
What is ACE-Brain-0.7? It is a 9B VLM fine-tuned from Qwen3.5-9B to support spatial grounding, task understanding, and progress estimation.
How was its training collection constructed? The 3.7 million question–answer pairs derive execution-related supervision from human videos, robot episodes, and simulation, with egocentric video as a major source.
What does the evaluation profile measure? It combines five public benchmarks into seventeen sub-items, separating task-structure judgments from judgments based on execution state.
How were RoboBench questions adapted for evaluation? For Q1 and Q2, prompts omit the skill library and worked examples, and models answer in numbered plain-language steps; Q3 is unchanged.
