AI Generates Articulated 3D Models at Scale Using Code Generation

AI Generates Articulated 3D Models at Scale Using Code Generation

Matt Zhou, Ruining Li, Xiaoyang Lyu, Zhaomou Song, Zhening Huang +4 more

7 min readMay 16, 2026

Articulated objects, such as cabinets, pliers, desk lamps, strollers, folding chairs, and industrial tools, are ubiquitous in the real world and thus important to applications in 3D content creation, gaming, physical design, and robotics. Because their function depends on the motion of their parts as much as on their geometry, modelling shape alone is insufficient: a useful representation must also capture part hierarchy, joints, and range of motion. We introduce Articraft, an agent that builds articulated 3D objects at scale by generating code. Articraft is designed around three principles: it should be automatic, producing an articulated asset without manual intervention; lightweight, avoiding heavy external graphics software to keep per-asset inference cost low; and expressive, covering a wide range of articulated categories including complex mechanisms.

The Agent Harness and SDK

The key technical innovation is the design of the agent, which rests on two components: an agent harness and an LLM-friendly Software Development Kit (SDK). Together, they enable off-the-shelf large language models (LLMs) to generate articulated 3D assets without retraining, drawing on the LLM's coding ability and its prior knowledge of how everyday objects are structured and move. This design distinguishes Articraft from prior LLM-based methods for articulated asset generation.

The SDK makes it easier for the LLM to understand how to generate assets. The LLM is asked to write a single program against this SDK; once executed, the program produces a complete articulated asset. The SDK is focused, expressive, and LLM-friendly: it spans a wide range of articulated categories while staying close enough to familiar coding patterns that generation remains reliable. It exposes both low-level primitives (e.g., adding a cylinder) and high-level abstractions (e.g., adding a hinge), keeping programs compact and readable, and it lets the LLM write and execute object-specific validation code to test structural integrity.

The Articraft Harness

The harness turns the LLM into an iterative agent rather than a single-pass generator. The harness exposes a minimal workspace and interface: edit a single program, execute it, and receive or request feedback on the resulting 3D asset. Its design encodes deliberate choices about how much context to expose, how that context is structured, how generated assets are validated, and when to stop iterating.

Differently from general coding agents, the Articraft agent does not operate directly on a complex codebase. Instead, the harness presents to the agent a workspace containing only one writable artifact, model.py, plus read-only access to the SDK documentation and curated examples.

Example of an articulated asset generated by Articraft showing part hierarchy and joint structure

Evaluating the Articraft Agent

To assess the quality of the generated assets, participants were presented with text prompts alongside the generated 3D assets from all six competing methods (including two variants of ours) in a randomized order. The perceptual study was distributed across 125 non-expert college students with diverse academic backgrounds, with each participant evaluating around 40 randomly assigned objects, yielding a total of 5000 submitted comparisons. Each participant was asked to select and rank the three best results out of six based on prompt alignment and overall quality.

Evaluating the Articraft-10K Dataset

Having validated the agent, we now assess the effectiveness of the new Articraft-10K dataset as training data and in applications. The compute requirements of Articraft are modest because generation does not train a model and does not require rendered visual feedback. The expensive inference step is provided by external LLM API backends, while the local harness runs model.py, CAD and mesh construction, URDF export, authored tests, and quality-control checks on CPU workers. No GPU is required for Articraft generation, dataset materialization, or the compile/QC loop.

Simulation Readiness

To validate the simulation-readiness of our proposed framework, we directly imported the generated URDF files into NVIDIA Isaac Sim. Physical properties for each component, such as damping factors and mass, were automatically assigned by leveraging a Large Language Model. The generated assets are inherently compatible with physics-based simulation environments. For practical task execution, the system retrieves global coordinates from the URDF and employs standard Inverse Kinematics and control algorithms. The high-fidelity and clean collision meshes ensure superior performance in physical interactions and collision accuracy.

Failure Cases and Validation Trade-offs

One class of failures is poor global shape quality despite passing local structural checks. For example, in a "screwcap bottle" case the bottle shell is visibly malformed but is not detected by the testing suite because the generated shell still compiles as a connected mesh, does not introduce an unallowed inter-part overlap, and satisfies the authored local tests for the cap, neck, and rotation axis. The checks verify structural consistency and selected geometric relations, but they do not fully judge category-level visual plausibility or global surface quality.

A second class of failures arises from mechanisms or shapes that are awkward to express compactly in the current SDK. For instance, a "trigger spray bottle" case captures several parts of the spray-head mechanism, but the trigger shape and motion are difficult to model cleanly, and the trigger can overlap the bottle during its motion. These cases suggest that some categories would benefit from richer mechanism-specific abstractions or additional pose checks.

Finally, some failures are sporadic in more complex categories. The agent may omit interior structure or fail to hollow out shapes even when the exterior and articulation are plausible, as in "rice cooker" and "refrigerator with hinged doors" cases. These errors reflect the current tradeoff between cheap validation and stronger semantic or functional checks: the harness can efficiently enforce many structural constraints, but it does not yet fully capture all category-specific notions of realism and completeness.

Diverse examples of articulated assets from the Articraft-10K dataset, including mechanical tools and household items

Conclusions

We introduced Articraft, an agentic system for scalable articulated 3D asset generation. By pairing a task-specific SDK with a restricted execution harness, Articraft turns articulated asset creation into an edit–execute–repair loop grounded in validation and geometric feedback, without relying on rendered visual inspection. Using this agentic system, we created Articraft-10K, a large-scale dataset of over 10K high-quality articulated 3D assets with source programs, semantic part structures, joint specifications, and generation traces. Our results suggest that the domain-specific programmatic interface and execution harness are key ingredients for scalable and robust articulated asset generation, enabling downstream applications in feed-forward model training, simulation, and VR interaction.

Frequently Asked Questions

How does Articraft generate articulated 3D assets without training a model? Articraft uses off-the-shelf large language models to write code against a specialized SDK, then executes that code to produce complete articulated assets in a lightweight, iterative loop.

What types of articulated objects can Articraft create? It can generate a wide range of categories including cabinets, pliers, desk lamps, strollers, folding chairs, quadcopter drones, screwcap bottles, and many more complex mechanisms.

Does Articraft require a GPU for asset generation? No, Articraft generation, dataset materialization, and the compile/QC loop run entirely on CPU workers, with the only GPU-dependent step being external LLM API calls.

What are the main failure modes of the Articraft system? Failures include poor global shape quality that passes local structural checks, difficulty expressing certain mechanisms compactly in the SDK, and sporadic omissions of interior structure in complex categories.

🍪 Cookie preferences

We use cookies to measure performance. Privacy Policy