FAMOS predicts the movable parts of an articulated object and their kinematic properties from sparse observations, given as partial point clouds, in a single forward pass using a Multi-state Articulation Transformer.
Multi-State Articulation Transformer
We build on recent query-based transformer formulations for feed-forward articulation prediction. Unlike prior work that operates on a single complete point cloud, our model must aggregate information scattered across multiple partial point clouds that differ in visible geometry and articulation state. Our method is designed around this requirement, combining state-wise geometric context with cross-state articulation cues.
Observed Articulation Span
Let the ground-truth joint value, such as an angle or displacement, for a part in an observation denote the articulation state. Let the set of observations in which a part is visible identify the observations available for that part.
The target is the articulation span computed from the observed joint values for parts that are visible in at least one observation. Joint values are given in radians for revolute joints and in object-normalized units for prismatic joints.
Training Objective
The model uses the Hungarian algorithm for assignment, where the mask logits of a point in an observation provide the predicted segmentation information. The assignment is computed jointly over all observations and is therefore shared across states. Unmatched queries are excluded from the loss computation.
Procedural Training Data Generation
We complement available articulated datasets with a parametric asset generator whose annotations are known by construction. Its modular architecture builds on basic shape primitives, allowing straightforward expansion to new object classes that decompose into the same constituent shapes.
Implementation Details
We source articulated assets from PartNet-Mobility and GRScenes. We filter both datasets to retain objects with pronounced, externally visible articulation and treat small components, such as buttons and knobs, as static.
This yields 1,020 objects across 25 classes from PartNet-Mobility and 968 objects across 15 classes from GRScenes. In addition, our procedural generator produces an unbounded stream of self-annotated assets.
Qualitative Results
Qualitative results cover all three evaluation benchmarks. Compared to the baselines, FAMOS identifies movable parts more reliably and estimates joint axes more accurately, while the baselines are more prone to under- and over-segmentation.
Conclusion
We presented FAMOS, a feed-forward approach for articulated object modeling from sparse monocular observations. Our method jointly reasons over multiple partial point clouds using a Multi-state Articulation Transformer that alternates between state-wise and global attention.
We further introduce an observed articulation span objective that encourages the model to leverage articulation cues across the full observation set. To increase the scale and diversity of training data, we design a procedural asset generator that synthesizes self-annotated articulated objects on-the-fly during training.
Experiments on three benchmarks demonstrate consistent improvements over existing feed-forward and optimization-based methods, while qualitative results show generalization to real-world captures.
Asset Families
To expand the scale of available datasets and avoid geometric memorization, our procedural generator synthesizes an unbounded stream of annotated 3D articulated objects. We group these assets into three main categories.
We design three families of cabinets:
- Hinged cabinets covering front, top, and side doors with single, double, or split openings.
- Drawer cabinets stacking a variable number of sliding drawers.
- Combo cabinets featuring two side-by-side bays combining doors and drawers.
Beyond cabinets, this group includes:
- Doors with swing, slide, and lift motions.
- Boxes incorporating split or hinged lids with sliding trays.
Frequently Asked Questions
What does FAMOS predict? FAMOS predicts the movable parts of articulated objects and their kinematic properties from sparse partial point-cloud observations.
What is the Multi-state Articulation Transformer? It is a transformer that aggregates geometric information across multiple partial point clouds with different visible geometries and articulation states.
What is the observed articulation span objective? It encourages the model to use articulation cues across the full set of observations in which a part is visible.
How is additional procedural training data generated? A parametric asset generator creates self-annotated articulated objects from modular shape primitives during training.