Weak supervision via synthetic renders of CAD models has been used to align this feature space with the geometry-focused Normalized Object Coordinates (NOCs), yet models trained on synthetic and isolated data struggle to generalize to the clutter, occlusion, and appearance variation present in real scenes. Beyond training data, nearest-neighbor matching in high-dimensional semantic feature spaces, as used in existing works, is costly in both memory and computation and fails to exploit the rigid geometric structure of the problem. This produces noisy matches that often require expensive iterative refinement to recover accuracy.
Problem Definition
Given an RGB image, camera intrinsics, and a candidate CAD model retrieved from off-the-shelf methods, the goal is to estimate a 9D transformation from the CAD coordinate frame to the camera coordinate frame.
Overall CAD Fitting Pipeline
SUFLECA follows the standard indirect CAD alignment pipeline used in prior work.
Datasets
The DiffCAD split covers six of the ScanNet25k object categories, evaluated without NMS and using non-fine-tuned depth. This split was introduced by DiffCAD, whose priors are available only for six categories.
Results
On CO3D, results are obtained under occluded masks and inexact retrieved CADs, posing significant front–back and left–right geometric ambiguities. SUFLECA grounds objects in 3D substantially better than competing methods in this challenging setting, with its features more effectively able to disambiguate such cases. In seen categories, all variants of SUFLECA perform comparably and outperform competing methods, highlighting the effectiveness of our weak supervision. For unseen categories, SUFLECA-blend generally outperforms other methods, as it compensates for the limited supervision coverage for uncommon objects at the cost of a larger feature representation.
Frequently Asked Questions
How does SUFLECA differ from existing CAD-to-image alignment methods? SUFLECA scales up feature learning with weak supervision from synthetic CAD renders, reducing reliance on expensive real-annotated data and improving generalization to cluttered scenes.
What are the key challenges SUFLECA addresses? The method targets occlusion, appearance variation, and geometric ambiguities like front–back confusion that arise when using isolated CAD models in real environments.
How is SUFLECA evaluated across different object categories? The approach is tested on the DiffCAD split of ScanNet25k (six categories) and the CO3D dataset, with separate performance reported for seen and unseen object categories.
What trade-off does SUFLECA-blend introduce? The blend variant uses a larger feature representation to compensate for limited supervision on uncommon objects, improving unseen-category performance at the cost of increased memory.