Researchers built ZONDA, a zero-shot object navigation framework that works across multiple floors while proactively avoiding moving pedestrians. By using a heuristic planner and multi-view verification, it achieves state-of-the-art success rates without requiring robot-specific training or static environments.
What the Researchers Built
ZONDA is a zero-shot object navigation framework designed for indoor robots operating in multi-floor buildings where people move around. Unlike previous methods, it requires no reinforcement learning policies tied to a specific robot—meaning it can be dropped onto different platforms without retraining. The system maintains three layered maps: an object instance map, a semantic heatmap of likely target locations, and a height-difference traversability map that detects stairs and ramps.
The framework has three core modules. First, a Heuristic Multi-Floor Planner groups nearby frontiers into “blocks” and scores them using semantic cues, then selects the best block to explore—and can autonomously navigate stair transitions. Second, a Multi-View Target Verification module uses a vision-language model (VLM) to examine multiple views of a candidate object before confirming a match, drastically reducing false positives. Third, a dynamic pedestrian avoidance pipeline detects moving people using Kalman filters and treats both current and predicted positions as obstacles, updating the occupancy grid in real time.
Key Results
The team evaluated ZONDA on three benchmarks: HM3D, MP3D, and a new dynamic benchmark HM3D-DYNA (which adds moving pedestrians). On HM3D, ZONDA achieved a success rate (SR) of 66.5% and success weighted by path length (SPL) of 33.0%, outperforming the prior state-of-the-art ASCENT by over 5 percentage points in SR and 7 points in SPL. On MP3D, it reached 48.2% SR and 21.5% SPL, again beating ASCENT.
Ablation studies quantify each module’s contribution:
| Variant | HM3D SR | HM3D SPL | MP3D SR | MP3D SPL |
|---|---|---|---|---|
| Full ZONDA | 66.5% | 33.0% | 48.2% | 21.5% |
| w/o heuristic blocks | 62.6% | 26.8% | 38.6% | 13.4% |
| w/o cross-floor navigation | 57.8% | 28.5% | 44.2% | 20.7% |
| w/o multi-view verification | 41.5% | 23.4% | 30.2% | 11.6% |
The multi-view verification module alone accounts for a 25-percentage-point SR drop when removed on HM3D, highlighting how critical it is to avoid premature false matches. On HM3D-DYNA with dynamic pedestrians, ZONDA maintained a SR of 62.1% and SPL of 29.8%, while a baseline without pedestrian avoidance collapsed to 38.5% SR.
How It Works
ZONDA operates on a three-layer map built from RGB-D input at a fixed grid resolution. The object instance map stores detected objects with incremental confidence scores, the semantic heatmap encodes likelihood of finding a target based on prior object co-occurrence, and the traversability map uses height differences to identify surfaces and stairs. All three maps are updated every frame.
The Heuristic Multi-Floor Planner groups unexplored frontiers into spatially contiguous blocks using DBSCAN clustering. Each block is scored by a heuristic that balances semantic relevance (how likely the block contains the target based on the heatmap) and information gain (how much new area would be uncovered). The robot then navigates to the highest-scoring block using A* on the occupancy grid. If a stair or ramp is detected in the traversability map, the planner triggers a cross-floor transition by steering the robot toward it and adjusting its height reference.
When the robot reaches a candidate target location, the Multi-View Target Verification module queries a buffer of recent observations for the best-quality views at multiple scales (near and far field). These images are fed as a batch to a VLM (e.g., GPT-4V), which reasons jointly about context and detail. For instance, a small radio may look like a TV in one close-up but is ruled out when the VLM sees the wider scene shows no TV-sized object.
Dynamic pedestrians are detected by segmenting RGB-D frames, back-projecting depth points inside instance masks to 3D, and tracking centroids with a constant-velocity Kalman filter. Data association uses the Hungarian algorithm. Both current and predicted positions (extrapolated for 2 seconds) are inflated by 0.50 m and projected onto the occupancy grid. The planner then selects waypoints that avoid these dynamic obstacles, and a low-level MPPI controller outputs smooth velocity commands for execution on the real robot.
Why This Matters for Robotics
Most object navigation systems assume static, single-floor environments—an unrealistic constraint for real-world deployments in offices, hospitals, or warehouses. ZONDA solves three practical problems at once: it can search across multiple floors without a trained stair-navigation policy, it avoids false positives that would stop a robot prematurely, and it navigates safely around people.
For warehouse operators, this means mobile robots can locate misplaced items across multiple levels without costly site-specific training. For service robotics in hospitals, a robot can autonomously deliver supplies while dodging staff and visitors. The zero-shot nature—no retraining per robot—makes it especially attractive for fleets of heterogeneous robots. The modular architecture also means the planner and verification modules can be reused on different platforms, from wheeled to biped robots.
Those upgrading existing fleets can explore warehouse robots that could benefit from this type of navigation stack. For bipedal or humanoid platforms, see browse humanoid robots on Robot Overflow.
Limitations and Open Questions
ZONDA currently requires a robust VLM for multi-view verification, which adds latency and computational cost. The paper reports real-time performance using an off-board workstation with an RTX 5060 Ti, but deploying entirely on-robot would require more efficient models or hardware acceleration. The dynamic avoidance pipeline assumes predictable pedestrian motion (constant velocity) and may fail with erratic movements or crowds. Additionally, the stair traversal strategy relies on traversability maps from height differences—it may struggle with steep or narrow stairs that don’t produce clear height gradients. Finally, the system has only been tested indoors; outdoor multi-floor navigation (e.g., ramps, uneven terrain) remains unexplored.
Frequently Asked Questions
What does “zero-shot” mean in ZONDA? ZONDA requires no task-specific training or fine-tuning on the target environment. It uses pretrained models for perception and a VLM for verification, then applies heuristic planning to navigate.
How does ZONDA handle stairs between floors? The traversability map detects height discontinuities that correspond to stairs or ramps. The planner treats them as navigable cells and adjusts the robot’s height reference during transition, avoiding the need for a learned stair policy.
Does ZONDA work with any robot platform? Yes. The low-level controller (discrete or continuous) is decoupled from the planning modules. The planner outputs waypoints, and any compatible local planner can execute them—demonstrated on a Direct Drive Tech TITA biped robot.
What benchmarks were used to evaluate ZONDA? The team tested on Habitat-based HM3D and MP3D benchmarks, plus a new HM3D-DYNA variant that adds moving pedestrians. Results show consistent improvement over prior methods across all three.
Conclusion
ZONDA sets a new standard for zero-shot object navigation by tackling multi-floor exploration, false-positive avoidance, and dynamic pedestrian safety in a single framework. Its modular, platform-agnostic design makes it highly practical for real-world deployment in human-populated indoor spaces.