An eight-thruster autonomous underwater vehicle (AUV) can track a fish-shaped target more reliably by combining stereo depth, target appearance, and motion history. The system turns those estimates into predictive, constraint-aware commands, separating yaw from translation so the vehicle can maintain a useful view while the target and water conditions change.
What did the researchers build?
The researchers built a complete underwater visual-servoing system: a camera-based perception and control stack that converts what an AUV sees into movement commands. The vehicle uses a forward-facing stereo camera, which captures two slightly different views so software can estimate scene depth from image disparity.
The central perception module is target-specific. Instead of accepting a noisy depth estimate for every pixel in the scene, it combines three types of evidence around the tracked object:
- Color cues, which help separate the target from the background.
- Disparity cues, which indicate how far visible surfaces are from the camera.
- Temporal cues, which use information from earlier frames to stabilize the target mask.
The resulting target depth and image position are filtered separately. That distinction matters because image motion and range changes behave differently when an underwater target rotates, deforms, or changes apparent size.
The system runs on an eight-thruster AUV. Four horizontal thrusters handle planar movement and yaw, while four vertical thrusters control heave, roll, and pitch. A host computer runs perception, state estimation, and high-level control through ROS 2; the onboard controller translates those commands into thruster actuation.
What results did the underwater tracking system achieve?
The evaluation covered video replay, simulation, and closed-loop experiments in a physical pool. The real-world perception study used 12 videos: four with mostly static disturbances and eight with dynamic conditions involving target motion, vehicle motion, attitude changes, and target yaw.
Every method received the same rectified stereo images, detection boxes, and disparity maps. Metrics were calculated separately for each video and then summarized using the median across all 12 sequences. This setup makes the comparison less sensitive to one unusually easy or difficult recording.
The reported findings show that target-specific color–disparity–temporal fusion produced more usable target-depth measurements than relying on scene-level stereo estimates alone. Separate filtering of image-plane and depth observations also improved the stability of the estimated relative state.
At the control level, the system remained effective when the target’s movement changed. Its adaptive model fusion combined a constant-velocity prediction with a zero-velocity prediction, rather than committing to one assumption about how the target would move. The pool experiments validated the complete perception-to-thruster loop on the eight-thruster AUV.
The supplied results description does not provide exact error reductions, tracking percentages, or latency values, so no numerical accuracy comparison can be stated responsibly. Its strongest evidence is system-level validation across replay, simulation, and physical tracking conditions.
How does the depth estimator and predictive controller work?
The system follows a position-based visual-servoing, or PBVS, design. Instead of controlling only the target’s pixel location, PBVS estimates the target’s three-dimensional position relative to the camera. That lets the controller explicitly regulate distance and preserve camera visibility.
The perception pipeline first receives stereo images, target detections, and disparity maps. A fused mask identifies which parts of the image are likely to belong to the target. Color helps reject background regions with unsuitable appearance; disparity helps distinguish surfaces at different ranges; and temporal information discourages frame-to-frame jumps in the target region.
Depth is then extracted from the target region rather than treated as a property of the entire scene. The estimated horizontal and vertical image position is filtered separately from range. This separation reduces the chance that a sudden depth outlier will destabilize the full relative-state estimate.
The controller uses a quadratic-programming model predictive control approach. MPC predicts how future commands will affect tracking error, then selects an action while respecting constraints such as actuator limits and vehicle motion. The design decouples yaw regulation from translational control: yaw keeps the target suitably aligned with the camera, while the translational optimizer handles forward, sideways, and vertical movement.
The controller also maintains two competing target-motion models:
- Constant velocity, useful when the target is moving steadily.
- Zero velocity, useful when the target pauses or appears stationary.
After each prediction horizon, the system compares the models with historical multi-step prediction errors. The model with the stronger recent record receives more weight, and the combined prediction is used for the next control decision. This gives the AUV a way to adapt without manually switching modes.
The experiments used a Unity-based underwater simulation connected to ROS 2, while the physical setup used calibrated camera parameters and shared algorithm settings across methods.
Why does this matter for real-world robotics?
Underwater robots operate with poor visibility, unstable lighting, suspended particles, reflections, and targets that can change shape or orientation. A controller that reacts only to the current image error can issue commands that arrive too late, overshoot the target, or move the camera away from the object it needs to observe.
This system addresses those problems in two practical ways. First, target-specific depth estimation focuses the stereo measurement on the object that matters, instead of trusting every scene pixel equally. Second, adaptive prediction gives the controller a way to handle both moving and temporarily stationary targets without relying on a fixed motion assumption.
The approach is relevant to underwater inspection, aquaculture monitoring, marine research, and robotic interaction with moving objects. The same design principles also apply outside the water: sensor-specific filtering, explicit three-dimensional goals, and control that respects actuator limits are useful when evaluating used industrial robots or autonomous platforms for difficult operating environments.
For robot buyers and operations teams, the work highlights an important distinction between visual detection and usable autonomy. Detecting an object is not enough; the robot must estimate where that object is, predict where it will be, and convert that estimate into safe movement.
What are the limitations and open questions?
The physical validation took place in a controlled pool with a fish-shaped model, not in open water with a living animal or an operational inspection target. Pool conditions make repeatable testing possible, but they do not fully represent turbidity, changing sunlight, currents, biofouling, or cluttered seabeds.
The system also depends on usable stereo disparity and continued target observations. Severe visibility loss, partial occlusion, or complete detection failure can still break the state estimate. The controller’s fixed camera calibration and algorithm parameters may require retuning for another vehicle or camera arrangement.
The researchers identify disparity refinement, observation loss, and open-water generalization as future priorities. Additional testing should also report computational latency, actuator saturation frequency, recovery after lost observations, and performance across different target shapes and speeds.
Frequently Asked Questions
What type of robot was tested?
An eight-thruster AUV with a forward-looking stereo camera was tested in simulation and physical pool experiments.
Why does the system estimate target-specific depth?
Scene-level stereo depth can be noisy underwater, so the system focuses depth estimation on pixels associated with the tracked target.
What does model fusion add to predictive control?
It blends constant-velocity and zero-velocity predictions, adjusting their influence according to recent multi-step prediction errors.
Was the system tested in open water?
No. The reported physical experiments used a controlled pool and a fish-shaped model.
Conclusion
The research combines target-focused stereo perception with adaptive model-fusion MPC to make underwater visual tracking more stable and more responsive to changing target motion. Simulation, video replay, and pool experiments demonstrate a complete perception-and-control pipeline on an eight-thruster AUV, while open-water robustness remains the next major test.