A hardware-software benchmarking platform for industrial dexterous manipulation is presented, comprising three custom-designed Industrial Dexterity Benchmark (IDB) boards targeting datacenter fiber switches, automotive cable harnesses, and planetary gearbox assembly. The work introduces DAG-ROS, a ROS2-based imitation-learning framework, and AG-iDP3, a multimodal diffusion-based policy architecture that fuses RGB images, point clouds, joint state, and wrist wrench inputs. Results on the datacenter cable manipulation board are reported across six sensor configurations.
Introduction
Real-world datacenter environments feature densely populated rack switches with sub-millimeter cable separation and routing at sharp angles. In such environments, even routine maintenance like connector cleaning or cable swapping requires careful coordination of perception, planning, and contact-aware control to complete tasks without disturbing the environment.
This work originated from efforts to solve the National Institute of Standards and Technology (NIST) Assembly Task Board #4 (ATB4) using classical computer vision paired with classical control techniques. That solution worked under controlled conditions but proved brittle and difficult to scale, prompting development of a new benchmarking standard to better mimic the variability and challenges of real-world datacenters.
Contributions and Outline
Three key contributions are outlined:
- The Industrial Dexterity Benchmark (IDB) Boards, inspired by the NIST Assembly Task Board set, comprising three custom benchmarking boards tailored to datacenter fiber switches (IDB Board #1), automotive cable harnesses (IDB Board #2), and planetary gearbox assembly (IDB Board #3)
- DAG-ROS, a ROS2-based imitation-learning framework that ties together teleoperation, sensor and sub-component integration, data collection, and real-time robotic arm control
- AG-iDP3, a framework for multimodal diffusion-based policies deployed and chained together using a PyTrees-based behavior tree architecture that takes RGB images, point clouds, joint state, and robot wrist wrench as inputs and outputs robot actions
Diffusion Policies for Manipulation
AG-iDP3 addresses gaps in existing approaches by fusing robot joint states, RGB images, 3-D point clouds, and wrist wrench into a unified diffusion policy. By incorporating a modality-gating mechanism, the architecture can selectively mask sensor inputs during training, enabling ablation studies across six distinct sensor configurations. The policies are trained on real demonstrations from solving the cable-cleaning task on IDB Board #1.
Industrial Dexterity Benchmarking (IDB)
To enable systematic evaluation and development of end-to-end policies on industrially relevant tasks, a family of benchmarking platforms was designed targeting distinct industrial manipulation challenges. The IDB program comprises three benchmark board designs: (1) Datacenter Cable Manipulation, (2) Automotive Cable Harness, and (3) Gearbox Assembly. All three follow the NIST assembly task board philosophy and are easily fabricated using 3-D printing, laser cutting, and off-the-shelf hardware with total cost in the low hundreds of USD. Results reported in this paper are exclusively on the Datacenter Cable Manipulation board; the remaining two designs are described for completeness.
Design 2: Automotive Cable Harness (IDB Board #2)
The benchmark includes eight connectors (two 6-pin, six 2-pin) distributed at varying heights, orientations, and levels of occlusion. The 2-pin connectors are equipped with LEDs that illuminate upon successful electrical connection, providing unambiguous pass/fail feedback. Receptacles in the enclosed area have multiple mounting positions allowing users to vary difficulty levels using the same board. An advanced routing option requires threading a cable through a narrow cutout in the wall, further stressing spatial awareness and robotic bimanual coordination.
Scoring evaluates both connection success (eight connectors, LED-verified) and cable routing compliance (whether cables pass through prescribed routing clips). No time limit is imposed and no penalties are assessed for re-attempts, enabling clean separation of capability from speed.
Design 3: Gearbox Assembly (IDB Board #3)
The correct sequence to successfully assemble the gearbox is as follows:
- Mount the ring frame onto the stepper motor with four M3 screws of length 8mm
- Insert the sun gear onto the motor shaft (D-spline alignment)
- Seat three planet gears, ensuring mesh with both the sun and ring gears
- Secure the frame cap with five M3 screws of length 12mm
For scoring, points are awarded for sub-tasks related to placement and fastening. One point is awarded for each of the following criteria:
- All bolts tightened such that they cannot be rotated by hand
- All planet gears meshing correctly
- The carrier assembly rotating freely after completion
Behavior Tree Architecture
Three of the root node's children are Parallel nodes (where multiple children run concurrently), each containing a policy node and an evaluator node that run concurrently. The evaluator is necessary because AI policies do not terminate on their own, as successive inferences keep generating new action chunks indefinitely. When the evaluator returns success, a stop request is sent to the policy node before the parent Parallel node itself returns success.
- Grasp — The AI policy locates and grips the connector on the rack. The evaluator detects when the gripper position drops below a closure threshold for a preset time.
- Clean — The AI policy wipes the connector face against the cleaning pad. The evaluator detects when the contact wrench magnitude exceeds a threshold for a preset time.
- Insert — The AI policy aligns the connector with the receptacle, inserts it, then opens the gripper. The evaluator detects when the gripper position rises above an opening threshold for a preset time.
This hybrid architecture combines learned policies for dexterous manipulation with classical motion primitives for reliable and safe movements, yielding a system that balances adaptability and reliability by applying each method where it is most appropriate.
Key Findings
The key takeaways from the experiments are:
- Whenever the policy has 3-D context available, through either point cloud or two RGB cameras providing different perspectives, the grasp phase performs well with success rates ranging from 88% to 98% across the five configurations that include 3-D context. Only in the single-RGB configuration, where 3-D context is entirely removed, does the grasp success rate drop to 48%.
- With the exception of the RGB-only baseline, insertion is the phase that differentiates total scores. Across configurations 1–5, insertion score varies by 48 percentage points while grasp success only varies by 10 percentage points.
- The best total score (78%) is achieved by the multi-view RGB configuration (RGB wrist + RGB scene) with R3M, which combines the strong grasping behavior of wrist RGB with the high-resolution scene context needed for fine alignment.
- In the direct head-to-head comparison between scene cameras, the EVAL-ADTF3175 ToF outperforms the RealSense D435i by 7 percentage points, suggesting that indirect ToF is better suited to industrial datacenter conditions.
- All four multimodal expansions outperform the single RGB camera DP baseline (36% total), and the egocentric point-cloud-only iDP3 (52% total) replicates the finding that an egocentric point-cloud policy beats an egocentric RGB policy.
- All three phases of the cable-cleaning benchmarking task (grasp, clean, and insert) required roughly 100 demonstrations each to train a well-performing policy across every ablation. At this demonstration count, AG-iDP3-trained policies are practical for on-site data collection and re-training whenever the task or hardware configuration changes.
Additional Considerations
Point-cloud resolution vs. industrial robustness. The resolution of both point cloud scene cameras, further downsampled in the PointNet encoder, was not fine enough to resolve the receptacle features needed for tight-clearance insertion. This is likely why the dual-RGB configuration outperformed both RGB-wrist + PC-scene configurations. In an industrial setting where lighting is not constant, equipment discolors with age, and airborne particulates are common, ToF sensing may provide more reliable output than passive stereo even when the experimental configuration favored the dual RGB configuration.
Scoring rationale. A scoring rule was fixed before experimentation. Summing grasp and insert successes (excluding cleaning, which always succeeded) reflects the operational reality that a successful insertion is meaningless without a successful grasp. If the robot fails to grasp and extract the cable, there is no opportunity for insertion to succeed; if the robot grasps and cleans but fails to insert, the task is partially complete.
Caveat on per-phase comparison for the RGB-only baseline. Per-phase comparisons against the RGB-wrist-only configuration should be interpreted carefully. Ports 2 and 3 had higher success rates than ports 1 and 4 on both phases. Because the RGB-only configuration's low grasp success rate filtered out a large fraction of insert attempts, the insert success rate for that configuration is not directly comparable to those of configurations that successfully grasp on most trials.
Robustness Observations
The learned policy was brittle to small visual scene changes. Two representative failure modes were observed: (i) removing a background object (a red polymer block) that was present at training time caused the policy to fail; (ii) changing the routing of a cable along the robot arm (loose vs. tied) also caused failure. These issues were mitigated by training and deploying on cropped images so that only the controlled task area is fed to the policy, and by adopting pulley-based cable management so cables return to the same position regardless of arm path. While these task-specific workarounds restored reliable behavior, the underlying robustness issue persists: the visual encoder appears to latch onto incidental scene features rather than task-relevant ones.
Future Directions
Ongoing research focuses on three threads:
Algorithmic improvements include richer pre-trained encoders for point-cloud, depth, and grayscale inputs (e.g., DINOv3, DiT), multi-task or semantic encoding, dynamic input resolution, dynamic prediction horizons, Cartesian-space action models, and a slow-fast policy that pairs a high-rate reactive controller with a low-rate planner. Robustness and data scaling can be addressed using domain randomization, simulation-based training with data augmentation, and reinforcement-learning fine-tuning.
Sensing and platform extensions include direct head-to-head evaluation of an ADI Tembin wrist ToF sensor against the D405; event cameras for low-latency motion cues; tactile sensing for local contact and slip detection; an ADI 6-DOF wrench sensor compared with the FR3's joint-torque-derived wrench; multi-arm bimanual coordination for routing, fixturing, and handoff; dexterous multi-finger hands for in-hand reorientation; and haptic teleoperation devices for richer contact-rich demonstrations.
Compute and deployment improvements target faster inference, higher action rates, lower latency, and better power efficiency. These goals can be addressed by moving from RTX-class workstations to edge deployment on neural processing units (NPUs) and analog neural-network accelerators.
Frequently Asked Questions
What hardware is needed to build the IDB boards? The IDB boards are designed for easy fabrication using 3-D printing, laser cutting, and off-the-shelf hardware, with total cost in the low hundreds of USD.
How many demonstrations are needed to train AG-iDP3 policies? Each phase of the cable-cleaning task (grasp, clean, insert) required roughly 100 demonstrations to train a well-performing policy across all ablation configurations.
Which sensor configuration achieved the best results? The multi-view RGB configuration (RGB wrist camera + RGB scene camera) with R3M achieved the best total score of 78%, combining strong grasping behavior with high-resolution scene context.
Why did the learned policies sometimes fail in new environments? The visual encoder tended to latch onto incidental scene features rather than task-relevant ones, causing failures when background objects were moved or cable routing changed.