FasTac is a curved fingertip sensor that combines visible and near-infrared imaging in a single camera to reconstruct contact geometry and three-axis forces at high speed. Its FPGA-powered pipeline delivers 0.0415 mm depth accuracy and completes a full perception cycle in just 1.09 milliseconds, making real-time tactile feedback practical for dexterous robot hands.
What Did the Researchers Build?
FasTac is a curved, fingertip-sized tactile sensor engineered for dexterous manipulation. It combines three modules: a light-guiding skeleton that houses a miniature camera, a flexible LED circuit with red, green, blue, and near-infrared sources, and a soft silicone sensing skin. The camera uses an OmniVision OV2736 RGB-IR sensor whose 4x4 color filter array captures visible and near-infrared light in a single exposure. Because both spectra share one optical center, the images align at the pixel level without the extra bulk or registration cost of a multi-camera system.
The sensing skin is a platinum-catalyzed silicone elastomer mixed with softener at a 1:1:3 ratio for high compliance. A silver-powder reflective coating blocks external light and boosts contrast, while laser-ablated black marker dots track lateral skin movement. The researchers also built HyperForce, a force-estimation model that fuses depth-derived normal displacement with marker-derived tangential displacement using position-aware dynamic convolution. Finally, the entire pipeline — image capture, 3D reconstruction, and force estimation — was deployed on an FPGA edge processor, eliminating the need to stream raw images off the sensor.
What Are the Key Results?
In benchmark tests against CAD-aligned ground truth, FasTac reconstructed contact geometry with 0.0415 mm mean absolute depth error using RGB-NIR illumination with a boundary depth prior — a 33% improvement over the 0.0618 mm achieved with RGB-only illumination. Without the depth prior, both configurations drift to roughly 0.27 mm error, confirming that the boundary constraint is what keeps curved-surface reconstruction stable.
HyperForce estimated normal force with a normalized mean absolute error of 2.74% and shear force with 2.39% error. The complete FPGA pipeline runs in 1.09 ms per frame, which equates to roughly 900 Hz of usable tactile feedback.
| 3D reconstruction depth error | RGB-NIR + depth prior | RGB-only |
|---|---|---|
| With depth prior (MAE) | 0.0415 mm | 0.0618 mm |
| Without depth prior (MAE) | ~0.27 mm | ~0.27 mm |
| Force estimation (HyperForce) | NMAE |
|---|---|
| Normal force | 2.74% |
| Shear force | 2.39% |
| Pipeline performance | Value |
|---|---|
| Full image-to-force latency | 1.09 ms/frame |
| Equivalent update rate | ~900 Hz |
Validation on real objects — a strawberry surface, a fingerprint, and a miniature LED array — showed that RGB-NIR reconstruction preserves fine surface pits, ridge-valley continuity, and small component details more completely than RGB-only reconstruction.
How Does FasTac Work?
Vision-based tactile sensors infer contact from how a deformable skin's reflective surface changes under load. FasTac reconstructs the skin's 3D shape using photometric stereo: the camera watches how the painted elastomer surface reflects light from multiple known illumination directions, and the intensity variations reveal the surface normal at every pixel.
Curved fingertips break the standard photometric stereo assumption that illumination comes from three known directions over a flat surface. FasTac solves this by adding near-infrared as a fourth independent light channel. The RGB-NIR sensor captures all four colors in one exposure, and because the light sources sit behind the translucent skeleton — which scatters them into a soft, diffuse field — specular highlights are reduced and illumination uniformity improves.
The pipeline then merges two displacement signals. Depth maps from the photometric reconstruction provide normal displacement (how hard the skin is pressed), while the laser-marked dots provide tangential displacement (how much the skin stretched sideways). HyperForce's position-aware dynamic convolution explicitly models the fact that a curved elastomer is stiffer in some regions than others, so identical deformation at different locations maps to different forces. The FPGA processes the entire chain on the sensor itself, producing force estimates without transferring raw images to a host computer.
Why Does This Matter for Robotics?
Fast, accurate touch feedback is one of the missing pieces in dexterous manipulation. At roughly 900 Hz effective throughput, FasTac gives controllers a tactile signal fast enough for slip detection, grip-force regulation, and vibration-based texture recognition — all inside a fingertip-sized curved package that fits humanoid hand architectures.
Edge processing matters beyond latency: keeping image data on the sensor cuts cable bandwidth, power draw, and host-computer load, which simplifies integration into multi-sensor hands. That combination of precision, speed, and form factor points toward practical applications in precision assembly, fragile-object handling, and warehouse pick-and-place. Teams evaluating tactile upgrades for existing fleets can compare FasTac-class sensors against the platforms they would support, from browse humanoid robots on Robot Overflow to warehouse robots.
What Are the Limitations and Open Questions?
FasTac's accuracy depends on a boundary depth prior; without it, reconstruction error jumps roughly 6.5x. That means the sensor needs a good geometric model of its own undeformed skin, which restricts how well it generalizes to unusual contact shapes or heavily deformed skins. The soft elastomer and silver reflective coating will wear over time, and long-term durability across millions of contact cycles remains unproven. The FPGA implementation also demands specialized hardware engineering, and the sensor covers only one fingertip — whole-hand tactile feedback would require coordinated arrays of these sensors. Force error values were measured under the paper's test conditions, so performance across the full range of real-world grasps remains an open question.
Frequently Asked Questions
How accurate is FasTac's 3D shape reconstruction? The sensor achieves 0.0415 mm mean absolute depth error against CAD-aligned ground truth with RGB-NIR illumination and a depth prior. That is roughly a third better than RGB-only illumination.
How fast is the full tactile perception pipeline? The complete FPGA pipeline — image capture, 3D reconstruction, and force estimation — runs in 1.09 ms per frame. That translates to roughly 900 Hz of closed-loop tactile feedback.
Why use near-infrared instead of just RGB light? The NIR channel provides a fourth independent illumination direction for photometric stereo on curved surfaces without adding a second camera. Because the RGB-IR sensor captures both spectra at the same optical center, the channels align at pixel level.
What force information does FasTac provide? HyperForce estimates three-axis contact force with normalized mean absolute errors of 2.74% for normal force and 2.39% for shear force. It fuses depth-derived normal displacement with marker-derived tangential displacement to account for the curved skin's position-dependent stiffness.
Conclusion
FasTac shows that vision-based tactile sensing can hit the precision and speed demands of real-time dexterous manipulation. Its multispectral single-sensor design and FPGA edge pipeline make high-fidelity touch a practical input for robot control rather than a post-hoc measurement.