A dual-arm robot system has played guqin strings using a custom fingertip that senses both pressing force and vibration. The design combines a soft fingerpad with a nail-like surface, enabling robotic tests of open strings, harmonics, and stopped notes while tactile contact timing triggers the other arm’s pluck. The work turns an unusually delicate musical technique into a measurable robotics test.
What Did the Researchers Build?
The researchers designed a human-inspired tactile fingertip for a robot playing the guqin, a traditional Chinese seven-string instrument known for quiet, highly controlled tones. The fingertip combines a soft fingerpad with a harder, nail-like section. The nail-like surface reduces friction and wear during sliding, while the soft section maintains contact when the robot presses a string.
The fingertip contains a 4 × 4 array of 16 piezoresistive sensing units. These sensors capture both sustained pressure and changing vibration, giving the robot information about when contact begins and how the string is behaving. That combination matters because guqin playing requires more than simply touching a string: the left hand must control pitch and resonance while the right hand plucks at the correct moment.
A dual-arm robot was used to test the design. One arm pressed the string with the tactile finger, while the other arm used its middle finger to pluck downward. The setup covered three basic sound-producing actions: an open string, a stopped note created by pressing the string against the instrument body, and a harmonic produced through light pressure and carefully timed release.
The guqin itself is roughly 122–125 centimetres long, 20 centimetres wide, and 6 centimetres thick. Its fretless layout makes accurate pressure and position control especially important.
What Were the Key Results?
The fingertip produced a repeatable contact signal that coordinated the two robot arms. In ten harmonic-playing trials, the average time between detected contact onset and the peak of the resulting audio signal was 481 milliseconds, with a standard deviation of 56 milliseconds. This shows that the tactile trigger consistently initiated the second arm’s pluck, although the timing still includes the robot’s movement and audio response.
The tests also mapped how pressing depth and release timing changed the sound. Harmonic tone strength peaked at an 8-millimetre string depth. At 10 millimetres, the string touched the guqin’s wooden soundboard and the sound changed into a stopped note.
| Test condition | Reported result |
|---|---|
| Tactile-event coordination trials | 10 |
| Contact-to-audio peak delay | 481 ± 56 ms |
| Tested string-depth range | 0–10 mm |
| Strongest harmonic response | 8 mm |
| Left-hand release delay tested | 120–200 ms |
| Tactile sensing units | 16 |
A shorter release delay produced a higher-pitched, louder sound closer to an open string. A longer delay lowered the pitch and created a deeper, more muted tone. The source reports no direct comparison with another robotic fingertip or control system, so these results represent task demonstrations rather than a head-to-head benchmark.
How Does the Robot Finger Work?
The fingertip uses two complementary sensing behaviours. One responds to relatively steady contact, such as the force needed to hold a guqin string down. The other captures faster changes caused by contact onset, vibration, and sliding. Together, they allow the system to distinguish a finger arriving at the string from a finger simply maintaining pressure.
Calibration separated the mechanical behaviour of the fingertip from the robot’s motion. A three-axis motion stage positioned a 1-millimetre spherical indenter over each of the 16 sensing units. A six-axis force/torque sensor measured the applied contact force while the fingertip’s electrical outputs were recorded. A separate vibration platform tested dynamic response from 1 to 100 hertz. All tactile channels were sampled at 1,000 hertz, giving the controller much faster feedback than the robot’s visual system.
The coordination experiment used an event-triggered controller rather than full tactile feedback control. The controller summed the signals from all 16 sensing units, filtered the result into a relevant frequency band, and calculated its rate of change. When that rate crossed a fixed threshold, the system treated the event as contact onset and commanded the right arm to perform one pre-planned pluck.
This distinction is important. The tactile signal controlled when the next action began, but it did not continuously adjust pressing force, finger position, or release timing. For the harmonic experiments, the researchers separately varied pressing depth and the delay between the right-hand pluck and the left-hand release. The audio envelope then showed how those timing choices shaped the resulting tone.
Why Does This Matter for Robotics?
Many robots can move accurately in space, but delicate manipulation often fails because contact is treated as a binary event: touched or not touched. Guqin playing exposes the missing layer. A robot must sense pressure, friction, vibration, timing, and release behaviour while coordinating two hands around a flexible string.
The work therefore provides a compact test case for dexterous manipulation. Musical performance makes errors easy to detect through sound, while the instrument demands the same capabilities needed for fragile assembly, cable handling, surface finishing, and contact-rich inspection. A tactile event that reliably triggers a second action can also support handover tasks, tool use, and coordinated manipulation in industrial environments.
The half-nail, half-fingerpad construction is another practical design idea. A robot fingertip that combines low-friction sliding with compliant contact could help extend the usefulness of used cobots for sale in tasks requiring both contact sensitivity and repeatable motion. The dual-arm coordination approach is also relevant to used industrial robots, although industrial deployment would require more robust sensing, calibration, and safety control.
The research does not make a robot a complete musician. It shows instead how a culturally specific, technically demanding task can expose limitations in tactile perception and bimanual control that standard pick-and-place tests often hide.
What Are the Limitations and Open Questions?
The system demonstrates selected string-contact actions rather than a complete guqin performance. Fast multi-string techniques, expressive vibrato, sliding, harmonics across changing hand positions, and full musical pieces remain unresolved.
The controller is also deliberately limited. Tactile input triggers the next pluck but does not continuously servo the robot’s force or trajectory. The visual system operates at 30 hertz, which constrains fusion between visual, tactile, and audio information when coordinated movements need control at 100–200 hertz. The ten-trial coordination test establishes repeatability but is too small to characterise long-term reliability, sensor drift, or performance across different instruments and strings.
Future work centres on aligning audio, human performance video, and Jianzipu notation so that demonstrated fingering can be transferred into robot motion sequences.
What Are the Frequently Asked Questions?
What is a stopped note on the guqin?
A stopped note is produced when the robot presses a string against the guqin’s wooden body. The fretless instrument requires pressure and position control rather than a fixed fret location.
What does the haptic fingertip sense?
It senses sustained contact forces and faster changes associated with vibration and contact onset. Its 16 sensing units are sampled together at 1,000 hertz.
Does the robot finger continuously control playing force?
No. In the demonstrated coordination task, tactile input detects contact onset and triggers a pre-planned pluck, while force and release timing are set separately.
Can this system play a complete guqin piece?
Not yet. The experiments cover open strings, harmonics, and stopped notes, while rapid multi-string techniques and expressive interpretation remain open challenges.
What Is the Takeaway?
The study shows that a purpose-built tactile fingertip can give a dual-arm robot useful contact information for delicate guqin techniques. Its strongest contribution is not autonomous musical performance, but a clear demonstration of how haptic sensing can connect physical contact to coordinated robotic action.