Plant phenotyping based on observable traits plays a critical role in modern agriculture, both for stress detection and for linking physiological responses to genotype in breeding programs. Vision-based systems can estimate macroscopic variables such as growth rate, biomass, or yield. However, in the context of climate change, understanding how different plant varieties respond to environmental stress requires access to physiological indicators that go beyond passive imaging.
Chlorophyll fluorescence is a direct reporter of photosynthetic activity and provides a sensitive indicator of plant stress. Traditional fluorescence measurements in greenhouse or field conditions rely on handheld devices that require physically clamping individual leaves and applying controlled light stimulation protocols. While accurate, this approach is labor-intensive, requires manual intervention, and does not scale well to high-throughput phenotyping.
Recent advances in miniaturized electronics and the widespread availability of components developed for wearable devices have enabled the development of more compact fluorescence sensors. A portable device provided by the Jan IngenHousz Institute allows long-term, leaf-mounted recordings. Although these compact systems produce high-quality temporal data, their deployment still requires manual positioning and retrieval, making large-scale and spatially resolved measurements time-consuming and difficult to automate.
A robotic platform automates targeted chlorophyll fluorescence measurements across multiple leaves and plants using a single portable sensor. The system integrates a robotic manipulator and a camera–sensor gimbal, enabling precise positioning of the fluorescence probe. The mechanical architecture of the platform and the integration and experimental validation of the fluorescence sensor are described first, followed by a perception pipeline for leaf detection and 3D localization. Motion planning strategies then generate collision-free trajectories to position the sensor under geometric and accessibility constraints.
By coupling active physiological sensing with robotic perception and manipulation, the system enables scalable, repeatable, and spatially resolved plant phenotyping, providing a robotics-driven solution for high-throughput agricultural inspection.
Hardware
The hardware configuration of the active sensing platform is designed to remain low-cost, with a total budget in the $1000–1500 range. The majority of the cost corresponds to the CNC stage and the aluminum structural frame supporting the arm.
The Five-Degrees-of-Freedom Gimbal
The sensing unit consists of a Pi Camera Module 3 and a compact chlorophyll fluorescence sensor mounted on a custom 3D-printed gimbal at the end effector of a lightweight arm fabricated from PLA, PETG, and aluminum elements.
Cartesian positioning is achieved using an X-Carve CNC platform providing a 750 × 750 mm planar workspace and 31.5 mm of z-axis motion, with a nominal accuracy of approximately 0.05 mm. Orientation control is provided by a two-axis pan–tilt mechanism driven by stepper motors, allowing full 360-degree rotation in both axes. Two slip rings ensure continuous power delivery and signal transmission without cable torsion.
Motion control of the CNC axes is handled by motor drivers connected to a Raspberry Pi 5. The pan–tilt stepper motors are driven by an Adafruit Feather M0 interfaced via USB. The gimbal carries a Raspberry Pi Zero W for local acquisition and control of the camera and fluorescence sensor, communicating wirelessly with the main controller via Wi-Fi.
The system is organized into two distributed nodes:
- The CNC motion node.
- The gimbal sensing node.
The nodes communicate through custom middleware implementing a request–response protocol with automatic module discovery.
The Fluorescence Sensor
The device is an active chlorophyll fluorescence sensor. It operates by illuminating the leaf with an LED at the excitation wavelength of chlorophyll and measuring the emitted fluorescence signal, which serves as a direct proxy for photosynthetic activity. Accurate measurements require near-contact positioning, at an approximately 5 mm standoff distance, and alignment of the optical axis as close as possible to the local surface normal of the leaf. These geometric constraints motivate the need for precise robotic positioning and orientation control.
The sensor is interfaced via USB through a serial adapter to the gimbal sensing node, and communication is handled through a command-based serial protocol. The system allows programmable excitation sequences, enabling arbitrary light stimulation protocols depending on the experimental requirements. This flexibility makes the platform compatible with standard fluorescence assays as well as custom dynamic stimulation paradigms.
To characterize the sensor response, a sequence of 1-second light pulses with increasing intensity was applied, with pulses separated by 30-second dark intervals. The average fluorescence signal during each pulse was recorded on two independent Pilea peperomioides plants. The resulting light-response curves exhibit the expected behavior: a linear increase in the low-light regime, followed by saturation and a gradual decrease at higher intensities due to non-photochemical quenching, a stress response to high light. These results confirm the sensor’s ability to capture physiologically meaningful fluorescence dynamics.
From these measurements, a ratio is computed from the relative fluorescence response during the transient decrease and the saturating flash. This scalar metric provides a compact and robust indicator of light stress and can be readily integrated into automated robotic phenotyping workflows.
Targeted Measurements
The fluorescence sensor requires precise positioning relative to the leaf. This is essential for reproducibility and for controlling the level of noise in the signal. Each leaf must therefore be detected, and the sensor pose must be set so that it faces the leaf surface plane at the sensor’s working distance.
Perception Component
Leaf localization requires accurate 3D reconstruction of the plant followed by geometric segmentation of individual leaves. To reconstruct the plant geometry, multiple RGB images are acquired from viewpoints distributed in a circle around the specimen. Eighty views were used in the experiments. Camera poses are estimated using a Structure-from-Motion pipeline.
Although Structure-from-Motion directly produces a sparse or dense point cloud, the recovered camera poses are instead used as input to a space-carving algorithm to obtain a volumetric reconstruction. In this setup, space carving yields a denser and less noisy point cloud than the direct output of Structure-from-Motion. A known limitation is that space carving assumes photo-consistency under silhouette constraints and cannot accurately reconstruct strong concavities. As a result, highly curled or self-occluding leaves may be partially reconstructed.
Because the acquisition setup uses a uniformly black background, simple color-based segmentation is sufficient to isolate the plant from the scene. An Excess Green index is computed to generate binary masks, which are then used as silhouettes for the 3D reconstruction pipeline.
From the reconstructed 3D point cloud, leaf segmentation is performed using local geometric descriptors. For each point, the covariance matrix of its local neighborhood is computed at multiple spatial scales. Three scales corresponding to neighborhoods of approximately 30, 100, and 300 points are used. Eigenvalue decomposition of this covariance matrix provides a compact representation of local surface geometry.
Planar structures, such as leaf laminae, exhibit two dominant eigenvalues and one near-zero eigenvalue, while linear structures such as stems exhibit a single dominant eigenvalue. These geometric signatures allow discrimination between leaf surfaces and supporting structures.
After classification, individual leaves are extracted using density-based clustering, which is robust to noise and variable point density. For each segmented leaf, its centroid is computed and the local surface normal at that point is estimated. These geometric features are subsequently used to define target poses for the fluorescence sensor, ensuring orthogonal alignment and accessible positioning for robotic measurement.
Motion Planning
Using the leaf centroids and surface normals estimated by the perception module, a motion planning strategy generates trajectories for the robotic arm that avoid collisions with the plant volume. The strategy operates in two phases:
- A global path-planning step routes the end effector between successive target leaves while circumventing the plant.
- A local approach step positions the sensor at a geometrically consistent standoff distance from each leaf surface.
Visualization Interface
A web-based interface implemented using the Dash framework provides interactive visualization of the reconstructed point cloud and the corresponding measurement locations on each leaf. The interface allows users to inspect the spatial distribution of sampled points in 3D, alongside the associated RGB image of the probed leaf and the recorded fluorescence time series.
This unified visualization enables qualitative assessment of targeting accuracy and physiological measurements.
Conclusion
A low-cost robotic platform was presented for automated active sensing of chlorophyll fluorescence in plants. By integrating 3D perception, geometric reasoning, and precise manipulation, the system enables targeted, repeatable fluorescence measurements at the leaf level. Using the proposed stimulation protocol, the platform demonstrated that it can discriminate between dark-adapted and light-exposed leaves based on a standard indicator of photosynthetic light stress.
While the binary discrimination between dark-adapted and light-stressed leaves validates the sensing and positioning capabilities of the system, more subtle stress heterogeneity will require higher-dimensional analysis of fluorescence dynamics. In this context, machine learning approaches such as dictionary learning could be leveraged to extract informative features from temporal fluorescence traces and improve sensitivity to complex physiological states. The results could also guide the placement of leaf-mounted devices based on heterogeneity across the plant.
The current evaluation was conducted on plants with relatively simple geometry. For specimens with highly complex or self-occluding structures, the reconstruction and segmentation pipeline may fail because of the limitations of the space-carving approach. More advanced 3D reconstruction methods, such as recent neural methods, could handle more complex geometries, potentially at the cost of noisier point clouds.
Finally, reducing reconstruction time is critical for deployment in greenhouse or field conditions. Future work will focus on accelerating the perception pipeline, for example by reducing the number of views. The integration of the sensing arm onto a mobile robotic platform is also of interest, enabling scalable and autonomous plant-level phenotyping in real agricultural environments.
Acknowledgment
The authors thank Ludovico Caracciolo and David Kramer from the Jan IngenHousz Institute, who allowed them to use a preliminary version of their fluorescence instrument, and Aliénor Lahlou from Paris Research, Sony CSL, for fruitful discussions and help calibrating the instrument.
An LLM assistant was used for clarifying and correcting language in the writing of this paper. All scientific content and interpretations are solely the responsibility of the authors.
Frequently Asked Questions
What plant signal does the robotic platform measure? It measures chlorophyll fluorescence, a direct reporter of photosynthetic activity and an indicator of plant stress.
How does the system locate individual leaves? It reconstructs the plant in 3D from multiple RGB views, then uses local geometric descriptors and density-based clustering to segment individual leaves.
Why must the fluorescence sensor be positioned precisely? Measurements require an approximately 5 mm standoff distance and optical alignment close to the leaf surface normal.
What are the main limitations of the current system? Space carving may fail on highly curled or self-occluding leaves, and reconstruction time must be reduced for greenhouse and field deployment.