Path-Safe Sim-to-Real Learning for Hydroponic Lettuce Harvesting

Path-Safe Sim-to-Real Learning for Hydroponic Lettuce Harvesting

Al Bashir, Partho Ghose, Yibin Wang, Robert G. Hardin, Joshua Peeples +2 more

4 min read•Oct 18, 2026

Methodology

The proposed PSA-PPO motion controller integrates with a previously developed lettuce cutting-point localization perception module and cut-and-hold mechanism to create a complete pipeline. The policy is trained entirely in simulation and transferred to the physical robot without real-world fine-tuning.

End-Effector Integration

The harvesting tool adapts a previously developed lettuce cut-and-hold end effector for direct actuation by an OnRobot 2FG7 gripper. The redesigned components were modeled in SolidWorks 2026 (Dassault Systèmes, Vélizy-Villacoublay, France) and 3D-printed using fused deposition modeling (FDM) with polylactic acid (PLA) filament.

The original 2FG7 fingers were replaced by custom brackets carrying paired holding fingers and shear blades while retaining the first-class lever cutting principle. A single gripper-closing action therefore actuates the shear mechanism and simultaneously retains the detached lettuce for transfer.

Custom end effector with paired holding fingers and shear blades

Experimental Setup

PSA-PPO training and simulation evaluation were conducted using NVIDIA Isaac Lab in Isaac Sim 5.1.0 with the RSL-RL backend. Training used GPU-parallel simulation on an Ubuntu 22.04 LTS workstation equipped with dual Intel Xeon Gold processors, 256 GB RAM, and an NVIDIA RTX A6000 GPU with 48 GB VRAM.

Candidate policies were trained without physical robot interaction. The selected actor was evaluated deterministically on a fixed set of randomly generated targets spanning the prescribed front workspace. Component-removal ablations were evaluated on the same targets, enabling paired target-wise comparison with the complete PSA-PPO formulation.

Physical experiments used an RB-VOGUI mobile platform (Robotnik Automation, Paterna, Spain) equipped with a UR10e manipulator (Universal Robots, Odense, Denmark), an OnRobot 2FG7 gripper (OnRobot, Odense, Denmark), and an Intel RealSense D435i RGB-D camera mounted near the tool center point.

The adapted end effector was mounted directly to the 2FG7. An onboard Intel i7-based computer executed perception, target transformation, harvesting sequencing, deterministic PSA-PPO inference, ROS 1 communication, and gripper control. The simulation-trained actor was transferred to this platform without fine-tuning.

PSA-PPO policy and UR10e robot setup for lettuce harvesting

Kinematic and Atypical-Trajectory Diagnostics

Within the sensitivity-selected sets, Cartesian+TOTG contained no orientation-revolution, single-joint-revolution, or near-singularity hard events. The RRTConnect selected set contained two orientation-revolution events, one single-joint-revolution event, three wrist-near-singularity cases, and one elbow-near-singularity case. The RRT selected set contained four orientation-revolution events, three single-joint-revolution events, two wrist-near-singularity cases, and one elbow-near-singularity case. These counts describe the selected diagnostic subsets, not the full target population*.

The highest-priority selected planner execution is shown alongside PSA-PPO on the same target. The Cartesian case is the highest-ranked non-hard anomaly because no Cartesian hard event was present in the selected set. The RRTConnect and RRT* examples contain hard diagnostic events under the predefined audit rule. These examples illustrate tail behavior only and are not classified as collision events.

Code Availability

The source code and supporting materials will be made available upon publication at the authors’ GitHub repository.

Acknowledgments

This research is partially supported by the United States Department of Agriculture (USDA)’s National Institute of Food and Agriculture (NIFA) Research Capacity Fund Hatch Program: TEX09954 (Accession No. 7002248) and Research Capacity Fund Multistate Hatch Program: TEX0-1-9916 (Accession No. 7008389).

This publication is also supported by the Texas A&M University Division of Research Targeted Proposal Teams (TPT) funding program and Texas A&M AgriLife Research. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the authors and should not be construed to represent any official USDA or U.S. Government determination or policy.

Frequently Asked Questions

How is the PSA-PPO policy trained and deployed? It is trained entirely in simulation and transferred to the physical robot without real-world fine-tuning.

What robot platform was used in the physical experiments? Experiments used an RB-VOGUI mobile platform with a UR10e manipulator, an OnRobot 2FG7 gripper, and an Intel RealSense D435i camera.

How does the adapted end effector cut and retain lettuce? A single gripper-closing action actuates the shear mechanism and retains the detached lettuce using paired holding fingers.

Were the diagnostic trajectory examples classified as collisions? No. They illustrate tail behavior and are not classified as collision events.