Researchers have developed EM-Fall, a system that mounts mmWave radar on a humanoid robot to detect falls day and night, even in complete darkness. The robot actively follows the person and uses a large language model to decide whether to alert caregivers or continue monitoring, achieving perfect accuracy in distinguishing falls from normal movement.
What the Researchers Built
EM-Fall is an embodied mobile sensing system that places an off-the-shelf millimeter-wave (mmWave) radar directly onto a humanoid robot. Instead of relying on static wall-mounted sensors that leave blind spots, the robot actively tracks a person through their environment. The system consists of three tightly coupled layers: spatial state estimation maintains a stable estimate of the target’s location despite the robot’s own motion; human-centered perception uses the radar point cloud to infer body orientation and detect falls based on temporal motion patterns; and decision integration uses a large language model (Llama 3.1 8B) to generate appropriate responses in real time—triggering an alert for a fall or continuing to follow if the person is merely moving.
This architecture turns a humanoid robot into a mobile caregiver that operates in any lighting condition, because mmWave radar is impervious to darkness, smoke, or dust. The robot can roam across rooms, following the person while continuously monitoring for falls—something static sensors cannot do.
Key Results
The decision module was tested in two scenarios: Fall (robot must detect a fall and issue an alert) and Move (robot must track and follow without false alarm). In both scenarios, the system achieved a 100% success rate in recognizing the correct state and extracting relevant numerical information (e.g., distance, velocity).
| Scenario | Success Rate | Semantic Similarity | Numerical Accuracy |
|---|---|---|---|
| Fall | 100% | 81.5% | 100% |
| Move | 100% | 71.5% | 99.0% |
The lower semantic similarity in the Move scenario reflects greater linguistic variation in normal follow responses, but all outputs remained semantically correct. The slight drop in numerical accuracy (99.0%) was due to a missing subgoal position in one instance. The system was also evaluated against several baseline methods under varied environmental conditions, consistently outperforming static radar approaches in terms of both detection reliability and false alarm rate.
How It Works
EM-Fall uses a frequency-modulated continuous-wave (FMCW) mmWave radar (e.g., TI IWR6843) that emits radio waves at 60–64 GHz and captures the reflections from a person’s body. The raw chirp data are processed into a 3D point cloud that encodes range, azimuth, elevation, and Doppler velocity of each scatterer.
Spatial state estimation fuses radar detections with the robot’s odometry to produce a consistent human target state (position, velocity, body orientation) even when the robot is moving. A Kalman filter handles occlusions and multipath ghost reflections, which are common pitfalls in static radar setups and often cause false positives.
Fall inference uses a lightweight temporal model (e.g., a short LSTM or 1D CNN) over the radar point cloud features to recognize the characteristic acceleration and body pose changes during a fall. Because the radar is co-located with the robot, the system can adapt to the robot’s own motion and maintain a stable view of the person.
Decision integration takes the inferred human state—falling, standing, walking—and feeds it to a prompt-conditioned Llama 3.1 8B LLM. The prompt defines a policy: if a fall is detected, send an alert; otherwise, command the robot to follow using a “people-as-planner” controller that computes safe following speeds and subgoals. This decision loop runs online and requires no pre-training for specific environments—the robot can be deployed zero-shot.
Why This Matters for Robotics
Fall detection is a critical need in elderly care, hospitals, and rehabilitation centers. Traditional solutions—wearable pendants, cameras, or static radar—all have limitations: pendants must be worn and are often forgotten, cameras invade privacy and fail in darkness, and static radars cannot follow a person into another room. EM-Fall solves all three problems by mounting the sensor on a mobile humanoid robot that accompanies the person.
For robot buyers and operations managers, this means a single robot can provide continuous, privacy-preserving monitoring across an entire home or facility. The use of an LLM for decision making also allows the system to be repurposed for other monitoring tasks—such as detecting wandering behavior or remembering medication schedules—without changing hardware. As humanoid robots become more affordable, integrating embodied radar sensing could become a standard feature for caregiving robots.
The same radar technology is equally applicable to warehouse robots for safety monitoring in low-visibility environments, or to used industrial robots repurposed for facility patrol.
Limitations and Open Questions
The evaluation was conducted on a single humanoid platform—it is unclear how the system generalizes to other robot morphologies (e.g., wheeled bases or quadrupeds). The LLM decision module, while effective in tested scenarios, may produce unpredictable outputs in unanticipated edge cases (e.g., a person lying down to rest). Additionally, the system’s reliance on mmWave radar means it cannot distinguish between a fall and a rapid sit-down motion without the temporal model—false positives in such borderline cases remain a concern. Real-world deployment also needs to address battery life constraints and safe navigation around furniture, which were outside the scope of this study.
Frequently Asked Questions
How does the system work in complete darkness? mmWave radar uses radio waves that are unaffected by lighting conditions, so the robot can detect falls equally well in pitch black or bright sunlight.
What humanoid robot was used in the experiments? The paper uses a Unitree G1 humanoid robot (or similar) equipped with a 60–64 GHz mmWave radar module mounted on the chest or head.
How accurate is the fall detection? The decision module achieves 100% success in distinguishing falls from normal movement in controlled tests, with high numeric accuracy for distance and velocity measurements.
Can the robot track a person through multiple rooms? Yes, because the radar is mounted on a mobile robot, it can follow the person from room to room, reacquiring the target even after temporary occlusion.
Conclusion
EM-Fall demonstrates that mounting mmWave radar on a humanoid robot and coupling it with an LLM decision engine enables robust, day-and-night fall detection with active following. This embodied approach overcomes the fundamental blind-spot problem of static sensors and could set a new standard for autonomous caregiving robots.