Drones have increasingly been deployed in wildfire response efforts. Human-piloted UAVs are used to detect hotspots, while others are equipped with tools to suppress flames. However, these systems are limited by their reliance on human operators, which reduces operational uptime and scalability. Recent advances in artificial intelligence (AI), particularly through the integration of unmanned aerial vehicles (UAVs) and deep learning models, have accelerated the development of more autonomous and effective wildfire management strategies.
Introduction
Despite these advancements, current unmanned aerial systems (UAS) face limitations in wildfire monitoring due to bandwidth constraints and the continued need for human oversight. To address these challenges, recent initiatives aim to establish hierarchical platforms of multiple UAVs for sustained fire coverage, develop low-computation real-time collaborative learning algorithms for onboard fire detection and mapping, and transmit final fire maps to relevant stakeholders.
In parallel, modeling efforts have focused on predicting fire spread under extreme conditions using probabilistic surrogate models, which approximate mathematical models since the outcome of interest cannot be easily computed. To reduce latency in rapidly evolving fire scenarios, large language model (LLM)-enhanced agents have been proposed for real-time decision-making in dynamic environments.
Multimodal LLMs have recently been integrated into UAV swarms to enable intelligent coordination and collaborative observation of wildfires. This approach could significantly improve situational awareness and response efficiency. Furthermore, swarm intelligence has shown promise in search-and-rescue operations, offering scalable solutions to locate individuals in disaster zones.
Replay and Normalization
Each agent stores its experience in a replay buffer, which includes observations, actions, clipped rewards, next obstacle states, and episode termination flags.
Critic updates follow a Twin Delayed Deep Deterministic Policy Gradient architecture, using clipped target Q-values and two critic networks. During each episode, Augmented Policy Exploration and Evaluation (APE2) is used to generate multiple action candidates with different Gaussian noise levels. The candidates are then evaluated by multiple critics.
The environment uses Welford’s online algorithm to compute the mean and variance of three key reward components: proximity, exploration, and energy. These components are normalized individually and then recombined using weighted coefficients. The proximity weight is 1.5, the exploration weight is 1.2, and the energy weight is the value specified by the original formula.
Training begins after a replay warm-up of 20,000 environment steps, with normalization starting after 2,000 collected transitions.
Policy and Candidate Selection
The actor network generates a base action vector for each drone, which serves as the foundation for candidate selection. To promote diversity, stochastic noise and fire-directed heuristics are applied at multiple levels—encouraging exploration when distant from fires and retreat when too close.
Each candidate is first scored based on its simulated immediate reward, incorporating proximity shaping, optimal bonus logic, and penalties for unsafe distances or stationary behavior.
To evaluate long-term value, each candidate is passed through the critic network, which averages Q-values across multiple critics. A small energy penalty is subtracted from each Q-value to discourage inefficient or erratic movement. Actor-side regularization is also applied to promote smooth control.
The candidate with the highest adjusted score is selected for execution. Candidate scores were computed using critic estimates and energy penalties, enabling drones to learn robust policies for fire tracking, exploration, and collision avoidance.
Discussion
Across all scenarios, training led to increased episode rewards, longer episode durations, reduced boundary collisions, and more frequent movement. Loss curves stabilized, and all scenario-specific metrics showed measurable improvements in drone behavior over the 1,000-episode training period.
Conclusion
These findings should be interpreted with caution. Improved navigation and fire tracking in simulation does not guarantee similar performance in real wildfire conditions. The simulated environment omits critical real-world factors such as wind, uneven terrain, sensor noise, and communication delays, all of which could influence drone behavior.
Moreover, the observed relationships between training components and performance improvements do not establish direct causation; no single element—reward shaping, curriculum design, or multi-agent coordination—can be identified as solely responsible. Instead, the results suggest that these components collectively support the learning process.
Future studies could explore more realistic fire dynamics through the incorporation of environmental disturbances, such as wind, or communication constraints. Testing alternative reward structures or comparing different reinforcement learning algorithms could also help determine which approaches are most effective for wildfire monitoring tasks. Expanding the number of scenarios and increasing the diversity of fire shapes would further strengthen the generalizability of the findings.
In summary, the results indicate that the trained agents developed safer and more structured navigation behaviors over the course of training. While these findings highlight the potential of reinforcement learning for wildfire monitoring, additional experiments and more complex simulations are needed to fully understand the strengths and limitations of this approach.
The study provides a foundation for future work aimed at improving autonomous wildfire-tracking systems and exploring broader applications in environmental monitoring. Together, these findings demonstrate the promise of deep reinforcement learning-driven UAV systems while highlighting the need for continued refinement in realistic, high-variability wildfire environments.
Frequently Asked Questions
What is the purpose of the multi-agent UAV system? The system is designed to support autonomous wildfire monitoring, fire tracking, exploration, and collision avoidance.
How are UAV experiences stored during training? Each agent stores observations, actions, clipped rewards, next obstacle states, and episode termination flags in a replay buffer.
How are action candidates selected? Candidates are evaluated using immediate simulated rewards, critic estimates, energy penalties, and actor-side regularization. The candidate with the highest adjusted score is executed.
What limitations affect the study’s conclusions? The simulation omits factors including wind, uneven terrain, sensor noise, and communication delays. Performance in simulation therefore does not guarantee equivalent performance in real wildfire conditions.