A goal-conditioned deep reinforcement learning framework enhanced with a Transformer-encoder and physics-informed rewards improves navigation stability and flight consistency for multirotor unmanned aerial vehicles navigating toward target destinations. Developed to overcome persistent challenges in autonomous aerial guidance, the system couples self-attention temporal modeling with physical flight principles to generate robust and dynamically consistent flight trajectories in simulation experiments.
Unmanned aerial vehicles are increasingly deployed across civilian and military sectors due to their operational mobility, design flexibility, and low energy consumption. Despite widespread adoption, conventional path-planning and navigation algorithms continue to suffer from heavy computational costs, limited environmental adaptability, and suboptimal trajectory generation, motivating the exploration of data-driven learning solutions.
Deep reinforcement learning offers a promising alternative by enabling unmanned aircraft to learn optimal navigation strategies directly from ongoing interactions with their operating environment. This approach is well suited for managing the continuous and high-dimensional action spaces that characterize multirotor flight dynamics, although standard deep reinforcement learning architectures frequently struggle to capture long-term temporal dependencies over extended trajectories.
To address these temporal modeling shortcomings, the proposed navigation framework introduces a combined architecture featuring a Transformer-encoder and a Long Short-Term Memory layer. Within this goal-conditioned design, the Transformer-encoder employs a self-attention mechanism to capture long-range dependencies, while the recurrent Long Short-Term Memory component preserves sequential temporal information across navigation episodes.
The architectural framework integrates two established probabilistic deep reinforcement learning algorithms: Proximal Policy Optimization and Advantage Actor-Critic. In comparative evaluations against their standalone baseline counterparts, the Transformer-enhanced models demonstrated marked improvements in validation stability, trajectory consistency, and overall trajectory robustness, with the most pronounced performance gains achieved by the enhanced Proximal Policy Optimization implementation.
Alongside the temporal network architecture, the researchers introduced a novel reward function tailored for continuous action spaces, combining multiple reward contributions with a physics-inspired term grounded in the Least Action Principle to encourage efficient and dynamically consistent flight. The resulting formulation successfully promoted smoother and more action-consistent vehicle behavior throughout the simulated navigation tasks.
To systematically evaluate the framework, the research team developed a multi-stage simulation pipeline that began with training on a quadrotor dynamic model, progressed through Software-in-the-Loop validation using PX4 and Gazebo, and concluded with Hardware-in-the-Loop deployment on a Raspberry Pi. This testing architecture facilitates rapid model prototyping while establishing a structured experimental pathway toward future real-world flight trials.
Subject of Research: Autonomous multirotor navigation using deep reinforcement learning
Article Title: Transformer-Enhanced Physics Informed Deep Reinforcement Learning for Quadrotor Navigation
Article References: Sopegno, L., Martini, S., Fagiolini, A., Livreri, P., Cirrincione, G., & Valavanis, K. P. (2026). Transformer-Enhanced Physics Informed Deep Reinforcement Learning for Quadrotor Navigation. Journal of Intelligent & Robotic Systems. https://doi.org/10.1007/s10846-026-02479-6
Image Credits: AI Generated
DOI: 10.1007/s10846-026-02479-6
Keywords: quadrotor navigation, deep reinforcement learning, unmanned aerial vehicles, transformer encoder, long short-term memory, proximal policy optimization, advantage actor-critic, least action principle, physics-informed learning, continuous action spaces, software-in-the-loop, hardware-in-the-loop, PX4 autopilot, Gazebo simulation, Raspberry Pi, trajectory consistency, multirotor flight
News Source: Katie Riggs. (November 18, 2026). Transformer Integration and Physics-Based Rewards Enhance Simulated Quadrotor Flight Navigation. Scienmag.



