A research team from Dalian Neusoft University of Information and China Railway Corporation has developed a hybrid deep reinforcement learning architecture designed to overcome the mechanical control limitations of six-wheeled rocker–bogie planetary exploration rovers. Operating autonomous mobility platforms across unstructured extraterrestrial terrains requires negotiating severe physical challenges, including nonlinear terramechanics, dynamic wheel-soil slip, chassis load shifts, and variable traction. Traditional robotic platforms rely on fixed-gain proportional–integral–derivative controllers, but these classical feedback structures assume approximately constant operating conditions. When rovers encounter abrupt transitions in soil sinkage, loose regolith, or uneven terrain, fixed parameters often induce significant tracking oscillations, transient overshoot, and excessive actuator strain. To address these operational hazards, the investigators constructed an online gain-scheduling supervisory architecture that combines classical control reliability with continuous reinforcement learning.
The novel control framework retains the transparent, deployable structure of classical proportional–integral–derivative loops while deploying two decoupled Deep Deterministic Policy Gradient actor–critic agents to continuously adapt feedback gains in real time. Rather than replacing the foundational control loops with an end-to-end neural network, the artificial intelligence agents function as deterministic meta-optimizers. One agent oversees the lateral steering subsystem, while the second agent independently regulates longitudinal velocity. This dual-agent separation prevents the destructive cross-coupling and policy oscillations that frequently arise when training high-dimensional single-agent networks across correlated action manifolds. The low-level proportional–integral–derivative loops execute at a high frequency of fifty hertz, ensuring that sharp terrain shocks and physical perturbations are immediately mitigated by the deterministic feedback law. Concurrently, the supervisory reinforcement learning agents evaluate the rover’s dynamic state every one hundred milliseconds, dynamically updating proportional, integral, and derivative gains to adapt to evolving environmental conditions.
To evaluate the system under high-fidelity physical dynamics, the researchers modeled a full-scale six-wheel rocker–bogie planetary exploration rover within a three-dimensional MATLAB Simscape Multibody environment. The calibrated rover configuration possesses an operational total mass of eighteen point five kilograms, an effective wheelbase of one point two zero meters, a track width of one point one zero meters, and rigid wheels with a nominal radius of zero point one five meters. The vehicle’s planar guidance relies on an Ackermann-equivalent bicycle representation paired with a Pure Pursuit path-tracking algorithm, which generates instantaneous steering curvature and forward velocity targets from discrete Cartesian waypoints. Meanwhile, the multibody simulation environment explicitly resolves the complex multi-joint suspension kinematics, actuator rotational damping, dynamic contact normal forces, and non-linear wheel-ground interactions under static and dynamic friction coefficients calibrated respectively to zero point eight zero and zero point five five.
A critical component of the proposed framework is a smoothness-shaped reward formulation engineered to guide policy convergence toward physically consistent control actions. Unregularized policy gradients often generate erratic gain corrections that trigger severe mechanical chatter and actuator saturation. To prevent these vulnerabilities, the composite reward functions penalize lateral tracking error, heading deviation, wheel slip ratio, and abrupt control variations. For the steering subsystem, the reward explicitly penalizes inter-joint mismatch variance across the steerable wheel assemblies to ensure synchronized turning. For the driving subsystem, the reward actively penalizes deviations from safe traction thresholds to suppress longitudinal wheel slip. The overall training formulation also incorporates a modest control-effort penalty, an early divergence penalty of negative twenty, and a sparse success bonus of positive ten, encouraging stable, energy-efficient navigation without aggressive control spikes.
State estimation for the real-time control pipeline is structured around a multi-rate error-state extended Kalman filter that fuses sensory feeds from an inertial measurement unit, wheel encoders, and visual-inertial odometry. Because parameters such as wheel-soil slip ratio and tracking error cannot be measured directly on planetary surfaces, the filter estimates the platform’s six-degree-of-freedom pose and calculates longitudinal slip by contrasting the circumferential speed of the wheel hubs with the fused vehicle longitudinal velocity. The resulting state vector delivers lateral error, heading error, their respective time derivatives, and dynamic slip indicators directly to the actor networks. Actor inputs are normalized using min-max scaling to bounded intervals, and the actor network outputs continuous action corrections through hyperbolic tangent activation functions before hard saturation operators constrain gains within stable, physically admissible boundaries.
Computational feasibility tests performed across representative embedded hardware platforms demonstrated that the dual-agent actor-critic architecture is well-suited for deployment on resource-constrained robotic systems. Benchmarking on an NVIDIA Jetson Nano powered by a quad-core ARM Cortex-A57 processor revealed an average dual-agent actor forward-pass latency of one point four five milliseconds. When integrated into the full gain-adaptation sequence—including three point two zero milliseconds for state estimation filtering, zero point one five milliseconds for Pure Pursuit kinematic projections, and zero point zero five milliseconds for gain allocation—the complete cycle required only four point eight five milliseconds. Evaluated against the one-hundred-millisecond reinforcement learning update period, this execution profile affords a computational margin of ninety-five point one five percent, leaving ample operational overhead for communication and low-level motor commutation.
The adaptive framework was subjected to rigorous comparative benchmarking against a conventional fixed-gain proportional–integral–derivative controller and a state-of-the-art Model Reference Adaptive Control baseline. To verify statistical validity under stochastic terramechanical conditions, the authors conducted fifty independent Monte Carlo simulation runs across randomly perturbed soil friction coefficients spanning from zero point three five to zero point seven five. The proposed hybrid framework achieved a root-mean-square tracking error of zero point zero four one meters, representing an eighty-three point three percent reduction compared to the fixed-gain baseline value of zero point two four six meters, and a seventy-three point one percent reduction compared to the model reference adaptive controller value of zero point one five two meters. Two-tailed Welch’s t-tests confirmed that these performance enhancements were statistically significant, yielding p-values well below ten to the negative sixth power.
Further quantitative evaluations assessed transient response dynamics and control efficiency across the Monte Carlo trials. Maximum lateral tracking error dropped from zero point four eight two meters under standard proportional–integral–derivative regulation to zero point zero nine eight meters with the proposed strategy. Settling time decreased from two point eight zero seconds in the baseline model to zero point nine five seconds in the adaptive framework, while transient overshoot fell from eighteen point four percent down to two point eight percent. Actuator wear and electrical expenditure were similarly curtailed: the cumulative control effort, computed as the integral of the squared control input, decreased significantly, demonstrating that the learning agent achieved superior tracking precision through subtle parameter modulation rather than brute-force actuator torque spikes.
To examine fault tolerance and disturbance rejection, the researchers subjected the simulated rover to extreme off-nominal operational scenarios. In an abrupt friction transition test, the terrain friction coefficients dropped sharply from nominal traction down to static and dynamic values of zero point three zero and zero point twenty, inducing sudden wheel slip. The adaptive controller swiftly dampened the transient error spike within ten seconds by modulating torque distribution to suppress dynamic slip. In a secondary challenge involving a sudden five-kilogram payload shift—simulating the operation or detachment of an auxiliary robotic climbing unit—the system rapidly recovered chassis stability despite altered wheel load distributions.
Comparative evaluations against heuristic optimization baselines, such as Genetic Algorithm-tuned controllers, further highlighted the necessity of real-time adaptability over static pre-tuning. The authors emphasized that while these simulation results decisively confirm the theoretical and computational efficacy of the hybrid deep reinforcement learning approach within multibody dynamic environments, hardware-in-the-loop validation and physical testing on actual rover hardware over natural regolith terrains remain essential future steps to bridge the sim-to-real transfer gap.
Subject of Research: Deep reinforcement learning adaptive control for six-wheeled rocker-bogie planetary rovers
Article Title: Deep reinforcement learning–driven model-based adaptive PID strategy for six-wheeled rocker–bogie mobile platform
Article References: Tian, H., Tu, Y., & Feng, H. (2026). Deep reinforcement learning–driven model-based adaptive PID strategy for six-wheeled rocker–bogie mobile platform. Frontiers in Neurorobotics, 20, Article 1905372. https://doi.org/10.3389/fnbot.2026.1905372
Image Credits: AI Generated
DOI: 10.3389/fnbot.2026.1905372
Keywords: planetary rovers, deep reinforcement learning, rocker-bogie suspension, adaptive control, proportional integral derivative, robotics, multibody dynamics, terramechanics, Deep, reinforcement, learning, driven
News Source: Denise Maddox. (November 7, 2026). Deep Reinforcement Learning Enhances Adaptive Steering and Drive Control in Planetary Rovers. Scienmag.
