A simulation study has demonstrated that coupling adaptive deep reinforcement learning with model reference control can substantially improve speed regulation for battery electric vehicles navigating intricate urban traffic conditions. Engineering researchers Abdul Imran and R. Srinu Naik formulated the integrated control scheme, termed ADRL-MRC, to overcome the persistent trade-offs between dynamic adaptability and mathematical stability that frequently hinder automated vehicle speed controllers. In complex metropolitan transit environments, electric vehicles must constantly respond to fluctuating traffic densities, abrupt stop-and-go patterns, sharp road curves, and signalized intersections. Conventional control strategies, such as standard proportional-integral-derivative controllers and classical model reference adaptive systems, often struggle to deliver precise tracking performance when confronted with severe nonlinearities and rapid operational disturbances. By deploying an actor-critic deep reinforcement learning agent to generate dynamic parameter adjustments alongside a robust model reference tracking baseline, the researchers created a dual-layer closed-loop architecture designed to regulate the vehicle’s traction drive voltage. Evaluated across five standardized urban driving schedules in numerical simulations, the hybrid system significantly outperformed conventional benchmark controllers across multiple error metrics.
Urban driving exposes battery-operated vehicles to extreme variations in operational demand, where sudden acceleration and decelerating events directly impact battery state-of-charge, energy consumption rates, and propulsion drive stability. Recent academic investigations into intelligent transportation have explored advanced learning paradigms, including Deep Deterministic Policy Gradient and Soft Actor-Critic algorithms, for energy management and eco-driving velocity optimization. However, existing reinforcement learning controllers operating in isolation often encounter severe convergence difficulties, unpredictable policy oscillations, and a lack of formal stability guarantees when operating within highly nonlinear city transit environments. The researchers sought to bridge this gap by designing an architecture wherein reinforcement learning continuously adapts to external traffic conditions while model reference control preserves systemic stability.
To evaluate the proposed control methodology, the investigators modeled a complete electric vehicle traction drive encompassing a battery power pack, a step-down buck converter, a three-phase voltage source inverter, and a brushless direct current motor. In modern electric vehicle propulsion systems, brushless direct current machines are widely favored due to their elevated operational efficiency, compact physical dimensions, rapid transient response, and high power density. The stator dynamics of the machine were mathematically represented across a three-phase stationary reference frame, capturing phase resistances, phase inductances, and back-electromotive force profiles induced by the permanent magnet rotor. Electronic commutation is coordinated through Hall effect position sensors and switching logic circuits, replacing traditional mechanical commutators and brushes to direct three-phase excitation signals from the inverter into the stator coils. Torque generation arises directly from the electromagnetic interaction between the stator phase currents and the permanent magnet flux linkage. The power electronics subsystem regulates motor shaft velocity by modulating the buck converter duty cycle, which scales the input direct-current voltage delivered from the battery storage pack to the inverter stage.
The mechanical load and chassis behavior were structured using an all-terrain vehicle dynamic configuration to rigorously evaluate propulsion performance under demanding traction circumstances. While all-terrain vehicle platforms originate in off-road, industrial, logistics, and military applications characterized by rough surfaces and steep gradients, their parameterized physical equations effectively represent the rigorous dynamic demands encountered during aggressive metropolitan maneuvers. The vehicle’s longitudinal motion is governed by a force balance equating the total inertial acceleration to the motor tractive force developed at the wheel contact patches minus the cumulative resistance forces. These resisting components encompass rolling resistance generated by tire deformation, aerodynamic drag opposing frontal motion at elevated velocities, and road gradient resistance induced by terrain elevation changes. Tractive thrust generated at the road interface is calculated directly from the brushless motor mechanical output torque scaled by the transmission gear ratio, overall drivetrain mechanical efficiency, and effective tire rolling radius. Modeling these mechanical dynamics within the simulation framework ensured that realistic rotational inertia and road loads were reflected in the motor torque requirements.
The learning component of the controller is grounded in the formal mathematics of Markov decision processes, which define stochastic control problems through continuous state spaces, action vectors, transition probabilities, and discounted reward expectations. Standard tabular reinforcement learning algorithms store state-action values within discrete matrices, rendering them computationally intractable when applied to continuous, high-dimensional automotive propulsion systems. To circumvent this dimensionality bottleneck, deep reinforcement learning deploys deep neural networks as universal nonlinear function approximators that iteratively learn optimal operational policies directly from environmental interactions. The state observation vector supplied to the learning agent at each discrete operational time step incorporates five vital indicators: instantaneous speed tracking error, the time rate of change of tracking error, instantaneous vehicle speed, battery state-of-charge, and instantaneous longitudinal acceleration. This five-dimensional state formulation provides comprehensive operational visibility into chassis velocity, error trajectories, and internal energy storage reserves throughout transient driving maneuvers.
The integrated control law coordinates actions through a linear superposition of the fundamental model reference control signal and an adaptive compensation signal synthesized by the deep reinforcement learning policy network. Within this architecture, the model reference control module enforces tracking convergence by compelling actual vehicle velocity to trace the output of a predefined first-order stable reference model. The resulting model-following error serves as the core feedback driving the baseline control signal toward asymptotic stability. Concurrently, the parameterized actor network evaluates the five-dimensional state observation vector and outputs an adaptive correction that adjusts the buck converter duty cycle between zero and unity. By directly altering the effective output voltage applied across the propulsion drive, the controller precisely manages the electromagnetic torque and rotational velocity of the brushless direct current motor. The cumulative control action ensures that systematic tracking errors are attenuated rapidly even when the vehicle encounters severe disturbances or unpredictable speed variations.
To balance accurate reference speed tracking with powertrain longevity and energy conservation, the investigators engineered a composite instantaneous reward function. The formulation penalizes the squared tracking error, the squared difference in duty cycle adjustments between successive time steps, and the depletion of the battery state-of-charge through designated weighting coefficients. Simultaneously, penalizing energy depletion incentivizes the policy network to select smooth, energy-efficient operational trajectories during driving maneuvers. The agent was trained over one hundred successive episodes utilizing the Deep Deterministic Policy Gradient algorithm. During early training phases, the agent yielded low or negative cumulative returns as it actively explored the state-action space without prior system knowledge. Tracking performance improved markedly as policy parameters converged, with cumulative episodic rewards crossing thirty units around the sixty-fifth episode and stabilizing at elevated plateaus in final training cycles.
The validation framework assessed the developed ADRL-MRC architecture against traditional proportional-integral-derivative control and standalone model reference adaptive control across five distinct urban driving cycles, designated UC1 through UC5. These standardized driving schedules were modeled within MATLAB/Simulink 2023a to replicate a comprehensive range of real-world metropolitan operating conditions. Cycle UC1 represents low-speed urban transit spanning 195 seconds with a peak velocity of 50 kilometers per hour. Cycle UC2 models a mixed urban-arterial profile lasting 400 seconds with high-speed segments reaching 120 kilometers per hour. Cycle UC3 captures a 400-second mixed driving profile with an intermediate peak velocity of 90 kilometers per hour. Cycle UC4 constitutes a demanding composite highway-urban schedule lasting 506 seconds with peak speeds of 91.5 kilometers per hour, characterized by harsh acceleration and deceleration demands. Finally, cycle UC5 represents an intensive high-demand urban driving schedule lasting 660 seconds with frequent stop-and-go maneuvers and a maximum velocity of 70 kilometers per hour.
Quantitative error analyses demonstrated that the hybrid ADRL-MRC controller achieved superior tracking precision compared to both benchmark controllers across every evaluated driving schedule. Across all five driving cycles, ADRL-MRC achieved an overall mean root mean square error of 0.357 kilometers per hour, representing an error reduction of 55.7 percent compared to standard model reference control, which averaged 0.806 kilometers per hour, and an error reduction of 78.6 percent compared to the proportional-integral-derivative controller, which exhibited a mean root mean square error of 1.665 kilometers per hour. The mean absolute error dropped to 0.242 kilometers per hour under ADRL-MRC, outperforming the 0.580 kilometers per hour recorded for standard model reference control and 1.202 kilometers per hour recorded for proportional-integral-derivative control. Across individual drive cycles, ADRL-MRC maintained its lowest root mean square error of 0.296 kilometers per hour during cycle UC5 and its highest error of 0.528 kilometers per hour during the aggressive conditions of cycle UC4. Peak instantaneous tracking error for ADRL-MRC remained restricted between 0.702 and 1.673 kilometers per hour, whereas peak errors under proportional-integral-derivative control escalated up to 6.547 kilometers per hour. In normalized multi-metric radar assessments benchmarking all metrics against proportional-integral-derivative performance, ADRL-MRC occupied the minimal area, maintaining normalized values of roughly 0.24 for root mean square error, 0.20 for mean absolute error, 0.23 for maximum instantaneous error, and 0.20 for integral of absolute error.
Despite the substantial numerical gains achieved across all five simulation schedules, the study authors explicitly noted key technical boundaries regarding the real-world deployment of the control architecture. All reported performance metrics, parameter convergence curves, and comparative error reductions were derived entirely from software simulations implemented within the MATLAB/Simulink computational environment. As a consequence, the authors emphasized that these simulation findings substantiate algorithmic feasibility rather than confirmed operational efficacy in production vehicles. Future investigations are planned to transition the proposed control law toward real-time embedded hardware platforms and incorporate predictive traffic data feeds to enhance urban driving responsiveness.
Subject of Research: Adaptive deep reinforcement learning model reference control for battery electric vehicles
Article Title: Adaptive deep reinforcement learning-based model reference control for battery electric vehicles under dynamic urban driving conditions
Article References: Imran, A., & Naik, R. S. (2026). Adaptive deep reinforcement learning-based model reference control for battery electric vehicles under dynamic urban driving conditions. Discover Vehicles, 2(1), Article 51. https://doi.org/10.1007/s44465-026-00055-8
Image Credits: AI Generated
DOI: 10.1007/s44465-026-00055-8
Keywords: battery electric vehicles, deep reinforcement learning, model reference control, brushless DC motor, buck converter, speed tracking, urban driving cycles, MATLAB Simulink, state of charge, power electronics, closed loop control, dynamic traffic, tracking error, actor critic algorithms, vehicle dynamics, electronic commutation, longitudinal dynamics
News Source: Denise Maddox. (November 13, 2026). Adaptive Deep Reinforcement Learning Controller Improves Electric Vehicle Speed Tracking in Simulations. Scienmag.
