New Simulation Framework Stress-Tests AI Traffic Signals Against Real-World Incidents

Reinforcement learning has been hailed as a transformative tool for urban traffic signal control, promising networks of intersections that learn to slice delays and smooth congestion on their own. Yet almost all of the evidence behind that promise comes from simulations of calm, orderly traffic.

The framework, described in a study published in European Transport Research Review, is built on the widely used SUMO traffic simulator and models how real networks behave when something goes wrong. Drivers in the simulation become aware of an incident through radio broadcasts, variable message signs, online sources or direct observation, and then decide probabilistically whether to reroute based on expected travel-cost savings. Vehicles approaching a blockage decelerate according to stopping-sight-distance calculations drawn from road design guidelines, and make anticipatory lane changes as they near the disruption. The result is congestion that propagates across intersections the way it does in real cities, rather than staying neatly confined to a single link.

The research team, including Dang Viet Anh Nguyen, Carlos Lima Azevedo, Tomer Toledo and Filipe Rodrigues, framed the problem as a decentralized partially observable Markov decision process, in which each signalized intersection is an agent that sees only its local traffic conditions and selects signal phases to minimize congestion and delay. Incidents are injected online through SUMO’s Traffic Control Interface: special blocker vehicles are inserted at randomized positions and times to occupy one or more lanes, with durations drawn from configurable distributions. Multiple incidents can occur simultaneously, and the framework supports accidents, stalled vehicles, roadworks and even environmental failures such as fog-related speed reductions or signal blackouts.

To measure robustness, the authors went beyond the usual queue lengths and travel times. They proposed a suite of learning-oriented metrics: a Learning Stability Index that quantifies fluctuations in training performance, a Final Performance Deviation measure that captures how much of a method’s best performance is retained by the end of training, a Convergence Rate, an Area Under the Learning Curve that penalizes costly exploration, and relative degradation indices comparing nominal and perturbed conditions. The authors caution that these metrics capture system-level effects associated with safety risks, such as prolonged congestion and unstable adaptation, but do not directly measure collisions.

The experiments spanned a synthetic 4-by-4 intersection grid and four real-world German networks drawn from established SUMO benchmarks: a three-intersection corridor and an eight-intersection region in Cologne, plus a seven-intersection corridor and a 21-intersection region in Ingolstadt, all featuring actual road layouts and calibrated demand. Four reinforcement learning methods were benchmarked against rule-based baselines including fixed-time, random, max-pressure and greedy control: an independent Deep Q-Network, independent Proximal Policy Optimization, the pressure-based MPLight, and the hierarchical Feudal Multi-agent Actor-Critic method known as FMA2C.

The headline finding is that no single learning paradigm dominates. Under normal conditions, the reinforcement learning controllers generally beat traditional baselines, with the independent value-based method reaching 242.88 seconds of average travel time in the Ingolstadt Region against 295.22 seconds for fixed-time control. But when incidents struck, that advantage often evaporated. In the Cologne Corridor, MPLight’s travel time climbed to 178.74 seconds, falling behind both greedy control at 157.74 seconds and fixed-time at 116.94 seconds. On the synthetic grid, every learning method underperformed the simple greedy baseline during incidents, with the independent Q-learning agent’s travel time rising from 145.79 to 191.92 seconds while greedy control rose only from 145.41 to 160.78 seconds.

The robustness metrics made the instability stark. The independent value-based method’s Learning Stability Index on the grid spiked from 7.33 to 814.66 under incidents, and its relative area-under-curve degradation reached 24.06 percent. The policy-gradient method fared even worse in some settings, with a relative degradation of 417.55 percent in the Ingolstadt Corridor. Negative convergence rates signaled outright convergence failure in turbulent conditions. The hierarchical FMA2C method proved the most robust, achieving the lowest degradation in several networks, including just 9.38 percent in the Ingolstadt Region, but paid for that resilience with slow convergence and heavy sampling demands.

Perhaps the most counterintuitive result came from the generalization experiments, where policies trained in one scenario were tested in another without further training. Policies trained on ordinary, incident-free traffic often outperformed those trained directly on incident scenarios when later evaluated under disruptions. The authors attribute this to epistemic uncertainty: because incidents during training vary randomly in location, severity and timing, incident-trained agents can overfit to those specific disruptions and become brittle when facing unseen ones. Base-trained policies, by contrast, appear to capture more stable and generalizable traffic patterns that transfer reasonably well to moderate disruptions.

The transfer experiment, in which controllers pre-trained without incidents continued learning after incidents were introduced, reinforced the fragility of independent learning. On the grid network, the independent Q-learning method’s travel time jumped from roughly 150 seconds to nearly 400 seconds upon transfer, while the pressure-based and hierarchical methods saw milder spikes of about 60 and 25 seconds. In the large, irregular Ingolstadt Region, however, the hierarchy paid off: FMA2C stabilized around 303.07 seconds after transfer, beating greedy control at 334.42 seconds and continuing to improve, while the pressure-based method failed to converge at all in that heterogeneous setting.

The authors conclude that current state-of-the-art reinforcement learning signal controllers are not yet ready for reliable real-world deployment, lacking robustness to distribution shifts and struggling with out-of-distribution generalization. They also report that T-REX scales efficiently, with runtime growing roughly linearly with network size and sub-linearly with vehicle demand, and incident scenarios roughly doubling or tripling computation time. The framework’s main limitation is behavioral: parameters governing driver awareness and response to incident information are currently inferred or assumed in the absence of empirical datasets. The full codebase is publicly available on GitHub, and the team invites the research community to adopt T-REX as a standardized platform for robustness-oriented benchmarking before any of these learning controllers reaches a live intersection.

Subject of Research: An open-source simulation framework for benchmarking the robustness of reinforcement learning-based traffic signal control under incident scenarios.

Article Title: A framework for benchmarking traffic signal control robustness under incidents: comparative study of reinforcement learning-based methods

Article References: Nguyen, D. V. A., Lima Azevedo, C., Toledo, T., & Rodrigues, F. (2026). A framework for benchmarking traffic signal control robustness under incidents: comparative study of reinforcement learning-based methods. European Transport Research Review, 18(1), Article 85. https://doi.org/10.1186/s12544-026-00841-1

Image Credits: AI Generated

DOI: 10.1186/s12544-026-00841-1

Keywords: reinforcement learning, traffic signal control, T-REX, SUMO simulator, traffic incidents, robustness benchmarking, intelligent transportation, decentralized control, hierarchical reinforcement learning, traffic congestion, simulation framework, urban mobility