RASA-Drive Diagnoses Long-Tail Safety Vulnerabilities in Scoring-Based Autonomous Driving

Safety-critical performance collapse in scoring-based autonomous driving systems concentrates within a small cluster of camera-stream perturbation cells rather than spreading uniformly across sensor disruptions. Investigating why standard average corruption metrics fail to highlight the specific planning tokens most vulnerable to operational breakdown, the newly introduced RASA-Drive evaluation framework examines the diagnostic behavior of trajectory scoring models. Using the official frozen SparseDriveV2 motion planner evaluated across the full-navtest split of the NAVSIM v2 benchmark, the investigation established a paired token-level failure atlas designed to track whether candidate-score geometry can successfully rank high-clean-to-collapse degradation events without altering the system’s selected driving trajectory.

The diagnostic protocol subjected the vision-based planning architecture to an organized battery of synthetic camera-stream corruptions structured around explicit score thresholds, deterministic manifests, and group-isolated evaluations. The experimental stress tests incorporated complete camera dropout, frame delay, visual occlusion, and photographic degradations comprising blur, exposure shifts, and noise patterns, with each disturbance systematically evaluated across three distinct severity levels. By isolating perturbation conditions within deterministic test manifests, the protocol isolated genuine vulnerability boundaries from stochastic evaluation noise, demonstrating that overall planner resilience cannot be inferred solely from aggregate degradation scores when individual planning tokens experience sharp localized failures.

To detect emerging trajectory hazards under finite computational budgets, the researchers evaluated a logistic risk monitor driven by the frozen geometry of candidate trajectory scores. Across token-level, scene-level, and log-group analytical benchmarks, the geometry-based logistic monitor consistently improved failure risk ranking when benchmarked against a baseline metadata prior informed by known corruption patterns. However, the internal geometry of candidate trajectory scores failed to sustain this predictive advantage when confronted with held-out temporal disruptions, exposing a sharp boundary in the monitor’s operational capabilities.

The statistical evaluation on held-out frame delay perturbations clearly delineated the limits of static score monitoring. When evaluated exclusively on candidate geometry, the logistic monitor achieved a receiver operating characteristic area under the curve of 0.493, with a 95 percent confidence interval spanning 0.466 to 0.522, representing performance no better than random guessing. Combining candidate geometry with the metadata prior increased the area under the curve to 0.572, with a 95 percent confidence interval between 0.548 and 0.597. Nevertheless, both configurations underperformed the standalone known-corruption metadata prior, which reached an area under the curve of 0.615 with a 95 percent confidence interval from 0.601 to 0.630, confirming that scorer confidence offers insufficient diagnostic traction when inputs suffer from temporal latency.

To verify the generalizability of these geometric signals across benchmark generations, the authors conducted a separate audit on the same SparseDrive architecture family utilizing the NAVSIM v1 benchmark. This audit confirmed that the geometric signal produced reproducible diagnostic tracking for visual occlusion events. However, cross-condition transfer degraded substantially under camera dropout scenarios, and the frame delay failure boundary persisted unresolved. This concrete negative outcome demonstrates that static candidate-score geometry cannot directly register temporal misalignment, establishing an inherent architectural ceiling for monitoring systems that rely solely on frozen candidate trajectory rankings without explicit temporal modeling.

In detailing the practical scope of the methodology, the study frames RASA-Drive as a reproducible diagnostic protocol tailored for finite-budget risk ranking in frozen, scoring-based autonomous driving systems. The authors explicitly emphasize that the framework provides modality-specific diagnostic utility rather than universal robustness transfer across all sensor impairments. Furthermore, the findings are bounded by careful scope restrictions: the protocol does not claim to deliver calibrated failure probabilities, establish closed-loop fallback benefits during active driving, or guarantee generalization to differing autonomous vehicle architectures.

Subject of Research: Autonomous driving safety diagnostics

Article Title: RASA-Drive: from average robustness to long-tail safety diagnosis in scoring-based end-to-end autonomous driving

Article References: Guo, J., Geng, G., & Sun, L. (2026). RASA-Drive: from average robustness to long-tail safety diagnosis in scoring-based end-to-end autonomous driving. Applied Intelligence, 56(16), Article 540. https://doi.org/10.1007/s10489-026-07556-9

Image Credits: AI Generated

DOI: 10.1007/s10489-026-07556-9

Keywords: autonomous driving, safety diagnosis, SparseDriveV2, NAVSIM benchmark, candidate-score geometry, camera perturbations, risk ranking, end-to-end planning, RASA-Drive, average, robustness, long-tail