Interactive Adversarial Scenario Generation for Autonomous Driving: A Continual Learning Framework with Safety Constraints

Jialin Fan^1,2, Ying Ni^1,2, Yuhang Chen^1,2, Siying Li^1,2, Jie Sun^1,2, and Jian Sun^1,2

¹ Department of Transportation Engineering, Tongji University, Shanghai, China

² Key Laboratory of Road and Traffic Engineering, Ministry of Education, Shanghai, China

Safety-critical scenarios are of significant value for testing and validating autonomous vehicles (AVs). However, their long-tailed distributions in real-world environments make efficient data collection challenging. Data-driven methods for targeted generation of challenging testing scenarios offer a promising solution, but adversarial approaches that solely focus on maximizing adversarial interactions often lead to inevitable collisions, leaving no space for the AV to make decisions. To address this challenge, we propose Constrained-Adversarial Policy Optimization (CAPO), an interactive scenario generation method that incorporates adversarial rationality. CAPO is built on a two-phase continual learning framework. In Phase I, multi-agent reinforcement learning (MARL) with safety-constraint function (SCF) is utilized to train agents to interact safely and accomplish driving tasks. In the Phase II, an AV expert is introduced to scenarios which are generated by adversarial agents considering the AV’s minimum safety constraint. These generated scenarios are adversarial yet solvable for the AV expert. Through open-loop and closed-loop tests, CAPO demonstrates its ability to produce more solvable and safety-critical scenarios, while significantly reducing unrealistic adversarial cases and unavoidable collisions.