Corner Case Generation and Analysis for Safety Assessment of Autonomous Vehicles
Abstract
Testing and evaluation is a crucial step in the development and deployment of connected and automated vehicles (CAVs). To comprehensively evaluate the performance of CAVs, it is necessary to test the CAVs in safety-critical scenarios, which rarely happen in a naturalistic driving environment. Therefore, how to purposely and systematically generate these corner cases becomes an important problem. Most existing studies focus on generating adversarial examples for perception systems of CAVs, whereas limited efforts have been put into decision-making systems, which is the highlight of this paper. As the CAVs need to interact with numerous background vehicles (BVs) for a long duration, variables that define the corner cases are usually high-dimensional, which makes the generation a challenging problem. In this paper, a unified framework is proposed to generate corner cases for decision-making systems. To address the challenge brought by high dimensionality, the driving environment is formulated based on the Markov decision process, and the deep reinforcement learning techniques are applied to learn the behavior policy of BVs. With the learned policy, BVs behave and interact with the CAVs more aggressively, resulting in more corner cases. To further analyze the generated corner cases, the techniques of feature extraction and clustering are utilized. By selecting representative cases of each cluster and outliers, the valuable corner cases can be identified from all generated corner cases. Simulation results of a highway driving environment show that the proposed methods can effectively generate and identify the valuable corner cases.
Get full access to this article
View all access and purchase options for this article.
References
1. Kalra N., Paddock S. M. Driving to Safety: How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Reliability? Transportation Research Part A: Policy and Practice, Vol. 94, 2016, pp. 182–193.
2. Li L., Huang W.-L., Liu Y., Zheng N.-N., Wang F.-Y. Intelligence Testing for Autonomous Vehicles: A New Approach. IEEE Transactions on Intelligent Vehicles, Vol. 1, No. 2, 2016, pp. 158–166.
3. Li L., Lin Y.-L., Zheng N.-N., Wang F.-Y., Liu Y., Cao D., Wang K., Huang W.-L. Artificial Intelligence Test: A Case Study of Intelligent Vehicles. Artificial Intelligence Review, Vol. 50, No. 3, 2018, pp. 441–465.
4. Koren M., Alsaif S., Lee R., Kochenderfer M. J. Adaptive Stress Testing for Autonomous Vehicles. Proc., 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China, IEEE, New York, 2018, pp. 1–7.
5. Cui L., Hu J., Park B. B., Bujanovic P. Development of a Simulation Platform for Safety Impact Analysis Considering Vehicle Dynamics, Sensor Errors, and Communication Latencies: Assessing Cooperative Adaptive Cruise Control under Cyber Attack. Transportation Research Part C: Emerging Technologies, Vol. 97, 2018, pp. 1–22.
6. Thorn E., Kimmel S. C., Chaka M., Hamilton B. A. A Framework for Automated Driving System Testable Cases and Scenarios. Technical Report. United States Department of Transportation, National Highway Traffic Safety Administration, Washington, D.C., 2018.
7. Li L., Wang X., Wang K., Lin Y., Xin J., Chen L., Xu L., Tian B., Ai Y., Wang J., Cao D. Parallel Testing of Vehicle Intelligence via Virtual-Real Interaction. Science Robotics, Vol. 4, No. 28, 2019, Article eaaw4106.
8. Wang P., Chen Y., Wang C., Liu F., Hu J., Van N. N. Development and Verification of Cooperative Adaptive Cruise Control via LTE-V. IET Intelligent Transport Systems, Vol. 13, No. 6, 2019, pp. 991–1000.
9. Liu P., Xu Z., Zhao X. Road Tests of Self-Driving Vehicles: Affective and Cognitive Pathways in Acceptance Formation. Transportation Research Part A: Policy and Practice, Vol. 124, 2019, pp. 354–369.
10. Liu P., Yang R., Xu Z. How Safe Is Safe Enough for Selfdriving Vehicles? Risk Analysis, Vol. 39, No. 2, 2019, pp. 315–325.
11. Li L., Zheng N., Wang F.-Y. A Theoretical Foundation of Intelligence Testing and Its Application for Intelligent Vehicles. IEEE Transactions on Intelligent Transportation Systems, 2020, pp. 1–10.
12. Feng S., Yan X., Sun H., Feng Y., Liu H. X. Intelligent Driving Intelligence Test for Autonomous Vehicles with Naturalistic and Adversarial Environment. Nature Communications, Vol. 12, 2021, p. 748. https://doi.org/10.1038/s41467-021-21007-8.
13. Yan X., Feng S., Sun H., Liu H. X. Distributionally Consistent Simulation of Naturalistic Driving Environment for Autonomous Vehicle Testing. arXiv Preprint arXiv:2101.02828, 2021.
14. Chen S.-T., Cornelius C., Martin J., Chau D. H. Robust Physical Adversarial Attack on Faster R-CNN Object Detector. arXiv Preprint arXiv:1804.05810, Vol. 2, No. 3, 2018, p. 4.
15. Evtimov I., Eykholt K., Fernandes E., Kohno T., Li B., Prakash A., Rahmati A., Song D. Robust Physical World Attacks on Machine Learning Models. arXiv Preprint arXiv:1707.08945, Vol. 2, No. 3, 2017, p. 4.
16. Eykholt K., Evtimov I., Fernandes E., Li B., Rahmati A., Xiao C., Prakash A., Kohno T., Song D. Robust Physical-World Attacks on Deep Learning Visual Classification. Proc., IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, IEEE, New York, 2018, pp. 1625–1634.
17. Xie C., Wang J., Zhang Z., Zhou Y., Xie L., Yuille A. Adversarial Examples for Semantic Segmentation and Object Detection. Proc., IEEE International Conference on Computer Vision, Venice, Italy, IEEE, New York, 2017, pp. 1369–1378.
18. Ding W., Xu M., Zhao D. Learning to Collide: An Adaptive Safety-Critical Scenarios Generating Method. arXiv Preprint arXiv:2003.01197, 2020.
19. Treiber M., Hennecke A., Helbing D. Congested Traffic States in Empirical Observations and Microscopic Simulations. Physical Review E, Vol. 62, No. 2, 2000, p. 1805.
20. Treiber M., Helbing D. MOBIL: General Lane-Changing Model for Car-Following Models. Disponıvel Acesso Dezembro, 2016.
21. Krauß S., Wagner P., Gawron C. Metastable States in a Microscopic Model of Traffic Flow. Physical Review E, Vol. 55, No. 5, 1997, p. 5597.
22. Bareiss D., van den Berg J. Generalized Reciprocal Collision Avoidance. The International Journal of Robotics Research, Vol. 34, No. 12, 2015, pp. 1501–1514.
23. Weng B., Rao S. J., Deosthale E., Schnelle S., Barickman F. Model Predictive Instantaneous Safety Metric for Evaluation of Automated Driving Systems. Proc., 2020 IEEE Intelligent Vehicles Symposium (IV), Las Vegas, NV, IEEE, New York, 2020, pp. 1899–1906.
24. Mnih V., Kavukcuoglu K., Silver D., Rusu A. A., Veness J., Bellemare M. G., Graves A., Riedmiller M., Fidjeland A. K., Ostrovski G., Petersen S. Human-Level Control through Deep Reinforcement Learning. Nature, Vol. 518, No. 7540, 2015, pp. 529–533.
25. Wold S., Esbensen K., Geladi P. Principal Component Analysis. Chemometrics and Intelligent Laboratory Systems, Vol. 2, No. 1–3, 1987, pp. 37–52.
26. Alsabti K., Ranka S., Singh V. An Efficient K-Means Clustering Algorithm. Electrical Engineering and Computer Science, Vol. 43, 1997.
27. Ester M., Kriegel H.-P., Sander J., Xu X. Density-Based Spatial Clustering of Applications with Noise. Proc., International Conference on Knowledge Discovery and Data Mining, Portland, OR, Vol. 240, 1996, p. 6.
28. Goodfellow I. J., Shlens J., Szegedy C. Explaining and Harnessing Adversarial Examples. arXiv Preprint arXiv:1412.6572, 2014.
29. Yuan X., He P., Zhu Q., Li X. Adversarial Examples: Attacks and Defenses for Deep Learning. IEEE Transactions on Neural Networks and Learning Systems, Vol. 30, No. 9, 2019, pp. 2805–2824.
30. Buckner C. Understanding Adversarial Examples Requires a Theory of Artefacts for Deep Learning. Nature Machine Intelligence, Vol. 2, No. 12, 2020, pp. 731–736.
31. Szegedy C., Zaremba W., Sutskever I., Bruna J., Erhan D., Goodfellow I., Fergus R. Intriguing Properties of Neural Networks. arXiv Preprint arXiv:1312.6199, 2013.
32. Kurakin A., Goodfellow I., Bengio S. Adversarial Examples in the Physical World. arXiv Preprint arXiv:1607.02533, 2017.
33. Lu J., Sibai H., Fabry E., Forsyth D. No Need to Worry about Adversarial Examples in Object Detection in Autonomous Vehicles. arXiv Preprint arXiv:1707.03501, 2017.
34. Lu J., Sibai H., Fabry E., Forsyth D. Standard Detectors Aren’t (Currently) Fooled by Physical Adversarial Stop Signs. arXiv Preprint arXiv:1710.03337, 2017.
35. Ma W.-H., Peng H. A Worst-Case Evaluation Method for Dynamic Systems. ASME Journal of Dynamic Systems, Measurement, and Control, Vol. 121, No. 2, 1999, pp. 191–199.
36. Zhao D., Lam H., Peng H., Bao S., LeBlanc D. J., Nobukawa K., Pan C. S. Accelerated Evaluation of Automated Vehicles Safety in Lane-Change Scenarios Based on Importance Sampling Techniques. IEEE Transactions on Intelligent Transportation Systems, Vol. 18, No. 3, 2016, pp. 595–607.
37. Zhao D., Huang X., Peng H., Lam H., LeBlanc D. J. Accelerated Evaluation of Automated Vehicles in Car-Following Maneuvers. IEEE Transactions on Intelligent Transportation Systems, Vol. 19, No. 3, 2017, pp. 733–744.
38. Feng S., Feng Y., Yu C., Zhang Y., Liu H. X. Testing Scenario Library Generation for Connected and Automated Vehicles, Part I: Methodology. IEEE Transactions on Intelligent Transportation Systems, Vol. 22, No. 3, 2020, pp. 1573–1582.
39. Feng S., Feng Y., Sun H., Bao S., Zhang Y., Liu H. X. Testing Scenario Library Generation for Connected and Automated Vehicles, Part II: Case Studies. IEEE Transactions on Intelligent Transportation Systems, 2020, pp. 1–13.
40. Feng S., Feng Y., Sun H., Zhang Y., Liu H. X. Testing Scenario Library Generation for Connected and Automated Vehicles: An Adaptive Framework. IEEE Transactions on Intelligent Transportation Systems, 2020, pp. 1–10. https://doi.org/10.1109/TITS.2020.3023668.
41. Feng S., Feng Y., Yan X., Shen S., Xu S., Liu H. X. Safety Assessment of Highly Automated Driving Systems in Test Tracks: A New Framework. Accident Analysis & Prevention, Vol. 144, 2020, Article 105664.
42. Akagi Y., Kato R., Kitajima S., Antona-Makoshi J., Uchida N. A Risk-Index Based Sampling Method to Generate Scenarios for the Evaluation of Automated Driving Vehicle Safety. Proc., 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand, IEEE, New York, 2019, pp. 667–672.
43. O’Kelly M., Sinha A., Namkoong H., Tedrake R., Duchi J. C. Scalable End-to-End Autonomous Vehicle Testing via Rare-Event Simulation. Advances in Neural Information Processing Systems, Vol. 31, 2018, pp. 9827–9838.
44. Karunakaran D., Worrall S., Nebot E. Efficient Statistical Validation with Edge Cases to Evaluate Highly Automated Vehicles. arXiv Preprint arXiv:2003.01886, 2020.
45. Shalev-Shwartz S., Shammah S., Shashua A. On a Formal Model of Safe and Scalable Self-Driving Cars. arXiv Preprint arXiv:1708.06374, 2017.
46. Ulbrich S., Menzel T., Reschka A., Schuldt F., Maurer M. Defining and Substantiating the Terms Scene, Situation, and Scenario for Automated Driving. Proc., 2015 IEEE 18th International Conference on Intelligent Transportation Systems, Gran Canaria, Spain, IEEE, New York, 2015, pp. 982–988.
47. Tai L., Paolo G., Liu M. Virtual-to-Real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigation. Proc., 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, IEEE, New York, 2017, pp. 31–36.
48. Sutton R. S., Barto A. G. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA, 2018.
49. Mnih V., Kavukcuoglu K., Silver D., Graves A., Antonoglou I., Wierstra D., Riedmiller M. Playing Atari with Deep Reinforcement Learning. arXiv Preprint arXiv:1312.5602, 2013.
50. Van Hasselt H., Guez A., Silver D. Deep Reinforcement Learning with Double Q-Learning. arXiv Preprint arXiv:1509.06461, 2015.
51. Hessel M., Modayil J., Van Hasselt H., Schaul T., Ostrovski G., Dabney W., Horgan D., Piot B., Azar M., Silver D. Rainbow: Combining Improvements in Deep Reinforcement Learning. arXiv Preprint arXiv:1710.02298, 2017.
52. Wang Z., Schaul T., Hessel M., Hasselt H., Lanctot M., Freitas N. Dueling Network Architectures for Deep Reinforcement Learning. Proc., International Conference on Machine Learning, PMLR, New York, NY, 2016, pp. 1995–2003.
53. Lillicrap T. P., Hunt J. J., Pritzel A., Heess N., Erez T., Tassa Y., Silver D., Wierstra D. Continuous Control with Deep Reinforcement Learning. arXiv Preprint arXiv:1509.02971, 2015.
54. Ference J. J. The Integrated Vehicle-Based Safety Systems Initiative. National Highway Traffic Safety Administration, Washington, D.C., 2006.
55. Leurent E. An Environment for Autonomous Driving Decision-Making. 2018. https://github.com/eleurent/highway-env.
56. Brockman G., Cheung V., Pettersson L., Schneider J., Schulman J., Tang J., Zaremba W. OpenAI Gym. arXiv Preprint arXiv:1606.01540, 2016.
57. Dosovitskiy A., Ros G., Codevilla F., Lopez A., Koltun V. CARLA: An Open Urban Driving Simulator. arXiv Preprint arXiv:1711.03938, 2017.
Cite article
Cite article
Cite article
OR
Download to reference manager
If you have citation software installed, you can download article citation data to the citation manager of your choice
Information, rights and permissions
Information
Published In
Article first published online: July 2, 2021
Issue published: November 2021
Authors
Author Contributions
The authors confirm contribution to the paper as follows: study concept and design: Haowei Sun, Shuo Feng, and Henry Liu; data collection: Haowei Sun, Xintao Yan; analysis and interpretation of results: Haowei Sun, Shuo Feng, and Henry Liu; draft manuscript preparation: Haowei Sun, Shuo Feng, and Henry Liu. All authors reviewed the results and approved the final version of the manuscript.
Metrics and citations
Metrics
Journals metrics
This article was published in Transportation Research Record: Journal of the Transportation Research Board.
VIEW ALL JOURNAL METRICSArticle usage*
Total views and downloads: 608
*Article usage tracking started in December 2016
Altmetric
See the impact this article is making through the number of times it’s been read, and the Altmetric Score.
Learn more about the Altmetric Scores
Articles citing this one
Receive email alerts when this article is cited
Web of Science: 0
Crossref: 2
- Dense reinforcement learning for safety validation of autonomous vehic...
- Novel Test Scenario Generation Technology for Performance Evaluation o...
Figures and tables
Figures & Media
Tables
View Options
Get access
Access options
If you have access to journal content via a personal subscription, university, library, employer or society, select from the options below:
loading institutional access options
Alternatively, view purchase options below:
Purchase 24 hour online access to view and download content.
Access journal content via a DeepDyve subscription or find out more about this option.
