AI assurance: Safe artificial intelligence for autonomous driving

Artificial intelligence (AI) is of vital importance for autonomous and highly automated driving, because it’s the job of AI-based systems to make sure that autonomous vehicles can cope with complex urban traffic situations. Autonomous vehicles must be able to perceive their environment and respond just right. The systems need to identify pedestrians at all times, for example, and initiate the correct response without fail. Still, the use of AI in autonomous vehicles raises a question: “Is AI demonstrably safe enough for use in safety-critical applications like road traffic?” After all, it’s not always transparent how AI systems reach their conclusions, and even minor changes to the input data or other elements can lead to a completely different result. In road traffic, however, the decisions made by the AI must be clear-cut and easy to verify so that functional safety is guaranteed at all times.

Proving the safety of AI systems must be possible

That’s why the “KI-Absicherung” project for AI assurance, an initiative by the German Association of the Automotive Industry (VDA), has defined its goal of making the safety of in-car AI systems verifiable. To this end, the project partners are developing a stringent, verifiable chain of arguments for the assurance of AI functions in highly automated vehicles. In total, 28 project partners are researching solutions to this challenge. The Fraunhofer Institute for Cognitive Systems IKS is also on board. The project was initiated as part of the VDA’s flagship initiative “Autonomous and Connected Driving” and is funded by the German Federal Ministry for Economic Affairs and Energy (BMWi). It is also part of the German federal government’s AI strategy.

Example: Identifying pedestrians — systematic AI assurance

The companies are working on five subprojects that have one overall objective: the assurance of AI functions in autonomous vehicles.

  • Subproject 1: AI function
  • Subproject 2: Synthetic data
  • Subproject 3: Methods and measures
  • Subproject 4: Assurance strategy
  • Subproject 5: Project management

The primary use case here is the reliable identification of pedestrians in road traffic. As such, the first subproject involves developing algorithms for identifying pedestrians. In the second subproject, synthetic data is being generated to map the many possible scenarios in which pedestrians can appear in road traffic. This synthetic data can then be used to produce data sets for training and tests, including corner cases — that is, cases where the usual operating parameters don’t apply. The data subsequently serves as the foundation for subproject 3, where researchers are building a toolkit with methods and measures, which they also assess for their safety.

Fraunhofer IKS is contributing its expertise to subproject 4, “Assurance strategy”, which applies and assesses the solutions developed in the preceding subprojects. The objective of this subproject is to develop a methodology that verifies whether the AI can meet the safety requirements and, if so, under what conditions. If the safety requirements cannot be met fully at all times, it is particularly important to have a transparent representation of the remaining risks. These risks, which take both technical and ethical aspects into account, are then incorporated in the reasoning structure. In addition, Fraunhofer IKS is taking the lead in developing a suitable schedule for evaluating the safety requirements and their reasoning structure. Overall, the first four subprojects aim to create a systematic assurance strategy for AI functions.

Lastly, the fifth subproject of the “KI-Absicherung” project engages in an exchange with standardization bodies and certification authorities. The objective here is to reach a common industry consensus as to how the assurance of artificial intelligence can be achieved. With these five elements, the project is working on a crucial prerequisite for autonomous driving in Germany.

More information


Autonomous Driving

Will the automobiles of the future drive autonomously? This vision of the future will become reality only when autonomous driving is safe. With this in mind, Fraunhofer IKS works on adaptive automobile software architectures.


Artificial Intelligence

The goal of Fraunhofer IKS is to take advantage of machine learning technologies in order to design a future that is safe. AI technologies must be 100 percent dependable, especially when it comes to safety-critical applications.


Safety Engineering

The electronics inside vehicles and machinery are growing increasingly complex. It takes more advanced safety mechanisms to tame this complexity. This is why safety engineering is a focal point of research at Fraunhofer IKS.