Continuous Safety Assurance for AI-based Driving Functions

The use of AI often enables the realization of new driving functions in the first place. However, deploying AI in safety-critical contexts introduces risks that are difficult to predict.

Fraunhofer IKS offers a coordinated portfolio of solutions to identify and mitigate these risks to an acceptable level.
 

In detail, this portfolio includes the following development-phase measures:

• Definition of acceptable residual risk and acceptance criteria: We support you in defining acceptable residual risk (e.g., using MEM, ALARP, GAMAB, or positive risk balance) and in breaking it down into measurable acceptance criteria for individual components and ML models.
• Definition of the Operational Design Domain (ODD): Define ODDs based on the latest standards and automatically link your ODD definitions to downstream safety analyses and V&V activities.
• Selection of relevant safety-related properties and metrics: Develop a traceable concept for how safety requirements can be met through a concrete combination of safety-relevant metrics.
• Safe ML architectures and monitors: Use proven ML architectures to save time in model selection and automatically derive runtime monitors from your safety argumentation.
• Efficient V&V: Use methods for targeted evidence collection and build empirically robust safety arguments. Fraunhofer IKS provides methods to integrate performance data – such as object detection accuracy, trajectory planning precision, localization accuracy, and collision rate – into a safety argumentation framework.
• Regulatory and standards compliance: Use our Assurance Framework for AI High-Risk Systems to achieve compliance with standards such as ISO 21448, ISO 8800, or the EU AI Act.
• LLMs & Agents: Want to take your safety engineering to the next level? We support the safe use of LLMs and agents in your development processes, e.g., for HARA or requirements engineering.

Contact us now

After all safety engineering steps have been performed to bring a safe AI-based product to market, it can be assumed that the system’s residual risk is acceptably low.

However, the obligation to ensure safety does not end there. Several standards and regulations, such as ISO 21448, require runtime monitoring to detect previously unknown risks in the field and take appropriate countermeasures.

Fraunhofer IKS supports companies in efficiently and safely meeting these requirements through the establishment of a Continuous Safety Engineering process (also known as Dynamic Safety). 
 

This process addresses the following aspects:

• Automated monitoring: By automatically generating monitoring components from a formalized safety argumentation, all uncertainty-related assumptions can be easily monitored in the field.
• Early risk mitigation: With multi-layered monitoring structures, deviations from expected behavior can be detected early through abnormal metric values, enabling proactive mitigation before incidents occur.
• Efficient root cause analysis: Automated storage of relevant data and feedback to development systems allows efficient root cause identification and rapid design of appropriate countermeasures.
• Safe updates: We support you in designing solutions to provide fast and safe over-the-air updates for vehicle fleets.

Contact us now

Building on the development-phase measures and runtime assurance, Fraunhofer IKS supports the creation of evidence-based, continuous safety cases that link requirements, V&V results, and runtime data consistently throughout the entire lifecycle. The resulting evidence-based safety arguments for AI-based functions align with ISO PAS 8800, SOTIF, and ISO 26262.

At the core lies a structured safety case architecture (e.g., GSN) that makes evidence, assumptions, and counterarguments (defeaters) transparent, while quantifying and propagating confidence from evidence to claim. We model risk, complexity, and uncertainties causally (triggering conditions, influencing factors, insufficiencies) and use these models for targeted evidence planning.

Dynamic safety cases integrate runtime evidence and safety performance indicators for true continuous assurance. Modularity and reusability are ensured via formalized assurance contracts. The result: standards-compliant, decision-ready arguments, supported by prototypes, tools, training, and reviews – proven in automotive, rail, and aviation domains.

This approach closes the gap between verification & validation (V&V) and safety demonstration – iteratively across the lifecycle – accelerating approvals and establishing a robust foundation for regulatory certification.

Causal risk models and evidence strategy (ISO PAS 8800 / ISO 21448 SOTIF)

We link causal risk models with a risk-based, traceable evidence strategy for AI functions – compliant with ISO PAS 8800 and ISO 21448 (SOTIF). The result: focused, auditable proofs with clear traceability from system to function level, iteratively over the lifecycle.

Consulting topics include:

• Development of causal risk models connecting ML influencing factors and triggering conditions to AI-level insufficiencies and resulting system-level risk

• Model-based prioritization of scenarios, measures, and tests

• Derivation of concrete data, model, and test evidence over the lifecycle (per ISO PAS 8800 / ISO 21448)

• Standard-compliant use of quality metrics, real and synthetic data, and integration of explainability and monitoring evidence

• Evidence management and tool/prototype integration into existing MLOps processes

• Compliance mapping and audit preparation (ISO PAS 8800, SOTIF, ISO 26262)

Structured safety argumentation

We develop structured, modular, and dialectical safety cases and make them regulatorily compliant for efficient approval and faster certification in mobility contexts.

Consulting topics include:

• Safety case blueprints (e.g., GSN) for AI components, functions, and systems

• Dialectical argumentation with defeaters, assumption and evidence management

• Modularization through assurance contracts and reuse in product lines

• End-to-end traceability to requirements, tests, and data artifacts at component and system levels

• Training, guidelines, and templates for teams

Confidence assessment of safety arguments

Our approach quantifies confidence from evidence to claim, transparently propagating uncertainty for decision-ready statements.

Consulting topics include:

• Development of confidence frameworks (qualitative/quantitative) for safety cases
• Modeling of uncertainty in evidence (measurement, bias, coverage) and claims

• Determining evidence independence, defeater analysis, and residual risk

• Modeling of confidence propagation, sensitivity, and stress analyses

Continuous assurance: dynamic safety cases with runtime evidence

We develop dynamic safety cases, integrate runtime evidence, and continuously evaluate confidence – enabling true continuous assurance during field operation.

Consulting topics include:

• Design and implementation of Safety Performance Indicators (SPIs)

• Pipelines for evidence capture (in-service monitoring, DataOps/MLOps integration)
• Dashboards and reports for releases, change impacts, and residual risk
• Governance and triggers for re-argumentation during model/data updates
• Pilots, tool prototypes, and operational guidelines

Contact us now

Fraunhofer IKS supports you in the selection, integration, optimization, testing, and validation of AI components for your (semi-)autonomous function using our APIKS software platform for developing and validating autonomous driving functions.
 

You want to test an AI function?

You want to develop, assure, and test an AI function (e.g., perception, trajectory planning, end-to-end architecture) in a standardized environment Benefit from:

• Reusable functional blocks: Use existing components to accelerate development.
• Independent development of target blocks: Work independently on specific target modules.
• Standards-based system architecture and interfaces: Architecture designed for modularity, compliance, and sustainability, ensuring consistent and reliable communication between functional blocks in accordance with ISO 4804 / ISO 5083.
   

You want to validate an AI function?

Already have an AI function and want to validate it in a standardized environment? Benefit from:

• (Co-)Simulations: Easy integration into advanced simulators like CARLA for comprehensive testing and validation.
• Preconfigured simulation scenarios: Test and validate new functions using ready-made simulation setups to ensure efficiency and reliability.
• Compatibility with autonomous driving software platforms: Compatible with leading AD software platforms such as AUTOWARE.

Contact us now