Adaptive safety and performance management

Cognitive systems are deployed in dynamic contexts. This leads to many uncertainties that make it difficult to predict the system context and validate the system accordingly. Even functions based on machine learning are sources of uncertainties since their behavior cannot be accurately predicted.

Conventional validation approaches have always assumed the worst case situation in these dynamic systems, leading to complicated, costly or ineffective system designs. However, dynamic safety and performance management make it possible to overcome these current barriers and easily, cost-effectively and efficiently validate cognitive systems, even in dynamic contexts.

The Fraunhofer Institute for Cognitive Systems IKS develops solutions designed for safety-critical systems that are often based on AI technology. In contrast to conventional methods, the Fraunhofer IKS approach specifically addresses the three interconnected issues of safety, reliability and cost. This permits the creation of new and innovative market-ready products for a wide range of industries, which otherwise would be unacceptable to society without the corresponding safety solutions.

More specifically, this involves adapting and enhancing proven software engineering methods, which includes processes such as determining and analyzing risks, carrying out error analyses, deriving safety concepts and generating safety verifications. The high degree of automation in model-based system development makes it possible to move partial aspects, such as the dynamic risk analysis for different operational design domains (ODDs), to the runtime environment. To do that researchers are developing models that a system can use to identify and consciously determine, or verifiably optimize, its own state with respect to safety, reliability and availability at runtime. The system is able to interpret the models on its own and derive corresponding measures to safeguard its resilience. Using risk models as a foundation, the system can also evaluate the risk for a specific situation. With these situation- and context-dependent safety management solutions, the desired cognitive system performance can be achieved while guaranteeing safety at the same time.