Ensuring Safety of Software-based Systems in the Defense Domain

Safety is a crucial requirement dictated by standards, but it is often perceived as a barrier to innovation and can incur significant costs, particularly when implemented post factum. When safety measures are either excessive or poorly suited to flawed architectures, they can result in brittle systems that lack reliability. Additionally, common fail-stop approaches may inadvertently create safety risks for human operators, compounding the challenges of ensuring operational effectiveness.

To navigate these complexities, organizations must embrace short development cycles that allow for the swift integration of new technologies and adaptation to highly variable or unpredictable mission conditions. However, the lack of clear safety compatibility between different product versions and their components complicates in-field repair efforts, further exacerbating the issue. Additionally, legacy components, whether they are software or hardware, embedded in safety functions require extra steps to innovate, making it even more difficult to achieve the desired balance between safety and innovation.

Our Solutions and Offerings

• Apply our design process and principles for resilience, to create safe and reliable systems even in case of unknown scenarios, damage or faults.
  • Graceful Degradation allows a system to maintain a reduced level of functionality in the event of a failure, rather than failing completely. This is crucial in military operations where maintaining some level of operational capability can mean the difference between mission success and failure.
  • Pre- / In- / Post-Mission Adaptations ensure that systems can be tailored to specific mission requirements before deployment, adjusted in real-time during operations, and evaluated for improvements after missions are completed. It enhances the system's resilience and effectiveness in unpredictable environments.
  • Self-Adaptivity / Autonomy allows the system to learn from its environment and make decisions independently, allowing for quick responses to changing battlefield conditions. This autonomy is particularly valuable in complex scenarios, where human operators may not always be able to react swiftly enough.
• Reshape safety functions to reduce the impact of legacy components and the need for post factum safety analysis. 

• Efficient integration of product lines, DevOps and platform systems (“App-Store”).

Contact us now

The deployment of Artificial Intelligence in defense requires dependable, real-time AI-systems for crucial tasks such as intelligence gathering, situational awareness, and force protection. Defense systems must work reliably and ensure mission-critical accuracy and resilience against active adversaries. We focus on safeguarding AI solutions to provide reliable functionality, even in difficult environments. This includes the following research topics:
 

Mission-Critical Accuracy and Uncertainty Estimation

The inability of a system to estimate its confidence level is a major obstacle in critical applications. For defense applications, where errors have high-stakes consequences, reliability is non-negotiable.

Our Solutions and Offerings 

• Precision and Reliability: The degree to which models and data correctly reflect the true value of the intended attributes in a particular context of use to prevent non-intended behaviour such as misidentification or missed threats.
• Explainable Confidence: Developing Interpretability capacity that allows human operators to understand the internal behavior of AI, which is vital when decisions are being made.
   

Resilient AI-based Functions

AI-based functions must be resilient against external factors such as unforeseen events or adversaries attempting to manipulate the system. It demands that the AI-based system can perform reliably in dynamic, non-cooperative, and uncertain environments.

Our Solutions and Offerings 

• Adaptive Anomaly Detection: Implementing mechanisms to identify and flag novel, un-anticipated patterns or Model Deception attempts, guaranteeing high operational fidelity even against unknown threats.
• Counter-Deception and Evasion: Developing AI with integrated features and increased capacity for AI self-protection to prevent non-intended third-party interactions, such as adversarial attacks designed to deceive or evade the model.
• System Resilience: Engineering systems ensuring the AI-enabled platform can recover operational condition quickly following an incident or attack, maintaining continuous threat monitoring.
   

Safety Assurance for AI-based Systems

The development of AI-based systems in defense requires new standards and assurance methods to qualify the system for deployment. Our research focuses on adapting existing safety standards to specific applications and the military domain's unique requirements for technical robustness and safety.

Our Solutions and Offerings 

• End-to-End Testing and Evaluation: Defining and implementing a Validation and Verification (V&V) process across the entire lifecycle to qualify AI systems.
• Certification and Qualification: Supporting the homologation/certification processes of our customers necessary to release AI-based systems for operation.
• Compliance with Legal Frameworks: Providing guidance during the development phase to ensure the design and use of sensing systems align with the necessary policies, regulations, and standards, forming a crucial foundation for robust AI systems.

Contact us now

In the defense domain, operational contexts are inherently unpredictable, making resilience an essential requirement for modern systems. At Fraunhofer IKS, we specialize in developing resilient systems that dynamically adapt to changing and unsafe environments while maintaining safety and optimizing utility. This capability is fundamental for dynamic defense applications, where the margin for error is minimal.

The Need for an "Emergency Go" Function

Traditional safety paradigms often emphasize stopping systems in the face of uncertainty. However, in defense scenarios, an "emergency go button" is often more critical than an "emergency stop button." This approach ensures that systems can continue to operate, even in degraded states, to provide essential functionality when it matters most, such as during life-or-death situations for tank crews or other mission-critical operations.

Dynamic Risk Adaptation for Real-World Missions

Conventional safety standards rely on static risk acceptance scales defined during the design phase. While these scales are suitable for controlled environments, they fall short in real-world defense missions, where even minimal functionality can be the difference between mission success and failure. Our approach extends traditional resilience by enabling systems to dynamically adjust their risk acceptance scales at runtime. This allows systems to prioritize survival and mission-critical functionality in high-stakes scenarios, transitioning from training-level safety assumptions to real-mission adaptability.

Our Solutions and Offerings

Our expertise in resilience cyber-physical systems such as autonomous vehicles, mobile robots, is particularly suited for defense technologies. By integrating resilience principles, we help ensure that these systems remain effective in unforeseen situations and optimize functionality and safety in a way that is appropriate for the most uncertain and dynamic operational contexts.

Fraunhofer IKS offers cutting-edge research and solutions tailored to the unique challenges of the defense sector. Whether you are looking to implement an "emergency go" function, enhance system adaptability, or explore dynamic safety concepts, we are here to support you. Contact us to discuss how we can help you build resilient systems that meet the demands of modern defense operations.

Key Features of Our Resilience Solutions

• Continuous Safety Management: Systems that continuously assess and adapt to changing risk levels in real time.
• Degradation: Ensuring that systems maintain critical functionality, even in degraded or unsafe conditions.
• Open-World Context Adaptation: Designing systems capable of operating effectively in unpredictable and undefined environments.
• Optimized Utility: Balancing safety and performance to maximize mission success under challenging conditions.
   

Contact us now

In the defense domain, the complexity of operations often extends beyond individual systems to interconnected Systems of Systems (SoS). These SoS are composed of multiple, independent systems that must collaborate seamlessly to achieve overarching mission objectives. The unpredictable and dynamic nature of defense scenarios makes resilience not just a desirable feature but an operational necessity for SoS. At Fraunhofer IKS, we specialize in developing resilient systems that adapt dynamically to changing and unsafe environments while maintaining safety and optimizing mission utility.

Dynamic Risk Adaptation in SoS

Traditional safety standards often fail to account for the dynamic and interdependent nature of SoS. To address this, we extend the concept of dynamic risk adaptation to the SoS level. This involves:

• Runtime Risk Scaling: Allowing the SoS to adjust its risk acceptance levels based on the current mission phase and context. For example, higher risks may be acceptable during a critical mission phase compared to routine operations.
• Distributed Decision-Making: Enabling individual systems within the SoS to make local decisions while aligning with the global mission objectives.
• Degradation at Scale: Ensuring that the SoS as a whole continues to function, even if individual systems are degraded or fail.

Our Solutions and Offerings

Fraunhofer IKS offers cutting-edge research and solutions tailored to the unique challenges of defense Systems of Systems, including integrating new and legacy systems to one SoS. Our expertise in resilience, dynamic safety, and adaptive systems ensures that your SoS can operate effectively in the most challenging and unpredictable environments.

Whether you're looking to implement an "emergency go" function at the SoS level, enhance inter-system adaptability, or explore dynamic safety concepts, we are here to support you. Contact us to discuss how we can help you build resilient SoS that redefine safety and performance for modern defense operations.
 

Contact us now

We research Quantum computing and quantum-inspired AI methods to address technological challenges in the defense industry.

Learn more on our quantum computing page.

Contact us now