Fraunhofer Institute for Cognitive Systems IKSFraunhofer IKS supports industrial companies that want to use AI agents, from the initial potential analysis to a safety-certified architecture. Our focus is on integrating AI agents into the system architecture in a way that is controllable, traceable, and ready for industrial use.
© iStock.com/gorodenkoff
Trustworthy AI Agents for Industry
Deploy AI agents safe, cost-effective, and in compliance with regulatory requirements
Contents
Why AI agents are now relevant
AI agents and multi-agent systems open up new possibilities for industrial automation: They can process information from various sources, plan tasks, prepare decisions, and coordinate complex workflows. This gives companies the opportunity to design processes that are more flexible, efficient, and data-driven. However, companies looking to deploy AI agents must address three key questions early on:
- Cost-effectiveness & token economy: Where does the computation- and token-intensive use of autonomous systems offer an advantage over deterministic software?
- Risks & liability: What risks arise from dynamic agent behavior, and how can compliance with the EU AI Act and product liability requirements be ensured?
- Operation & control: How can system behavior be monitored and controlled in real time during ongoing operations?
Deploying AI agents is a strategic decision that depends on market conditions and economic viability:
Tabbed contents
Cost-effectiveness and efficiency potential
Cost-effectiveness and efficiency potential
In capacity planning, adaptive production control, and intralogistics, autonomous agents achieve efficiency gains of up to 30%. However, since LLMs incur ongoing OPEX costs, the preliminary analysis by Fraunhofer IKS ensures that AI is deployed only where it generates a measurable ROI.
Regulatory Issues and Liability Risks
Regulatory requirements and liability risks
The EU AI Act and stricter European product liability rules place obligations on operators. They require full traceability and systematic risk management for autonomous systems. Unsecured agents pose significant financial and legal risks.
Token Economy
Token economy
Every step – from reading, reasoning to responding - consumes tokens. The amount depends on model size, number of users, latency, and data location. While individual prompts are becoming more cost-effective, Agentic AI multiplies the token requirement per output through autonomous prompt chains, tool calls, and database queries. The location of processing (device, edge, cloud) determines the total cost. Fraunhofer IKS optimizes your architecture for maximum efficiency with minimal token consumption.
Lifecycle safety: Autonomy needs control
As systems become more autonomous, safety, compliance, and liability risks become more unpredictable. Since agents act dynamically and develop their own plans, static model tests are insufficient. Therefore, Fraunhofer IKS uses end-to-end validation:
Assurance before implementation
- Hazard and Risk Analysis (HARA): Systematic identification of potential malfunctions
- Operational Design Domain (ODD): Precise definition of permissible operating parameters (e.g., load limits, data sources)
- Architectural design: Design of integrated safety guards and deterministic fallback levels
Assurance during operation
- Runtime monitoring & assurance: Continuous real-time monitoring of agent decisions through upstream validation instances
- Dynamic assurance cases: Digital, automated chains of evidence for safe operation within defined safety boundaries
- Human-in-the-Loop: Clear escalation paths for seamless handover of critical or ambiguous decisions to humans
The Fraunhofer IKS approach: System-level assurance
Static model validations quickly become outdated. Fraunhofer IKS offers a model-independent assurance framework. We do not validate the individual, volatile LLM, but rather the overarching process and interaction structures. Even with model updates, your security and cost architecture remains intact.
Our SCALE framework:
- Strategy: Definition of goals, risks, and potential
- Concepts: Development of solution approaches and system architectures
- Assurance: Identification, analysis, and mitigation of critical risks
- Learning: Prototype testing and continuous learning
- Execution: Transition to production
Readiness Check: Potential Analysis for AI Agents
Readiness Check: Potential Analysis for AI Agents
Do you want to find out whether AI agents make economic sense for your process – and where?
As part of the potential analysis at Fraunhofer IKS, we evaluate your specific use case for technological feasibility and maturity. In doing so, we compare the use of AI agents with traditional automation approaches. This gives you a solid foundation for making strategic decisions regarding the deployment of AI agents.
At a Glance:
- Challenge: You want to avoid investing in AI agents that are economically or legally unsuitable.
- Our service: Fraunhofer IKS evaluates your use case from both technical and economic perspectives in terms of feasibility, token costs, and ROI
- Your result: You’ll receive a strategic decision paper including a make-or-buy assessment, a business case, and a concrete roadmap for implementation
Agentic AI Risk Check
Agentic AI Risk Check
Would you like to assess the risks and costs of existing AI pilot projects?
With the Fraunhofer IKS Agentic AI Risk Check, we conduct a technology and maturity assessment of existing pilot projects and compare LLM-based and rule-based approaches. This allows us to identify critical vulnerabilities before costly missteps occur.
At a glance:
- Challenge: You want to identify unknown security risks and potential cost pitfalls in existing pilot projects or AI prototypes at an early stage.
- Our service: Fraunhofer IKS audits your agent architecture for AI-specific vulnerabilities, false triggers, and inefficient token chains.
- Your result: You’ll receive a technical report with risk mapping and recommendations to help you avoid costly missteps.
Assurance Concept & Compliance
Assurance Concept & Compliance
Do you need transparent assurance and clear regulatory classification for your AI agents?
We analyze the risks in complex agent interactions, develop safety arguments, and prepare the technical documentation required for regulations such as the EU AI Act. In this way, Fraunhofer IKS creates transparent reliability metrics and a robust foundation for compliance with industry standards, legal requirements, and accelerated system approval.
At a glance:
- Challenge: Before the system rollout, you need liability frameworks and safety evidence to meet regulatory requirements (EU AI Act).
- Our service: Fraunhofer IKS develops formal safety arguments, defines transparency metrics, and structures the documentation.
- Your result: You receive an AI Act-compliant compliance dossier, including structured safety evidence (assurance cases) for audits and certifications.
Framework Integration & Co-Development
Framework Integration & Co-Development
Do you want to integrate AI agents into your existing system landscape in a controlled manner?
We implement the process-oriented assurance framework developed by Fraunhofer IKS directly into your architecture (device, edge, core, or cloud) and develop customized runtime safeguards. This creates a robust, controllable, and future-proof system architecture that is ready for real-world operation. Model updates or the transition to more efficient LLMs can therefore be integrated flexibly without requiring recertification or architectural overhauls.
At a glance:
- Challenge: You want to integrate AI agents into existing IT/OT landscapes without creating new security or budget risks.
- Our service: Fraunhofer IKS supports you as a co-engineering partner in implementing our model-independent verification framework, including runtime safeguards and token guards.
- Your result: You’ll receive a functional, hardened software prototype with a real-time monitoring dashboard and documented interface architecture.
Ready for the next step?
Integrate AI agents into industrial operations in a controlled manner
Schedule a no-obligation 30-minute consultation with the AI safety and architecture experts at Fraunhofer IKS. Together, we’ll take a first look at your use case and assess its potential.