AI Governance: How to Keep AI Agents Under Control

Traditional software does exactly what is specified in the code. AI systems—especially AI agents—use their degrees of freedom as the situation warrants. This makes them more powerful, but also harder to control. AI governance, therefore, must not be just another item on a checklist right before go-live. It must be part of the architecture from the very beginning.

Intelligent systems require advanced capabilities for management, control, and traceability. Our approach: Not every action performed by a AI system is treated the same. Depending on the context and risk, it is automatically approved, secured by rules, subject to additional approval, or blocked. This takes into account both legal requirements and our experience from customer projects.

1. No AI has more authority than the user or the business process on whose behalf it acts.
2. Strict security and business rules are enforced deterministically and outside the agents.
3. Autonomy grows only to the extent that its effects are limited, observable, and reversible.
4. In addition to standard checks, data science methods ensure that systems and their output remain manageable and explainable.

These principles apply regardless of platform, model, or use case—for both traditional and generative AI.

Every action that an AI system can perform is assigned to one of five risk classes:

• Class 0 – Searching for and reading shared data: automatically allow, log, and check on a random basis.
• Class 1 – Create a draft, complete a ticket, update an internal record: deterministic policy, as well as schema and business rule validation.
• Level 2 – Sending emails, submitting minor requests, exporting customer data: step-up policy, limits, and user confirmation if necessary.
• Level 3 – Modify permissions, create quotes, cost-intensive actions, bulk deletion: dual-control principle, human approval, or defined workflow.
• Class 4 – Policy circumvention, privilege escalation, unauthorized data use: Block and treat as a security event.

This way, non-critical tasks run smoothly, while critical decisions remain the responsibility of humans. Class 4 actions are not tasks to be automated.

Whether an action is permitted depends on more than just the tool call itself. Among other things, the following factors are relevant:

• Identity of the User and the Agent, and the Agent's Mandate
• Versions of Agent, Prompt, Model, and Tool
• Tool called, operation, parameters, target resource, and destination
• Type of Data Involved and Source of Information Relevant to Decision-Making
• Reversibility and Idempotence of the Action
• Session, Trace ID, and Applicable Policy Versions

This makes it clear why a system is allowed to do something, why it isn't, or why additional authorization is required.

No single measure is enough. AI governance is effective when both worlds work together:

• Traditional tools: IAM/RBAC, policies, gateways and rate limits, secret management, tool and API controls, approval and escalation workflows, monitoring and audit trails, encryption.
• Data science tools: guardrails, anonymization and redaction, drift detection, protection against the reconstruction of training data, fraud detection, human-in-the-loop, model fingerprinting, evaluation and testing, as well as transactions and time travel, so that incorrect decisions can be reversed.

To this end, we rely on both open-source solutions and services from our technology partners.

You need explainability, traceability, and controllability for every component in the AI system. A solution that works from a technical standpoint but cannot be controlled becomes a risk during operation. That is why we incorporate AI governance from the very beginning when implementing agent-based and AI solutions and implement it technically. Only when traditional, deterministic, and data science methods work together can a trustworthy and controllable overall result be achieved.