EU AI Act: New Regulations for the Use of AI

The EU AI ACT requires companies to ensure that AI applications are not misused. In order to meet the new legal requirements for transparency, risk management and documentation, existing control structures must be expanded. Operational SAP operations and the IT organization are also affected. The new regulations will largely apply from August 2026.

The EU AI Act adopted by the EU Council in May 2024 creates a uniform framework for the use of artificial intelligence in the European Union. The AI regulation, the only one of its kind in the world to date, will largely go live in Germany in August 2026.
The AI regulation is thus gradually changing the compliance obligations for the use of AI in the Finance and accounting and the audit agenda of Internal Audit:

• New obligations for inventorying & risk classification of KI applications
• increased transparency and documentation requirements
• as well as further audit and governance tasks

1. Risk classification as a new control instance

   The ICS must include processes that assign each AI system used to one of the four risk classes (unacceptable, high, limited, minimal). The requirements for the ICS are particularly high for high-risk AI systems (e.g., in personnel recruitment or credit checks).

2. Implementation of AI risk management

   The regulation prescribes a dedicated risk management system for high-risk systems. This includes:

   • The systematic identification and analysis of risks to health, safety, and fundamental rights.
   • The definition of mitigation measures requires that they be regularly tested for their effectiveness.
   • Extended documentation and logging obligations: The ICS must ensure that technical documentation is always up to date and that AI systems automatically log their processes (logging) to ensure traceability.
   • Data governance: In the future, internal controls must also monitor the quality of training and test data to minimize bias and errors in AI output.

3. Human Oversight

   Control mechanisms must ensure that AI decisions can be checked by humans and corrected or stopped if necessary.

• AI-based credit assessments may be classified as high-risk. This leads to mandatory risk analyses, bias tests, and documentation.
• Transparency regarding training data, model documentation, and explainability to regulators and data subjects is required. Accounting and valuation processes must remain auditable.
• The processing of personal data in AI models increases GDPR risks. Risks of discrimination (bias) require continuous monitoring and targeted countermeasures.
• Auditors must be able to review technical documentation, risk analyses, and test logs.
• External audits are becoming more likely. “Unpopular” disciplines, such as audit trail documentation, are gaining greater importance.

In change and release management, the requirements for documented validation are increasing, as every AI model change in SAP systems requires test logs and approvals.

Third-party risks associated with purchased models and/or cloud agents (e.g., SAP partner models) require contractual assurances regarding data provenance and model transparency. The following SAP AI applications may be affected by the EU AI Act:

 

• SAP Business AI (Joule, Agents): role-based AI assistants and ready-made AI scenarios for HR, Procurement, Accounting Accruals Agent, Cash Management Agent
• SAP AI Core & AI Launchpad: Infrastructure for training, deployment, lifecycle management, and monitoring of own ML models on the SAP BTP.
• Generative AI Hub / Generative AI functions: Integration of generative models for text/report generation, summaries, and semantic search (e.g., in Joule).
• Embedded AI in SAP S/4HANA: preconfigured AI scenarios (e.g., automatic invoice processing, classification, cash forecasting).
• SAP BTP AI Services (e.g., document classification, OCR, tabular models): reusable services for data extraction and predictions.
• SAP Analytics Cloud (AI functions such as "Just Ask") and Predictive Services: Analysis and explainability functions for financial reports.
• Intelligent Scenario Lifecycle Management (ISLM) / Joule Studio / Build Tools: Tools for development, governance, and lifecycle of AI use cases in SAP.