The Path to Sovereign AI

Artificial intelligence has long been a part of our lives, although a few years ago, many people were still suspicious and uncertain about how AI would develop. Due to the rapid progress of recent years, especially the hype surrounding generative AI (e.g., ChatGPT), the dependence on global technology companies and their often opaque algorithms is growing alongside the possibilities.

Building an independent AI infrastructure presents companies that public administration and the healthcare industry face considerable technical, economic, and regulatory challenges to public administration and the healthcare industry but also offers numerous opportunities. These players are increasingly asking themselves how they can confidently and securely operate AI and automation solutions confidently and securely without becoming dependent on large providers while minimizing the entry risks.

Many companies offer powerful, easy-to-integrate AI models and services that often appear cost-efficient. However, using many of these offerings quickly leads to lock-in effects, making it difficult to switch between AI solutions and models. In addition, many models are black box systems whose calculation and decision-making processes are challenging to understand and sometimes result in losing data sovereignty if the data is processed and stored externally. Therefore, Sovereign AI should offer data protection by design to ensure transparency and secure data processing.

In addition to the aspects of intransparency, there is also the high demand for computing capacity and energy consumption. This is because high-performance AI requires immense computing power, which companies often cannot provide themselves. Without their own data centers, these companies are forced to rely on cloud services, bringing new dependencies and security issues.

Implementing a sovereign AI in your company requires careful planning and a strategic approach. In the following three steps, you will learn how to lay the necessary foundations, identify the right use cases, and involve the right personnel to implement your AI initiatives successfully.

1. Building a sovereign AI infrastructure

   The efficient use of GPU resources is a key success factor in implementing powerful AI systems. A well-thought-out infrastructure strategy can not only optimize computing capacities but also take costs and data protection requirements into account. While cloud solutions offer flexible scaling options, on-premises servers have advantages in terms of data control. Especially with GDPR, local data centers can be a secure alternative that provides expandable computing power. This is a crucial aspect for the public sector and healthcare in particular.

   The number of GPUs required is highly dependent on the respective use case. While training complex models requires considerable computing capacity, the demand for inference, i.e., the process in which a trained model makes decisions based on new inputs, is significantly lower. Real-time applications benefit from GPUs with high memory bandwidth, whereas batch processes, i.e., data processing in groups or batches, often manage with fewer resources. Model size also plays a decisive role. Large AI models such as GPT-4 require several GPUs, while more compact architectures can often be operated efficiently with a single high-end GPU.

   When choosing a GPU, models with high memory bandwidth and optimized computing operations are usually used - such as the NVIDIA H100, which was specially developed for AI workloads.

   In addition to the model size, the amount of data and the batch size also significantly impact memory requirements. Larger batches, i.e., bundles of data, require more VRAM, making selecting powerful GPUs necessary. In addition to the available memory, necessary specifications also include the support of optimized calculation methods such as FP16, TF32, or INT8, as well as the availability of specialized computing units such as tensor cores, specially developed computing units for matrix operations that can process large matrices in a short time. Strategies such as data or model parallelism allow workloads to be distributed across several GPUs, thus significantly reducing training times.

   However, it is not just technical factors that play a role - economic considerations are also crucial. While cloud GPUs enable on-demand use, on-premises solutions can be more cost-efficient for long-term use, provided the infrastructure is designed accordingly. Therefore, a well-founded requirements analysis should consider the required computing power, the underlying model architecture, and the data size. Ideally, this is done based on established benchmarks.

2. Identification of use cases and models for sovereign AI

   Once the appropriate infrastructure has been selected, the specific use cases must be precisely defined to determine the actual computing requirements. This can be done, for example, through a requirements analysis or process mapping. This involves discovering where the AI will be used and what volume it will process. Among other things, this could use valuable data from your specialist departments or process or quality management. As a rule, this provides important information on the requirements of your AI for the respective use case. The use cases can be very individual, such as the analysis and automation of application processes in traditional public administration, traditional processing at health insurance companies, diagnostic support for doctors in the healthcare sector, or the processing of large unstructured data sets at pharmaceutical companies, for example.

   Relevant parameters here are the number of model parameters, the required FLOPs, the unit of measurement that specifies how many floating-point operations (e.g., addition, multiplication) a hardware can perform per second, and the model architecture, such as transformer networks or convolutional neural networks (CNNs), which are optimized for parallel processing.

   In addition to proprietary AI models, open-source alternatives such as Meta's Llama models can also be an attractive option for implementing use cases. As the market for AI models is changing rapidly, you should consult experts when choosing your AI models to make the best long-term decision and always stay up-to-date. Therefore, consider carefully which use cases you want to implement and determine the computing power required to run suitable AI models on your required infrastructure.

3. Building a qualified team for sovereign AI

   After all, setting up a powerful AI infrastructure requires not only the right hardware and qualified specialists. In particular, public administration faces a major challenge in this respect with an increasingly aging workforce. For this reason, establishing internal competence centers should be started at an early stage to retain specialist knowledge internally. However, these should also ensure that employees are trained, internal knowledge transfer is secured, and cooperation within the company is expanded. This way, a sustainable AI strategy can be created, combining technological innovation with economic efficiency and digital self-determination.

Do you already have initial ideas for using sovereign AI but don't know how to get started, or have you already started but don't know which models are right for your use cases?

This is exactly where we come in as Arvato Systems. We support our customers from the outset in making decisions regarding their sovereign AI strategy and enable them to build their first AI use cases on a sovereign infrastructure.

We also address the regulatory requirements of the AI Act, GDPR requirements, and individual requirements of our customers. Thanks to the capabilities of our data centers and the many years of experience of our AI experts, we combine technological know-how with strategic approaches and ensure all-around carefree support for your future systems.

Our goal is not only to achieve technological sovereignty for our customers but also to provide our customers with artificial intelligence that is competitive and compliant in the long term and to continuously develop it in line with the latest standards. Building a sovereign AI is challenging, but with the right partner at your side, it is a strategic and valuable investment in your future.