Managed Artificial Intelligence Services

AI projects are usually based on a multi-stage process model. First, it is necessary to identify relevant use cases. Based on the actual situation, it must be analyzed whether the existing data, systems and processes are suitable for implementing the favored use case. Then, the prototype's goals and requirements have to be defined, transformed into a concept, and the best solution approach has to be developed. The third step is to transform the verified prototype into a Minimum Viable Product (MVP). Continuous function, load, and integration tests ensure stable integration of the AI service into the operating environment. The finished AI Service can then be integrated into applications, processes, and systems via interfaces and scalable. Many AI projects fail in the fourth phase: It is essential to operate the AI Service reliably in a real production environment.

Professional Support for AI Services

To prevent AI projects from failing in the last mile, it makes sense for companies to consider having their AI services managed by experienced partners. To obtain so-called Managed AI Services is also an option. While classic Managed Services are concerned with getting recurring IT services according to defined Service Level Agreements (SLA) - from operation and monitoring to updating systems and applications - Managed AI Services have a different thrust. In addition to the question of where an AI service is to be operated - in the company's own data center, on-premises at a service provider, or in the cloud - one aspect is crucial to success: the AI service must be continuously monitored and its underlying model must be adapted again and again - in live operation. An example: A plant manufacturer uses an AI application that detects worn components. The basis is video surveillance of the plants. For a functioning AI service, the existing video material must be viewed, a model must be created, and the AI must be trained with real data to identify signs of wear or leaks. If the company commissions a new plant or replaces individual machine parts, the model must be adapted, and the AI trained again - a task that requires a great deal of know-how and ties up many resources. With the Managed AI Services of a specialized service provider, the plant builder can instead concentrate on his daily business. The service provider's Data & AI experts take care of retraining and going live. 

Keeping the deployment under Control

An interdisciplinary project team must provide Managed AI Services professionally, which includes a Data Scientist or Machine Learning Engineer, Data Engineer or Data Architect, Cloud Architect, and DevOps Engineer. The data scientist transforms the task into automated procedures with the help of AI and machine learning. The Data Engineer is responsible for collecting and consolidating the required data. While the Cloud Architect sets up a secure and highly available IT infrastructure for development and subsequent operation, the DevOps Engineer forms the indispensable interface between development and operation.

An Endless Cycle

To transfer an AI service into productive operation, the project team must work together smoothly. The data scientist develops an AI model in a so-called "sandbox." In doing so, he experiments with test data. On the other hand, the Data Engineer establishes the permanent connection between the trained AI model and the real operating data, and the DevOps Engineer accompanies the incredibly important part of the production setup. Thus, the AI service leaves the "sandbox" and moves to secure infrastructure and must function reliably in real-time. From this point on, it must be continuously maintained and improved. In productive operation, an AI service generates a vast amount of data. Therefore, it must be checked whether the original model is still plausible with the generated data. If this is not the case, the Data Scientist must adapt it - just like the respective processes. To do this, he must be able to access existing operating models and data. To integrate the adapted model into the production environment a second time under the DevOps Engineer's supervision, the AI must be trained (retrained) and tested in advance. And then the process starts all over again - an endless cycle that only runs smoothly if development and deployment are seamlessly integrated. Also, external circumstances and the requirements for an AI service can change abruptly. Reacting flexibly to these changes is an absolute must. As they were possible in the development phase, adjustments in the trial-and-error procedure are taboo in live operation. Agile methods such as continuous integration, continuous delivery, and continuous deployment are therefore recommended. They shorten the time until a new model goes live.

Monitoring Is Mandatory

Reliable end-to-end monitoring of each individual AI service is essential to identify any need for adaptation. After all, it must be ensured that the AI-based system always works. Therefore, even 24/7 monitoring may be required for particularly critical AI services. It is essential that the service provider defines company-specific key figures, measurement, and threshold values and integrates these into standard processes following ITIL (Information Technology Infrastructure Library) as part of IT Service Management (ITSM). A distinction must be made between the monitoring of infrastructure and application. Infrastructure monitoring ensures optimum availability, accessibility, performance, and utilization with the help of appropriate event and incident management processes. The monitoring of the applications is carried out by monitoring the interfaces and regular queries. 

Taking a Step Back in Case of Doubt

It is also essential to continuously monitor and historicize critical figures so that adjustments can be reset in case of doubt. Despite careful pre-analysis, an AI service may behave differently in the real operating environment than assumed in the test phase. In such a case, it is crucial to switch back to the previous version quickly.

Maximum Flexibility

Also, there is another aspect to be considered in AI projects. It is crucial to avoid a vendor lock. The underlying model must be designed so that an AI service can be transferred to a different infrastructure - whether it be to another cloud, to use it as an on-premises solution at a data center service provider, or even to operate it in the company's own data center. Such flexibility is guaranteed if the service provider provides the finished model via an application programming interface (API), takes care of operation and monitoring of the AI service, and offers accompanying support. When trying to provide all these services themselves, companies often reach their limits.

Find the Right Partner

Developing, operating, and updating AI services is complex. Companies must, therefore, ask themselves whether they can or want to meet this challenge alone. In most cases, companies ultimately rely on cooperation with an appropriate partner. Then the success of AI-based applications stands and falls with the service provider. It is vital that the service provider offers managed AI services from a single source, has a great deal of expertise concerning the many different tasks involved, and makes the transition from development to secure operation seamlessly. That requires experienced experts with specialized skills - depending on their role in an AI project. It is advantageous if the service provider can draw on extensive experience in the operation of infrastructures and therefore transfer proven concepts and procedures into artificial intelligence. With such professional support, companies can concentrate on the respective use case, noticeably accelerate the associated processes thanks to AI and effectively drive their business forward. In this combination, Managed AI Services provides companies with comprehensive support in the use of AI-based applications.

For questions please contact Niels Pothmann or Andree Kupka.