Arvato Systems Creates AI Shopping Wizard

Artificial intelligence makes it possible: Arvato Systems is currently developing a digital shopping assistant that will help customers choose their clothes. The recommendations made by the digital assistant are based on the current outfit, the desired occasion, age, gender, and the personal preferences of the buyer.

What will it be? A suit, casual wear or perhaps a suitable sports outfit? A digital shopping assistant with AI features could soon revolutionize the way we shop. Most online shops rely on experienced human shopping consultants in the background or general, automatically generated suggestions. Arvato Systems, on the other hand, analyzes photos of customers with the help of "Cognitive Services" of Microsoft's Azure Cloud. The results are suitable outfit suggestions based on the age, gender, and fashionable clothing of the test persons.

The system was designed in mid-2018 by Björn Brockschmidt together with a team from TU Munich and initially presented on a Pepper robot. However, it also works on POS systems, smart TVs, and even smartphones as a PaaS or SaaS offering.

Massive potential for online shops and stationary trade

It is not without reason that many online shops now offer not only an extensive range but also individual consulting solutions, mostly with trained shopping assistants in the background. Among the largest providers of online shopping advice are currently:

• Outfittery
• Zalon (by Zalando)
• Modomoto
• AIsura
• Style Agency (by About You)

According to a Statista study conducted in 2017, 38 percent of respondents took advantage of one of these counseling services within twelve months. According to a further study, almost half of those surveyed believe that AI support could also stimulate the retail sector (44%). For an even more full acceptance of such systems, the quality of the recommendations and the simplicity of operation are certainly the most critical factors.

An AI system, as recently introduced by Arvato Systems, can provide valuable assistance in this respect in the retail sector: The potential customer first uploads a photo of himself to the cloud using a robot with a built-in webcam. The system then selects from numerous fashion shots those recommendations that might suit the type of user. Individual differences and personal tastes should be taken into account as well as the occasion for which the outfit is planned.

As part of the Bertelsmann Hackathon 2018 (more information here), the AI shopping solution was designed as a cloud-native system and presented in its basic version in action for the first time. Randomly selected test persons were given suggestions for outfits in the areas of casual, sports, and business. But this is only the beginning. Gradually, the system is to become even more accurate in its proposals and analyses.

A digital shopping assistant based on Cognitive Services

Björn Brockschmidt (expert for Robotic Process Automation and Artificial Intelligence at Arvato Systems) and his team from TU Munich convinced the jury at the Bertelsmann Hackathon 2018 with the idea of an AI-based shopping solution and took first place. The execution took place via a small, white robot of the Pepper series called L.I.S.A., which presented various test persons with a series of suitable suggestions for outfit design. In addition to its humanoid appearance, the Pepper L.I.S.A. robot has an integrated tablet and a camera with which it takes high-resolution photos of test subjects.

The photo taken by the robot is uploaded to the Microsoft Azure Cloud and analyzed by "Cognitive Services." In the background, a PaaS (Platform-as-a-Service) solution works, which compares the photo taken with a previously learned model and evaluates it according to specific, pre-defined criteria. By using the existing interface, photos with known content were uploaded and converted into a suitable learning model.

As a first result, the system provides an assessment of the user's age and gender. In the next step, the user can decide to have his current outfit evaluated for a specific occasion. The AI system now analyzes what kind of gear it is. These three areas are currently being considered:

• Business: Does my outfit fit a business occasion?
• Casual: Can I wear this outfit well in my spare time?
• Sport: Is this outfit suitable for sports activities?

Alternatively, the user can also display suitable suggestions for a specific occasion. For this purpose, the software examines the image for further characteristics that affect the current clothing style of the person depicted and identifies suitable suggestions.

Finally, the user is presented with three clothing suggestions for the chosen occasion - and has the option to receive an evaluation of his or her current outfit.

That is how AI-supported outfit consulting from Arvato Systems works

Using the most considerable possible amount of data whose structure and content are known, the AI system (in this case the Microsoft Azure Cloud and it's Cognitive Services module) learns to recognize commonalities and differences between data sets.

The result could look like this:

• The user is male with a probability of 80%.
• The age of the person is probably approximately 35 years.
• The outfit of the person depicted has a 65% match with the clothing style "leisure." 25% could also be "business," but only 10% "sport."
• Currently in development: The user likes to wear the colors dark green and ochre.

The next step is to identify suitable suggestions from the previously uploaded image pool and propose them to the user.

Of course, many other criteria are conceivable, such as the characteristics of the user, his preferences, or the given occasion. The system introduced in 2018 and developed within only two days is only the beginning when it comes to the vast possibilities of AI-supported purchasing consulting.

The shopping experience of the future

The AI-supported shopping solution from Arvato Systems is currently further developed under the working title "ShopRobi 4.0" for various application areas. It can later be used on the Pepper robot mentioned above as well as on other robots or POS systems. A SaaS solution is working in the background, which can be utilized off-premise and does not require any investments in server hardware.

It is also conceivable to add Conversational Agent functions, which will enable the consulting system to respond even better to the wishes of customers or to subsequently correct system suggestions according to their preferences.

Completely mobile: Outfit consultation on the Smartphone or Tablet

Smartphones and tablets have two significant advantages: They are usually on the move and have a built-in camera. That means that the ShopRobi system can also be used without requiring a robot. It is conceivable, for example, that the customer picks up his device in a particular shop, opens the corresponding app and uploads a picture of himself or herself (preferably in front of a mirror to capture the whole person).

Within seconds, the user not only receives outfit suggestions but is also guided directly to the department which stores the shown outfit by the use of smartphones and indoor navigation.

And even at home - also with the help of smartphones and mirrors - such a system provides valuable decision-making aids. In this way, it creates needs from which the stationary retail business may benefit: Anyone who knows in advance where they can find a particular outfit will be happy to go to the store and try it on, without desperately searching a different number of stores. An app is already obligatory for an online shop today, and for this reason, the solution developed by Arvato Systems can also run on Android and the iOS operating system.

Comfortable from the sofa: Consulting on the Smart-TV

Modern TV sets can do far more than the one-way irrigation of earlier times: Today, most of them are "smart" and internet-enabled. Your significant advantage: The considerable bigger screens when compared to smartphones and tablets gives you as a customer a better idea of whether the proposed outfit suits your type. Even if the TV set lacks the necessary camera, your photo can still be uploaded via another Internet-enabled device. This software solution also works with standard web browsers and does not require any installation.

In action on site: POS terminals with AI-supported purchasing advice

Almost every significant shopping mile now has terminals that show visitors the way to the right shops. However, most and especially smaller retail stores have so far lacked such a solution: So why not set up a terminal there as well and offer digital shopping advice in addition to the store information? In this way, customers may find what they are looking for faster, which increases store satisfaction because the customers do not have to wait for a freelance salesperson.

With the help of a personal advice card or an NFC-enabled smartphone, the customer could identify himself again at a later point in time and find out which new outfits might suit him. The Arvato-Systems solution has a flexible structure and can be used both in a protected web browser and as a native, individually adapted app solution under Windows or Linux.

Guide to the right department: The robot shows the way

In Japan, futuristic sales advice has already found its way into everyday life: In some department stores, Pepper robots such as our Pepper L.I.S.A. are already prominent. There they show customers the way to the right department - but functions that go beyond this are rare. A robot as a shopping assistant would be the next step. The robot would remain with the customer for the duration of the purchase and provides the appropriate, AI-supported (and thus neutral) feedback for each outfit that has just been tried on.

At the Munich Airport, another Pepper named Josie is already helping passengers find their way to the right terminal. This type of functionality could also be realized: At Arvato Systems, the predecessor of L.I.S.A., Roberto, has already guided one or two customers through the large in-house data center.

Big data analysis with the Microsoft Azure Cloud

With the help of Microsoft's Azure Cloud system and its modules, vast amounts of data can be analyzed in a matter of seconds and converted into big data models. As a result, new data sets of unknown content can be examined and evaluated without noticeable delay for the user. That is made possible by Hyperscale methods, among others. Not dozens, but even thousands or millions of servers are used for the implementation. That makes it possible to obtain available analysis results from vast amounts of data and their complexity, especially with image or video material, in the shortest possible time.