Whether they meet customers online, offline, or in some combination, retailers share a big problem: How can they offer the right choices, when and how the customer wants, without overwhelming (and often losing) the buyer?

More than anything, this is an information problem. As such, it’s a good candidate for using artificial intelligence (AI) for greater success. Here’s how Richemont tackled the problem.

Richemont owns a portfolio of leading luxury goods brands, recognized for their distinctive heritage, craftsmanship and creativity. It has strengths and specialties in jewelry (Cartier, Van Cleef & Arpels), luxury watches (IWC, Jaeger-LeCoultre, Panerai, Vacheron Constantin), and fashion & accessories (Chloé, Montblanc, dunhill).

People shop for such goods in a number of ways, from online searching to individual meetings in boutiques, and Richemont must be prepared for every context. Understanding which shoppers are likely to buy or repurchase, when to engage directly, and what creation to suggest enables sales associates to spend quality time with clients, engaging at the right time with meaningful advice. Richemont solves these retail challenges with an integrated Client Platform leveraging Google Cloud and its AI/ML capabilities.

Enticing peoples’ desires with Machine Learning

Richemont began by posing two questions:

1. Which prospects or clients need extra attention? Specifically, who is likely to convert or to repurchase?
2. What would be meaningful items to suggest to each client and prospect?

Both questions were addressed with machine learning algorithms. Their challenges included deploying and monitoring algorithms at scale for several brands across the globe, while addressing the specific business needs for each brand. For instance, it may be more relevant to recommend in-season items for fashion brands, while for watchmakers it is more about cross-fertilization across each brand’s iconic creations.

This graph summarizes the prediction process implemented by Richemont:

Engagement data (email opened, clicked, SMS/MMS, website visits…) was found crucial to predict conversion of prospects for whom per definition no transaction history is available. For website interactions Richemont leverages the Google x Salesforce Connector.

To deploy the Machine Learning algorithms and to monitor them, Richemont leveraged Vertex AI, along with BigQuery, Cloud Functions and Google Storage, all orchestrated with Google Cloud Composer.

The role of product recommendation algorithms

Richemont used the deep learning library TensorFlow Recommenders to perform the product recommendation tasks. This library enables companies to build state of the art deep learning algorithms to achieve relevant and robust predictions.

Unlocking client value with integrated technology

Richemont’s innovations show how technology that considers many parts of the customer experience creates more value. In this case, the company used in store applications to invite people with a strong propensity to buy for boutique visits, while others at a different point in the purchasing journey were offered different options more suited to their tastes and inclinations.This solution, now deployed across 11 brands in over 25 countries, shows just one way that AI can improve customer experience, for better customer loyalty.

Key to the process, here and elsewhere, is the way a retailer and its partners put customer understanding at the center of the process. As AI becomes more important not only in retail, but in every industry, this human understanding will become even more important as a fundamental organizing principle. Much is changing, but once again, the winners will be the companies that focus best on their customers.

With the advent of the pandemic, contact center traffic has increased by as much as 300%, taxing center capabilities. To help handle the surge, and keep up with heightened customer demands, many customer experience providers have deployed automation in the form of virtual agents that serve as the first—and, sometimes the last—line of customer support. How effective these virtual agents are in delivering timely, complete customer service depends in large part on the contact center’s infrastructure and the power of the automation solution.

Customer support providers can now start to reimagine customer experience with Conversational AI and take advantage of the Google Cloud-Automation Anywhere partnership to modernize their operations. 

Bringing together the Google Cloud Contact Center AI (CCAI) solution and the Automation Anywhere cloud-native Automation 360 platform, the partnership helps contact centers remain competitive by improving their performance metrics and customer satisfaction to exceed the continuously growing expectations.

Information here, there, anywhere—with limited access

Over the years, many contact centers have accumulated a multitude of systems, often creating a disconnected infrastructure. As a result, both human and virtual agents alike are challenged to keep up with growing customers’ expectations for timely, accurate, and complete service. Without up-to-date software and infrastructure, human agents have to perform a “swivel-chair” maneuver, logging in to the different systems, sifting through records, copying the needed information, and deciding what the next action should be. This approach is not conducive to achieving lower average handle times (or AHT), reduced processing errors, or increased customer satisfaction.

In such a siloed environment, virtual agents may not connect to all the relevant data systems and applications. That also limits what they can do to support human agents. Typically, a virtual agent can handle basic customer requests, such as providing banking customers with their account balances. For more complex requests, such as applying for a line of credit, a virtual agent still must transfer customers to human agents, and the swivel-chair maneuver begins. 

Enter the RPA-assisted AI-powered virtual agent

The combination of Google Cloud Contact Center AI and Automation Anywhere’s Automation 360 RPA platform can help contact centers get the maximum benefit from utilizing virtual agents, enriching customer engagements. Further, by integrating automation, opening up APIs, and creating new processes for virtual agents, customer experience teams can help streamline operations by enabling users to quickly access the information they require. This helps minimize the need for the “swivel-chair” maneuver.

Automation 360 makes it possible for the CCAI virtual agents to access all systems and applications in both legacy and modern infrastructure, helping resolve every case quickly and easily. Not only can the virtual agents respond faster with answers, but they can complete more complex end-to-end requests. With RPA, customers are reporting 66% improved efficiency of contact center operations while exceeding their AHT reduction goals. 

This video illustrates how such automation can dramatically reduce the processing time of a customer service request, and you can read more about the details of contact center automation, as well.

Additionally, a comprehensive development platform, the Google Cloud CCAI Dialogflow, powers virtual agents—chatbots and voicebots—with Conversational AI to deliver lifelike customer experiences anytime a customer reaches out to a brand. CCAI Agent Assist helps human agents with turn-by-turn guidance, ready-to-deliver answers, and ready-to-send responses. CCAI Insights identifies key metrics such as call drivers and customer sentiment for workflow optimization. 

Living up to their potential

TELUS International, a leading digital customer experience provider and long-time partner of both Google Cloud and Automation Anywhere, has been leveraging virtual agents to enhance the employee and customer experience.  

“We are very proud of the enormous value that we can provide our clients through combining our expertise in customer experience and digital transformation alongside the innovative solutions of our technology partners,” Jim Radzicki, CTO of TELUS International, explains. “For instance, through leveraging Google Cloud Contact Center AI and Automation Anywhere’s RPA integration, we are able to expand the capabilities of virtual agents to process a wider variety of customer requests while allowing our team members to focus on the most critical conversations and creating a meaningful connection with every customer.” 

With CCAI and Automation 360, virtual agents can help contact centers deliver 24/7, comprehensive, accurate service, all of which helps eliminate wait times—even with heavy traffic. And this is just the start. 

With deeper integration planned between Automation 360 and CCAI, RPA can augment CCAI Agent Assist’s abilities to help human agents by bringing untapped case-sensitive information to their fingertips. Furthermore, Automation 360 Bot Insight can complement CCAI Insights’ as well as TELUS International’s Intelligent Insights, a tool-agnostic platform to monitor and manage RPA solutions and bots, with backend data access metrics.

At Google Cloud and Automation Anywhere, we’ll continue developing our contact center solution to extend automation capabilities for better customer service and greater customer satisfaction. Instead of constantly swiveling, agents can once again be at the center of the call center action, taking their work—and the experience of their customers—to the next level.

When you build a Machine Learning (ML) product, consider at least two MLOps scenarios. First, the model is replaceable, as breakthrough algorithms are introduced in academia or industry. Second, the model itself has to evolve with the data in the changing world.

We can handle both scenarios with the services provided by Vertex AI. For example:

• AutoML capability automatically identifies the best model based on your budget, data, and settings.
• You can easily manage the dataset with Vertex Managed Datasets by creating a new dataset or adding data to an existing dataset.
• You can build an ML pipeline to automate a series of steps that start with importing a dataset and end with deploying a model using Vertex Pipelines.

This blog post shows you how to build this system. You can find the full notebook for reproduction here. Many folks focus on the ML pipeline when it comes to MLOps, but there are more parts to building MLOps as a “system”. In this post, you will see how Google Cloud Storage (GCS) and Google Cloud Functions manage data and handle events in the MLOps system.

Architecture

Figure 1 shows the overall architecture presented in this blog. We cover the components and their connection in the context of two common workflows of the MLOps system.

Components

Vertex AI is at the heart of this system, and it leverages Vertex Managed Datasets, AutoML, Predictions, and Pipelines. We can create and manage a dataset as it grows using Vertex Managed Datasets. Vertex AutoML selects the best model without your knowing much about modeling. Vertex Predictions creates an endpoint (RestAPI) to which the client communicates.

It is a simple, fully managed yet somewhat complete end-to-end MLOps workflow moves from a dataset to training a model that gets deployed. This workflow can be programmatically written in Vertex Pipelines. Vertex Pipelines outputs the specification for an ML pipeline allowing you to re-run the pipeline whenever or wherever you want. Specify when and how to trigger the pipeline using Cloud Functions and Cloud Storage.

Cloud Functions is a serverless way to deploy your code in Google Cloud. In this particular project, it triggers the pipeline by listening to changes on the specified Cloud Storage location. Specifically, if a new dataset is added, for example, a new span number is created; the pipeline is triggered to train the dataset, and a new model is deployed.

Workflow

This MLOps system prepares the dataset with either Vertex Dataset’s built-in user interface (UI) or any external tools based on your preference. You can upload the prepared dataset into the designated GCS bucket with a new folder named SPAN-NUMBER. Cloud Functions then detects the changes in the GCS bucket and triggers the Vertex Pipeline to run the jobs from AutoML training to endpoint deployment.

Inside the Vertex Pipeline, it checks if there is an existing dataset created previously. If the dataset is new, Vertex Pipeline creates a new Vertex Dataset by importing the dataset from the GCS location and emits the corresponding Artifact. Otherwise, it adds the additional dataset to the existing Vertex Dataset and emits an artifact.

When the Vertex Pipeline recognizes the dataset as a new one, it trains a new AutoML model and deploys it by creating a new endpoint. If the dataset isn’t new, it tries to retrieve the model ID from Vertex Model and determines whether a new AutoML model or an updated AutoML model is needed. The second branch determines whether the AutoML model has been created. If it hasn’t been created, the second branch creates a new model. Also, when the model is trained, the corresponding component emits the artifact as well.

Directory structure that reflects different distributions

In this project, I have created two subsets of the CIFAR-10 dataset, SPAN-1 and SPAN-2. A more general version of this project can be found here, which shows how to build training and batch evaluation pipelines pipelines. The pipelines can be set up to cooperate so they can evaluate the currently deployed model and trigger the retraining process.

ML Pipeline with Kubeflow Pipelines (KFP)

We chose to use Kubeflow Pipelines to orchestrate the pipeline. There are a few things that I would like to highlight. First, it’s good to know how to make branches with conditional statements in KFP. Second, you need to explore AutoML API specifications to fully leverage AutoML capabilities, such as training a model based on the previously trained one. Last, you also need to find a way to emit artifacts for Vertex Dataset and Vertex Model to consume that Vertex AI can recognize them. Let’s go through these one by one.

Branching strategy

In this project, there are two main conditions and two sub-branches inside the second main branch. The main branches split the pipeline based on a condition if there is an existing Vertex Dataset. The sub-branches are applied in the second main branch, which is selected when there is an exciting Vertex Dataset. It looks up the list of models and decides to train an AutoML model from scratch or a previously trained one.

ML pipelines written in KFP can have conditions with a special syntax of kfp.dsl.Condition. For instance, we can define the branches as follows:

from google_cloud_pipeline_components import aiplatform as gcc_aip


# try to get Vertex Dataset ID
dataset_op = get_dataset_id(...) 

with kfp.dsl.Condition(name="create dataset", 
                       dataset_op.outputs['Output'] == 'None'):
    # Create Vertex Dataset, train AutoML from scratch, deploy model

with kfp.dsl.Condition(name="update dataset", 
                       dataset_op.outputs['Output'] != 'None'):
    # Update existing Vertex Dataset
    ...

    # try to get Vertex Model ID
    model_op = get_model_id(...)

    with kfp.dsl.Condition(name='model not exist',
                           model_op.outputs['Output'] == 'None'):
    # Create Vertex Dataset, train AutoML from scratch, deploy model

    with kfp.dsl.Condition(name='model exist',
                           model_op.outputs['Output'] != 'None'):
        # Create Vertex Dataset, train AutoML based on trained one, deploy model

get_dataset_id and get_model_id are custom KFP components used to determine if there is an existing Vertex Dataset and Vertex Model respectively. Both return “None” if a model is found and some other value if a model isn’t found. They also emit Vertex AI-aware artifacts. You will see what this means in the next section.

Emit Vertex AI-aware artifacts

Artifacts track the path of each experiment in the ML pipeline and display metadata in the Vertex Pipeline UI. When Vertex AI aware artifacts are released into in the pipeline, Vertex Pipeline UI displays links for its internal services such as Vertex Dataset, so that users can visit a web page for more information.

So how could you write a custom component to generate Vertex AI-aware artifacts? To do this, custom components should have Output[Artifact] in their parameters. Then you need to replace the resourceName of the metadata attribute with a special string format.

The following code example is the actual definition of get_dataset_id used in the previous code snippet:

@component(
    packages_to_install=["google-cloud-aiplatform", 
                         "google-cloud-pipeline-components"]
)
def get_dataset_id(project_id: str, 
                     location: str,
                 dataset_name: str,
                 dataset_path: str,
                      dataset: Output[Artifact]) -> str:
    from google.cloud import aiplatform
    from google.cloud.aiplatform.datasets.image_dataset import ImageDataset
    from google_cloud_pipeline_components.types.artifact_types import VertexDataset

    
    aiplatform.init(project=project_id, location=location)
    
    datasets = aiplatform.ImageDataset.list(project=project_id,
                                            location=location,
                                            filter=f'display_name={dataset_name}')
    
    if len(datasets) > 0:
        dataset.metadata['resourceName'] = 
               f'projects/{project_id}/locations/{location}/datasets/{datasets[0].name}'
        return f'projects/{project_id}/locations/{location}/datasets/{datasets[0].name}'
    else:
        return 'None'

As you see, the dataset is defined in the parameters as Output[Artifact]. Even though it appears in the parameter, it is actually emitted automatically. You just need to provide the necessary data as if it is a function variable.

The dataset component retrieves the list of Vertex Dataset by calling the aiplotform.ImageDataset.list API. If the length of it is zero, it simply returns ‘None’. Otherwise, it returns the found resource name of the Vertex Dataset and provides the dataset.metadata[‘resourceName’] with the resource name at the same time. The Vertex AI-aware resource name follows a special string format, which is ‘projects/<project-id>/locations/<location>/<vertex-resource-type>/<resource-name>’.

The <vertex-resource-type>can be anything that points to an internal Vertex AI service. For instance, if you want to specify that the artifact is the Vertex Model, then you should replace <vertex-resource-type> with models. The <resource-name> is the unique ID of the resource, and it can be accessed in the name attribute of the resource found by the aiplatform API. The other custom component, get_model_id, is written in a very similar way as well.

AutoML based on the previous model

You sometimes want to train a new model on top of the previously best model. If that is possible, the new model will probably be much better than the one trained from scratch, because it leverages previously learned knowledge.

Luckily, Vertex AutoML comes with the ability to train a model using a previous model. AutoMLImageTrainingJobRunOp component lets you train a model by simply providing the base_model argument as follows:

training_job_run_op =
gcc_aip.AutoMLImageTrainingJobRunOp(
…,
base_model=model_op.outputs['model'],
…
)

When training a new AutoML model from scratch, you pass ‘None‘ in the base_model argument, and it is the default value. However, you can set it with a VertexModel artifact, and the component will trigger an AutoML training job based on the other model.

One thing to be careful of is that VertexModel artifacts can’t be constructed in a typical way of Python programming That means you can’t create an instance of VertexModel artifact by setting the id found in the Vertex Model dashboard. The only way you can create one is to set the metadata[‘resourceName’] parameters properly. The same rule applies to other Vertex AI-related artifacts such as VertexDataset. You can see how the VertexDataset artifact is constructed properly to get an existing Vertex Dataset to import additional data into it. See the full notebook of this project here.

Cost

You can reproduce the same result from this project with the free $300 credit when you create a new GCP account.

At the time of this blog post, Vertex Pipelines costs about $0.03/run, and the type of underlying VM for each pipeline component is e2-standard-4, which costs about $0.134/hour. Vertex AutoML training costs about $3.465/hour for image classification. GCS holds the actual data, which costs about $2.40/month for 100GiB capacity, and Vertex Dataset is free.

To simulate two different branches, the entire experiment took about one to two hours, and the total cost for this project is approximately $16.59. Please find more detailed pricing information about Vertex AI here.

Conclusion

Many people underestimate the capability of AutoML, but it is a great alternative for app and service developers who have little ML background. Vertex AI is a great platform that provides AutoML as well as Pipeline features to automate the ML workflow. In this article, I have demonstrated how to set up and run a basic MLOps workflow, from data injection to training a model based on the previously-achieved best one, to deploying the model to a Vertex AI platform. With this, we can let our ML model automatically adapt to the changes in a new dataset. What’s left for you to implement is to integrate a model monitoring system to detect data/model drift. One example is found here.