Establishing a mature MLOps practice to build and operationalize ML systems can take years to get right.  We recently published our MLOps framework to help organizations come up to speed faster in this important domain.  

As you start your MLOps journey, you might not need to implement all of these processes and capabilities. Some will have a higher priority than others, depending on the type of workload and business value that they create for you, balanced against the cost of building or buying processes or capabilities. 

To help ML practitioners translate the framework into actionable steps, this blog post highlights some of the factors that influence where to begin, based on our experience in working with customers. 

The following table shows the recommended capabilities (indicated by check marks) based on the characteristics of your use case, but remember that each use case is unique and might have exceptions. (For definitions of the capabilities, see the MLOps framework.)

Your use case might have multiple characteristics. For example, consider a recommender system that’s retrained frequently and that serves batch predictions. In that case, you need the data processing, model training, model evaluation, ML pipelines, model registry, and metadata and artifact tracking capabilities for frequent retraining. You also need a model serving capability for batch serving.

In the following sections, we provide details about each of the characteristics and the capabilities that we recommend for them.

Pilot

Example: A research project for experimenting with a new natural language model for sentiment analysis.

For testing a proof of concept, your focus is typically on data preparation, feature engineering, model prototyping, and validation. You perform these tasks using the experimentation and data processing capabilities. Data scientists want to set up experiments quickly and easily and track and compare them. Therefore, you need the ML metadata and artifact tracking capability in order to debug, to provide traceability and lineage, to share and track experimentation configurations, and to manage ML artifacts. For large-scale pilots, you might also require dedicated model training and evaluation capabilities.

Mission-critical

Example: An equities trading model where model performance degradation in production can put millions of dollars at stake.

In a mission-critical use case, failure with the training process or production model has a significant negative impact on the business (a legal, ethical, reputational, or financial risk). The model evaluation capability is important to identify bias and fairness, as well as to provide explainability of the model. Additionally, monitoring is essential to assess the quality of the model during training and to assess how it performs in production. Online experimentation lets you test newly trained models against the one in production using a controlled environment before you replace the deployed model. Such use cases also need a robust model governance process to store, evaluate, check, release, and report on models and to protect against risks. You can enable model governance by using the model registry and metadata and artifact tracking capabilities. Additionally, datasets and feature repositories provide you with high-quality data assets that are consistent and versioned.

Reusable and collaborative

Example: Customer Analytic Record (CAR) features that are used across various propensity modeling use cases.

Reusable and collaborative assets allow your organization to share, discover, and reuse AI data, source code, and artifacts. A feature store helps you standardize the processes of registering, storing, and accessing features for training and serving ML models. Once features are curated and stored, they can be discovered and reused by multiple data science teams. Having a feature store helps you avoid reengineering features that already exist, and saves time on experimentation. You can also use tools to unify data annotation and categorization. Finally, by using ML metadata and artifacts tracking, you help provide consistency, testability, security and repeatability of the ML workflows. 

Ad hoc retraining

Example: An object detection model to detect various car parts, which needs to be retrained only when new parts are introduced. 

In ad hoc retraining, models are fairly static and you do not retrain them except when the model performance degrades. In these cases, you need data processing, model training, and model evaluation capabilities to train the models. Additionally, because your models are not updated for long periods, you need model monitoring. Model monitoring detects data skews, including schema anomalies, as well as data and concept drifts and shifts. Monitoring also lets you continuously evaluate your model performance, and it alerts you when performance decreases or when data issues are detected. 

Frequent retraining

Example: A fraud detection model that’s trained daily in order to capture recent fraud patterns.

Use cases for frequent retraining are ones where model performance relies on changes in the training data. The retraining might be based on time intervals (for example, daily or weekly), or it could be triggered based on events like when new training data becomes available. For this scenario, you need ML pipelines to connect multiple steps like data extraction, preprocessing, and model training. You also need the model evaluation capability to ensure that the accuracy of the newly trained model meets your business requirements. As the number of models you train grows, both a model registry and metadata and artifact tracking help you keep track of the training jobs and model versions.

Frequent implementation updates

Example: A promotion model with frequent changes to the architecture to maximize conversion rate.

Frequent implementation updates involve changes to the training process itself. That might mean switching to a different ML framework, such as changing the model architecture (for example, LSTM to Attention) or adding a data transformation step in your training pipeline. Such changes in the foundation of your ML workflow require controls to ensure that the new code is functional and that the new model matches or outperforms the previous one. Additionally, the CI/CD process accelerates the time from ML experimentation to production, as well as reducing the possibility for human error. Because the changes are significant, online experimentation is necessary to ensure that the new release is performing as expected. You also need other capabilities such as experimentation, model evaluation, model registry, and metadata and artifact tracking to help you operationalize and track your implementation updates. 

Batch serving

Example: A model that serves weekly recommendations to a user who has just signed up for a video-streaming service.

For batch predictions, there is no need to score in real time. You precompute the scores and you store them for later consumption, so latency is less of a concern than in online serving. However, because you process a large amount of data at a time, throughput is important. Often batch serving is a step in a larger ETL workflow that extracts, pre-processes, scores, and stores data. Therefore, you need the data processing capability and ML pipelines for orchestration. In addition, a model registry can provide your batch serving process with the latest validated model to use for scoring.

Online serving

Example: A RESTful microservice that uses a model to translate text between multiple languages. 

Online inference requires tooling and systems in order to meet latency requirements. The system often needs to retrieve features, to perform inference, and then to return the results according to your serving configurations. A feature repository lets you retrieve features in near real time, and model serving allows you to easily deploy models as an endpoint. Additionally, online experiments help you test new models with a small sample of the serving traffic before you roll the model out to production (for example, by performing A/B testing).

Get started with MLOps using Vertex AI

We recently announced Vertex AI, our unified machine learning platform that helps you implement MLOps to efficiently build and manage ML projects throughout the development lifecycle. You can get started using the following resources: 

• MLOps: Continuous delivery and automation pipelines in machine learning
• Getting started with Vertex AI
• Best practices for implementing machine learning on Google Cloud

^{Acknowledgements: I’d like to thank all the subject matter experts who contributed,  including Alessio Bagnaresi, Alexander Del Toro, Alexander Shires, Erin Kiernan, Erwin Huizenga, Hamsa Buvaraghan, Jo Maitland, Ivan Nardini, Michael Menzel, Nate Keating, Nathan Faggian, Nitin Aggarwal, Olivia Burgess, Satish Iyer, Tuba Islam, and Turan Bulmus. A special thanks to the team that helped create this, Donna Schut, Khalid Salama, and Lara Suzuki, and Mike Pope for his ongoing support.}

In our conversations with customers, few generative AI use cases have driven as much enthusiasm as generative search. Leaders at enterprises know the limits of traditional enterprise search, with queries producing a list of links based on pattern matching, and significant manual investigation required to find the more relevant answers. In generative AI, these leaders see an opportunity to leverage their data more effectively and deeply, including applying it to conversational apps that can answer complex questions, produce accurate summaries that synthesize many sources, and help people get the information they need, faster. 

Enterprise Search on Generative AI App Builder (Gen App Builder) lets organizations create custom chatbots and semantic search applications in as little as a few minutes, with minimal coding needed to get started and enterprise-grade management and security built in. Customers can combine their internal data with the power of Google’s search technologies and generative foundation models, delivering relevant, personalized search experiences for enterprise applications or consumer-facing websites. 

To date, most approaches to combining generative AI and search technology have been inadequate for the scale and reliability needed for enterprise use. For example, building search by breaking long documents into chunks and feeding each segment into an AI assistant typically isn’t scalable and doesn’t effectively provide insights across multiple sources. Likewise, many solutions are limited in the data types they can handle, prone to errors, and susceptible to data leakage. Bespoke, do-it-yourself approaches are generally no easier, with production-grade solutions often requiring complex tasks like integrating embeddings data with foundation models and significant use case data testing. Even when organizations make these efforts, the resulting solutions still tend to lack feature completeness and reliability, with significant investments of time and resources required to achieve high-quality results.

These challenges demonstrate that to effectively implement generative search, organizations typically need more than access to powerful foundation models. They also likely need the ability to ground model outputs in specific data, so that outputs are more relevant and less likely to include mistakes or “hallucinations.” They generally need safeguards that protect their data, how it is accessed, and how it is used. And they generally need the process to be high-performant and scalable out of the box, making the functionality easy to use even if the organization lacks data science and machine learning expertise.

Let’s look at how Enterprise Search on Gen App Builder helps customers bypass these scale and reliability challenges, so they can start leveraging generative search quickly. 

The intersection of generative AI and enterprise data 

Enterprise Search on Gen App Builder lets developers create search engines that help ground outputs in specific data sources for accuracy and relevance, can handle multimodal data such as images, and include controls over how answer summaries are generated. Multi-turn conversations are supported so that users can ask follow up questions as they peruse outputs, and customers have control over their data—including the ability to support HIPAA compliance for healthcare use cases. All of this is available as a fully managed service, so developers can focus on building rather than cloud complexity. 

Gen App Builder’s out-of-box capabilities can remove the need for data chunking, generating embeddings, or managing indexes, hiding complexity behind a straightforward interface that lets developers build apps in minutes with little or no coding and no prior machine learning experience. With the ability to ingest large volumes of documents and support for both unstructured and structured data, apps built with Gen App Builder help customers solve the long-standing headache of finding relevant information across the organization, turning tasks that used to take hours into quick searches or conversational explorations with an app. 

These capabilities are underpinned by both Google’s foundation models and a variety of Google Search technologies, including:

• Semantic search, which helps deliver more relevant results than traditional keyword-based search techniques by using natural language processing and machine learning techniques to infer relationships within the content and intent from the user’s query input. 
• Google’s understanding of how users search for information. 
• Google’s expertise in understanding relevance, which considers factors such as content popularity and user personalization of content when determining the order in which search results are displayed. 

For more advanced use cases, Gen App Builder can be easily integrated with Vertex AI for in-depth foundation model tuning, enabling flexible input and output search formats. 

As always with our AI products, Gen App Builder has been evaluated for alignment with our AI Principles, which is reflected by the product’s many safeguards against bias, toxic content, and unhelpful outputs. Whether for prototypes built in minutes or apps with many custom components, Enterprise Search on Gen App Builder offers a robust suite of user-friendly tools for customers across industries and levels of expertise. 

How customers are innovating with Enterprise Search 

After gaining access to Enterprise Search on Gen App Builder via our trusted tester program, a number of customers are already leveraging the product for novel use cases. 

Priceline is harnessing Gen App Builder and Vertex AI for a range of projects, including internal search engines for employees and a new chatbot to assist customers as they make travel plans. Slated to be available across both desktop and mobile experiences, Priceline’s chatbot will help customers find the right information faster via always-on, personalized experiences, including answering nuanced questions like, “What are the best 4-star hotel options in midtown Manhattan within walking distance to Central Park?” and “Can you help me extend my hotel reservation for an additional night?”

“Priceline is charting a course to transform the novelty of generative AI into lasting value for our customers and our business. We believe it’s not just about having the latest technology; it’s also about practically targeting innovation to the right challenges and opportunities,” said Marty Brodbeck, Chief Technology Officer, Priceline. “With Google Cloud as our AI innovation partner, we’re doubling down on our commitment to delivering the fastest, most seamless and informative booking experience for our customers, from personalized planning and travel inspiration to customer service.”

Vodafone is experimenting with Enterprise Search and foundation models on Gen App Builder to build a tool that can rapidly and securely query documents, search, and understand specific commercial terms and conditions. Vodafone Voice and Roaming Services has over 10,000 contracts with other telecommunications companies worldwide, in a variety of formats such as PDFs, images, and complex tables. Searching this document repository is often a time consuming process for employees. 

“Every day we introduce and manage new services like 5G to a roaming footprint comprising more than 700 operators in 210 countries to ensure both Vodafone and our partners’ business customers, holidaymakers, and Internet of Things devices stay connected when abroad. With Enterprise Search on Gen App Builder, we are building an intelligent assistant to securely and quickly search contracts.” said Sherif Bakir, CEO of Vodafone Voice and Roaming Services. “With generative AI, we’re accelerating otherwise protracted processes, increasing productivity and operational efficiency.”

Software startup Trender.ai, which creates customer monitoring and intelligence solutions for B2B relationships, is using Gen App Builder to build a product that can synthesize information from social media, public sources, and CRM data so that users can form more productive, personal relationships with prospects and customers.

“With Enterprise Search on Gen App Builder, we have been able to do things in the last month that we had projected to take 12-18 months on our prior roadmap,” said Betsy Bilhorn, co-founder at Trender.ai. “We had expected it would take at least 12 months to build and train models from an individual’s public social, web, and other data, and to then be able to ask questions like ‘What things are most important to this person?’ or ‘When and how is the best way to make an introduction to this prospect that they’ll respond to?’ Last year, we thought getting to this vision was a bit of a moonshot for a startup our size — but now we were able to achieve this in under a month.” 

Stop searching for solutions — start discovering insights

We’re excited to see how customers use Enterprise Search on Gen App Builder to leverage data in powerful ways, discover new insights, and create useful, personal, and efficient experiences. 

Today, we’re pleased to help our customers accomplish these goals, with the general availability of Enterprise Search on Gen App Builder for customers on the allowlist (i.e., approved for access). Please contact your Google Cloud sales team for access and pricing details. We are also launching two new features within Enterprise Search on Gen App Builder that are available today in our preview offering: multi-turn search, which supports asking follow-up questions, and content recommendations to find semantically relevant content. Access to the preview offering is available via Google Cloud’s trusted tester program. Read more about Enterprise Search on Gen App Builder and sign up for access on our webpage. 

We’re also bringing generative AI features of Enterprise Search to our existing solutions like Contact Center AI and Document AI. As an example, starting this month, customers can preview generative AI-powered search in Document AI Warehouse. To keep up with our latest generative AI news, don’t miss The Prompt or our generative AI primer for executives on Transform with Google Cloud.

In the last year, there’s been an upward trend in cloud adoption in India. In fact, NASSCOM finds that cloud spending in India is estimated to grow at 30% per annum to cross the US$7 billion mark by 2022.

At Google, in our conversations with customers, discussions have evolved beyond cost savings and efficiencies. While those are still very relevant reasons for adopting cloud technologies, Indian enterprises are looking to Google Cloud to help them drive digital transformation, identify new revenue generating business models, reach previously untapped consumer markets, and build customer loyalty through greater insight and personalization.

Here are some companies and their stories.

Tata Steel: Mining data and maximizing its power

Tata Steel is a great example of an established enterprise from a traditional industry that is modernizing and embracing cloud computing. With an ambition to be a leader in manufacturing in India and a digital-first organization by 2022, Tata Steel believes smart analytics is key to enhancing operational efficiency and gaining business advantage. 

To organize data from siloed systems across the organization and make it easily accessible to all employees, Tata Steel is using Cloud Search and plans to scale it to more than one million documents and 28 disparate enterprise content sources including enterprise resource planning (ERP) and SharePoint. In fact, Tata Steel is one of the first Indian enterprises to harness the power of Cloud Search to meet some of the most aggressive ingestion demands, with indexing durations reduced from weeks to seconds.

They are also leveraging Google Cloud Platform (GCP) services like Google Cloud Storage and BigQuery to build their data lake and enterprise data warehouse so they can take advantage of advanced analytics and machine learning. Managed services such as AI Platform further enable Tata Steel to manage end-to-end AI/ML workflows within the GCP console. This complements their existing on-premise reporting and analytics tools, and brings data management to the forefront of everything they do—from forecasting market demand to predictive equipment maintenance.

“Digital is not just a goal, it’s become a way of life. We are digitizing everything from the deployment of factory vehicles to improving material throughput to marketing and sales. As a result, we have petabytes of structured and unstructured data that is not only waiting to be mined, but that we can generate intelligence from to create opportunities across our multiple lines of business using GCP,” said Sarajit Jha, Chief Business Transformation & Digital Solutions at Tata Steel.

Helping L&T Financial Services reach customers in rural communities

In rural communities, quick access to financial services can make a tremendous difference to livelihoods. L&T Financial Services provides farm-equipment finance, micro loans and two-wheeler finance to consumers across rural India backed by a strong digital and analytics platform. Their digital-loan approval app, which runs on GCP, makes it significantly faster and easier for people to apply for financial assistance to purchase important things such as farming equipment and two-wheelers. It also helps rural women entrepreneurs get quicker access to funds for their businesses through micro loans.

L&T Financial found G Suite to be a far better collaborative tool to help staff work together efficiently. Employees can interact with each other in real time using Hangouts Meet, and the task of information sharing is more seamless and secure through Drive. BigQuery also helps L&T Financial Services generate behavior scorecards to track credit quality of its micro-loan customers.

“Cloud is the technology that enables us to achieve scale and reach. Today there are countless data points available about rural consumers which enable us to personalize our products to serve them better. With access to faster compute power, we can also on-board consumers more efficiently. Our rural businesses have clocked a disbursement CAGR of 60% over the past three years.” said Sunil Prabhune, Chief Executive-Rural Finance, and Group Head-Digital, IT and Analytics, L&T Financial Services.

Creating conversational connections for Digitate’s customers

Digitate, a venture of TCS (Tata Consultancy Services), has integrated Dialogflow into its flagship brand ignio, an award-winning artificial intelligence platform for driving IT operations, workload operations and ERP operations for diverse enterprises. This integration is the next step in ignio’s product development journey, and will enable users to chat or talk with ignio to detect issues, triage problems, resolve them and even predict system behavior.

“ignio combines its unique self-healing AIOps capabilities for enterprise IT and business operations with Dialogflow’s AI/ML-based, easy to use, natural and rich conversational capabilities to create an unparalleled, intuitive and feature-rich experience for our customers,” says Akhilesh Tripathi, Head of Digitate.

Indian enterprises going G Suite

The base of Indian enterprises that are making the switch to G Suite to streamline their productivity and collaboration also continues to grow. Sharechat, BookMyShow, Hero MotorCorp, DB Corp and Royal Enfield are now able to move faster within their organizations, using intelligent, cloud-based apps to transform the way they work.

A hybrid and multi-cloud future in India

IDC predicts that by 2023, 55% of India 500 organizations will have a multi-cloud management strategy that includes integrated tools across public and private clouds. (IDC FutureScape: Worldwide Cloud 2019 Predictions  — India Implications (# AP43922319). We look forward to sharing more success stories of Indian enterprises that have taken the next step in their digital transformation journey.