Editor’s note: In part one of this blog, Wayfair shared how it supports each of its 30 million active customers using machine learning (ML). Wayfair’s Vinay Narayana, Head of ML Engineering, Bas Geerdink, Lead ML Engineer, and Christian Rehm, Senior Machine Learning Engineer, take us on a deeper dive into the ways Wayfair’s data scientists are using Vertex AI to improve model productionization, serving, and operational readiness velocity. The authors would like to thank Hasan Khan, Principal Architect, Google for contributions to this blog.

When Google announced its Vertex AI platform in 2021, the timing coincided perfectly with our search for a comprehensive and reliable AI Platform. Although we’d been working on our migration to Google Cloud over the previous couple of years, we knew that our work wouldn’t be complete once we were in the cloud. We’d simply be ready to take one more step in our workload modernization efforts, and move away from deploying and serving our ML models using legacy infrastructure components that struggle with stability and operational overhead. This has been a crucial part of our journey towards MLOps excellence, in which Vertex AI has proved to be of great support.

Carving the path towards MLOps excellence

Our MLOps vision at Wayfair is to deliver tools that support the collaboration between our internal teams, and enable data scientists to access reliable data while automating data processing, model training, evaluation and validation. Data scientists need autonomy to productionize their models for batch or online serving, and to continuously monitor their data and models in production. Our aim with Vertex AI is to empower data scientists to productionize models and easily monitor and evolve them without depending on engineers. Vertex AI gives us the infrastructure to do this with tools for training, validating, and deploying ML models and pipelines.

Previously, our lack of a comprehensive AI platform resulted in every data science team having to build their own unique model productionization processes on legacy infrastructure components. We also lacked a centralized feature store, which could benefit all ML projects at Wayfair. With this in mind, we chose to focus our initial adoption of the Vertex AI platform on its Feature Store component.

An initial POC confirmed that data scientists can easily get features from the Feature Store for training models, and that it makes it very easy to serve the models for batch or online inference with a single line of code. The Feature Store also automatically manages performance for batch and online requests. These results encouraged us to evaluate the adoption of Vertex AI Pipelines next, as the existing tech for workflow orchestration at Wayfair slowed us down greatly. As it turns out, both of these services are fundamental to several models we build and serve at Wayfair today.

Empowering data scientists to focus on building world-class ML models

Since adopting Vertex AI Feature Store and AI Pipelines, we’ve added a couple of capabilities at Wayfair to significantly improve our user experience and lower the bar to entry for data scientists to leverage Vertex AI and all it has to offer:

1. Building a CI/CD and scheduling pipeline

Working with the Google team, we built an efficient CI/CD and scheduling pipeline based on the common tools and best practices at Wayfair and Google. This enables us to release Vertex AI Pipelines to our test and production environments, leveraging cloud-native services.

Keeping in mind that all our code is managed in GitHub Enterprise, we have dedicated repositories for Vertex AI Pipelines where the Kubeflow code and definitions of the Docker images are stored. If a change is pushed to a branch, a build starts in the Buildkite tool automatically. The build contains several steps, including unit and integration tests, code linting, documentation generation and automated deployment. The most important artifacts that are released at the end of the build are the Docker image and the compiled Kubeflow template. The Docker image is released to the Google Cloud Artifact Registry and we store the Kubeflow template in a dedicated Google Cloud Storage Bucket, fully versioned and secured. This way, all the components we need to run a Vertex AI Pipeline are available once we run a pipeline (manually or scheduled).

To schedule pipelines, we developed a dedicated Cloud Function that has the permissions to run the pipeline. This Function listens to a Pub/Sub topic where we can publish messages with a defined schema that indicates which pipeline to run with which parameters. These messages are published from a simple cron job that runs according to a set schedule on Google Kubernetes Engine. This way, we have a decoupled and secure environment for scheduling pipelines, using fully-supported and managed infrastructure.

Abstracting Vertex AI services with a shared library

We abstracted the relevant Vertex AI services currently in use with a thin shared Python library to support the teams that develop new software or migrate to Vertex AI. This library, called wf-vertex, contains helper methods, examples, and documentation for working with Vertex AI, as well as guidelines for Vertex AI Feature Store, Pipelines, and Artifact Registry.

One example is the run_pipeline method, which publishes a message with the correct schema to the Pub/Sub topic so that a Vertex AI pipeline is executed. When scheduling a pipeline, the developer only needs to call this method without having to worry about security or infrastructure configuration:

@cli.command()
def trigger_pipeline() -> None:
    from wf_vertex.pipelines.pipeline_runner import run_pipeline

    run_pipeline(
       template_bucket= f"wf-vertex-pipelines-{env}/{TEAM}",  # this is the location of the template, where the CI/CD has written the compiled templates to
       template_filename="sample_pipeline.json",  # this is the filename of the pipeline template to run
       parameter_values= {"import_date": today()}  # it’s possible to add pipeline parameters
)

Most notable is the establishment of a documented best practice for enabling hyperparameter tuning in Vertex AI Pipelines, which speeds up hyperparameter tuning times for our data scientists from two weeks to under one hour.

Because it is not yet possible to combine the outputs of parallel steps (components) in Kubeflow, we designed a mechanism to enable this. It entails defining parameters at runtime and executing the resulting steps in parallel via the Kubeflow parallel-for operator. Finally, we created a step to combine the results of these parallel steps and interpret the results. In turn, this mechanism allows us to select the best model in terms of accuracy from a set of candidates that are trained in parallel:

Our CI/CD, scheduling pipelines, and shared library have reduced the effort of model productionization from more than three months to about four weeks. As we continue to build the shared library, and as our team members continue to gain expertise in using Vertex AI, we expect to further reduce this time to two weeks by the end of 2022.

Looking forward to more MLOps capabilities

Looking ahead, our goal is to fully leverage all the Vertex AI features to continue modernizing our MLOps stack to a point where data scientists are fully autonomous from engineers for any of their model productionization efforts. Next on our radar are Vertex AI Model Registry and Vertex ML Metadata alongside making more use of AutoML capabilities. We’re experimenting with Vertex AI for AutoML models and endpoints to benefit some use cases at Wayfair next to the custom models that we’re currently serving in production.

We’re confident that our MLOps transformation will introduce several capabilities to our team, including: automated data and model monitoring steps to the pipeline, as well as metadata management, and architectural patterns in support of real-time models requiring access to Wayfair’s network. We also look forward to performing continuous training of models by fully automating the ML pipeline that allows us to achieve continuous integration, delivery, and deployment of model prediction services.

We’ll continue to collaborate and invest in building a robust Wayfair-focused Vertex AI shared library. The aim is to eventually migrate 100% of our batch models to Vertex AI. Great things to look forward to on our journey towards MLOps excellence.

“Cloud Wisdom Weekly: for tech companies and startups” is a new blog series we’re running this fall to answer common questions our tech and startup customers ask us about how to build apps faster, smarter, and cheaper. In this installment, we explore how to leverage artificial intelligence (AI) and machine learning (ML) for faster innovation and efficient operational growth.

Whether they’re trying to extract insights from data, create faster and more efficient workflows via intelligent automation, or build innovative customer experiences, leaders at today’s tech companies and startups know that proficiency in AI and ML is more important than ever.

AI and ML technologies are often expensive and time-consuming to develop, and the demand for AI and ML experts still largely outpaces the existing talent pool. These factors put pressure on tech companies and startups to allocate resources carefully when considering bringing AI/ML into their business strategy. In this article, we’ll explore four tips to help tech companies and startups accelerate innovation and reduce costs with AI and ML.

4 tips to accelerate innovation and reduces costs with AI and ML

Many of today’s most innovative companies are creating services or products that couldn’t exist without AI—but that doesn’t mean they’re building their AI and ML infrastructure and pipelines from scratch. Even for startups whose businesses don’t directly revolve around AI, injecting AI into operational processes can help manage costs as the company grows. By relying on a cloud provider for AI services, organizations can unlock opportunities to energize development, automate processes, and reduce costs.

1. Leverage pre-trained ML APIs to jumpstart product development

Tech companies and startups want their technical talent focused on proprietary projects that will make a difference to the business. This often involves the development of new applications for an AI technology, but not necessarily the development of the AI technology itself. In such scenarios, pre-trained APIs help organizations quickly and cost-effectively establish a foundation on which higher-value, more differentiated work can be layered.

For example, many companies building conversational AI into their products and services leverage Google Cloud APIs such as Speech-to-Text and Natural Language. With these APIs, developers can easily integrate capabilities like transcription, sentiment analysis, content classification, profanity filtering, speaker diarization, and more. These powerful technologies help organizations focus on creating products rather than having to build the base technologies.

See this article for examples of why tech companies and startups have chosen Google Cloud’s Speech APIs for use cases that range from deriving customer insights to giving robots empathetic personalities. For an even deeper dive, see

• our AI product page to explore other APIs, including Translation, Vision, and more;
• and the Google Cloud Skills Boost for ML APIs.

2. Use managed services to scale ML development and accelerate deployment of models to production

Pre-trained models are extremely useful, but in many cases, tech companies and startups need to create custom models to either derive insights from their own data or to apply new use cases to public data. Regardless of whether they’re building data-driven products or generating forecasting models from customer data, companies need ways to accelerate the building and deployment of models into their production environments.

A data scientist typically starts a new ML project in a notebook, experimenting with data stored on the local machine. Moving these efforts into a production environment requires additional tooling and resources, including more complicated infrastructure management. This is one reason many organizations struggle to bring models into production and burn through time and resources without moving the revenue needle.

Managed cloud platforms can help organizations transition from projects to automated experimentation at scale or the routine deployment and retraining of production models. Strong platforms offer flexible frameworks, fewer lines of code required for model training, unified environments across tools and datasets, and user-friendly infrastructure management and deployment pipelines.

At Google Cloud, we’ve seen customers with these needs embrace Vertex AI, our platform for accelerating ML development, in increasing numbers since it launched last year. Accelerating time to production by up to 80% compared to competing approaches, Vertex AI provides advanced end-to-end ML Ops capabilities so that data scientists, ML engineers, and developers can contribute to ML acceleration. It includes low-code features, like AutoML, that make it possible to train high performing models without ML expertise.

Over the first half of 2022, our performance tests found that the number of customers utilizing AI Workbench increased by 25x. It’s exciting to see the impact and value customers are gaining with Vertex AI Workbench, including seeing it help companies speed up large model training jobs by 10x and helping data science teams improve modeling precision from the 70-80% range to 98%.

If you are new to Vertex AI, check out this video series to learn how to take models from prototype to production. For deeper dives, see

• this article about Vertex AI’s role in an ambitious project to measure climate change with AI;

• BigQuery has built-in Machine Learning (ML) and Analytics that you can use to create no-code predictions using just SQL queries.
• this blog about how Vertex AI and BigQuery work together to make data analysis easier and more powerful;
• and this blog about Example-based explanations, one of our most recent updates to make model iteration more intuitive and efficient.

3. Harness the cloud to match hardware to use cases while minimizing costs and management overhead

ML infrastructure is generally expensive to build, and depending on the use case, specific hardware requirements and software integrations can make projects costly and complicated at scale. To solve for this, many tech companies and startups look to cloud services for compute and storage needs, attracted by the ability to pay only for resources they use while scaling up and down according to changing business needs.

At Google Cloud, customers share that they need the ability to optimize around a variety of infrastructure approaches for diverse ML workloads. Some use Central Processing Units (CPUs) for flexible prototyping. Others leverage our support for NVIDIA Graphics Processing Units (GPUs) for image-oriented projects and larger models, especially those with custom TensorFlow operations that must run partially on CPUs. Some choose to run on the same custom ML processors that power Google applications—Tensor Processing Units (TPUs). And many use different combinations of all of the preceding.

Beyond matching use cases to the right hardware and benefiting from the scale and operational simplicity of a managed service, tech companies and startups should explore configuration features that help further control costs. For example, Google Cloud features like time-sharing and multi-instance capabilities for GPUs — as well as features like Vertex AI Training Reduction Server — are built to optimize GPU costs and usage.

Vertex AI Workbench also integrates with the NVIDIA NGC catalog for deploying frameworks, software development kits and Jupyter Notebooks with a single click—another feature that, like Reduction Server, speaks to the ways organizations can make AI more efficient and less costly via managed services.

4. Implement AI for operations

Besides using pre-trained APIs and ML model development to develop and deliver products, startup and tech companies can improve operational efficiency, especially as they scale, by leveraging AI solutions built for specific business and operational needs, like contract processing or customer service.

Google Cloud’s DocumentAI products, for instance, apply ML to text for use cases ranging from contract lifecycle management to mortgage processing. For businesses whose customer support needs are growing, there’s Contact Center AI, which helps organizations build intelligent virtual agents, facilitate handoffs as appropriate between virtual agents and human agents, and generate insights from call center interactions. By leveraging AI to help manage operational processes, startups and tech companies can allocate more resources to innovation and growth.

Next steps toward an intelligent future

The tips in this article can help any tech company or startup find ways to save money and boost efficiency with AI and ML. You can learn more about these topics by registering for Google Cloud Next, kicking off October 11, where you’ll hear Google Cloud’s latest AI news, discussions, and perspectives—in the meantime, you can also dive into our Vertex AI quickstarts and BigQuery ML tutorials. And for the latest on our work with tech companies and startups, be sure to visit our Startups page.