Transcend from Prototype to Production: Train Your ML models with Vertex AI
2620
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
You’re working on a new machine learning problem, and the first environment you use is a notebook. Your data is stored on your local machine, and you try out different model architectures and configurations, executing the cells of your notebook manually each time. This workflow is great for experimentation, but you quickly hit a wall when it comes time to elevate your experiments up to production scale. Suddenly, your concerns are more than just getting the highest accuracy score.
Sound familiar?
Developing production applications or training large models requires additional tooling to help you scale beyond just code in a notebook, and using a cloud service provider can help. But that process can feel a bit daunting.
To make things a little easier for you, we’ve created the Prototype to Production video series, which covers all the foundational concepts you’ll need in order to build, train, scale, and deploy machine learning models on Google Cloud using Vertex AI.
Let’s jump in and see what it takes to get from prototype to production!
Getting started with Notebooks for machine learning
Episode one of this series shows you how to create a managed notebook using Vertex AI Workbench. With your environment set up, you can explore data, test different hardware configurations, train models, and interact with other Google Cloud services.
Storing data for machine learning
When working on machine learning problems, it’s easy to be laser focused on model training. But the data is where it all really starts.
If you want to train models on Vertex AI, first you need to get your data into the cloud. In episode 2, you’ll learn the basics of storing unstructured data for model training and see how to access training data from Vertex AI Workbench.
Training custom models on Vertex AI
You might be wondering, why do I need a training service when I can just run model training directly in my notebook? Well, for models that take a long time to train, a notebook isn’t always the most convenient option. And if you’re building an application with ML, it’s unlikely that you’ll only need to train your model once. Over time, you’ll want to retrain your model to make sure it stays fresh and keeps producing valuable results.
Manually executing the cells of your notebook might be the right option when you’re getting started with a new ML problem. But when you want to automate experimentation at scale, or retrain models for a production application, a managed ML training option will make things much easier.
Episode 3 shows you how to package up your training code with Docker and run a custom container training job on Vertex AI. Don’t worry if you’re new to Docker! This video and the accompanying codelab will cover all the commands you’ll need.
CODELAB: Training custom models with Vertex AI
How to get predictions from an ML model
Machine learning is not just about training. What’s the point of all this work if we don’t actually use the model to do something?
Just like with training, you could execute predictions directly from a notebook by calling model.predict. But when you want to get predictions for lots of data, or get low latency predictions on the fly, you’re going to need something more than a notebook. When you’re ready to use your model to solve a real world problem with ML, you don’t want to be manually executing notebook cells to get a prediction.
In episode 4, you’ll learn how to use the Vertex AI prediction service for batch and online predictions.
CODELAB: Getting predictions from custom trained models
Tuning and scaling your ML models
By this point, you’ve seen how to go from notebook code, to a deployed model in the cloud. But in reality, an ML workflow is rarely that linear. A huge part of the machine learning process is experimentation and tuning. You’ll probably need to try out different hyperparameters, different architectures, or even different hardware configurations before you figure out what works best for your use case.
Episode 5, covers the Vertex AI features that can help you with tuning and scaling your ML models. Specifically, you’ll learn about hyperparameter tuning, distributed training, and experiment tracking.
CODELAB: Hyperparameter tuning on Vertex AI
CODELAB: Distributed Training on Vertex AI
We hope this series inspires you to create ML applications with Vertex AI! Be sure to leave a comment on the videos if you’d like to see any of the concepts in more detail, or learn how to use the Vertex AI MLOps tools.
If you’d like try all the code for yourself, check out the following codelabs:
VCP Peering and Private Endpoints on Vertex AI to Better Security and Predictions in Near Real-time
4496
Of your peers have already read this article.
2:00 Minutes
The most insightful time you'll spend today!
One of the biggest challenges when serving machine learning models is delivering predictions in near real-time. Whether you’re a retailer generating recommendations for users shopping on your site, or a food service company estimating delivery time, being able to serve results with low latency is crucial. That’s why we’re excited to announce Private Endpoints on Vertex AI, a new feature in Vertex Predictions. Through VPC Peering, you can set up a private connection to talk to your endpoint without your data ever traversing the public internet, resulting in increased security and lower latency for online predictions.
Configuring VPC Network Peering
Before you make use of a Private Endpoint, you’ll first need to create connections between your VPC (Virtual Private Cloud) network and Vertex AI. A VPC network is a global resource that consists of regional virtual subnetworks, known as subnets, in data centers, all connected by a global network. You can think of a VPC network the same way you’d think of a physical network, except that it’s virtualized within GCP. If you’re new to cloud networking and would like to learn more, check out this introductory video on VPCs.
With VPC Network Peering, you can connect internal IP addresses across two VPC networks, regardless of whether they belong to the same project or the same organization. As a result, all traffic stays within Google’s network.
Deploying Models with Vertex Predictions
Vertex Predictions is a serverless way to serve machine learning models. You can host your model in the cloud and make predictions through a REST API. If your use case requires online predictions, you’ll need to deploy your model to an endpoint. Deploying a model to an endpoint associates physical resources with the model so it can serve predictions with low latency.
When deploying a model to an endpoint, you can specify details such as the machine type, and parameters for autoscaling. Additionally, you now have the option to create a Private Endpoint. Because your data never traverses the public internet, Private Endpoints offer security benefits in addition to reducing the time your system takes to serve the prediction when it receives the request. The overhead introduced by Private Endpoints is minimal, achieving performance nearly identical to DIY serving on GKE or GCE. There is also no payload size limit for models deployed on the private endpoint.
Creating a Private Endpoint on Vertex AI is simple.
In the Models section of the Cloud console, select the model resource you want to deploy.
Next, select DEPLOY TO ENDPOINT
In the window on the right hand side of the console, navigate to the Access section and select Private. You’ll need to add the full name of the VPC network for which your deployment should be peered.
Note that many other managed services on GCP support VPC peering, such as Vertex Training, Cloud SQL, and Firestore. Endpoints is the latest to join that list.
What’s Next?
Now you know the basics of VPC Peering and how to use Private Endpoints on Vertex AI. If you want to learn more about configuring VPCs, check out this overview guide. And if you’re interested to learn more about how to use Vertex AI to support your ML workflow, check out this introductory video. Now it’s time for you to deploy your own ML model to a Private Endpoint for super speedy predictions!
3139
Of your peers have already downloaded this article
22:30 Minutes
The most insightful time you'll spend today!
Companies everywhere are seeking to leverage the power of AI. And rightly so. The smart applications of AI enable organizations to improve, to scale, and to accelerate the decision-making process across most business functions, so as to work both more efficiently and more effectively. It can also open up new avenues and new revenue streams, providing the organization with an additional competitive edge.
In short, many believe (as we do) that the enterprises that invest in building industry-specific AI solutions today are positioning themselves to be the global economic leaders of tomorrow. But the path to building an effective AI capability is not an easy one. There are many challenges to overcome. Challenges with the technology to develop platforms and solutions. With the people who will implement and manage that technology. With the data that fuels the technology. And with the processes that govern the whole of it. How do you harness the power inherent in AI, while avoiding any potential missteps?
That’s where Google Cloud comes in. Our framework for AI adoption provides a guide to technology leaders who want to build an effective AI capability, one that enables them to leverage the power of AI to enhance and streamline their business, smoothly and smartly. The framework is informed by Google’s own evolution, innovation, and leadership in AI, including experience deploying AI in production through products such as Gmail and Google Photos. It is also inspired by many years of experience helping cloud customers — from startups to enterprises, in various industries — to solve complex challenges.
With Google Cloud’s AI Adoption Framework, you’ll be able to create and evolve your own transformative AI capability. You’ll have a map for assessing where you are in the journey and where, at the end of it, you’d like to be. You’ll have a structure for building scalable AI capabilities to create better insights from big data with powerful algorithms across the entire business.
With Google Cloud as your guide, the path to AI is considerably smoother.
Download this whitepaper to find out:
- A map for assessing where you are in your AI journey and where you want to be
- A comprehensive structure for building an effective AI capability across your entire organisation to create actionable insights from data
- A technical deep dive for technology leaders
Costa Mesa Sanitary District Demonstrates How Public Utilities Can Leverage ML for Management and Upkeep of Manholes
3087
Of your peers have already read this article.
2:00 Minutes
The most insightful time you'll spend today!
Local governments are embracing more modern and scalable ways to support their communities. In an effort to save both time and money, Costa Mesa Sanitary District (“CMSD”) used machine learning to automate and streamline manhole maintenance. Manhole maintenance is an essential part of the upkeep of cities. Manholes provide critical access points for underground public utilities, allowing inspection, maintenance, and system upgrades. But failure to upkeep manholes can cause a multitude of problems, from road hazards to sewer blockages, and can make it difficult for workers to access underground public utilities, which can lead to other safety issues. Manhole maintenance is an essential part of Costa Mesa Sanitary District maintenance, but this process requires the work of an outside consultant and costs the CMSD over $100,000.
ML to the rescue
CMSD, in collaboration with SpringML and Google Cloud, developed a project to streamline manhole maintenance by leveraging the power of machine learning (ML) to detect sewer manholes and rate their conditions. This solution saves CMSD $40,000 every year, freeing up funds for other public service projects.
Every quarter, one member of the CMSD drives a car outfitted with a GoPro camera. This car travels through the entire District area, which includes the city of Costa Mesa and small portions of Newport Beach, CA, which is about 218 street miles, and records the roads to detect approximately 5,000 manholes. At the end of the recording day, CMSD members transfer images and videos from the GoPro SD card into a local server. Then these files are automatically ingested into Google Cloud Storage for processing. From here, the machine learning algorithms detect which manholes need repair.
Google Cloud products are used throughout this project. Once the images and videos are in Google Cloud Storage, a workflow with Cloud Scheduler spins up the VM every night to detect if there are new videos on Cloud Storage. If there are, this triggers cloud machine learning, which reviews the numerous images and videos, rates each manhole, and determines if any require maintenance.
Machine learning to detect and grade manholes
SpringML applied a very systematic approach to detecting and grading the manholes. First, image processing ensures that the region of interest is only the section of the road in front of the vehicle to avoid any privacy concerns. Then SpringML applied a 5-step process using machine learning to detect and grade the manholes.
- SpringML developed two separate custom TensorFlow-based Mask R-CNN models. Mask-R-CNN is a deep neural network that is used for image segmentation tasks, which means it can separate different objects in an image or a video.
The first Mask-R-CNN model was created to accurately detect if an image showed a manhole cover. This was a critical step because sewer manhole covers look similar to water main covers. To make the model accurate, it was important that there be no false positives. SpringML used around 50 images of sewer manholes and other images to train and validate that the algorithm could successfully detect sewer manhole covers. - Once a manhole is accurately detected, the surrounding area is masked using image processing to focus on the manhole and then the cropped images are sent to the second model which is used to detect damage in the area around the manhole, called the apron. To reduce the processing time, the second model only does the inference if a manhole cover is detected.
- The second Mask-R-CNN model uses CMSD Guidelines to decide what types of damage contributed to the rating system. For training purposes, SpringML uses a TensorFlow-Keras inside a Virtual Machine on Google Cloud. The initial model was trained and the Keras weights were saved on Google Cloud Storage. This helped create a versioned system of the models as the model gets refined over time.
- Then, duplicated detections are cleaned up to have a unique detection for each manhole cover.
- Finally, Manholes are graded from a 1-5 rating, with 5 representing high damage and 1 representing low damage.
Once cloud machine learning has analyzed the new videos and images, the final scores are stored in BigQuery. Results are then served to members of CMSD via a simple web application where they can see which manholes need to be maintained. Two staff members review the ML results and determine priorities for repairs. One of the most interesting features of this project is that the model gets continually retrained based on feedback submitted in the web application. For example, if the model inaccurately detects a manhole, a member of CMSD can mark that in the web application and their feedback is immediately used to refine the model.
This solution shows how leveraging machine learning can streamline a necessary local government project and save money and labor while being highly scalable! Not only does this system streamline the manhole maintenance process, but it also allows for more frequent review of manhole conditions and provides a historical view of how the District’s manholes change over time.
Want to learn more about Google Cloud machine learning? Check out this tutorial to learn TensorFlow and Keras and check out more machine learning tools in our AI Platform.
3000
Of your peers have already watched this video.
26:30 Minutes
The most insightful time you'll spend today!
What’s Next for Personalization on Google Cloud
Customer shopping behavior has changed for good. With fewer in-store shopping visits retailers have had to shore up their digital storefronts and explore new ways to meaningfully engage with their customers.
Delivering a superior customer experience has become even more of a differentiator for the early movers and personalized recommendations have emerged as one of the strongest potential drivers of revenue lift.
But as many retailers have discovered delivering recommendations at scale can actually be quite complex and time consuming.
Learn how to deliver highly-personalized product recommendations with Google Cloud Recommendations AI.
Recommendations AI is now fully open access and self-serve, with more built-in integrations with Google Shopping Merchant Center and Google Analytics, as well as more controls over how you create recommendation pipelines and manage your costs.
You will also hear how Google Cloud partners like Qubit and BigCommerce have successfully deployed Recommendations AI for their customers and made us an integral part of their solution offerings.
How Google Cloud’s PSO Supports Customers’ Migration Goals
5048
Of your peers have already read this article.
2:00 Minutes
The most insightful time you'll spend today!
Google Cloud’s Professional Services Organization (PSO) engages with customers to ensure effective and efficient operations in the cloud, from the time they begin considering how cloud can help them overcome their operational, business or technical challenges, to the time they’re looking to optimize their cloud workloads.
We know that all parts of the cloud journey are important and can be complex. In this blog post, we want to focus specifically on the migration process and how PSO engages in a myriad of activities to ensure a successful migration.
As a team of trusted technical advisors, PSO will approach migrations in three phases:
- Pre-Migration Planning
- Cutover Activities
- Post-Migration Operations
While this post will not cover in detail all of the steps required for a migration, it will focus on how PSO engages in specific activities to meet customer objectives, manage risk, and deliver value. We will discuss the assets, processes and tools that we leverage to ensure success.
Pre-Migration Planning
Assess Scope
Before the migration happens, you will need to understand and clarify the future state that you’re working towards. From a logistical perspective, PSO will be helping you with capacity planning to ensure sufficient resources are available for your envisioned future state.
While migration into the cloud does allow you to eliminate many of the considerations for the physical, logistical, and financial concerns of traditional data centers and co-locations, it does not remove the need for active management of quotas, preparation for large migrations, and forecasting. PSO will help you forecast your needs in advance and work with the capacity team to adjust quotas, manage resources, and ensure availability.
Once the future state has been determined, PSO will also work with the product teams to determine any gaps in functionality. PSO captures feature requests across Google Cloud services and makes sure they are understood, logged, tracked, and prioritized appropriately with the relevant product teams. From there, they work closely with the customer to determine any interim workarounds that can be leveraged while waiting for the feature to land, as well as providing updates on the upcoming roadmap.
Develop Migration Approach and Tooling
Within Google Cloud, we have a library of assets and tools we use to assist in the migration process. We have seen these assets help us successfully complete migrations for other customers efficiently and effectively.
Based on the scoping requirements and tooling available to assist in the migration, PSO will help recommend a migration approach. We understand that enterprises have specific needs; differing levels of complexity and scale; regulatory, operational, or organization challenges that will need to be factored into the migration. PSO will help customers think through the different migration options and how all of the considerations will play out.
PSO will work with the customer team to determine the best migration approach for moving servers from on-prem to Google Cloud. PSO will walk customers through different migration approaches, such as refactoring, lift-shift, or new installs. From there, the customer can determine the best fit for their migration. PSO will provide guidance on best practices and use cases from other customers with similar use cases.
Google offers a variety of cloud native tools that can assist with asset discovery, the migration itself, and post-migration optimization. PSO, as one example, will help work with project managers to determine the best tooling that accommodates the customer’s requirements for migrating servers. PSO will also engage Google product team to ensure the customer fully understands the capabilities of each tool and the best fit for the use case. Google understands from a tooling perspective, one size does not fit all, thus PSO will work with the customer on determining the best migration approach and tooling for different requirements.
Cutover Activities
Once all of the planning activities have been completed, PSO will assist in making sure the cutover is successful.
During and leading up to critical customer events, PSO can provide proactive event management services which deliver increased support and readiness for key workloads. Beyond having a solid architecture and infrastructure on the platform, support for this infrastructure is essential and TAMs will help ensure that there are additional resources to support and unblock the customer where challenges arise.
As part of event management activities, PSO liaises with the Google Cloud Support Organization to ensure quick remediation and high resilience for situations where challenges arise. A war room is usually created to facilitate quick communication about the critical activities and roadblocks that arise. These war rooms can give customers a direct line to the support and engineering teams that will triage and resolve their issues.
Post-Migration Activities
Once cutover is complete, PSO will continue to provide support in areas such incident management, capacity planning, continuous operational support, and optimization to ensure the customer is successful from start to finish.
PSO will serve as the liaison between the customer and Google engineers. If support cases need to be escalated, PSO will ensure the appropriate parties are involved and work to get the case resolved in a timely manner. Through operational rigor, PSO will work with the customer in determining if certain Google Cloud services will be beneficial to the customer objectives. If services will add value to the customer, PSO will help enable the services so it aligns with the customer’s goal and current cloud architecture. In cases where there are missing gaps in services, PSO will proactively work with the customer and Google engineering teams to close the gaps by enabling additional functionality in the services.
PSO will continue to work with the engineering teams to consistently review and provide recommendations on the customer’s cloud architecture in ensuring the most optimal and cost efficient design along with adhering to Google’s best practices guidelines.
Aside from migrations, PSO is also responsible for providing continuous training of Google Cloud to customers. To ensure consistent development of Google Cloud, PSO will work with the customer to jointly develop a learning roadmap to ensure the customer has the necessary skills to succeed in delivering successful projects in Google Cloud.
Conclusion
Google PSO will be actively engaged throughout the customer’s cloud journey to ensure the necessary guidance, methodology, and tools are presented to the customer. PSO will engage in a series of activities from pre-migration planning to post migration in key areas such as capacity planning to ensure sufficient resources are allocated for future workloads to providing support on technical cases for troubleshooting. PSO will serve as a long-term trusted advisor who will be the voice of the customer and provide the reliability and stability of the customer’s Google Cloud environment.
Click here if you’d like to engage with our PSO team on your migration. Or, you can also get started with a free discovery and assessment of your current IT landscape.
More Relevant Stories for Your Company
What Are India’s Biggest Companies Doing on Google Cloud?
In the last year, there’s been an upward trend in cloud adoption in India. In fact, NASSCOM finds that cloud spending in India is estimated to grow at 30% per annum to cross the US$7 billion mark by 2022. At Google, in our conversations with customers, discussions have evolved beyond cost savings
Partnering with Google Cloud is the Key Behind Recent Healthcare Innovations
It’s simply amazing to witness how some of our systems integrators employ Google Cloud solutions to drive innovation in ways we at Google may never have considered—especially in healthcare. According to analyst firm MarketsandMarkets, the market for the Cloud in healthcare is projected to grow 43% between 2020 and 2025
City of San Jose Ensures Critical Services Reach Community Using AI Translation
San José is one of the most diverse U.S. cities, with residents speaking more than 100 languages. Several years ago, we set out to improve city community interactions through more equitable service management and delivery. This demanded a new approach to automating the intake of requests from a majority population
Transform Your Marketing Strategy with Tinyclues and Google Cloud CDP
Editor’s note: The post is part of a series highlighting our awesome partners, and their solutions, that are Built with BigQuery. What are Customer Data Platforms (CDPs) and why do we need them? Today, customers utilize a wide array of devices when interacting with a brand. As an example, think
