How Vertex Vizier’s Automated Hyperparameter Tuning Improves ML Models
5026
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
We recently launched Vertex AI to help you move machine learning (ML) from experimentation into production faster and manage your models with confidence—speeding up your ability to improve outcomes at your organization.
But we know many of you are just getting started with ML and there’s a lot to learn! In tandem with building the Vertex AI platform, our teams are dropping as much best practices content as we can to help you come up to speed. Plus, we have a dedicated event on June 10th, Applied ML Summit, with sessions on how to apply ML technology in your projects, as well as grow your skills in this field.
In the meantime, we couldn’t resist a quick lesson on hyperparameter tuning, because (a) it’s incredibly cool (b) you will impress your coworkers (c) Google Cloud has some unique battle tested tech in this area and (d) you will save time by getting better ML models into production faster. Vertex Vizier, on average, finds optimal parameters for complex functions in over 80% fewer trials than traditional methods.
So it’s incredibly cool, but what is it?
While machine learning models automatically learn from data, they still require user-defined knobs which guide the learning process. These knobs, commonly known as hyperparameters, control, for example, the tradeoff between training accuracy and generalizability. Examples of hyperparameters are the optimizer being used, its learning rate, regularization parameters, the number of hidden layers in a DNN, and their sizes.
Setting hyperparameters to their optimal values for a given dataset can make a huge difference in model quality. Typically, optimal hyperparameter values are found via grid searching a small number of combinations, or tedious manual experimentation. Hyperparameter tuning automates this work for you by searching for the best configuration of hyperparameters for optimal model performance.
Vertex Vizier enables automated hyperparameter tuning in several ways:
- “Traditional” hyperparameter tuning: by this we mean finding the optimal value of hyperparameters by measuring a single objective metric which is the output of an ML model. For example, Vizier selects the number of hidden layers and their sizes, an optimizer and its learning rate, with the goal of maximizing model accuracy.
- When hyperparameters are evaluated, models are trained and evaluated on splits of the data set. If evaluation metrics are streamed to Vizier (e.g. as a function of epoch) as the model is trained, Vizier’s early stopping algorithms can predict the final objective value, and recommend which unpromising trials should be early stopped. This conserves compute resources and speeds up convergence.
- Oftentimes, models are tuned sequentially on different data sets. Vizier’s built in transfer learning learns priors from previous hyperparameter tuning studies, and leverages them to converge faster on subsequent hyperparameter tuning studies.
- AutoML is a variant of #1, where Vertex Vizier performs both model selection, and also tunes architectures/non-architecture modifying hyperparameters. AutoML usually requires more code on top of Vertex Vizier (to ingest data etc), but Vizier is in most cases the “engine” behind the process. AutoML is implemented by defining a tree like (DAG) search space, rather than a “flat” search space (like in #1). Note that you can use DAG search spaces for any other purpose where searching over a hierarchical space makes sense.
- There are times when you may wish to optimize more than one metric. For example, we would like to optimize model accuracy, while minimizing model latency. Vizier can find the Pareto frontier, which presents tradeoffs for multiple metrics, allowing users to choose the appropriate tradeoff. Simple example: I want to make a more accurate model, but would like to minimize serving latency. I do not know ahead of time what’s the tradeoff between the two metrics. Vizier can be used to explore and plot a tradeoff curve, so users can select on the most appropriate one. For example, “a latency decrease of 200ms will only decrease accuracy by 0.5%”
Google Vizier is all yours with Vertex AI
Google published the Vizier research paper in 2017, sharing our work and use cases for black-box optimization—i.e. The process of finding the best settings for a bunch of parameters or knobs when you can’t peer inside a system to see how well the knobs are working. The paper discusses our requirements, infrastructure design, underlying algorithms, and advanced features such as transfer learning that the service provides. Vizier has been essential to our progress with machine learning at Google, which is why we are so excited to make it available to you on Vertex AI.
Vizier has already tuned millions of ML models at Google, and its algorithms are continuously improved for faster convergence and handling of real-life edge cases. Vertex Vizier’s models are very well calibrated and are self-tuning (they adapt to user data), and offer unique power features, such as hierarchical search spaces and multi-objective optimization. We believe Vertex Vizier’s set of features is a unique capability to Google Cloud, and look forward to optimizing the quality of your models by automatically tuning hyperparameters for you.
To learn more about Vertex Vizier, check out these docs and if you are interested in what’s coming in machine learning over the next five years, tune in to our Applied ML Summit on June 10th, or watch the sessions on demand in your own time.
Beyond Traditional Learning: AI-based Online Learning Platform and Google Cloud Solutions Push Learners to Get Ahead
10311
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
The combination of a vital need for IT experts among businesses and a digital skills gap is making lifelong learning increasingly critical. Beyond professional development, learning new skills offers additional rewards from building peer connections to boosting your creativity. That’s why in 2020 Krishna Deepak Nallamilli and I launched KIMO.ai to reimagine how people approach learning, especially in developing markets. Our team is building the artificial intelligence needed to generate individual learning paths through a wide range of quality digital learning content.
Google Cloud and its Startup Program have been instrumental in connecting our team with the tools, people, processes, and best practices to grow our business.
Existing learning platforms lack engagement
Outside of traditional education settings, massive open online learning courses (MOOCs)—often modeled after university courses—can provide a flexible and affordable way to upskill or reskill. But the vast majority of people who participate in MOOC programs fail to complete courses. Based on our research, the challenge with existing learning platforms is a lack of engagement, primarily caused by limited direction on which skills to learn, whether AI, fintech, blockchain, or other in-demand disciplines.
We’ve also received feedback that many corporate learning management systems–developed as online training systems to upskill employees–tend to be poorly designed and time-consuming to use.
Overall, a significant challenge with most existing learning platforms is that they’re generic. For example, suppose you’re interested in learning about AI. In that case, you need AI-related coursework that applies to your industry and the job you want because AI in medicine is vastly different from AI in financial services. Today’s online learning options typically take a one-size-fits-all approach and fail to capture the nuances of what learners really need to get ahead.
Building a future-proof learning platform
The commitment to highly personalized, accessible learning inspired KIMO.ai, a platform that we believe is the future of education. Depending on your goals, current skills, location, and other factors, our AI-based platform will identify which coursework (and where to find those classes) to build the skills you need. The more personalized, relevant learning recommendations even take into account people’s preferences for podcasts, MOOCs, books, articles, videos, courses, publications, and more.
In a mix of cooperation and competition we call “coopetition,” KIMO.ai will regularly recommend courses from other established online learning systems if, based on our automated assessment, it’s the best option for a learner. There’s also the option to access free content only.
Google cultural alignment fosters trust
Our platform started with one developer exploring NLP models and Google APIs. As we’ve grown our team and launched our beta to 110,000 users in developing markets, we discovered there is a lot of interest in our platform, and we believe we can make a significant impact. In feedback forums, we also learned that we need to focus our efforts on the mobile experience to improve engagement since 99% of the beta testers use mobile devices.
Beyond our team’s high level of trust in Google Cloud solutions, our team also appreciates the cultural alignment with Google. We value Google’s developer-centric approach and rely on tools like Dataflow for batch data processing and Cloud TPU to reliably run machine learning models with AI services on Google Cloud. We also build all of our deployments on Google Kubernetes Engine (GKE), which makes it easy to manage all our containerized workloads
On the front end, Google App Engine makes it easy to deploy apps and experiment, and it integrates seamlessly with Firebase for authentication and more. BigQuery is our serverless data warehouse that efficiently scales to support the millions of articles, videos, and other learning resources we need to analyze to provide the targeted coursework recommendations our learners require.
As we grow our business having a network of trusted advisors is also extremely valuable. By working closely with DoIT International, the 2020 Google Cloud Global Reseller Partner of the Year, our team has access to their cloud, Kubernetes, and machine learning expertise. DoIT has already helped us quickly resolve IT issues and create analytics dashboards that give us insights to continually enhance our services.
Building for a growing industry
The dynamic edtech market is growing rapidly and estimated to become an $11B industry by 2025. We’re proud to be part of the next wave of personalized education that has the potential to empower people in developing markets and beyond to grow their skills with coursework tailored to their exact needs and how they like to learn. This year, we will deliver our platform to at least 400,000 more people. We’re excited to see how they use it and where it takes them.
If you want to learn more about how Google Cloud can help your startup, visit our Startup Program application page here to get more information about our program, and sign up for our communications to get a look at our community activities, digital events, special offers, and more.
3825
Of your peers have already downloaded this article
2:30 Minutes
The most insightful time you'll spend today!
Every business is a digital business. That’s what you’ll hear from technology folks these days. But, what exactly is a digital business? How does one define it?
Simply put, digital businesses are those that have thoroughly capitalized on the opportunity to connect people with technology. There are four parts to a digital business:
Real-time data and analytics: To stay relevant in the age of Big Data, businesses must analyze copious amounts of data to derive actionable insights—both from historical data, and in real time.
Fast, flexible application development: Rapid and continuous delivery of software to your stakeholders is no longer optional. Businesses need platforms for their applications strategy — from using container-based development tools to fully managed serverless platforms.
Secure, reliable infrastructure: How secure is your on-prem datacenter? What happens if it goes down? How many dedicated security engineers do you have on staff? What’s the cost of a system upgrade? What digital businesses need is a secure and reliable infrastructure that can power their applications.
Constant collaboration and productivity: Digital businesses are designed to keep teams seamlessly connected not only to each other, but also to the applications that keep the company running. This enables everything and everyone to work together, no matter where they sit—across the office or across the ocean.
Download this infographic to know more.
3790
Of your peers have already downloaded this article
10:30 Minutes
The most insightful time you'll spend today!
Digital transformation is about how digital technologies can connect people and processes to solve challenges that traditional methodologies could not.
While this transformation is imperative for businesses of all sizes, the Frost & Sullivan Enterprise Cloud Maturity Index (ECMI) assessment indicates that around 85% of CXOs want to digitally transform their enterprises, but only 39% of them have a clear plan to achieve this transformation.
Most business leaders are exploring new business ideas around cloud to extract better outcomes, increase productivity, create opportunities and redefine customer engagement processes as part of their internal strategies.
It has been proved time and again that enterprises need to bank on emerging technologies like cloud that provide ‘do more with less’ approach. Cloud has the potential to transform IT departments through infrastructure consolidation and optimization.
To understand the interest and level of cloud adoption in enterprises, Google Cloud partnered with Frost & Sullivan to recognize the unique business drivers and key challenges in Cloud adoption that come the enterprise way during their business transformation journey.
Read the report to find out the challenges, solutions and benefits of digital transformation and the cloud.
Google Introduces ML-based Predictive Autoscaling to Forecast Capacity and Match Scaling Demands
4973
Of your peers have already read this article.
3:00 Minutes
The most insightful time you'll spend today!
At Google Cloud, we believe you get most benefits from the cloud when you scale infrastructure based on changing demand. Compute Engine allows you to configure autoscaling to save costs during periods of low demand, and add capacity to support peak loads.
When you use a managed instance group (MIG), you can have an autoscaler automatically create or delete virtual machine (VM) instances based on increases or decreases in load. However, if your application takes several minutes to initialize, creating VMs in response to growing load might not increase your application’s capacity quickly enough. For example, if there’s a large increase in load (like when users first wake up in the morning), some users might experience delays while your application is initializing on new instances.
A good way to solve this problem would be to create VMs ahead of demand so that your application has enough time to initialize beforehand. This requires knowing upcoming demand. If only we could predict the future… Well, now we can!
Introducing predictive autoscaling
Predictive autoscaling uses Google Cloud’s machine learning capabilities to forecast capacity needs. It creates VMs ahead of growing demand allowing enough time for your application to initialize.
How does it work?
Predictive autoscaling uses your instance group’s CPU history to forecast future load and calculate how many VMs are needed to meet your target CPU utilization. Our machine learning adjusts the forecast based on recurring load patterns for each MIG.
You can specify how far in advance you want autoscaler to create new VMs by configuring the application initialization period. For example, if your app takes 5 minutes to initialize, autoscaler will create new instances 5 minutes ahead of the anticipated load increase. This allows you to keep your CPU utilization within the target and keep your application responsive even when there’s high growth in demand.
Many of our customers have different capacity needs during different times of the day or different days of the week. Our forecasting model understands weekly and daily patterns to cover for these differences. For example, if your app usually needs less capacity on the weekend our forecast will capture that. Or, if you have higher capacity needs during working hours, we also have you covered.
Why should you try it?
Predictive autoscaling continuously adapts forecasted capacity to best match upcoming demand. Autoscaler checks the forecast several times per minute and creates or deletes VMs to match its prediction. The forecast itself is updated every few minutes to match recent load trends so if your growth rate is higher or lower than usual we will adjust the forecast accordingly. This gives you capacity needed to cover peak load while saving on cost when demand goes down.
You can start using predictive autoscaling without worry as it’s fully compatible with the current autoscaler. Autoscaler will calculate enough VMs to cover both forecasted as well as real-time CPU load—whichever is higher. This works with other autoscaling features as well: you can scale based on schedule, your Load Balancer request target or Cloud Monitoring metrics. Autoscaler provides enough capacity to all of your configurations by taking the highest number of VMs needed to meet all your targets.
Getting started
You can enable predictive autoscaling in the Google Cloud Console. Select an autoscaled MIG from the instance groups page and click Edit group. Change predictive autoscaling configuration from Off to Optimize for availability.
To better understand whether predictive autoscaling is good for your application, click the link See if predictive autoscaling can optimize your availability. This will show you a comparison of the last seven days with your current autoscaling configuration vs. with predictive autoscaling enabled.
In the above chart,
- Average VM minutes overloaded per day shows how often your VMs exceed your CPU utilization target. This happens when demand is higher than available capacity. Predictive autoscaling can reduce this by starting VMs ahead of anticipated load.
- Average VMs per day is a proxy for cost. This shows how much additional VM capacity you need to keep your CPU utilization within the target you have set. You can optimize your cost by adjusting Minimum instances andCPU utilization as explained below.
Optimizing your configuration
Make sure your Cool down period reflects how long it takes for your application to initialize from VM boot time until it’s ready to serve the load. Predictive autoscaling will use this value to start VMs ahead of forecasted load. If you set it to 10 minutes (600 seconds) your VMs will start 10 minutes before the load is expected to increase.
Review your autoscaling CPU utilization target and Minimum number of instances. With predictive autoscaling you no longer need a buffer to compensate for the time it takes for a VM to start. If your application works best at 70% CPU utilization you don’t need to set target to a much lower value as predictive autoscaling will start VMs ahead of usual load. A higher CPU utilization and lower Minimum number of instances allows you to reduce the cost as you don’t need to pay for additional capacity to prepare for growing demand.
Try predictive autoscaling today
Predictive autoscaling is generally available across all Google Cloud regions. For more information on how to configure, simulate and monitor predictive autoscaling, consult the documentation.
Building a Stronger South Bend through Cloud Computing Education
3023
Of your peers have already read this article.
2:00 Minutes
The most insightful time you'll spend today!
The city of South Bend, Indiana has announced they are working with Google Cloud to connect 500 local job seekers with no-cost access to Google Cloud Skills Boost through Upskill SB. This initiative will train, upskill, and certify residents in cloud computing skills to create pathways to technical jobs.
As companies, nonprofits, and governments increasingly leverage cloud computing technology to strengthen their organizations, skilled cloud talent is in high demand both nationally, in Indiana, and in South Bend. According to Lightcast, there were 27,180 unique cloud roles open in the U.S. as of October 2022. Cloud skills are the most in-demand skill set by IT departments: more than one-third of tech learners said professionals with cloud skills are the most challenging to find.
“This partnership with Google Cloud will help residents get on a no-cost path to a cloud computing career at a time when the field is rapidly growing,” said Caleb Bauer, Executive Director of Community Investment for the City of South Bend. “This will help residents prepare to take the next step in their career with Google Cloud Skills Boost and will pave the way to build in-demand skills.”
Google Cloud Skills Boost is the destination for all Google Cloud learning and training opportunities. Local colleges and universities are also able to offer free access to Google Cloud Skills Boost for students through the Google Cloud Higher Education program. Students can request access to free online labs, quests, and courses. Faculty can take advantage of free Google Cloud Computing Foundations curriculum and coach their students to prepare for Google Cloud certifications.
“Google is proud to offer no-cost access to Google Cloud Skills Boost for jobseekers in South Bend,” said Alice Kamens, Workforce Development lead at Google Cloud. “Through this partnership, we’re advancing our work to ensure everyone has access to the technical training needed to pursue cloud computing careers.”
Those interested in learning more about the program or applying for Upskill SB should visit the Google Cloud page on Upskill SB. Scholarship recipients need access to a computer, hand-held device or smartphone and the internet.
More Relevant Stories for Your Company
Google Cloud Extends Research Credits to Non-profit Research Projects
To fuel the breakthroughs of the future, today’s researchers need easy access to the most innovative cloud technologies. That’s why we are delighted to announce the latest expansion of Google Cloud research credits beyond its previous scope of government and academic research institutions to researchers at nonprofit institutions. This initiative is part
Kohl’s Leverages Google Cloud Platform for Omnichannel Retail
Kohl’s is an omnichannel retailer focused on driving traffic, operational efficiency and delivering seamless omnichannel customer experiences. Ratnakar Lavu is Kohl’s Senior Executive Vice President and Chief Technology Officer. He, and Kohl’s, are at the forefront of retail technology innovation, focusing on a frictionless customer journey across digital, mobile and
Pharma Firm Drives 80% Improvement in Speed with SAP on Google Cloud
FFF Enterprises is a leading supplier of critical-care biopharmaceuticals, plasma products, and vaccines. Their passion for patient safety and product efficacy drives their mission of Helping Healthcare Care. For FFF Enterprises if they have to focus on ERP infrastructure, that takes away from getting products to patents. Learn why FFF
Telehealth Improves Patient and Clinical Experiences across Continuum of Care
According to a National Academy of Medicine discussion paper, social determinants of health (SDoH) account for upwards of 80% of a population's health outcomes. Breaking this down further, 50% come just from socioeconomic and physical environment factors such as education, employment, income, family and social support, community safety, air and water
