We recently launched Vertex AI to help you move machine learning (ML) from experimentation into production faster and manage your models with confidence—speeding up your ability to improve outcomes at your organization.

But we know many of you are just getting started with ML and there’s a lot to learn! In tandem with building the Vertex AI platform, our teams are dropping as much best practices content as we can to help you come up to speed. Plus, we have a dedicated event on June 10th, Applied ML Summit, with sessions on how to apply ML technology in your projects, as well as grow your skills in this field. 

In the meantime, we couldn’t resist a quick lesson on hyperparameter tuning, because (a) it’s incredibly cool (b) you will impress your coworkers (c) Google Cloud has some unique battle tested tech in this area and (d) you will save time by getting better ML models into production faster. Vertex Vizier, on average, finds optimal parameters for complex functions in over 80% fewer trials than traditional methods. 

So it’s incredibly cool, but what is it?

While machine learning models automatically learn from data, they still require user-defined knobs which guide the learning process. These knobs, commonly known as hyperparameters, control, for example, the tradeoff between training accuracy and generalizability.  Examples of hyperparameters are the optimizer being used, its learning rate, regularization parameters, the number of hidden layers in a DNN, and their sizes.

Setting hyperparameters to their optimal values for a given dataset can make a huge difference in model quality. Typically, optimal hyperparameter values are found via grid searching a small number of combinations, or tedious manual experimentation. Hyperparameter tuning automates this work for you by searching for the best configuration of hyperparameters for optimal model performance. 

Vertex Vizier enables automated hyperparameter tuning in several ways:

1. “Traditional” hyperparameter tuning: by this we mean finding the optimal value of hyperparameters by measuring a single objective metric which is the output of an ML model.  For example, Vizier selects the number of hidden layers and their sizes, an optimizer and its learning rate, with the goal of maximizing model accuracy.
2. When hyperparameters are evaluated, models are trained and evaluated on splits of the data set. If evaluation metrics are streamed to Vizier (e.g. as a function of epoch) as the model is trained, Vizier’s early stopping algorithms can predict the final objective value, and recommend which unpromising trials should be early stopped. This conserves compute resources and speeds up convergence.
3. Oftentimes, models are tuned sequentially on different data sets. Vizier’s built in transfer learning learns priors from previous hyperparameter tuning studies, and leverages them to converge faster on subsequent hyperparameter tuning studies.
4. AutoML is a variant of #1, where Vertex Vizier performs both model selection, and also tunes architectures/non-architecture modifying hyperparameters. AutoML usually requires more code on top of Vertex Vizier (to ingest data etc), but Vizier is in most cases the “engine” behind the process. AutoML is implemented by defining a tree like (DAG) search space, rather than a “flat” search space (like in #1). Note that you can use DAG search spaces for any other purpose where searching over a hierarchical space makes sense.
5. There are times when you may wish to optimize more than one metric. For example, we would like to optimize model accuracy, while minimizing model latency. Vizier can find the Pareto frontier, which presents tradeoffs for multiple metrics, allowing users to choose the appropriate tradeoff. Simple example: I want to make a more accurate model, but would like to minimize serving latency. I do not know ahead of time what’s the tradeoff between the two metrics. Vizier can be used to explore and plot a tradeoff curve, so users can select on the most appropriate one. For example, “a latency decrease of 200ms will only decrease accuracy by 0.5%”

Google Vizier is all yours with Vertex AI

Google published the Vizier research paper in 2017, sharing our work and use cases for black-box optimization—i.e. The process of finding the best settings for a bunch of parameters or knobs when you can’t peer inside a system to see how well the knobs are working. The paper discusses our requirements, infrastructure design, underlying algorithms, and advanced features such as transfer learning that the service provides. Vizier has been essential to our progress with machine learning at Google, which is why we are so excited to make it available to you on Vertex AI.

Vizier has already tuned millions of ML models at Google, and its algorithms are continuously improved for faster convergence and handling of real-life edge cases. Vertex Vizier’s models are very well calibrated and are self-tuning (they adapt to user data), and offer unique power features, such as hierarchical search spaces and multi-objective optimization. We believe Vertex Vizier’s set of features is a unique capability to Google Cloud, and look forward to optimizing the quality of your models by automatically tuning hyperparameters for you.

To learn more about Vertex Vizier, check out these docs and if you are interested in what’s coming in machine learning over the next five years, tune in to our Applied ML Summit on June 10th, or watch the sessions on demand in your own time.

Google Cloud Results

• Enables accurate allocation of marketing spend to underserved areas
• Supports expansion into international markets
• Helps ensure accurate forecasting of inventory levels
• Improves accuracy of forecasted delivery times by at least 60%

Operating in India since 2011, Rebel Foods has grown from a brick-and-mortar business that provided wraps to customers to a cloud kitchen that delivers cuisine to about one million consumers per month. “We started with the Faasos food brand and now we have scaled up to 10 brands,” says Soumyadeep Barman, Chief Technology Officer at Rebel Foods. “We have doubled our revenue every year from 2014 until now, and we operate kitchens in 15 cities across India. Each kitchen offers at least seven of our brands to customers.”

Barman attributes Rebel Foods’ success to the fact that it is a full stack company. “We procure, we have our own inventory, we prepare the food, we deliver the food to customers, and we make sure customers are delighted every time they order,” he says.

The rapid emergence and adoption of mobile technologies and services in India gave the business its opportunity to expand quickly. “The boom in applications and the web really got going in India in about 2014,” Barman says. “The subsequent emergence of smart devices and mobile applications opened up new markets, including older people who had not really used a computer until then.”

The business released the first iteration of its mobile application in 2013 on servers in an on-premises data center. “However, we experienced breakages because our infrastructure was not scalable or dependable enough, and we decided to move to another solution,” Barman says.

Google Maps Platform delivers opportunity

In 2014, Rebel Foods decided to move to the cloud and selected Google Cloud because of its stability, reliability, and scalability.

The business also wanted to take advantage of the opportunities Google Maps Platform presented to improve the efficiency and effectiveness of its delivery service. With 175 kitchens delivering to about 900 locations across India, Rebel Foods needs to provide estimated delivery times and meet delivery guarantees, while accounting for all the factors that might affect how quickly a rider can reach a customer’s doorstep.

The business turned to Google Maps Platform Premier Partner Searce for support in leveraging Google Maps Platform APIs to deliver a compelling customer experience and improve its efficiency. “Searce helped us determine the Google Maps Platform APIs we should use across our mobile applications and websites, and how many licenses we needed to conduct activities like calculating estimated delivery time and reviewing order heat maps,” Barman says. “Thanks to the firm’s support, Google Maps Platform APIs were a game changer for us.”

Mapping customer locations

Customers accurately pinpoint their location in a map through functionality made available through the Places API and Geocoding API, in conjunction with the JavaScript API. Drivers use the Directions API to identify the quickest route to customers.

Customers can also track the progress of delivery and estimated time of arrival using an Android or iOS application, or the brand websites.

Deploying Google Maps Platform APIs enabled Rebel Foods to improve by up to 60 percent the accuracy of forecasted delivery times. “Rather than tell a customer we can reach them in, say, 45 minutes, based on previous experience and gut feeling, we can retrieve an accurate traffic scenario and calculate delivery times based on traffic congestion levels and likely average speeds,” Barman says

Allocating budget effectively

Google Maps Platform also allows the business to combine mapping of customers to individual kitchens and to how often customers place orders – and for what value. This enabled the business to understand where to allocate budget for local marketing to stimulate demand in underserved areas.

Google Maps Platform technologies complement Rebel Foods’ use of Google Cloud Platform services such as the BigQuery analytics data warehouse to process data used to forecast inventory levels and provide recommendations to customers based on previous usage and behaviors. The business also runs its key applications in Kubernetes Engine to achieve cost-effective scalability, so it can expand to international markets.

“We are targeting growth into a range of international markets in January 2019, including Australia, the Middle East, and Southeast Asia,” Barman says. “With the user data and experience provided by Google Maps Platform in particular, we are poised for success.”

In order to be successful, research needs to be replicable, so scientists can build on past work and insights. However, an article in Nature warned that as much as 50% of published drug development research could not be reproduced in subsequent trials. As a result, promising drug candidates sometimes led to disappointment, as well as wasted time and money, when key findings could not be replicated. 

The shift to cloud computing helps solve this problem because it allows researchers to use open-source tools that work across platforms. As demand for cloud computing rises, our customers have asked us for more ready-made solutions to assure reproducibility of results by their collaborators, regardless of the platform they are using. They asked for secure and effective collaboration tools as well as faster time-to-insight from any type of data.

We listened. Google’s new research and data insights solution includes three sets of functionalities to address these key challenges. Each can be activated on demand and may be eligible for subscription pricing. “HPC in a box” offers abstract complexity to run high performance computing (HPC) workloads by automatically managing your cluster in the most effective manner. It integrates seamlessly with some of the industry’s most-used schedulers like Slurm and PBS. It makes it easier than ever to answer bigger questions faster by accessing Google’s fast, powerful hardware like TPUs and GPUs, all for one predictable flat fee for eligible workloads. Healthcare Innovation Hub provides healthcare-specific functionality to help ingest, aggregate, and de-identify any type of healthcare data in its original format. It unlocks cross-modality analysis and collaboration and empowers researchers with harmonization tools to overcome healthcare interoperability issues. Google Cloud Real-World Insights (formerly FDA MyStudies) accelerates and streamlines drug development and clinical trials to address urgent medical challenges with reproducible results.

The solution enables researchers to ask new questions, get answers more quickly, and work more collaboratively–with no wait times or down times. Institutions can scale to more ambitious projects and generate actionable, real-time insights from any data source–all while staying within budget.

Many top research centers have already found it faster and more cost effective to shift from downloading and storing data on their own servers to storing and analyzing data on Google Cloud. Here are some of the real-world projects already yielding breakthroughs:

• Clemson analyzed 8,500 hours of traffic camera feeds with 2.1M vCPUs in three hours to improve evacuation routes for disaster planning.
• The Colorado Center for Personalized Medicine saved $2.9M by building their Compass data warehouse on Google Cloud rather than on on-prem.
• The Broad Institute slashed costs of genomic sequencing by 90% with Google Cloud.

Our partners, such as Atos, Burwood, Omnibond, Mavenwave, Quantiphi, and Deloitte, can help you first design and develop, then install and implement your own solution, including training. To assess your institution’s needs and develop a customized plan for your next-generation research solution with research and insights, contact our sales team.