We recently launched Vertex AI to help you move machine learning (ML) from experimentation into production faster and manage your models with confidence—speeding up your ability to improve outcomes at your organization.

But we know many of you are just getting started with ML and there’s a lot to learn! In tandem with building the Vertex AI platform, our teams are dropping as much best practices content as we can to help you come up to speed. Plus, we have a dedicated event on June 10th, Applied ML Summit, with sessions on how to apply ML technology in your projects, as well as grow your skills in this field. 

In the meantime, we couldn’t resist a quick lesson on hyperparameter tuning, because (a) it’s incredibly cool (b) you will impress your coworkers (c) Google Cloud has some unique battle tested tech in this area and (d) you will save time by getting better ML models into production faster. Vertex Vizier, on average, finds optimal parameters for complex functions in over 80% fewer trials than traditional methods. 

So it’s incredibly cool, but what is it?

While machine learning models automatically learn from data, they still require user-defined knobs which guide the learning process. These knobs, commonly known as hyperparameters, control, for example, the tradeoff between training accuracy and generalizability.  Examples of hyperparameters are the optimizer being used, its learning rate, regularization parameters, the number of hidden layers in a DNN, and their sizes.

Setting hyperparameters to their optimal values for a given dataset can make a huge difference in model quality. Typically, optimal hyperparameter values are found via grid searching a small number of combinations, or tedious manual experimentation. Hyperparameter tuning automates this work for you by searching for the best configuration of hyperparameters for optimal model performance. 

Vertex Vizier enables automated hyperparameter tuning in several ways:

1. “Traditional” hyperparameter tuning: by this we mean finding the optimal value of hyperparameters by measuring a single objective metric which is the output of an ML model.  For example, Vizier selects the number of hidden layers and their sizes, an optimizer and its learning rate, with the goal of maximizing model accuracy.
2. When hyperparameters are evaluated, models are trained and evaluated on splits of the data set. If evaluation metrics are streamed to Vizier (e.g. as a function of epoch) as the model is trained, Vizier’s early stopping algorithms can predict the final objective value, and recommend which unpromising trials should be early stopped. This conserves compute resources and speeds up convergence.
3. Oftentimes, models are tuned sequentially on different data sets. Vizier’s built in transfer learning learns priors from previous hyperparameter tuning studies, and leverages them to converge faster on subsequent hyperparameter tuning studies.
4. AutoML is a variant of #1, where Vertex Vizier performs both model selection, and also tunes architectures/non-architecture modifying hyperparameters. AutoML usually requires more code on top of Vertex Vizier (to ingest data etc), but Vizier is in most cases the “engine” behind the process. AutoML is implemented by defining a tree like (DAG) search space, rather than a “flat” search space (like in #1). Note that you can use DAG search spaces for any other purpose where searching over a hierarchical space makes sense.
5. There are times when you may wish to optimize more than one metric. For example, we would like to optimize model accuracy, while minimizing model latency. Vizier can find the Pareto frontier, which presents tradeoffs for multiple metrics, allowing users to choose the appropriate tradeoff. Simple example: I want to make a more accurate model, but would like to minimize serving latency. I do not know ahead of time what’s the tradeoff between the two metrics. Vizier can be used to explore and plot a tradeoff curve, so users can select on the most appropriate one. For example, “a latency decrease of 200ms will only decrease accuracy by 0.5%”

Google Vizier is all yours with Vertex AI

Google published the Vizier research paper in 2017, sharing our work and use cases for black-box optimization—i.e. The process of finding the best settings for a bunch of parameters or knobs when you can’t peer inside a system to see how well the knobs are working. The paper discusses our requirements, infrastructure design, underlying algorithms, and advanced features such as transfer learning that the service provides. Vizier has been essential to our progress with machine learning at Google, which is why we are so excited to make it available to you on Vertex AI.

Vizier has already tuned millions of ML models at Google, and its algorithms are continuously improved for faster convergence and handling of real-life edge cases. Vertex Vizier’s models are very well calibrated and are self-tuning (they adapt to user data), and offer unique power features, such as hierarchical search spaces and multi-objective optimization. We believe Vertex Vizier’s set of features is a unique capability to Google Cloud, and look forward to optimizing the quality of your models by automatically tuning hyperparameters for you.

To learn more about Vertex Vizier, check out these docs and if you are interested in what’s coming in machine learning over the next five years, tune in to our Applied ML Summit on June 10th, or watch the sessions on demand in your own time.

In November 2021, we announced the general availability of Tau VMs. Since then, Google Cloud’s Tau VMs with Google Kubernetes Engine (GKE) have unlocked value for many customers who are now using Tau VMs for their production workloads, such as: Ascend, who achieved over 125% higher performance; Nylas, who gained over 40% higher price-performance; and OpenX, who achieved 40% better price-performance while at the same time reducing their application latency by 62%.

T2D is the first instance type in the Tau VM family and is built on the latest 3rd generation AMD EPYCTM processors, offering 42% higher price-performance compared to general-purpose VMs from any of the leading public cloud vendors. Tau VMs offer a leading combination of performance, price, and full x86 compatibility, offering customers the lowest cost solution for scale-out workloads. Tau VMs are available in predefined shapes, with up to 60vCPUs per VM, 4GB of memory per vCPU, networking up to 32 Gbps and a slew of storage options including Standard, Balanced and Performance PD. Tau VMs are also available as Spot VMs, offering an over 60% discount compared to on-demand pricing.

For customers looking for advanced container orchestration, GKE delivers high levels of reliability, security, and scalability, and has supported Tau VMs since the day they became available on Google Cloud. Tau VMs are ideal for CPU-bound workloads such as web-serving with encryption, video encoding, compression/decompression, image processing and horizontally-scaled applications. Using Tau VMs along with GKE’s cost-optimization best practices can help lower your total cost of ownership. You can add Tau VMs to new or existing GKE clusters by specifying the Tau T2D machine type in your GKE node-pools through the Cloud console or by using –machine-type in gcloud.

Here is what some of our customers have to say about Tau VMs:

Ascend provides a unified analytics and data engineering platform, and chose Tau VMs along with GKE to run their data-intensive workload — primarily because of Tau’s absolute performance and price-performance advantage.

“Our core capability at Ascend is bringing together data ingestion, transformation, delivery, orchestration and observability into a single platform. To operate at scale and keep pace with our telemetry data production rates, high single-threaded performance is critical. With Google Cloud’s Tau VMs with Google Kubernetes Engine (GKE), we are able to achieve over 125% higher performance than previous generation families. This has completely changed our ability to query historical metrics. Where previously metric queries against historical data over ranges longer than a couple hours were difficult, we can now easily query data ranges of multiple weeks.” – Joe Stevens, Tech Lead – Infrastructure, Ascend.io

Nylas is a pioneer and leading provider of productivity infrastructure solutions for modern software. In the past year, Nylas has been using GKE in their journey to reinvent their architecture and provide their enterprise customers with a bi-directional universal email sync, security compliance with the highest enterprise standards, and industry-specific machine learning services.

“For our core application, Google’s Tau VMs with Google Kubernetes Engine delivers over 40% better price-performance than Amazon’s Graviton-based VMs. Further, Tau VMs maintain x86 compatibility and eliminate the need to maintain a separate stack for ARM. We are moving our workload from Amazon Web Services to Google Cloud to take advantage of these benefits.” – David Ting, SVP of Engineering, Nylas

OpenX operates an independent ad exchange. Operating 100% on Google Cloud has enabled OpenX to achieve improved performance, scalability, speed and global reach.

“At OpenX, our ad-exchange services over 200 billion requests every day. Getting the best combination of performance and price from the infrastructure is critically important for us. We use multiple Google Kubernetes Engine (GKE) clusters across geographic regions with autoscaling to power our ad-delivery components. Running Google Cloud’s Tau VMs with GKE has enabled over 40% better price-performance and 62% latency reduction for our application as compared to the prior generation family. We have made the move to Tau VMs for our application to take advantage of these benefits.” – Paul T.Ryan, CTO, OpenX

We are excited to see Tau VMs adding value for so many of our customers by enabling industry leading price-performance for a variety of workloads.

If you haven’t tried Tau VMs yet, give them a try today in our Iowa, Netherlands and Singapore regions and move your production workloads to Tau VMs. Tau VMs will be arriving in additional regions and zones in the coming weeks. You can provision GKE node pools based on Tau VMs and explore how you can take advantage of improved price-performance for your scale-out containerized workloads.

To get started, go to the Google Cloud Console, select Google Kubernetes Engine, and choose Tau T2D for your GKE nodes. To learn more about Tau VMs or other Compute Engine VM options, check out our machine types and our pricing pages.

Cloud computing is globally recognized as the single most effective, agile and scalable path to digitally transform and drive value creation. It has been a critical catalyst for growth, allowing private organizations and governments to support consumers and citizens alike, delivering services quickly without prohibitive capital investment. European organizations—in both the public and private sectors—want a provider to deliver a cloud on their terms, one that meets their requirements for security, privacy, and digital sovereignty, without compromising on functionality or innovation.

Last year, we set out an ambitious vision of sovereignty along three distinct pillars: data sovereignty (including control over encryption and data access), operational sovereignty (visibility and control over provider operations), and software sovereignty (providing the ability to run and move cloud workloads without being locked-in to a particular provider, including in extraordinary situations such as stressed exits). After extensive dialogue with customers and policymakers, we are today unveiling ‘Cloud. On Europe’s Terms’. As part of  this initiative, we will continue to demonstrate our commitment to deliver cloud services that provide the highest levels of digital sovereignty, all while enabling the next wave of growth and transformation for Europe’s businesses and organizations.

Google Cloud’s baseline controls and security features offer strong protections, meet current robust security and privacy requirements, and address many customer needs. Yet each country in Europe has its own characteristics and expectations. Certain customers in Europe may require more flexibility than current public and private cloud offerings may provide. We want to deliver a platform that allows customers to deploy workloads with the desired local control, without losing the transformational benefits of the public cloud.

We are now delivering on this new vision collaboratively with trusted local technology providers in Europe, starting with T-Systems in Germany. Today, together with T-Systems, we announced a partnership to build a Sovereign Cloud offering in Germany for private and public sector organizations. The offering will become available in mid 2022 with additional features being added over time.  

In this new joint offering, T-Systems will manage sovereignty controls and measures, including encryption and identity management of the Google Cloud Platform. In addition, as part of their offering, T-Systems will operate and independently control key parts of the Google Cloud infrastructure for T-Systems Sovereign Cloud customers in Germany.

We are committed to building trust with European governments and enterprises with a cloud that meets their digital sovereignty, sustainability and economic objectives. We are starting with T-Systems today and will continue by partnering with trusted technology providers in selected markets across the region. 

Customers in other markets across Europe will be able to use these trusted partner offerings or use Google Cloud’s controls to exercise autonomous control over data access and use; exercise choice over the infrastructure that is used to process that data; and avoid cloud vendor lock-in. 

With Google Cloud, our customers also automatically benefit from sustainable business transformation on the cleanest cloud in the industry. Today, we are the largest annual corporate purchaser of renewable energy globally, and by 2030, we aim to operate entirely on 24/7 carbon-free energy in all of our cloud regions worldwide. 

We’ll continue to listen to our customers and key stakeholders across Europe who are setting policy and helping shape requirements for customer control of data. Our goal is to make Google Cloud the best possible place for sustainable, digital transformation for European organizations on their terms—and there is much more to come.

As Google Cloud has become a choice for more startups, I’ve experienced an increase in founders asking how they should think about cloud services. Though each startup is different and requirements may vary across industries and regions, I’ve seen a few core best practices that help startups to succeed—as well as several traps to avoid. 

For example, If you go with a public cloud provider, you’re ideally not starting from ground zero like you would running your own data center, but it’s important not to introduce similar complexity in a virtualized environment. Just because you’re using public cloud infrastructure doesn’t mean you want to manage it.

Instead, you want to leverage platforms that abstract away complexity so your team can focus on delivering value to customers. That’s where serverless comes in. Serverless platforms are fully managed by the provider, offering automatic scaling for workloads, as well as provisioning, configuring, patching, and management of servers and clusters. Freed from these resource-intensive tasks, your technical talent can focus on the things that differentiate your business, not on IT curation. 

This applies to not only running stateless applications, but also managing and analyzing data. You should be collecting data points about which features are most popular on your platform, what people are buying, the types of activities that help your customers get their jobs done, and so on. This information is essential to building and executing on a product roadmap that will serve your customers. However, it’s not enough to collect this data, you need to make your data accessible, secure, and easy for your team to analyze—so where do you run and host it? 

On Google Cloud, this is where options like Spanner and Cloud SQL can play a large role, as can BigQuery for analysis. You won’t have to worry about standing up infrastructure or patching servers—you can just stream your data to our data management platforms, where it’s available whenever someone needs to run a query. By leveraging a serverless architecture, your startup can be data-driven without having to invest in the traditional complexity of database administration and management—and that can significantly change your playing field. 

Serverless is just one of the factors you should consider as you build out your tech stack. To hear my thoughts on a range of other topics relevant to startups — such as security, cloud credits, and the differences among managed services — check out the below video or visit our Build and Grow page.

https://www.youtube.com/embed/gn4RjLbs8Hc?enablejsapi=1&