This past year brought both new challenges and successes for our customers around the world. We thought life was going to return to normal… and then it didn’t. Despite the uncertainty, it was exciting to see customers make huge transformations in when, where, and how they run their businesses, with cloud infrastructure as a major enabler. 

According to IDC, “by 2024, 50% of organizations will use applications built on abstraction provided by managed services including cloud-native technologies to enable consistency in running in any and many locations.”¹ Over the past year we were honored to work with so many of you to start to make this prediction a reality.

Let’s take a deeper look at where our infrastructure progressed in 2021, and where we’re headed in the new year.

Google Cloud is recognized by independent experts for its leadership

For the fourth consecutive year, Google Cloud was named a Leader in the Gartner Magic Quadrant for Cloud Infrastructure and Platform Services.

The Forrester Wave for AI infrastructure placed Google at the topmost position, a testament to Google’s investments in tomorrow’s AI-first world. 

Google Cloud also won the HPC Wire Editor’s Choice award for the best use of HPC in the cloud.    

We made it easier to migrate enterprise applications

To support enterprises in their transformation journeys we focused on expanding our support for enterprise applications in our cloud. The Home Depot just wrote about its successful migration of SAP to Google Cloud, highlighting how crucial it is to have good processes and strong partnerships. For other companies using SAP applications we also launched Filestore Enterprise, which delivers 99.99% regional availability ​​backed by an SLA. 

Our partner NetApp deepened its integration with Google Cloud to enable easier and faster Windows-based application migration, and provided flexible deployment options to modernize workloads. We also added new Network Connectivity Center partners to include companies like Cisco, Palo Alto Networks, and VMware, so as you migrate over time you can easily connect all your networking resources together in one place. 

For those of you considering migration to cloud, Forrester dove deeper into the economics of migrating enterprise applications to cloud. (Spoiler alert: companies are finding major cost savings and performance benefits!) It’s so important to us that companies trust that their biggest, most complex applications will run smoothly and securely with us, and we’re pleased at the progress we’ve made toward supporting more enterprise applications this year.

We also made it easier for you to migrate VMware workloads to the cloud by expanding Google Cloud VMware Engine to 12 regions worldwide and enabling several new capabilities across networking, compliance, and scale. Customers such as Mitel and Carrefour migrated their VMware estates to Google Cloud to transform their applications with Google services to increase agility, save money (up to 45%) and reduce energy costs (about 35%) compared to running on-premises. 

For High Performance Computing (HPC), we continued to add capabilities such as optimized VM images for HPC, enhanced integration with schedulers such as SchedMD, Slurm, and Altair PBSPro.    

New cost and performance options

To further support customers doing high performance computing for things like real-time advertising, dynamic e-commerce, and gaming, we launched a new VM family: Tau VMs. Tau VMs offer 56% higher absolute performance and 42% higher price-performance compared to general-purpose VMs from any of the leading public cloud vendors.  We are seeing companies such as Nylas switching from other cloud platforms to Google Cloud to make the most of Tau VMs.

That isn’t the only VM advancement we made. For customers with higher levels of fault tolerance and looking for greater cost efficiencies, we launched Spot VMs, which open up access to Google Cloud’s idle capacity so you can run your application at the lowest price possible. With Spot VMs, you save anywhere from 60-91% off the price of on-demand VMs. 

We also launched more options for the highest performance ephemeral block storage — 6TB and 9TB Local SSDs, which offer greater IOPS per dollar when attached to general purpose N2 Compute Engine VMs. 

Distributed cloud brings us closer to you

For years, the industry has been telling companies, “move to cloud!” but not all workloads can move to the public cloud immediately or entirely. Factors include industry- or region-specific compliance and data sovereignty needs, low latency or local data-processing requirements, or the need to run applications close to other services. 

In October, we expanded our distributed cloud strategy and announced Google Distributed Cloud, a portfolio of fully managed hardware and software solutions that extend Google Cloud’s infrastructure and services to data centers and the edge. This is a major step forward in our ability to meet you where you are, in private data centers and at the edge, and we’re excited to see how this helps you as we move into 2022.

Data protection and security that never sleeps

This year we worked hard to help improve your data protection capabilities. We doubled down on our Actifio acquisition by launching several releases that deepened its integration with Google Cloud. For customers with containerized applications, we took a big step forward with the launch of Backup for GKE. We’re particularly excited about how this new option for GKE users allows you to more easily meet your service-level objectives, automate common backup and recovery tasks, and show reporting for compliance and audit purposes. 

We also enhanced Cloud Storage with data protection features like custom region selection for dual-region buckets. Previously, Google Cloud assigned dual-region pairs for you to choose from. With this release, you can  select your own region pairs that meet your regulatory or compliance requirements, or optimize your app performance. We also launched Turbo Replication for dual-region buckets, which replicates 100% of your data between regions in 15 minutes or less, backed by a Service Level Agreement, a first from a leading cloud provider. 

Security is a key area of focus for us and we partnered with Palo Alto Networks to deliver their industry-leading Intrusion Detection System (IDS) solution natively on Google Cloud. With the recent Apache Log4j vulnerability, we automatically updated Cloud IDS to help detect exploit attempts and protect our customers’ environments quickly. We also integrated Cloud Armor with reCAPTCHA Enterprise to deliver a best-in-class bot and fraud management solution to prevent volumetric attacks. Cloud Armor is deployed with our Cloud Load Balancer and Cloud CDN, extending its security benefits at the network edge for traffic coming into Google Cloud. 

We made it easier to build and manage applications

Developers want to focus on code, not configuring, managing or scaling infrastructure. To help, we worked on simplifying networking with new services such as Private Service Connect, which allows you to connect VPCs to applications and services securely without configuring all the network underlay. To make it easier to run workloads in hybrid environments, we introduced BYOIP so you don’t need to change your IP address, and we enhanced Cloud Load Balancer with advanced traffic management, regional and hybrid app delivery to load balance traffic between on-prem and cloud workloads. We also introduced IPv6, DNS policy manager, Cloud Domains, GKE Gateway Controller, eBPF data plane, and Service Directory, all with the goal of making networking easier for developers.

We continue to invest in visibility and observability for efficient day 2 operations. Network Intelligence Center now has dynamic reachability within the Connectivity Tests module, interconnect and VPN visualization in Network Topology, and the Firewall Insights module, which provides visibility into firewall rules to ensure they are being used appropriately and as intended. 

And while not new this year, developers continue to tell us how much they love the ease of architecting storage using Cloud Storage. See how easy it is to configure a dual-region bucket. As a result, developers can treat a continent like a single bucket, dramatically simplifying the application programming model. Google Cloud is unique in offering this capability among major public cloud vendors.

We continue to be the most sustainable cloud for you

Another area Google has invested in deeply and that is becoming increasingly important to more organizations is sustainability. Many cloud providers have a vision for a sustainable future, and many aim to match their electricity consumption with 100% renewable energy by 2025 or 2030. We accomplished 100% renewable energy in 2017, which means we’re the only hyperscale cloud to do this today and on data centers that are twice as energy efficient as the average data center.

We also announced a number of new features like our new Carbon Footprint tool. Every Google Cloud user — that means you! — can see the gross carbon emissions associated with the services you use in Google Cloud. This means picking the data center that meets your sustainability goals, and reducing the overall emissions tied to your infrastructure and applications. And we have some pretty big goals for the future. For more, don’t miss our full sustainability recap.

A big thank you to our customers

While we were busy building out our infrastructure in 2021, we’d be nowhere without the partnership of our amazing customers. It’s hard to pick from all the exciting stories we heard this year, but here are just a few: Pega migrated its SAP applications to Google Cloud with zero downtime. Paypal exited its non-strategic data centers and achieved major cost savings. And Wix used our network to serve tens of millions of requests per day. 

I’m proud to look back at how our cloud infrastructure has evolved this year to support each of our customers on their transformation journey. As we head into 2022 I’m excited to continue to partner with you to solve your most difficult challenges together. 

Interested in hearing more about Google’s vision for the future? Listen to Thomas Kurian’s thoughts from Google Cloud Next ‘21.

^{1. IDC FutureScape: Worldwide Cloud 2022 Predictions}

Cloud offers a proven way to accelerate end-to-end chip design flows. In a previous blog, we demonstrated the inherent elasticity of the cloud, showcasing how front-end simulation workloads can scale with access to more compute resources. Another benefit of the cloud is access to a powerful, modern and global infrastructure. On-prem environments do a fantastic job of meeting sustained demand but Electronic Design Automation (EDA) tooling upgrades happen much more frequently (every six to nine months) than typical on-prem data center infrastructure upgrades (every three to five years). 

What this means is that your EDA tool can provide much better performance if given access to the right infrastructure. This is especially useful in certain phases of the design process.

Take for example, a physical verification workload. Physical verification is typically the last step in the chip design process. In simplified terms, the process consists of verifying design rule checks (or DRCs) against the process design kit (PDK) provided by the foundry. It ensures that the layout produced from the physical synthesis process is ready for handoff to a foundry (in-house or otherwise) for manufacturing. Physical verification workloads tend to require machines with large memories (1TB+) for advanced nodes. Having access to such compute resources enables more physical verification to run in parallel, increasing your confidence in the design that is being taped out (i.e., sent to manufacturing).

At the other end of the spectrum are functional verification workloads. Unlike the physical verification process described above, functional verification is normally performed in the early stages of design and typically requires machines with much less memory. Furthermore, functional verification (dynamic verification in particular) accounts for the most time (translating directly to the availability of compute) in the design cycle. Verifying faster, an ambition for most design teams, is often tied to availability of right-sized compute resources. 

The intermittent and varied infrastructure requirements for verification (both functional and physical) can be a problem for organizations with on-prem data centers. On-prem data centers are optimized for maximizing utilization—this does not directly address access to right-sized compute to deliver the best tool performance. Even if the IT and Computer Aided Design (CAD) departments choose to provision additional suitable hardware, the process of provisioning, acquiring and setting up new hardware on-prem typically takes months for even the most modern organizations. A “hybrid” flow that enables use of on-prem clusters most of the time, but provides seamless access to cloud resources as needed would be ideal.

Hybrid chip design in action

You can improve a typical verification workflow simply by utilizing a hybrid environment that provides instantaneous access to better compute. To illustrate, we chose a front-end simulation workflow, and designed an environment that replicates on-prem and cloud clusters. We also took a few more liberties to simplify the environment (described below). The simplified setup is provided in a GitHub repository for you to try out.

In any hybrid chip design flow, there are a few key considerations:

1. Connectivity between on-prem infrastructure and the cloud: Establishing connectivity to the cloud is one of the most foundational aspects of the flow. Over the years, this has also become a very well-understood field, and secure, high availability connectivity is a reality in most setups. 
   
   In our tutorial, we represent both on-prem and cloud clusters as two different networks in the cloud where all traffic is allowed to pass between these networks. While this is not a real-world network configuration, it is sufficient to demonstrate the basic connectivity model.
   
2. Connection to license server: Most chip design flows utilize tools from EDA vendors. Such tools are typically licensed, and you need a license server with valid licenses to operate the tool. License servers may remain on-prem in the hybrid flow, so long as latency to the license server is acceptable. You can also install license servers in the cloud on a Compute Engine VM (particularly sole-tenant nodes) for lower latency. Check with your EDA vendors to understand if you can rehost your license services in the cloud.
   
   In our tutorial, we use an open source tool (Icarus Verilog Simulator) and therefore, do not need a license server.
   
3. Identifying data sources and syncing data: There are three important aspects in running EDA jobs: the EDA tools themselves, the infrastructure where the tools run, and the data sources for the tool run. Tools don’t change much, and can be installed on cloud infrastructure. Data sources, on the other hand, are primarily created on-prem and updated regularly. These could be SystemVerilog files that describe the design, the testbenches or the layout files. It is important to sync data between on-prem and cloud to maintain parity. Furthermore, in production environments, it’s also important to maintain a high-performance syncing mechanism.
   
   In our tutorial, we create a file system hierarchy in the cloud that is similar to one you’d find on-prem. We transfer the latest input files before invoking the tool.
   
4. Workload scheduler configuration and job submission transparency: Most environments that leverage batch jobs use job schedulers to access a compute farm. An ideal environment finds the balance between cost and performance, and builds parameters in the system to enable predictive (and prescriptive) wrappers to job schedulers (see picture below).
   
   In our tutorial, we use the open-source SLURM job scheduler and an auto-scaling cluster. For simplicity, the tutorial does not include a job submission agent.

Other cloud-native batch processing environments such as Kubernetes can also provide further options for workload management.

Our on-prem network is called ‘onprem’ and the cloud cluster is called ‘burst’. Characteristics of the on-prem and burst clusters are specified below:

Once set up, we ran the OpenPiton regression for single and two-tile configurations. You can see the results below:

Regressions run on “burst” clusters were on average 30% faster than on “onprem”, delivering faster verification sign-off and physical verification turnaround times. You can find details about the commands we used in the repository. 

Hybrid solutions for faster time to market

Of course, on-prem data centers will continue to play a pivotal role in chip design. However, things have changed. Cloud-based, high performance compute has proved itself to be a viable and proven technology for extending on-prem data centers during the chip design process. Companies that successfully leverage hybrid chip design flows will be able to better address the fluctuating needs of their engineering teams. To learn more about silicon design on Google Cloud, read our whitepaper “Using Google Cloud to accelerate your chip design process”.