Customers love Cloud Spanner because it gives them the benefits of relational semantics and SQL while also delivering the scale and availability of non-relational databases. Many of these customers want to move even more of their work to Spanner, and have requested smaller instance sizes to support development, testing and small production workloads. We’re happy to announce that they’ll soon get what they asked for: more granular instance sizing is coming to Spanner.

Granular instance sizing will be available in Public Preview soon. With this feature, you can run workloads on Spanner at as low as 1/10th the cost of regular instances, equating to approximately $65/month.

In addition, we are launching new enterprise capabilities to break down operational silos for real-time insights and provide greater database observability to developers: 

• Datastream, now in public preview, is a change data capture (CDC) service that allows enterprises to synchronize data across heterogeneous databases and applications. With Spanner support in Datastream, users will be able to stream data from MySQL or Oracle to Spanner reliably and with minimal latency. 
• BigQuery federation to Spanner (coming soon) lets users query transactional data residing in Spanner, from BigQuery, without moving or copying data. 
• Key Visualizer, available now in public preview, provides interactive monitoring so developers can quickly identify trends and usage patterns in Spanner. 

Democratizing access with more granular instance sizing 

Today, customers provision Spanner instances by specifying the number of nodes they need to run their workloads. Each node can provide up to 10,000 queries per second (QPS) of reads or 2,000 QPS of writes (writing single rows at 1 KB of data per row) and 2 TB of storage. A Spanner node is replicated across 3 zones for regional instances, and 5 or more zones for multi-regional instances. The choice of node count determines the amount of serving and storage resources that are available to the databases in a given instance. 

Historically, the most granular unit for provisioning resources on Spanner has been one node. To enable more granular control, we are introducing Processing Units (PUs); one Spanner node is equal to 1,000 PUs. Customers can now provision in batches of 100 PUs, and get a proportionate amount of compute and storage resources. This will allow teams to run smaller workloads on Spanner at much lower cost. With this feature, customers can start at 100 PUs and scale up as needed in batches of 100 PUs, to up to 1,000 PUs (1 node), all with zero downtime. Subsequently, customers can continue to scale up by adding more nodes, just like what they do today. Customers do have the choice of using either PUs or nodes to provision resources within workloads, when those workloads occupy multiple nodes of capacity. 

To illustrate with an example, let’s say a game developer creates a Spanner instance called “baseball-game” in us-central1, with 100 PU compute capacity at $65/month price.

As you can see below, proportional maximum storage of 205 GB is assigned to the 100 PU instance.

Once the game starts becoming popular and resource demands increase, the user edits the instance to increase the compute capacity to 500 PUs with proportional maximum 1 TB of storage.

The game grows in popularity, and the gaming company prepares to launch a much-awaited capability in the game. Anticipating a sharp increase in usage at launch day, the game developers increase the compute capacity to 3,000 PUs. Compute capacity assigned to the instance over a period of time can be viewed in Cloud Monitoring graphs:

Request Access 
You can request early access to granular instance sizing feature by filling this form.

Breaking down operational silos with Datastream CDC and BigQuery federation

BigQuery Federation. BigQuery has made analytics easy by bringing together data from multiple sources for seamless analysis. Soon you’ll also be able to analyze data in Spanner directly in BigQuery. With  Spanner’s BigQuery federation, you’ll be able to instantly query data residing in Spanner in real-time without moving or copying the data. Simply set up the Spanner as an external data source in BigQuery, as shown below.

Change Data Capture ingest. Now in public preview, Datastream lets you stream change data into Google Cloud from MySQL and Oracle databases. As of today, you can ingest this change data directly into Spanner using a built-in Dataflow template. This lets you migrate data from MySQL and Oracle databases into Spanner in near-real time.

Understanding performance and resource usage with Key Visualizer

Key Visualizer is a new interactive monitoring tool that lets developers and administrators analyze usage patterns in Spanner. It reveals trends and outliers in key performance and resource metrics for databases of any size, helping to optimize queries and reduce infrastructure costs. Designed for performance tuning and instance sizing, Key Visualizer is available today in public preview in the web-based Cloud Console for all Spanner databases at no additional cost. Learn more in the Key Visualizer blog. 

Learn more

• To request early access to granular instance sizing in Spanner, fill out this form.
• To get started with Spanner today, create an instance or try it out with a Spanner Qwiklab.

When you choose to run your business on Google Cloud you benefit from the same planet-scale infrastructure that powers Google’s products such as Maps, YouTube, and Workspace. 

We have picked 10 ways in which Google Cloud Infrastructure services outshine alternatives in the market in how they simplify your operations, save money, and secure your data. 

1. Custom Machine Types means no wasted resources

Compute Engine offers predefined machine types that you can use when you create a VM instance. A predefined machine type has a preset number of vCPUs and a preset amount of memory; each type is billed at a set price as described on the Compute Engine pricing page. 

If predefined machine types don’t meet your needs, you can create a VM instance with a custom number of vCPUs and custom amount of memory, effectively building a custom machine type. Custom machine types are available only for general-purpose machine families. When you create a custom machine type, you are deploying a custom machine type from the E2, N2, N2D, or N1 machine family on GCP.  No other leading cloud vendor offers custom machine types so extensively.

Custom machine types are a good idea for workloads that aren’t a good fit for the predefined machine types and for workloads that require more processing power or memory but don’t need all of the upgrades provided by the next machine type level. This translates into lower operating costs.   They are also useful for controlling software licensing costs that are based on the number of underlying compute cores. 

Jeremy Lloyd, Infrastructure and Application Modernization Lead at Appsbroker, a Google partner: 

“Custom machine types coupled with Google’s StratoZone data center discovery tool provides Appsbroker with the flexibility we need to provide cost efficient virtual machines matched to a virtual machine’s actual utilization. As a result, we are able to keep our customers’ operating costs low while still providing the ability to scale as needed.”

2. Compute Engine Virtual Machines are optimized for scale-out workloads 

For scale-out workloads, T2D, the first instance type in the Tau VM family, is based on 3rd Gen AMD EPYC processors and leapfrogs VMs for scale-out workloads of any leading public cloud provider today, both in terms of performance and price-performance. Tau VMs offer 56% higher absolute performance and 42% higher price-performance compared to general-purpose VMs from any leading public cloud vendor (source). The x86 compatibility provided by these AMD EPYC processor-based VMs gives you market-leading performance improvements and cost savings, without having to port your applications to a new processor architecture. Sign up here  if you are interested in trying out T2D instances in Preview. 

For SAP HANA, Google Cloud has demonstrated with SAP how we can run the world’s largest scale-out HANA system in the public cloud (96TB).   With such innovation, you are covered as your business grows exponentially.

3. Largest single node GPU-enabled VM

Google is the only public cloud provider to offer up to 16 NVIDIA A100 GPUs in a single VM, making it possible to train very large AI models. Users can start with one NVIDIA A100 GPU and scale to 16 GPUs without configuring multiple VMs for single-node ML training, without crossing the VM layer. 

Additionally, customers can choose smaller GPU configurations—1, 2, 4 and 8 GPUs per VM—providing the flexibility to scale their workload as needed. 

The A2 VM family was designed to meet today’s most demanding applications—workloads like CUDA-enabled machine learning (ML) training and inference, for example. This family is built on the A100 GPU which offers up to 20x the compute performance compared to the previous generation GPU and comes with 40 GB of high-performance HBM2 GPU memory. To speed up multi-GPU workloads, the A2 VMs use NVIDIA’s HGX A100 systems to offer high-speed NVLink GPU-to-GPU bandwidth that delivers up to 600 GB/s. A2 VMs come with up to 96 Intel Cascade Lake vCPUs, optional Local SSD for workloads requiring faster data feeds into the GPUs and up to 100 Gbps of networking. A2 VMs provide full vNUMA transparency into the architecture of underlying GPU server platforms, enabling advanced performance tuning. Google Cloud offers these GPUs globally. 

4. ​​Non-disruptive maintenance means you worry less about planned downtime

Compute Engine offers live migration (non-disruptive maintenance) to keep your virtual machine instances running even when a host system event, such as a software or hardware update, occurs. Google’s Compute Engine live migrates your running instances to another host in the same zone without requiring your VMs to be rebooted. Live migration enables Google to perform maintenance that is integral to keeping infrastructure protected and reliable without interrupting any of your VMs. When a VM is scheduled to be live-migrated, Google provides a notification to the guest that a migration is imminent. 

Live migration keeps your instances running during:

• Regular infrastructure maintenance and upgrades
• Network and power grid maintenance in the data centers
• Failed hardware such as memory, CPU, network interface cards, disks, power, and so on. This is done on a best-effort basis; if a hardware component fails completely or otherwise prevents live migration, the VM crashes and restarts automatically and a hostError is logged.
• Host OS and BIOS upgrades
• Security-related updates
• System configuration changes, including changing the size of the host root partition, for storage of the host image and packages

Live migration does not change any attributes or properties of the VM itself. The live migration process transfers a running VM from one host machine to another host machine within the same zone. All VM properties and attributes remain unchanged, including internal and external IP addresses, instance metadata, block storage data and volumes, OS and application state, network settings, network connections, and so on. This has the benefit of reducing operational and maintenance overhead, helps you build a more robust security posture where infrastructure can be consciously revamped from a known good state and minimizes risks for advanced persistent threats. 

Refer to Lessons learned from a year of using live migration in production on Google Cloud from the Google engineering team.

5. Trusted Computing: Shielded VMs guard you against advanced, persistent attacks

Establishing trust in your environment is multifaceted, involving hardware and firmware, as well as host and guest operating systems. Unfortunately, threats like boot malware or firmware rootkits can stay undetected for a long time, and an infected virtual machine can continue to boot in a compromised state even after you’ve installed legitimate software. 

Shielded VMs can help you protect your system from attack vectors like:

• Malicious guest OS firmware, including malicious UEFI extensions
• Boot and kernel vulnerabilities in the guest OS
• Malicious insiders within your organization

To guard against these kinds of advanced persistent attacks, Shielded VMs use:

• Unified Extensible Firmware Interface (UEFI) BIOS: Helps ensure that firmware is signed and verified
• Secure and Measured Boot: Helps ensure that a VM boots an expected, healthy kernel
• Virtual Trusted Platform Module (vTPM): Establishes root-of-trust, underpins Measured Boot, and prevents exfiltration of vTPM-sealed secrets
• Integrity Monitoring: Provides tamper-evident logging, integrated with Stackdriver, to help you quickly identify and remediate changes to a known integrity state

The Google approach allows customers to deploy Shielded VMs with only a simple click, thereby easing implementation. 

6. Confidential Computing encrypts data while in use

Google Cloud was a founding member of the Confidential Computing Consortium. Along with encryption of data in transit and at rest using customer-managed encryption keys (CMEK) and customer-supplied encryption keys (CSEK), Confidential VM adds a “third pillar” to the end-to-end encryption story by encrypting data while in use. Confidential Computing uses processor-based technology that allows data to be encrypted in use while it is being processed in the public cloud. Confidential VM allows you to to encrypt memory in use on a Google Compute Engine VM by checking a single checkbox. 

All Confidential VMs support the previously mentioned Shielded VM features under the covers—you can think of Shielded VM as helping to address VM integrity, while Confidential VM addresses the memory encryption aspect which relies on CPU features. With the confidential execution environments provided by Confidential VM and AMD Secure Encrypted Virtualization (SEV), Google Cloud keeps customers’ sensitive code and other data encrypted in memory during processing. Google does not have access to the encryption keys. In addition, Confidential VM can help alleviate concerns about risk related to either dependency on Google infrastructure or Google insiders’ access to customer data in the clear. 

See what Google Cloud partners say about Confidential Computing here. 

7. Advanced networking delivers full-stack networking and security services with fast, consistent, and scalable performance

Google Cloud’s network delivers low latency, reduces operational costs and ensures business continuity, enabling organizations to seamlessly scale up or down in any region to meet business needs. Our planet-scale network uses advanced software-defined networking and security with edge caching services to deliver fast, consistent, and scalable performance. With 28 regions, 85 zones, and 146 PoPs connected by 16 subsea fiber cables around the world, Google Cloud’s network offers a full stack of layer 1 to layer 7 services for enterprises to run their workloads anywhere. Enterprises can be assured that they have best-in-class networking and security services connecting their VMs, containers, and bare metal resources in hybrid and multi-cloud environments with simplicity, visibility, and control. 

Google Cloud’s network has protected customers from one of the world’s largest DDoS attacks at 2.54 Tbps. With our multi-layer security architecture and products such as Cloud Armor, our customers ran their business with no disruptions. Furthermore, our recent integration of Cloud Armor with reCAPTCHA Enterprise adds best-in-class bot and fraud management to prevent volumetric attacks. Cloud Armor is deployed with our Cloud Load Balancer and Cloud CDN, extending the secure benefits at the network edge for traffic coming into Google Cloud so customers have security, performance, and reliability all built in. Furthermore, we are excited to offer Cloud IDS in preview, which was co-developed with security industry leader, Palo Alto Networks, to run natively in Google Cloud. 

Our advanced networking capabilities also extends to GKE and Anthos networking. With the GKE Gateway controller, customers can manage internal and external HTTPS load balancing for a GKE cluster or a fleet of GKE clusters with multi-tenancy while maintaining centralized admin policy and control. Unlike other Kubernetes offerings, we offer eBPF dataplane which brings powerful tooling such as Kubernetes network policy and logging to GKE. eBPF is known to kernel engineers as a “superpower” for its unique architecture to load and unload modules in kernel space, and now this capability is built in with Google Cloud networking. 

For observability and monitoring, our customers deploy Network Intelligence Center, Google Cloud’s comprehensive network monitoring, verification and optimization platform. With four key modules in Network Intelligence Center, and several more to come, we are working towards realizing our vision of proactive network operations that can predict and heal network failures, driven by AI/ML recommendations and remediation. Network Intelligence Center provides unmatched visibility into your network in the cloud along with proactive network verification. Centralized monitoring cuts down troubleshooting time and effort, increases network security and improves the overall user experience.  

8. Regional Persistent Disk for High Availability

Regional Persistent Disk is a storage option that provides synchronous replication of data between two zones in a region. Regional Persistent Disks can be a great building block if you need to ensure high availability of your critical applications as they offer cost-effective durable storage and replication of data between two zones in the same region. 

Regional Persistent Disks are also easy to set up within the Google Cloud Console. If you are designing robust systems or high availability services on Compute Engine, Regional Persistent Disks combined with other best practices such as backing up your data using snapshots enable you to build an infrastructure that is highly available and recoverable in a disaster. Regional Persistent Disks are also designed to work with regional managed instance groups. In the unlikely event of a zonal outage, Regional Persistent Disks allow continued I/O through failover of your workloads to another zone. Regional Persistent Disks can help meet zero RPO and near-zero RTO requirements and other stringent SLAs that your critical applications might require by maximizing application availability and protection of data during events such as host/VM failures and zonal outages. 

9. Cloud Storage’s single namespace for dual-region and multi-region means managing regional replication is incredibly simple

Similar to how Persistent Disk makes data more available by replicating data across zones, Cloud Storage provides similar benefits for object storage. Cloud Storage within a region is cross-zone by definition, reducing the risk that a zonal outage would take down your application. Cloud Storage adds to this by also providing a cross-region option that can protect against a regional outage and gets your data closer to distributed users. This comes in the form of Dual-region or Multi-region settings for a bucket. These are the simplest to implement cross-region replication offerings in the industry—just a simple button or API call to enable them. In addition to being simple to implement, they offer an added advantage of using a single bucket name that spans regions. 

This is unique in the industry. Competitive offerings currently require setting up and managing two distinct buckets, one in each region and they don’t offer the strong consistency properties Cloud Storage offers across regions. Operations and app development are burdened by this design. Google’s single namespace approach dramatically simplifies application development (the app runs on single region or dual/multi-region without any changes), and provides simpler application restarts and testing for DR.

10. Predictive autoscaling 

Customers use predictive autoscaling to improve response times for applications with long initialization times or for applications with workloads that vary predictably with daily or weekly cycles. When you enable predictive autoscaling, Compute Engine forecasts future load based on your Managed Instance Group’s history and scales out the MIG’s in advance of predicted load, so that new instances are ready to serve when the load arrives. Without predictive autoscaling, an autoscaler can only scale a group reactively, based on observed changes in load in real time. 

With predictive autoscaling enabled, the autoscaler works with real-time data as well as with historical data to cover both the current and forecasted load. Forecasts are refreshed every few minutes (faster than competing clouds) and consider daily and weekly seasonality, leading to more accurate forecasts of load patterns.

For more information, see How predictive autoscaling works and Checking if predictive autoscaling is suitable for your workload.

These are just a few examples of customer-centric innovation that set Google Cloud infrastructure apart.  Bring your applications and let the platform work for you.   

Get started by learning about your options for migration, or talk to our sales team to join the thousands of customers who have embarked upon this journey.

Acknowledgement

Special thanks to Dheeraj Konidena (Google) for contributing to this article.

Scale-out workloads demand the best combination of performance and price to bring down the cost of delivering applications, all while providing an excellent user experience. We are excited to announce a new virtual machine (VM) family, Tau VMs, coming to Google Cloud. Tau VMs extend Compute Engine’s VM offerings with a new option optimized for cost-effective performance of scale-out workloads. 

T2D, the first instance type in the Tau VM family, is based on 3rd Gen AMD EPYC^TM processors and leapfrogs the VMs for scale-out workloads of any leading public cloud provider available today, both in terms of performance and workload total cost of ownership (TCO). The x86 compatibility provided by these AMD EPYC processor-based VMs gives you market-leading performance improvements and cost savings, without having to port your applications to a new processor architecture. 

As illustrated below, Tau VMs offer 56% higher absolute performance and 42% higher price-performance (est. SPECrate2017_int_base) compared to general-purpose VMs from any of the leading public cloud vendors.

What our customers and partners are saying

Snap
“At Snap, it is critical for our business to continue improving our scale-out compute infrastructure for key Snapchat capabilities like AR, Lenses, Spotlight and Maps,” said Cody Powell, Senior Engineering Manager, Snap Inc. “We were impressed when we tested Google Cloud’s new Tau VMs with Google Kubernetes Engine. While it’s early days, we believe we can gain double digits in infrastructure performance improvements for key workloads—enabling us to do more with less and invest even more in new features for our amazing Snapchat community.”

Twitter
“High performance at the right price point is a critical consideration as we work to serve the global public conversation,” said Nick Tornow, Platform Lead, Twitter. “We are excited by initial tests that show potential for double digit performance improvement. We are collaborating with Google Cloud to more deeply evaluate benefits on price and performance for specific compute workloads that we can realize through use of the new Tau VM family.”

DoiT
“DoiT partners with leading cloud vendors who are focused on growth and cost optimization,” said Yoav Toussia-Cohen, CEO, DoiT International. “In our preliminary testing of Google’s new Tau VMs with the Coremark benchmark, we were thrilled to see the incredible performance at 50% better than a comparable ARM-based offering from another leading public cloud. With Tau VMs, Google Cloud has set a new bar for price-performance, making the cloud even more accessible to digital-native companies. We are excited to bring Google’s Tau VMs to our joint customers.”

Designed for demanding scale-out workloads 

Tau VMs bring the benefit of Google’s long-standing experience engineering platforms for scale-out workloads to our customers. They come in multiple predefined VM shapes, with up to 60vCPUs per VM, and 4GB of memory per vCPU. They offer up to 32 Gbps networking bandwidth and a wide range of network attached storage options, making Tau VMs ideal for scale-out workloads including web servers, containerized microservices, data-logging processing, media transcoding, and large-scale Java applications. 

Google Kubernetes Engine support

Google Kubernetes Engine (GKE) is the de facto standard for organizations looking for advanced container orchestration, delivering the highest levels of reliability, security, and scalability. GKE supports Tau VMs on day 1, helping you optimize price-performance for your containerized workloads. You can add Tau VMs to your GKE clusters by specifying the T2D machine type in your GKE node-pools. 

Pricing

Tau VMs will be priced to support significant TCO and price-performance improvements for your cloud applications. A 32vCPU VM with 128GB RAM will be priced at $1.3520 per hour for on-demand usage in us-central1. 

Coming soon to a Google Cloud region near you

If you are interested in trying out T2D VMs when they become available in Q3 2021 please sign-up here.

Editor’s note: This post was originally published in TechPulse Belgium, where Jack Buser, Google Cloud’s Director of Game Industry Solutions, shared his trends for the industry this year.

The year 2022 will hold many surprises (with a few already dropping!), but there’s one near certainty: By this time next year there will be millions more gamers worldwide. It’s thanks to the industry’s little-noted technology growth drivers, which are getting stronger. 

Games are no longer a niche hobby; it’s global, mainstream entertainment. There were 3.1 billion gamers worldwide in 2020. It’s estimated that by 2024 there will be another 500 million. With a global population of 7.7 billion, that’s nearly half the planet. Revenues in 2021 may reach $175 billion—likely more than movies, music, and books combined. 

It’s a strength unique to the games industry, driven by the way its creators deploy new technology. Advances we see in everything from pricing and emerging markets, to backend computing and delivery suggests it will grow from here, on things like artificial intelligence and planet-scale networks.

Games forever, game better

It’s no surprise that videogames have become a huge business. Playing games is one of our deepest human expressions. There is a 4,500 year old board game that people still play, a testimony to the human love of testing cleverness, honing reactions, building trust, or just enlivening a day.

Video games do all that and for decades game makers have taken the most cutting-edge computing tech to make things better. We’ve seen electronic ping pong become shooting asteroids, to simple (yet iconic) 2D sprites evolve into believable, expressive 3D characters, all leading to today’s immersive worlds hosting tens of millions of players worldwide.

There’s more to come. I work at technology’s cutting edge for games, offering the world’s leading game creators access to powerful cloud computing, a low-latency global network, Machine Learning and AI, and much more. As usual, game innovators are taking up all these tools, building new narratives, new ways to play, new markets, and new jaw-dropping moments.

And game creators aren’t just thinking about players; they’re also thinking about viewers. More and more people are watching eSports, and creators are using the games medium to engage their viewers in increasingly immersive ways.

Streaming like never before 

AI and ML are sharpening games and attracting new players in other ways. These technologies increasingly power everything from network performance to global player matching. The delicate but essential work of encouraging newcomers and discouraging bad actors, optimizing monetization, or building and scaling up new player environments.

Another important area we’re seeing cloud tools being put to use is in coordinating the workflows of global teams of developers. It’s not just a question of building faster anymore, but of collaborating and responding more effectively. The distributed work trends that featured so strongly during the pandemic had already featured in much of the game industry, increasingly it’s an industry standard. This will likely mean even faster and more diverse game development, addressing the needs of new markets where we’re seeing double-digit growth.

Tools like cloud streaming, 5G, and edge computing are likely to accelerate the use of Game Streaming, Augmented Reality, and other types of immersive gameplay. If my metaverse doesn’t have games, count me out! We’re looking at some exciting new developments in coming months with some major industry players. Stay tuned.

A big year for the games industry

As a premier provider of solutions, tools and services to games companies, our engineering job is complex and global. Fortunately, our mission is simple: Help game companies transform to meet new global opportunities with planet scale solutions. Technology has brought the gaming industry to astonishing heights, and is the perfect compliment to great storytelling and creative ingenuity. 

As a decades-long industry veteran, I’ve never been more impressed with what technology can do for our industry. Looking past 2022, when we’ll see more compute, more AI, more AR, and more innovation around healthy and exciting game play, there is just one near certainty: More growth.