Over the past several years, Google Cloud has made significant investments in our infrastructure product portfolio. We launched new Tau T2D VMs, which deliver 42% better price-performance vs. other leading cloud providers. We upgraded Cloud Storage to offer more flexibility to support customers’ enterprise and analytics workloads, with dual-region buckets and upcoming Turbo Replication. And we’ve delivered numerous improvements to our global network, including expansion to 29 cloud regions.

However, from conversations with customers, we’ve also learned we can do more to align our capabilities and pricing with their varied workloads. So, today, we are announcing we will adjust our infrastructure product and pricing structure to give customers more choice in how they pay for what they use alongside new, flexible SKUs with new product options and capabilities. These changes are designed to help ensure better product fit for our customers’ use cases across a wider array of workloads. They are also designed to better align with how other leading cloud providers charge for similar products, so customers can more easily compare services between leading cloud providers.

Some of these changes will provide new, lower-cost options and features for Google Cloud products. Other changes will raise prices on certain products. Ultimately, our goal is to provide more flexible pricing models and options for how customers are using our cloud services. Here’s an overview of what customers can expect:

Which services are changing? What new services are being introduced?

We are changing prices for some storage, compute, and networking products. The changes provide customers with new ways to optimize their spending based on workload type and size, or data portability needs, as well as reducing costs on some services. Specific changes include:

• Cloud Storage pricing changes for data mobility, including replication of data written to a dual- or multi-region storage bucket, and inter-region data access
• Introduction of a new lower-cost archive snapshot option for Persistent Disk (PD), so that compliance/archiving use cases are charged less than compute-intensive DevOps workloads
• New outbound data processing pricing for Cloud Load Balancing, in line with other leading cloud providers
• New pricing for Network Topology, which will include Performance Dashboard within Network Intelligence Center at no additional charge

Will customers’ bills increase? Decrease?

The impact of the pricing changes depends on customers’ use cases and usage. While some customers may see an increase in their bills, we’re also introducing new options for some services to better align with usage, which could lower some customers’ bills. In fact, many customers will be able to adapt their portfolios and usage to decrease costs. We’re working directly with customers to help them understand which changes may impact them.

When will the new prices go into effect?

Today, we sent customers a six-month notice on the price changes, which go into effect on October 1, 2022. Customers under existing commit contracts with a floating or fixed discount will not face any changes until renewal. Our goal is to help our customers manage any impact of these changes and allow time for them to adjust or modify their implementations.

What should customers do next?

There are a number of things customers can do to prepare for the changes:

• Read through the Mandatory Service Announcement (MSA) sent on March 14.
• Consider what actions, if any, they may want to take based on current storage, networking, and compute needs. Many of these changes may have simple choices associated with them.
• Consider using the Storage Transfer Service to select the right Cloud Storage bucket locations. Storage Transfer Service will be available free-of-cost for transfers within Cloud Storage, starting April 2 until the end of the year.

For those customers under contract, Google Cloud account representatives are available to discuss these changes. Please visit our pricing page and the links below for more details on our updates to storage, networking, and PD pricing, including information on how to modify your implementations if needed. If you do not have an account manager and still have questions please review our public FAQ, which will be updated regularly, as well as the resource links below.

Note: This pricing analysis is valid as of February 2022.

Resources:

• Cloud Storage Pricing Announcements 
• Load Balancing pricing Announcements
• Network Intelligence Center pricing Announcements
• PD Pricing Announcements
• Public FAQ

Google Cloud’s AI-optimized infrastructure makes it possible for businesses to train, fine-tune, and run inference on state-of-the-art AI models faster, at greater scale, and at lower cost. We are excited to announce the preview of inference on Cloud TPUs. The new Cloud TPU v5e enables high-performance and cost-effective inference for a broad range AI workloads, including the latest state-of-the-art large language models (LLMs) and generative AI models.

As new models are released and AI becomes more sophisticated, businesses require more powerful and cost efficient compute options. Google is an AI-first company, so our AI-optimized infrastructure is built to deliver the global scale and performance demanded by Google products like YouTube, Gmail, Google Maps, Google Play, and Android that serve billions of users — as well as our cloud customers. 

LLM and generative AI breakthroughs require vast amounts of computation to train and serve AI models. We’ve custom-designed, built, and deployed Cloud TPU v5e to cost-efficiently meet this growing computational demand.

Cloud TPU v5e is a great choice for accelerating your AI inference workloads: 

• Cost Efficient: Up to 2.5x more performance per dollar and up to 1.7x lower latency for inference compared to TPU v4.
• Scalable: Eight TPU shapes support the full range of LLM and generative AI model sizes, up to 2 trillion parameters. 
• Versatile: Robust AI framework and orchestration support. 

In this blog, we’ll dive deeper into how you can leverage TPU v5e effectively for AI inference.

Up to 2.5x more performance per dollar and up to 1.7x lower latency for inference

Each TPU v5e chip provides up to 393 trillion int8 operations per second (TOPS), allowing complex models to make fast predictions. A TPU v5e pod consists of 256 chips networked over ultra-fast links. Each TPU v5e pod delivers up to 100 quadrillion int8 operations per second, or 100 PetaOps, of compute power.

We optimized the Cloud TPU inference software stack to take full advantage of this powerful hardware. The inference stack leverages XLA, Google’s AI compiler, which generates highly-efficient code for TPUs to maximize performance and efficiency.

The combined hardware and software optimizations, including int8 quantization, enable Cloud TPU v5e to achieve up to 2.5x greater inference performance per dollar than Cloud TPU v4 on state-of-the-art LLM and generative AI models, including Llama 2, GPT-3, and Stable Diffusion 2.1:

Google Internal Data. August 2023. Normalized to single-chip throughput. Precision: Llama 2 7B, 13B, 70B, GPT-J 6B: int8; GPT-J 175B, Stable Diffusion 2.1: bf16.

On latency, Cloud TPU v5e achieves up to 1.7x speedup compared to TPU v4:

Google Internal Data. August 2023. Precision: Llama 2 7B, 13B and 70B: int8; GPT-3 175B: bf16.

Google Cloud customers have been running inference on Cloud TPU v5e, and some have seen even greater speedups on their particular workloads.

AssemblyAI offers dozens of AI models to their customers for speech recognition and understanding with over 25 million inference calls on a daily basis. 

“Cloud TPU v5e consistently delivered up to 4X greater performance per dollar than comparable solutions in the market for running inference on our production model. The Google Cloud software stack is optimized for peak performance and efficiency, taking full advantage of the TPU v5e hardware that was purpose-built for accelerating the most advanced AI and ML models. This powerful and versatile combination of hardware and software dramatically accelerated our time to solution: instead of spending weeks hand-tuning custom kernels, within hours we optimized our model to meet and exceed our inference performance targets.” – Domenic Donato, VP of Technology, AssemblyAI

Scale to the full range of LLM and Generative AI model sizes 

LLMs and generative AI models continue to grow in size and computational cost. The largest models require the combined compute and memory of hundreds of hardware accelerators. Cloud TPU v5e enables inference for a wide range of model sizes. A single v5e chip can run models with up to 13B parameters. From there, you can scale up to hundreds of chips and run models with up to 2 trillion parameters.

Google Internal Data. August 2023. Batch size = 1. Multi-head attention based decoder only language models: prefix length = 2048, decode steps = 256, beam size = 32 for sampling.

Gridspace leverages Google Cloud TPU infrastructure to power its full-stack conversational AI platform – building and integrating real-time conversational ASR, LLMs, semantic search, and neural TTS.

“We’re a huge fan of Google Cloud TPUs. Our benchmarks are demonstrating a 5X increase in the speed of AI models when training and running on Google Cloud TPU v5e. We are also seeing a 6x improvement in the scale of our inference metrics. We’ve scaled our AI models to billions of conversations per year across financial services, capital markets, and healthcare with Google Cloud’s AI infrastructure. Our Grace bots are powered by models trained using Cloud TPUs and served at scale on GKE with support for PCI, HITRUST, and SOC 2 compliance.” – Wonkyum Lee, Head of Machine Learning, Gridspace 

Robust AI framework and orchestration support

Leading AI frameworks, including PyTorch, JAX, and TensorFlow, provide robust support for inference on Cloud TPU v5e. This means you can now train and serve models end-to-end on Cloud TPUs: what you train is what you serve.

Google Cloud offers you many choices to run inference on Cloud TPUs easily and reliably. From GKE and Vertex AI, to popular open-source frameworks such as Ray and Slurm, you can leverage Google Cloud TPUs in your preferred way to fit your development process.

Try Cloud TPU v5e for inference today

Cloud TPU v5e provides a high-performance, cost-efficient, scalable, and reliable inference platform for LLMs and generative AI models. Leading AI companies are leveraging the power of Cloud TPU v5e to serve AI models at scale:

To get started with inference on Cloud TPU, reach out to your Google Cloud account manager or contact Google Cloud sales.

Businesses that operate in complex cloud environments, large fleets, or sophisticated security operations all require visibility into their cloud assets in order to keep their teams nimble and their data secure. Cloud Asset Inventory (CAI) helps these teams understand their Google Cloud  and Anthos environments by providing complete visibility, real-time monitoring, and powerful asset analysis capabilities. Today, Cloud Asset Inventory gets four new capabilities that help you understand your environment more clearly and easily than ever before.

New user interface eases asset and insight discovery 

Cloud Asset Inventory console preview is now publicly available for GCP and Anthos customers. This preview provides insights into your cloud footprint, history and details of resource usage with powerful filtering and search capabilities. For example, you can view your global distribution of resources and policies, how your GCE VM footprint has been changing over time, as well as full metadata and change history for all your assets. The CAI console can be filtered at the organization, folder, or project-level, so each user can view the resources they have permissions for down to project level granularity.

Asset discovery and Datadog integration

A new asset list service in CAI provides quick and comprehensive asset discovery, including asset history, without needing to export the data to a storage destination. Datadog, a leading multi-cloud monitoring and security service provider, relies on deep integration with CAI for service and asset discovery. Datadog has been piloting and taking full advantage of the newly released asset list service. Datadog Product Manager, Steve Harrington, commented: 

“Google’s new Cloud Asset Inventory API provides us with an immensely valuable, single source of truth for determining the resources present in a given GCP environment. Along with its rich metadata, this enables us to enhance multiple aspects of our integration with GCP, including streamlined metric collection and ingestion of custom labels. We plan to continue building around Cloud Asset Inventory in the future to improve existing features, and are envisioning ways it could help us provide entirely new insights to our customers.”

Answer “who can access what resources?”

Determining authoritative answers to security-related questions like “Who can read data from my storage bucket that contains PII?” or “Does a terminated employee still have any remaining access to my system?” can be difficult and time consuming. This is why access management and identity certification is one of the top security priorities for enterprises running workloads in the cloud. To help alleviate this challenge, the new Policy Analyzer capability in CAI thoroughly analyzes the relationship between IAM policies and resources. The analysis includes powerful and efficient group expansion, service account impersonation, conditional access analysis, resource expansion, and more. You can even export the results to a BigQuery table or Cloud Storage bucket for further analysis and record keeping. CAI’s enhanced UI makes it even easier for you to build your own flexible queries and quickly get to a comprehensive answer.

Create asset posture visibility 

Cloud Asset Inventory now provides seven types of Asset Insights through the Active Assist platform. These new asset insights help proactively detect anomalies within your organization’s IAM policies, which may be opportunities to improve your security posture. The insights can be aggregated at the Organization, Folder or Project level. 

The seven new Asset Insights include: 

• External members in IAM policies. 
• External users that impersonate your service accounts.
• External members as policy editors.
• External users who can view cloud storage buckets.
• Terminated users/groups that are still in IAM policies
• IAM policies containing all users or all authenticated users.
• Projects with only terminated users as owners.

As a Google Cloud customer you can get started and use all the recently released capabilities and features immediately; check out our documentation to see how. We’d love to hear your feedback; email us with any questions or concerns!

Quanta Services is the leading specialty contractor with the largest and highly-skilled trained workforce in North America. It provides fully-integrated solutions for the electric power

pipeline industrial and telecommunications industries the company’s geographic footprint which includes North America Latin America and Australia.

It’s network of companies ensures world-class execution with local delivery and has over 40,000 employees.

“The exciting thing about Backstory is it allows us to land massive amounts of data from all of our different security tools and then Chronicle worries about how to correlate and aggregate this information for us. We can then focus on the highest priority threats.”

Richard Breaux, Manager Security Operations Quanta Services

Because Quanta Services’ customers provide energy and telecommunications to their customers, they have world-class cybersecurity requirements.

“We must ensure that Quanta leads the industry given the fact that our business is building the core infrastructure that powers people’s lives. It’s crucial that we meet their cybersecurity requirements,” says James Stinson, VP-IT, Quanta Services.

To support this goal Quanta implemented a number of security tools that generate terabytes of security telemetry. Over time, however, these systems generated an information overload for the company’s limited pool of qualified security resources.

“Our logging tools also couldn’t keep up with the rapidly growing information. With limited resources we need to focus our security analysts time on high quality work instead of digging through mountains of data,” says Stinson.

They achieve this by implementing Backstory and Chronicle, which is on Google Cloud.

“The exciting thing about Backstory is it allows us to land massive amounts of data from all of our different security tools and then Chronicle worries about how to correlate and aggregate this information for us. We can then focus on the highest priority threats,” says Richard Breaux, Manager Security Operations Quanta Services.

As a result, the company’s security  team spends less time getting to the core information they need to address these incidents.

“What used to take us 15 minutes or more, we can now accomplish in seconds,” says Breaux.