As the saying goes, “it’s hard to make predictions, especially about the future.” Some organizations find it challenging to predict what cloud resources they’ll need in months or years ahead. Every organization is on its own unique cloud journey. To help, we’re developing new ways for customers to consume and pay for Google Cloud services. We’re doing this by removing barriers to entry, aligning cost to consumption and providing contractual and product flexibility. Read on to learn how we’re rolling out several new go-to-market programs across these key areas to help our customers purchase and consume Google Cloud services more easily.

Removing barriers to entry with Google Cloud Flex Agreements

Many customers choose multi-year commitments because they provide better line-of-sight into IT spend and budgeting. However, these commitments can create difficulty for those who don’t have clear visibility into their future cloud consumption needs. That’s why today we’re launching Flex Agreements, which enable customers to migrate their workloads to the cloud with no up-front commitments. As part of this new licensing option, Google Cloud customers still get access to unique incentives, such as monthly spend discounts¹, committed use discounts, cloud credits, and access to professional services, based on monthly spend and workloads migrated to Google Cloud.

Flex Agreements are just one example of how we are removing barriers to help customers start using Google Cloud. In 2022, we launched the Innovators Plus annual subscription, which gives developers a curated toolkit to accelerate their expertise, including access to live and on-demand training through Google Cloud Skills Boost, Google Cloud credits, and more. 

We also recently expanded trials for Google Cloud products. For example, the new Spanner free trial instance is good for 90 days, allowing developers to create Google Standard SQL or PostgreSQL databases, explore Spanner capabilities, and prototype applications—with no commitment or contract needed. 

Contractual and feature flexibility  

Contractual flexibility has always been one of our core principles. Committed Use Discounts (CUDs), for example, provide discounted prices in exchange for a commitment to use a minimum level of resources for a specified term. Last year, we introduced Flexible CUD, spend-based commitments that offer predictable and simple flat-rate discounts that apply across multiple virtual machine families and regions.

In addition to contractual flexibility, our customers also need the flexibility to choose features and functionality based on their stages of cloud adoption and the complexity of their business requirements. Therefore, over the next few quarters, we will launch new product pricing editions—Standard, Enterprise, and Enterprise Plus—in parts of our cloud portfolio. This new commercial packaging model will help give customers more choice and flexibility to optimize their cloud spend.

For customers running workloads such as those in regulated industries like banking and public sector, the higher-end Enterprise Plus tier will offer compute, storage, networking and analytics services with high availability, multi-region support, regional failover and disaster recovery, advanced security, and a broad range of regulatory compliance support. The Enterprise pricing tier will include a broad range of features designed for customers with workloads that demand a high level of scalability, flexibility, and reliability. The Standard pricing tier will offer cost-efficient and easy-to-use managed services that include all essential capabilities such as autoscaling to meet the core workload requirements of customers.

Align costs to consumption with autoscaling

At Google Cloud, a core requirement for the products we build is providing customers industry-leading capabilities to automatically scale (autoscale) services up and down to match capacity with real-time demand. Autoscaling improves uptime, reduces infrastructure costs, and removes the operational burden of managing resources.  

Many Google Cloud products include autoscaling capabilities to help customers manage unplanned variations in demand. For example, Dataflow vertical and horizontal autoscaling, in combination with granular adaptive resource configuration (aka “right-fitting”), has resulted in up to 50% saving in infrastructure costs for streaming by automatically choosing the right number of instances required to run the jobs and dynamically re-allocating more or fewer instances during the runtime of jobs. Bigtable also provides native autoscaling capabilities, and Spanner’s autoscale is an open source tool that works across regional and multi-regional Spanner deployments. 

Similarly, we added multiple features such as Cluster Autoscaler, Horizontal Pod Autoscaling, Vertical Pod Autoscaling, and Node Auto-Provisioning to GKE for elasticity and cost efficiency. 

For L.L.Bean, the ability to quickly scale capacity to meet changing usage patterns (e.g., during the holidays), as well as to rapidly perform load tests to test capacity, are “night and day” with Google Cloud compared to L.L.Bean’s legacy on-premises IT system.

“We won’t have to pay for peak capacity to have it available during peak shopping times. We just scale capacity up or down as needed.” — Randy Dyer, Enterprise Architect, L.L.Bean

We are now taking these capabilities to the next level by enabling autoscaling in BigQuery at a more granular level so you never pay more than what you use. This allows you to provision additional capacity in smaller increments, so you never overprovision and overpay for underutilized capacity. BigQuery customers can now try the new BigQuery autoscaler (currently in public preview) in their Google Cloud console.

A commitment to flexibility and choice

At Google Cloud, we remain deeply committed to the success of our customers and partners, and we are uniquely positioned to help organizations transform their business. By providing you with more flexibility and choice in how to purchase our products, we are empowering you to be more efficient and resilient.

Join Google Data Cloud & AI Summit to hear the latest announcements around innovations in Google Data Cloud for databases, data analytics, business intelligence, and AI. Gain expert insights, new solutions, and strategies that can help you transform customer experiences with modern apps, boost revenue, and reduce costs.

^{1. Not available for customers buying through Partner Advantage.}

Many people start a small business that aligns with their passions, but soon discover the day-to-day running of a business is very different than anticipated. Dealing with accounting, finance, and other daily activities can quickly overwhelm even the most promising of new businesses.

Recognizing the unique challenges facing small businesses, we founded Beany. With our personalized accounting platform, small business owners can relax knowing that we will look after all their compliance and advisory needs. Business owners who use Beany also benefit because they end up freeing more time to focus on what they enjoy and do best—likely the main reasons they started their business in the first place. We prepare financial statements, minimize tax and keep an eye that our clients are paying the right amount at the right time. We also advise on business purchase and sale, valuations and forecasting for banks. In fact, we do everything that a business owner needs from their accountant, including a help desk filled with domain experts to provide free & unlimited advice and support. 

As we expand our team, we’ll continue to introduce new solutions, services, and integrations that make it even easier for small businesses to prioritize business planning, balance budgets, and keep up with ever-changing tax laws. We also plan to launch Beany in new markets and extend our global footprint beyond New Zealand, Australia, and the UK.    

Migrating and scaling Beany

Beany was initially developed and hosted on a virtual private server (VPS) with limited capabilities. As our subscriber base grew, we realized we needed a more scalable, secure solution to eliminate downtime and deliver a reliable customer experience. We also understood it would be challenging for our small engineering team to cost-effectively test and deploy new features without a more agile development environment. 

With that in mind, we chose the secure-by-design infrastructure of Google Cloud to power Beany and help us scale our innovative accounting platform. We now use Compute Engine to cost-effectively create virtual machines (VMs) that automatically select optimal amounts of processing and memory. 

In addition, we encrypt, store, and archive everything on highly secure Cloud Storage, leveraging a combination of solid-state drives (SSDs) and hard disk drives (HDDs) for hot, nearline, and coldline data. With Cloud Storage, we can replicate data between regions in under 15 minutes to enable rapid recovery and business continuity. This has been really useful moving images to data centers on the other side of the planet quickly as we primarily operate out of Sydney and London. 

We also connect, create, and collaborate on Google Workspace using Google Gmail, Docs, Calendar, and Meet. Beany smoothly integrates with Google Sheets to provide live reporting from our application direct to our sales, marketing and support staff. It also makes sharing account data with our accounting clients easy and secure.

Analyzing financial data with Google Cloud AI and machine learning

Joining the Google for Startups Cloud Program gave us immediate access to Google Cloud credits which we use to cost-effectively trial and deploy additional Google Cloud solutions. We continue to evaluate Cloud Code which will help our developers more efficiently create, deploy, and integrate applications directly on Google Cloud. We’re also exploring how Google Cloud AI and machine learning products such as Vertex AI and AutoML can further revolutionize accounting with advanced financial analyses and end-to-end automation of business processes.

We can’t wait to see what we accomplish next as we grow our team and introduce new services and solutions that enable small businesses to streamline operations, lower costs, and increase profits. It’s exciting to help small business owners manage accounting and financial planning so they can spend more time doing what they love and realize their business vision.

Beany team members

If you want to learn more about how Google Cloud can help your startup, visit our page here to get more information about our program, and sign up for our communications to get a look at our community activities, digital events, special offers, and more.

I’m excited to share our first Cloud CISO Perspectives post of 2022. It’s already shaping up to be an eventful year for our industry and we’re only in month one. There’s a lot to recap in this post, including the U.S. government’s recent efforts to address critical security issues, like open source software security and zero trust architectures. We’ve also released new resources from our Google Cybersecurity Action Team like the Cloud Security Megatrends and the Boards of Directors whitepaper on cloud risk governance. 

Cloud Security Megatrends 

We’re often asked if the cloud is more secure than on-prem (and why) so we shared our answer in a recent blog post. At Google Cloud, security by design is our priority. We’ve long adopted zero-trust principles for our baseline security architectures and built a global network that relies on defense in depth layers to protect against configuration errors and attacks. But security is always evolving and that is why we also take advantage of the following megatrends:

1. Economy of scale: Decreasing the marginal cost of security raises the baseline level of security. 
2. Shared fate: A flywheel of increasing trust drives more transition to the cloud, which compels even higher security and even more skin-in-the-game from the cloud provider.
3. Healthy competition: The race by deep-pocketed cloud providers to create and implement leading security technologies is the tip of the spear of innovation. 
4. Cloud as the digital immune system: Every security update the cloud gives the customer is informed by some threat, vulnerability, or new attack technique often identified by someone else’s experience. Enterprise IT leaders use this accelerating feedback loop to get better protection.
5. Software-defined infrastructure: Cloud is software defined, so it can be dynamically configured without customers having to manage hardware placement or cope with administrative toil. From a security standpoint, that means specifying security policies as code, and continuously monitoring their effectiveness.
6. Increasing deployment velocity: Because of cloud’s vast scale, providers have had to automate software deployments and updates, usually with automated continuous integration/continuous deployment (CI/CD) systems. That same automation delivers security enhancements, resulting in more frequent security updates.
7. Simplicity: Cloud becomes an abstraction-generating machine for identifying, creating and deploying simpler default modes of operating securely and autonomically. 
8. Sovereignty meets sustainability: The cloud’s global scale and ability to operate in localized and distributed ways creates three pillars of sovereignty. This global scale can also be leveraged to improve energy efficiency.

If you’re an IT decision maker, pay attention to these megatrends that will continue to drive and reinforce cloud security and will outpace the security of on-prem infrastructure well into the future. 

U.S. Federal government cybersecurity momentum 

• Open source software security: Earlier this month, Google participated in the White House Summit on open source software security. The meeting came at a critical time for the industry following December’s Log4j vulnerabilities and was both a recognition of the challenge and an important first step towards addressing it. The open source software ecosystem is not homogenous, despite the fact that the industry often thinks of or treats it this way. Some of it, like Linux, is highly curated, while other critical software is supported through diffuse communities including technology companies and other stakeholders. There is also a long tail of many other critical projects driven by a dedicated community of maintainers around the world, including Googlers. In light of this reality, we welcomed the chance to share our recommendations to advance the future of open source software security. Some work we’ve done includes founding the Open Source Security Foundation, which has been instrumental already in making security improvements. We’ve also helped drive a number of key security initiatives within the open source community including security scorecards, the SLSA framework to improve the security and integrity of open source packages, and Secure Open Source Rewards to financially incentivize improvements to critical open source security projects.  
• OMB’s Federal zero trust strategy: The publication of the Office of Management and Budget’s zero trust architecture strategy marks an important step for the U.S. federal government’s efforts to modernize under Executive Order 14028. Google Cloud supports this approach, which recognizes the immense security benefits offered by modern computing architectures. For the past decade, Google has successfully applied zero trust principles through our BeyondCorp and BeyondProd frameworks for providing end-user access and securing our cloud workloads. And we’ve brought these best practices from our own journey to global governments and businesses of any size through solutions like BeyondCorp Enterprise and capabilities like Binary Authorization and Anthos Service Mesh, which are embedded in Anthos, our managed application platform. For Federal agencies embarking on this zero trust journey, the Google Cybersecurity Action Team will offer our expertise by conducting Zero Trust Foundations strategy workshops, which can help organizations in the public and private sectors develop actionable and achievable strategies and plans for zero trust implementation. 

Google Cybersecurity Action Team Highlights 

Here are the latest updates, products, services and resources across our security teams this month: 

Security

• Democratizing security operations: We recently announced that Siemplify, a leading security orchestration, automation and response (SOAR) provider, is joining Google Cloud to help companies better manage their threat response. Providing a proven SOAR capability with Chronicle’s approach to security analytics is an important step forward in our vision to advance invisible security and democratize security operations for every organization.
• Security by design: The Highmark Health security team is using “secure-by-design” techniques to address the security, privacy, and compliance aspects of its Living Health solution with Google Cloud’s Professional Services Organization (PSO). Google has long advocated for and followed security by design principles, which is why we’re continuously building enhanced security, controls, resiliency and more into our cloud products and services. 
• Secure collaboration for hybrid work environments: The Google Workspace team shared its recommendations for businesses as they prepare for the future of work,  where the hybrid/flexible work model is becoming standard practice and a new approach to security is essential.
• Anthos Policy Controller CIS Benchmark enforcement: A big part of our shared fate philosophy is to build secure products and not just security products. A recent example of this in action is embedding CIS benchmark policy conformance in the Anthos Policy Controller. We believe the more we embed approaches like this into our products, the more application and infrastructure teams can intrinsically embed security at the start and reduce toil for the security team.
• DevOps for technology-driven organizations and startups: A key success factor for many security programs is the partnership and integration with development teams, and there are some great resources and lessons in our DORA research.
• Security by design with Chrome OS: ABN AMRO’s Asia-Pacific region team recently shared how they are using Chrome OS and CloudReady to work securely in the cloud, reduce total cost of ownership, and add flexibility for employees. This is a great example of secure by design principles in the use of Chromium.

Risk & Compliance

• Boards of Directors summary guide to cloud risk governance: The latest whitepaper from the Google Cybersecurity Action Team outlines how boards of directors can prioritize safe, secure, and compliant adoption processes for cloud technologies within their organizations.  
• TruSight Risk Assessment of Google Cloud: TruSight recently released a comprehensive
  risk assessment report on Google Cloud. Our Enterprise Trust team collaborated on this robust assessment of Google Cloud services to validate the design and implementation of controls. TruSight’s risk assessment of our security controls will help customers accelerate and complete their risk management due diligence.
• Data governance: Check out this new blog series on data governance where our teams explain the role of data governance, its importance, and the necessary processes to run an effective data governance program. Implementing data governance will help maximize value derived from business data, build user trust, and ensure compliance with required security measures.

Controls and Products

• Encrypting Data Fusion: To help meet the security, privacy and compliance requirements of customers in regulated industries like finance or public sector, we announced the general availability of Customer Managed Encryption Keys (CMEK) integration for Cloud Data Fusion, which enables encryption of both user data and metadata at rest with a key that customers can control through our Cloud Key Management Service (KMS). 

Don’t forget to sign-up for our newsletter if you’d like to have our Cloud CISO Perspectives post delivered every month to your inbox. We’ll be back next month with more updates and security-related news.

Google Cloud’s AI-optimized infrastructure makes it possible for businesses to train, fine-tune, and run inference on state-of-the-art AI models faster, at greater scale, and at lower cost. We are excited to announce the preview of inference on Cloud TPUs. The new Cloud TPU v5e enables high-performance and cost-effective inference for a broad range AI workloads, including the latest state-of-the-art large language models (LLMs) and generative AI models.

As new models are released and AI becomes more sophisticated, businesses require more powerful and cost efficient compute options. Google is an AI-first company, so our AI-optimized infrastructure is built to deliver the global scale and performance demanded by Google products like YouTube, Gmail, Google Maps, Google Play, and Android that serve billions of users — as well as our cloud customers. 

LLM and generative AI breakthroughs require vast amounts of computation to train and serve AI models. We’ve custom-designed, built, and deployed Cloud TPU v5e to cost-efficiently meet this growing computational demand.

Cloud TPU v5e is a great choice for accelerating your AI inference workloads: 

• Cost Efficient: Up to 2.5x more performance per dollar and up to 1.7x lower latency for inference compared to TPU v4.
• Scalable: Eight TPU shapes support the full range of LLM and generative AI model sizes, up to 2 trillion parameters. 
• Versatile: Robust AI framework and orchestration support. 

In this blog, we’ll dive deeper into how you can leverage TPU v5e effectively for AI inference.

Up to 2.5x more performance per dollar and up to 1.7x lower latency for inference

Each TPU v5e chip provides up to 393 trillion int8 operations per second (TOPS), allowing complex models to make fast predictions. A TPU v5e pod consists of 256 chips networked over ultra-fast links. Each TPU v5e pod delivers up to 100 quadrillion int8 operations per second, or 100 PetaOps, of compute power.

We optimized the Cloud TPU inference software stack to take full advantage of this powerful hardware. The inference stack leverages XLA, Google’s AI compiler, which generates highly-efficient code for TPUs to maximize performance and efficiency.

The combined hardware and software optimizations, including int8 quantization, enable Cloud TPU v5e to achieve up to 2.5x greater inference performance per dollar than Cloud TPU v4 on state-of-the-art LLM and generative AI models, including Llama 2, GPT-3, and Stable Diffusion 2.1:

Google Internal Data. August 2023. Normalized to single-chip throughput. Precision: Llama 2 7B, 13B, 70B, GPT-J 6B: int8; GPT-J 175B, Stable Diffusion 2.1: bf16.

On latency, Cloud TPU v5e achieves up to 1.7x speedup compared to TPU v4:

Google Internal Data. August 2023. Precision: Llama 2 7B, 13B and 70B: int8; GPT-3 175B: bf16.

Google Cloud customers have been running inference on Cloud TPU v5e, and some have seen even greater speedups on their particular workloads.

AssemblyAI offers dozens of AI models to their customers for speech recognition and understanding with over 25 million inference calls on a daily basis. 

“Cloud TPU v5e consistently delivered up to 4X greater performance per dollar than comparable solutions in the market for running inference on our production model. The Google Cloud software stack is optimized for peak performance and efficiency, taking full advantage of the TPU v5e hardware that was purpose-built for accelerating the most advanced AI and ML models. This powerful and versatile combination of hardware and software dramatically accelerated our time to solution: instead of spending weeks hand-tuning custom kernels, within hours we optimized our model to meet and exceed our inference performance targets.” – Domenic Donato, VP of Technology, AssemblyAI

Scale to the full range of LLM and Generative AI model sizes 

LLMs and generative AI models continue to grow in size and computational cost. The largest models require the combined compute and memory of hundreds of hardware accelerators. Cloud TPU v5e enables inference for a wide range of model sizes. A single v5e chip can run models with up to 13B parameters. From there, you can scale up to hundreds of chips and run models with up to 2 trillion parameters.

Google Internal Data. August 2023. Batch size = 1. Multi-head attention based decoder only language models: prefix length = 2048, decode steps = 256, beam size = 32 for sampling.

Gridspace leverages Google Cloud TPU infrastructure to power its full-stack conversational AI platform – building and integrating real-time conversational ASR, LLMs, semantic search, and neural TTS.

“We’re a huge fan of Google Cloud TPUs. Our benchmarks are demonstrating a 5X increase in the speed of AI models when training and running on Google Cloud TPU v5e. We are also seeing a 6x improvement in the scale of our inference metrics. We’ve scaled our AI models to billions of conversations per year across financial services, capital markets, and healthcare with Google Cloud’s AI infrastructure. Our Grace bots are powered by models trained using Cloud TPUs and served at scale on GKE with support for PCI, HITRUST, and SOC 2 compliance.” – Wonkyum Lee, Head of Machine Learning, Gridspace 

Robust AI framework and orchestration support

Leading AI frameworks, including PyTorch, JAX, and TensorFlow, provide robust support for inference on Cloud TPU v5e. This means you can now train and serve models end-to-end on Cloud TPUs: what you train is what you serve.

Google Cloud offers you many choices to run inference on Cloud TPUs easily and reliably. From GKE and Vertex AI, to popular open-source frameworks such as Ray and Slurm, you can leverage Google Cloud TPUs in your preferred way to fit your development process.

Try Cloud TPU v5e for inference today

Cloud TPU v5e provides a high-performance, cost-efficient, scalable, and reliable inference platform for LLMs and generative AI models. Leading AI companies are leveraging the power of Cloud TPU v5e to serve AI models at scale:

To get started with inference on Cloud TPU, reach out to your Google Cloud account manager or contact Google Cloud sales.