Do you want to fire up a bunch of techies? Talk about multicloud! There is no shortage of opinions. I figured we should tackle this hot topic head-on, so I recently talked to four smart folks—Corey Quinn of Duckbill Group, Armon Dadgar of Hashicorp, Tammy Bryant Butow of Gremlin, and James Watters of VMware—about what multicloud is all about, key considerations, and why you should (or shouldn’t!) do it.

Five important insights came out of these discussions. If you’re on a multicloud journey or considering one, keep reading.

Do: Choose to do multicloud for the right reasons

Don’t do multicloud because Gartner says so, implores Corey Quinn. Before embarking on a multicloud, define a “why” focused on business value journey, says Armon Dadger. For example, you might want to use services from each public cloud because of their differentiated services, according to Tammy Bryant Butow. Armon also calls out regulatory reasons, existing business relationships, and accommodating mergers and acquisitions. On the topic of M&A, Corey points out that if you acquire a company that uses another cloud, it’s usually expensive and difficult to consolidate. It can be smarter to stay put.

https://youtube.com/watch?v=xFSDexQhCUY%3Fenablejsapi%3D1%26

Don’t: Over-engineer for workload or data portability

Thinking that you’ll build a system that moves seamlessly among the various cloud providers? Hold up, says our group of experts. Armon points out that aspects of your toolchain or architecture may be multicloud—think of some of your workflows or global network routing—but that shifting workloads or data is far from simple. Corey says that trying to engineer for “write once, run anywhere” can slow you down, and ignores the inherent uniqueness that’s part of each platform. Specifically, Corey calls out the per-cloud stickiness of identity management, security features, and even network functionality. And data gravity is still a thing, says James, that causes some to dismiss multicloud outright.

If you’re using multiple public clouds, you take advantage of the distinct value each offers, Armon says. Use native cloud services where possible so that you see the benefits from useful innovations, built-in resilience, and baked-in best practices. The value from that cloud-infused workload may outweigh the benefits of seamless portability.

https://youtube.com/watch?v=B1VH56_L8f8%3Fenablejsapi%3D1%26

Do: Recognize different stakeholder interests and needs

James smartly points out that many multicloud debates happen because people are arguing from different perspectives. Context matters. If you’re an infrastructure engineer who invests heavily in a given cloud’s identity and access management model, multicloud looks tricky. Or if you’re a data engineer with petabytes of data homed in a particular cloud, multicloud may look unrealistic. James highlights that many developers default to multicloud because their local tools—where all the work happens—are multicloud. A developer’s IDE and preferred code framework(s) aren’t tied to any given cloud. Be aware that groups within your organization will come at multicloud from distinct directions. And this may impact your approach!

https://youtube.com/watch?v=I9sqXDqkKBM%3Fenablejsapi%3D1%26

Don’t: Go it alone

Corey talks about the importance of asking others what worked, and what didn’t. Tammy offers her best practices around sharing results from experiments. It’s about sharing knowledge and tapping into it for community benefit. Others have probably tried what you’re trying, and can help you avoid common pitfalls. If you’ve just made an architectural choice that didn’t work out, share it, and help others avoid the pain. 

Read research from analysts, go to conferences or watch videos to observe case studies, and join online communities that offer a safe place to share mistakes and learn from others.

https://youtube.com/watch?v=mrSb5vqOfuI%3Fenablejsapi%3D1%26

Do: Experiment first using techniques like multi-region deployments

If you think you can operate systems across clouds, how about you first try doing it across regions in a specific cloud, suggests Corey. Getting a system to properly work across cloud regions isn’t trivial, he says, and that experience can help you uncover where you have architectural or operational constraints that will be even worse across cloud providers.

This is great guidance if your multicloud aspirations involve using multiple clouds to power one application—versus the more standard definition of multicloud where you use different clouds for different applications—but can also surface issues in your support process or toolchain that fail when faced with distributed systems. Start with muti-region deployments and chaos engineering experiments before aggressively jumping into multicloud architectures.

The Google Cloud take

Do the things above. It’s great advice. I’ll add three more things that we’ve learned from our customers.

1. Don’t fear multicloud. You’re already doing it. You don’t single-source everything. As Corey mentioned, you probably already have one cloud for productivity tools, another for source code, another for cloud infrastructure. You’ll use software and application services from a mix of providers for a single app. You have that experience in your team and have been doing that for decades. What people do rightly worry about is using more than one infrastructure service beneath an application, as that can introduce latency, security, and logistical hurdles. Make sure you know which model your team is considering.
2. Embrace the right foundational components, including Kubernetes. Will everything run on Kubernetes? Of course not. Don’t try to do that. But it also represents the closest thing we have to a multicloud API. Companies are using Kubernetes to stripe a consistent experience across clouds. And this isn’t just to orchestrate containers, but also to manage infrastructure and cloud-native services. Also, consider where you need other fundamental consistency across clouds, including areas like provisioning and identity federation.
3. Use Google Cloud as your anchor. Here’s a fundamental question you have to decide for yourself: Are you going to bring your on-premises technology and practices to the cloud, or bring cloud technology and practices on-prem? We sincerely believe in the latter. Anchor to where you’re trying to get to. We offer Anthos as a way to build and run distributed Kubernetes fleets in Google Cloud and across clouds. By using a cloud-based backplane instead of an on-prem one, you’re offloading toil, leveraging managed services for scale and security, and introducing modern practices to the rest of your team.

We learned a lot about multicloud through these discussions, and it seems like others did too. That’s why we’re going to do a second round of interviews with a new crop of experts so that we can keep digging deeper into this topic. Stay tuned!

Google Cloud’s AI-optimized infrastructure makes it possible for businesses to train, fine-tune, and run inference on state-of-the-art AI models faster, at greater scale, and at lower cost. We are excited to announce the preview of inference on Cloud TPUs. The new Cloud TPU v5e enables high-performance and cost-effective inference for a broad range AI workloads, including the latest state-of-the-art large language models (LLMs) and generative AI models.

As new models are released and AI becomes more sophisticated, businesses require more powerful and cost efficient compute options. Google is an AI-first company, so our AI-optimized infrastructure is built to deliver the global scale and performance demanded by Google products like YouTube, Gmail, Google Maps, Google Play, and Android that serve billions of users — as well as our cloud customers. 

LLM and generative AI breakthroughs require vast amounts of computation to train and serve AI models. We’ve custom-designed, built, and deployed Cloud TPU v5e to cost-efficiently meet this growing computational demand.

Cloud TPU v5e is a great choice for accelerating your AI inference workloads: 

• Cost Efficient: Up to 2.5x more performance per dollar and up to 1.7x lower latency for inference compared to TPU v4.
• Scalable: Eight TPU shapes support the full range of LLM and generative AI model sizes, up to 2 trillion parameters. 
• Versatile: Robust AI framework and orchestration support. 

In this blog, we’ll dive deeper into how you can leverage TPU v5e effectively for AI inference.

Up to 2.5x more performance per dollar and up to 1.7x lower latency for inference

Each TPU v5e chip provides up to 393 trillion int8 operations per second (TOPS), allowing complex models to make fast predictions. A TPU v5e pod consists of 256 chips networked over ultra-fast links. Each TPU v5e pod delivers up to 100 quadrillion int8 operations per second, or 100 PetaOps, of compute power.

We optimized the Cloud TPU inference software stack to take full advantage of this powerful hardware. The inference stack leverages XLA, Google’s AI compiler, which generates highly-efficient code for TPUs to maximize performance and efficiency.

The combined hardware and software optimizations, including int8 quantization, enable Cloud TPU v5e to achieve up to 2.5x greater inference performance per dollar than Cloud TPU v4 on state-of-the-art LLM and generative AI models, including Llama 2, GPT-3, and Stable Diffusion 2.1:

Google Internal Data. August 2023. Normalized to single-chip throughput. Precision: Llama 2 7B, 13B, 70B, GPT-J 6B: int8; GPT-J 175B, Stable Diffusion 2.1: bf16.

On latency, Cloud TPU v5e achieves up to 1.7x speedup compared to TPU v4:

Google Internal Data. August 2023. Precision: Llama 2 7B, 13B and 70B: int8; GPT-3 175B: bf16.

Google Cloud customers have been running inference on Cloud TPU v5e, and some have seen even greater speedups on their particular workloads.

AssemblyAI offers dozens of AI models to their customers for speech recognition and understanding with over 25 million inference calls on a daily basis. 

“Cloud TPU v5e consistently delivered up to 4X greater performance per dollar than comparable solutions in the market for running inference on our production model. The Google Cloud software stack is optimized for peak performance and efficiency, taking full advantage of the TPU v5e hardware that was purpose-built for accelerating the most advanced AI and ML models. This powerful and versatile combination of hardware and software dramatically accelerated our time to solution: instead of spending weeks hand-tuning custom kernels, within hours we optimized our model to meet and exceed our inference performance targets.” – Domenic Donato, VP of Technology, AssemblyAI

Scale to the full range of LLM and Generative AI model sizes 

LLMs and generative AI models continue to grow in size and computational cost. The largest models require the combined compute and memory of hundreds of hardware accelerators. Cloud TPU v5e enables inference for a wide range of model sizes. A single v5e chip can run models with up to 13B parameters. From there, you can scale up to hundreds of chips and run models with up to 2 trillion parameters.

Google Internal Data. August 2023. Batch size = 1. Multi-head attention based decoder only language models: prefix length = 2048, decode steps = 256, beam size = 32 for sampling.

Gridspace leverages Google Cloud TPU infrastructure to power its full-stack conversational AI platform – building and integrating real-time conversational ASR, LLMs, semantic search, and neural TTS.

“We’re a huge fan of Google Cloud TPUs. Our benchmarks are demonstrating a 5X increase in the speed of AI models when training and running on Google Cloud TPU v5e. We are also seeing a 6x improvement in the scale of our inference metrics. We’ve scaled our AI models to billions of conversations per year across financial services, capital markets, and healthcare with Google Cloud’s AI infrastructure. Our Grace bots are powered by models trained using Cloud TPUs and served at scale on GKE with support for PCI, HITRUST, and SOC 2 compliance.” – Wonkyum Lee, Head of Machine Learning, Gridspace 

Robust AI framework and orchestration support

Leading AI frameworks, including PyTorch, JAX, and TensorFlow, provide robust support for inference on Cloud TPU v5e. This means you can now train and serve models end-to-end on Cloud TPUs: what you train is what you serve.

Google Cloud offers you many choices to run inference on Cloud TPUs easily and reliably. From GKE and Vertex AI, to popular open-source frameworks such as Ray and Slurm, you can leverage Google Cloud TPUs in your preferred way to fit your development process.

Try Cloud TPU v5e for inference today

Cloud TPU v5e provides a high-performance, cost-efficient, scalable, and reliable inference platform for LLMs and generative AI models. Leading AI companies are leveraging the power of Cloud TPU v5e to serve AI models at scale:

To get started with inference on Cloud TPU, reach out to your Google Cloud account manager or contact Google Cloud sales.

Road construction is booming in India. A recent report prepared by the India Brand Equity Foundation—a trust established by the Department of Commerce—pointed out that in FY 2019 alone, the country added 10,855 kilometers of highways to a road network that spans 5.89 million kilometers. This network is the second largest in the world and transports 90% of passenger traffic and 64.5% of all goods in the country.

The Indian Government has also earmarked road construction as key to plans to increase the nation’s GDP to $5 trillion in coming years, targeting road construction worth $212.8 billion in the two years from April 2020.

G R Infraprojects Limited is well positioned to support the government’s program. The business, which started as a contractor building roads in rural villages in India, now specializes in road engineering, procurement, and construction (EPC), a model whereby private construction firms build roads funded by the government.

The business now undertakes the processing of bitumen, manufacture of thermoplastic road-marking paint and road signage, and fabrication and galvanizing road-crash barriers. Its in-house integration model includes a design and engineering team, as well as manufacturing facilities in Rajasthan, Assam, and Gujarat.

The business recently expanded into rail—another area expected to benefit from extensive government investment—with its competencies including earthworks, materials supply, track lining, and bridge construction.

In this environment—and despite the economic impact of the coronavirus pandemic—G R Infraprojects Limited aims to substantially increase turnover and manpower over the next five years.

Best-in-class infrastructure key to success

Digital transformation is key to enabling growth while best-in-class IT infrastructure is one of the foundations on which the business seeks to build success.

G R Infraprojects Limited’s digital initiatives include deployment of a new document management system and corporate systems that enable remote monitoring, live tracking, effective real-time communication, and efficient data management.

Providing a scalable, reliable cost-effective infrastructure

But most important of all is providing a scalable, reliable, and cost-effective infrastructure to support a business-critical SAP enterprise resource planning system. Over the last few years, versions of SAP have enabled the organization to digitize processes and seamlessly run business-critical functions such as inventory management and finance.

G R Infraprojects Limited initially went live with SAP ECC6.0, with a few hundred team members using the system for business-critical tasks such as tracking stock level and movement and generating financial statements and reports for review and action.

“Google Cloud is a very big brand, so we were easily able to secure the trust from our executive and business teams to run an important system such as SAP S/4HANA on the platform.”—Sachin Kumar Agarwal, Head, Transformation, G R Infraprojects Limited

Lowering maintenance costs

However, as G R Infraprojects Limited grew, projects proliferated, and new markets emerged, the business elected to move to the cloud from an on-premises infrastructure. “We wanted to move because there were so many maintenance costs in on-premises solutions and cloud provided convenience to IT and the broader organization,” explains Sachin Kumar Agarwal, Head, Transformation at G R Infraprojects Limited.

The business also wanted to move to SAP S/4HANA to take advantage of features such as AI, advanced analytics, and machine learning to transform business processes. The system runs on the HANA database, an in-memory database with fast processing speeds and a simplified data model.

G R Infraprojects Limited selected Google Cloud to run SAP S/4HANA because, Sachin says, it is “much better than any other platform,” incorporates a wide range of features, and meets uptime requirements. The cloud service could also scale to support forecast growth without a sharp increase in cost.

Furthermore, Sachin adds, “Google Cloud is a very big brand, so we were easily able to secure the trust from our executive and business teams to run an important system such as SAP S/4HANA on the platform.”

“With Google Cloud, our speed and availability are controlled and optimized day by day. With such a scalable and dynamic platform, we are very happy with the performance.”—Sachin Kumar Agarwal, Head, Transformation, G R Infraprojects Limited

Successful transition with Infrabeat

G R Infraprojects Limited completed the project with assistance from partner InfraBeat over three months and SAP S/4HANA on Google Cloud went live mid-2019. “Our dedicated SAP team worked closely with Infrabeat to deliver the project successfully,” says Sachin. “We needed an experienced partner to assist with the implementation process and Infrabeat performed that role admirably. Both teams supported each other and worked to plan to deliver a great result.”

SAP S/4HANA runs on an infrastructure comprising virtual machine instances delivered through Compute Engine, Cloud Storage, and Cloud NAT to enable the secure transmission and receipt of packets to and from the internet.

G R Infraprojects Limited estimates the cost of running SAP S/4 HANA in Google Cloud is significantly lower than on alternative infrastructure options—freeing up budget for other business priorities.

In addition, moving SAP S/4 HANA to infrastructure as a service through Google Cloud has eliminated the need to assign internal team members to infrastructure management, allowing them instead to focus on higher-value activities.

Google Cloud also incorporates the security needed to protect the data and processes of SAP S/4 HANA from intrusion or disruption and ensure the uptime and continuity required of a business-critical system.

Optimized speed and availability

G R Infraprojects Limited’s decision to run SAP S/4 HANA on Google Cloud is delivering benefits on a daily basis. “With Google Cloud, our speed and availability are controlled and optimized day by day,” says Sachin. “We are very happy with the performance.”

Success with SAP S/4HANA has helped the business decide to move its remaining apps and data to the cloud when its on-premises servers and other equipment reach end of life. G R Infraprojects Limited is already running Active Directory in Google Cloud and Sachin says the cloud platform’s “fast and excellent services” made the decision easy.

“There are definitely instances of various upgrades of our systems and within the organization, and as we talk about our application and mobility requirements, we see Google Cloud playing a crucial role.”—Sachin Kumar Agarwal, Head, Transformation, G R Infraprojects Limited

Adding value

As G R Infraprojects Limited grows, Sachin adds, the business expects Google Cloud to continue to add value. “There are definitely instances of various upgrades of our systems and within the organization, and as we talk about our application and mobility requirements, we see Google Cloud playing a crucial role.”

In our conversations with technology leaders about data-driven transformation using Google Data Cloud –  industry’s leading unified data and AI solution – , one important topic is incorporating continuous intelligence to move from answering questions such as “What has happened? to questions like “What is happening?” and “What might happen?”. The core to this evolution is the need for an underlying data processing that not only provides powerful real-time capabilities for events happening close to origination, but also brings together existing data sources under one unified data platform to enable organizations to draw insights and take actions holistically. Dataflow, Google’s cloud-native data processing and streaming analytics platform, is a key component of any modern data and AI architecture and data transformation journey, along with BigQuery, Google’s internet-scale warehouse with built-in streaming, BI engine and ML; Pub/Sub, a global no-ops event delivery service; and Looker, a modern BI and embedded analytics platform. One of the key evaluation factors is potential economic value of Dataflow to their organization, particularly in the context of engaging other stakeholders is key for many of the leaders that we engage with. So we commissioned Forrester Consulting to conduct a comprehensive study on the impact that Dataflow had on their organization by interviewing actual customers . 

Today we’re excited to share our commissioned study conducted by Forrester Consulting, the Total Economic Impact™ of Google Cloud Dataflow, which allows data leaders to understand and quantify the benefits of Dataflow, and use cases it enables. Forrester conducted interviews with Dataflow customers to evaluate the benefits, costs, and risks of investing in Dataflow across an organization. Based on their interviews, Forrester identified major financial benefits across four different areas: business growth, infrastructure cost savings, data engineer productivity, and administration efficiency. In fact, Forrester found that customers adopting Dataflow can achieve a 55% boost in developer productivity and a 50% reduction in infrastructure costs. In fact, Forrester projects that customers adopting Dataflow can achieve a range of up to 171% Return on Investment (ROI) and a less than six months payback period. Customers can now use figures in the report to compute their own Return on Investment (ROI) and payback period.

“Dataflow is integral to accelerating time-to-market, decreasing time-to-production, reducing time to figure out how to use data for use cases, focusing time on value-add tasks, streamlining ingestion, and reducing total cost of ownership.” – Lead technical architect, CPG

Let’s take a deeper look at the ways that Forrester found that Dataflow can help you achieve your goals and unlock your business potential. 

Benefit #1: Increase data engineer productivity by 55%

Developers can choose among a variety of programming languages to define and execute data workflows. Dataflow also seamlessly integrates with other Google Cloud Platform and open source technologies to maximize value and applicability to a wide variety of use cases. Dataflow streamlined workflows with code reusability,dynamic templates, and the simplicity of a managed service. Engineers trusted pipelines to run correctly and adhere to governance. Data engineers avoided laborious issue-monitoring and remediation tasks that were common in the legacy environments such as poor performance, lack of availability, and failed jobs. Teams valued the language flexibility and open source base.

“Dataflow provided us with ETL replacement that opened limitless potential use cases and enabled us to do smarter data enhancement while data remains in motion.” — Director of data projects, financial services

Benefit #2: Reduce infrastructure costs by up-to 50% for batch and streaming workloads 

Dataflow’s serverless autoscaling and discrete control of job needs, scheduling, and regions eliminated overhead and optimized technology spending. Consolidating global data processing solutions to Dataflow further eliminated excess costs while ensuring performance, resilience, and governance across environments. Dataflow’s unified streaming and batch data platform gives organizations the flexibility to define either workload in the same programming model, run it on the same infrastructure, and manage it from a single operational management tool. 

“Our costs with our cloud data platform using Dataflow are just a fraction of the costs we faced before. Now we only pay for cloud infrastructure consumption because the open source base helps us avoid licensing costs. We spend about $120,000 per year with Dataflow, but we’d be spending millions with our old technologies.” – Lead technical architect, CPG

Benefit #3: Increase top-line revenue by improving customer experience and retention with payback time of < 6 months

Streaming analytics is an essential capability in today’s digital world to gain real-time actionable insights. Likewise, organizations must also have flexible, high- performance batch environments to analyze historical data for building machine learning models, business intelligence, and advanced analytics. Dataflow enabled real-time streaming use cases, improved data enrichment, encouraged data exploration,improved performance and resiliency, reduced errors, increased trust, and eliminated barriers to scale. As a result, organizations provided customers with more accurate, relevant, and in-the-moment data-backed services and insights — boosting customer experience, creating new revenue streams, and improving acquisition, retention, and enrichment.

“It’s already been proven that we are getting more business [with Dataflow] because we can turn around results faster for customers.” – VP of technology, financial services technology

“When we provide data to our customers and partners with Dataflow, we are much more confident in those numbers and can provide accurate data within a minute. Our customers and partners have taken note and commented on this. It’s reduced complaints and prevented churn.” – Senior software engineer, media

Other benefits 

Eliminated administrative overhead and toil

As a cloud-native managed service, all administration tasks such as provisioning, scaling, and updates are automatically handled by Google Cloud. Teams no longer need to manage servers and related software for legacy data processing solutions. Admins also streamlined processes for setting up data sources, adding pipelines, and enforcing governance.

Saved business operations costs for support teams and data end users

Dataflow improved the speed, quality, reliability, and ease of access to data for insights for general business users, saving time and empowering users to drive better data-backed outcomes. It also reduced support inquiry volume while automating manual job creation.

What’s next?

Download the Forrester Total Economic Impact study today to dive deep into the economic impact Dataflow can deliver your organization. We would love to partner with you to explore the potential Dataflow can unlock in your teams. Please reach out to our sales team to start a conversation about your data transformation with Google Cloud.