Forrester Research has named Google Cloud a Leader in The Forrester Wave™: AI Infrastructure, Q4 2021 report authored by Mike Gualtieri and Tracy Woo. In the report, Forrester evaluated dimensions of AI architecture, training, inference and management against a set of pre-defined criteria. Forrester’s analysis and recognition gives customers the confidence they need to make important platform choices that will have lasting business impact. 

Google received the highest possible score in 16 Forrester Wave evaluation criteria: architecture design, architecture components, training software, training data, training throughput, training latency, inferencing throughput, inferencing latency, management operations, management external, deployment efficiency, execution roadmap, innovation roadmap, partner ecosystem, commercial model, and number of customers.

We believe that Google’s vision to be a unified data and AI solution provider for the end-to-end data science experience is recognized by Forrester, through high scores in the areas of architecture and innovation. We are focused on building the most robust yet cohesive experience to enable our customers to leverage the best of Google every step of the way. Here are four key areas where Google excels, among the many highlighted in this report. 

AI Infrastructure: Leverage the building blocks of innovation

When an organization chooses to run its business on Google Cloud, it benefits from innovative infrastructure available globally. Google offers users a rich set of building blocks such as Deep Learning VMs and containers, the latest GPUs/TPUs and a marketplace of curated ISV offerings to help architect your own custom software stack on VMs and/or Google Kubernetes Engine (GKE). 

Google provides a range of GPU & TPU accelerators for various use cases, including high performance training, low cost inferencing and large-scale accelerated data processing. Google is the only public cloud provider to offer up to 16 NVIDIA A100 GPUs in a single VM, making it possible to train very large AI models on a single node. Users can start with one NVIDIA A100 GPU and scale to 16 GPUs without configuring multiple VMs for single-node ML training.  Google also provides TPU pods for large-scale AI research with PyTorch, TensorFlow, and JAX.  The new fourth generation TPU pods deliver exaflop-scale peak performance with leading results in recent MLPerf benchmarks which included a 480 billion parameter language model.   

Google Kubernetes Engine provides the most advanced Kubernetes services with unique capabilities like Autopilot, highly automated cluster version upgrades, and cluster backup/restore. GKE is a good choice for a scalable multi-node bespoke platform for training, inference and Kubeflow pipelines, given its support for 15,000 nodes per cluster, auto-provisioning,  auto-scaling and various machine types (e.g. CPU, GPU, TPU and on-demand, spot). ML workloads also benefit from GKE’s support for dynamic scheduling, orchestrated maintenance, high availability, job API, customizability, fault tolerance and ML frameworks.  When a company’s footprint grows to a fleet of GKE clusters, its data teams can leverage Anthos Config Management to enforce consistent configurations and security policy compliance. 

Comprehensive MLOps: Build models faster and more easily without skimping on governance 

Google’s fully managed Vertex AI platform provides services for ML lifecycle management, from data ingestion and preparation all the way up to model deployment, monitoring, and management. Vertex AI requires nearly 80% fewer lines of code to train a model versus competitive platforms1, enabling data scientists and ML engineers across all levels of expertise to implement Machine Learning Operations (MLOps) so they can efficiently build and manage ML projects throughout the entire development lifecycle. 

Vertex AI Workbench provides data scientists with a single environment for the entire data-to-ML workflow, enabling data scientists to build and train models 5x faster than traditional notebooks. This is enabled by integrations across data services (like Dataproc, BigQuery, Dataplex, and Looker), which significantly reduce context switching.  Users are also able to access NVIDIA GPUs, modify hardware on the fly, and set up idle shutdown to optimize infrastructure costs. 

Organizations can then build and deploy models built on any framework (including TensorFlow, PyTorch, Scikit learn or XGBoost) with Vertex AI, with built-in tooling to track a model’s performance.  Vertex Training also provides various approaches for developing large models including Reduction Server to optimize bandwidth and latency of multi-node distributed training on NVIDIA GPUs for synchronous data parallel algorithms.  Vertex AI Prediction is serverless, and performs automatic provisioning and deprovisioning of nodes behind the scenes to provide low latency online predictions. It also provides the capability to split traffic between multiple models behind an endpoint. Models trained in Vertex AI can also be exported to be deployed in private or other public clouds.Google’s strengths in its current offering are in architecture, training, data throughput, and latency. Its sweet spot is in its product offering, Vertex AI, which has core AI compute capabilities and MLOps services for end-to-end AI lifecycle management.The Forrester Wave:™ AI Infrastructure, Q4 2021

In addition to building models, it is important to deploy tools for governance, security, and auditability. These tools are crucial for compliance in regulated industries, and they help teams to protect data, understand why given models fail, and determine how models can be improved. 

For orchestration and auditability, Vertex Pipelines and  Vertex ML Metadata tracks the inputs and outputs of an ML pipeline and the lineage of artifacts. Once models are in production, Vertex AI Model Monitoring supports feature skew and drift detection, alerting data scientists. These capabilities speed up debugging and create the visibility required for regulatory compliance and good data hygiene in general.For explainability, Vertex Explainable AI helps teams understand their model’s outputs for classification and regression tasks. Vertex AI tells how much each feature in the data contributed to the predicted result. Data teams can then use this information to verify that the model is behaving as expected, recognize bias in the model, and get ideas for ways to improve the model and training data.

These services together aim to simplify MLOps for data scientists and ML engineers, so that businesses can accelerate time to value for ML initiatives.

Security: Protect data while keeping ML pipelines flowing  

The Google stack builds security through progressive layers that deliver defense in depth. To accomplish data protection, authentication, authorization and non-repudiation, we have measures such as boot-level signature and chain-of-trust validation. 

Ubiquitous data encryption delivers unified control over data at-rest, in-use, and in-transit, with keys that are held by customers themselves. 

We offer options to run in fully encrypted confidential environments utilizing managed Hadoop or Spark with Confidential Dataproc or Confidential VMs.  

Partner Ecosystem: Work with world-class AI specialists 

Google works with certified partners globally to help our customers design, implement and manage complex AI systems. We have a growing list of partners with Machine Learning specializations on Google who have demonstrated customer success across industries, including deep partnerships with the largest Global System Integrators. The Google Cloud  Marketplace also provides a list of technology partners who allow enterprises to deploy machine learning applications on Google’s AI infrastructure.

Our dedication to being your partner of choice for ML Needs  

Leading organizations like OTOY, Allen Institute for AI and DeepMind (an Alphabet subsidiary) choose Google for ML, and enterprises like Twitter, Wayfair and The Home Depot shared more about their partnership with Google in their recent sessions at Google Next 2021.

Establishing well-tuned and appropriately managed ML systems has historically been challenging, even for highly skilled data scientists with sophisticated systems. With the key pillars of Google’s investments above, organizations can build, deploy, and scale ML models faster, with pre-trained and custom tooling, within a unified AI platform. 

We look forward to continuing to innovate and to helping customers on their digital transformation journey. To download the full report, click here. Get started on Vertex AI, learn what’s upcoming with infrastructure for AI and ML at Google here, and talk with our sales team.

As data environments become more complex, companies are turning to streaming analytics solutions that analyze data as it’s ingested and deliver immediate, high-value insights into what is happening now. These insights enable decision makers to act in real time to take advantage of opportunities or respond to issues as they occur.

While understanding what is happening now has great business value, forward-thinking companies are taking things a step further, using real-time analytics integrated with artificial intelligence (AI) and business intelligence (BI) to answer the question, “what might happen in the future?” Arkansas-based Tyson Foods has embraced AI/BI analytics to enable predictive insights that unlock new opportunities and drive future growth.

Creating a digital twin for connected intelligence company wide

Before using AI/BI, Tyson’s analytics capabilities consisted of traditional BI solutions focused on KPIs and simplifying data so that humans could understand it. Tyson wanted to leverage its data to uncover ways to improve current processes and grow its business. But with BI alone, Tyson struggled to use data to run the simulations and scenarios essential to make educated decisions. To keep growing, it had to embrace the complexity of its data, building ways to analyze it and use it to inform decision making. 

Tyson’s on-premises analytics solutions limited its ability to be aggressive and make intelligent, timely, prescriptive decisions. The solution was to create a digital twin to scale optimizations within business processes, moving from local optimizations to system-wide connected optimizations. Doing so meant shifting entirely to cloud computing, with an initial focus on building the ingestion component of the digital twin platform.

Investing in a digital twin enabled Tyson to accelerate new capabilities like supply chain simulation “what-if” scenarios, prescriptive price elasticity recommendations, and improvement of customer intimacy. 

Solving the ingestion problem for faster time to insights

Before its migration to Google Cloud, analytics projects that Tyson suffered from uncertainty over how to obtain the data. This problem was prolific and caused project times to be extended for weeks or even months due to the need to write and support one-off data ingestion processes at the front end. This problem also prevented the IT team from delivering analytics solutions fast enough for the business to take full advantage of them. 

To solve this analytics problem, the team created Data Ingestion Compute Engine (DICE). DICE is a Google Cloud-hosted, open-source, cloud-native ingestion platform developed to provide configuration-based, no-ops, code-free ingestion from disparate enterprise data systems, both internal and external. It is centered on three high-level goals:

1. Accelerate the speed of delivery of IT analytics solutions
2. Enable growth of IT capabilities to produce meaningful insight
3. Reduce long-term total cost of ownership for ingestion solutions

Creating DICE ingestion platform with Google Cloud services

Teams use DICE to set up secure data ingestion jobs in minutes without having to manage complex connections or write, deploy, and support their own code. DICE enables unbound scale, highly parallel processing, DevSecOps, open source, and the implementation of Lambda Data Architecture.

A DICE job is the logical unit of work in the DICE platform, consisting of immutable and mutable configurations persisted as JSON documents stored in Firestore. The job exists as an instruction set for the DICE data engine, which is Apache Beam running Dataflow to instruct which data to pull, how to pull it, how often to pull it, how to process it, when it changes, and where to direct it.

Two of DICE’s primary layers include the metadata engine and the data engine. The metadata engine is responsible for the creation and management of DICE job configuration and orchestration. It is made up of many microservices that interact with multiple Google Cloud services, including the job configuration creation API, job build configuration helper API, and job execution scheduler API.

The data engine is responsible for the physical ingestion of data, the change detection processing of that data, and the delivery of that data to specified targets. The data engine is Java code that uses the Apache Beam unified programming model and runs in Dataflow. It is comprised of streaming, jobs, and Dataflow flex template batch jobs. Logically, the data engine is segmented across three layers: the inbound processing layer, the DICE file system layer, and the target processing layer, which takes the data from the DICE file system and moves it to targets.

Rolling DICE for thousands of ingestion jobs each day

DICE was first deployed to a production environment in November 2019, and just two years later, it has more than 3,000 data ingestion jobs from more than a hundred disparate data systems, both internal and external to Tyson Foods. Most of these jobs run multiple times a day. On a daily basis the DICE environment sees more than 25,000 Dataflow jobs running and an average of 3.25 terabytes of new data being ingested.

DICE supports ingestion from many different types of technologies, including BigQuery, SQL Server, SAP HANA, Postgres, Oracle, MySQL, Db2, various types of file systems, and FTP servers. Additionally, DICE supports target platform technologies for ingestion jobs that include multiple JDBC targets, multiple file system targets, and BigQuery and queue-based store and forward technologies. 

The platform continues to see linear growth of DICE jobs, all while keeping platform costs relatively flat. With increasing demand for the platform, Tyson’s IT team is constantly enhancing DICE to support new sources and targets.

This intelligent platform keeps adding new value and makes it simple for Tyson to take advantage of its data. This innovation is a necessity in this fast-changing world of digital business in which companies must transform a high volume of complex data into actionable insight.

Organizations across industries are looking to AI to turn troves of data into intelligence, powered by the latest advances in generative AI. Yet for many organizations, there is a barrier to adopting the latest models because they can be costly to train or serve. A new class of cloud GPUs is needed to lower the cost of entry for businesses that want to tap the power of AI. 

Today, we’re introducing G2, the newest addition to the Compute Engine GPU family in Google Cloud. G2 is the industry’s first cloud VM powered by the newly announced NVIDIA L4 Tensor Core GPU, and is purpose-built for large inference AI workloads like generative AI. G2 delivers cutting-edge performance-per-dollar for AI inference workloads that run on GPUs in the cloud. By switching from NVIDIA A10G GPUs to G2 instances with L4 GPUs, organizations can lower their production infrastructure costs up to 40%. We also found that customers switching from NVIDIA T4 GPUs to L4 GPUs can achieve 2x-4x better performance. As a universal GPU offering, G2 instances also help accelerate other workloads, offering significant performance improvements on HPC, graphics, and video transcoding. Currently in private preview, G2 VMs are both powerful and flexible, and scale easily from one up to eight GPUs. 

Currently organizations require end-to-end enterprise ready infrastructure that will future proof their AI and HPC initiatives for a new era. G2s will be ready to be deployed on Vertex AI, GKE, and GCE, giving customers the freedom to architect their own custom software stack to meet their performance requirements and budget. With optimized Vertex AI support for G2 VMs, AI users can tap the latest generative AI models and technologies. With an easy to use UI and automated workflows, customers can access, tune and serve modern models for video, text, images, and audio without the toil of manual optimizations. The combination of these services with the power of G2 will help customers harness the power of complex machine models for their business.

NVIDIA L4 GPUs with Ada Lovelace Architecture

G2 machine families enable machine learning customers to run their production infrastructure in the cloud for a variety of applications such as language models, image classification, object detection, automated speech recognition, and language translation. Built on the Ada Lovelace architecture with fourth-generation Tensor Cores, the NVIDIA L4 GPU provides up to 30 TFLOPS of performance for FP32, and 242 TFLOPs for FP16. Newly added FP8 support, on top of existing INT8, BFLOAT16 and TF32 capabilities, makes the L4 ideal for ML inference. 

With the latest third-generation RT Cores and DLSS 3.0 technology, G2 instances are also great for graphics-intensive workloads such as rendering and remote workstations when paired with NVIDIA RTX Virtual Workstation. NVIDIA L4 provides 3x video encoding and decoding performance, and adds new AV1 hardware-encoding capabilities. For example, G2 can enable gaming customers running game engines such as Unreal and Unity with modern graphics cards to run real-time applications. Likewise, media and entertainment customers that need GPU-enabled virtual workstations can use the L4 to create photo-realistic, high-resolution 3D content for movies, games, and AR/VR experiences using applications such as Autodesk Maya or 3D Studio Max.

What customers are saying

A handful of early customers have been testing G2 and have seen great results in real-world applications. Here are what some of them have to say about the benefits that G2 with NVIDIA L4 GPUs bring:

AppLovin

AppLovin enables developers and marketers to grow with market leading technologies. Businesses rely on AppLovin to solve their mission-critical functions with a powerful, full stack solution including user acquisition, retention, monetization and measurement.

“AppLovin serves billions of AI powered recommendations per day, so scalability and value are essential to our business,” said Omer Hasan, Vice President, Operations at AppLovin. “With Google Cloud’s G2 we’re seeing that NVIDIA L4 GPUs offer a significant increase in the scalability of our business, giving us the power to grow faster than ever before.”

WOMBO

WOMBO aims to unleash everyone’s creativity through the magic of AI, transforming the way content is created, consumed, and distributed. 

“WOMBO relies upon the latest AI technology for people to create immersive digital artwork from users’ prompts, letting them create high-quality, realistic art in any style with just an idea,” said Ben-Zion Benkhin, Co-Founder and CEO of WOMBO. “Google Cloud’s G2 instances powered by NVIDIA’s L4 GPUs will enable us to offer a better, more efficient image-generation experience for users seeking to create and share unique artwork.”

Descript

Descript’s AI-powered features and intuitive interface fuel YouTube and TikTok channels, top podcasts, and businesses using video for marketing, sales, and internal training and collaboration. Descript aims to make video a staple of every communicator’s toolkit, alongside docs and slides. 

“G2 with L4’s AI Video capabilities allow us to deploy new features augmented by natural-language processing and generative AI to create studio-quality media with excellent performance and energy efficiency” said Kundan Kumar, Head of Artificial Intelligence at Descript.

Workspot

Workspot believes that the software-as-a-service (SaaS) model is the most secure, accessible and cost-effective way to deliver an enterprise desktop and should be central to accelerating the digital transformation of the modern enterprise.

“The Workspot team looks forward to continuing to evolve our partnership with Google Cloud and NVIDIA. Our customers have been seeing incredible performance leveraging NVIDIA’s T4 GPUs. The new G2 instances with L4 GPUS through Workspot’s remote Cloud PC workstations provide 2x and higher frame rates at 1280×711 and higher resolutions” said Jimmy Chang, Chief Product Officer at Workspot.

Pricing and availability

G2 instances are currently in private preview in the following regions: us-central1, asia-southeast1 and europe-west4. Submit your request here to join the private preview, or to receive a notification as when the public preview begins. Support will be coming to Google Kubernetes Engine (GKE), Vertex AI, and other Google Cloud services as well. We’ll share G2 public availability and pricing information later in the year.