Smart Reply: How the AI-augmented Chat Helps Scale Google's Tech Support Operations - Build What's Next
Blog

Smart Reply: How the AI-augmented Chat Helps Scale Google’s Tech Support Operations

2892

Of your peers have already read this article.

3:00 Minutes

The most insightful time you'll spend today!

Smart Reply, Google's product based on machine learning, natural language understanding, and conversation modeling to respond to IT queries at scale and in real-time. Read the blog to learn how AI powers Google Techstop to resolves Googlers' queries!

As Googlers transitioned to working from home during the pandemic, more and more turned to chat-based support to help them fix technical problems. Google’s IT support team looked at many options to help us meet the increased demand for tech support quickly and efficiently. 

More staff? Not easy during a pandemic. 

Let service levels drop? Definitely not. 

Outsource? Not possible with our IT requirements. 

Automation? Maybe, just maybe…

How could we use AI to scale up our support operations, making our team more efficient?

The answer: Smart Reply, a technology developed by a Google Research team with expertise in machine learning, natural language understanding, and conversation modeling. This product provided us with an opportunity to improve our agents’ ability to respond to queries from Googlers by using our corpus of chat data.  Smart Reply trains a model that provides suggestions to techs in real time. This reduces the cognitive load when multi-chatting and helps a tech drive sessions towards resolution.

chat sessions

In the solution detailed below, our hope is that IT teams in a similar situation can find best practices and a few shortcuts to implementing the same kind of time saving solutions. Let’s get into it!

Challenges in preparing our data

Our tech support service for Google employees—Techstop—provides a complex service, offering support for a range of products and technology stacks through chat, email, and other channels. 

Techstop has a lot of data. We receive hundreds of thousands of requests for help per year. As Google has evolved we’ve used a single database for all internal support data, storing it as text, rather than as protocol buffers. Not so good for model training. To protect user privacy, we want to ensure no PII (personal identifiable information – e.g. usernames, real names, addresses, or phone numbers) makes it into the model.

To address these challenges we built a FlumeJava pipeline that takes our text and splits each message sent by agent and requester into individual lines, stored as repeated fields in a protocol buffer. As our pipe is executing this task, it also sends text to the Google Cloud DLP API, removing personal information from the session text, replacing it with a redaction that we can later use on our frontend. 

With the data prepared in the correct format, we are able to begin our model training. The model provides next message suggestions for techs based on the overall context of the conversation. To train the model we implemented tokenization, encoding, and dialogue attributes.

Splitting it up

The messages between the agent and customer are tokenized: broken up into discrete chunks for easier use. This splitting of text into tokens must be carefully considered for several reasons:

  • Tokenization determines the size of the vocabulary needed to cover the text.
  • Tokens should attempt to split along logical boundaries, aiming to extract the meaning of the text.
  • Tradeoffs can be made between the size of each token, with smaller tokens increasing processing requirements but enabling easier correlation between different spans of text.

There are many ways to tokenize text (SAFT, splitting on white spaces, etc.), here we chose sentence piece tokenization, with each token referring to a word segment. 

Prediction with encoders

Training the neural network with tokenized values has gone through several iterations. The team used an Encoder-Decoder architecture that took a given vector along with a token and used a softmax function to predict the probability that the token was likely to be the next token in the sentence/conversation. Below, a diagram represents this method using LSTM-based recurrent networks. The power of this type of encoding comes from the ability of the encoder to effectively predict not just the next token, but the next series of tokens.

encoder decoder

This has proven very useful for Smart Reply. In order to find the optimal sequence, an exponential search over each tree of possible future tokens is required. For this we opted to use beam search over a fixed-size list of best candidates, aiming to avoid increasing the overall memory use and run time for returning a list of suggestions. To do this we arranged tokens in a trie, and used a number of post processing techniques, as well as calculating a heuristic max score for a given candidate, to reduce the time it takes to iterate through the entire token list. While this improves the run time, the model tends to prefer shorter sequences. 

In order to help reduce latency and improve control we decided to move to an Encoder-Encoder architecture. Instead of predicting a single next token and decoding a sequence of following predictions with multiple calls to the model, it instead encodes a candidate sequence with the neural network.

encoder

In practice, the two vectors – the context encoding and the encoding of a single candidate output – are combined with dot product to arrive at a score for the given candidate. The goal of this network is to maximize the score for true candidates – e.g. candidates that did appear in the training set – and minimize false candidates.

Choosing how to sample negatives affects the model training greatly. Below are some strategies that can be employed:

  • Using positive labels from other training examples in the batch.
  • Drawing randomly from a set of common messages. This assumes that the empirical probability of each message is sampled correctly. 
  • Using messages from context.
  • Generating negatives from another model.

As this encoding generates a fixed list of candidates that can be precomputed and stored, each time a prediction is needed, only the context encoding needs to be computed, then multiplied by the matrix of candidate embeddings. This reduces both the time from the beam search method and the inherent bias towards shorter responses.

Dialogue Attributes

Conversations are more than simple text modeling. The overall flow of the conversation between participants provides important information, changing the attributes of each message. The context, such as who said what to whom and when, offers useful bits of input for the model when making a prediction. To that end the model uses the following attributes during its prediction:

  • Local User ID’s – we set a finite number of participants for a given conversation to represent the turn taking between messages, assigning values to those participants. In most cases for support sessions there are 2 participants, requiring ID 0, and 1.
  • Replies vs continuations – initially modeling focused only on replies. However, in practice conversations also include instances where participants are following up on the previously sent message. Given this, the model is trained for both same-user suggestions and “other” user suggestions.
  • Timestamps  – gaps in conversation can indicate a number of different things. From a support perspective, gaps may indicate that the user has disconnected. The model takes this information and focuses on the time elapsed between messages, providing different predictions based on the values. 

Post processing

Suggestions can then be manipulated to get a more desirable final ranking. Such post-processing includes:

  1. Preferring longer suggestions by adding a token factor, generated by multiplying the number of tokens in the current candidate.
  2. Demoting suggestions with a high level of overlap with previously sent messages.
  3. Promoting more diverse suggestions based on embedding distance similarities.

To help us tune and focus on the best responses the team created a priority list. This gives us the opportunity to influence the model’s output, ensuring that responses that are incorrect can be de-prioritized. Abstractly it can be thought of as a filter that can be calibrated to best suit the client’s needs. 

Getting suggestions to agents

With our model ready we now needed to get it in the hands of our techs. We wanted our solution to be as agnostic to our chat platform as possible, allowing us to be agile when facing tooling changes and speeding up our ability to deploy other efficiency features. To this end we wanted an API that we could query either via gRPC or via HTTPs. We designed a Google Cloud API, responsible for logging usage as well as acting as a bridge between our model and a Chrome Extension we would be using as a frontend.

The hidden step, measurement

Once we had our model, infrastructure, and extension in place we were left with the big question for any IT project. What was our impact? One of the great things about working in IT at Google is that it’s never dull. We have constant changes, be it planned or unplanned. However, this does complicate measuring the success of a deployment like this. Did we improve our service or was it just a quiet month?

In order to be satisfied with our results we conducted an A/B experiment, with some of our techs using our extension, and the others not. The groups were chosen at random with a distribution of techs across our global team, including a mix of techs with varying levels of experience ranging from 3 to 26 months. 

Our primary goal was to measure tech support efficiency when using the tool. We looked at two key metrics as proxies for tech efficiency: 

  1. The overall length of the chat. 
  2. The number of messages sent by the tech.

Evaluating our experiment

To evaluate our data we used a two-sample permutation test. We had a null hypothesis that techs using the extension would not have a lower time-to-resolution, or be able to send more messages, than those without the extension. The alternative hypothesis was that techs using the extension would be able to resolve sessions quicker or send more messages in approximately the same time.

We took the mid mean of our data, using pandas to trim outliers greater than 3 standard deviations away. As the distribution of our chat lengths is not normal, with significant right skew caused by a long tail of longer issues, we opted to measure the difference in means, relying on central limit theorem (CLT) to provide us with our significance values. Any result with a p-value between 1.0 and 9.0 would be rejected. 

Across the entire pool we saw a decrease in chat lengths of 36 seconds.

distribution 1

In reference to the number of chat messages we saw techs on average being able to send 5-6 messages more in less time. 

distribution 2

In short, we saw techs were able to send more messages in a shorter period of time. Our results also showed that these improvements increased with support agent tenure, and our more senior techs were able to save an average of ~4 minutes per support interaction.

distribution 3

Overall we were pleased with the results. While things weren’t perfect, it looked like we were onto a good thing.

So what’s next for us?

Like any ML project, the better the data the better the result. We’ll be spending time looking into how to provide canonical suggestions to our support agents by clustering results coming from our allow list. We also want to investigate ways of making improvements to the support articles provided by the model, as anything that helps our techs, particularly the junior ones, with discoverability will be a huge win for us.

How can you do this?

A successful applied AI project always starts with data. Begin by gathering the information you have, segmenting it up, and then starting to process it. The interaction data you feed in will determine the quality of the suggestions you get, so make sure you select for the patterns you want to reinforce.

Our Contact Center AI allows tokenization, encoding and reporting, without you needing to design or train your own model, or create your own measurements. It handles all the training for you, once your data is formatted properly. 

You’ll still need to determine how best to integrate its suggestions to your support system’s front-end. We also recommend doing statistical modeling to find out if the suggestions are making your support experience better. 

As we gave our technicians ready-made replies to chat interactions, we saved time for our support team. We hope you’ll try using these methods to help your support team scale.

Blog

Three Data Insights That Set Marketing Leaders Apart from Marketing Laggards

3322

Of your peers have already read this article.

1:15 Minutes

The most insightful time you'll spend today!

Google partnered with MIT Sloan Management Review and MIT Technology Review for a deep dive into the mindsets of marketing leaders who use machine learning and AI to get better results from their marketing activities, compared to marketing executives who don't.

With insights directing the bulk of today’s marketing decisions, leading marketers are driving growth by embracing three core mindset shifts. Marketing leaders are working toward a holistic view of consumers; they are investing in machine learning to support their activities; and they believe how they apply their data is crucial to success. Google partnered with MIT Sloan Management Review and MIT Technology Review for a deeper dive into the mindsets around these beliefs. Here’s what we found.

1. Leading marketers are working toward a holistic view of consumers.

  • 63% of leading marketers agree they are using KPIs to develop a single integrated view of the customer.
  • 66% of leading marketers agree they should build teams for end-to-end customer experiences and journeys, across channels and devices.
  • Marketing leaders are 60% more likely than laggards to believe that marketing teams should own a data-driven customer strategy that supports all organizational stakeholders.

2.  Leading marketers are investing in machine learning to support their activities.

  • Measurement-leaders are more than 2X as likely as their measurement-challenged counterparts to agree that their organization is already investing in automation and machine learning technologies to drive marketing activities.
  • 75% of marketers who use machine learning to drive marketing activities said they were satisfied with how their KPIs inform and influence decision-making across their enterprise.
  • 73% of marketing leaders who have invested in machine learning have shifted more than 10% of their time from manual activation to strategic insight generation.

3. Leading marketers believe how they apply their data is crucial to success.

  • 66% of marketing leaders believe how companies apply their data will play a key role in their ability to thrive.
  • 60% of leading marketers believe data-driven attribution is essential to understanding journeys of high-value customers.
  • Marketing leaders are 53% more likely than laggards to say machine learning processes data signals to help marketers better understand consumer intent.
Blog

Google Dataflow Named Leader in The 2021 Forrester Wave™: Streaming Analytics

6625

Of your peers have already read this article.

2:00 Minutes

The most insightful time you'll spend today!

Google Dataflow shines with the highest scores in Forrester's 12 important criteria and bags the title of a Leader in the The Forrester Wave™, Streaming Analytics. Read further to understand Dataflow's strength in harnessing real-time data.

We are excited to announce that Google has been named a Leader in The Forrester Wave™: Streaming Analytics, Q2 2021 report. Thank you to our strong community of customers and partners for working with us to deliver a customer focused product. We believe Forrester’s recognition is an acknowledgement of our leadership across an integrated set of capabilities that rely on data to drive transformation. We were also honored to be named a leader in The Forrester Wave™: Cloud Data Warehouse, Q1 2021

Forrester gave Dataflow a score of 5 out of 5 across 12 different criteria and according to the report: “Google Cloud Dataflow has strengths in data sequencing, advanced analytics, performance, and high-availability. Google Dataflow’s sweet spot is for enterprises that have a preponderance of real-time data generated on Google Cloud Platform or wish to simplify all data processing by using a single platform that unifies both streaming and batch jobs.” 

Harnessing the power of real-time data

The speed with which businesses are able to respond to change is the difference between those that successfully navigate the future and those that get left behind. In order to accelerate their digital transformation, reimagine their business and leverage the power of real-time data,  today’s data leaders require a streaming analytics platform that provides both depth and breadth.  

Cloud Pub/Sub and Cloud Dataflow, based on more than a decade of experience in internet scale systems for Google’s own needs, provide customers with a reliable, scalable, performant platform. In addition, we’ve designed these products for ease of use to make streaming analytics accessible to more users, which is why customers such as Sky and others from across all industries use Dataflow to run streaming analytics workloads.

5 out of 5 across key streaming analytics criteria

While Forrester gave Dataflow a score of 5 out of 5 in 12 criteria, the product achieved the highest possible scores in areas that are top of mind for our customers.

streaming analytics criteria.jpg

We continue to be focused on solving problems that matter to you. For example, just in the last month we announced Dataflow Prime and  Auto Sharding for BigQuery – two new auto tuning capabilities that bring efficiency and simplicity to your streaming pipelines. 

Dataflow achieves highest score possible in strategy 

With Google, organizations gain an industry leading product and a partner that has the vision and strategy to help you tackle new business challenges and provide delightful experiences to your customers.

highest score in strategy.jpg

In summary, we are honored to be a Leader in The Forrester Wave™, Streaming Analytics, and look forward to continuing to innovate and partner with you on your digital transformation journey. 

Download the full report: The Forrester Wave™: Streaming Analytics, Q2 2021 and check out these smart analytics reference patterns. To learn more about Dataflow, visit our website and get to know the product by taking an interactive tutorial. You can also watch recordings from the Data Cloud Summit event (May 2021), where we provided an in-depth view of new product innovations in Dataflow and other data analytics products.

Case Study

US County, the Size of Mangalore, Uses AI to Offer Voice-Enabled Virtual Agent

3056

Of your peers have already read this article.

5:30 Minutes

The most insightful time you'll spend today!

California’s Placer County utilizes AI technology via smart speakers, smartphones, tablets, PCs, and webpages to help citizens get the information they need. Citizens can ask questions like: "How to adopt a pet?" or “What historical engineering work has been done on my property?”

From the historic Gold Country to the rugged heights of the Sierra Nevada, Placer County encompasses more than 1,500 square miles—and provides services to nearly 400,000 residents. Across the county, residents access resources in person, over the phone, and through the county website. In 2018, the county piloted a suite of eServices through its Community Development Resource Agency (CDRA) with the goal of helping its geographically dispersed population more easily apply for permits, make appointments, and get immediate answers to specific questions.

Now, with the help of Google Cloud Dialogflow and Speech-to-Text API, the county has created a virtual agent that enables anyone, anywhere, to simply start a conversation by saying “Ask Placer County” into a variety of voice assistant hardware. This virtual agent builds on the success of eServices and helps the public access information through their smartphones or smart home devices, which is especially helpful for individuals without immediate access to a computer—such as the traveling public or a contractor working on-site, for example.

The pilot program has been an opportunity to explore new technology and create another communication channel to our citizens. “Our ambitious and long-term goal is that ‘Ask Placer County’ will be like having a personal assistant to everything related to Placer County. While the virtual agent is currently limited to a few departments, we plan to expand it countywide,” says Ben Palacio, senior IT analyst for Placer County.

“Our ambitious and long-term goal is that ‘Ask Placer County’ will be like having a personal assistant to everything related to Placer County. While the virtual agent is currently limited to a few departments, we plan to expand it countywide.”

Ben Palacio, Senior IT Analyst, Placer County

New eServices get a boost from AI virtual agent

The CDRA, one of many departments and agencies within Placer County, deployed interactive eServices for residents. As part of this initiative, the CDRA made a special request to the information technology (IT) department: Build an AI-based virtual agent that would spotlight the eServices on the website and make them easily available through a conversational interface.

“Having a specific technology requirement voiced by an agency was visionary and very welcome. It meant they were engaged and excited about the possibilities that these new technologies had to offer,” says Mike Spak, IT Manager for Placer County.

Today, the CDRA’s eServices help residents answer important questions, such as: “Is my land zoned for adding a garage?” “What historical engineering work has been done on my property?” And, “How can I make an appointment at a permitting office?”

“Having a specific technology requirement voiced by an agency was visionary and very welcome. It meant they were engaged and excited about the possibilities that these new technologies had to offer.”

Mike Spak, IT Manager, Placer County

Google Cloud fit easily into Placer County’s multivendor environment

With the help of consultants from Dito, a Google Cloud Premier Partner, Placer County chose Google Cloud to help with “Ask Placer County.”

Placer County runs a multivendor IT environment. “Especially with the cloud, the easier it is to integrate a vendor’s capabilities with others, the better for everybody,” says Palacio. “Google Cloud was able to integrate with other vendors’ capabilities for this project.”

For example, Placer County uses the Google Cloud operations suite (formerly Stackdriver) to monitor, troubleshoot, and improve its cloud infrastructure and application performance. And it uses Dialogflow, which powers the natural language processing (NLP) interface of the “Ask Placer County” virtual agent. Both integrate easily with tools from other vendors.

“Especially with the cloud, the easier it is to integrate a vendor’s capabilities with others, the better for everybody. Google Cloud was able to integrate with other vendors’ capabilities for this project.”

Ben Palacio, Senior IT Analyst, Placer County

Real-time conversations powered by “Ask Placer County”

Dito got “Ask Placer County” up and running using App Engine and Datastore. Today, the “Ask Placer County” virtual agent provides information ranging from how to adopt a pet to short-term-rental compliance.

Users can say “Ask Placer County” into their smartphones or their computer microphones and access targeted information from the pilot departments. IT staff reviews the asked questions and are constantly updating the virtual agent to provide more accurate and responsive information.

“You could be in your backyard with a contractor trying to get a project started, and if the contractor has a question about permitting or zoning, they can use their smartphone to get an answer right then and there, rather than having to interrupt the meeting, get back in their car, and drive to a county office,” says Palacio.

The county hopes “Ask Placer County” will improve the efficiency of in-person interactions between the public and county employees as well. Because the public can access common questions and information online or through the virtual agent, employees’ interactions with the public can be more focused and productive.

“We are providing more effective and efficient service to our customers through 24/7 access to information and by reducing the proportion of staff’s time in responding to emails and voicemails to answer our customer questions,” says Shawna Purvines, Principal Planner for Placer County.

According to Placer County, the virtual agent currently answers a monthly average of around 200 questions for the CDRA, and the county continues to develop more and more complete answers across participating departments.

“We are providing more effective and efficient service to our customers through 24/7 access to information and by reducing the proportion of staff’s time in responding to emails and voicemails to answer our customer questions.”

Shawna Purvines, Principal Planner, Placer County

Constantly evolving based on users’ needs

To best serve the needs of the CDRA, and potentially other departments over time, “Ask Placer County” continues to evolve. The county can add and retire questions based on the needs of constituents. This technology can be tailored to address current needs, such as responding to COVID-19. “Our hope is that, in quickly evolving situations, the virtual agent can be a resource for the public to access real-time information about public services,” says Palacio.

As a component of the eServices group, “Ask Placer County” has kept construction and development in Placer County operational during the COVID-19 pandemic, and it continues to provide opportunities for the public to access county resources without coming in to the counters, reducing the need for in-person visits. In fact, according to Placer County data, permit applications have increased by 17%.

Placer County also performs analytics using the Google Cloud operations suite to gain insights into the most popular questions and, just as importantly, the questions that are not being answered. “We get insight into what we’re missing—for those times when the virtual agent can’t find a response in our database,” says Palacio. Placer County uses these insights to improve answers and ultimately improve the efficacy of the virtual agent over time.

“This continual, real-time learning is the exciting part of the application that lets us help constituents in entirely new ways. It’s really cool,” says Palacio.

County IT workers actively analyze CDRA webpages to determine which are the most visited and to craft questions and answers to add to “Ask Placer County.” According to Placer County, the number of questions the virtual agent can answer is now more than 600. But the CDRA pages are just a small subset of the 5,000 pages that make up the Placer County website. A future goal is to provide a backstop for customer questions that can’t be answered, transferring those customers to county staff for resolution.

“Looking ahead, this technology has the ability to provide answers to thousands and thousands of questions,” says Palacio.

“This continual, real-time learning is the exciting part of the application that lets us help constituents in entirely new ways. It’s really cool.”

Ben Palacio, Senior IT Analyst, Placer County

Scaling “Ask Placer County” countywide

The limited rollout of the virtual agent technology allowed IT to monitor its effectiveness and make initial adjustments. As the virtual agent evolves, the Placer County chief information officer sees opportunities for it to engage the public countywide.

The “Ask Placer County” virtual agent can truly be a concierge, helping the public navigate county resources. It can help citizens become more engaged with their elected officials, it can provide information, and it can more efficiently connect the public with the services they seek.

Some of the future possibilities include providing current information on board meetings and individual elected officials, checking on permit status and burn days, and getting the latest county news. “There are so many ways we can use this solution to tackle issues within the county that we’re going to have to somehow prioritize them,” says Palacio.

6387

Of your peers have already watched this video.

2:00 Minutes

The most insightful time you'll spend today!

Blog

CCAI Insights: Answer Customers’ Queries & Understand Them Better with Conversation Data

With CCAI Insights, businesses can drive contact center efficiency, solve customer problems and leverage data from customer interactions to understand them better!

CCAI Insights, a core piece of the Google Cloud’s Contact Center AI product suite is built to help contact center management dive into data to adjust business needs, preempt problems with timely analysis of customer conversations and keep agents prepared. Additionally, businesses can automatically feed data into Insights from other areas of CCAI like Dialogflow CX or another product sources. Watch the video to find out more benefits and capabilities of CCAI Insights in elevating CX.

Blog

Revolutionizing Cloud Computing: Introducing G2 VMs with NVIDIA L4 GPUs

1542

Of your peers have already read this article.

2:30 Minutes

The most insightful time you'll spend today!

Experience unmatched cloud computing performance with G2 VMs and NVIDIA L4 GPUs, a breakthrough in cloud technology. Discover how this industry-first innovation can revolutionize the way you work in the cloud. Read more!

Organizations across industries are looking to AI to turn troves of data into intelligence, powered by the latest advances in generative AI. Yet for many organizations, there is a barrier to adopting the latest models because they can be costly to train or serve. A new class of cloud GPUs is needed to lower the cost of entry for businesses that want to tap the power of AI. 

Today, we’re introducing G2, the newest addition to the Compute Engine GPU family in Google Cloud. G2 is the industry’s first cloud VM powered by the newly announced NVIDIA L4 Tensor Core GPU, and is purpose-built for large inference AI workloads like generative AI. G2 delivers cutting-edge performance-per-dollar for AI inference workloads that run on GPUs in the cloud. By switching from NVIDIA A10G GPUs to G2 instances with L4 GPUs, organizations can lower their production infrastructure costs up to 40%. We also found that customers switching from NVIDIA T4 GPUs to L4 GPUs can achieve 2x-4x better performance. As a universal GPU offering, G2 instances also help accelerate other workloads, offering significant performance improvements on HPC, graphics, and video transcoding. Currently in private preview, G2 VMs are both powerful and flexible, and scale easily from one up to eight GPUs. 

Currently organizations require end-to-end enterprise ready infrastructure that will future proof their AI and HPC initiatives for a new era. G2s will be ready to be deployed on Vertex AI, GKE, and GCE, giving customers the freedom to architect their own custom software stack to meet their performance requirements and budget. With optimized Vertex AI support for G2 VMs, AI users can tap the latest generative AI models and technologies. With an easy to use UI and automated workflows, customers can access, tune and serve modern models for video, text, images, and audio without the toil of manual optimizations. The combination of these services with the power of G2 will help customers harness the power of complex machine models for their business.

NVIDIA L4 GPUs with Ada Lovelace Architecture

G2 machine families enable machine learning customers to run their production infrastructure in the cloud for a variety of applications such as language models, image classification, object detection, automated speech recognition, and language translation. Built on the Ada Lovelace architecture with fourth-generation Tensor Cores, the NVIDIA L4 GPU provides up to 30 TFLOPS of performance for FP32, and 242 TFLOPs for FP16. Newly added FP8 support, on top of existing INT8, BFLOAT16 and TF32 capabilities, makes the L4 ideal for ML inference. 

With the latest third-generation RT Cores and DLSS 3.0 technology, G2 instances are also great for graphics-intensive workloads such as rendering and remote workstations when paired with NVIDIA RTX Virtual Workstation. NVIDIA L4 provides 3x video encoding and decoding performance, and adds new AV1 hardware-encoding capabilities. For example, G2 can enable gaming customers running game engines such as Unreal and Unity with modern graphics cards to run real-time applications. Likewise, media and entertainment customers that need GPU-enabled virtual workstations can use the L4 to create photo-realistic, high-resolution 3D content for movies, games, and AR/VR experiences using applications such as Autodesk Maya or 3D Studio Max.

What customers are saying

A handful of early customers have been testing G2 and have seen great results in real-world applications. Here are what some of them have to say about the benefits that G2 with NVIDIA L4 GPUs bring:

AppLovin

AppLovin enables developers and marketers to grow with market leading technologies. Businesses rely on AppLovin to solve their mission-critical functions with a powerful, full stack solution including user acquisition, retention, monetization and measurement.

“AppLovin serves billions of AI powered recommendations per day, so scalability and value are essential to our business,” said Omer Hasan, Vice President, Operations at AppLovin. “With Google Cloud’s G2 we’re seeing that NVIDIA L4 GPUs offer a significant increase in the scalability of our business, giving us the power to grow faster than ever before.”

WOMBO

WOMBO aims to unleash everyone’s creativity through the magic of AI, transforming the way content is created, consumed, and distributed. 

“WOMBO relies upon the latest AI technology for people to create immersive digital artwork from users’ prompts, letting them create high-quality, realistic art in any style with just an idea,” said Ben-Zion Benkhin, Co-Founder and CEO of WOMBO. “Google Cloud’s G2 instances powered by NVIDIA’s L4 GPUs will enable us to offer a better, more efficient image-generation experience for users seeking to create and share unique artwork.”

Descript

Descript’s AI-powered features and intuitive interface fuel YouTube and TikTok channels, top podcasts, and businesses using video for marketing, sales, and internal training and collaboration. Descript aims to make video a staple of every communicator’s toolkit, alongside docs and slides. 

“G2 with L4’s AI Video capabilities allow us to deploy new features augmented by natural-language processing and generative AI to create studio-quality media with excellent performance and energy efficiency” said Kundan Kumar, Head of Artificial Intelligence at Descript.

Workspot

Workspot believes that the software-as-a-service (SaaS) model is the most secure, accessible and cost-effective way to deliver an enterprise desktop and should be central to accelerating the digital transformation of the modern enterprise.

“The Workspot team looks forward to continuing to evolve our partnership with Google Cloud and NVIDIA. Our customers have been seeing incredible performance leveraging NVIDIA’s T4 GPUs. The new G2 instances with L4 GPUS through Workspot’s remote Cloud PC workstations provide 2x and higher frame rates at 1280×711 and higher resolutions” said Jimmy Chang, Chief Product Officer at Workspot.

Pricing and availability

G2 instances are currently in private preview in the following regions: us-central1, asia-southeast1 and europe-west4. Submit your request here to join the private preview, or to receive a notification as when the public preview begins. Support will be coming to Google Kubernetes Engine (GKE), Vertex AI, and other Google Cloud services as well. We’ll share G2 public availability and pricing information later in the year.

More Relevant Stories for Your Company

Whitepaper

Data Science Teams: A Practitioners Guide to MLOps

Across industries, DevOps and DataOps have been widely adopted as methodologies to improve quality and reduce the time to market of software engineering and data engineering initiatives. With the rapid growth in machine learning (ML) systems, similar approaches need to be developed in the context of ML engineering, which handle

Case Study

Sainsbury’s Uses AI to Figure Out How the World Eats

Retail will forever be an industry that must constantly reinvent itself in response to, and anticipation of, ever-changing consumer demands. Digital transformation is fueling these changes and we've previously spoken about how businesses including Ulta Beauty and Kohl’s are taking advantage of Google Cloud to put data at the center of what they do

Case Study

Insurer Uses Google Cloud AI to Battle Slow Growth: It Improves Sales by 5% in 8 Weeks

For a business to succeed in the long term, it needs to learn not just to adapt to inevitable change, but to harness it. South Africa-based PPS has been an insurance company since 1941 and today is the biggest mutual insurance provider in the country. As a mutual company, PPS is owned

Case Study

Apollo24|7 partnered with Google Cloud to build the Clinical Decision Support System (CDSS) together

Clinical Decision Support System (CDSS) is an important technology for the healthcare industry that analyzes data to help healthcare professionals make decisions related to patient care. The market size for the global clinical decision support system appears poised for expansion, with one study predicting a compound annual growth rate (CAGR)

SHOW MORE STORIES