Conversational AI drives better customer experiences

3362
Of your peers have already read this article.
3:30 Minutes
The most insightful time you'll spend today!
Conversational AI is opening up a new world of possibilities in areas like customer experience, user engagement, and access to content.
In Cloud AI, we’ve taken Google’s groundbreaking machine learning models in speech and natural language processing and applied them to the contact center space, radically improving the customer experience while also driving down operational costs. It’s shifting the focus of contact centers from the backroom, out of sight (and mind) to the boardroom and the strategic heart of the business.
Our solution, called Contact Center AI (CCAI), is an accelerator of digital transformation as organizations all over the world figure out how to support their customers during these challenging times. One customer that’s redefined the possibilities of AI-powered conversation using CCAI, is Verizon.
Whether through voice calls or chat, Verizon customers will no longer need to go through menu prompts or option trees; they simply say or type their question, and CCAI’s natural-language recognition feature finds the best way to assist them. No stilted speech or robot-like commands.
“These customer service enhancements, powered by the Verizon collaboration with Google Cloud, offer a faster and more personalized digital experience for our customers while empowering our customer support agents to provide a higher level of service,” said Shankar Arumugavelu, SVP & CIO at Verizon.
CCAI is also driving cost savings without cutting corners on customer service. In the past, to improve customer satisfaction (CSAT), you had to hire more agents, increasing operating costs. Conversely, if you reduced the number of agents, your CSAT scores went down. Contact Center AI (CCAI) lets you have both.
CCAI does this by expediting customer requests using virtual agents, assisting live agents, and providing insights on customer interactions to drive improvements back into the system. The result is a better experience for your customers—they get answers faster—and you get lower operating costs. It allows human agents to focus on higher value tasks in their customer interactions.
New products and features add richer customer experience
We are continuing to invest in CCAI and today we are excited to announce the following new capabilities:
Dialogflow CX is the latest version of Dialogflow, a development suite used by over a million developers for building natural, rich conversational experiences into mobile and web applications, smart devices, bots, interactive voice response systems, popular messaging platforms and more. This new version of Dialogflow is optimized for large contact centers that deal with complex (multi-turn) conversations and it is truly omnichannel – you build it once and deploy it everywhere – in your contact centers and digital channels. Dialogflow CX features a new visual builder to create, build and manage virtual agents. It’s available now, in beta.
Lukasz Rewerenda
Dialogflow CX brings conversation state management to a whole new level
Principal Solutions Architect, Randstad (Netherlands)
Agent Assist for Chat is a new module for Agent Assist that provides agents with continuous support over “chat” in addition to voice calls, by identifying intent and providing real-time, step-by-step assistance. Agent Assist enables agents to be more agile and efficient and spend more time on difficult conversations, giving both the customer and the agent a better experience. It transcribes calls in real time, identifies customer intent, provides real-time, step by step assistance (recommended articles, workflows, etc.), and automates call dispositions.
For customers in regulated industries, Agent Assist can remove the risk of agents providing inaccurate information (which can happen due to high agent turnover and limited training). Agent Assist can also surface the latest discount information, deals and special offers, which can be hard for agents to keep track of as this information changes frequently.
Custom Voice, available in beta, is a new capability for CCAI and our Text-to-Speech API that lets you create a unique voice to represent your brand across all your customer touchpoints, instead of using a common voice shared with other organizations. By taking advantage of the custom Text-to-Speech model created with Custom Voice, you can define and choose the voice profile that suits your business and adjust to changes without scheduling studio time with voice actors to record new phrases.
At Google Cloud, we are committed to ensuring that our products and features are in alignment with our AI Principles. If you are interested in Custom Voice, there is a review process to ensure that your use case is aligned with our AI principles. Learn more about Custom Voice and how to sign up here.
Bring Google’s groundbreaking speech models on-prem
For many customers, an all-in approach to public cloud is not an option, which is why we’re extending our AI capabilities to run on-prem. Last week we announced Speech-to-Text On-Prem, the first of our hybrid AI offerings, now generally available. Speech-to-Text On-Prem gives you full control over speech data and since it runs in your own data center, it’s easy to comply with your data residency and compliance requirements. At the same time, Speech-to-Text On-Prem uses state-of-the-art speech models from Google researchers that are more accurate, smaller, and require less computing resources to run than existing solutions.
Take a look at how businesses saved costs and improved customer experience with CCAI in the new commissioned study conducted by Forrester Consulting: “New Technology: The Projected Total Economic Impact™ Of Google Cloud Contact Center AI.” To learn more about CCAI and conversational AI, see how our customers are using the solution and check out these sessions at Next OnAir:
The Future of Retail: Automated Customer Journeys Powered by Technology

3672
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
Editor’s note: To kick off the new year and in preparation for NRF The Big Show, we invited partners from across our retail ecosystem to share stories, best practices, and tips and tricks on how they are helping retailers transform during a time that continues to see tremendous change. Please enjoy this entry from our partner.
If you put a pot of water on the stove, it doesn’t heat up instantly. It simmers slowly at first, eventually picking up steam to reach a rolling boil. The retail landscape is not so different. Capgemini’s research shows the sector has evolved over four generations, from the early days of fragmented outlets to omnichannel and customer-centric focuses, with a fifth generation on the horizon that promises to be centered on consumption.
- Generation 1: Fragmented outlets
- Generation 2: Chain concentration
- Generation 3: Omnichannel
- Generation 4: Consumer-centric
- Generation 5: Consumption-centric
Although incremental change allows companies to experiment and iterate during their digital journeys, the COVID-19 pandemic rapidly accelerated the evolution of online and contactless shopping. Evolution became a revolution, with most Consumer Product and Retail (CPR) companies still mastering the omnichannel generation of their digital transformation to create a seamless shopping experience. Companies that are more digitally mature are already aspiring to the consumer-centric phase, embracing opportunities made possible by technology such as personalization and automation.
What consumers want
Customers have more choices in how they shop and engage with brands. This has made it harder for brands to predict and anticipate needs across customer journeys. And that’s convincing some companies to innovate more quickly as consumer demand drives the need for speed and scale.
Think about your own online behavior. Say you’re shopping for an item and searching online for the closest store in your neighborhood. Google is likely your go-to for finding that information. Looking to troubleshoot an issue with a product or seek out a service? Again, you’ll likely hit up Google first, not even considering going directly to a brand’s website for answers.
Both scenarios point to a disconnect between virtual and physical worlds, a gap technology can bridge in numerous ways such as breaking down silos and integrating fragmented media channels. Interestingly, though, not everything will be centered online all the time. In our recent study on consumer behavior, The great consumer reset, Capgemini discovered 57 percent of shoppers plan to return to brick-and-mortar stores post-pandemic, which is basically unchanged from the 59 percent who often interacted with physical stores before.
But business as usual? Not even close. Consumers have come to expect a frictionless shopping experience (buy online, pick up in store) or an immersive one (products displayed online using augmented reality), and are not content to return to in-store lineups, empty shelves, or a one-size-fits-all approach. Moreover, customers want personalized interactions while ensuring their data and privacy are protected.
So the role of the store is changing. In fact, many online-only brands are opening brick-and-mortar establishments to drive customer experience. In our research on “smart stores,” we found the majority of consumers (66 percent) believe automation can improve their shopping experience by solving the challenges they face at retail stores.
From personalization to serendipity
Retailers must recognize that they have to win consumer trust and confidence. Many consumers believe retailers’ use of tech is focused on reducing costs rather than easing friction. And they’re right. That same Capgemini research found that only one-third (35 percent) of retailers consider “solving customer pain points” as the most important criteria when deciding which automation use cases to implement.
“Retailers are largely in the early stages of adopting automation, and that’s an opportunity to rethink how they’re using technology, not just to smooth out friction and engender consumer trust but to build unexpected consumer benefits,” says Neerav Vyas, Head of Customer First, Co-Chief Innovation Officer, Insights & Data, North America, Capgemini. “We’re trying to move towards this idea of delivering serendipitous experiences to bridge the physical and digital divide.”
The focus is not solely on shoppers seeking out a specific product. “When consumers are in an exploratory mood, retailers can recommend products and services customers didn’t even know they wanted,” says Vyas. For example, business teams that use personalization platforms as part of an integrated media strategy can optimize algorithms against outcomes such as improving conversion and driving engagement.
Vyas says the elevated experience from “personalization to serendipity” fosters trust in the ability of recommendation architectures to persuade and influence consumers’ choices in beneficial ways. A case in point: our research found that half (52 percent) of spending by millennials goes towards experience-related purchases. As always, the key is to meet consumers where they are. Even better, according to Vyas, is to anticipate and understand when signals like customer intent are changing.
How to create value throughout the customer journey
One solution companies can implement right now is an integrated media spend platform that incorporates reporting, planning, and strategy across the entire customer journey. This offers value throughout the customer journey by using technology to reduce friction along the way. Think of it as starting with the customer looking for a product (search and discovery), moving on to the purchase (omnichannel basket, “shoppable” screens) and pick up/delivery (QR code scan in store), and through to post-purchase engagement with the retailer (Google Contact Center AI).
Such a holistic approach also accelerates data acquisition, integration, and reporting using advanced analytics to break down silos and emphasize the importance of privacy and first-party data. This in turn guides end-to-end interventions across customer journeys that enable optimized media spend, empowers businesses to analyze their spend distribution, fine-tunes owned and paid tactics with agencies, and promotes stewardship to support audit efficacy.
A culture of experimentation
With this data-driven focus, CPRs can create a 360-degree perspective of the customer. That intelligence can be used to enhance and humanize automated shopping experiences by putting the customer in control, whether online, in store, or across company brands. Moreover, by using Google Cloud’s emerging technologies such as artificial intelligence (AI), we help companies accelerate value across the spectrum, from supply-chain optimization and customer innovation to consumer experience.
Building a culture of experimentation is a team effort. “It’s not ever just one person who had a big idea. It’s all incremental steps,” says Jennifer Marchand, Google Cloud COE Leader, Capgemini. “Finding the right use case and timing is everything.” Take Google Glass Enterprise, she continues. For greater consumer experience, it can enable in-store associates to better serve with hands-free checkout, customer personalization and recommendations, and special offers. At the same time, Computer Vision and Smart Shelves can help prioritize tasks for employees, notifying them of low stock or a spill in the store.
Marchand points out that companies and consumers alike might not be ready to fully adopt some technology like facial recognition, but since almost everyone has a smartphone these days, they can benefit from automation with ease. “What’s interesting,” she adds, “is the way Google thinks about these types of problems, solving them for the long term.”
The store of tomorrow
Imagine a truly frictionless shopping experience, where state-of-the-art computer vision and AI identifies the products you pick up, put back, and keep, allowing you to head home, completely bypassing the checkout, with a 99 percent accuracy rate and receipts sent directly to your mobile app. That utopian experience is already taking shape at CornerShop, Capgemini’s live experimental store in London, UK.
Jamie MacLoud, Transformation & Strategy Consultant at frog, part of Capgemini, describes the retail space, which runs on Google Cloud, as the store of tomorrow, and not the distant future. It’s an experiential space where retailers and brands can explore, develop, and test technological shopping innovations in real-time. The outcome is a clearer understanding of how digital innovation can enable new ways to progress the customer experience, improve in-store operations, and help consumers to rediscover the joy of in-person retail through new ways to shop and engage with brands.
“We build, test, and learn about store concepts of tomorrow that we believe could be implemented into actual stores in the next one to two years. Getting these experiences in front of real customers in the CornerShop allows us to generate tangible learnings that we can share with our clients and use to shape future store strategy” says MacLeod. CornerShop was opened to the public in two eight-week stints, which allowed real-time testing to see what technologies resonated with customers, which brands can adapt and scale. Frictionless checkout, not surprisingly, was a big win for customers, but the technology underpinning the “virtual try-on” of clothing was deemed more suitable for the store of the future.
So, unlike an innovation lab, CornerShop lets companies experiment risk free, speeding up the process from hypothesis to full-scale implementation. It’s also another step toward solving the challenges customers face, while delivering those serendipitous experiences that build brand loyalty and longevity.
Learn more about how Capgemini is partnering with Google Cloud to help retailers create next-generation shopping experiences today.
We would like to acknowledge Jamie MacLoud, Transformation & Strategy Consultant at frog, part of Capgemini and Neerav Vyas, Head of Customer First, Co-Chief Innovation Officer, Insights & Data, North America at Capgemini who supported with invaluable insights and subject matter expertise in the writing of this blog post.
Trend 2: Google Research on Machine Learning Themes for 2022 and Beyond!

2878
Of your peers have already read this article.
6:00 Minutes
The most insightful time you'll spend today!
Trend 2: Continued Efficiency Improvements for ML
Improvements in efficiency — arising from advances in computer hardware design as well as ML algorithms and meta-learning research — are driving greater capabilities in ML models. Many aspects of the ML pipeline, from the hardware on which a model is trained and executed to individual components of the ML architecture, can be optimized for efficiency while maintaining or improving on state-of-the-art performance overall. Each of these different threads can improve efficiency by a significant multiplicative factor, and taken together, can reduce computational costs, including CO2 equivalent emissions (CO2e), by orders of magnitude compared to just a few years ago. This greater efficiency has enabled a number of critical advances that will continue to dramatically improve the efficiency of machine learning, enabling larger, higher quality ML models to be developed cost effectively and further democratizing access. I’m very excited about these directions of research!
Continued Improvements in ML Accelerator Performance
Each generation of ML accelerator improves on previous generations, enabling faster performance per chip, and often increasing the scale of the overall systems. Last year, we announced our TPUv4 systems, the fourth generation of Google’s Tensor Processing Unit, which demonstrated a 2.7x improvement over comparable TPUv3 results in the MLPerf benchmarks. Each TPUv4 chip has ~2x the peak performance per chip versus the TPUv3 chip, and the scale of each TPUv4 pod is 4096 chips (4x that of TPUv3 pods), yielding a performance of approximately 1.1 exaflops per pod (versus ~100 petaflops per TPUv3 pod). Having pods with larger numbers of chips that are connected together with high speed networks improves efficiency for larger models.
ML capabilities on mobile devices are also increasing significantly. The Pixel 6 phone features a brand new Google Tensor processor that integrates a powerful ML accelerator to better support important on-device features.

Our use of ML to accelerate the design of computer chips of all kinds (more on this below) is also paying dividends, particularly to produce better ML accelerators.
Continued Improvements in ML Compilation and Optimization of ML Workloads
Even when the hardware is unchanged, improvements in compilers and other optimizations in system software for machine learning accelerators can lead to significant improvements in efficiency. For example, “A Flexible Approach to Autotuning Multi-pass Machine Learning Compilers” shows how to use machine learning to perform auto-tuning of compilation settings to get across-the-board performance improvements of 5-15% (and sometimes as much as 2.4x improvement) for a suite of ML programs on the same underlying hardware. GSPMD describes an automatic parallelization system based on the XLA compiler that is capable of scaling most deep learning network architectures beyond the memory capacity of an accelerator and has been applied to many large models, such as GShard-M4, LaMDA, BigSSL, ViT, MetNet-2, and GLaM, leading to state-of-the-art results across several domains.

Human-Creativity–Driven Discovery of More Efficient Model Architectures
Continued improvements in model architectures give substantial reductions in the amount of computation needed to achieve a given level of accuracy for many problems. For example, the Transformer architecture, which we developed in 2017, was able to improve the state of the art on several NLP and translation benchmarks while simultaneously using 10x to 100x less computation to achieve these results than a variety of other prevalent methods, such as LSTMs and other recurrent architectures. Similarly, the Vision Transformer was able to show improved state-of-the-art results on a number of different image classification tasks despite using 4x to 10x less computation than convolutional neural networks.
Machine-Driven Discovery of More Efficient Model Architectures
Neural architecture search (NAS) can automatically discover new ML architectures that are more efficient for a given problem domain. A primary advantage of NAS is that it can greatly reduce the effort needed for algorithm development, because NAS requires only a one-time effort per search space and problem domain combination. In addition, while the initial effort to perform NAS can be computationally expensive, the resulting models can greatly reduce computation in downstream research and production settings, resulting in greatly reduced resource requirements overall. For example, the one-time search to discover the Evolved Transformer generated only 3.2 tons of CO2e (much less than the 284t CO2e reported elsewhere; see Appendix C and D in this joint Google/UC Berkeley preprint), but yielded a model for use by anyone in the NLP community that is 15-20% more efficient than the plain Transformer model. A more recent use of NAS discovered an even more efficient architecture called Primer (that has also been open-sourced), which reduces training costs by 4x compared to a plain Transformer model. In this way, the discovery costs of NAS searches are often recouped from the use of the more-efficient model architectures that are discovered, even if they are applied to only a handful of downstream uses (and many NAS results are reused thousands of times).

NAS has also been used to discover more efficient models in the vision domain. The EfficientNetV2 model architecture is the result of a neural architecture search that jointly optimizes for model accuracy, model size, and training speed. On the ImageNet benchmark, EfficientNetV2 improves training speed by 5–11x while substantially reducing model size over previous state-of-the-art models. The CoAtNet model architecture was created with an architecture search that uses ideas from the Vision Transformer and convolutional networks to create a hybrid model architecture that trains 4x faster than the Vision Transformer and achieves a new ImageNet state of the art.

The broad use of search to help improve ML model architectures and algorithms, including the use of reinforcement learning and evolutionary techniques, has inspired other researchers to apply this approach to different domains. To aid others in creating their own model searches, we have open-sourced Model Search, a platform that enables others to explore model search for their domains of interest. In addition to model architectures, automated search can also be used to find new, more efficient reinforcement learning algorithms, building on the earlier AutoML-Zero work that demonstrated this approach for automating supervised learning algorithm discovery.
Use of Sparsity
Sparsity, where a model has a very large capacity, but only some parts of the model are activated for a given task, example or token, is another important algorithmic advance that can greatly improve efficiency. In 2017, we introduced the sparsely-gated mixture-of-experts layer, which demonstrated better results on a variety of translation benchmarks while using 10x less computation than previous state-of-the-art dense LSTM models. More recently, Switch Transformers, which pair a mixture-of-experts–style architecture with the Transformer model architecture, demonstrated a 7x speedup in training time and efficiency over the dense T5-Base Transformer model. The GLaM model showed that transformers and mixture-of-expert–style layers can be combined to produce a model that exceeds the accuracy of the GPT-3 model on average across 29 benchmarks using 3x less energy for training and 2x less computation for inference. The notion of sparsity can also be applied to reduce the cost of the attention mechanism in the core Transformer architecture.

The use of sparsity in models is clearly an approach with very high potential payoff in terms of computational efficiency, and we are only scratching the surface in terms of research ideas to be tried in this direction.
Each of these approaches for improved efficiency can be combined together so that equivalent-accuracy language models trained today in efficient data centers are ~100 times more energy efficient and produce ~650 times less CO2e emissions, compared to a baseline Transformer model trained using P100 GPUs in an average U.S. datacenter using an average U.S. energy mix. And this doesn’t even account for Google’s carbon-neutral, 100% renewable energy offsets. We’ll have a more detailed blog post analyzing the carbon emissions trends of NLP models soon.
Google and Fervo Agreement to Shape-up Plans for 24/7 Carbon-free Energy by 2030

4787
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
When Google announced our plan to go beyond purchasing renewable power for 100% of our energy usage and operate on 24/7 carbon-free energy by 2030, we noted that achieving this goal will require new transaction structures, advancements in clean energy policy, and innovative new technologies. Today, we’re pleased to announce that one of these new technologies—a first-of-its-kind, next-generation geothermal project—will soon begin adding carbon-free energy to the electric grid that serves our data centers and infrastructure throughout Nevada, including our Cloud region in Las Vegas.
Google and clean-energy startup Fervo have just signed the world’s first corporate agreement to develop a next-generation geothermal power project, which will provide an “always-on” carbon-free resource that can reduce our hourly reliance on fossil fuels. In 2022, Fervo will begin adding “firm” geothermal energy to the state’s electric grid system, where Google’s commitments already include one of the world’s largest corporate solar-plus-storage power purchase agreements.
Importantly, this collaboration also sets the stage for next-generation geothermal to play a role as a firm and flexible carbon-free energy source that can increasingly replace carbon-emitting fossil fuels—especially when aided by policies that expand and improve electricity markets; incentivize deployment of innovative technologies; and increase investments in clean energy research, development, and demonstration (RD&D).
Next-generation geothermal technology
Traditional geothermal already provides carbon-free baseload energy to a number of power grids. But because of cost and location constraints, it accounts for a very small percentage of global clean energy production.
That’s one reason this new approach is so exciting; by using advanced drilling, fiber-optic sensing, and analytics techniques, next-generation geothermal can unlock an entirely new class of resource. And the US Department of Energy has found that with advancements in policy, technology, and procurement, geothermal energy could provide up to 120 GW of firm, flexible generation capacity in the US by 2050.
As part of our agreement, Google is partnering with Fervo to develop AI and machine learning that could boost the productivity of next-generation geothermal and make it more effective at responding to demand, while also filling in the gaps left by variable renewable energy sources. Although this project is still in the early stages, it shows promise.

Using fiber-optic cables inside wells, Fervo can gather real-time data on flow, temperature, and performance of the geothermal resource. This data allows Fervo to identify precisely where the best resources exist, making it possible to control flow at various depths. Coupled with the AI and machine learning development outlined above, these capabilities can increase productivity and unlock flexible geothermal power in a range of new places.
This won’t be the first time that Google is applying software solutions to clean energy applications: we’ve just announced an update to our carbon-intelligent computing program that helps us reduce emissions associated with running applications at Google data centers. And other forms of AI and machine learning are currently being used to increase the value of wind energy.
Already this year, Google has taken significant strides toward sourcing 24/7 carbon-free energy for all our data centers, office campuses, and Cloud regions. On Earth Day, our CEO Sundar Pichai announced that for the first time, five of our global data center sites operated near or at 90% carbon-free energy in 2020.
Not only does this Fervo project bring our data centers in Nevada closer to round-the-clock clean energy, but it also acts as a proof-of-concept to show how firm clean energy sources such as next-generation geothermal could eventually help replace carbon-emitting power sources around the world.
Enabling Sustainable Agriculture: InstaDeep uses Cloud TPU v4

2642
Of your peers have already read this article.
3:30 Minutes
The most insightful time you'll spend today!
You are what you eat. We’ve all been told this, but the truth is what we eat is often more complex than we are – genetically at least. Take a grain of rice. The plant that produces rice has 40,000 to 50,000 genes, double that of humans, yet we know far more about the composition of the human genome than of plant life. We need to close this knowledge gap quickly if we are to answer the urgent challenge of feeding 8 billion people, especially as food security around the globe is likely to worsen with climate change.
For this reason, AI company InstaDeep has teamed up with Google Cloud to train a large AI model with more than 20 billion parameters on a dataset of reference genomes for cereal crops and edible vegetables, using the latest generation of Google’s Tensor Processing Units (Cloud TPU v4), which is particularly suited for training efficiency at scale. Our aim is to improve food security and sustainable agriculture by creating a tool that can analyze and predict plants’ agronomic traits from genomic sequences. This will help identify which genes make some crops more nutritious, more efficient to grow, and more resilient and resistant to pests, disease and drought.
Genomic language models for sustainable agriculture
Ever since farming began, we have been, directly or indirectly, trying to breed better crops with higher yields, better resilience and, if we’re lucky, better taste too. For thousands of years, this was done by trial and error, growing crops year-on-year while trying to identify and retain only the most beneficial traits as they naturally arise from evolutionary mutations. Now that we have access to the genomic sequences of plants, we hope to directly identify beneficial genes and predict the effect of novel mutations.
However, the complexity of plant genomes often makes it difficult to identify which variants are beneficial. Revolutionary advances in machine learning (ML) can help to understand the link between DNA sequences and molecular phenotypes. This means we now have precise and cost-effective prediction methods to help us close the gap between genetic information and observable traits. These predictions can help identify functional variants and accelerate our understanding of which genes link to which traits – so we can make better crop selections.
Moreover, thanks to the vast library of available crop genetic sequences, training large models on hundreds of plant genomes means we can transfer the knowledge from thoroughly-studied species to those that are less understood but important for food production – especially in developing countries. And by doing this digitally, AI can quickly map and annotate the genomes of both common and rare crop variants.
One of the major limitations of traditional ML methods for plant genomics has been they mostly rely on supervised learning techniques. They need labeled data. Such data is scarce and expensive to collect, severely limiting these methods. Recent advances in natural language processing (NLP), such as Transformer architectures and BERT-style training (Bidirectional Encoder Representations from Transformers), allow scientists to train massive language models on raw text data to learn meaningful representations. This unsupervised learning technique changes the game. Once learned, the representations can be leveraged to solve complex regression or classification tasks – even when there is a lack of labeled data.
InstaDeep partners with Google Cloud to train the new generation of AI models for genomics on TPUs
Researchers have demonstrated that large language models can be especially effective in proteomics. To understand how this works, imagine reading amino acids as words and proteins as sentences. The treasure trove of raw genomics data – in sequence form – inspired InstaDeep and Google Cloud to apply similar technologies on nucleotides, this time reading them as words and chunks of genomes as sentences.
Moreover, the representations that the system learned improved in line with the size of the models and datasets, NLP research studies showed. This finding led InstaDeep researchers to train a set of increasingly larger language models on genomics datasets ranging from 1 billion to 20 billion parameters.
- Models of 1 billion and 5 billion parameters were trained on a dataset comprising the reference genomes for several edible plants, including fruit, cereal and vegetables for a total of 75 billion nucleotides.
- The training dataset must increase in the same proportion as the model capacity, recent work has shown. Thus, we created a larger dataset gathering all reference genomes available on the National Center for Biotechnology Information (NCBI) database including human, animal, non-edible plant and bacteria genomes. This dataset, which we used to train a 20-billion-parameter Transformer model, comprised 700 billion tokens, exceeding the size of most datasets typically used for NLP applications, such as the Common Crawl or Wikipedia dataset.
- Both teams announced that the 1 billion-parameter model will be shared with the scientific community to further accelerate plant genomics research.
The compact and meaningful representations of nucleotide sequences learned by these models can be used to tackle molecular phenotype prediction problems. To showcase their ability, we trained a model to predict the gene function and gene ontology (i.e. a gene’s attribute) for different edible plant species.
Early results have demonstrated that this model can predict these characteristics with high accuracy – encouraging us to look deeper at what these models can tell us. Based on these results, we decided to annotate the genomes of three plant species with considerable importance for many developing countries: cassava, sweet potato, and yam. We are working on making these annotations freely available to the scientific community and hope that these will be used to further guide and accelerate new genomic research.

Overcoming scaling challenges with massive models and datasets with Cloud TPUs
The compute requirement for training our 20 billion-parameter model with billions of tokens is massive. While modern accelerators offer impressive peak performance per chip, to utilize this performance often requires tightly coupled hardware and software optimizations. Moreover, maintaining this efficiency when scaling to hundreds of chips presents additional system design challenges. The Cloud TPU’s tightly-coupled hardware and software stack is especially well suited to such challenges. The Cloud TPU software stack is based on the XLA Compiler which offers out-of-the-box optimizations (such as compute and communication overlap) and an easy programming model for expressing parallelism.
We successfully trained our large models for genomics by leveraging Google Tensor Processing Units (TPUv4). Our code is implemented with the JAX framework. JAX provides a functional programming-based approach to express computations as functions that can be easily parallelized using JAX APIs powered by XLA. This helped us to scale from a single host (four chips) configuration to a multi-host configuration without having to tackle any of the system design challenges. The TPU’s cost-effective inter- and intra-communication capabilities led to an almost linear scaling between the number of chips and training time. This allowed us to train the models quickly and efficiently on a grid of 1024 TPUv4 cores (512 chips).
Conclusion
Ultimately, our hope is that the functional characterization of genomic variants predicted by deep learning models will be critical to the next era in agriculture, which will largely depend on genome editing and analysis. We envisage that novel approaches, such as in-silico mutagenesis – the assessment of all possible changes in a genomic region by a computer model – will be invaluable in prioritizing mutations that improve plant fitness and guiding crop improvements. Attempting similar work in wet-lab experiments would be difficult to scale and nearly impossible in nature. By making our current and future annotations available to the research community, we also hope to help democratize breeding technologies so that they can benefit all of global agriculture.
Further Reading
To learn more about the unique features of Cloud TPU v4 hardware and software stack we encourage readers to explore Cloud TPU v4 announcement. To learn more about scaling characteristics, please see this benchmark and finally we recommend reading PJIT Introduction to get started with JAX and SPMD parallelism on Cloud TPU.
This research was made possible thanks to the support of Google’s TPU Research Cloud (TRC) Program which enabled us to use the Cloud TPUv4 chips that were critical to this work.
Quick Recap on Google Cloud: Latest News, Launches, Updates, Events and More

12872
Of your peers have already read this article.
2:00 Minutes
The most insightful time you'll spend today!
Want to know the latest from Google Cloud? Find it here in one handy location. Check back regularly for our newest updates, announcements, resources, events, learning opportunities, and more.
Tip: Not sure where to find what you’re looking for on the Google Cloud blog? Start here: Google Cloud blog 101: Full list of topics, links, and resources.
Week of May 24-May 28 2021
- Google Cloud for financial services: driving your transformation cloud journey–As we welcome the industry to our Financial Services Summit, we’re sharing more on how Google Cloud accelerates a financial organization’s digital transformation through app and infrastructure modernization, data democratization, people connections, and trusted transactions. Read more or watch the summit on demand.
- Introducing Datashare solution for financial services–We announced the general availability of Datashare for financial services, a new Google Cloud solution that brings together the entire capital markets ecosystem—data publishers and data consumers—to exchange market data securely and easily. Read more.
- Announcing Datastream in Preview–Datastream, a serverless change data capture (CDC) and replication service, allows enterprises to synchronize data across heterogeneous databases, storage systems, and applications reliably and with minimal latency to support real-time analytics, database replication, and event-driven architectures. Read more.
- Introducing Dataplex: An intelligent data fabric for analytics at scale–Dataplex provides a way to centrally manage, monitor, and govern your data across data lakes, data warehouses and data marts, and make this data securely accessible to a variety of analytics and data science tools. Read more.
- Announcing Dataflow Prime–Available in Preview in Q3 2021, Dataflow Prime is a new platform based on a serverless, no-ops, auto-tuning architecture built to bring unparalleled resource utilization and radical operational simplicity to big data processing. Dataflow Prime builds on Dataflow and brings new user benefits with innovations in resource utilization and distributed diagnostics. The new capabilities in Dataflow significantly reduce the time spent on infrastructure sizing and tuning tasks, as well as time spent diagnosing data freshness problems. Read more.
- Secure and scalable sharing for data and analytics with Analytics Hub–With Analytics Hub, available in Preview in Q3, organizations get a rich data ecosystem by publishing and subscribing to analytics-ready datasets; control and monitoring over how their data is being used; a self-service way to access valuable and trusted data assets; and an easy way to monetize their data assets without the overhead of building and managing the infrastructure. Read more.
- Cloud Spanner trims entry cost by 90%–Coming soon to Preview, granular instance sizing in Spanner lets organizations run workloads at as low as 1/10th the cost of regular instances, equating to approximately $65/month. Read more.
- Cloud Bigtable lifts SLA and adds new security features for regulated industries–Bigtable instances with a multi-cluster routing policy across 3 or more regions are now covered by a 99.999% monthly uptime percentage under the new SLA. In addition, new Data Access audit logs can help determine whether sensitive customer information has been accessed in the event of a security incident, and if so, when, and by whom. Read more.
- Build a no-code journaling app–In honor of Mental Health Awareness Month, Google Cloud’s no-code application development platform, AppSheet, demonstrates how you can build a journaling app complete with titles, time stamps, mood entries, and more. Learn how with this blog and video here.
- New features in Security Command Center—On May 24th, Security Command Center Premium launched the general availability of granular access controls at project- and folder-level and Center for Internet Security (CIS) 1.1 benchmarks for Google Cloud Platform Foundation. These new capabilities enable organizations to improve their security posture and efficiently manage risk for their Google Cloud environment. Learn more.
- Simplified API operations with AI–Google Cloud’s API management platform Apigee applies Google’s industry leading ML and AI to your API metadata. Understand how it works with anomaly detection here.
- This week: Data Cloud and Financial Services Summits–Our Google Cloud Summit series begins this week with the Data Cloud Summit on Wednesday May 26 (Global). At this half-day event, you’ll learn how leading companies like PayPal, Workday, Equifax, and many others are driving competitive differentiation using Google Cloud technologies to build their data clouds and transform data into value that drives innovation. The following day, Thursday May 27 (Global & EMEA) at the Financial Services Summit, discover how Google Cloud is helping financial institutions such as PayPal, Global Payments, HSBC, Credit Suisse, AXA Switzerland and more unlock new possibilities and accelerate business through innovation. Read more and explore the entire summit series.
- Announcing the Google for Games Developer Summit 2021 on July 12th-13th–With a surge of new gamers and an increase in time spent playing games in the last year, it’s more important than ever for game developers to delight and engage players. To help developers with this opportunity, the games teams at Google are back to announce the return of the Google for Games Developer Summit 2021 on July 12th-13th. Hear from experts across Google about new game solutions they’re building to make it easier for you to continue creating great games, connecting with players and scaling your business. Registration is free and open to all game developers. Register for the free online event at g.co/gamedevsummit to get more details in the coming weeks. We can’t wait to share our latest innovations with the developer community. Learn more.
More Relevant Stories for Your Company
ESG Did the Math: It’s Cheaper and Smarter to Migrate Enterprise Data Warehouses to Google BigQuery. Way Smarter
Enterprise data warehouses (EDWs) are often deemed the most valuable asset in the data center, serving as the backbone of the business. The ongoing insight gained from these solutions has justified the significant up-front capital investments and ongoing operational costs, but the rigidity of the traditional EDW is forcing organizations

What’s Next for Personalization on Google Cloud
Customer shopping behavior has changed for good. With fewer in-store shopping visits retailers have had to shore up their digital storefronts and explore new ways to meaningfully engage with their customers. Delivering a superior customer experience has become even more of a differentiator for the early movers and personalized recommendations

2021 was the Momentum for Contact Center AI!
2021 has been a high-stakes year for call centers, with many organizations forced to rapidly scale up their call center operations in response to ongoing pandemic disruptions. We’re proud that 2021 has also been an amazing year for Google Cloud’s Contact Center AI (CCAI), which has helped our customers adapt

Vector Search: The Tech Powering Billions of Search Results for Google Users
Recently, Google Cloud partner Groovenauts, Inc. published a live demo of MatchIt Fast. As the demo shows, you can find images and text similar to a selected sample from a collection of millions in a matter of milliseconds: Image similarity search with MatchIt Fast Give it a try — and either select a preset







