Communication service providers (CSPs) are at an inflection point. From stagnating revenues, to network strain in meeting the demands of 5G, to challenges in delivering innovative customer experiences, there’s enormous pressure on the telecommunications industry to transform.

Over the past few years, CSPs around the globe have turned to artificial intelligence (AI) to address some of these challenges, but the lion’s share of an operator’s operational expenses are still spent on infrastructure and data management. This has limited their ability to capitalize on core data assets and develop differentiated customer experiences that meet individual needs. 

Enter generative AI, a type of machine intelligence that’s received enormous attention as of late. We’ve all marveled over its ability to generate text that reads like it was written by a person, to create new images, and to build even musical scores. It’s a riveting addition to the AI toolset — and one that complements machine learning (ML) and its ability to identify patterns to make predictions, spot efficiencies, or interpret large data sets. 

But while there’s a lot of hype around generative AI, at Google Cloud we view it through a much more practical lens for the telecommunications industry. Generative AI can accelerate the transformation already underway, with its potential to streamline many of the tools and processes that CSPs engage with daily, bringing a new level of natural interaction between people and computers, and enabling machines to be programmed to carry out an action with a spoken request, and respond in natural, interactive ways. 

Generative AI builds on existing Google Cloud data, AI, and ML services. For example, Contact Center AI, with human-like interactions between callers and computers, has been successfully adopted by CSPs for many years, increasing the satisfaction of both customers and call center workers. As we add generative AI to this technology, CSPs and their customers will see even greater capabilities and impact, for example, virtual agents that not only provide helpful information, but also let customers make payments and execute other transactions. With generative AI, CSPs will be able to harness customer call summaries to better understand customer sentiment and identify cross-sell and up-sell opportunities. CSPs could also easily and quickly build and deploy virtual agents informed by customer conversations that enable more innovative and personalized customer interactions. And that’s just the start.

Three key areas

The contact center is just one of the areas where practical and generative AI will help drive new value. When reflecting on the key challenges facing CSPs today, three areas in particular stand out where generative AI may be transformative:

1. Personalized experiences: In addition to further improvements in customer call center interactions, generative AI can deliver improved personalization in ecommerce interactions — a big factor in helping customers sort through their choices of phones and calling plans. Personalization is also important for lowering churn, offering relevant new services, and managing the customer lifecycle. For example, generative AI could enable CSPs to produce marketing campaign content customized for select themes, and target individual customers with customized text and images. 
2. Autonomous networks: Generative AI will also help to pave the way for autonomous networks by connecting multiple complex AI/ML models used across network planning and operations with large language models (LLMs) that can understand network behaviors and create action plans in areas including network capacity planning and performance. For example, generative AI will enable CSPs to train models with customer experience and sentiment data to build better prediction capabilities. Importantly, the customer data sets used to tune these models are not public, but curated internal customer data — significantly enhancing privacy, factuality, and relevance, while protecting intellectual property. In addition, generative AI will be able to help network planning and design, which requires high levels of reporting and analysis.
3. Streamlined operations: Both operations center uptime and field service efficiency are crucial for managing costs and improving customer satisfaction. In particular, applying generative AI to field service devices can speed up diagnostics and analysis, and can even help with installation, parts, and troubleshooting, and minimizing the number of times companies have to send out trucks and improve field-service training. Generative AI will also provide a productivity boost to IT development processes, enabling code generation and troubleshooting to deliver reliable software products and services.

Data security and reliability

An under-discussed area in generative AI is the importance of data quality and data security in building and training the LLMs that power the technology. Many CSPs are rightly concerned about intellectual property leaking both into and out of LLMs, risking the security of their systems and their intellectual property. We’ve long offered industry-leading data security and privacy technologies, and with generative AI integrated with Vertex AI, we can ensure all data is secured within the CSP’s environment.

Meanwhile, to ensure that their LLMs generate accurate information, CSPs are building out scenarios and use cases for training on smaller, controlled amounts of their own data, sometimes accompanied by highly trusted sources from partners and others. Google Cloud also provides tools including Prompt Engineering, Tuning, and Reinforcement Learning from Human Feedback to further ensure data factuality and reliability. This will likely result in the first generative AI applications targeted to smaller, high-impact problems, like optimizing network topologies. 

The human element

Of course, people are critical to the success of generative AI, whether that means solving problems in call centers, field service workers combining AI information with their own knowhow, marketing and creative teams brainstorming with generative AI to make new presentations and marketing materials, or operations engineers augmenting and approving an AI suggestion. We’ve built a lot of amazing technology to help augment what people cannot do: synthesize millions, if not billions, of records and sources to help energize new workflows and productivity.

Telecommunications is a fast-changing industry that is tech savvy and hungry to learn and deploy the best possible new technologies, and that includes generative AI. Each meeting with a CSP inspires new ideas, sparks more use cases, and leads to more industry-changing initiatives. It’s exciting to see this pace of change, and we are only just getting started.

We look forward to showing you more soon as we deep dive into some of the exciting CSP industry use cases in upcoming blog posts. We also invite you to find out more about how Google Cloud is partnering with CSPs around the world to deliver a holistic cloud transformation.

For more than 10 years, NMK Textile Mills has manufactured bed linens for major retailers in the United States and Canada. As e-commerce exploded, the company’s co-founder saw an opportunity to grow the business. So, in addition to manufacturing bed linens wholesale for retail customers, NMK Textile Mills reworked its complete supply chain to manufacture products for California Design Den, which became an e-commerce retailer selling its own fashion-forward products directly to consumers online.

With California Design Den’s push into e-commerce, it became apparent the SMB company (with about 250 global employees) faced the same tough supply chain questions as large retail customers, including maintaining enough inventory to meet customer demand in a timely, efficient way.

California Design Den depended upon a myriad of systems to track its complex forecasting and reordering processes. Team members typically planned inventory manually using desktop spreadsheet software, which could lead to excess inventory. Accurately forecasting demand and supply was essential to the company’s financial success—but it was also a challenge.

A couple years ago, California Design Den partnered with Pluto7, a technology solutions provider that offers a software as a service (SaaS) called Planning In A Box. Leveraging Google Cloud Platform machine learning and artificial intelligence, Planning In A Box intelligently helps predict demand and balances it with supply.

But that was just the beginning. “Along the way, we realized that to compete with larger retailers, make quicker decisions, and move faster, we needed to go further,” says Deepak Mehrotra, Co-founder and Chief Adventurer at California Design Den.

With guidance from Pluto7, California Design Den began migrating its database to Google Cloud Platform. Using Google BigQuery, Google Compute Engine, Google Cloud SQL, and Google Cloud Storage, and experimenting with Google Cloud Vision and Google Cloud AutoML, the company is reducing inventory carryovers by more than 50%, improving the accuracy of demand planning quarter over quarter, and gaining granular insights into how individual SKUs are performing.

“We would need an army of data scientists to make faster decisions on pricing and inventory levels. With Google Cloud Platform machine learning and artificial intelligence, we don’t need that. We can make much faster pricing decisions to optimize profitability and move inventory.”

—Deepak Mehrotra, Co-founder and Chief Adventurer, California Design Den

No need for army of data scientists

“Using Google Cloud Platform machine learning and AI was essential for California Design Den if it was to compete successfully with larger retailers,” Deepak says.

For example, tastes and fashions in bed linens can change quickly and consumer prices fluctuate. With more than 2,500 SKUs, it wasn’t possible for California Design Den’s team to continuously monitor product demand and experiment with competitive pricing in real time.

“We would need an army of data scientists to make faster decisions on pricing and inventory levels,” says Deepak. “With Google Cloud Platform machine learning and artificial intelligence, we don’t need that. We can make much faster pricing decisions to optimize profitability and move inventory.”

By integrating all its data onto Google Cloud Platform, California Design Den’s team gains deeper insights into product sales over time, which in turn helps the company improve demand planning by better determining which styles to manufacture and sell in the future.

“When experienced employees leave, their knowledge leaves with them. By having all data in one place, and with machine learning and AI, California Design Den can go back in its history, look at products made or sold years ago, and analyze product performance.”

—Deepak Mehrotra, Co-founder and Chief Adventurer, California Design Den

Merging visuals with data

Before Google Cloud Platform, team members had to dig through spreadsheets and run scenarios to get a sense of how particular products had sold. The next step was to perform keyword searches across the company’s photo library in the cloud to find each product’s image. From there, a team member would insert the product images into a presentation, along with relevant data points, to provide a report for stakeholders on how particular styles performed.

Today, California Design Den, with the help of Pluto7, is integrating its entire product image library with its database on Google Cloud Platform. Experimenting with Google Cloud Vision and Google Cloud AutoML, California Design Den is moving towards a day when team members can run sales scenarios and get deep background data on individual product performance while viewing images of the relevant products.

Merging product visuals with data will help designers and team members better understand sales patterns over time and in context. In the past, making correlations between things like which sheet colors sold well in California, compared to how the same sheet colors performed on the East Coast, was something that California Design Den employees primarily did in their heads.

“When experienced employees leave, their knowledge goes with them,” says Manjunath Devadas, Founder and CEO at Pluto7. “By having all data in one place, and with machine learning and AI, California Design Den can go back in its history, look at products made or sold years ago, and analyze product performance.”

“We are literally growing the complexity of our business on all levels, including designing, manufacturing, selling, reordering, inventory holding—everything.”

—Deepak Mehrotra, Co-founder and Chief Adventurer, California Design Den

Reimagining supply-demand balancing

Pluto7’s mission statement is to democratize supply demand balancing with machine learning and AI. California Design Den is a case in point, as the combination of Planning In A Box and Google Cloud Platform gives the company greater control over its destiny.

“Big retailers used to tell us what to manufacture and how much they would pay for it,” says Deepak. “That was our primary business, and if we didn’t accept the terms, a competitor would.” Today, in addition to continuing to make products for retailers, California Design Den can design, make, and sell a variety of designs for itself, including custom and limited-edition products, thanks to Google Cloud Platform and Pluto7 software offerings. The payoff is not only in having a more diversified business. California Design Den also receives more favorable profit margins by selling its own products.

“We are literally growing the complexity of our business on all levels, including designing, manufacturing, selling, reordering, inventory holding—everything,” Deepak says. “We can make smaller batches. We can connect directly with consumers. We can identify the missing pieces—what should we produce next, when, and how much? We otherwise couldn’t afford the level of talent it would take to do this.”

Cutting through the noise

Google and Pluto7 software helped California Design Den reduce inventory carryovers by more than 50%. Inventory tracking and distribution, along with insights and visibility into product sales, are faster, more efficient, and accurate. Google Cloud Platform flexible pricing, speed, reliability, security, and scalability enable California Design Den to stay relevant and be more competitive.

In addition to benefiting from Google Cloud Platform, California Design Den relies on G Suite—also part of Google Cloud—to enhance collaboration among its global teams. Previously, the company’s email server would sometimes crash, due to the heavy load of sharing product photos and other data. “Gmail and Google Drive handle the everyday demands on the business effortlessly and reliably,” Deepak says.

“Google machine learning and AI enable us to cut through all the noise from raw data, so we can see what’s important. We can focus on analytics to guide us to success today and in the future.”

—Deepak Mehrotra, Co-founder and Chief Adventurer, California Design Den

The company is exploring additional ways to leverage Google Cloud Platform in the near future. For example, one possibility is to import customer reviews from sites where products are sold into Google BigQuery, and to use that data to perform sentiment analysis via Google Cloud Natural Language. It could provide another valuable data source to help California Design Den’s team decide where to focus future designs.

“Google machine learning and AI enable us to cut through all the noise from raw data, so we can see what’s important,” Deepak says. “We can focus on analytics to guide us to success today and in the future.”

Be it marketing, sales, or even security, most departments in large organizations today have a range of SaaS tools at their disposal to help make their work more efficient. But people in corporate finance and accounting have been underserved in that respect. Their days are still spent on routine, mundane tasks that keep them away from more stimulating work. Auditoria.AI aims to change that by automating those functions with AI and natural language technology.

We strive to improve the lives of finance and accounting professionals by automating the routine, repetitive, and laborious parts of the finance function, such as copy-and-pasting data, validating documents, and checking for errors on spreadsheets, freeing these teams to focus instead on providing valuable, strategic insights to the business. 

To that end, the problems we’re solving affect three major finance functions:

• Accounts Payable, responsible for sending money out of the company, such as bill payments. 
• Accounts Receivable, responsible for bringing money into the company, such as invoicing for services. 
• General accounting, a broader term consisting of functions of the general ledger team and the CFO, including closing books. 

Historically, making these processes more efficient entailed dedicating more personnel to them. But this didn’t necessarily mean more work was done faster and to the highest standards. Many finance professionals are often overworked, dedicating extra hours, weekends, and sometimes holidays to process invoices, collect payments, and close the books on time. 

We created solutions for these three finance functions, with our SmartBots taking care of the back-and-forth communications between finance teams, vendors, and customers. In large companies, these micro-transactions add up to thousands per day, resulting in finance professionals spending entire days reading inquiries, interpreting requests, looking for relevant information, and answering as many as possible. But with our AR helpdesk, for example, accounts receivable tasks, such as a request for a copy of an invoice, get automated. Our technology reads emails and attachments to understand what is being requested. Then it connects to the Enterprise Resource Planning (ERP) software to grab the relevant information and attach it to the email, so the recipient gets a response within 60 seconds. 

Building the smart assistant that finance teams need

In our automation flow, we constantly handle different types of documents, from invoices and tax forms to receipts and email messages. But processing the interaction between computers and human language is complex. You must detect intent and facts, and understand the context before finding the specific slots of information extraction that may be relevant to specific processes and requests. Our solution adds value by extracting the right information in the right context, from the right document, for the relevant finance function. Instead of building everything from scratch, we turned to Google Cloud’s Document AI to support extracting data from unstructured documents to understand and analyze them. 

Document AI comes with pre-built models that help analyze specific parts of our post-production lifecycle. For example, Invoice Parser extracts text and values from invoices, including invoice number, supplier name, invoice amount, tax amount, and invoice due date, all of which are necessary for our SmartBots to execute an extraction workflow. These out-of-the-box features significantly accelerate our own product development process and time-to-market, which are critical for the performance of a startup such as ours. 

To ensure a high quality of information extraction, we used to do document readings in-house. Having automated some of that with Document AI, we’re at 85% accuracy extracting files, and with some additional customization efforts, we will achieve 95%+ extraction accuracy.

Meanwhile, we have now streamlined internal processes, which ultimately translates into faster services for our customers. For example, assuming all the information has been provided, it generally took up to 15 minutes to process a tax form. We now do that in seconds. 

The value of automation doesn’t stop there. Using DocumentAI to automate structured data extraction from documents, we have managed to:

• Speed up the collection of general ledger entries by 90%+
• Reduce errors and omissions by 85%+
• Close books 20% faster
• Improved the productivity of full-time employees by 60%+
• Reduce process workload by 75%+
• Improve vendor serviceability by 75%+
• Reduce vendor risk and fraud by 50%+

Leveraging automation to focus on more innovation

Automating some of our processes with Document AI also means we have more time to focus on developing new features and further improving our solution. 80-90% of the time used for extracting custom fields from documents has now been automated with an OCR metadata library. 

With Document AI taking care of standard extraction, we focus on the intelligence we add to post-extraction. For example, when an invoice comes in from a vendor, our application needs to figure out which vendor it is to match it to the correct records in the ERP. But variations in the documents could interfere with that extraction process. The vendor’s trading name might be slightly different from the company’s name registered in our internal system, delaying the process. With more time on our hands, we’re now working on features enabling our models to leverage logos and other elements extracted from documents to swiftly match them to the correct company registered in our systems.  

With the benefits we’ve seen thus far, we look forward to accelerating our international growth. We’ll be relying on Google Cloud’s Document AI to automate operations, potentially in different languages, as we continually remove friction from the work lives of finance and accounting people worldwide.

Access to the cloud has advanced the distribution and consumption of financial information on a global scale. In parallel, the global financial data landscape has been transformed by an influx of alternative data sources, including social media, meteorological data, satellite imagery, and other data. Exchanges and market data providers now find they need to include these new datasets to enrich their products and compete, which has meant they now must consider cloud-based models to keep up with the demands of their customers who expect easy, quick, flexible and cost-efficient ways to consume market data.

To address these needs, today we’re announcing the general availability of Datashare for financial services, a new Google Cloud solution that brings together the entire capital markets ecosystem—data publishers, and data consumers—to exchange market data securely and easily.

Datashare helps organize third-party financial information, making it accessible and useful to market data publishers and data consumers. We open-sourced the entire Datashare solution so market data publishers can now onboard their licensed datasets to Google Cloud securely, quickly and easily, while data consumers can consume that data as a service in tools of their preference, such as BigQuery.

Three ways to distribute and consume your data

Batch data delivery

Datashare provides a batch data delivery mechanism for data publishers to deliver their reference data, historical tick data, alternative market data sources and more via BigQuery, reducing the administrative burden on data consumers to extract insights from data. 

Real-time data streaming delivery

By using this event-based data delivery channel for rapidly changing instrument prices, tick data, orders, news and others via Pub/Sub, data consumers can reliably process individual messages or rewind to a point in time to replay a prior market scenario and test model changes.

Monetizing licensed datasets

Market data publishers can onboard their licensed datasets to Google Cloud and make them available via a one-stop-shop on Google Cloud Marketplace, enabling a new sales channel to expand market reach.

Reference architecture

Check out the diagram below to see how you can share your batch and real-time data directly to your Google Cloud customers with BigQuery and Pub/Sub.

As you can see in the above reference architecture, both publishers and consumers can derive several benefits from the solution:

Benefits for data publishers

• You no longer have to maintain your own delivery and licensing infrastructure.
• You can easily package and deliver granular data products and experiments with SQL.
• You can have a solution that scales with your business as data volumes and number of customers grow.

Benefits for data consumers

• Your data is ready for analysis and machine learning (ML)— you no longer have to maintain extract, transform, and load (ETL) pipelines to load files and transform data.
• You can avoid the expense and burden of maintaining multiple copies of large data files.
• You can be more targeted with consumption of data using BigQuery queries, improving performance, and compliance, and reducing cost.

Accessing the datasets

Google Cloud has been working with multiple industry firms on innovating in the market data space. By using Datashare for publishing, data publishers can make their entire datasets available on Google Cloud. Early adopters of Datashare include firms such as OneTick and Accern. OneTick’s datasets include reference and historical futures data (that can be accessed in our console with your login). Accern’s datasets include alternative data such as market sentiment and credit analysis data (that can be accessed in our console with your login).

To make it more helpful, we partnered with Accern to create a hypothetical scenario to describe the data acquisition and analytics process step-by-step.

Accern use case 

As a sustainability analyst, you require an economic, social and governance (ESG) dataset to determine which sector is the most widely covered ESG sector by analysts, and to also identify the sector with the lowest ESG sentiment score. Now, you can discover and acquire an ESG dataset in Google Cloud.

Step 1. Navigate to the Financial Services solutions page in the Google Cloud console:

Step 2. Click a dataset, for example Accern AI-Generated ESG Insights, then review the overview details, plans and pricing, documentation and support information. To view the available pricing tiers, click ‘View All Plans’. Once you’ve decided on a tier that you would like to subscribe to, click ‘Select’, choose a billing account and review and accept the terms of service to complete the subscription. Once the steps are complete, click ‘Subscribe’ at the bottom. An overlay window will appear, click ‘Register with Accern’ to activate and complete the subscription.

Step 3. Once activation is complete, you’ll be directed to the Datashare ‘My Products’ screen. Voila! You are now subscribed to Accern’s ESG Scores dataset and can access it in your Google Cloud instance using BigQuery. To access the data, click the hour glass icon on the corresponding ‘My Products’ record that you just purchased. An overlay will present you with the details on the dataset and/or table. Click the ‘Navigate to Table’ button to navigate through to the BigQuery console.

Step 4. Now that you have access and are in the BigQuery console, it’s time to generate data insights.

^{For this example, we’ve eliminated the company identifying information that is included as part of the subscription and aggregated company ESG in a view where each row represents a day, an industry sector, a specific identified ‘ESG Issues’ (event_group and event) and the respective ‘ESG Sentiment’ per issue.}

For example, row 1 indicates that within the ‘Healthcare’ sector, there was a ‘Social – Civil Society’ issue identified and it had a negative ESG sentiment score of -15.35.

Step 5. Generate a report by exporting it to Data Studio to build visualizations and conduct additional analysis on the ESG data.

Select ‘Export’ and ‘Explore with Data Studio’.

Step 6. Build a simple/basic report.

Now that the ESG data appears in Data Studio, you can start by building a simple chart to help you understand which industry sectors have the highest volume of discussions around ESG and the overall ESG Sentiment per industry sector.

To build the chart:

• Select the chart type ‘Table’.

• Include Entity_Sector as your dimension to aggregate results by ‘Industry Sector.’
• Include Signal_ID as a measure to count the number of ESG passages identified per ‘Industry Sector.’
• Include AVG(Event_Sentiment) as a measure to display the overall ESG Sentiment per ‘Industry Sector’ across ESG Issues.

You can see sectors that are  most discussed when it comes to ESG related topics and their corresponding ‘ESG Sentiment’ scores.

Step 7. Build your final report in Data Studio.

As a next step you can further drill into the data to understand ESG data specific to each ‘Industry Sector’ and identify positive and negative ESG practices. 

Accern has built a more complex sample dashboard and made it available publicly here. You can interact with this report and play around with the data. The dashboard can help to identify material ESG insights for each sector to inform your investment and risk processes. If you have additional questions, you can reach out to Accern directly.

Discovering, accessing and analyzing licensed datasets is quick and easy. Stay tuned for more updates on new licensed datasets.

Publishing your data via Datashare

If you are a publisher of market data, alternative, or exotic data, you can use Datashare to get it published on Google Cloud Marketplace.

Start by joining the Partner Advantage program by registering for the Partner Advantage Portal and applying for the Partner Advantage Build Model engagement. Visit our getting started guide for information to get started on publishing licensed datasets in the Marketplace. Stay tuned for a future blog post about using Datashare to publish datasets in the Marketplace.

More solutions for capital markets

Check out other Google Cloud solutions for capital markets.

Over the past few years, advances in training large language models (LLMs) have moved natural language processing (NLP) from a bleeding-edge technology that few companies could access, to a powerful component of many common applications. From chatbots to content moderation to categorization, a general rule for NLP is that the larger the model, the greater the accuracy it’s able to achieve in understanding and generating language.

But in the quest to create larger and more powerful language models, scale has become a major challenge. Once a model becomes too large to fit on a single device, it requires distributed training strategies, which in turn require extensive compute resources with vast memory capacity and fast interconnects. You also need specialized algorithms to optimize the hardware and time resources.

Cohere engineers are working on solutions to this scaling challenge that have already yielded results. Cohere provides developers a platform for working with powerful LLMs without the infrastructure or deep ML expertise that such projects typically require. In a new technical paper, Scalable Training of Language Models using JAX pjit and TPUv4, engineers at Cohere demonstrate how their new FAX framework deployed on Google Cloud’s recently announced Cloud TPU v4 Pods addresses the challenges of scaling LLMs to hundreds of billions of parameters. Specifically, the report reveals breakthroughs in training efficiency that Cohere was able to achieve through tensor and data parallelism.

This framework aims to accelerate the research, development, and production of large language models with two significant improvements: scalability and rapid prototyping. Cohere will be able to improve its models by training larger ones more quickly, delivering better models to its customers faster. The framework also supports rapid prototyping of models that address specific objectives — for example, creating a generative model that powers customer-service chatbot — by experimenting and testing new ideas. The ability to switch back and forth among model types and optimize for different objectives will ultimately allow Cohere to offer models optimized for particular use cases.

The FAX framework relies heavily on the partitioned just-in-time compilation (pjit) feature of JAX, which abstracts the relationship between device and workload. This allows Cohere engineers to optimize efficiency, and performance by aligning devices and processes in the ideal configuration for the task at hand. Pjit works by compiling an arbitrary function into a single program (an XLA computation), that runs on multiple devices — even those residing on different hosts.

Cohere’s new solution also takes advantage of Google Cloud’s new TPU v4 Pods to perform tensor parallelism. which is more efficient than the earlier pipeline parallelism implementation. As the name suggests, the pipeline parallel approach uses accelerators in a linear fashion to scale a workload, like a single long assembly line. Accelerators must process each micro-batch of data before passing it along to the next one, and then run the backward pass in reverse order.

Tensor parallelism eliminates the accelerator idle time of pipeline parallelism, also known as the pipeline bubble. Tensor parallelism involves partitioning large tensors (mathematical arrays that define the relationship among multiple objects such as the words in a paragraph) across accelerators to perform computations at the same time on multiple devices. If pipeline parallelism is an ever-lengthening assembly line, tensor parallelism is a series of parallel assembly lines — one making the engine, the other the body, etc. — that simultaneously come together to form a complete car in a fraction of the time.

These computations are then collated, a process made practical thanks to Google Cloud TPU v4 VMs, which more than double the computational power. The superior performance of v4 chips has enabled Cohere to iterate on ideas and validate them 1.7X faster in computation than before.

At Cohere, we build cutting-edge natural language processing (NLP) services, including APIs for language generation, classification, and search. These tools are built on top of a set of language models that Cohere trains from scratch on Cloud TPUs using JAX. We saw a 70% improvement in training time for our largest model when moving from Cloud TPU v3 Pods to Cloud TPU v4 Pods, allowing faster iterations for our researchers and higher quality results for our customers. The exceptionally low carbon footprint of Cloud TPU v4 Pods was another key factor for us.

Aidan Gomez
CEO and co-founder, Cohere

Why Google Cloud for LLM training?

As part of a multiyear technology partnership, Cohere leverages Google Cloud’s advanced AI and ML infrastructure to power its platform. Cohere develops and deploys its products on Cloud TPUs, Google Cloud’s custom-designed machine learning chips that are optimized for large-scale ML. Cohere’s recently announced their new model improvements and scalability by training an LLM using FAX on Google Cloud TPUs, and this model has demonstrated that transitioning from TPU v3 to TPU v4 has so far enabled them to achieve a total speedup of 1.7x. In addition to a significant performance boost, TPUs provide an excellent user experience with the new TPU VM architecture. Importantly, Google Cloud ensures that Cohere’s state-of-the-art ML training is achieved with the highest standards of sustainability, powered by 90% carbon-free energy in the world’s largest publicly available ML hub.

By adopting Cloud TPUs, Cohere is making LLM training faster, more economical, and more agile. This helps them provide larger and more accurate LLMs to developers, and put NLP technology in the hands of developers and businesses of all sizes.

To learn more about these LLM training advances, you can read the full paper, Scalable Training of Language Models using JAX pjit and TPUv4. To learn more about Cohere’s best practices and AI principles, you can check this article co-authored with Open AI and AI 21 Labs.