Google’s Record-breaking Performance Tops the MLPerf Benchmark Results
3115
Of your peers have already read this article.
4:00 Minutes
The most insightful time you'll spend today!
The latest round of MLPerf benchmark results have been released, and Google’s TPU v4 supercomputers demonstrated record-breaking performance at scale. This is a timely milestone since large-scale machine learning training has enabled many of the recent breakthroughs in AI, with the latest models encompassing billions or even trillions of parameters (T5, Meena, GShard, Switch Transformer, and GPT-3).
Google’s TPU v4 Pod was designed, in part, to meet these expansive training needs, and TPU v4 Pods set performance records in four of the six MLPerf benchmarks Google entered using TensorFlow and JAX. These scores are a significant improvement over our winning submission from last year and demonstrate that Google once again has the world’s fastest machine learning supercomputers. These TPU v4 Pods are already widely deployed throughout Google data centers for our internal machine learning workloads and will be available via Google Cloud later this year.
Figure 1: Speedup of Google’s best MLPerf Training v1.0 TPU v4 submission over the fastest non-Google submission in any availability category – in this case, all baseline submissions came from NVIDIA. Comparisons are normalized by overall training time regardless of system size. Taller bars are better.1
Let’s take a closer look at some of the innovations that delivered these ground-breaking results and what this means for large model training at Google and beyond.
Google’s continued performance leadership
Google’s submissions for the most recent MLPerf demonstrated leading top-line performance (fastest time to reach target quality), setting new performance records in four benchmarks. We achieved this by scaling up to 3,456 of our next-gen TPU v4 ASICs with hundreds of CPU hosts for the multiple benchmarks. We achieved an average of 1.7x improvement in our top-line submissions compared to last year’s results. This means we can now train some of the most common machine learning models in a matter of seconds.
Figure 2: Speedup of Google’s MLPerf Training v1.0 TPU v4 submission over Google’s MLPerf Training v0.7 TPU v3 submission (exception: DLRM results in MLPerf v0.7 were obtained using TPU v4). Comparisons are normalized by overall training time regardless of system size. Taller bars are better. Unet3D not shown since it is a new benchmark for MLPerf v1.0.2
We achieved these performance improvements through continued investment in both our hardware and software stacks. Part of the speedup comes from using Google’s fourth-generation TPU ASIC, which offers a significant boost in raw processing power over the previous generation, TPU v3. 4,096 of these TPU v4 chips are networked together to create a TPU v4 Pod, with each pod delivering 1.1 exaflop/s of peak performance.
Figure 3: A visual representation of 1 exaflop/s of computing power. If 10 million laptops were running simultaneously, then all that computing power would almost match the computing power of 1 exaflop/s.
In parallel, we introduced a number of new features into the XLA compiler to improve the performance of any ML model running on TPU v4. One of these features provides the ability to operate two (or potentially more) TPU cores as a single logical device using a shared uniform memory access system. This memory space unification allows the cores to easily share input and output data – allowing for a more performant allocation of work across cores. A second feature improves performance through a fine-grained overlap of compute and communication. Finally, we introduced a technique to automatically transform convolution operations such that space dimensions are converted into additional batch dimensions. This technique improves performance at the low batch sizes that are common at very large scales.
Enabling large model research using carbon-free energy
Though the margin of difference in topline MLPerf benchmarks can be measured in mere seconds, this can translate to many days worth of training time on the state-of-the-art models that comprise billions or trillions of parameters. To give an example, today we can train a 4 trillion parameter dense Transformer with GSPMD on 2048 TPU cores. For context, this is over 20 times larger than the GPT-3 model published by OpenAI last year. We are already using TPU v4 Pods extensively within Google to develop research breakthroughs such as MUM and LaMDA, and improve our core products such as Search, Assistant and Translate. The faster training times from TPUs result in efficiency savings and improved research and development velocity. Many of these TPU v4 Pods will be operating at or near 90% carbon free energy. Furthermore, cloud datacenters can be ~1.4-2X more energy efficient than typical datacenters, and the ML-oriented accelerators – like TPUs – running inside them can be ~2-5X more effective than off-the-shelf systems.
We are also excited to soon offer TPU v4 Pods on Google Cloud, making the world’s fastest machine learning training supercomputers available to customers around the world. Cloud TPUs support leading frameworks such as TensorFlow, PyTorch, and Jax, and we recently released an all-new Cloud TPU system architecture that provides direct access to TPU host machines, greatly improving the user experience.
Want to learn more?
Please contact your Google Cloud sales representative to request early access to Cloud TPU v4 Pods. We are excited to see how you will expand the machine learning frontier with access to exaflops of TPU computing power!
1. All results retrieved from www.mlperf.org on June 30, 2021. MLPerf name and logo are trademarks. See www.mlperf.org for more information. Chart uses results 1.0-1067, 1.0-1070, 1.0-1071, 1.0-1072, 1.0-1073, 1.0-1074, 1.0-1075, 1.0-1076, 1.0-1077, 1.0-1088, 1.0-1089, 1.0-1090, 1.0-1091, 1.0-1092.
2. All results retrieved from www.mlperf.org on June 30, 2021. MLPerf name and logo are trademarks. See www.mlperf.org for more information. Chart uses results 0.7-65, 0.7-66, 0.7-67, 1.0-1088, 1.0-1090, 1.0-1091, 1.0-1092.
4540
Of your peers have already watched this video.
31:00 Minutes
The most insightful time you'll spend today!
How Google Secures its Data Centers: Watch Video
Security is in the DNA of Google Cloud’s dozens of data centers, complex network and workloads scattered the globe. Take a tour to the nucleus of data center’s six layers of physical security designed to keep unauthorized access at bay, and also learn about Google Cloud’s security fundamentals to leverage the same philosophy on Google Cloud. Watch now!
Guide to Create and Manage Datasets with Vertex AI
3050
Of your peers have already read this article.
2:00 Minutes
The most insightful time you'll spend today!
At Google I/O this year, we introduced Vertex AI to bring together all our ML offerings into a single environment that lets you build and manage the lifecycle of ML projects. In a previous post, we gave you an overview of Vertex AI, sharing how it supports your entire ML workflow—from data management all the way to predictions. Today, we’ll talk a little about how to manage ML datasets with Vertex AI.
Many enterprises want to use data to make meaningful predictions that can bolster their business or help them venture into new markets. This often requires using custom machine learning models—something not every business knows how to create or use. This is where Vertex AI can help. Vertex AI provides tools for every step of the machine learning workflow—from managing data sets to different ways of training the model, evaluating, deploying, and making predictions. It also supports varying levels of ML expertise, so you don’t need to be an ML expert to use Vertex AI.https://www.youtube.com/embed/CN2X6oIlnmI?enablejsapi=1&
Types of data you can use in Vertex AI
Datasets are the first step of the machine learning lifecycle—to get started you need data, and lots of it. Vertex AI currently supports managed datasets for four data types—image, tabular, text, and videos.
Image
Image datasets let you do:
- Image classification—Identifying items within an image.
- Object detection—Identifying the location of an item in an image
- Image segmentation—Assigning labels to pixel level regions in an image.
To ensure your model performs well in production, use training images similar to what your users will send. For example, if users are likely to send low quality images, be sure to have blurry and low resolution images in your data set. Don’t forget to include different angles, backgrounds, and resolutions. We recommend you include at least 1,000 images per label (item you want to identify), but you can always get started with 10 per label. The more examples you provide, the better your model will be.
Tabular
Tabular datasets enable you to do:
- Regression—Predicting a numerical value.
- Classification—Predicting a category associated with a particular example.
- Forecasting—Predicting the likelihood of sudden events or demands.
Tabular data sets support hundreds of columns and millions of rows.
Text
With text datasets, you can do:
- Classification—Assigning one or more labels to an entire document.
- Entity extraction—Identifying custom text entities within a document, like “too expensive” or “great value”.
- Sentiment analysis—Identifying the overall sentiment expressed in a block of text, for example, if a customer was happy or upset or frustrated.
Video
Video datasets enable:
- Classification—Labeling entire videos, shots, or frames.
- Action recognition—Identifying clips video clips where specific actions occur.
- Object tracking—Tracking specific objects in a video.
Creating and managing datasets in Vertex AI
Now that we’ve covered the different types of data you can use, let’s shift to creating and managing those datasets. In the Cloud Console, go to Vertex AI dashboard page and click Datasets, then click Create Project.
Say you want to classify items within a set of photos. Create an image dataset and select image classification. You can import files directly from your computer, which will be stored in Cloud Storage. Then, you’ll need to add the corresponding labels (items you want to identify) for your images. If you already have labels, you can use the Import File option to import a CSV with your image URLs and their labels. If your data is not labeled and you would like human help to label it, you can use the Vertex AI data labeling service. Once the files are uploaded, you can create labels and assign them to the images. You can also analyze the images in the data set, the number of images per label, and a few other properties.
Depending on the type of data you use, your options might vary slightly. For example, if you want to use tabular data, you could upload a CSV file from your computer, use one from Cloud Storage, or select a table from BigQuery directly. Once you select the table, the data is available for analysis.
More to come
This concludes our overview of creating and managing datasets in Vertex AI. In a future installment, we’ll go over the next phase of the machine learning workflow: building and training ML models.
If you enjoyed this post, keep an eye out for more AI Simplified episodes on YouTube. In the meantime, here’s where you can learn more about Vertex AI.
Google is a Leader in the 2023 Gartner® Magic Quadrant™ for Enterprise Conversational AI Platforms
1291
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
We’re excited to share that Gartner has recognized Google as a Leader in the 2023 Gartner® Magic Quadrant™ for Enterprise Conversational AI Platforms, authored by Bern Elliot and Gabriele Rigon.
We believe this recognition is a testament to Google Cloud’s robust investments and commitment to innovation in AI, coupled with a deep understanding of enterprise customer needs. Enterprises are increasingly investing in AI-driven solutions that balance addressing customer expectations with operational efficiency. At a time when the demand for quality, performant, and trustworthy conversational AI has never been higher, we’re thrilled to continue to deliver best-in-class technologies, purpose-built to solve our customers’ most critical use cases.
In 2022, Google Cloud delivered cutting-edge conversational AI technologies with many launches to our Conversational AI API portfolio. Together, these enabled developers to leverage Google’s technologies to power their applications with our end-to-end Contact Center AI (CCAI) suite, designed to solve the needs of Customer Experience (CX) and contact center leaders.
Google Cloud’s Conversational AI APIs include pre-trained models for Speech to Text, Text to Speech and Natural Language Understanding. This conversational AI core leverages Google Research’s technology for speech, understanding and interaction, enabling and orchestrating high-quality conversational experiences at scale.
With Contact Center AI, organizations see improved customer satisfaction, higher agent productivity and reduced costs through increased agent efficiency. By focusing on user needs, Google Cloud provides comprehensive and integrated solutions that are ready for the enterprise. CCAI encompasses a comprehensive set of offerings to address the needs of the contact center.
In 2022, we launched Contact Center AI Platform, our AI-first, mobile-first, user-first contact center as a service (CCaaS), providing AI-powered experiences, CRM-centered design and deployment flexibility in a single platform without the need for multiple providers. Contact Center AI Platform auto-scales on the backend, with capacity for up to 100k concurrent users on a single tenant. It also offers multi-provider voice resiliency with global low-latency routing for best possible call quality and is optimized for enhanced customer/business data security, reduced downtime, and increased agent productivity. Segra, one of the largest independent fiber infrastructure bandwidth companies in the Eastern U.S., is leveraging CCAI Platform to reimagine their customer experience through predictive flows for common experiences and greatly expanding their channels for customer interaction.
CCAI also includes Dialogflow for building virtual agents, enabling businesses to meet their customers across multiple channels. It offers robust, flexible self-service voice and chat interactions that are just as natural as a live agent. Dialogflow enables both a great customer experience and a cost-effective way to scale services.
Our Agent Assist service gives businesses the ability to transition a call from a virtual agent to a human agent while maintaining context. It efficiently guides the agent to an accurate response, while providing real-time suggestions, more accurate responses and informed recommendations.
To improve contact center operations, CCAI Insights analyzes all customer conversations to provide leaders with real-time, actionable data points on customer queries, agent performance, and sentiment trends. Its topic modeling capabilities enable deeper understanding of key investment areas and greater classification accuracy.
To ensure our enterprise customers deploying CCAI realize value faster, Google Cloud offers CCAI through three defined transformation stages with out-of-the-box packages. The first stage starts with efficiency basics in the first week that include transcription and summarization. The second stage covers automation basics within six months using Agent Assist and Insights. The final stage is full automation within a year with industry use cases and pre-built components. All of this leads to higher agent efficiency, improved customer satisfaction and increased containment.
As we look forward to the rest of 2023 and beyond, elevating the customer experience through user-first design, AI-first capabilities and accelerating time-to-value will be our north star. We plan to announce exciting new capabilities over the next few months to enable that vision to become a reality for many more organizations.
We are honored to be a Leader in the 2023 Gartner® Magic Quadrant™ for Enterprise Conversational AI Platforms, and look forward to continuing to innovate and partner with customers on their digital transformation journeys.
Download the complimentary copy of the report: 2023 Gartner® Magic Quadrant™ for Enterprise Conversational AI Platforms.
Learn more about how organizations are transforming their business with Google Cloud solutions with Contact Center AI.
GARTNER is a registered trademark and service mark of Gartner and Magic Quadrant is a registered trademark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and are used herein with permission. All rights reserved.
This graphic was published by Gartner, Inc. as part of a larger research document and should be evaluated in the context of the entire document. The Gartner document is available upon request from Google.
Gartner does not endorse any vendor, product or service depicted in its research publications and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.
5581
Of your peers have already watched this video.
1:20 Minutes
The most insightful time you'll spend today!
Nippon India Mutual Fund Re-Invents How Indians Buy with AI
Nippon India Mutual Fund, formerly Reliance Mutual Funds is changing the way Indians purchase funds making it easier and faster, with the help of Google Cloud.
In India, only 3-4 percent of the population has invested in mutual funds. There’s a sizeable market to tap into for mutual fund houses–if they can find ways to make it easier for first-time investors to take the plunge.
As the leading retail asset management company in the country, Reliance Mutual Funds, decided to use voice to facilitate transactions.
“That would create a delightful experience for the investor,” says Arpan Saha, Head of Digital Business, Nippon India Mutual Fund.
That’s exactly what the company did using the Google Cloud Platform.
Today, the company has over 10,000 interactions using the Google AI Platform.
“Today we see more consumer coming and doing more transactions with us, and we only see this going up as we make this experience razor-sharp,” says Saha.
AI Features in Apigee X Helps Build and Manage APIs at Scale
5261
Of your peers have already read this article.
2:00 Minutes
The most insightful time you'll spend today!
APIs are the backbone of digital transformation. Via APIs, you can securely share data and functionality with developers both inside and outside of your organizational boundaries, letting you build applications faster, seamlessly connect and interact with partners, and drive new business revenue.
Because APIs encompass business-critical information, any downtime or performance degradation can lead to significant loss in revenue, customers, and brand value. Therefore, there’s mounting pressure on operations teams to ensure that APIs are always available and performing as expected. If the APIs go down, so too do the services that fuel customer experiences and on which the organization relies for collaboration and business processes.
However, as you build and scale your API programs, it becomes practically impossible for API operators to manually monitor and manage all your APIs. To help, we brought the power of industry-leading AI and ML technologies to API operations via Apigee X, a major release of our API management platform. Apigee X seamlessly weaves together Google Cloud’s expertise in AI, security and networking to help you efficiently build and manage APIs at scale.
Put your API data into action
Apigee applies machine learning to your API metadata and provides you the required tools that simplify various aspects of API operations. A great example of AI for APIs is anomaly detection:
- AI-powered rules trigger alerts based on a set of predefined conditions that are determined by applying Google’s industry-leading machine learning models to your historical API data.
- Auto-thresholds adjust the monitoring criteria of your APIs and set them to pattern-based values.
- Reduce overhead results because operators don’t have to manually monitor anomalies or adjust the monitoring thresholds on APIs.
“By applying AI and ML models to our historical API data, these advanced features are able to alert us about scenarios we haven’t thought of. Such automation capabilities significantly reduce our upfront efforts. And from a security perspective, the actionable insights help us ensure that our proxies are exposed only over secure HTTPs ports and adhere to compliance requirements. We’re also able to closely monitor user activity and quickly pull out reports during audits.” – Adam Brancato, Sr. Manager, Global Technology and Security at Citrix
As our customers scale their API programs, they find it extremely useful to harness AI-powered capabilities. In our recent State of the API Economy 2021 report, we found a 230% increase in enterprises’ use of anomaly detection, bot protection, and security analytics features.
To learn more about Apigee X, and see AI and machine learning in action, check out this video, and to try Apigee X for free, click here.
More Relevant Stories for Your Company
Leading Verve Group’s CX Innovation with Google Cloud Vertex AI
Verve Group is an ecosystem of demand and supply technologies fusing data, media, and technology to deliver results and growth to both advertisers and publishers – no matter the screen or location, no matter who, what, or where a customer is. Classifying massive amounts of this unstructured data at scale is
Time-series Model on Google Cloud Allows Better Transparency on Fishing and Marine Activities
Who would have known that today technology would enable us with the ability to use machine learning to track vessel activity, and make pattern inferences to help address IUU (illegal, unreported, and unregulated) fishing activities. What’s even more noteworthy is that we now have the computing power to share this
Supercharging Security with Generative AI
At Google Cloud, we continue to invest in key technologies to progress towards our true north star on invisible security: making strong security pervasive and simple for everyone. Our investments are based on insights from our world-class threat intelligence teams and experience helping customers respond to the most sophisticated cyberattacks. Customers can tap into
Supercharge Marketing with Ready-Made, Centralized Smart Analytics
Your company has lots of valuable marketing data, but it’s spread across many systems and teams. That's a problem. In fact, only 13% of organisations feel like they're making the most out of their available customer data today. And the reason is that data--marketing and customer data--is spread across many
