Google Submits Two Large Language Model Benchmarks into the Open Division
2922
Of your peers have already read this article.
3:00 Minutes
The most insightful time you'll spend today!
Recently, models with billions or trillions of parameters have shown significant advances in machine learning capabilities and accuracy. For example, Google’s LaMDA model is able to engage in a free-flowing conversation with users about a large variety of topics. There is enormous interest within the machine learning research and product communities in leveraging large models to deliver breakthrough capabilities. The high computational demand of these large models requires an increased focus on improving the efficiency of the model training process, and benchmarking is an important means to coalesce the ML systems community towards realizing higher efficiencies.
In the recently concluded MLPerf v1.1 Training round1, Google submitted two large language model benchmarks into the Open division, one with 480 billion parameters and a second with 200 billion parameters. These submissions make use of publicly available infrastructure, including Cloud TPU v4 Pod slices and the Lingvo open source modeling framework.
Traditionally, training models at these scales would require building a supercomputer at a cost of tens or even hundreds of millions of dollars – something only a few companies can afford to do. Customers can achieve the same results using exaflop-scale Cloud TPU v4 Pods without incurring the costs of installing and maintaining an on-premise system.
Large model benchmarks
Google’s Open division submissions consist of a 480 billion parameter dense Transformer-based encoder-only benchmark using TensorFlow and a 200 billion-parameter JAX benchmark. These models are architecturally similar to MLPerf’s BERT model but with larger dimensions and number of layers. These submissions demonstrate large model scalability and high performance on TPUs across two distinct frameworks. Notably, these benchmarks, with their stacked transformer architecture, are fairly comparable in terms of their compute characteristics with other large language models.
Our two submissions were benchmarked on 2048-chip and 1024-chip TPU v4 Pod slices, respectively. We were able to achieve an end-to-end training time of ~55 hours for the 480B parameter model and ~40 hours for the 200B parameter model. Each of these runs achieved a computational efficiency of 63%- calculated as a fraction of floating point operations of the model together with compiler rematerialization over the peak FLOPs of the system used2.
Next-generation ML infrastructure for large Model training
Achieving these impressive results required a combination of several cutting edge technologies. First, each TPU v4 chip provides more than 2X the compute power of a TPU v3 chip – up to 275 peak TFLOPS. Second, 4,096 TPU v4 chips are networked together into a Cloud TPU v4 Pod by an ultra-fast interconnect that provides 10x the bandwidth per chip at scale compared to typical GPU-based large scale training systems. Large models are very communication intensive: local computation often depends on results from remote computation that are communicated across the network. TPU v4’s ultra-fast interconnect has an outsized impact on computational efficiency of large models by eliminating latency and congestion in the network.
The performance numbers demonstrated by our submission also rely on our XLA linear algebra compiler and leverage the Lingvo framework. XLA transparently performs a number of optimizations, including GSPMD based automatic parallelization of many of the computation graphs that form the building blocks of the ML model. XLA also allows for reduction in latency by overlapping communication with the computations. Our two submissions demonstrate the versatility and performance of our software stack across two frameworks, TensorFlow and JAX.
Large models in MLPerf
Google’s submissions represent an important class of models that have become increasingly important in ML research and production, but are currently not represented in MLPerf’s Closed division benchmark suite.
We believe that adding these models to the benchmark suite is an important next step and can inspire the ML systems community to focus on addressing the scalability challenges that large models present.
Our submissions demonstrate 63% computational efficiency, cutting edge in the industry. This high computational efficiency enables higher experimentation velocity through faster training. This directly translates into cost savings for Google’s Cloud TPU customers.
Please visit the Cloud TPU homepage and documentation to learn more about leveraging Cloud TPUs using TensorFlow, PyTorch, and JAX.
1. The MLPerf name and logo are trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use is strictly prohibited. See www.mlcommons.org for more information.
2. Computational efficiency and end-to-end training time are not official MLPerf metrics
2977
Of your peers have already watched this video.
9:00 Minutes
The most insightful time you'll spend today!
Satellites Can Help Map Carbon Emissions By Looking at Images of Power Plants!
Did you know that Satellites can now help track power plants and determine if they are on or off? And did also know that compute processing that classifies over 59 trillion bytes of data from over 11,000 sensors from 300 satellites is done on on Google Cloud?
Google Cloud’s sustainability initiatives along with its cutting-edge, powerful technology and products are leveraged worldwide by organizations, non-profits and governments to keep a check on greenhouse gas emissions and build self-reporting, power monitoring platforms. Climate TRACE, a collaborative data sharing project with 50+ organizations is dedicated towards this cause by building a visual and meaningful way on a web app. In this video you can learn how generate geo-spatial model that can view the images of power plants gathered from satellite called Sentinel-2 to assess emissions from power plants. Watch the video to learn Google Cloud’s vehement role in funding and staffing special initiatives with Googlers who are experts in AI/ML, UX, data analytics and more!
6365
Of your peers have already watched this video.
2:00 Minutes
The most insightful time you'll spend today!
Google Cloud’s ML-based Image Classification App: A Key to Global Wildlife Conservation
Wildlife provides critical benefits to support nature and people. Unfortunately, wildlife is slowly but surely disappearing from our planet and we lack reliable and up-to-date information to understand and prevent this loss. By harnessing the power of technology and science, we can unite millions of photos from [motion sensored cameras] around the world and reveal how wildlife is faring, in near real-time…and make better decisions
wildlifeinsights.org/about
ML Models Built on Google Cloud Solutions Help You Virtually Participate in National Muffin Day!
2964
Of your peers have already read this article.
2:00 Minutes
The most insightful time you'll spend today!
If you’re here you’re probably wondering: what on Earth is the connection between muffins and machine learning, and what is National Muffin Day? To understand this, let’s start with National Muffin Day: an annual holiday co-founded by Jacob and his friend Julia Levy in 2015 to bake muffins and raise money for homelessness. National Muffin Day will occur on Sunday, February 20 this year. For more information on how to participate in National Muffin Day (which involves delicious baked goods and donations to people in need), please see the information at the bottom of this post. Last year, a colleague connected Jacob with Sara, who had done several baking projects that used machine learning to generate new recipes. They decided to collaborate for this year’s National Muffin Day, adding a new muffin recipe created with the help of machine learning.
In this post, we’ll explain how Sara used Google Cloud to develop a new muffin recipe, show you how you can participate virtually in National Muffin Day, and of course—share the recipe.
Machine learning for muffins
At its core, machine learning is the process of finding patterns in data and using those patterns to make predictions on new data. After a lot of baking over the past few years, Sara learned that baking is also based on patterns. For example, the ratio of flour, fat, liquid, and sugar that make up a cookie is very different from the ratio of those ingredients for a bread, a pie crust, or a muffin. She used that discovery to create a recipe for a hybrid cake + cookie, and a cake filled with Maltesers. Next up: muffins!
The first step was figuring out how to translate the task of generating a new muffin recipe into a machine learning task. To solve this, she planned to use numerical data on the amounts of different ingredients in a muffin recipe to train the model. Sara considered two types of models for this task: classification and regression. A classification model would categorize muffin recipes into different muffin types based on their ingredient amounts, and a regression model would do the reverse: take a type of muffin and return the amount of each ingredient needed to make it. She decided to build a regression model, since it would be more fun for the model to return ingredient amounts, rather than tell you which type of muffin recipe you’re already making.
Implementing this first required identifying a few muffin categories and collecting recipe data. This presented a new challenge, since her previous baking models used categories for distinct baked goods (i.e. cakes, cookies, breads). After scouring through quite a few recipes, Sara discovered that many muffins fall into two types: those that use only traditional ingredients as their base (flour, sugar, butter, milk, etc.), and those that include an alternative ingredient, most commonly a pureed fruit, to make the base (like bananas, applesauce, or pumpkin). Using those two categories, the model would take the type of muffin as input and return the amounts of base ingredients required to make that recipe. Here, the inputs can be any values adding up to 100%:
The next step in the ML process was collecting recipe data to use for model training and narrowing down the ingredients used to train the model. Sara wanted the model to learn the combination of core ingredients that make up a muffin batter, rather than flavorings and additions like blueberries, vanilla extract, or chocolate chips. These tasty additions could be added after the model helped create the muffin batter. Once she gathered enough recipes, she removed extra ingredients for training purposes and converted ingredients from different recipes into the same unit (grams, milliliters, and teaspoons).
Building a muffin model with Vertex AI
Sara uploaded the muffin ingredient data into BigQuery, and then created a notebook instance in Vertex AI Workbench to analyze the data. With the new Workbench managed instances, you can interactively query BigQuery tables directly from your instance and copy the code to download your data to a notebook as a Pandas DataFrame:
From her notebook instance, Sara experimented with different ML frameworks and model types. She landed on a Scikit-learn regression model to solve this task, and to mimic a real-world production environment, decided to convert this workflow into a ML pipeline. Using the Kubeflow Pipelines SDK, she ran the following on Vertex Pipelines:
The first component reads the ingredient data from BigQuery and converts it into a Pandas DataFrame which is passed to the next pipeline step. In this step, we train a custom Scikit-learn model on the recipe data. Finally, this model is deployed to an endpoint in Vertex AI. To put it all together, Sara built a web app that allowed her to easily generate ingredient amounts for different muffin types. The web app uses the Vertex AI SDK to call the deployed model endpoint and return ingredient amounts.
The recipe
With a deployed recipe generation model, the only thing left to do was test recipes in the kitchen! Because the model only returns ingredient amounts, there were still many key human elements to complete the baking process: adding yummy additions to the core muffin batter, making adjustments to optimize taste, determining the method for adding ingredients, baking time, and more. After testing a few recipes generated by the model, we landed on a favorite which we’re very excited to share with you here.
Berry ML Muffins
Makes 12 muffins
Flour 285 grams (2 cups)
Granulated sugar 250 grams (1 cup)
Baking powder 2 teaspoons
Baking soda ¼ teaspoon
Salt ½ teaspoon
Cinnamon ½ teaspoon
Milk 170 ml (⅔ cup), room temperature
Butter 55 grams (¼ cup), melted and slightly cooled
Eggs 1 egg plus 1 egg white, room temperature
Canola or vegetable oil 50 grams (¼ cup)
Sour cream 50 grams (3 tablespoons + ¾ teaspoon), room temperature
Vanilla extract 1 ½ teaspoons
Blueberries or raspberries 240 grams (1 ½ cups)
Coarse sugar, like demerara or turbinado (optional for topping) 1 tablespoon
- Measure your three cold ingredients and allow them to come to room temperature: 1 egg + 1 egg white, sour cream, and milk.
- Preheat the oven to 375 F / 190 C. Line a 12-muffin tin with cupcake liners or lightly grease with baking spray.
- In a large bowl, whisk together flour, baking powder, baking soda, salt, and cinnamon. Set aside.
- Melt your butter in a medium heat proof bowl, and allow it to cool slightly for a few minutes. Whisk in sugar until combined. Then add egg, oil, and vanilla, milk, and sour cream and whisk until fully incorporated.
- Pour the wet ingredients into the dry ingredients, mixing with a spatula until just combined. Be careful not to overmix, it’s ok if there are a few lumps in your batter.
- Prepare your fruit. If you can’t decide whether to use blueberries or raspberries, divide your batter into two bowls and do both! Crush half of your fruit and fold it into the batter. Then mix in the remaining whole berries.
- Divide the mixture evenly into the muffin tin. Optionally (but extra tasty), sprinkle the tops of each muffin with about ⅛ teaspoon of coarse sugar. Turbinado or demerara sugar work well for this. This will caramelize and add a nice texture to the tops of your muffins.
- Bake at 375 for 22 – 24 minutes, or until a toothpick inserted in the center comes out clean. For best results, do a toothpick test in a few muffins since not all ovens have an even temperature throughout. Let the muffins cool in the muffin tin for a few minutes, then transfer to a wire rack to cool completely.
- Enjoy!
How can you participate in National Muffin Day?
Participation in National Muffin Day is as easy as 1-2-3!
- On February 20, Bake Muffins. It’s time to dust those muffin tins, grab your blueberries, chocolate chips, rhubarb, and favorite ingredients, and create some magical scrumdiddlyumptiousness! If you want to join Jacob in a virtual baking party, you can register here.
- Then, Give. In non-pandemic years, we asked our bakers to personally hand muffins to hungry folks in their cities. While this is a valuable and rewarding experience, the current state of Covid means this practice is still unsafe, so we request that you refrain from doing this. Instead, if it feels safe, we encourage you to take your delicious baked goods and donate them to local homeless shelters, which can distribute them to those in need. Alternatively, you can share your muffins with friends and families and then make a donation to an organization that benefits people experiencing homelessness, like the ones listed below in step 3.
- Share Your Muffin Pics on Social Media. We’d love to see your muffins! Share your pictures on Twitter or Instagram with the hashtags #givemuffins, or share them to our official Facebook Event page. For each individual baker who participates, we will make donations to Project Homeless Connect, which provides much needed resources to people experiencing homelessness in San Francisco, Family promise, which supports unhoused families nationwide, and Pine Street Inn which provides resources for people experiencing homelessness in Boston. Donations will be on a per-baker basis (with up to $80 donated per baker!), so please feel free to loop in your significant others, kids, nieces and nephews, roommates, friends, and anybody else with a giving spirit who loves deliciousness!
Google’s Record-breaking Performance Tops the MLPerf Benchmark Results
3130
Of your peers have already read this article.
4:00 Minutes
The most insightful time you'll spend today!
The latest round of MLPerf benchmark results have been released, and Google’s TPU v4 supercomputers demonstrated record-breaking performance at scale. This is a timely milestone since large-scale machine learning training has enabled many of the recent breakthroughs in AI, with the latest models encompassing billions or even trillions of parameters (T5, Meena, GShard, Switch Transformer, and GPT-3).
Google’s TPU v4 Pod was designed, in part, to meet these expansive training needs, and TPU v4 Pods set performance records in four of the six MLPerf benchmarks Google entered using TensorFlow and JAX. These scores are a significant improvement over our winning submission from last year and demonstrate that Google once again has the world’s fastest machine learning supercomputers. These TPU v4 Pods are already widely deployed throughout Google data centers for our internal machine learning workloads and will be available via Google Cloud later this year.
Figure 1: Speedup of Google’s best MLPerf Training v1.0 TPU v4 submission over the fastest non-Google submission in any availability category – in this case, all baseline submissions came from NVIDIA. Comparisons are normalized by overall training time regardless of system size. Taller bars are better.1
Let’s take a closer look at some of the innovations that delivered these ground-breaking results and what this means for large model training at Google and beyond.
Google’s continued performance leadership
Google’s submissions for the most recent MLPerf demonstrated leading top-line performance (fastest time to reach target quality), setting new performance records in four benchmarks. We achieved this by scaling up to 3,456 of our next-gen TPU v4 ASICs with hundreds of CPU hosts for the multiple benchmarks. We achieved an average of 1.7x improvement in our top-line submissions compared to last year’s results. This means we can now train some of the most common machine learning models in a matter of seconds.
Figure 2: Speedup of Google’s MLPerf Training v1.0 TPU v4 submission over Google’s MLPerf Training v0.7 TPU v3 submission (exception: DLRM results in MLPerf v0.7 were obtained using TPU v4). Comparisons are normalized by overall training time regardless of system size. Taller bars are better. Unet3D not shown since it is a new benchmark for MLPerf v1.0.2
We achieved these performance improvements through continued investment in both our hardware and software stacks. Part of the speedup comes from using Google’s fourth-generation TPU ASIC, which offers a significant boost in raw processing power over the previous generation, TPU v3. 4,096 of these TPU v4 chips are networked together to create a TPU v4 Pod, with each pod delivering 1.1 exaflop/s of peak performance.
Figure 3: A visual representation of 1 exaflop/s of computing power. If 10 million laptops were running simultaneously, then all that computing power would almost match the computing power of 1 exaflop/s.
In parallel, we introduced a number of new features into the XLA compiler to improve the performance of any ML model running on TPU v4. One of these features provides the ability to operate two (or potentially more) TPU cores as a single logical device using a shared uniform memory access system. This memory space unification allows the cores to easily share input and output data – allowing for a more performant allocation of work across cores. A second feature improves performance through a fine-grained overlap of compute and communication. Finally, we introduced a technique to automatically transform convolution operations such that space dimensions are converted into additional batch dimensions. This technique improves performance at the low batch sizes that are common at very large scales.
Enabling large model research using carbon-free energy
Though the margin of difference in topline MLPerf benchmarks can be measured in mere seconds, this can translate to many days worth of training time on the state-of-the-art models that comprise billions or trillions of parameters. To give an example, today we can train a 4 trillion parameter dense Transformer with GSPMD on 2048 TPU cores. For context, this is over 20 times larger than the GPT-3 model published by OpenAI last year. We are already using TPU v4 Pods extensively within Google to develop research breakthroughs such as MUM and LaMDA, and improve our core products such as Search, Assistant and Translate. The faster training times from TPUs result in efficiency savings and improved research and development velocity. Many of these TPU v4 Pods will be operating at or near 90% carbon free energy. Furthermore, cloud datacenters can be ~1.4-2X more energy efficient than typical datacenters, and the ML-oriented accelerators – like TPUs – running inside them can be ~2-5X more effective than off-the-shelf systems.
We are also excited to soon offer TPU v4 Pods on Google Cloud, making the world’s fastest machine learning training supercomputers available to customers around the world. Cloud TPUs support leading frameworks such as TensorFlow, PyTorch, and Jax, and we recently released an all-new Cloud TPU system architecture that provides direct access to TPU host machines, greatly improving the user experience.
Want to learn more?
Please contact your Google Cloud sales representative to request early access to Cloud TPU v4 Pods. We are excited to see how you will expand the machine learning frontier with access to exaflops of TPU computing power!
1. All results retrieved from www.mlperf.org on June 30, 2021. MLPerf name and logo are trademarks. See www.mlperf.org for more information. Chart uses results 1.0-1067, 1.0-1070, 1.0-1071, 1.0-1072, 1.0-1073, 1.0-1074, 1.0-1075, 1.0-1076, 1.0-1077, 1.0-1088, 1.0-1089, 1.0-1090, 1.0-1091, 1.0-1092.
2. All results retrieved from www.mlperf.org on June 30, 2021. MLPerf name and logo are trademarks. See www.mlperf.org for more information. Chart uses results 0.7-65, 0.7-66, 0.7-67, 1.0-1088, 1.0-1090, 1.0-1091, 1.0-1092.
Want to Code for the Cloud? Get Started with the Native App Development Track
6763
Of your peers have already read this article.
2:00 Minutes
The most insightful time you'll spend today!
Earlier this year, we launched the Google Cloud skills challenge, which provides 30 days of free access to training to build your cloud knowledge and an opportunity to earn skill badges that showcase your Google Cloud competencies. Today, we’re adding a Native App Development track to the skills challenge, joining the Getting Started, Data Analytics, Kubernetes, Machine Learning (ML) and Artificial Intelligence (AI) tracks.
The Native App Development track is designed for cloud developers who want to learn to build serverless web apps and Google Assistant applications on Google Cloud using Cloud Run and Firebase. Specifically, you’ll have an opportunity to earn three skill badges in the Native App Dev track: Serverless Firebase Development, Serverless Cloud Run Development, and Build Interactive Apps with Google Assistant. To earn a skill badge, you complete a series of hands-on labs and take a final assessment challenge lab to test your skills.
Here’s an overview of each badge.
Serverless Firebase Development
To earn this skill badge, you’ll learn how to build serverless web apps, import data into a serverless database, and build Google Assistant applications using Firebase, Google’s backend-as-service platform for creating mobile and web applications.
Serverless Cloud Run Development
For this badge, you’ll discover how to use Cloud Run, a fully managed serverless platform, to connect and leverage data stored in Cloud Storage. You’ll learn how to use Cloud Run to build a resilient, asynchronous system with Pub/Sub, build a REST API gateway as well as build and expose services.
Build Interactive Apps with Google Assistant
To earn the final skills badge, you’ll build Google Assistant applications by creating a project in the Actions console, integrating Dialogflow, testing your action in the Actions simulator, and adding Cloud Translation API to your assistant application.
Ready to jump into the skills challenge? Sign up here.
You can also check out this quick video below to learn how to join the skills challenge.
More Relevant Stories for Your Company
Streamline Your Business Processes with Google Cloud’s Custom Document Splitter
Businesses rely on processing an inflow of documents to drive processes and make decisions. Many such documents are combined into a single file. For example, a loan application may have a driver’s license, paystub, W2, bank statement, and other document types within a single file. The complexity of handling many
Making Hybrid Work Human: Google Workspace and Economist Impact Survey
Google Workspace recently commissioned Economist Impact to complete a global survey (October 2021)* on the state of hybrid work, including its challenges and opportunities. We already knew that the pandemic had fundamentally changed the world of work, but the survey emphasizes the scale, reach, and longevity of those changes.. Over 75% of
This Diagnostic Company is Revolutionising Healthcare Delivery with AI
Dr. Elliot Smith cannot be accused of lacking ambition. A high achiever with a Ph.D. in Electrical Engineering and a specialist in magnetic resonance imaging (MRI) systems, Smith aims to deliver top quality healthcare to anyone in the world -- regardless of their location or wealth. Dr. Smith has already
Google Unveils New Cloud Region in Delhi NCR to Power India’s Digitization
In the past year, Google has worked to surface timely and reliable health information, amplify public health campaigns, and help nonprofits get urgent support to Indians in need. Now, we are continuing to focus on helping India’s businesses accelerate their digital transformation, deepening our commitment to India’s digitization and economic recovery.
