Today, ML-driven innovation is fundamentally transforming computing, enabling entirely new classes of internet services. For example, recent state-of-the-art lage models such as PaLM and Chinchilla herald a coming paradigm shift where ML services will augment human creativity. All indications are that we are still in the early stages of what will be the next qualitative step function in computing. Realizing this transformation will require democratized and affordable access through cloud computing where the best of compute, networking, storage, and ML can be brought to bear seamlessly on ever larger-scale problem domains.

Today’s release of MLPerf™ 2.0 results from the MLCommons® Association highlights the public availability of the most powerful and efficient ML infrastructure anywhere. Google’s TPU v4 ML supercomputers set performance records on five benchmarks, with an average speedup of 1.42x over the next fastest non-Google submission, and 1.5x vs our MLPerf 1.0 submission. Even more compelling — four of these record runs were conducted on the publicly available Google Cloud ML hub that we announced at Google I/O. ML Hub runs out of our Oklahoma data center, which uses over 90% carbon-free energy.

Let’s take a closer look at the results.

Performance at scale…and in the public cloud

Our 2.0 submissions1, all running on TensorFlow, demonstrated leading performance across all five benchmarks. We scaled two of our submissions to run on full TPU v4 Pods. Each Cloud TPU v4 Pod consists of 4096 chips connected together via an ultra-fast interconnect network with an industry-leading 6 terabits per second (Tbps) of bandwidth per host, enabling rapid training for the largest models.

Hardware aside, these benchmark results were made possible in no small part by our work to improve the TPU software stack. Scalability and performance optimizations in the TPU compiler and runtime, including faster embedding lookups and improved model weight distribution across the TPU pod, enabled much of these improvements, and are now widely available to TPU users. For example, we made a number of performance improvements to the virtualization stack to fully utilize the compute power of both CPU hosts and TPU chips to achieve peak performance on image and recommendation models. These optimizations reflect lessons from Google’s cutting-edge internal ML use cases across Search, YouTube, and more. We are excited to bring the benefits of this work to all Google Cloud users as well.

Translating MLPerf wins to customer wins

Cloud TPU’s industry-leading performance at scale also translates to cost savings for customers. Based on our analysis summarized in Figure 3, Cloud TPUs on Google Cloud provide ~35-50% savings vs A100 on Microsoft Azure (see Figure 3). We employed the following methodology to calculate this result:2

We compared the end-to-end times of the largest-scale MLPerf submissions, namely ResNet and BERT, from Google and NVIDIA. These submissions make use of a similar number of chips — upwards of 4000 TPU and GPU chips. Since performance does not scale linearly with chip count, we compared two submissions with roughly the same number of chips.

To simplify the 4216-chip A100 comparison for ResNet vs our 4096-chip TPU submission, we made an assumption in favor of GPUs that 4096 A100 chips would deliver the same performance as 4216 chips.

For pricing, we compared our publicly available Cloud TPU v4 on-demand prices ($3.22 per chip-hour) to Azure’s on-demand prices for A1003 ($4.1 per chip-hour). This once again favors the A100s since we assume zero virtualization overhead in moving from on-prem (NVIDIA’s results) to Azure Cloud.

The savings are especially meaningful given that real-world models such as GPT-3 and PaLM are much larger than the BERT and ResNet models used in the MLPerf benchmark: PaLM is a 540 billion parameter model, while the BERT model used in the MLPerf benchmark has only 340 million parameters — a 1000x difference in scale. Based on our experience, the benefits of TPUs will grow significantly with scale and make the case all the more compelling for training on Cloud TPU v4.

Have your cake and eat it too — a continued focus on sustainability

Performance at scale must take environmental concerns as a primary constraint and optimization target. The Cloud TPU v4 pods powering our MLPerf results run with 90% carbon-free energy and a Power Usage Efficiency of 1.10, meaning that less than 10% of the power delivered to the data center is lost through conversion, heat, or other sources of inefficiency. The TPU v4 chip delivers 3x the peak FLOPs per watt relative to the v3 generation. This combination of carbon-free energy and extraordinary power delivery and computation efficiency makes Cloud TPUs among the most efficient in the world.4

Making the switch to Cloud TPUs

There has never been a better time for customers to adopt Cloud TPUs. Significant performance and cost savings at scale as well as a deep-rooted focus on sustainability are why customers such as Cohere, LG AI Research, Innersight Labs, and Allen Institute have made the switch. If you are ready to begin using Cloud TPUs for your workloads, please fill out this form. We are excited to partner with ML practitioners around the world to further accelerate the incredible rate of ML breakthroughs and innovation with Google Cloud’s TPU offerings.

1. MLPerf™ v2.0 Training Closed. Retrieved from https://mlcommons.org/en/training-normal-20/ 29 June 2022, results 2.0-2010, 2.0-2012, 2.0-2098, 2.0-2099, 2.0-2103, 2.0-2106, 2.0-2107, 2.0-2120. The MLPerf name and logo are trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use is strictly prohibited. See www.mlcommons.org for more information.
2. MLPerf v1.0 and v2.0 Training Closed. Retrieved from https://mlcommons.org/en/training-normal-20/ 29 June 2022, results 1.0-1088, 1.0-1090, 1.0-1092, 2.0-2010, 2.0-2012, 2.0-2120.
3. ND96amsr A100 v4 Azure VMs, powered by eight 80 GB NVIDIA Ampere A100 GPUs (Azure’s flagship Deep Learning and Tightly Coupled HPC GPU offering with CentOS or Ubuntu Linux) is used for this benchmarking
4. Cost to train is not an official MLPerf metric and is not verified by MLCommons Association. Azure performance is a favorable estimate as described in the text, not an MLPerf result. Computations are based on results from MLPerf v2.0 Training Closed. Retrieved from https://mlcommons.org/en/training-normal-20/ 29 June 2022, results 2.0-2012, 2.0-2106, 2.0-2107.

Over the past few years, many organizations have experienced the benefits of migrating their SAP solutions to Google Cloud. But this migration can do more than reduce IT maintenance costs and make data more secure. By leveraging BigQuery, SAP customers can complement their SAP investments and gain fresh insights by consolidating enterprise data and easily extending it with powerful datasets and machine learning from Google. 

BigQuery is a leading cloud data warehouse, fully managed and serverless, and allows for massive scale, supporting petabyte-scale queries at super-fast speeds. It can easily combine SAP data with additional data sources, such as Google Analytics or Salesforce, and its built-in machine learning lets users operationalize machine learning models using standard SQL — all at a comparatively low cost. 

If your SAP-powered organization is looking to supercharge its analytics with the strength of BigQuery, read on for considerations and recommendations for modeling with SAP data. These guidelines are based on our real-world implementation experience with customers and can serve as a roadmap to the analytics capabilities your business needs.

Considerations for data replication 

Like most technology journeys, this one should start with a business objective. Keeping your intended business value and goals in mind is critical to making the right decisions in the early steps of the design process.

When it comes to replicating the data from an SAP system into BigQuery, there are multiple ways to do it successfully. Decide which method will work best for your organization by answering these questions:

• Does your business need real-time data? Will you need to time travel into past data?
• Which external datasets will you need to join with the replicated data?
• Are the source structures or business logic likely to change? Will you be migrating the SAP source systems any time soon? For instance, will you be moving from SAP ECC to SAP S/4HANA?

You’ll also need to determine whether replication should be done on a table-by-table basis or whether your team can source from pre-built logic. This decision, along with other considerations such as licensing, will influence which replication tool you should use.

Replicating on a table-by-table basis
Replicating tables, especially standard tables in their raw form, allows sources to be reused and ensures more stability of the source structure and functional output. For example, the SAP table for sales order headers (VBAK) is very unlikely to change its structure across different versions of SAP, and the logic that writes to it is also unlikely to change in a way that affects a replicated table. 

Something else to consider: Reconciliation between the source system and the landing table in BigQuery is linear when comparing raw tables, which helps avoid issues in consolidation exercises during critical business processes, such as period-end closing. Since replicated tables aren’t aggregated or subject to process-specific data transformation, the same replicated columns can be reused in different BigQuery views. You can, for instance, replicate the MARA table (the material master) once and use it in as many models as needed. 

Replicating pre-built logic
If you replicate pre-built models, such as those from SAP extractors or CDS views, you don’t need to build the logic in BigQuery, since you’re using existing logic. Some of these extraction objects have embedded delta mechanisms, which may complement a replication tool that can’t handle deltas. This will save initial development time, but it can also lead to challenges if you create new columns, or if customizations or upgrades change the logic behind the extraction. 

It’s also important to note that different extraction processes may transform and load the same source columns multiple times, which creates redundancy in BigQuery and can lead to higher maintenance needs and costs. However, replicating pre-built models may still be a good choice, since doing so can be especially useful for logic that tends to be immutable, such as flattening a hierarchy, or logic that is highly complex.

How you approach replication will also depend on your long-term plans and other key factors — for example, the availability (and curiosity) of your developers, and the time or effort they can put into applying their SQL knowledge to a new data warehouse. 

With either replication approach, bear in mind when designing your replication process that BigQuery is meant to be an append-always database — so post-processing of data and changes will be required in both cases. 

Processing data changes

The replication tool you choose will also determine how data changes are captured (known as CDC – change data capture). If the replication tool allows for it (for example as SAP SLT does) the same patterns described in the CDC with BigQuery documentation also apply to SAP data. 

Because some data, like transactions, are known to be less static than others (e.g., master data), you need to decide what should be scanned in real time, what will require immediate consistency, and what can be processed in batches to manage costs. This decision will be based on the reporting needs from the business.

Consider the SAP table BUT000, containing our example master data for business partners, where we have replicated changes from an SAP ERP system:

In an append-always replication in BigQuery, all updates are received as new records. For example, deleting a record in the source will be represented as a new record in BigQuery with a deletion flag. This applies to whether the records are coming from raw tables like BUT000 itself or pre-aggregated data, as from a BW extractor or a CDS view.

Let’s take a closer look at data coming particularly from the partners “LUCIA” and “RIZ”. The operation flag tells us whether the new record in BigQuery is an insert (I), update (U) or deletion (D), while the timestamps help us identify the latest version of our business partner.

If we want to find the latest updated record for the partners LUCIA and RIZ, this is what the query would look like:

  SELECT partner,
        ARRAY_AGG(i1 ORDER BY i1.recordstamp DESC LIMIT 1) AS row
FROM SAP_ECC.but000 i1 
WHERE partner in ('LUCIA','RIZ')
    GROUP BY partner

With the following result:

After identifying stale records for “LUCIA” and “RIZ” business partners, we can proceed to deleting all stale records for “LUCIA” if we do not want to retain the history. In this example, we are using a different table to which the same replication has been done, for the purpose of comparison and to check that all stale records have been deleted for the selection made and that we only kept last updated records. For example:

  DELETE SAP_HANA.but000 i1
WHERE
i1.recordstamp < TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 3 HOUR) AND
i1.recordstamp < (SELECT MAX(recordstamp) FROM SAP_HANA.but000 i2
WHERE
i1.partner = i2.partner
and partner="LUCIA")

You can also use the following query to retrieve stale records for “LUCIA” partner before moving forward with deletion

  SELECT partner, operation_flag, recordstamp  FROM SAP_HANA.but000 i1
WHERE
i1.recordstamp < TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 3 HOUR) 
AND
i1.recordstamp < (SELECT MAX(recordstamp) FROM SAP_HANA.but000 i2 
WHERE 
i1.partner = i2.partner
and partner="LUCIA")

Which produces all of the records, except the latest update:

Partitioning and clustering

To limit the number of records scanned in a query, save on cost and achieve the best performance possible, you’ll need to take two important steps: determine partitions and create clusters. 

Partitioning
A partitioned table is one that’s divided into segments, called partitions, which make it easier to manage and query your data. Dividing a large table into smaller partitions improves query performance and controls costs because it reduces the number of bytes read by a query.

You can partition BigQuery tables by:

• Time-unit column: Tables are partitioned based on a “timestamp,” “date,” or “datetime” column in the table.
• Ingestion time: Tables are partitioned based on the timestamp recorded when BigQuery ingested the data.
• Integer range: Tables are partitioned based on an integer column.

Partitions are enabled when the table is created, as in the example below.  A great tip is to always include the partition filter as shown on the left-hand side of the query.

Clustering
Clustering can be created on top of partitioned tables by applying the fields that are likely to be used for filtering. When you create a clustered table in BigQuery, the table data is automatically organized based on the contents of one or more of the columns in the table’s schema. The columns you specify are then used to colocate related data.

Clustering can improve the performance of certain query types — for example, queries that use filter clauses or that aggregate data. It makes a lot of sense to use them for large tables such as ACDOCA, the table for accounting documents in SAP S/4HANA. In this case, the timestamp could be used for partitioning, and common filtering fields such as the ledger, company code, and fiscal year could be used to define the clusters.

A great feature is that BigQuery will also periodically recluster the data automatically.

Materialized views

In BigQuery, materialized views are precomputed views that periodically cache the results of a query for better performance and efficiency. BigQuery uses precomputed results from materialized views and, whenever possible, reads only the delta changes from the base table to compute up-to-date results quickly. Materialized views can be queried directly or can be used by the BigQuery optimizer to process queries to the base table.

Queries that use materialized views are generally completed faster and consume fewer resources than queries that retrieve the same data only from the base table. If workload performance is an issue, materialized views can significantly improve the performance of workloads that have common and repeated queries. While materialized views currently only support single tables, they are very useful common and frequent aggregations like stock levels or order fulfillment.

Further tips on performance optimization while creating select statements can be found in the documentation for optimizing query computation.

Deployment pipeline and security

For most of the work you’ll do in BigQuery, you’ll normally have at least two delivery pipelines running — one for the actual objects in BigQuery and the other to keep the data staging, transforming, and updated as intended within the change-data-capture flows. Note that you can use most existing tools for your Continuous Integration / Continuous Deployment (CI/CD) pipeline — one of the benefits of using an open system like BigQuery. But, if your organization is new to CI/CD pipelines, this is a great opportunity to gradually gain experience. A good place to start is to read our guide for setting up a CI/CD pipeline for your data-processing workflow.  

When it comes to access and security, most end-users will only have access to the final version of the BigQuery views. While row and column-level security can be applied, as in the SAP source system, separation of concerns can be taken to the next level by splitting your data across different Google Cloud projects and BigQuery datasets. While it’s easy to replicate data and structures across your datasets, it’s a good idea to define the requirements and naming conventions early in the design process so you set it up properly from the start. 

Start driving faster and more insightful analytics

The best piece of advice we can give you is this: Try it yourself. Anyone with SQL knowledge can get started using the free BigQuery tier. New customers get $300 in free credits to spend on Google Cloud during the first 90 days. All customers get 10 GB storage and up to 1 TB queries/month, completely free of charge. In addition to discovering the massive processing capabilities, embedded machine learning, multiple integration tools, and cost benefits, you’ll soon discover how BigQuery can simplify your analytics tasks. 

If you need additional assistance, our Google Cloud Professional Services Organization (PSO) and Customer Engineers will be happy to help show you the best path forward for your organization. For anything else, contact us at cloud.google.com/contact.

Alphabet CEO Sundar Pichai has compared the potential impact of artificial intelligence (AI) to the impact of electricity—so it may be no surprise that at Google Cloud, we expect to see increased AI and machine learning (ML) momentum across the spectrum of users and use cases.

Some of the momentum is more foundational, such as the hundreds of academic citations that Google AI researchers earn each year, or products like Google Cloud Vertex AI accelerating ML development and experimentation by 5x, with 80% fewer lines of code required. Some are more concrete, like mortgage servicer Mr. Cooper using Google Cloud Document AI to process documents 75% faster with 40% cost savings; Ford leveraging Google Cloud AI services for predictive maintenance and other manufacturing modernizations; and customers across a wide range of industries deploying ML platforms atop Google Cloud.

Together, these proof points reflect our belief that AI is for everyone, and that it should be easy to harness in workflows of all kinds and for people of all levels of technical expertise. We see our customers’ accomplishments as validation of this philosophy and a sign that we are taking away the right things from our conversations with business leaders. Likewise, we see validation in recognition from analysts, which recently includes Google being named a Leader by

Gartner® in the 2022 Magic Quadrant™ for Cloud AI Developer Services report

Forrester in the Forrester Wave™: AI Infrastructure, Q4 2021 report, the Forrester Wave™: Document-Oriented Text Analytics Platforms, Q2 2022 report, and The Forrester Wave™: People-Oriented Text Analytics Platforms, Q2 2022 report

In June, we talked about four pillars that guide our approach to creating products for MLOps and to accelerate development of ML models and their deployment into product. In this article, we’ll look more broadly at our AI and ML philosophy, and what it means to create “AI for everyone.”

AI should be for everyone

One of the pillars we discussed in June was “meeting users where they are,” and this idea extends far beyond products for data scientists. Technical expertise should not be a barrier to implementing AI—otherwise, use cases where AI can help will languish without modernization, and enterprises without well-developed AI practices will risk falling behind their competitors.

To this end, we focus on creating AI and ML services for all kinds of users, e.g.:

• DocumentAI, Contact Center AI, and other solutions that inject AI and ML into business workflows without imposing heavy technical requirements or retraining on users;
• Pre-trained APIs, ranging from Speech to Fleet Optimization, that let developers leverage pre-trained ML models and free them from having to develop core AI technologies from scratch;
• BigQuery ML to unite data analysis tasks with ML;
• AutoML for abstracted and low-code ML production without requiring ML expertise;
• Vertex AI to speed up ML experimentation and deployment, with every tool you need to build deploy and the lifecycle of ML projects
• AI Infrastructure options for training deep learning and machine learning models cost effectively. Including Deep Learning VMs optimized for data science and machine learning tasks and AI accelerators for every use case, from low-cost inference to high-performance training.

It’s important to provide not only leading tools for advanced AI practitioners, but also leading AI services for users of all kinds. Some of this involves abstracting or automating parts of the ML workflow to meet the needs of the job and technical aptitude of the user. Some of it involves integrating our AI and ML services with our broader range of enterprise products, whether that means smarter language models invisibly integrated into Google Docs or BigQuery making ML easily accessible to data analysts. Regardless of any particular angle, AI is turning into a multi-faceted, pervasive technology for businesses and users the world over, so we feel technology providers should reflect this by building platforms that help users harness the power of AI by meeting them wherever they are.

How we’re powering the next generation of AI

Creating products that help bring AI to everyone requires large research investments, including in areas where the path to productization may not be clear for years. We feel a foundation in research combines with our focus on business needs and users to inform sustainable AI products that are in keeping with our AI principles and encourages responsible use of AI.

Many of our recent updates to our AI and ML platforms began as Google research projects. Just consider how DeepMind’s breakthrough AlphaFold project has led to the ability to run protein prediction models in Vertex AI. Or how research into neural networks helped create Vertex AI NAS, which lets data science teams train models more accurately with lower latency and power requirements.

Research is crucial, but also only one way of validating an AI strategy. Products have to speak for themselves when they reach customers, and customers need to see their feedback reflected as products are iterated and updated. This reinforces the importance of seeing customer adoption and success across a range of industries, use cases, and user types. In this regard, we feel very fortunate to work with so many great customers, and very proud of the work we help them accomplish.

I’ve already mentioned Ford and Mr. Cooper, but those are just a small sampling. For example, Vodafone Commercial’s “AI Booster” platform uses the latest Google technology to enable cutting-edge AI use cases such as optimizing customer experiences, customer loyalty, and product recommendations. Our conversational AI technologies are used by companies ranging from Embodied, whose Moxie robot helps children overcome developmental challenges, to HubSpot connecting meeting notes to CRM data. Across our products and across industries around the world, customer stories grow by the day.

We also see validation in our partner network. As we noted in the pillars discussed in June, partners like Nvidia help us to ensure customers have freedom of choice when building their AI stacks, and partners like Neo4j help our customers to expand our services into areas like graph structures. Partners support our mission to bring AI to everyone, helping more customers use our services for new and expanded use cases.

Accelerating the momentum

Overall, to create products that reflect AI’s potential and likely future ubiquity, we have to take all of the preceding factors, from research to customer and analyst conversations to working with partners, and turn them into products and product updates. We’ve been very active over the last year, from the launch of Call Center AI Platform in March, to the new Speech model we released in May, to a range of announcements at the Google Cloud Applied ML Summit in June. We have much more planned in coming months, and we’re excited to work with customers not just to maintain the pace of AI momentum, but to accelerate it. To learn more about Google Cloud’s AI and ML services, visit this link or browse recent AI and ML articles on the Google Cloud Blog.

GARTNER and MAGIC QUADRANT are registered trademarks and service marks of Gartner, Inc. and/or its affiliates in the U.S. and internationally and are used herein with permission. All rights reserved. Gartner does not endorse any vendor, product or service depicted in its research publications and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s Research & Advisory organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.

While the shift to digital business and the cloud has been well under way for some years now, organizations today have a new sense of urgency due to COVID-19. Delivering digital transformation is no longer a ‘nice to have’ option, rather, it is an operational imperative. Taking advantage of the infrastructure, platform and solution gains that cloud and microservices architecture provide is a must for brands today. 

At Google Cloud, we understand the pressures and challenges organizations of all sizes, across all industries are facing. The pandemic has dramatically impacted global commerce at-large, exposing (for many organizations across multiple sectors) gaps in omnichannel capabilities, business continuity and forecasting plans, not to mention spots in supply chain agility, resilience and responsiveness. 

A rapidly evolving consumer-driven commerce landscape has put innovation squarely in the spotlight for supply chain teams all over the world, with the effects of the global pandemic making it increasingly difficult for manufacturers, wholesalers, third party logistics providers and retailers (in particular) to weather the perfect storm of fast-moving consumer trends and a need for ‘always on’ digital innovation. 

These same effects have driven increasing interest and uptake of technology like the Manhattan Active® suite of solutions, as well as our own cloud platform; both of which afford organizations the levels of agility, flexibility and scalability needed to insulate their people, processes and long-term business strategies against unforeseen future obstacles such as global pandemics or international trade disputes.

An excellent example of this agility, flexibility and scalability in action is PVH’s response to the global pandemic. One of the most admired fashion and lifestyle companies with such iconic brands as Calvin Klein, TOMMY HILFIGER, Van Heusen, and IZOD, PVH was forced to temporarily close its physical stores and, as a result, experienced a sudden massive increase in online sales. The retailer was able to quickly pivot by adjusting its business rules in Manhattan Distributed Order Management (part of Manhattan Active Omni) to expose store inventory to online consumers and reroute its fulfillment processes. Thanks to Manhattan’s solution delivered through Google Cloud, in a matter of days, PVH was able to leverage both its distribution centers and vast store network to fulfill its online orders.

“The events of 2020 have accelerated retail and ecommerce operations forward,” said David Herridge, executive vice president of Global Value Chain Technologies for PVH. “With quick, creative thinking and the right partner, we were able to pivot operations, satisfy our customers and prepare for the future.”

Manhattan’s products have been recognized for their ability to solve real-world challenges through innovation, and used by many of the world’s top brands to solve some of their most complex commerce and supply chain challenges: the latest recognition is Manhattan’s position as sole leader in the 2021 Forrester Wave™ for Order Management Solutions. 

Since December 2018, Google Cloud has been collaborating closely with the team at Manhattan and its ‘always on’, versionless approach to innovation. And, during the last two and a half years, Manhattan has significantly accelerated its cloud-first solutions and market adoption, resulting in tremendous growth in its overall cloud business efforts. 

By building cloud native solutions on Google Cloud, the teams at Manhattan continue to deliver the high-performance, elastic, high-redundancy, secure solutions their customers rely on. Moreover, it means both Google Cloud and Manhattan continue to innovate and push the boundaries of what is possible in terms of the supply chain and omnichannel innovations that underpin global commerce – innovation that is needed more now than maybe ever before.

Our commitment to distributed cloud solutions and ongoing innovation, not to mention the fact Google Cloud operates a net carbon-neutral cloud, means that the working partnership between both industry leading teams continues to be a perfect match of brand values; not just from a technology perspective, but also a long-term sustainability and environmental one too.

More information on the partnership can be found here.