Scaling Data-Driven Insights Across a Complex Global Organization with Looker and BigQuery - Build What's Next

3376

Of your peers have already watched this video.

16:30 Minutes

The most insightful time you'll spend today!

Case Study

Scaling Data-Driven Insights Across a Complex Global Organization with Looker and BigQuery

In this video, SpringML and Iron Mountain share how migrating data to Google Cloud not only saved hundreds of thousands of dollars in licensing consolidation but allows stakeholders across a complex global organization to:

  • Consolidate 1,200 data management applications into one data lake
  • Unify information governance practices across disparate corporate verticals and departments
  • Provide a single view of operational and customer data for enhanced decision making across 43 countries and 200 acquisitions
  • Unlock the power of data science and predictive analytics

Standardizing across varying systems and processes onto one platform took just a matter of weeks for a project that could have taken about a year.

SpringML and Iron Mountain share how this was done, lessons learned and next steps for their collaboration adopting Looker and BigQuery into Iron Mountain.

Blog

What’s New in Retail: Bits from Google Cloud’s Retail & Consumer Goods Summit

7983

Of your peers have already read this article.

3:00 Minutes

The most insightful time you'll spend today!

Google Cloud transforms the retail industry with solutions for digital and omnichannel growth, data-driven and customer-focused experiences, and operational improvement.

Today we’re hosting our Retail & Consumer Goods Summit, a digital event dedicated to helping leading retailers and brands digitally transform their business. For me, this is a personally exciting moment, as I see tremendous opportunities for those companies that choose to focus on their customers and leverage technology to elevate experiences.

Our event includes breakout sessions to help retailers and brands become customer centric, embrace the digital moment and transform their operations. Some of my favorite sessions include: 

I’ll be speaking in our Retail Spotlight session, discussing the current retail landscape and our industry approach, followed by conversations with Albert Bertilsson, Head of Engineering – Edge at IKEA Retail (Indga Group) and Neelima Sharma, Senior Vice President, Technology Ecommerce, Marketing and Merchandising at Lowe’s. 

Let me share a bit more about the topics we’ll discuss in that session.

In retail specifically, digital-first shopping journeys are blurring the lines between the physical and digital brand experience. Shoppers want to know what’s available before they visit your stores, and they expect fulfillment options like curbside pickup. We see this when tracking trends for interest in curbside pickup or in-stock items.

google search results.jpg

This has left many retailers asking how they can get smarter with their data, tackle the $300 billion dollar problem of “search abandonment,” move faster to create new customer experiences, and do a better job of connecting their employees and customers – with confidence.

Our team has been spending time thinking about how we can rise and succeed in this new era together. We continue to focus on areas where we can bring the best of our capabilities to our retail customers around the world. And we’re focused on ways we can bring the best of what Google has to offer through cloud integrations.

Our goal is to help retailers become customer-centric and data-driven, capture digital and omni-channel revenue growth, create the modern store and drive operational improvement.

ways we're helping retailers transform.jpg

Let’s dig into each of these strategic pillars in a bit more detail. 

Become customer centric and data driven

Customers today expect experiences that are timely, targeted, and tailored for them and their needs, and reject experiences that can’t deliver these features. Data modeling, legacy technology, and siloed systems often prevent retailers from providing that level of personalized experience. 

At Google Cloud, we work with global retailers and our ecosystem partners to activate and bring value from first-party data, particularly in the field of customer data platforms (CDPs). This includes integrations from Google Cloud, such as our business intelligence platform Looker and other popular platforms to power one source of customer data through the organization. We also help retailers modernize their data warehouse with Looker for gathering business intelligence across their organization. This is important not just for consumer data, but inventory, supply chain, and store operations as well. 

Capture Digital and Omnichannel Growth 

We power some of the largest e-commerce sites in the world, helping them scale for Black Friday, Cyber Monday, and other holiday events. While scale is critically important, it’s also important to consider the quality of the online experience. How do your customers find products? How can you help deliver seamless online and omnichannel experiences? 

To help, we’re building product discovery solutions that bring together the best of our technologies that help retailers drive engagement with their consumers. Retail Search, for example, gives retailers the ability to provide Google-quality search on their own digital properties – search that is customizable for their unique business needs and built upon Google’s advanced understanding of user intent & context. 

The imperative is clear. Recent research found that retailers lose more than $300 billion to search abandonment — when purchase intent is not converted into a sale due to bad search results — every year in the US alone. 

Today, we announced that Retail Search is available to a larger set of retailers. If you are interested in learning more about Retail Search you can contact your sales representative for additional details.

Create the modern store  

With the rise of buying trends like curbside pickup and proximity-based search, our Google Maps Platform team is working on new products and features to help raise inventory awareness for your shoppers. We want to help you make it easier for them to understand what’s available to purchase in their channel of choice.

With Product Locator, each product page connects customers with information they need for local pickup and delivery options. This ensures customers are aware of pickup and delivery options throughout the buying journey—not just checkout. 

Awareness of local inventory can boost a wide range of key metrics for your business. Shopify recently shared that shoppers who opt for local pickup over delivery had a +13% higher conversion rate and that 45% of local pickup customers make an additional purchase upon arrival.

This is just one quick example of how our Google Maps Platform team can improve experiences for your shoppers.

Operational improvement

It can be challenging to operate in a world and at a time when consumer behavior and supply chains are so disrupted and volatile, and where entire retail teams had to go remote during the pandemic and beyond. 

We’re working with retailers to leverage artificial intelligence (AI) to improve consumer experience through chat bots or conversational commerce that solves problems for customers from anywhere. You can learn more about these offerings in our Conversational Commerce with Google breakout session, featuring Albertsons.

As the need for digital transformation continues to accelerate, Google Cloud is helping retailers stay ahead of the curve with solutions for digital and omnichannel growth, data-driven and customer-focused experiences, and operational improvement. For every era of cloud technologies, from the past into the future, Google Cloud is committed to providing solutions to retailers.

Read more about our solutions for retail, and check out additional sessions, including the CPG Industry Spotlight Session How To Grow Brands in Times of Rapid Change – Featuring L’Oréal at our Retail & Consumer Goods Summit.

3377

Of your peers have already watched this video.

16:30 Minutes

The most insightful time you'll spend today!

Case Study

Scaling Data-Driven Insights Across a Complex Global Organization with Looker and BigQuery

In this video, SpringML and Iron Mountain share how migrating data to Google Cloud not only saved hundreds of thousands of dollars in licensing consolidation but allows stakeholders across a complex global organization to:

  • Consolidate 1,200 data management applications into one data lake
  • Unify information governance practices across disparate corporate verticals and departments
  • Provide a single view of operational and customer data for enhanced decision making across 43 countries and 200 acquisitions
  • Unlock the power of data science and predictive analytics

Standardizing across varying systems and processes onto one platform took just a matter of weeks for a project that could have taken about a year.

SpringML and Iron Mountain share how this was done, lessons learned and next steps for their collaboration adopting Looker and BigQuery into Iron Mountain.

10537

Of your peers have already watched this video.

2:15 Minutes

The most insightful time you'll spend today!

Case Study

The True Story of How HotStar Broke a World-Record–Thanks to Firebase and Google BigQuery

Hotstar, India’s largest video streaming platform with 150 million monthly active users around the world, provides live-streaming of TV shows, movies, sports, and news on the go.

By using a combination of Firebase products together, Hotstar safely rolled out new features to its watch screen during a major live-streaming event without disrupting users, sacrificing stability, or releasing a new build. They also used Firebase with BigQuery to analyze their event data and reduce app startup time.

“We have an ambitious mission, but our engineering team is only a fraction of the size of most of our competitors. But we are still keeping up, and we are doing it with the help of Firebase,” says Ayushi Gupta, Android Engineer, Hotstar.

Blog

Notebook Executor Feature of Vertex AI Workbench to Schedule Notebooks Ad Hoc or on Recurring Basis

4453

Of your peers have already read this article.

3:00 Minutes

The most insightful time you'll spend today!

The launch of notebook executor feature of Vertex AI Workbench will scale notebook workflows, configuring different hardware options, passing in parameters for experimentation, and setting an execution schedule, all via the Console UI!

When solving a new ML problem, it’s common to start by experimenting with a subset of your data in a notebook environment. But if you want to execute a long-running job, add accelerators, or run multiple training trials with different input parameters, you’ll likely find yourself copying code over to a Python file to do the actual computation. That’s why we’re excited to announce the launch of the notebook executor, a new feature of Vertex AI Workbench that allows you to schedule notebooks ad hoc, or on a recurring basis. With the executor, your notebook is run cell by cell on Vertex AI Training. You can seamlessly scale your notebook workflows by configuring different hardware options, passing in parameters you’d like to experiment with, and setting an execution schedule, all via the Console UI or the notebooks API

Built to Scale

Imagine you’re tasked with building a new image classifier. You start by loading a portion of the dataset into your notebook environment and running some analysis and experiments on a small machine. After a few trials, your model looks promising, so you want to train on the full image dataset. With the notebook executor, you can easily scale up model training by configuring a cluster with machine types and accelerators, such as NVIDIA GPUs, that are much more powerful than the current instance where your notebook is running.

Your model training gets a huge performance boost from adding a GPU, and you now want to run a few extra experiments with different model architectures from TensorFlow Hub. For example, you can train a new model using feature vectors from various architectures, such as InceptionResNet, or MobileNet, all pretrained on the ImageNet dataset. Using these feature vectors with the Keras Sequential API is simple; all you need to do is pass the TF Hub URL for the particular model to hub.KerasLayer.

1 Vertex AI Workbench.jpg

Instead of running these trials one by one in the notebook, or making multiples copies of your notebook (inceptionv3.ipynb, resnet50.ipynb, etc) for each of the different TF Hub URLs, you can experiment with different architectures by using a parameter tag. To use this feature, first select the cell you want to parameterize. Then click on the gear icon in the top right corner of your notebook.

2 Vertex AI Workbench.jpg

Type “parameters” in the Add Tag box and hit Enter. Later when configuring your execution, you’ll pass in the different values you want to test.

3 Vertex AI Workbench.jpg

In this example, we create a parameter called feature_extractor_model, and we’ll pass in the name of the TF hub model we want to use when launching the execution. That model name will be substituted into the tf_hub_uri variable, which is then passed to the hub.KerasLayer, as shown in the screenshot above.  

After you’ve discovered the optimal model architecture for your use case, you’ll want to track the performance of your model in production. You can create a notebook that pulls the most recent batch of serving data that you have labels for, gets predictions, and computes the relevant metrics. By scheduling these jobs to execute on a recurring basis, you’ve created a lightweight monitoring system that tracks the quality of your model predictions over time. The executor supports your end-to-end ML workflow, making it easy to scale up or scale out notebook experiments written with Vertex AI Workbench.

Configuring Executions

Executions can be configured through the Cloud Console UI or the Notebooks API.

In your notebook, click on the Executor icon.

4 Vertex AI Workbench.jpg

In the side panel on the right specify the configuration for your job, such as the machine type and the environment. You can select an existing image, or provide your own custom docker container image.

5 Vertex AI Workbench.jpg

If you’ve added parameter tags to any of your notebook cells, you can pass in your parameter values to the executor.

6 Vertex AI Workbench.jpg

Finally, you can choose to run your notebook as a one time execution, or schedule recurring executions.

7 Vertex AI Workbench.jpg

Then click SUBMIT to launch your job.

8 Vertex AI Workbench.jpg

In the EXECUTIONS tab, you’ll be able to track the status of your notebook execution.

9 Vertex AI Workbench.jpg

When your execution completes, you’ll be able to see the output of your notebook by clicking VIEW RESULT.

10 Vertex AI Workbench.jpg

You can see that an additional cell was added with the comment # Parameters, that overrides the default value for feature_extractor_model, with the value we passed in at execution time. As a result, the feature vectors used for this execution came from a ResNet50 model instead of an Inception model.

What’s Next?

You now know the basics of how to use the notebook executor to train with a more performant hardware profile, test out different parameters, and track model performance over time. If you’d like to try out an end-to-end example, check out this tutorial. It’s time to run some experiments of your own!

Case Study

This Chart, from Home Depot, Dramatically Demonstrates the Power of a Cloud Data Warehouse

7627

Of your peers have already read this article.

2:30 Minutes

The most insightful time you'll spend today!

When Home Depot moved it's gigantic enterprise data warehouse to Google Cloud, it could not have imagined how much faster it could crunch data--for a variety of uses cases.

The Home Depot (THD) is the world’s largest home-improvement chain, growing to more than 2,200 stores and 700,000 products in four decades. Much of that success was driven through the analysis of data. This included developing sales forecasts, replenishing inventory through the supply chain network, and providing timely performance scorecards.

However, to compete in today’s business world, THD has taken this data-driven approach to an entirely new level of success on Google Cloud, providing capabilities not practical on legacy technologies.

The Home Depot BigQuery installation performance table
Percent reduction in time that specific workloads took using BigQuery versus on-premises data warehousing.

The pressures of contemporary growth that drove much of the work are familiar to many businesses. In addition to everything it was doing, THD needed to better integrate the complexities in its related businesses, like tool rental and home services. It needed to better empower teams, including a fast-growing data analysis staff and store associates with mobile computing devices. It wanted to better use online commerce and artificial intelligence to meet customer needs, while maintaining better security.

Even before addressing these new challenges, THD’s existing on-premises data warehouse was under stress as more data was required for analytics and data analysts were utilizing the data with increasingly complex use cases. This drove rapid growth of the data warehouse, but also created constant challenges for the team in managing priorities, performance, and cost.

In order to add capacity to the environment, it was a major planning, architecture, and testing effort. In one case, adding on-premises capacity took six months of planning and a three-day service outage. Within a year, capacity was again scarce, impacting performance and ability to execute all the reporting and analytics workloads required. The capacity refresh cycles were shrinking, and the expecations for data were growing. There had to be a better way.

Still, THD did not take its move to the cloud lightly. A large-scale enterprise data warehouse migration involves tremendous effort among people, process, and technology. After careful consideration, THD chose Google Cloud’s BigQuery for its cloud enterprise data warehouse.

BigQuery, a scalable serverless data warehouse, was better on cost, infrastructure agility, and analytics capability, driving better insights with improved performance. There are no service interruptions when capacity is added, and that capacity can be added within a week (and soon same day). It doesn’t require complex system administration, and its standard SQL support means people can easily ramp up quickly. Valuable BigQuery products like Identity and Access Management meant THD could create many separate Google Cloud projects, while ensuring that different teams weren’t interfering with each other or accessing protected data.

THD also utilizes BigQuery’s flat-rate monthly pricing model that allows teams to budget their capacity based on need and provides billing predictability. The capacity not being used by a given project is available for enterprise use. This ensures no surprises when the monthly bill arrives and provides all analytical users access to significant computing power.

While THD’s legacy data warehouse contained 450 terabytes of data, the BigQuery enterprise data warehouse has over 15 petabytes. That means better decision-making by utilizing new datasets like website clickstream data and by analyzing additional years of data.

As for performance, look at this chart:

With the cloud EDW migration complete, and the legacy on-premises data warehouse retired, analysts now execute more complex and demanding workloads that they would not have been able to complete before, such as utilizing Datalab for orchestrating analytics through Python Notebooks, utilizing BigQuery ML for machine learning directly against the BigQuery data (no movement of large datasets), and AutoML to help determine the best model for predictions.

Additionally, engineers at THD have adapted BigQuery to monitor, analyze, and act on application performance data across all its stores and warehouses in real time, something that was not practical in the on-premises system.

With over 600 projects that THD now has on Google Cloud, the BigQuery story is just one of the many ways that Google Cloud is working with THD to deliver meaningful business results, every day.

More Relevant Stories for Your Company

Case Study

Google Cloud Platform Gives Us 5x the Processing Power to Analyze Physician Performance at 75% Lower Cost

Patients about to undergo a healthcare procedure understandably want the best medical professionals they can get. But how can they know which doctors have had the most experience and the best outcomes with that particular procedure? How can they make an informed decision about which doctor to select when the

Case Study

Serverless and BigQuery Together on Google Cloud: Behind the L’Oreal Beauty Tech Data Platform

Editor's note: In Today's guest post we hear from beauty leader L'Oréal about their approach to building a modern data platform on fully managed services: managing the ingest of diverse datasets into BigQuery with Cloud Run, and orchestrating transformations into relevant business domain representations for stakeholders across the organization. Learn

Podcast

Cloud Dataprep: The Easiest Way to Cleanse Data

Anyone who has worked with data has felt the pain of data preparation. It’s a struggle—if you are working with the wrong tools—to massage data or cleanse data of anomalies, outliers, and just plain old dirty data. Eric Anderson, a Product Manager at Google working on Cloud Dataprep and a

Explainer

The New and Upcoming Infrastructure for Google Cloud’s AI and ML Solutions

How does Google manage to provide its customers a differentiated compute platform experience and define ways to fully leverage its infrastructure supporting its cutting-edge AI and ML offerings? Easy-to-use, scalable and ability to create innovative products and services to end-users at low cost of ownership is the narrative behind Google

SHOW MORE STORIES