With nearly 2,300 stores, The Home Depot is the world’s largest home-improvement chain — a brand that professional contractors and DIYers alike have come to depend on. The home improvement industry continues to experience unprecedented demand and dramatic increases in online ordering accompanied by expanding consumer expectations for things like curbside pickup and same day delivery. The Home Depot’s decision to migrate to cloud-based infrastructure, including the migration of the company’s SAP applications on Google Cloud which began in 2017, has set it up for success in an increasingly digital world, and helped the company adapt to changing market conditions quickly. 

Interconnected retail at scale

Building on a strong customer-first philosophy, The Home Depot aims to create what it calls interconnected retail—allowing customers to shop however, whenever, and wherever they want. “So many companies are focused on omni-channel retail,” explains Sam Moses, Vice President of Corporate Systems. “At The Home Depot, we wanted to take it to the next level. Interconnected retail puts the customer at the center of everything and enables them to shop in store, online, or both. Customers can begin a transaction online and continue in-store, or vice-versa.”

To support this strategy, the company’s SAP environment needed to be more agile. Running  everything on-premises, from central finance to POS systems, meant that The Home Depot’s IT teams experienced redundancy and repetitive, manual processes. Their data warehouse needed an upgrade to process and analyze growing and increasingly diverse data sets. The Home Depot chose to migrate its SAP environment to Google Cloud to support both the velocity and scale needed for the business as well as critical analytics capabilities needed for its bold digital initiatives. “We chose Google Cloud to support our SAP implementation. Our decision had a lot to do with the relationship between Google Cloud and SAP and also for the applications and services that are offered by Google Cloud, like BigQuery, which are helping to enable data and analytics within our organization,” Moses explains.

After migrating its SAP applications—including S/4HANA, its customer activity repository (CAR), general ledger, e-commerce system, enterprise data warehouse and more to Google Cloud, the company now has the speed, scale and flexibility to tackle enormous spikes in the business, all while staying fully available for their customers. Additionally, The Home Depot was able to transform its financial systems and make them more agile to deliver critical information across multiple business functions in real time.

Maximizing data insights to support customer experiences

By migrating to Google Cloud, The Home Depot is leveraging Google Cloud analytics to build   the industry’s most efficient supply chain including more robust demand forecasting, supplier lead times, estimated delivery times and more, all while maintaining better security than before. “We experienced unprecedented change in our customers’ behavior and their buying patterns, which puts a lot of pressure on our supply chain,” explains Moses. “So having the ability to leverage data and analytics gives us insights to know exactly what it is that our customers need.” 

The company’s analysts now use BigQuery ML for machine learning directly against the company’s BigQuery data and use AutoML to determine the best model for predictions. The Home Depot’s engineers have also adapted BigQuery to monitor, analyze, and act on application performance data across all its stores and warehouses in real time—capabilities that were not as seamless in the on-premises environment.

With hundreds of projects on Google Cloud, The Home Depot’s cloud journey is well on track, but the company is always looking to the future. “As our customers’ needs have continued to evolve, and as technology has continued to evolve, our relationship with Google will continue to advance — to be able to innovate together, to be able to find new solutions together, to better serve our customers.”

Learn more about how The Home Depot is renovating its retail operation with SAP on Google Cloud.

The ability to gain real-time insights from IoT data can redefine competitiveness for businesses. Intelligence allows connected devices and assets to interact efficiently with applications and with human beings in an intuitive and non-disruptive way. After your IoT project is up and running, many devices will be producing lots of data. You need an efficient, scalable, affordable way to both manage those devices and handle all that information. 

IoT Core is a fully managed service for managing IoT devices. It supports registration, authentication, and authorization inside the Google Cloud resource hierarchy as well as device metadata stored in the cloud, and the ability to send device configuration from other GCP or third-party services to devices. 

Main components

The main components of Cloud IoT Core are the device manager and the protocol bridges:

• The device manager  registers devices with the service, so you can then monitor and configure them. It provides:
  • Device identity management 
  • Support for configuring, updating, and controlling individual devices
  • Role-level access control
  • Console and APIs for device deployment and monitoring
• Two protocol bridges (MQTT and HTTP) can be used by devices to connect to Google Cloud Platform for:
  • Bi-directional messaging
  • Automatic load balancing
  • Global data access with Pub/Sub

How does Cloud IoT Core work?

Device telemetry data is forwarded to a Cloud Pub/Sub topic, which can then be used to trigger Cloud Functions as well as other third-party apps to consume the data. You can also perform streaming analysis with Dataflow or custom analysis with your own subscribers.

Cloud IoT Core supports direct device connections as well as gateway-based architectures. In both cases the real time state of the device and the operational data is ingested into Cloud IoT Core and the key and certificates at the edge are also managed by Cloud IoT Core. From Pub/Sub the raw input is fed into Dataflow for transformation, and the cleaned output is populated in Cloud Bigtable for real-time monitoring or BigQuery for warehousing and machine learning. From BigQuery the data can be used for visualization in Looker or Data Studio and it can be used in Vertex AI for creating machine learning models. The models created can be deployed at the edge using Edge Manager (in experimental phase). Device configuration updates or device commands can be triggered by Cloud Functions or Dataflow to Cloud IoT Core, which then updates the device.  

Design principles of Cloud IoT Core

As a managed service to securely connect, manage, and ingest data from global device fleets, Cloud IoT COre is designed to be:

• Flexible, providing easy provisioning of device identities and enabling devices to access most of Google Cloud
• IThe industry leader in IoT scalability and performance
•  Interoperable, with supports for the most common industry-standard IoT protocols

Use cases

IoT use cases range across numerous industries. Some typical examples include:

• Asset tracking, visual inspection, and quality control in retail, automotive, industrial, supply chain and logistics
• Remote monitoring and predictive maintenance in oil & gas, utilities, manufacturing, and transportation
• Connected homes and consumer technologies.
• Vision intelligence in retail, security, manufacturing, and industrial sectors
• Smart living in commercial, residential, and smart spaces 
• Smart factories with predictive maintenance and real-time plant floor analytics

 For a more in-depth look into Cloud IoT Core check out the documentation.  

https://youtube.com/watch?v=76v16P-Wqe4%3Fenablejsapi%3D1%26

For more #GCPSketchnote, follow the GitHub repo. For similar cloud content follow me on Twitter @pvergadia and keep an eye out on thecloudgirl.dev.

May was a very busy month for data analytics product innovation. If you didn’t have the chance to attend our inaugural Data Cloud Summit, video replays of all our sessions are now available so feel free to watch them at your own pace. 

In this blog, I’d like to share some background behind the innovations we released in May, why we built them the way we did, and the type of value they can bring your company and your team.

But first, a huge thank you!

This week, we had the honor to announce that Google has been named a Leader in The Forrester Wave™: Streaming Analytics, Q2 2021 report. Forrester gave Dataflow a score of 5 out of 5 across 12 different criteria, stating: “Google Cloud Dataflow has strengths in data sequencing, advanced analytics, performance, and high-availability”. 

Google has more than a decade of experience in building real-time and internet-scale systems for its own needs, and we are excited to see that our ability to provide customers with a reliable, scalable, and performant platform is bearing fruit. 

This announcement comes on the back of the release of The Forrester Wave™: Cloud Data Warehouse, Q1 2021 report, which also named Google Cloud as a Leader.

We couldn’t be more excited about the recognition and appreciate all your feedback and trust in the work that we do to support your goal in accelerating data-powered innovation.

Innovation galore

Your feedback and your passion is the fuel that drives our ambition to deliver more and better services to you. That’s why, this year, we didn’t want to wait until Google Cloud Next to share some great products we have been working on. On May 26, our team announced a slew of new products, services and programs. Watch a quick summary below:

https://youtube.com/watch?v=DG1mOPMXJvw%3Fenablejsapi%3D1%26

Meeting you where you are

An important design principle behind all of our services is “meeting you where you are”. This means we aim to provide you with the tools and software you need to innovate on your own terms. Here are three new services that will help you do just that:

Datastream

Datastream, our new serverless change data capture (CDC) and replication service, allows your company to synchronize data across heterogeneous databases, storage systems, and applications reliably and with minimal latency to support real-time analytics, database replication, and event-driven architectures. Datastream delivers change streams from Oracle and MySQL databases into Google Cloud services such as BigQuery, Cloud SQL, Cloud Storage, and Cloud Spanner, saving time and resources while ensuring your data is accurate and up-to-date. 

• Under the hood, Datastream reads CDC events (inserts, updates, and deletes) from source databases, and writes those events with minimal latency to a data destination. It leverages the fact that each database source has its own CDC log—binlog for MySQL and LogMiner for Oracle—which it uses for its own internal replication and consistency purposes. 
• Datastream integrates with purpose-built and extensible Dataflow templates to pull the change streams written to Cloud Storage, and create up-to-date replicated tables in BigQuery for analytics. It also leverages Dataflow templates to replicate and synchronize databases into Cloud SQL or Cloud Spanner for database migrations and hybrid cloud configurations. 
• Datastream also powers a Google-native Oracle connector in Cloud Data Fusion’s new replication feature for easy ETL/ELT pipelining. By delivering change streams directly into Cloud Storage, customers can leverage Datastream to implement modern, event-driven architectures.

Looker and BigQuery Omni on Microsoft Azure

Research on multi cloud adoption is unequivocal — 92% of businesses in 2021 report having a multi cloud strategy. We want to continue supporting your choice by providing the flexibility you need to see your strategy through. 

• This past month, we introduced Looker, hosted on Microsoft Azure. For the first time, you can now choose Azure, Google Cloud, or AWS for your Looker instance. You can also self-host your Looker instance on-premises.
• We also introduced BigQuery Omni for Azure, which along with last year’s introduction of BigQuery Omni for AWS, will help you access and securely analyze data across Google Cloud, AWS, and Azure. 

The cost of moving data between cloud providers isn’t sustainable for many, and it’s still difficult to seamlessly work across clouds. BigQuery Omni represents a new way of analyzing data stored in multiple public clouds, which is made possible by BigQuery’s separation of compute and storage. By decoupling these two, BigQuery provides scalable storage that can reside in Google Cloud or other public clouds, and stateless resilient compute that executes standard SQL queries. 

• Unlike competitors, BigQuery Omni doesn’t require you to move or copy your data from one public cloud to another, where you might incur egress costs. You also benefit from the same BigQuery interface on Google Cloud, enabling you to query data stored in Google Cloud, AWS, and Azure without any cross-cloud movement or copies of data. 
• BigQuery Omni’s query engine runs the necessary compute on clusters in the same region where your data resides. For example, you can query Google Analytics 360 Ads data stored in Google Cloud and query logs data from your ecommerce platform and applications that are stored in AWS S3 and/or Microsoft Azure. 

Then, using Looker, you can build a dashboard that allows you to visualize your audience behavior and purchases alongside your advertising spend. 

Dataplex

We understand that most organizations still struggle to make high-quality data easily discoverable and accessible for analytics, across multiple silos, to a growing number of people and tools within their organization. 

They are often forced to make tradeoffs. For instance, moving and duplicating data across silos to enable diverse analytics use cases or leaving their data distributed but limiting the agility of decisions. 

• Dataplex provides an intelligent data fabric that enables you to centrally manage, monitor, and govern your data across data lakes, data warehouses, and data marts, while also ensuring data is securely accessible to a variety of analytics and data science tools. 
• One of the core tenets of Dataplex is letting you organize and manage your data in a way that makes sense for your business, without data movement or duplication. For that, we provide logical constructs like lakes, data zones, and assets. These constructs enable you to abstract away the underlying storage systems and become the foundation for setting policies around data access, security, lifecycle management, and so on. 
• For example, you can create a lake per department within your organization (e.g. Retail, Sales, Finance, etc.) and create data zones that map to data readiness and usage (e.g. landing, raw, curated_data_analytics, curated_data_science, etc.). 

Once you have your lakes and zones setup, you can attach data to these zones as assets. You can add data from different types of storage (e.g. GCS Bucket and BigQuery dataset) under the same zone. You can also attach data across multiple projects under the same zone. You can ingest data into your lakes and zones using the tools of your choice, including services such as Dataflow, Data Fusion, Dataproc, Pub/Sub, or choose from one of our partner products. Dataplex comes with built-in 1-click templates for common data management tasks. 
To find out more about Dataplex, head to cloud.google.com/dataplex or watch the video below:

https://youtube.com/watch?v=bbFeAt7cw1g%3Fenablejsapi%3D1%26

Helping you innovate everyday

Sharing data is hard. Traditional data sharing techniques use batch data pipelines that are expensive to run, create late arriving data, and can break with any changes to the source data. These techniques also create multiple copies of data, which brings unnecessary costs and can bypass data governance processes. They also fail to offer features for data monetization, such as managing subscriptions and entitlements. Altogether, these challenges mean that organizations are unable to realize the full potential of transforming their business with shared data.

Analytics Hub

To address these limitations, we are introducing Analytics Hub, a new fully managed service that helps organizations unlock the value of data sharing, leading to new insights and increased business value. 

This new service is built on the tremendous experience and feedback we have received over the years. For example, BigQuery has had cross-organizational, in-place data sharing capabilities since its inception in 2010—and the functionality is very popular. Over a 7-day period in April, we had over 3,000 different organizations sharing over 200 petabytes of data. These numbers don’t include data sharing between departments within the same organization.

Analytics Hub takes sharing to the next level, making it easy for you to publish, discover, and subscribe to valuable datasets that you can combine with your own data to derive unique insights. 

This includes: 

• Shared datasets: As a data publisher, you create shared datasets that contain the views of data that you want to deliver to your subscribers. Data subscribers can search through the datasets that are available across all exchanges for which they have access and subscribe to relevant datasets. In addition, the publisher can track subscribers, disable subscriptions, and see aggregated usage information for the shared data.
• Curated, self-service data exchanges: Exchanges are collections used to organize and secure shared datasets. By default, exchanges are completely private, but granular roles and permissions make it easy to deliver data to the right audience—whether internal or public. 

This is just the beginning for Analytics Hub. Please sign up for the preview, which is scheduled to be available in the third quarter of 2021.

Dataflow Prime

At Google Cloud, we have the great privilege of working with some of the most innovative organizations in the world. And this work provides us with a unique perspective into the future of big data processing. Dataflow Prime is a new platform based on a serverless, no-ops, and auto-tuning architecture that brings unparalleled resource utilization and radical operational simplicity to big data processing. This new service introduces a large number of exciting capabilities but I’d like to highlight three key aspects of the product:

• Vertical Autoscaling: Dataflow Prime dynamically adjusts the compute capacity allocated to each worker based on utilization, detecting when jobs are limited by worker resources and automatically adding more resources. Vertical Autoscaling works hand in hand with Horizontal Autoscaling to seamlessly scale workers to best fit the needs of the pipeline. As a result, it no longer takes hours or days to determine the perfect worker configuration to maximize utilization. 
• Right Fitting: Each stage of a pipeline typically has a different resource requirement than the others. Until now, either all workers in the pipeline would have had the higher memory and GPU, or none of them would. Pipelines either had to waste resources or suffer slower workloads. Right Fitting solves this problem by creating stage-specific pools of resources, optimized for each stage. 
• Smart Recommendations: Smart Recommendations automatically detects problems in your pipeline and shows potential fixes. For example, if your pipeline is running into permissions issues, a Smart Recommendation will detect which IAM permissions you need to enable to unblock your job. If you are using an inefficient coder in your job, Smart Recommendations will surface more performant coder implementations that can help you save on costs.

What’s next

We’re excited to hear your thoughts and feedback about all these exciting new services. I would also highly recommend that you connect with members of the community to learn more about their story and journey. A good example to start with is the Data To Value customer panel we produced at our inaugural Data Cloud Summit with the Chief Data Officers of Keybank and Rackspace. You can watch it for free below:

https://youtube.com/watch?v=ITI2Q3MkxuA%3Fenablejsapi%3D1%26

We introduced the Google Cloud Ready – Sustainability designation earlier this year to showcase those partners committed to help global businesses and governments accelerate their sustainability programs. These partners build solutions that enhance the capabilities and ease the adoption of powerful Google Cloud technologies, such as Google Earth Engine and BigQuery, allowing customers to leverage data-rich solutions that help reduce their carbon footprints.

Today, we’re pleased to announce growth of the Google Cloud Ready – Sustainability program, with 12 new partners joining the initiative and bringing their climate, ESG, and sustainability platforms to Google Cloud. These partners include:

Aclima is pioneering an entirely new way to diagnose the health of our air and track climate-changing pollution. Powered by its network of roving and stationary sensors, Aclima measures air pollution and greenhouse gasses at unprecedented scales and with block-by-block resolution.

Sustainability at Airbus means uniting and safeguarding the world in a safe, ethical, and socially and environmentally responsible way. Airbus has a comprehensive sustainability strategy built on four core commitments, which guide the company’s approach to the way it does business and how it designs its products and services: Lead the journey toward clean aerospace, respect human rights and foster inclusion, build the business on the foundation of safety and quality, and exemplify business integrity.

Atlas AI is a predictive analytics platform that analyzes, monitors, and forecasts regions of growth, vulnerability, and opportunity around the world to offer insight into where organizations can grow most successfully, and where investment can boost historically underserved communities. Atlas AI’s platform has been used to expand water and sanitation infrastructure, promote new electrification, target community health services, and broaden internet access in countries across Sub-Saharan Africa and South Asia.

BlueSky Resources makes sense of sensors from both public and private sources by harmonizing inputs from ground, aerial and space based inputs. Expertise in atmospheric science and cloud technology allows BlueSky to provide understanding and insights related to the correlation of emissions insights to assets and activities. This powerful combination of data, climate science and delivery of insights is enabling focused sustainability impact across clients in various industries including energy, waste management, industry and natural resource management.

Electricity Maps provides companies with actionable data quantifying the carbon intensity and origin of electricity. This data is available on an hourly basis across 50+ countries and more than 160 regions. Electricity Maps’ mission is to organize the world’s electricity data to drive the transition toward a truly decarbonized electricity system.

FlexiDAO is a global climate tech company based in the Netherlands and Spain. The company works closely with other critical stakeholders to co-create the international standard around energy-related emissions compliance. Thanks to FlexiDAO’s end-to-end 24/7 Carbon-free Energy platform, companies can quantify and confidently showcase their contribution to society’s decarbonization.

LevelTen Energy helps organizations achieve carbon-free energy usage targets (on an annual and 24/7 basis) by delivering access to the world’s largest clean energy marketplace, and the software, data, analytics, and expertise required for efficient transactions. The LevelTen Platform connects energy buyers and over 40 sustainability advisors with more than 1,800 carbon-free energy projects in 24 countries across North America and Europe.

Ren is a SaaS platform built on Google Cloud that enables companies with global supply chains to source the cleanest energy possible. Despite using country-sized amounts of energy, most companies have no idea how to transition to renewables due to complex financial, technical, and logistical challenges. Ren unlocks cost savings, provides the cleanest energy possible, and ensures companies meet their carbon commitments on time.

Sidewalk Labs, an urban innovation unit in Google, builds products to radically improve quality of life in cities for all. Delve is a product that helps real estate teams design more sustainable buildings and neighborhood blocks, faster. Mesa automates building controls to deliver savings and comfort to commercial building owners and tenants. With these products and others, Sidewalk Labs helps commercial real estate developers, building owners and city planners make more sustainable choices for the built environment that are better for communities and the planet.

Tomorrow.io is The World’s Weather and Climate Security Platform, helping countries, businesses, and individuals manage their weather and climate security challenges. The platform is fully customizable to any industry impacted by the weather. Customers around the world use Tomorrow.io to dramatically improve operational efficiency. Tomorrow.io was built from the ground up to help teams prepare for the business impact of weather by automating decision-making and enabling climate adaptation at scale.

UP42 is a geospatial developer platform and marketplace bringing together industry-leading data and ready-to-use processing algorithms. The platform enables organizations to build, run, and scale geospatial products. With the ability to choose from a wide range of high-resolution commercial and open satellite data, aerial, weather, and others, solution providers can apply best-in-class machine learning and/or processing modules to gain valuable geospatial insights and streamline their processes.

Woza is a sustainable innovation platform that leverages deep geospatial knowledge and existing best-in-class technologies to develop a new generation of streamlined analytics workflows focused on sustainability. Companies in agri-food, energy, and public sector are partnering with Woza to accelerate their journey to Industry 4.0.

Adding expertise to accelerate sustainability use cases

New partners in the initiative join our existing Google Cloud Ready – Sustainability partners like Carto, Climate Engine, Geotab, NGIS, and Planet Labs PBC bringing a wealth of industry knowledge, offering solutions for sustainability challenges ranging from first-mile sustainable sourcing and spatial finance to fleet electrification and rich geospatial visualizations.

CARTO is the world’s leading Location Intelligence platform, enabling organizations to use spatial data and analysis for more efficient delivery routes, better behavioral marketing, strategic store placements, and much more. The company’s solutions extend the geospatial capabilities available in BigQuery, while leveraging the near limitless scalability that Google Cloud provides. When it comes to sustainability, CARTO’s platform is trusted by a wide range of organizations, including Greenpeace, Vizzuality, Litterati, Indigo, WWF, the Marine Conservation Institute, The World Bank, and the Institute for Sustainable Cities.

Climate Engine leverages data from Google Earth Engine and other ecosystem partners to help organizations improve their climate change-related risk planning in areas such as water use, agriculture, storm risk, and wildfire spread. By linking the economy and the environment, organizations can understand how environmental risks are affecting their markets and discover opportunities to reduce their emissions and potential supply chain or operational disruptions from climate-related events.

Geotab is advancing security, connecting commercial vehicles to the cloud, and providing data-driven analytics to help customers better manage their fleets. Processing billions of data points daily, Geotab helps businesses improve and optimize fleet productivity, enhance safety, and achieve sustainability goals and stronger compliance.

Geospatial solutions provider NGIS built a SaaS-based first-mile sustainable sourcing solution called TraceMark using Google Cloud’s geospatial platform and technologies from other ecosystem partners. Several global CPG firms have already used TraceMark to modernize their geospatial workflows and help facilitate the use of space-based data for supply chain sustainability transformation.

Planet Labs PBC operates the largest fleet of Earth imaging satellites in history, with approximately 200 satellites in orbit. Planet’s mission is to image the whole earth’s landmass every day to make global change visible, accessible, and actionable. As a Public Benefit Corporation, Planet’s Public Benefit Purpose is to accelerate humanity toward a more sustainable, secure, and prosperous world by illuminating environmental and social change.

How the Google Cloud Ready – Sustainability program works

If you are a Google Cloud partner with sustainability solutions and expertise to share, the Google Cloud Ready – Sustainability program is open for applications. Entry into the program requires that the partner solution delivers quantifiable results for climate mitigation, adaptation, or reporting needs. To apply for the Google Cloud Ready – Sustainability designation, the solution must:

• Be available on Google Cloud
• Address ESG risk, and assist customers in achieving ESG targets and/or support typical ESG goal frameworks, such as the United Nations’ SDGs
• Demonstrate repeatability
• Meet minimum Google Cloud application development best practices, including security, performance, scalability, availability, and carbon footprint reporting for available services
• Have Google Cloud Carbon Footprint Reporting enabled
• Have at least one public customer case study available.

The selection process begins with an evaluation of the solution. If a partner meets the above criteria, Google Cloud provides a suggested roadmap for tier progression within the program and then issues a formal acknowledgement of participation in the Google Cloud Ready – Sustainability program. Together, Google Cloud sustainability partners can deliver platforms that are helping businesses and governments accelerate progress aligned to their environmental goals.

Google Cloud will showcase the validated solutions on the Google Cloud Partner Directory Listing, Google Cloud Ready Sustainability Partner Advantage page, and — if applicable — via the Google Cloud Marketplace. We hope to help customers better understand how these technologies can help them meet their ESG goals, find the right solution for their particular challenge, and implement a solution faster.

Prospective partners can visit the Partner Portal to learn more about the Google Cloud Ready – Sustainability program or complete an application.