Building Data Lakes on Google Cloud - Build What's Next

3134

Of your peers have already watched this video.

38:30 Minutes

The most insightful time you'll spend today!

How-to

Building Data Lakes on Google Cloud

A data lake is a centralized and a secure data platform for ingesting, storing, processing, and analyzing all your data–structured semi-structured and unstructured data–at scale.

“When we work with our customers we find out that the traditional approach for building data lakes, where you bring data from different sources and land them into a single central store has not worked. The traditional approach ended up creating swamps instead of lakes,” says Nitin Motgi, Product Manager, Google Cloud.

In this video, Motgi will talk about how to build data lakes correctly.

Google Cloud provides all the capabilities enterprises need to create and manage data lakes. Enterprises can use Google Cloud to aggregate their data and efficiently analyze it using cloud-native or open source tools irrespective of where the data is managed.

Also, learn how Google is making investments to simplify data lake deployment and management.

Whitepaper

Moving Your Data Warehouse to the Cloud? Here’s What You Need to Know

DOWNLOAD WHITEPAPER

5499

Of your peers have already downloaded this article

10:30 Minutes

The most insightful time you'll spend today!

Modernizing your data warehouse is one way to keep up with evolving business requirements and harness new technology. For many companies, cloud data warehousing offers a fast, flexible, and cost-effective alternative to traditional on-premises solutions.

In a report sponsored by Google Cloud, TDWI examines the rise of cloud-based data warehouses and identifies associated opportunities, benefits, and best practices.

Featuring strategic advice from Google experts, it answers questions such as:

  • What’s driving businesses to consider the cloud for their data warehousing strategy?
  • What are the advantages of a cloud-native data warehouse?
  • How can you coordinate data warehouse modernization with other modernization projects?
  • What’s the first step in migrating your data warehouse to the cloud?
  • How will cloud data warehousing affect your business’s security posture?

Download the complete report to learn more about cloud data warehousing and how it can help your business transform with the times — and prepare for the future.

Blog

AI-powered Business Messages for Timely, Engaging and Helpful Conversations with Customers

9386

Of your peers have already read this article.

2:00 Minutes

The most insightful time you'll spend today!

Bot-in-a-Box based on Google AI tools Dialogflow-part of the Google Cloud Contact Center AI helps build capabilities in Business Messages that helps brand of all sizes leverage Conversational AI and automate conversations. Learn more.

Over the last two years, we’ve seen a significant uptick in the number of people using messaging to connect with businesses. Whether it was checking hours of operation, verifying what was in stock, or scheduling a pick-up, the pandemic caused a significant shift in consumer behavior.

With the rise in demand for messaging, consumers expect communication with businesses to be  speedy, simple, and convenient. For businesses, keeping up with customer inquiries can be a labor-intensive process, and offering 24/7 support outside of store hours can be costly.  

To help businesses seamlessly deliver helpful, timely, and engaging conversations with customers when and where they need help, we introduced AI-powered Business Messages.

https://youtube.com/watch?v=fcgP3RHjBLY%3Fenablejsapi%3D1%26

What is AI-powered Business Messages?

With AI-powered Business Messages, you can connect with your customers in their moment of need, in the places they’re looking for answers—such as Google Search, Google Maps, or any brand-owned channel. For instance, check out how Walmart customers in the US are able to receive real-time information on product availability, straight from a search results page.

1 search and chat.jpg
Customers can search and chat with Walmart and quickly get help on inventory availability

People turn to Google when they are searching for answers to their questions, looking to buy something, or trying to accomplish a particular task with one of our many tools. In fact, 68% of all online experiences begin with a search engine.

At Google, we know how important it is for interactions with a brand to be personalized, helpful, and simple. With AI-powered Business Messages, customers are able to chat with virtual agents that understand, interact, and respond in natural ways.

We are also combining smart automation with the ability for customers to chat with live agents when it’s really needed. This approach saves your customers precious time, while also saving you money. And with Business Messages automatically handling many customer inquiries in the background, businesses have the option to distribute their human customer service agents to address other needs. 

Getting started with conversational AI is easy with Bot-in-a-Box

We know it can be difficult to get started with AI. That’s why we are utilizing existing Google AI tools like Dialogflow—part of Google Cloud Contact Center AI—to create the capability within Google’s Business Messages called Bot-in-a-Box, which makes getting started with Conversational AI easy. Bot-in-a-Box allows for fast and effective adoption of automation for businesses of all sizes. 

Enabling Business Messages with Bot-in-a-Box can be as simple as leveraging an existing customer FAQ document you already have, whether it’s from a web page or an internal document. And since the conversational AI is powered by Business Messages and Dialogflow working together, your chat bot is able to understand and respond to customer questions automatically without the need to write code. 

Bot-in-a-Box also supports other critical journeys like “Custom Intents.” That means that your bot is able to understand the different ways customers express a similar question and respond accurately by using machine learning capabilities.

2 Custom Intents match and respond.jpg
Custom Intents match and respond accurately to variations in customer input.

Finding success 

In April 2021, Wake County courthouses in North Carolina partnered with Tango Technology to implement Business Messages when it became apparent that being able to provide the public and attorneys with around-the-clock access to information would significantly reduce the pressure on courthouse staff. Using Bot-in-a-Box, Tango Technology was able to customize a solution for Wake County Courthouse, Justice Center, and Clerk of Superior Court in just four days.

“With the combination of Google’s Business Messages, GCP, and Dialogflow, we were able to spin up an AI-driven bot for the courts in days. And the technology stack allows us to continually improve by adding functionality in an agile process.”— Mike Lotz, Co-Founder, Tango Technology

3 The public can search and chat.png
The public can search and chat with the Wake County Justice Center any time of the day.

With Business Messages, North Carolina courthouses saw a 37% decrease in the call volume handled by courthouse staff.  With 398,298 fewer phone calls during the first year of operation, the AI-based messages helped Wake County Courthouse work more efficiently and productively.

We’ve seen many brands benefiting from AI-powered Business Messages. For instance, Levi’s saw a 30% increase in off-hours shoppers and surpassed 85% customer satisfaction scores after implementing Business Messages. They also drove 30x more store-related questions than Levi’s website chat. 

Bring Google’s conversational AI to your storefront with Business Messages

Google’s Business Messages makes it easier for businesses of all sizes to engage their existing or potential customers in a virtual conversation, when and where they need it. 

To learn more, watch our Cloud Next session here or visit us at g.co/businessmessages. We have specialized services to help you get started  and can share the wisdom of our channel partners and dedicated experts who specialize in unleashing the potential of conversational AI.

Case Study

The Fantastic Story of How BMG Enables a Micropayments Strategy So Music Artists Get Paid

7447

Of your peers have already read this article.

3:30 Minutes

The most insightful time you'll spend today!

BMG has the tough job of ensuring music artists get paid royalty every time their music is bought, downloaded or streamed. Here's how it does that today, seamlessly, and across the world. And how it's cloud platform now allows it to create additional channels of revenue.

The music industry is rapidly changing. Only 20 years ago, the availability of music and the infrastructure that was required to make an album a sales success were incredibly complex and expensive. With the decline of physical sales and a fundamental shift to digital, music streaming now accounts for more than half of all sales globally.

At the same time, technology has democratized music-making; in many ways, it has made the musical landscape more diverse. Artists can upload their music with the click of a button. But while it’s easier for creators to share their songs with audiences, getting paid has become more fragmented.

Although music is booming, people no longer buy it outright. Instead, listeners download music digitally or subscribe to streaming services to have their libraries with them at all times. To monetize digital content effectively, artists need to know when, where, and how often their songs are played on each service. To help them navigate this complicated royalties landscape and maximize their profits, Berlin-based international music company BMG provides customized, transparent, and fair services to songwriters and artists.

With publishing and recording divisions under one roof, the subsidiary of international media giant Bertelsmann works with both emerging artists and established stars, including John Legend, Kylie Minogue, Mick Jagger, and Keith Richards. With the MyBMG web and mobile application, clients can view and analyze their royalty details in real time and collect payment. When a new record is released, BMG uses data to maximize its impact and revenue for its creators.

“We make sure that everyone who uses our clients’ music knows who needs to be paid the associated royalties, then we collect these royalties and share them out quickly and transparently,” explains Sebastian Hentzschel, Chief Information Officer at BMG. “When our artists release new music, we make sure that it’s marketed and promoted effectively around the world.”

“We needed a scalable solution for our royalty workloads that was intuitive for our developers. We also wanted a partner, not a client-vendor relationship. With autoscaling via BigQuery, excellent customer support, and a clean and simple user interface, Google Cloud ticks every box for us.”

Gaurav Mittal, Vice President Group Technology, BMG

Getting up to speed with a new way of paying artists

In this digital world, artists aren’t just paid every time a fan buys an album—they’re paid a small amount, or royalty, for each song downloaded or streamed by a listener. So, when the industry shifted to digital, the volume of data that BMG needed to handle grew exponentially. “One CD sale is equivalent to about 1,500 streamed songs or plays,” says Gaurav Mittal, Vice President Group Technology at BMG. “That means IT departments have to process 1,500 times the amount of data to calculate payments for artists, and this makes scalable micropayment processing very important.”

Until 2019, BMG’s infrastructure was entirely hosted on-premises. Hardware limitations made it challenging to scale on-demand, making it harder to handle the data peaks that royalty processing can bring. “With our on-premises infrastructure, we were going to hit a ceiling in a few years,” says Gaurav. “We still managed to process royalty payments for our clients, but it was increasingly time consuming and expensive. To keep focusing on our clients, rather than our infrastructure, we decided to migrate to Google Cloud.”

From the outset, Gaurav and his team had a clear vision for the partnership: “Most importantly, we needed a scalable solution for our royalty workloads that was intuitive for our developers. We also wanted a partner, not a client-vendor relationship,” he says. “With autoscaling via BigQuery, excellent customer support, and a clean and simple user interface, Google Cloud ticks every box for us.”

Keeping artists happy with business-as-usual payouts during migration

To move applications to the cloud while keeping payment cycles on track for its artists, BMG teamed up with Google Cloud partner Rackspace Technology. “We selected Rackspace Technology because it combines strong technical muscle and a global footprint, with the customer service of a local boutique firm,“ shares Gaurav.

BMG’s own technology team put together the outline for the Google Cloud architecture, which they passed on to Rackspace Technology for optimizations and the ultimate stamp of approval. Whenever Gaurav and his developers needed support, Rackspace Technology was ready to go. “So far, we’ve migrated 17 applications successfully, and Rackspace Technology has been 100% spot-on with each suggestion,” says Gaurav. “I can’t recall a single flaw in a Rackspace Technology-approved architecture, and that really says something.”

When BMG began its migration in August 2019, the team developed an ambitious two-year plan. Only 14 months later, however, the project is more than 75% complete. Among the solutions that BMG is using today are Cloud Storage to securely store 130 TB of data, and Cloud SQL as its standard database technology. The web applications run on Compute EngineApp Engine, and Google Kubernetes Engine.

“After successfully moving a few applications, it was clear that with the strong teamwork of BMG and Rackspace Technology, together with the ease of use of Google Cloud, we could speed up the project without sacrificing quality,” says Gaurav. “We’re set to complete our migration six months before schedule, helping us to quickly move out of our hybrid environment.”

“Our income-tracking teams are very savvy on the data, and the simplicity of Google Cloud empowers them to self-serve analytics, rather than wait for IT. Our teams are much more productive.”

Gaurav Mittal, Vice President Group Technology at BMG

Royalty reporting and processing with BigQuery and Dataproc

So far, all of BMG’s critical workloads are up and running on Google Cloud. Royalty calculations, for example, which require incredible processing power and the collection of many micropayments to ensure full and timely payout, run entirely on Dataproc with output stored on BigQuery for downstream integration and reporting.

Enabling more harmonious workflows through self-serve analytics

As the new beating heart of BMG’s royalty reporting, BigQuery changed the rhythm of collaboration company-wide. In the past, income tracking teams had to contact IT departments if they needed deeper data insights for their work. By integrating Data Catalog with BigQuery, BMG has made the data more accessible to all teams. This helps them detect missing income and new revenue streams independently, maximizing profits for artists.

“Our income-tracking teams are very savvy on the data, and the simplicity of Google Cloud empowers them to self-serve analytics, rather than wait for IT,” says Gaurav. “Our teams are much more productive.”

“Google Cloud enables us to be more client focused and deliver better features faster. We believe it’s just the beginning. We offer rights and royalty services for music publishing, recorded music, neighboring rights, and books. Without scalability limitations, it’s absolutely conceivable to offer our platform as a service to other companies or industries, such as gaming. Google Cloud has opened a world of possibilities.”

Sebastian Hentzschel, Chief Information Officer, BMG

With a leaner IT environment, BMG can focus its effort on the needs of its clients. Beyond improvements in royalty processing, it can concentrate on app development, releasing new features and enhancements more frequently. By hosting applications on Google Kubernetes Engine, App Engine, and Compute Engine, BMG has built a CI/CD pipeline with automated deployments and testing to significantly speed up workflows.

“In our old system, it could take several weeks to set up an environment,” says Gaurav. “With Google Kubernetes Engine, any of our developers can complete the process in a few clicks. Having that autonomy makes our developers more motivated and self-driven.”

“Google Cloud enables us to be more client focused and deliver better features faster,” adds Sebastian. “We believe it’s just the beginning. We offer rights and royalty services for music publishing, recorded music, neighboring rights, and books. Without scalability limitations, it’s absolutely conceivable to offer our platform as a service to other companies or industries, such as gaming. Google Cloud has opened a world of possibilities.”

With the migration almost complete, BMG is looking forward to its next technology project. It plans to leverage AutoML to further scale and automate royalty tracking with machine learning. On the marketing side, advanced analytics will help BMG determine the effectiveness of promotional campaigns around the world, further increasing profits for artists. By connecting Google Data Studio to BigQuery, BMG will increase the quality of its analyses, helping musicians better understand the reach of their music around the world.

In the end, Gaurav shares, helping musicians is what it all comes down to. “We’re a new kind of music company because we build our services around our artist, songwriter, and publisher clients, not the other way around,” he says. “Google Cloud is helping us maintain strong relationships with our clients, and that’s music to our ears.”

2989

Of your peers have already watched this video.

39:00 Minutes

The most insightful time you'll spend today!

Case Study

Verizon Media Shows the Solution Architecture it Uses for a 100+ PB Analytics Platform

Verizon Media owns and operates more than a dozen brands including Yahoo Mail, Yahoo News, AOL, Huffington Post, TechCrunch, and Engadget among others.

These web properties are visited by millions of users on a daily basis.

In this session, Shakil Memon, Customer Engineer, Google Cloud and Nikhil Mishra, Sr Director, Engineering, Verizon Media, present how Verizon Media is generating actionable insights from the wealth of data that they have.

They will discuss:

  • Challenge with large scale and large volumes of data
  • Basic principles of a data warehousing and data analytics project
  • Showcase a reference architecture
  • A complete end-to-end solution architecture for building a 100+ PB internet-scale analytics platform on Google Cloud, including how Looker fits in the end-to-end solution and how actionable insights are generated from data.

They will also provide unique perspectives, behind-the-scenes thinking, and some insight on how the architecture has evolved in its current form over the years.

6815

Of your peers have already watched this video.

2:30 Minutes

The most insightful time you'll spend today!

Explainer

What is BigQuery?

BigQuery is Google Cloud’s enterprise data warehouse designed to help you ingest, store, analyze, and visualize big data with ease.

Organizations rely on data warehouses to aggregate data from disparate sources, process it, and make it readily available for data analysis that supports their strategic decision-making.

You can ingest data into BigQuery either through batch uploading or streaming data directly to deliver real-time insights.

As a fully-managed data warehouse, Google takes care of the infrastructure so you can focus on analyzing your data up to petabyte scale.

BigQuery supports the same Structured Query Language, or SQL, for analyzing your data, which you may be familiar with if you’ve worked with ANSI-compliant relational databases in the past.

If you’re looking to create machine learning models using your enterprise data, you can do so with BigQuery ML.

With only a few lines of SQL, you can train and execute models on your BigQuery data without needing to move it around.

When it comes time to visualize your data, BigQuery integrates with Looker, as well as several other business intelligence tools across our partner ecosystem.

Now, how do you use BigQuery?

Luckily, it’s straightforward to get up and running with BigQuery.

After creating a GCP project, you can immediately start querying public data sets, which Google Cloud hosts and makes available to all BigQuery users, or you can load your own data into BigQuery to analyze.

Interacting with BigQuery to load data, run queries, or even create ML models can be done in three different ways.

First is by using the UI and the Cloud Console. Second is by using the BigQuery command line tool. And third is by making calls to the BigQuery API, using client libraries available in several languages.

BigQuery is integrated with Google Cloud’s Identity and Access Management Service so you can securely share your data and analytical insights across the organization.

What does it cost to use BigQuery?

With BigQuery, you pay for storing and querying data and streaming inserts.

Loading and exporting data are free of charge.

Storage costs are based on the amount of data stored and have two rates based on how often the data is changing.

Query costs can be either on demand, meaning you are charged per query by the amount of data processed, or flat rate for customers who want to purchase dedicated resources.

More Relevant Stories for Your Company

How-to

How TeamSnap Improved Return on Ad Spend Significantly

Anyone who has ever coached or played on a sports team, or had a child involved in sports, knows how difficult scheduling and logistics can be. From game and practice schedules, to uniforms and who’s bringing the snacks, it can be a lot for coaches, administrators, parents, and players to

Blog

Predictive Model Built on Google Cloud Helps You Get a 7-day Mosquito Forecast Report!

Mosquitoes aren’t just the peskiest creatures on Earth; they infect more than 700 million people a year with dangerous diseases like Zika, Malaria, Dengue Fever, and Yellow Fever. Prevention is the best protection, and stopping mosquito bites before they happen is a critical step. SC Johnson—a leading developer and manufacturer

Case Study

Bit Capital Rolls Out a Digital Financial Solution in Under 3 Months and at 2/3 the Cost

In the past few years, a series of new technologies and regulatory changes has been transforming Brazil’s financial industry. The concept of blockchain added security and agility to financial transactions. The open banking system being deployed by the country’s Central Bank (BC) allows for platform integration and data sharing -with

Case Study

How Constellation Brands’ Direct-to-Customer Tech Delivers Economic Impact across Business Portfolio

Editor’s note: Today we’re hearing from Ryan Mason, Director, Head of DTC Growth & Strategy, at alcoholic beverage firm, Constellation Brands on the company’s shift to Direct-to-Consumer (DTC) sales and how Google Cloud’s powerful technology stack helped with this transformation.  It’s no secret that consumer businesses have been up-ended in

SHOW MORE STORIES