What's New in BigQuery, Google Cloud's Modern Data Warehouse - Build What's Next

2912

Of your peers have already watched this video.

36:00 Minutes

The most insightful time you'll spend today!

Explainer

What’s New in BigQuery, Google Cloud’s Modern Data Warehouse

The demands that today’s enterprise has from data go far beyond the capabilities of traditional data warehousing. For many leaders, the need to digitally transform their businesses is a key driver for data analytics spending.

Businesses want to make real-time decisions from fresh information as well as make predictions from their data in order to remain competitive.

In this video, watch Sudhir Hasbe, Director of Product Management, Data Analytics, Google Cloud and Tino Tereshko, Product Manager, BigQuery, Google Cloud demonstrate what’s new in BigQuery, Google Cloud’s enterprise data warehouse, and hear about all the latest feature innovations and see some amazing demos.

Case Study

How Ather Energy is leveraging the Cloud to build and scale smart mobility solutions for India

8391

Of your peers have already read this article.

8:30 Minutes

The most insightful time you'll spend today!

Ather Energy, India’s first-ever electric scooter, turns to Google Cloud to support the smooth running of its vehicles, lower costs, improve time to market, and create a great customer experience.

In 2013, long before the world was discussing clean energy and sustainable practices, two IIT Madras graduates — Swapnil Jain and Tarun Mehta — had an idea to develop India’s first-ever electrical scooter.

This was at a time when auto manufacturers were still focusing on fossil-fuel-driven vehicles and ‘eco-friendly’ mobility solutions were more a trendy alternative catering to a niche market.

The duo founded Ather Energy in 2013 and launched their first fully-electric scooter, the Ather S340, in Bengaluru in 2016. Since then, the company has released several new models into the market and is planning to expand to eight more cities by the end of the year.

To support the smooth running of their vehicles, lower costs, improve time to market, and create great customer experience, Ather turned to Google Cloud.

Read the Full Story on YourStory

3038

Of your peers have already watched this video.

24:00 Minutes

The most insightful time you'll spend today!

Case Study

How to Modernize Your Data infrastructure and Analytics: Case Study

Data modernization is key to digital transformation. Legacy systems are slow, expensive and cannot keep up with changing requirements. Data teams often spend more time on data pipelines than they do on data analysis or data science.

in this video, Harish Ramachandraiah, Director Eng. & Analytics, Sunrun, speaks with Joel Mckelvey, Product Marketing Manager, Google Cloud. They discuss how Sunrun leveraged Looker and BigQuery to reduce the complexity of legacy extract-transform-load processes, improve database performance, and adapt quickly to changes in data.

Now Sunrun makes data accessible where it’s needed, in near-real time.

Mckelvey shows how a modern data stack helps companies simplify the data pipeline, consolidate vital data, and accelerate analytics performance while improving agility, efficiency, and data governance.

Case Study

Mrs. T’s Pierogies Moves SAP Systems to Google Cloud for Faster Analytics Capabilities for its SAP Data

6846

Of your peers have already read this article.

1:30 Minutes

The most insightful time you'll spend today!

Leading manufacturer of frozen Pierogies, Mrs. T’s Pierogies seamlessly migrated its legacy on-prem infrastructure to SAP S/4HANA solution on Google Cloud with minimal downtime. This created fast and flexible analytical capabilities for SAP data!

Pierogies might just be the ultimate comfort food. But when Mrs. T’s Pierogies — the leading manufacturer of frozen pierogies in the US — learned it needed to transition its existing on-premises SAP ECC to S/4HANA, the company sought a little comfort for itself. 

Founded in 1952, Mrs. T’s Pierogies now produces more than 650 million pierogies a year. Moving that many pierogies requires a powerful ERP system — and an equally powerful IT infrastructure on which to run it. Mrs. T’s realized that the SAP-mandated transition of its ERP solution to SAP S/4HANA was an opportunity to move its SAP systems to Google Cloud and gain real-time analytics capabilities for faster sales forecasting, more effective trade promotions, and more sophisticated planning.

From necessity to opportunity

Mrs. T’s successfully ran its SAP ECC solution on its own on-premises servers for years. But after SAP decided to sunset the product, Mrs. T’s realized that it faced multiple challenges, including: 

  • Migrating its SAP ECC 6.0 system from an on-premises leagcy OS to a cloud-based Linux environment
  • Moving data from its SAP DB2 database to HANA
  • Transitioning from ECC to SAP S/4HANA

That complex transition needed to take place with little or no downtime, since nearly all of the company’s invoices, warehouse movements, and transfer orders used the Electronic Data Interchange (EDI) protocol. Missing even a few hours of EDI transactions would put significant revenue at stake. Adding to the challenge: Some Mrs. T’s customers would not accept an invoice past five days, which left little room for error. Mrs. T’s chose Rackspace Technology, a longtime Google Cloud partner, to oversee the move. 

Mrs. T’s could have chosen to run S/4HANA on its legacy hardware but saw migration to Google Cloud as an opportunity to improve key aspects of its business, in particular data analytics. Historically, sales planning and forecasting involved time-consuming manual processes. But the speed, availability, and scalability of Google Cloud meant that Mrs. T’s could take advantage of S/4HANA’s embedded analytics capabilities. Migrating could also open the door to leveraging Google Cloud’s native integration of SAP data to power Google tools such as BigQueryGoogle Cloud AI Building Blocks, and more. 

The move to Google Cloud also gave Mrs. T’s an opportunity to update its disaster recovery process. Previously, the company backed up to tape. So, if its SAP systems went down, a member of the IT department would have to drive the most recent tape backups an hour to its cold site, where the disaster recovery partner would load the SAP backup tape, boot the system up, switch network connections to that site, and cross their fingers. Not only would downtime be significant, but restored data would be limited to the periodic tape backup. 

“Everything just worked”

Once Mrs. T’s decided to migrate to Google Cloud, the company worked with Rackspace Technology to implement the system. The first phase focused on moving the SAP production environment from on-premises infrastructure to Google Cloud and updating its database to HANA, which took place over 12 weeks. The switchover occurred over a weekend and was all but invisible to users. “We came in on Monday and everything just worked,” recalls Timothy Coyle, Director of Information Systems & Technology. In a second four-month phase, the company transitioned from ECC to S/4. 

The move to Google Cloud paid dividends immediately. Batch transactions occurred twice as fast and on-screen end-user transactions rendered instantly. The upgrade also gave the finance team access to embedded analytics and monitoring for the first time. With everything now in the cloud, disaster recovery could be dynamic and nearly instantaneous, with a worst-case scenario of just 5 to 10 minutes of downtime. 

“Mrs. T’s needed a skilled and experienced partner that could move its SAP environment to Google Cloud with no negative impacts to its business. We knew this migration was a key initiative in Mrs. T’s digital transformation journey,” says Chuck Britton, Google Partner Development Manager at Rackspace. “We also knew that running SAP on Google Cloud would give the business the fast and flexible analytical capabilities it needed for its SAP data.” 

From ideation to production, Mrs. T’s migrated from its legacy on-premises infrastructure to a modern SAP S/4HANA solution on Google Cloud in only seven months, with minimal downtime and zero disruptions. Now that the migration is complete, Mrs. T’s has a flexible, highly scalable environment to run the SAP applications and data that fuel the business. Says Coyle, “Our strategic intent for IT is to make processes simpler, people more productive, and infrastructure more secure. This project fits right square in the middle of that strategic philosophy.”

Learn more about ways in which Google Cloud can transform your SAP experience and about Rackspace Google Cloud solutions for SAP customers.

6413

Of your peers have already watched this video.

2:00 Minutes

The most insightful time you'll spend today!

Blog

CCAI Insights: Answer Customers’ Queries & Understand Them Better with Conversation Data

With CCAI Insights, businesses can drive contact center efficiency, solve customer problems and leverage data from customer interactions to understand them better!

CCAI Insights, a core piece of the Google Cloud’s Contact Center AI product suite is built to help contact center management dive into data to adjust business needs, preempt problems with timely analysis of customer conversations and keep agents prepared. Additionally, businesses can automatically feed data into Insights from other areas of CCAI like Dialogflow CX or another product sources. Watch the video to find out more benefits and capabilities of CCAI Insights in elevating CX.

Case Study

Tyson Foods’ Story of Unlocking Opportunities by Integrating Real-time Analytics with AI and BI

6489

Of your peers have already read this article.

3:00 Minutes

The most insightful time you'll spend today!

Forward-thinking companies look beyond the here and now to solve and unlock opportunities to drive future business growth. Google Cloud-hosted, Ingestion platform based on analytics integrated with AI & BI helps Tyson Foods turn data into insights!

As data environments become more complex, companies are turning to streaming analytics solutions that analyze data as it’s ingested and deliver immediate, high-value insights into what is happening now. These insights enable decision makers to act in real time to take advantage of opportunities or respond to issues as they occur.

While understanding what is happening now has great business value, forward-thinking companies are taking things a step further, using real-time analytics integrated with artificial intelligence (AI) and business intelligence (BI) to answer the question, “what might happen in the future?” Arkansas-based Tyson Foods has embraced AI/BI analytics to enable predictive insights that unlock new opportunities and drive future growth.

Creating a digital twin for connected intelligence company wide

Before using AI/BI, Tyson’s analytics capabilities consisted of traditional BI solutions focused on KPIs and simplifying data so that humans could understand it. Tyson wanted to leverage its data to uncover ways to improve current processes and grow its business. But with BI alone, Tyson struggled to use data to run the simulations and scenarios essential to make educated decisions. To keep growing, it had to embrace the complexity of its data, building ways to analyze it and use it to inform decision making. 

Tyson’s on-premises analytics solutions limited its ability to be aggressive and make intelligent, timely, prescriptive decisions. The solution was to create a digital twin to scale optimizations within business processes, moving from local optimizations to system-wide connected optimizations. Doing so meant shifting entirely to cloud computing, with an initial focus on building the ingestion component of the digital twin platform.

Investing in a digital twin enabled Tyson to accelerate new capabilities like supply chain simulation “what-if” scenarios, prescriptive price elasticity recommendations, and improvement of customer intimacy. 

Solving the ingestion problem for faster time to insights

Before its migration to Google Cloud, analytics projects that Tyson suffered from uncertainty over how to obtain the data. This problem was prolific and caused project times to be extended for weeks or even months due to the need to write and support one-off data ingestion processes at the front end. This problem also prevented the IT team from delivering analytics solutions fast enough for the business to take full advantage of them. 

To solve this analytics problem, the team created Data Ingestion Compute Engine (DICE). DICE is a Google Cloud-hosted, open-source, cloud-native ingestion platform developed to provide configuration-based, no-ops, code-free ingestion from disparate enterprise data systems, both internal and external. It is centered on three high-level goals:

  1. Accelerate the speed of delivery of IT analytics solutions
  2. Enable growth of IT capabilities to produce meaningful insight
  3. Reduce long-term total cost of ownership for ingestion solutions

Creating DICE ingestion platform with Google Cloud services

Teams use DICE to set up secure data ingestion jobs in minutes without having to manage complex connections or write, deploy, and support their own code. DICE enables unbound scale, highly parallel processing, DevSecOps, open source, and the implementation of Lambda Data Architecture.

A DICE job is the logical unit of work in the DICE platform, consisting of immutable and mutable configurations persisted as JSON documents stored in Firestore. The job exists as an instruction set for the DICE data engine, which is Apache Beam running Dataflow to instruct which data to pull, how to pull it, how often to pull it, how to process it, when it changes, and where to direct it.

Two of DICE’s primary layers include the metadata engine and the data engine. The metadata engine is responsible for the creation and management of DICE job configuration and orchestration. It is made up of many microservices that interact with multiple Google Cloud services, including the job configuration creation API, job build configuration helper API, and job execution scheduler API.

The data engine is responsible for the physical ingestion of data, the change detection processing of that data, and the delivery of that data to specified targets. The data engine is Java code that uses the Apache Beam unified programming model and runs in Dataflow. It is comprised of streaming, jobs, and Dataflow flex template batch jobs. Logically, the data engine is segmented across three layers: the inbound processing layer, the DICE file system layer, and the target processing layer, which takes the data from the DICE file system and moves it to targets.

DICE @ Tyson Platform in Numbers

Rolling DICE for thousands of ingestion jobs each day

DICE was first deployed to a production environment in November 2019, and just two years later, it has more than 3,000 data ingestion jobs from more than a hundred disparate data systems, both internal and external to Tyson Foods. Most of these jobs run multiple times a day. On a daily basis the DICE environment sees more than 25,000 Dataflow jobs running and an average of 3.25 terabytes of new data being ingested.

DICE @ Tyson Platform in Numbers

DICE supports ingestion from many different types of technologies, including BigQuery, SQL Server, SAP HANA, Postgres, Oracle, MySQL, Db2, various types of file systems, and FTP servers. Additionally, DICE supports target platform technologies for ingestion jobs that include multiple JDBC targets, multiple file system targets, and BigQuery and queue-based store and forward technologies. 

The platform continues to see linear growth of DICE jobs, all while keeping platform costs relatively flat. With increasing demand for the platform, Tyson’s IT team is constantly enhancing DICE to support new sources and targets.

This intelligent platform keeps adding new value and makes it simple for Tyson to take advantage of its data. This innovation is a necessity in this fast-changing world of digital business in which companies must transform a high volume of complex data into actionable insight.

More Relevant Stories for Your Company

Explainer

Analytics in a Multi-Cloud World with BigQuery Omni

Enterprises have more data at hand than they have ever had in the past. But are unable to leverage it fully. “We have all of this data at our fingertips. But we just can't quite get to it because we're living in this world of data silos,” says Emily Rapp,

Blog

Explore the Innovations and Architecture Powering Spanner and BigQuery

Previously, databases had architectures with tightly coupled storage and compute. This resulted in higher latency, and with faster networks these constraints no longer surface. With Google Cloud's BigQuery and CloudSpanner, the storage and compute architecture have been separated, allowing for better scalability and availability to address businesses' high throughput data

Blog

Google’s Intelligent Products Essentials Assist Manufacturers in Product Development Journey

Expectations for both consumer and commercial products have changed. Consumers want products that evolve with their needs, adapt to their preferences, and stay up-to-date over time. Manufacturers, in turn, need to create products that provide engaging customer experiences not only to better compete in the marketplace, but also to provide

Webinar

Chronicle Security Analytics: Key to Address Security Data Overload

Google Cloud's Chronicle is a security analytics platform built for modern use cases to combat modern threats. In today's world, enterprises have undergone significant changes in all aspects, and must adapt to their security needs to counter threats and attacks. Watch the video to learn Google Cloud's initiative to help

SHOW MORE STORIES