From the historic Gold Country to the rugged heights of the Sierra Nevada, Placer County encompasses more than 1,500 square miles—and provides services to nearly 400,000 residents. Across the county, residents access resources in person, over the phone, and through the county website. In 2018, the county piloted a suite of eServices through its Community Development Resource Agency (CDRA) with the goal of helping its geographically dispersed population more easily apply for permits, make appointments, and get immediate answers to specific questions.

Now, with the help of Google Cloud Dialogflow and Speech-to-Text API, the county has created a virtual agent that enables anyone, anywhere, to simply start a conversation by saying “Ask Placer County” into a variety of voice assistant hardware. This virtual agent builds on the success of eServices and helps the public access information through their smartphones or smart home devices, which is especially helpful for individuals without immediate access to a computer—such as the traveling public or a contractor working on-site, for example.

The pilot program has been an opportunity to explore new technology and create another communication channel to our citizens. “Our ambitious and long-term goal is that ‘Ask Placer County’ will be like having a personal assistant to everything related to Placer County. While the virtual agent is currently limited to a few departments, we plan to expand it countywide,” says Ben Palacio, senior IT analyst for Placer County.

“Our ambitious and long-term goal is that ‘Ask Placer County’ will be like having a personal assistant to everything related to Placer County. While the virtual agent is currently limited to a few departments, we plan to expand it countywide.”

—Ben Palacio, Senior IT Analyst, Placer County

New eServices get a boost from AI virtual agent

The CDRA, one of many departments and agencies within Placer County, deployed interactive eServices for residents. As part of this initiative, the CDRA made a special request to the information technology (IT) department: Build an AI-based virtual agent that would spotlight the eServices on the website and make them easily available through a conversational interface.

“Having a specific technology requirement voiced by an agency was visionary and very welcome. It meant they were engaged and excited about the possibilities that these new technologies had to offer,” says Mike Spak, IT Manager for Placer County.

Today, the CDRA’s eServices help residents answer important questions, such as: “Is my land zoned for adding a garage?” “What historical engineering work has been done on my property?” And, “How can I make an appointment at a permitting office?”

“Having a specific technology requirement voiced by an agency was visionary and very welcome. It meant they were engaged and excited about the possibilities that these new technologies had to offer.”

—Mike Spak, IT Manager, Placer County

Google Cloud fit easily into Placer County’s multivendor environment

With the help of consultants from Dito, a Google Cloud Premier Partner, Placer County chose Google Cloud to help with “Ask Placer County.”

Placer County runs a multivendor IT environment. “Especially with the cloud, the easier it is to integrate a vendor’s capabilities with others, the better for everybody,” says Palacio. “Google Cloud was able to integrate with other vendors’ capabilities for this project.”

For example, Placer County uses the Google Cloud operations suite (formerly Stackdriver) to monitor, troubleshoot, and improve its cloud infrastructure and application performance. And it uses Dialogflow, which powers the natural language processing (NLP) interface of the “Ask Placer County” virtual agent. Both integrate easily with tools from other vendors.

“Especially with the cloud, the easier it is to integrate a vendor’s capabilities with others, the better for everybody. Google Cloud was able to integrate with other vendors’ capabilities for this project.”

—Ben Palacio, Senior IT Analyst, Placer County

Real-time conversations powered by “Ask Placer County”

Dito got “Ask Placer County” up and running using App Engine and Datastore. Today, the “Ask Placer County” virtual agent provides information ranging from how to adopt a pet to short-term-rental compliance.

Users can say “Ask Placer County” into their smartphones or their computer microphones and access targeted information from the pilot departments. IT staff reviews the asked questions and are constantly updating the virtual agent to provide more accurate and responsive information.

“You could be in your backyard with a contractor trying to get a project started, and if the contractor has a question about permitting or zoning, they can use their smartphone to get an answer right then and there, rather than having to interrupt the meeting, get back in their car, and drive to a county office,” says Palacio.

The county hopes “Ask Placer County” will improve the efficiency of in-person interactions between the public and county employees as well. Because the public can access common questions and information online or through the virtual agent, employees’ interactions with the public can be more focused and productive.

“We are providing more effective and efficient service to our customers through 24/7 access to information and by reducing the proportion of staff’s time in responding to emails and voicemails to answer our customer questions,” says Shawna Purvines, Principal Planner for Placer County.

According to Placer County, the virtual agent currently answers a monthly average of around 200 questions for the CDRA, and the county continues to develop more and more complete answers across participating departments.

“We are providing more effective and efficient service to our customers through 24/7 access to information and by reducing the proportion of staff’s time in responding to emails and voicemails to answer our customer questions.”

—Shawna Purvines, Principal Planner, Placer County

Constantly evolving based on users’ needs

To best serve the needs of the CDRA, and potentially other departments over time, “Ask Placer County” continues to evolve. The county can add and retire questions based on the needs of constituents. This technology can be tailored to address current needs, such as responding to COVID-19. “Our hope is that, in quickly evolving situations, the virtual agent can be a resource for the public to access real-time information about public services,” says Palacio.

As a component of the eServices group, “Ask Placer County” has kept construction and development in Placer County operational during the COVID-19 pandemic, and it continues to provide opportunities for the public to access county resources without coming in to the counters, reducing the need for in-person visits. In fact, according to Placer County data, permit applications have increased by 17%.

Placer County also performs analytics using the Google Cloud operations suite to gain insights into the most popular questions and, just as importantly, the questions that are not being answered. “We get insight into what we’re missing—for those times when the virtual agent can’t find a response in our database,” says Palacio. Placer County uses these insights to improve answers and ultimately improve the efficacy of the virtual agent over time.

“This continual, real-time learning is the exciting part of the application that lets us help constituents in entirely new ways. It’s really cool,” says Palacio.

County IT workers actively analyze CDRA webpages to determine which are the most visited and to craft questions and answers to add to “Ask Placer County.” According to Placer County, the number of questions the virtual agent can answer is now more than 600. But the CDRA pages are just a small subset of the 5,000 pages that make up the Placer County website. A future goal is to provide a backstop for customer questions that can’t be answered, transferring those customers to county staff for resolution.

“Looking ahead, this technology has the ability to provide answers to thousands and thousands of questions,” says Palacio.

“This continual, real-time learning is the exciting part of the application that lets us help constituents in entirely new ways. It’s really cool.”

—Ben Palacio, Senior IT Analyst, Placer County

Scaling “Ask Placer County” countywide

The limited rollout of the virtual agent technology allowed IT to monitor its effectiveness and make initial adjustments. As the virtual agent evolves, the Placer County chief information officer sees opportunities for it to engage the public countywide.

The “Ask Placer County” virtual agent can truly be a concierge, helping the public navigate county resources. It can help citizens become more engaged with their elected officials, it can provide information, and it can more efficiently connect the public with the services they seek.

Some of the future possibilities include providing current information on board meetings and individual elected officials, checking on permit status and burn days, and getting the latest county news. “There are so many ways we can use this solution to tackle issues within the county that we’re going to have to somehow prioritize them,” says Palacio.

Editor’s note: This blog has been updated since it originally published to reflect the total number of generative AI courses that have become available since the publish date.

An AI/ML (artificial intelligence/machine learning) career path can be a great specialty area within the cloud—and one of the most accessible! Because this area is constantly developing, most recently with the rise of generative AI, I want to share some recommendations to help you chart a sustainable career as AI/ML continues to evolve.

Generative AI falls under the overall category of AI and offers a new and exciting way of interacting with information, brands, and other people. Let’s talk about how these disciplines work together and about materials available to help you upskill in these areas.

To this end, we are happy to announce a new set of generative AI training content available at no cost. So, whether you are just getting started or already have a more advanced role, read on to find ways to help reach your desired position. 

ML compared to AI

While the terms ML and AI are often used interchangeably, there are distinct differences. AI is an umbrella concept wherein machines are taught to perform tasks normally associated with human intelligence, such as decision-making and language interaction. ML is a subset of AI dedicated to taking data from the past and training algorithms to create models that can perform highly complex tasks without being explicitly programmed. It’s the basis for most forms of AI that people interact with, like virtual assistants, music recommendations, and chatbots.

Data engineers and ML engineers work with data scientists to get insights from data. They are needed to create software models and get clear results, and develop deployable applications. These skilled roles are needed in every industry!

Change the world for the better with generative AI

I recently wrote about four key pillars of technology trends expected in the next decade, including the role of AI/ML in the cloud environment as one of those pillars, and how you can bridge the skills gap and build your career.

Generative AI is a new type of ML that has made a lot of headlines recently. Research from CIO Dive finds that seven in 10 executives say their companies are investigating or exploring generative AI. Now is a great time to become an expert, while we are at the cusp of this technology becoming more widely adopted. 

To get to generative AI, we need to talk first about deep learning. Deep learning is a subset of ML that uses artificial neural networks to process more complex patterns than traditional ML. Generative AI sits still further down the funnel, as a subset of deep learning that typically involves the Transformer architecture. Essentially, it’s a type of AI that can map long-range dependencies and patterns in large training sets, then use what it learns to produce new content, including text, imagery, audio, and synthetic data.

Generative AI relies on large models, such as large language models (LLMs) that can classify and generate text, answer questions, and summarize documents. For more detail about how this works, check out the video below from Google I/O 2023, which features great information from Dr. Gwendolyn Stripling, Artificial Intelligence Technical Curriculum Developer for Google Cloud.

How Google is offering generative AI

Here at Google, we’ve been heavily invested for decades in research and innovation aimed at helping businesses, governments, and developers maximize the potential of AI. Our vision is to empower builders, innovators, developers, and doers to use AI in unique, responsible, productive ways. 

To deliver on this, we recently announced a variety of solutions to bring generative AI into our offerings, beginning with Generative AI support on Vertex AI and Gen App Builder. 

Vertex AI is Google Cloud’s ML platform for training and deploying ML models and AI applications. With Generative AI support on Vertex AI, data science teams and developers can access foundation models from both Google and other sources, helping them to quickly build, customize, and deploy models for their own use cases. 

As part of generative AI support on Vertex AI, Model Garden and Generative AI Studio are now in preview, including access to PaLM 2 for Text and Chat and Embeddings API for Text, while other features and services are available to select trusted testers. If you’d like early access to Google Cloud’s AI products, join the waitlist here.

We announced our latest PaLM model in production at I/O 2023: PaLM 2. Today, it powers more than 25 products! Additionally, we’ve fine-tuned PaLM 2 for specific use cases, including security and medical domains. You’ve probably also heard of Bard, our experiment for conversational AI, which is fully running on PaLM 2. 

Our Gen App Builder aims to make customer, partner, and employee interactions more effective and helpful. With it, developers — even those with limited data science expertise — can quickly create bots, chat apps, digital assistants, custom search engines, and more. Gen App Builder even makes it possible to create some generative AI apps without any coding skills. 

Now, let’s bring these products to life! Not sure exactly the use case or in need of a little inspiration? Check out this blog from Google Cloud CEO Thomas Kurian to see how customers and partners in the ecosystem are bringing ideas to life with generative AI. And watch this video to see generative AI in action!

Build your AI/ML skills and validate your knowledge to grow your career

There is a lot of excitement around generative AI—it’s truly a brand new path to consider for your AI/ML career! Check out my recommendations below for training options for AI/ML roles. These will help you gain critical skills as generative AI becomes more widely available.

*NEW* training materials, specific to generative AI technologies
Generative AI Learning Path – no cost training 

• [Course] Introduction to Generative AI (1 day)
• [Course] Introduction to Large Language Models (1 day)
• [Course] Attention Mechanism (1 day)
• [Course] Transformer Models and BERT Model (1 day)
• [Course] Introduction to Image Generation (1 day)
• [Course] Create Image Captioning Models (1 day)
• [Course] Encoder-Decoder Architecture (1 day)

Additional AI/ML training – varying learning credits required to complete on Google Cloud Skills Boost 
Introductory level

• [Course with completion badge] How Google Does Machine Learning (1 day)
• [Course with completion badge] MLOps: Getting Started (1 day)
• [Skill badge] Get started with TensorFlow on Google Cloud (8 hours)  
• [Skill badge] Perform foundational ML, AI and data tasks in Google Cloud (7 hours)
• [Course with completion badge] Language, Speech, Text, and Translation with Google Cloud APIs (5 hours)

Intermediate/Multi-level

• [Learning Path] Machine Learning Engineer (Collection of 15 video courses and labs) 
• [Skill badge] Build and Deploy Machine Learning Solutions on Vertex AI (1 day)

Advanced

• [Training + certification exam] Google Cloud Professional ML Engineer Certification (varied time)
• [Skill badge] Machine Learning with TensorFlow in Vertex AI (90 minutes)
• [Course with completion badge] Natural Language Processing in Google Cloud (1 day)

You can also catch up on hands-on demos in our most recent Cloud OnBoard: From Data to AI with BigQuery and Vertex AI. In it, we guide you through all steps of the data-to-AI solution.

Coming up on June 29, we are hosting Getting Started with Vertex Generative AI as part of Innovators Live. Register now!

I also encourage you to consider an Innovators Plus subscription. You get access to over 700 labs, skill badges and courses (including many that are specific to AI/ML), a certification exam voucher, and up to $1,000 in Google Cloud credits. You’ll also get access to live learning events and technical briefings with Google Cloud experts, plus 1:1 consultations (talk with us about your upcoming generative AI project!). It’s a $1,500 package for only $299. Learn more!

October 23rd (this past Saturday!) was my 4th Googlevarsery and we are wrapping an incredible Google Next 2021!

When I started in 2017, we had a dream of making BigQuery Intelligent Data Warehouse that would power every organization’s data driven digital transformation. 

This year at Next, It was amazing to see Google Cloud’s CEO, Thomas Kurian, kick off his keynote with CTO of WalMart, Suresh Kumar , talking about how his organization is giving its data the “BigQuery treatment”.

AS  I recap Next 2021 and  reflect on our amazing journey over the past 4 years, I’m so proud of the opportunity I’ve had to work with some of the world’s most innovative companies from Twitter to Walmart to Home Depot, Snap, Paypal and many others.   

So much of what we announced at Next is the result of years of hard work, persistence and commitment to delivering the best analytics experience for customers. 

I believe that one of the reasons why customers choose Google for data is because we have shown a strong alignment between our strategy and theirs and because we’ve been relentlessly delivering innovation at the speed they require. 

Unified Smart Analytics Platform 

Over the past 4 years our focus has been to build industries leading unified smart analytics platforms. BigQuery is at the heart of this vision and seamlessly integrates with all our other services. Customers can use BigQuery to query data in BigQuery Storage, Google Cloud Storage, AWS S3, Azure Blobstore, various databases like BigTable, Spanner, Cloud SQL etc. They can also use  any engine like Spark, Dataflow, Vertex AI with BigQuery. BigQuery automatically syncs all its metadata with Data Catalog and users can then run a Data Loss Prevention service to identify sensitive data and tag it. These tags can then be used to create access policies. 

In addition to Google services, all our partner products also integrate with BigQuery seamlessly. Some of the key partners highlighted at Next 21 included Data Ingestion (Fivetran, Informatica & Confluent), Data preparation (Trifacta, DBT),  Data Governance (Colibra), Data Science (Databricks, Dataiku) and BI (Tableau, PowerBI, Qlik etc).

Planet Scale analytics with BigQuery

BigQuery is an amazing platform and over the past 11 years we have continued to innovate in various aspects. Scalability has always been a huge differentiator for BigQuery. BigQuery has many customers with more than 100 petabytes of data and our largest customer is now approaching  an exabyte of data. Our large customers have run queries over trillions of rows. 

But scale for us is not just about storing or processing a lot of data. Scale is also how we can reach every organization in the world. This is the reason we launched BigQuery Sandbox which enables organizations to get started with BigQuery without a credit card. This has enabled us to reach tens of thousands of customers. Additionally to make it easy to get started with BigQuery we have built integrations with various Google tools like Firebase, Google Ads, Google Analytics 360, etc. 

Finally, to simplify adoption we now provide options for customers to choose whether they would like to pay per query, buy flat rate subscriptions or buy per second capacity. With our autoscaling capabilities we can provide customers best value by mixing flat rate subscription discounts with auto scaling with flex slots.

Intelligent Data Warehouse to empower every data analyst to become a data scientist

BigQuery ML is one of the  biggest innovations that we have brought to market over the past few years. Our vision is to make every data analyst a data scientist by democratizing Machine learning. 80% of time is spent in moving, prepping and transforming data for the ML platform. This also causes a huge data governance problem as now every data scientist has a copy of your most valuable data.  Our approach was very simple.  We asked:”what if we could bring ML to data rather than taking data to an ML engine?” 

That is how BigQuery ML was born. Simply write 2 lines of SQL code and create ML models. 

Over the past 4 years we have launched many models like regression, matrix factorization, anomaly detection, time series, XGboost, DNN etc. These  models are used by customers to solve complex  business problems simply from segmentation, recommendations, time series forecasting, package delivery estimation etc. The service is very popular: 80%+ of our top customers are using BigQueryML today.  When you consider that the average adoption rate of ML/AI is in the low 30%, 80% is a pretty good result!

We announced tighter integration of BQML with Vertex AI. Model explainability will provide the ability to explain the results of predictive ML classification and regression models by understanding how each feature contributes to the predicted result. Also users will be able to manage, compare and deploy BigQuery ML models in Vertex; leverage Vertex Pipelines to train and predict BigQuery ML models.

Real-time streaming analytics with BigQuery 

Customer expectations are changing and everyone wants everything in an instant: according to Gartner, by the end of 2024, 75% of enterprises will shift from piloting to operationalizing AI, driving a 5X increase in streaming data and analytics infrastructures.

The BigQuery’s storage engine is optimized for real-time streaming. BigQuery supports streaming ingestion of 10s of millions of events in real-time and there is no impact on query performance. Additionally customers  can use materialized views and BI Engine (which is now GA) on top of streaming data. We guarantee always fast, always fresh data. Our system automatically updates MVs and BI Engine. 

Many customers also use our PubSub service to collect real-time events and process these through Dataflow prior to ingesting into BigQuery. This is a streaming ETL pattern which is very popular. Last year,we announced PubSub Lite to  provide customers with a 90% lower price point and aTCO that is lower than any DIY Kafka deployment. 

We also announced Dataflow Prime, it is our next generation platform for Dataflow. Big Data processing platforms have only focused on horizontal scaling to optimize workloads. But we have seen new patterns and use cases like streaming AI where you may have a few steps in pipelines that perform data prep and then customers  have to run a GPU based model. Customers  want to use different sizes and shapes of machines to run these pipelines in the most optimum manner. This is exactly what Dataflow Prime does. It delivers vertical auto scaling with the right fitting for your pipelines. We believe this should lower costs for pipelines significantly.

With Datastream as our change data capture service (built on Alooma technology), we have solved the last key problem space for customers. We can automatically detect changes in your operational databases like MySQL, Postgres, Oracle etc and sync them in BigQuery.

Most importantly, all these products work seamlessly with each other through a set of templates. Our goal is to make this even more seamless over next year. 

Open Data Analytics with BigQuery

Google has always been a big believer in Open Source initiatives. Our customers love using various open source offerings like Spark, Flink, Presto, Airflow etc. With Dataproc & Composer our customers have been able to run various of these open source frameworks on GCP and leverage our scale, speed and security. Dataproc is a great service and delivers massive savings to customers moving from on-prem Hadoop environments. But customers want to focus on jobs and not clusters. 

That’s why we launched Dataproc Serverless Spark (GA) offering at Next 2021. This new service adheres to one of our key design principles we started with: make data simple.  

Just like with BigQuery, you can simply RUN QUERY. With Spark on Google Cloud, you simply RUN JOB.  ZDNet did a great piece on this.  I invite you to check it out!

Many of our customers are moving to Kubernetes and wanted to use that as the platform for Spark. Our upcoming Spark on GKE offering will give the ability to deploy spark workloads on existing Kubernetes clusters.  

But for me the most exciting capability we have is, the ability to run Spark directly on BigQuery Storage. BigQuery storage is highly optimized analytical storage. By running Spark directly on it, we again bring compute to data and avoid moving data to compute. 

BigSearch to power Log Analytics

We are bringing the power of Search to BigQuery. Customers already ingest massive amounts of log data into BigQuery and perform analytics on it. Our customers have been asking us for better support for native JSON and Search. At Next 21 we announced the upcoming availability of both these capabilities.

Fast cross column search will provide efficient indexing of structured, semi-structured and unstructured data. User friendly SQL functions let customers rapidly find data points without having to scan all the text in your table or even know which column the data resides in. 

This will be tightly integrated with native JSON, allowing customers to get BigQuery performance and storage optimizations on JSON as well as search on unstructured or constantly changing  data structures. 

Multi & Cross Cloud Analytics

Research on multi cloud adoption is unequivocal — 92% of businesses in 2021 report having a multi cloud strategy. We have always believed in providing customers choice to our customers and meeting them where they are. It was clear that all our customers wanted us to take our gems like BigQuery to other clouds as their data was distributed on different clouds. 

Additionally it was clear that customers wanted cross cloud analytics not multi-cloud solutions that can just run in different clouds. In short, see all their data with a single pane of glass, perform analysis on top of any data without worrying about where it is located, avoid egress costs and finally perform cross cloud analysis across datasets on different clouds.

With BigQuery Omni, we deliver on this vision, with a new way of analyzing data stored in multiple public clouds.  Unlike competitors, BigQuery Omni does not create silos across different clouds. BigQUery provides a single control plane that shows an analyst all data they have access to across all clouds. Analyst just writes the query and we send it to the right cloud across AWS, Azure or GCP to execute it locally. Hence no egress costs are incurred. 

We announced BQ Omni GA for both AWS and Azure at Google Next 21 and I’m really proud of the team for delivering on this vision.  Check out Vidya’s session and learn from Johnson and Johnson how they innovate in a multi-cloud world.

Geospatial Analytics with BigQuery and Earth Engine

We have partnered with our Google Geospatial team to deliver GIS functionality inside BigQuery over the years. At Next we announced that customers will be able to integrate Earth Engine with BigQuery, Google Cloud’s ML technologies, and Google Maps Platform. 

Think about all the scenarios and use-cases your team’s going to be able to enable sustainable sourcing, saving energy or understanding business risks.

We’re integrating the best of Google and Google Cloud together to – again – make it easier to work with data to create a sustainable future for our planet.  

BigQuery as a Data Exchange & Sharing Platform

BigQuery was built to be a sharing platform. Today we have 3000+ organizations sharing more than 250 petabytes of data across organizations. Google also brings more than 150 public datasets to be used across various use cases. In addition to this, we are also bringing some of the most unique datasets like Google Trends to BigQuery. This will enable organizations to understand in real-time trends and apply to their business problems.

I am super excited about the Analytics Hub Preview announcement. Analytics Hub will provide the ability for organizations to build private and public analytics exchanges. This will include data, insights, ML Models and visualizations. This is built on top of the industry leading security capabilities of BigQuery.

Breaking Data Silos

Data is distributed across various systems in the organization and making it easy to break the data silo and make all this data accessible to all is critical. I’m also particularly excited about the Migration Factory we’re building with Informatica and the work we are doing for data movement, intelligent data wrangling with players like Trifacta and FiveTran, with whom we share over 1,000 customers (and growing!).  Additionally we continue to deliver native Google service to help our customers. 

We acquired Cask in 2018 and launched our self service Data Integration service in Data Fusion. Now Fusion allows customers to create complex pipelines with just simple drag and drop. This year we focused on unlocking SAP data for our customers. We have launched various SAP connectors and accelerators to achieve this.

At GCP Next we also announced our BigQuery Migration service in preview. Many of our customers are migrating their legacy data warehouses and data lakes to BigQuery. BigQuery Migration Service provides end-to-end tools to simplify migrations for these customers. 

And today, to make migrations to BigQuery easier for even more customers, I am super excited to announce the acquisition of CompilerWorks. CompilerWorks’ Transpiler is designed from the ground up to facilitate SQL migration in the real world and will help our customers accelerate their migrations. It supports migrations from over 10 legacy enterprises data warehouses and we will be making it available as part of our BigQuery Migration service in the coming months.

Data Democratization with BigQuery

Over the past 4 years we have focused a lot on making it very  easy to derive actionable insights from data in BigQuery. Our priority has been to provide a strong ecosystem of partners that can provide you with great tools to achieve this but also deliver native Google capabilities. 

With our BI engine GA announcement which we introduced in 2019, previewed earlier this year and showcased with tools like Microsoft PowerBI and Tableau, is now available for all to play with.

BigQuery + Data Studio are like peanut butter and Jelly. They just work well together. We launched BI Engine first with Data Studio and scaled it to all the users. More than 40% of our BigQuery customers use Data Studio. Once we knew BI Engine works extremely well we now have made it an integral part of BigQuery API and launched it for all our internal and partner BI tools. 

We announced GA for BI Engine at Next 2021 but we were already GA with Data Studio for the past 2 years. We recently moved the Data Studio team back into Google Cloud making the partnership even stronger. If you have not used Data Studio, I encourage you to take a look and get started for free today here!! 

Connected Sheets for BigQuery is one of my favorite combinations. You can give every business user in your organization the ability to analyze billions of records using standard Google Sheets experience. I personally use it everyday to analyze all our product data. 

We acquired Looker in Feb 2020 with a vision of providing a semantic modeling layer to our customers with a governed BI solution. Looker is tightly integrated with BigQuery including BigQuery ML. Our latest partnership with Tableau where Tableau customers will soon be able to leverage Looker’s semantic model, enabling new levels of data governance while democratizing access to data. 

Finally, I have a dream that one day we will bring Google Assistant to your enterprise data. This is the vision of Data QnA. We are in early innings on this and we will continue to work hard to make this vision a reality. 

Intelligent Data Fabric to unify the platform

Another important trend that shaped our market is the Data Mesh.  Earlier this year, Starburst invited me to talk about this very topic. We have been working for years on this concept, and although we would love for all data to be neatly organized in one place, we know that our customers’ reality is that it is not (If you want to know more about this, read about my debate on this topic with Fivetran’s George Fraser, a16z’s Martin Casado and Databricks’ Ali Ghodsi).

Everything I’ve learned from customers over my years in this field is that they don’t just need a data catalog or a set of data quality and governance tools, they need an intelligent data fabric.  That is why we created Dataplex, whose general availability we announced at Next.

Dataplex enables customers to centrally manage, monitor, and govern data across data lakes, data warehouses, and data marts, while also ensuring data is securely accessible to a variety of analytics and data science tools.  It lets customers organize and manage data in a way that makes sense for their business, without data movement or duplication. It provides logical constructs – lakes, data zones, and assets – which enable customers to abstract away the underlying storage systems to build a foundation for setting policies around data access, security, lifecycle management, and so on.  Check out Prajakta Damle’s session and learn from Deutsche Bank how they are thinking about a unified data mesh across distributed data.

Closing Thoughts

Analysts have recognized our momentum and, as I look back at this year, I couldn’t thank our customers and partners enough for the support they provided my team and I across our large Data Analytics portfolio: in March, Google BigQuery was named a Leader in The Forrester Wave™: Cloud Data Warehouse, Q1 2021.  And in June, Dataflow was named a Leader in The Forrester Wave™: Streaming Analytics, Q2 2021 report.

If you want to get a taste for why customers choose us over other hyperscalers or cloud data warehousing, I suggest you watch the Data Journey series we’ve just launched, which documents the stories of organizations modernizing to the cloud with us.

The Google Cloud Data Analytics portfolio has become a leading force in the industry and I couldn’t be more excited to have been part of it.  I do miss you, my customers and partners, and I’m frankly bummed that we didn’t get to meet in person like we’ve done so many times before (see a photo of my last in-person talk before the pandemic), but this Google Next was extra special, so let’s dive into the product innovation and their themes.

I hope that I will get to see you in person next time we run Google Next!

Retailers are becoming more skilled at making individual customers feel heard and valued. This is a necessity given the fact that 66% of respondents to a McKinsey survey stated that they expect email marketing messages to be tailored to their needs. While marketing personalization expertise is growing, it’s still difficult to manage, especially at scale.

Ocado Retail, one of the world’s largest dedicated online grocery retailers, delivering to over 645,000 customers in the U.K., has made personalization integral to its success. 

Let’s look at how Ocado retail worked with Google Cloud and partner Cognizant to develop a new data platform to power its personalization efforts from the ground up. 

Unifying data for more holistic, powerful personalization

To achieve its goals of personalization at scale, Ocado Retail needed a central data warehouse that could turn all forms of merchandising, advertising, business, and customer intelligence data into actionable insights. It wanted a means to accelerate customer segment identification, as well as the ideation and launch of relevant campaigns. 

“We standardized on Google Cloud, including BigQuery, as the foundation for our data platform because we knew it was the right solution for now and the future,” says Kieren Johnson, Head of IT at Ocado Retail. “We have an incredibly lean team and we needed a partner with exceptional expertise to help build an ambitious enterprise data warehouse to provide powerful insights. Cognizant was also the clear choice to help us get there.”

Cognizant worked closely with Ocado Retail to make sure its expertise in Google Cloud and other technologies aligned with Ocado Retail’s vision to drive more advanced personalization at higher scales using machine learning. The partner helped build the foundation on BigQuery, and then incorporated other Google Cloud tools such as Cloud Run, Vertex AI, and Vertex AI Natural Language to provide no-ops, all-code warehousing, and analytics capabilities.

Cognizant also took advantage of the Google Cloud Partner Success Services (PSS) program to ensure best practices were being followed throughout the project. PSS provided advisory services that guided Cognizant through the highly complex process of building the new data warehouse for Ocado Retail on Google Cloud.

Building the enterprise data platform in this way allows Ocado Retail to leverage the full power of cloud-based analytics while maintaining a lean team. It also allows the company to greatly scale up its personalization efforts.

Making customers feel valued at every touch

The work Ocado Retail has done with Google Cloud and Cognizant has positioned it to make its growing customer base feel valued, understood, and supported in every interaction. Before launching the project with Cognizant and Google Cloud, Ocado Retail was only able to run a couple of campaigns per week and knew it lacked optimal insight into each campaign’s efficacy. 

“We now run 10 times the number of campaigns we used to with the help of the data platform Cognizant built on Google Cloud,” says Kieren. “We run multiple campaigns every day for different customer segments, and all of that increased activity is entirely driven by data insights. The positive impacts on our marketing and customer service performance have been clear. We’re now working to expand what we do.”

Ocado Retail has enjoyed solid growth since the new data platform went live, including a 13% rise in active customers during fiscal year 2022, and has also seen a reduction in churn. It attributed these improvements to being better able to tailor products and communications to specific customer preferences.

Throughout the project, Cognizant supported data clean up while maximizing the scalable, flexible, and future-proofed data analytics infrastructure offered by Google Cloud.

Increasing data-driven actions

Ocado Retail plans to provide more data-driven insights to its commercial suppliers through a product called Beet Insights. So far, Beet Insights offers suppliers with intelligence about how their products are performing on the shelves. The result has been improving the role data plays throughout the supply chain, from production to purchase and beyond.

“By putting real-time insights about costs, marketing spend, and supply-funded activities into the hands of our commercial team and buyers, we are better positioned to improve our profits,” says Kieren. “At the same time, building data analytics into every part of our business will allow us to build on our personalization efforts.”

From the project’s inception, Ocado Retail ensured that the platform would be scalable and dynamic. Now, it is working to feed more data sources and streams into the warehouse to accelerate time to insights. Ocado Retail believes this next step in the evolution of the platform will unlock even more opportunities to initiate high-impact programs that transform personalization and every customer interaction.

Learn more about what Google Cloud and partners like Cognizant can do for your customer intelligence and personalization.