BigQuery offers strong query performance, but it is also a complex distributed system with many internal and external factors that can affect query speed. When your queries are running slower than expected or are slower than prior runs, understanding what happened can be a challenge.

The query execution graph provides an intuitive interface for inspecting query execution details. By using it, you can review the query plan information in graphical format for any query, whether running or completed.

You can also use the query execution graph to get performance insights for queries. Performance insights provide best-effort suggestions to help you improve query performance. Since query performance is multi-faceted, performance insights might only provide a partial picture of the overall query performance.

Execution graph

When BigQuery executes a query job, it converts the declarative SQL statement into a graph of execution, broken up into a series of query stages, which themselves are composed of more granular sets of execution steps. The query execution graph provides a visual representation of the execution stages and shows the corresponding metrics. Not all stages are made equal. Some are more expensive and time consuming than others. The execution graph provides toggles for highlighting critical stages, which makes it easier to spot the potential performance bottlenecks in the query.

Query performance insights

In addition to the detailed execution graph BigQuery also provides specific insights on possible factors that might be slowing query performance.

Slot contention

When you run a query, BigQuery attempts to break up the work needed by your query into tasks. A task is a single slice of data that is input into and output from a stage. A single slot picks up a task and executes that slice of data for the stage. Ideally, BigQuery slots execute tasks in parallel to achieve high performance. Slot contention occurs when your query has many tasks ready for slots to start executing, but BigQuery can’t get enough available slots to execute them.

Insufficient shuffle quota

Before running your query, BigQuery breaks up your query’s logic into stages. BigQuery slots execute the tasks for each stage. When a slot completes the execution of a stage’s tasks, it stores the intermediate results in shuffle. Subsequent stages in your query read data from shuffle to continue your query’s execution. Insufficient shuffle quota occurs when you have more data that needs to get written to shuffle than you have shuffle capacity.

Data input scale change

Getting this performance insight indicates that your query is reading at least 50% more data for a given input table than the last time you ran the query and hence experiencing query slowness. You can use table change history to see if the size of any of the tables used in the query has recently increased.

What’s next?

We continue to work on improving the visualization of the graph. We are working on adding additional metrics to each step and adding more performance insights that will make query diagnosis significantly easier. We are just getting started.

Some of the most important data at your company isn’t living in databases, but in documents, and most business processes begin, involve or end with a document. 

Yet most companies are still manually entering data and reliant on guesswork to make sense of it all as the volume and variety of data explodes. Organizations are also leaving heaps of value on the table in the form of new and better customer experiences that can be unlocked with artificial intelligence (AI) applied to documents. 

The latest releases of Document (Doc) AI platform, Lending DocAI and Procurement DocAI, built on decades of AI innovation at Google, bring powerful and useful solutions to these challenges. Under the hood are Google’s industry-leading technologies:

• Computer vision (including OCR) and Natural Language Processing (NLP) that creates pre-trained models for high-value, high-volume documents. 
• Google Knowledge Graph to validate and enhance the fields in your documents.
• Training and creation of your own custom document models. 
• Human interaction with AI to ensure accuracy where needed.

Google Cloud DocAI platform, Lending DocAI and Procurement DocAI are now generally available. Thousands of customers have tried these products in the preview phase—and DocAI has already processed tens of billions of pages of documents across lending, insurance, government and other industries.

“The vast majority of enterprise content still resides in unstructured sources like documents. Google Cloud’s Document AI brings a fresh new perspective to the problem informed by the company’s decades of experience making sense of the largest unstructured corpus in the world—the world wide web.” —Ritu Jyoti, VP of AI Research, IDC

Cut document processing costs by up to 60%

Lending DocAI helps banks, mortgage brokers and other lending institutions fast track the loan application process from weeks to days, dramatically reducing the cost of issuing a loan. And Procurement DocAI enables companies to automate procurement data capture at scale, lowering processing costs by up to 60%.

These solutions are built on DocAI platform, a unified console for document processing that lets you quickly access all parsers and tools. From the platform, you can automate and validate documents to streamline workflows, reduce guesswork, and keep data accurate and compliant. 

Get more value from AI with DocAI’s industry-specific solutions

According to Accenture’s AI: Built to Scale report: “Companies that scale successfully see 3x the return on their AI investments compared to those who have not fully rolled out AI capabilities.” 

Core to our strategy at Google Cloud is the creation of industry-specific solutions that help companies get maximum value out of their investments in AI. We announced Lending DocAI, our first solution designed specifically for the financial services industry, at the Mortgage Bankers Association convention last year. It processes borrowers’ income and asset documents using a set of specialized machine learning (ML) models, and automates routine document reviews so that mortgage providers can focus on more important work. 

Lending DocAI is now generally available and includes more specialized parsers for critical loan documents including paystubs, bank statements, and more. Our goal is to provide the right tools to help borrowers and lenders have a better experience and close home loans faster. For more, watch this video.

Procurement DocAI is also now generally available. This solution helps companies accelerate document processing for invoices, receipts, and other valuable documents in the procurement cycle. 

Automating data capture is helping our customers increase accuracy and also lower their procure-to-pay processing costs. We are continually expanding the types of documents Procurement DocAI can process—the latest is a utility parser for electric, water and other bills. In addition, Procurement DocAI leverages Google Knowledge Graph to validate and enrich parsed information to make the data even more useful. Check out this overview video for more details. 

One company that lives and breathes AI-enabled document management is AODocs. It uses Procurement DocAI to simplify invoice processing for enterprise customers and launched a new Gmail add-on, Invoice to Sheet, for SMB customers who just want to track their invoices in Google Sheets.

“Google Cloud’s Procurement DocAI service allows our document management platform to better automate the processing of invoices; AODocs customers who have tested our new account payables workflow estimate that the productivity of their A/P team has more than doubled, thanks to the reduction of manual data input brought by the Procurement DocAI.”—Stéphan Donzé, Founder and CEO, AODocs

The new specialized parsers for Lending and Procurement DocAI can be used alongside our existing AutoML Text & Document Classification and AutoML Document Extraction services. These technologies provide a state-of-the-art toolset for creating new document models and have been widely deployed by customers in financial services and other industries. 

Partner to accelerate your AI deployment and results

Having the right partner to ease the complexity of rolling out your AI-strategy in mortgage document processing is critical to transforming your customers’ experience. We’re excited to announce a partnership with Mr. Cooper, a leader in mortgage servicing, to provide customers with more automation and workflow tools throughout their entire mortgage life cycle. As part of this agreement, both companies will collaborate on digitizing Mr. Cooper’s core mortgage platform, creating a more personal customer experience utilizing AI, and driving a broader culture of innovation to imagine and develop services and solutions that will transform the mortgage experience for American homeowners.

“Over the last few years, we have made substantial investments in our servicing technology and core mortgage platform that have revolutionized the customer experience, while providing dramatic efficiencies in operating cost. Our partnership with Google Cloud AI will build on those advances and help make these technologies available for the mortgage industry.” —Jay Bray, Chairman and CEO, Mr. Cooper Group

This builds upon the robust partner ecosystem we’re creating to help customers revolutionize the home loan experience, which includes last year’s partnership announcement with Roostify.

Integrate human review into ML predictions 

Next up is the general availability of Human-in-the-Loop AI, a new DocAI feature that will help companies achieve higher document processing accuracy with the assurance of human review. Adding human review can increase accuracy and help businesses interpret predictions using purpose-built tools to enable those reviews. 

Processing documents quickly and cost-effectively is important. But it’s often necessary to have a high level of assurance on data accuracy for compliance. CIOs and IT decision-makers need highly accurate ML predictions to fulfill compliance requirements, improve employee experience (e.g. less rework), and raise customer satisfaction (e.g. fewer data errors). Including human participation in ML processes allows AI and humans to work together for the best possible results.

Human-in-the-Loop AI provides the workflow to manage human review tasks and produces a percentage confidence score of how “sure” it is that the AI ingested the document correctly. Document AI extracts data from documents with ML, and when paired with Human-in-the-Loop AI, human reviewers are able to verify the data captured. This system is customizable, providing the flexibility to set different thresholds and assign individual groups of reviewers to various stages of the workflow. With Human-in-the-Loop AI, developers can choose trusted reviewers to assign to the task; these reviewers can be from within their own or partner organizations.

More Document AI resources

To learn more, check out the Document AI webpage and watch a demo of how to process sample forms in AI Platform notebooks to inspect data extraction and confidence scores. For more on how customers and partners like Workday, AODocs, and Mr. Cooper are using Document AI, listen to our fireside chat. And stay tuned for the exciting evolution of these technologies in future releases of DocAI.

“Hello, Marks & Spencer. How may we help you?”

As one of the biggest and best-loved retail brands in the UK, Marks & Spencer (M&S) is known for the personalized service it provides to its 30 million loyal customers. For 135 years and in 57 countries around the world, M&S has worked to meet and exceed customer expectations for quality and service. Since the advent of the telephone, this has meant cheerfully servicing customers who call in to M&S branches or into their contact centers, no matter what their request might be.

“Retail is in a state of flux and M&S is transforming to better serve our customers so that we can compete and win. Automating calls into our stores with Google voice recognition gives us every opportunity to get things right for our customers and keep them coming back again and again.”

—Akash Parmar, Enterprise Architect (Digital Customer Engagement), Marks & Spencer

The company now has a goal of bringing one third of its business online by 2022. In order to engage more customers online, it has opened a new voice channel hosted on Google Cloud.

The company previously had switchboards in 13 different stores across the UK and Ireland (UKI), handling up to nine million calls a year. But as its retail offering evolved across multiple channels, it was becoming increasingly difficult to quickly and effectively answer customer service requests using an outdated switchboard model.

Customers might call in to order an outfit they had seen in a store, to inquire about returning a dress they bought online, or to recover a lost umbrella in a food hall. Each of these different requests required a different routing response from staff, and if the company didn’t act soon, it knew that the cost of managing the increase in call volume would lead to a significant cost impact. M&S decided it was time to make a technological leap forward to meet customers’ expectations in the new, omnichannel retail environment.

“Retail is in a state of flux and M&S is transforming to better serve our customers so that we can compete and win,” says Akash Parmar, Enterprise Architect for Digital Customer Engagement at M&S. “Automating calls into our stores with Google voice recognition gives us every opportunity to get things right for our customers and keep them coming back again and again.”

Boosting opportunities for customer engagement with voice recognition

M&S customers were used to dealing with their local store for anything they needed. But as stores were completely separate from the online business, customers weren’t able to purchase something they’d seen online by calling stores because the store staff didn’t have access to platforms needed to place an online order securely. For a company that places a very high importance on customer experience, this was unacceptable.

Akash Parmar, Enterprise Architect for Customer Engagement, was set a the goal by Chris McGrath, M&S Programme Manager, to ensure that the right channel and the right level of assistance was available to customers at any point before, during, or after purchase. Akash set himself the challenge of building a platform that could adapt to all of these channels and scale very quickly.

“We didn’t have the resources to build a speech recognition platform. DVELP removed that obstacle. It understood what we wanted to achieve and how Google Voice APIs and Twilio could get us there. Whatever we want, DVELP builds it for us. DVELP always presents options, never problems.”

—Akash Parmar

In 2018, Akash reached out to Google Cloud partner DVELP, one of the UK’s leading experts on the Twilio programmable contact center platform and Google speech recognition technology. DVELP recommended a Google Cloud-based natural language speech recognition platform that leverages the audio stream intent detection functionality in the Contact Center AI solution, Dialogflow, as the heart of an inbound-call-handling strategy. This strategy was designed to improve routing accuracy, give customers more self-service options, and increase analyst visibility into customer journeys.

“We didn’t have the resources to build a speech recognition platform. DVELP removed that obstacle,” says Akash. “They understood what we wanted to achieve and how Google Cloud Voice APIs and Twilio could get us there. Whatever we want, DVELP builds it for us. DVELP always presents options, never problems.”

Using Google speech recognition to improve customer experience

M&S wanted to use natural language to enable customers to speak and state what help they required rather than choose from a list of options. This would help them answer the millions of calls coming in and figure out what customers needed quickly.

In order to do that, DVELP needed to consider how best to address tying customer intent to actions, while maintaining flexibility. DVELP recommended the unconventional choice of not referencing intent in the application layer, but mapping the available actions to the information required to perform them. These actions were then used to build a “declarative dictionary” for the customer service team.

By focusing on actions rather than intents, the solution enables the customer services team to configure actions to intents in virtually any combination of key-value pairs. Leveraging the fact that Dialogflow can detect and respond to customer intents in real time, M&S has already reached 92% accuracy in translating customer declarations to actionable intents.

“With Google Cloud speech recognition and Contact Center AI solutions such as Dialogflow, there’s no information that we can’t make sense of. No matter where you call from, who you are, your age, your gender: you speak, and we understand.”

—Akash Parmar

Akash recalls that once customers became comfortable with the prompt, “in a few words, how may we help you?” they started providing simple, concise responses, and the customer learning curve quickly leveled out. At that point, Google Cloud speech recognition and Dialogflow took over. “The technology worked perfectly and the result was like magic,” says Akash.

“With Google Cloud speech recognition and Contact Center AI solutions such as Dialogflow, there’s no information that we can’t make sense of,” he says. “No matter where you call from, who you are, your age, your gender: you speak, and we understand.”

Enabling self-service contact center improvements with Dialogflow

It was important to M&S that contact center employees be self-sufficient in updating the platform to reflect changes in demand. They needed to be able to easily react to a spike in inquiries about a special offer, for example, without relying on the engineering team. At the same time, neither Akash nor the DVELP team wanted the staff to have to learn error-prone JSON inside contexts, or write responses in order to get necessary information from customers.

DVELP’s creative solution was to fill out the “Action and parameters” section of every intent. This is usually reserved for collecting information from customer declarations, but was also easily adapted to implementing custom key-value pairs. This is particularly helpful in making sure that the contact center is ready to handle new promotions as they arise. As sales and special events are communicated to the contact center from the head office, staff can program specific vocabulary directly into Dialogflow, thanks to Contact Center AI, ensuring that the M&S system is immediately ready to handle related customer calls.

Rolling out the platform to the UK and Ireland

In just a few months after going live, calls are being efficiently routed to the contact center and its existing customer service platform. At the contact center, staff can quickly and easily respond to customer requests, place orders, and process returns. Thanks to the natural language capabilities of Google Cloud, a simple customer request like “order the red children’s dress in the Bath high street window in size six” not only gets correctly routed, but provides data points for future personalized interactions.

Being able to accurately recognize customer intent 92% of the time after less than four months since deployment is an important milestone for M&S. With a concurrent 89% voice-to-text accuracy rate for Dialogflow transcriptions, M&S has rolled out the successful speech recognition platform to all of its stores in UK and Ireland and customer service contact centers.

Akash is so pleased with the performance of the new Google Cloud platform that he’s focusing on what new functionality he can add next to improve customer experience even more.

“We’re working on collecting product codes from customers using natural language so we can give them stock availability details,” he shares. “We also want to use Google Cloud to enable a more conversational experience when customers are searching for help or FAQs on our website. At the same time, we’re looking at Contact Center AI and Dialogflow to provide a virtual assistant experience for our webchat journey. Thanks to our new voice solution, we can clearly understand the key issues that our customers face on a day-to-day basis; the aim now is to start solving these issues through self-service and automation.”

In the United Kingdom, the popularity of online grocery shopping is expected to surge from about 6% of the market today to 9% by 2021, according to market research firm Mintel. One of the pioneers of online-only grocery retailing is Ocado, based in Hatfield, Hertfordshire in the U.K. Since starting commercial deliveries in 2002, the company has grown to 600,000 active customers, 260,000 weekly orders, and £1.39 billion in annual revenue.

Ocado takes supermarket trips out of the equation by enabling shoppers to purchase items online through its convenient web and mobile applications. Items are then picked and packed in automated warehouses and shipped directly to customers in a one-hour time slot of their choosing. Ocado’s delivery punctuality is 95%, order accuracy is 99%, and its service footprint now reaches more than 70% of the U.K. population.

“Google Cloud Platform gives us the flexibility and performance to tackle the large and complex data challenges unique to our business.”

—Paul Clarke, Chief Technology Officer, Ocado

The company achieved its success by building in-house almost all the technology and automation that powers its end-to-end e-commerce, fulfillment, and logistics platform. Ocado also developed a new platform, the Ocado Smart Platform (OSP), which offers large brick-and-mortar grocery retailers around the world access to a best-in-class solution for online grocery.

Democratizing machine learning

The shopping journey for online grocery retailing differs significantly from other e-businesses. Customers often buy dozens of products at once, a single household may have multiple buyers using multiple devices, and product shelf life may only be a couple of days.

“We often say that having built an end-to-end platform that can do online grocery scalably and profitably, we can do other forms of online retail; but the reverse does not necessarily follow,” says Paul Clarke, Chief Technology Officer at Ocado. “Google Cloud Platform gives us the flexibility and performance to tackle the large and complex data challenges unique to our business.”

The Ocado business model takes advantage of consumers’ shifting preferences and the links between digital technology and shopping experiences.

“Google Cloud Machine Learning Engine gives us the agility we need. Our developers were able to try out TensorFlow and see firsthand the benefits of machine learning in the cloud.”

—Paul Clarke, Chief Technology Officer, Ocado

The company has been building machine learning into its systems for over five years. Until recently, Ocado machine learning applications required specialist data scientists, typically with PhDs in machine learning, who would build these solutions from the ground up. It also required the specialist who set up the system and costly on-premises infrastructure to train and run these systems.

However, working with Google as a private alpha testing site for Google Cloud Machine Learning Engine accelerated its adoption of artificial intelligence (AI).

“We’ve been talking about how the cloud could democratize AI for some time,” says Paul. “Google Cloud Machine Learning Engine gives us the agility we need. Our developers were able to try out TensorFlow and see firsthand the benefits of machine learning in the cloud.”

TensorFlow is an open source software library for machine learning developed by the Google Brain team. Ocado developers, engineers, and data scientists now use TensorFlow for many of their machine learning projects. They deploy the models they build on Google Cloud Machine Learning Engine, which lets them train models faster across servers, desktop computers, and mobile devices through a single application program interface (API). Additionally, Google Cloud Machine Learning Engine integrates easily with the other Google Cloud Platform products used widely at Ocado.

What do customers really want?

One of the first TensorFlow models Ocado created was a machine learning algorithm that tags and categorizes customer emails and then prioritizes them for response.

The contact center receives thousands of emails each day and Ocado wanted to automate determining which ones needed to be answered immediately and which ones could wait.

For example, a first-time customer expressing their delight in using Ocado doesn’t need to be responded to with the same urgency as a customer who is missing an item from their order or who won’t be home to receive the delivery.

“Enabling agents to respond without having to sort through less-urgent emails improves Ocado’s responsiveness and customer service.”

—James Donkin, General Manager, Ocado

“We get a lot of emails from customers saying, ‘Our service was great,’ or ‘The driver was very courteous,'” says James Donkin, General Manager, Ocado. “But when issues like weather or road conditions potentially affect delivery, we often get surges of urgent questions. Enabling agents to respond without having to sort through less-urgent emails improves Ocado’s responsiveness and customer service.”

Using Google Cloud Machine Learning Engine, TensorFlow, and a large data set culled from several years’ worth of manually categorized customer emails, Ocado experimented on which kind of neural network architecture would best prioritize emails. After testing its models, Ocado implemented the highest-performing one and has been able to respond to urgent messages four times faster. The company also discovered that 7% of its emails don’t require a response at all, which means call center representatives now have more time to devote to higher priority messages.

“Without Google Cloud Machine Learning Engine, it would have been a lot harder to succeed on a project like email classification,” says Roland Plaszowski, who has recently managed several big data projects and initiatives at Ocado.

“Even if we invested significantly in infrastructure, it would be difficult to manage because of the computational intensity. It’s challenging and expensive to run machine learning projects at the same time without infrastructure that you can scale easily.”

Ocado also uses machine learning to predict customer behavior and improve experiences. By analyzing order data, Ocado makes shopping as frictionless as possible. For example, the ordering system can pre-populate customers’ shopping carts with items they are most likely to purchase, remind customers about items they may have forgotten, and notify them of multi-buy offers they haven’t completed, for example, only buying one of a buy one, get one free offer. Based on machine learning from previous purchase data, the Ocado system can also offer new products that are likely to delight customers.

“You will regularly see items that are more personally relevant to you instead of items that are being promoted more generally,” says James. “I’m a vegetarian, so I’m offered specials for vegetarian products that I normally buy and new ones that I’ve never bought. I’m also less likely to see things that I’m not interested in.”

Machines and machine learning

Within the Internet of Things (IoT), Ocado is looking to enhance its warehouse robots with machine learning. An integral part of the OSP, thousands of robots continually stream data into Google Cloud Storage and Google BigQuery.

Ocado data scientists apply machine learning to create a type of swarm intelligence that enables warehouse robots to work cooperatively to achieve a common goal. Projects include modules to search robot telemetry data, such as whether a battery pack is operating within standard tolerances or whether firmware has been successfully loaded, and use it to optimize maintenance schedules or detect patterns in wear and tear.

“Another challenge we’re looking at is how to embed machine learning directly into robots so they become smarter in terms of self-testing, exception handling, and error recovery,” says Paul. “This is a challenging combination of IoT, data analytics, and machine learning that we believe Google BigQuery and Google Cloud Machine Learning are particularly well suited to helping Ocado achieve.”

Scaling for new business

Scalability is also a major reason behind some of Ocado’s cloud initiatives, including the migration of all its on-premises data to the cloud. Ocado wanted to improve customer experiences, empower business teams with greater insight, and reduce IT overhead, so it consolidated onto Google Cloud Platform.

“The old databases just weren’t fast enough,” says Paul. “We needed a solution that could scale with the amount of data we generate and how we use it. Google Cloud Storage and Google BigQuery now provide the backbone, from a data point of view, for the Ocado Smart Platform.”

Ocado estimates its business, product, and transaction data is approaching two petabytes. Combining customer and supply chain data helps both internal Ocado operations and the company’s ambitions to commercialize OSP.

“When compared with other options for expansion internationally, selling OSP as a managed service lets us turn companies that could have been competitors into customers,” says Paul. “We want to build OSP once and then turn it on for multiple business-to-business customers.”

Each time Ocado adds a new hosting customer to OSP, it will launch a customized instance to fit that customer’s requirements. The capacity and performance of each new OSP instance must be able to scale quickly as the backend platform for established retailers with large numbers of products, customers, and transactions.

Ocado’s first OSP customer, Morrisons, is already benefiting from this first-of-a kind solution. Morrisons is one of the UK’s four largest supermarkets and uses OSP to power its online retail business. Using Google Cloud Platform, Ocado has stored, processed, and analyzed terabytes of Morrisons’ data using a dedicated data lake and Google BigQuery.

In addition to using Google Cloud Platform for OSP, Ocado also adopted it for its own online grocery retail business operation. Ocado originally used the Apache Spark and Apache Hadoop open-source frameworks on Google Compute Engine for its data platform. Moving to Google BigQuery frees Ocado business analysts from the complex query setup and workflows associated with Spark and Hadoop. Plus, it lets Ocado share data analytics with suppliers and partners.

Google BigQuery is well integrated with TensorFlow on Google Cloud Machine Learning Engine and Google Cloud Dataproc, the Apache Spark and Apache Hadoop service that lets Ocado use open source data tools for batch processing, querying, streaming, and machine learning. Google Cloud Dataflow and Google Cloud Dataproc handle cluster management, and provide an easy-to-use framework so developers can spend less time and money on administration and more time on delivering valuable business features.

Switching from Hadoop to Google BigQuery revealed a series of cost and performance improvements. For example, Ocado no longer needed to decide how many instances to bring up in a cluster or wait for the instances to spin up. Google handled everything.

“We simply ran our queries and paid for the resources that we use,” adds Roland. “One big win with Google BigQuery is we don’t have to do maintenance. Best of all, we saw Google BigQuery outperform our Hadoop cluster by over 80 times on our largest dataset, and for only two-thirds the cost.”