Today, application development teams are more agile and are shipping features faster than ever before. In addition to these rapid development cycles and the rise of microservices architectures, the end-to-end ownership of feature development (and performance monitoring) has moved to a shared responsibility model between advanced database administrators and full-stack developers. However, most developers don’t have the years of experience or the time needed to debug complex query performance issues and database administrators are now a scarce resource in most organizations. As a result, there is a dire need for tools for developers and DBAs alike to quickly diagnose performance issues.

Introducing Query Insights for Spanner

We are delighted to announce the launch of Query Insights for Spanner, a set of visualization tools that provide an easy way for developers and database administrators to quickly diagnose query performance issues on Spanner. Using Query Insights, users can now troubleshoot query performance in a self-serve way. We’ve designed Query Insights using familiar design patterns with world-class visualizations to provide an intuitive experience for anyone who is debugging issues with query performance on Spanner. Query Insights is available at no additional cost.

By using out-of-the-box visual dashboards and graphs, developers can visualize aberrant behavior like peaks and troughs in various performance metrics over a time-series and quickly identify problematic queries. Time series data provides significant value to organizations because it enables them to analyze important real-time and historical metrics. Data is valuable only if it’s easy to comprehend;. that’s where being able to view intuitive dashboards becomes a force multiplier for organizations looking to expose their time series data across teams.

Follow a visual journey with pre-built dashboards

With Query Insights, developers can seamlessly move from detection of database performance issues to diagnosis of problematic queries using a single interface. Query Insights will help identify query performance issues easily with pre-built dashboards.

The user could do this by following a simple journey where they can quickly confirm, identify and analyze query performance issues. Let’s walk through an example scenario.

Understand database performance

This journey will start by the user setting up an alert on Google Cloud Monitoring for CPU utilization going above a certain threshold. The alert could be configured in a way that if this threshold is crossed, the user will be notified with an email alert, with a link to the “Monitoring” dashboard.

Once the user receives this alert, they would click on the link in the email, and navigate to the “Monitoring” dashboard. If they observe high CPU Utilization and high read latencies, the possible root cause could be expensive queries. A spike in CPU Utilization could be a strong signal that the system is using more compute than it usually would, due to an inefficient query.

The next step is to identify which query might be the problem, this is where Query Insights comes in. The user can get to this tool by clicking on Query Insights in the left navigation of your Spanner Instance. Here, they can drill down into the CPU usage by query and observe that for a specific database, CPU Utilization (attributed to all queries) is spiking for a particular time window. This confirms that the CPU utilization is due to inefficient queries.

Identifying a problematic query

The user now observes the TopN (Top queries by CPU Utilization) query graph to see the TopN queries by CPU Utilization. From the graph, it is very easy to visualize and identify the top queries which could be causing the spike in CPU Utilization.

In the above screenshot, we can see that the first query in the table is showing a clear spike at 10:33 PM consuming 48.81% of total CPU. This is a clear indication that this query could be problematic, and the user should investigate further.

Analyzing the query performance

Once they have identified the problematic query, they can now drill down into this query shape to confirm, identify the root cause of the high CPU utilization.

They can do this by clicking on the Fingerprint ID for the specific query from the topN table, and navigating to the Query Details page where they will be able to see a list of metrics (Latency, CPU Utilization, Execution count, Rows Scanned / Rows Returned) over a time series for that specific query.

In this example, we notice that the average number of rows scanned for this specific query are very high (~ 600k rows scanned to return ~ 12k rows), which could point to a poor query design, resulting in an inefficient query. We can also observe that latency is high (1.4s) for this query.

Fixing the issue

To fix the problem in this scenario, the user could optimize this query by specifying a secondary index in the query using a FORCE_INDEX query hint to provide an index directive. This would provide more consistent performance, make the query more efficient, and lower CPU utilization for this query.

In the screenshot below, you can see that after specifying the index in the query, the query performance dramatically increases in terms of CPU, rows scanned (54K vs 630k) and also in terms of query latency (536 ns vs 1.4 s).

Unoptimized Query:

Optimized Query:

By following this simple visual journey, the user can easily detect, diagnose and debug inefficient queries on Spanner.

Get started with Query Insights today

To learn more about Query Insights, review the documentation here. Query Insights is enabled by default. In the Spanner console, you can click on Query Insights in the left navigation and start visualizing your query performance metrics!

New to Spanner? Get started in minutes with a new database.

A growing number of Indian enterprises are leveraging the Google Cloud to run their database workloads.

This is because Google Cloud offers fully managed, scalable database services to support all applications today and tomorrow.

In fact, Forrester has named Google as a Leader in The Forrester Wave™: Database-as-a-Service, Q2 2019.

The first question many {$persona}s ask is: Which database type is best for my specific workload or use case? Here are two tables that will help you get that answer. Click on the images.

With Google Cloud database services, you can supercharge your applications, accelerate adoption with broad open-source database compatibility, and do more with your data through integrations with analytics and ML/AI.

So, you want to store objects on the cloud? But you’re really new to Google Cloud or Cloud Storage and would like someone to walk you through the process step by step?

Today is your lucky day! 

I will help you understand the big steps involved in setting up and using Cloud Storage. For a walkthrough of all the steps involved that even includes highlighting of things to select so you don’t spend all your time playing “Where’s Waldo” on the page looking for that one button, click try the tutorial!

Creating buckets, storing objects in them, and adjusting permissions may feel challenging at first, but after you’ve walked through these steps, you’ll be able to create and use the Cloud Storage your project needs.

Resource Hierarchy

Projects contain all of the related parts of your application. Inside projects, you may have buckets, the top level containers in Cloud Storage. Files and folders are stored in buckets. You can grant access to a bucket, folder, or file using Identity and Access Management.

You can do all of that in this interactive tutorial. So let’s get started. I suggest you follow along in the tutorial so you can see the specific keys and screens as I describe them.

Creating a project

Everything in your application will be in a project, even if the only service you are using is storage. Projects need to be associated with a billing account, so if you don’t have one, consider signing up for the Free Trial. You’ll start in the Cloud Console.

Creating a bucket

Once you have a project, you can use storage by navigating to the Cloud Storage page by using either the Navigation menu at the top left of the console or by searching for “Cloud Storage” using the search box at the top of the console.

At the Buckets page of Cloud Storage, you’ll be able to create a bucket. The name of your bucket needs to be globally unique; that is no other bucket in Google Cloud Storage can have the same name. This is because if you or your organization allow your bucket to be accessible on the Internet, it will be at the URL https://storage.googleapis.com/<bucket name>. You can also consider implementing a security feature to prevent data exfiltration, organization restriction headers when you’re finished creating your bucket.. Pick a region close to you for your bucket. In some cases it may make sense to use multi-region or dual-region, but the details are more than I want to get into here.  Don’t worry; you can find lots more information on them! Additionally, I won’t go into the details about selecting the appropriate storage class based on access frequency and longevity, but more information can be found here as well our recently introduced Autoclass tiering feature..

Since buckets can be accessed from the Internet, you need to be careful to only make the things you want to be public, public. The default value is to keep your data off of the Internet, so when you create the bucket, there’s an option to “Enforce public access prevention on this bucket” that is already selected. Since we’ll be making items in this bucket public later on, clear the option before confirming the bucket information. Don’t worry; you’ll see how to set that option back later.

Take a look at the Bucket detail page. There are tabs to see the objects in the bucket, the permissions on the bucket, and much more. There are commands to create folders and to upload files and folders.  We’ll do some of this in the next section. You can even find out much more from the Learn option in the upper right.

Adding items to the bucket

Find a file you’d like to upload to this bucket, remembering it will be made public to the Internet later. If you don’t have anything available, there’s a picture of a cat (of course, it’s a cat picture, this is the Internet!) in the interactive tutorial.

Go to the bucket details for your bucket. Using the “Upload files” button, you’ll just upload the file to the bucket by selecting the file from your computer.   We won’t get into service classes here, but when you’re ready, there are links in the references.

Of course, if you have a lot of files in a bucket, you probably will want some sort of organization. You can do this by creating folders in your bucket with the “Create Folder” option. Once you’ve done that, you can go to the overflow menu (three vertical dots at the end line with the object information) and select Move. Remember, if you need help finding this, the interactive tutorial will point it out. 

Making a bucket public 

No surprise, the Bucket details hold lots of information about your bucket. You’ll change the bucket’s permissions in the Permission tab. (I hope that’s not a surprise.) You can then give access to individuals, groups, or all users. For this tutorial, follow the steps to grant access to allUsers. There are a variety of different levels of access you can grant to a user. In this case, just grant the ability to view the object using Storage Object Viewer.

Then go into the Objects tab to find the URL for the object you uploaded. Copy that URL and ensure you can indeed access the object from another tab or incognito window.

Now that you’ve done that, go back to the Permission tab for the bucket and select Prevent Public Access to secure the bucket again.

So, in a nutshell, to make all files in a bucket public, in the Permissions tab:

• Uncheck the option “Remove Public Access Prevention”
• Use Grant Access to give allUsers the Storage Object Viewer role in Cloud Storage.

To make all files in a bucket private, in the Permissions tab select Prevent Public Access.

Cleaning up

There is a charge to keep items in Cloud Storage, so you’ll probably want to delete the object you uploaded by selecting it and pressing Delete. You can delete the bucket in a similar way. The tutorial has details, of course.

While there are lots more details about Cloud Storage, you’ve got the big picture now. Go do something creative and useful with it!

So what now?

If you haven’t walked through the interactive tutorial, give it a try!

Cloud Spanner is the first scalable, enterprise-grade, globally-distributed, and strongly consistent database service built for the cloud specifically to combine the benefits of relational database structure with non-relational horizontal scale.

This combination delivers high-performance transactions and strong consistency across rows, regions, and continents with an industry-leading 99.999% availability SLA, no planned downtime, and enterprise-grade security.

Cloud Spanner revolutionizes database administration and management and makes application development more efficient.

Find out How Streak built a graph database on Cloud Spanner to wrangle billions of emails

What Are Cloud Spanner’s Strengths?

What is Cloud Spanner Useful For?

What Does it Cost?

Pricing for Cloud Spanner is simple and predictable. You are only charged for the number of nodes in your instance, the amount of storage that your tables and secondary indexes use (not pre-provisioned), and the amount of network bandwidth used. Note that there is no additional charge for replication. For more detailed pricing information, please view the pricing guide.

When you’re serving customers online, speed is essential for a good experience. As the amount of data in a database grows, queries that used to be fast can slow down.

For example, if a query has to scan every row because a table is missing an index, response times that were acceptable with a thousand rows can turn into multiple seconds of waiting once you have a million rows.

If this query is executed every time a user loads your web page, their browsing experience will slow to a crawl, causing user frustration. Slow queries can also impact automated jobs, causing them to time out before completion.

If there are too many of these slow queries executing at once, the database can even run out of connections, causing all new queries, slow or fast, to fail. 

The popular open-source databases MySQL and Google Cloud Platform‘s fully managed version, Cloud SQL for MySQL, include a feature to log slow queries, letting you find the cause, then optimize for better performance.

However, developers and database administrators typically only access this slow query log reactively, after users have seen the effects and escalated the performance degradation.

With Stackdriver Logging and Monitoring, you can stay ahead of the curve for database performance with automatic alerts when query latency goes over the threshold, and a monitoring dashboard that lets you quickly pinpoint the specific queries causing the slowdown.

To get started, import MySQL’s slow query log into Stackdriver Logging.

Once the logs are in Stackdriver, it’s straightforward to set up logs-based metrics that can both count the number of slow queries over time, which is useful for setting up appropriate alerts, and also provide breakdowns by slow SQL statement, allowing speedy troubleshooting.

What’s more, this approach works equally well for managed databases in Cloud SQL for MySQL and for self-managed MySQL databases hosted on Compute Engine. 

For a step-by-step tutorial to set up slow query monitoring, check out Monitoring slow queries in MySQL with Stackdriver. For more ideas about what else you can accomplish with Stackdriver Logging, check out Design patterns for exporting Stackdriver Logging.