Picking the right database for your application is not easy. The choice depends heavily on your use case—transactional processing, analytical processing, in-memory database, and so on—but it also depends on other factors. This post covers the different database options available within Google Cloud across relational (SQL) and non-relational (NoSQL) databases and explains which use cases are best suited for each database option. 

Relational databases 

In relational databases information is stored in tables, rows and columns, which typically works best for structured data. As a result they are used for applications in which the structure of the data does not change often. SQL (Structured Query Language) is used when interacting with most relational databases. They offer ACID consistency mode for the data, which means:

• Atomic: All operations in a transaction succeed or the operation is rolled back.
• Consistent: On the completion of a transaction, the database is structurally sound.
• Isolated: Transactions do not contend with one another. Contentious access to data is moderated by the database so that transactions appear to run sequentially.
• Durable: The results of applying a transaction are permanent, even in the presence of failures.

Because of these properties, relational databases are used in applications that require high accuracy and for transactional queries such as financial and retail transactions. For example: In banking when a customer makes a funds transfer request, you want to make sure the transaction is possible and it actually happens on the most up-to-date account balance, in this case an error or resubmit request is likely fine.

There are three relational database options in Google Cloud: Cloud SQL, Cloud Spanner, and Bare Metal Solution.

Cloud SQL: Provides managed MySQL, PostgreSQL and SQL Server databases on Google Cloud. It reduces maintenance cost and automates database provisioning, storage capacity management, back ups, and out-of-the-box high availability and disaster recovery/failover. For these reasons it is best for general-purpose web frameworks, CRM, ERP, SaaS and e-commerce applications.

Cloud Spanner: Cloud Spanner is an enterprise-grade, globally-distributed, and strongly-consistent database that offers up to 99.999% availability, built specifically to combine the benefits of relational database structure with non-relational horizontal scale. It is a unique database that combines ACID transactions, SQL queries, and relational structure with the scalability that you typically associate with non-relational or NoSQL databases. As a result, Spanner is best used for applications such as gaming, payment solutions, global financial ledgers, retail banking and inventory management that require ability to scale limitlessly with strong-consistency and high-availability. 

Bare Metal Solution: Provides hardware to run specialized workloads with low latency on Google Cloud. This is specifically useful if there is an Oracle database that you want to lift and shift into Google Cloud. This enables data center retirements and paves a path to modernize legacy applications. 

Non-relational databases

Non-relational databases (or NoSQL databases) store compex, unstructured data in a non-tabular form such as documents. Non-relational databases are often used when large quantities of complex and diverse data need to be organized. Unlike relational databases, they perform faster because a query doesn’t have to access several tables to deliver an answer, making them ideal for storing data that may change frequently or for applications that handle many different kinds of data. 

For example, an apparel store might have a database in which shirts have their own document containing all of their information, including size, brand, and color with room for adding more parameters later such as sleeve size, collars, and so on.

Qualities that make NoSQL databases fast:

• Eventual consistency: stores usually exhibit consistency at some later point (e.g., lazily at read time)
• Horizontal scaling, usually using hashed distributions
• Typically, they are optimized for a specific workload pattern (i.e., key-value, graph, wide-column)
• Typically, they don’t support cross shard transactions or flexible isolation modes.

Because of these properties, non-relational databases are used in applications that require large scale, reliability, availability, and frequent data changes.They can easily scale horizontally by adding more servers, unlike some relational databases, which scale vertically by increasing the machine size as the data grows. Although, some relations databases such as Cloud Spanner support scale-out and strict consistency.

Non-relational databases can store a variety of unstructured data such as documents, key-value, graphs, wide columns, and more. Here are your non-relational database options in Google Cloud: 

• Document databases: Store information as documents (in formats such as JSON and XML). For example: Firestore
• Key-value stores: Group associated data in collections with records that are identified with unique keys for easy retrieval. Key-value stores have just enough structure to mirror the value of relational databases while still preserving the benefits of NoSQL. For example: Datastore, Bigtable, Memorystore
• In-memory database: Purpose-built database that relies primarily on memory for data storage. These are designed to attain minimal response time by eliminating the need to access disks. They are ideal for applications that require microsecond response times and can have large spikes in traffic. For example: Memorystore
• Wide-column databases: Use the tabular format but allow a wide variance in how data is named and formatted in each row, even in the same table. They have some basic structure while preserving a lot of flexibility. For example: Bigtable
• Graph databases: Use graph structures to define the relationships between stored data points; useful for identifying patterns in unstructured and semi-structured information. For example: JanusGraph

There are three non-relational databases in Google Cloud:

• Firestore: Is a serverless document database which scales on demand and acts as a backend-as-a-service. It is DBaaS that increases the speed of building applications. It is perfect for all general purpose uses cases such as ecommerce, gaming, IoT and real time dashboards. With Firestore users can interact with and collaborate on live and offline data making it great for real-time application and mobile apps.  
• Cloud Bigtable: Cloud Bigtable is a sparsely populated table that can scale to billions of rows and thousands of columns, enabling you to store terabytes or even petabytes of data. It is ideal for storing very large amounts of single-keyed data with very low latency. It supports high read and write throughput at sub-millisecond latency, and it is an ideal data source for MapReduce operations. It also supports the open-source HBase API standard to easily integrate with the Apache ecosystem including HBase, Beam, Hadoop and Spark along with Google Cloud ecosystem.
• Memorystore: Memorystore is a fully managed in-memory data store service for Redis and Memcached at Google Cloud. It is best for in-memory and transient data stores and automates the complex tasks of provisioning, replication, failover, and patching so you can spend more time coding. Because it offers extremely low latency and high performance, Memorystore is great for web and mobile, gaming, leaderboard, social, chat, and news feed applications.

Conclusion

Choosing a relational or a non-relational database largely depends on the use case. Broadly, if your application requires ACID transactions and your data structure is not going to change much, select a relational database. 

In Google Cloud use Cloud SQL for any general-purpose SQL database and Cloud Spanner for large-scale globally scalable, strongly consistent use cases. In general, if your data structure may change later and if scale and availability is a bigger requirement than consistency then a non-relational database is a preferable choice.  Google Cloud offers Firestore, Memorystore, and Cloud Bigtable to support a variety of use cases across the document, key-value, and wide column database spectrum.

For more comparison resources on each database check out the overview. For more hands-on experience with Bigtable, check out our on-demand training here and learn about migrating databases to managed services check out this whitepaper.  

https://youtube.com/watch?v=2TZXSnCTd7E%3Fenablejsapi%3D1%26

For more #GCPSketchnote, follow the GitHub repo. For similar cloud content follow me on Twitter @pvergadia and keep an eye out on thecloudgirl.dev.

In a perfect cloud world, you would host all your applications in containers running on Kubernetes and Istio, benefitting from the portability and improved resource utilization of containers, plus a robust orchestration platform with advanced application management, networking, and security functionality. This is easy to do if you’re developing a new application, but it can be hard for existing applications to take advantage of those capabilities.

Many of the applications that you may want to move to the cloud have been around a long time, and you may not have the application-specific knowledge that would be required to rewrite them to be more cloud-native—or it would be incredibly time-consuming to do so.

Another option is to lift-and-shift to a virtual machine (VM) hosting platform like Compute Engine, but that means you still need to maintain the VMs. Even if you’re not able to fully modernize an existing app, it would still be great to get some of the benefits of containers and Kubernetes.

What is Migrate for Anthos?

Enter Migrate for Anthos, a fast and easy way to modernize your existing applications with a service that encapsulates them in a container. Moving your physical servers or existing VMs into Kubernetes containers gives you crucial portability and resource utilization benefits without having to rewrite the underlying application.

Since Migrate for Anthos is built for Google Kubernetes Engine (GKE), you also automatically capture the scaling and flexibility benefits of a managed Kubernetes environment in the cloud. Migrate for Anthos recently became generally available.

Converting an application with Migrate for Anthos happens in two phases. First, it creates a generic container wrapper around your application that makes it seem like it is still running in a full VM environment. Then, you launch Migrate for Anthos software on your Kubernetes cluster that runs the containerized application.

You can find more details about this in the documentation and in our blog post: Migrating from Compute Engine to Kubernetes Engine with Migrate for Anthos.

As the name suggests, Migrate for Anthos works with Anthos GKE. However, you can also use Migrate for Anthos with only GKE—all it requires is your application and a GKE cluster running the Migrate for Anthos software. 

Getting started with Migrate for Anthos

Migrate for Anthos works with a variety of workloads, but not all. It’s particularly adept at migrating legacy applications, stand-alone applications, and monolithic applications. As you start the modernization process, here are some questions to ask to determine whether to use Migrate for Anthos with your applications:

1. Should this app be in the cloud?
By its nature, the cloud may not be able to support some characteristics of your on-prem environment, such as geography and legal compliance. The best way to find out whether the cloud will work for each of your applications is to plan out a full migration. That will allow you to identify groups of applications that can benefit from cloud offerings such as a global network and ease of resizing resources. After that, try out a proof of concept by testing the apps in the cloud to see if it fits your business needs.

2. Should this app be in Kubernetes?
Containerizing an application simplifies workload administration, improves scalability (both up and down), and increases host utilization. Kubernetes orchestrates the containers and GKE handles node upgrades, while add-ons like Istio let you manage network and security policies independently of your application.

With those advantages, it’s easy to think that containers are always the right way to go, but there are some cases where it may make sense to stick with VMs. Strict hardware requirements, specialized kernel modules, and license constraints may be harder to run with containers, negating their advantages.

3. Should this app migration use Migrate for Anthos?
Migrating your apps or workloads to the cloud isn’t just about shifting where the compute resources run; it’s also an opportunity to modernize them with containers. Using Migrate for Anthos (or Migrate for Compute Engine) gives you the ability to get your workloads in the cloud quickly, with minimal upfront downtime that’s easy to plan for. 

However, even if you use the Migrate for Anthos wrapper, your application is still the same application. The benefits of the modern platform may not outweigh a legacy application and a rewrite may be the only way to meet your business needs. There are also some specific services from your VM that may not work with Migrate for Anthos, for example licensing requirements.

Migrate for Anthos can also be the first step in a larger migration effort. Once you’ve migrated the application to GKE, you can gradually break up a monolithic app into microservices by manually rewriting parts. Spreading out the migration effort gets you in the cloud sooner, giving you more time to modernize.

Next steps

A successful modernization starts with creating a full migration plan, testing the workloads, and monitoring them. You can experience the benefits of modernization with Migrate for Anthos by picking a small workload and trying it out for yourself!

Over the decades, technology has helped us organize large amounts of physical information in ways that are streamlined, efficient, and easily accessible. Rows upon rows of encyclopedias are no longer needed; simply punch in or speak a query into Google Search and find numerous results at your fingertips. There’s no need to haul around cases of CDs or cassettes either, when you can access hundreds of thousands of songs on music streaming services. 

We wanted to bring that same level of accessibility to one specific type of publication: the printed car manual. Here’s how we shifted the paper manual to become an easy-to-access, voice-activated digital experience for owners of the all-new Sienna. It’s helped this resource become just as modern and useful as Toyotas themselves.

Putting cloud technology in the driver’s seat

Far too often, the printed car manual remains unused, collecting dust in the glove compartment—that is, until it’s needed during a roadside emergency or to solve the meaning of a mysterious light popping up on the dashboard. Even then, thumbing through hundreds of pages during high-stakes moments can be stressful. On top of that, these manuals can be expensive to print and update.

Digital versions, such as a PDF file, are a nice start. But, they’re often little more than reformatted flat documents. In poor visual conditions on the side of a road, the last thing a driver wants to do is squint at a phone or scroll through pages of tiny text. And similar to printed manuals, digital versions don’t facilitate ongoing, real-time conversations between drivers and automakers when it matters the most; they’re often only text and pictures, and at best, schematic drawings.

With that in mind, we set out to elevate and personalize the car manual by creating a voice-activated digital owner’s manual experience, all powered by Google Cloud. A voice-based assistant was the clear choice because, as it felt like the most intuitive, natural option, especially while driving. In fact, 51% of U.S. adults have used a voice assistant while driving, and 95% of all drivers expect to use a voice assistant in the next three years. 

We’re now putting this technology on the road by powering the Toyota Driver’s Companion for the all-new 2021 Toyota Sienna model. Accessible through the existing Toyota app, this companion provides real-time assistance, any time of the day. Drivers can ask questions and the companion will efficiently provide helpful answers in convenient ways, from voice to 3D walkthroughs and explorable environments. 

What a modern car manual should do

The Toyota Driver’s Companion has interactive features to help drivers discover the Sienna’s dashboard, set up the car’s interior and exterior appearance, better understand vehicle maintenance, and explore Dynamic Radar Cruise Control, Lane Departure Alert and other features. 

Here are a few other additional key features to call out within the Toyota Driver’s Companion: 

• An easily accessible virtual voice through the Toyota Driver’s Companion lets app users ask personal questions about their 2021 Sienna such as “what’s the height of my car?” and receive immediate answers either by voice, display or interactive input. 
• The manual automatically connects with the purchased vehicle’s VIN number to create a completely personalized experience, curated specifically for the driver. For example, if an unfamiliar light on the dashboard pops up, the Toyota Driver’s Companion can help identify the light’s meaning.
• Interactive hotspots throughout the vehicle’s interior let drivers explore the cabin virtually. Drivers can discover button functionalities, find specific dials, and learn more about car functions, such as how to slide seats or open doors, to become acclimated with their new vehicle.

To bring this experience to life, we tapped into some of our key Google Cloud solutions:

• APIs powered by Google Cloud artificial intelligence technology make accessing specific vehicle information easy and effortless, by leveraging Google’s natural language processing:
  • Google Cloud DialogFlow API serves as the decision tree that gives intelligence for both finding an answer for a question, i.e., how the Companion responds to the end user’s questions. 
  • One of our Text-to-Speech APIs—called Wavenet—creates the Companion’s realistic voice. 
  • And finally, our Speech-to-Text API “listens” to the user’s voice and finds the correct information to craft responses. That means a driver can ask a question multiple ways, and the Companion will still respond with the right answer. 

Our Firebase mobile app dev platform gleans analytic insights that help improve the overall experience and services for OEMs and drivers alike.https://www.youtube.com/embed/66QxWS-PzIM?enablejsapi=1&

We’re encouraging better consumer experiences by providing faster access to fresh information, in a natural, accessible format—voice. But these new voice-activated experiences aren’t only an opportunity to help out drivers; it’s also about strengthening connections between drivers, their vehicles, and automakers, too. 

Our hope is that through this information exchange, drivers can provide feedback on the most frequently misunderstood features, enabling OEMs to address questions early on. By understanding the most requested features, OEMs can also predict and inform driver questions about features. We’re incredibly excited to help make the driver’s experience more connected and helpful.