Picking the right database for your application is not easy. The choice depends heavily on your use case—transactional processing, analytical processing, in-memory database, and so on—but it also depends on other factors. This post covers the different database options available within Google Cloud across relational (SQL) and non-relational (NoSQL) databases and explains which use cases are best suited for each database option. 

Relational databases 

In relational databases information is stored in tables, rows and columns, which typically works best for structured data. As a result they are used for applications in which the structure of the data does not change often. SQL (Structured Query Language) is used when interacting with most relational databases. They offer ACID consistency mode for the data, which means:

• Atomic: All operations in a transaction succeed or the operation is rolled back.
• Consistent: On the completion of a transaction, the database is structurally sound.
• Isolated: Transactions do not contend with one another. Contentious access to data is moderated by the database so that transactions appear to run sequentially.
• Durable: The results of applying a transaction are permanent, even in the presence of failures.

Because of these properties, relational databases are used in applications that require high accuracy and for transactional queries such as financial and retail transactions. For example: In banking when a customer makes a funds transfer request, you want to make sure the transaction is possible and it actually happens on the most up-to-date account balance, in this case an error or resubmit request is likely fine.

There are three relational database options in Google Cloud: Cloud SQL, Cloud Spanner, and Bare Metal Solution.

Cloud SQL: Provides managed MySQL, PostgreSQL and SQL Server databases on Google Cloud. It reduces maintenance cost and automates database provisioning, storage capacity management, back ups, and out-of-the-box high availability and disaster recovery/failover. For these reasons it is best for general-purpose web frameworks, CRM, ERP, SaaS and e-commerce applications.

Cloud Spanner: Cloud Spanner is an enterprise-grade, globally-distributed, and strongly-consistent database that offers up to 99.999% availability, built specifically to combine the benefits of relational database structure with non-relational horizontal scale. It is a unique database that combines ACID transactions, SQL queries, and relational structure with the scalability that you typically associate with non-relational or NoSQL databases. As a result, Spanner is best used for applications such as gaming, payment solutions, global financial ledgers, retail banking and inventory management that require ability to scale limitlessly with strong-consistency and high-availability. 

Bare Metal Solution: Provides hardware to run specialized workloads with low latency on Google Cloud. This is specifically useful if there is an Oracle database that you want to lift and shift into Google Cloud. This enables data center retirements and paves a path to modernize legacy applications. 

Non-relational databases

Non-relational databases (or NoSQL databases) store compex, unstructured data in a non-tabular form such as documents. Non-relational databases are often used when large quantities of complex and diverse data need to be organized. Unlike relational databases, they perform faster because a query doesn’t have to access several tables to deliver an answer, making them ideal for storing data that may change frequently or for applications that handle many different kinds of data. 

For example, an apparel store might have a database in which shirts have their own document containing all of their information, including size, brand, and color with room for adding more parameters later such as sleeve size, collars, and so on.

Qualities that make NoSQL databases fast:

• Eventual consistency: stores usually exhibit consistency at some later point (e.g., lazily at read time)
• Horizontal scaling, usually using hashed distributions
• Typically, they are optimized for a specific workload pattern (i.e., key-value, graph, wide-column)
• Typically, they don’t support cross shard transactions or flexible isolation modes.

Because of these properties, non-relational databases are used in applications that require large scale, reliability, availability, and frequent data changes.They can easily scale horizontally by adding more servers, unlike some relational databases, which scale vertically by increasing the machine size as the data grows. Although, some relations databases such as Cloud Spanner support scale-out and strict consistency.

Non-relational databases can store a variety of unstructured data such as documents, key-value, graphs, wide columns, and more. Here are your non-relational database options in Google Cloud: 

• Document databases: Store information as documents (in formats such as JSON and XML). For example: Firestore
• Key-value stores: Group associated data in collections with records that are identified with unique keys for easy retrieval. Key-value stores have just enough structure to mirror the value of relational databases while still preserving the benefits of NoSQL. For example: Datastore, Bigtable, Memorystore
• In-memory database: Purpose-built database that relies primarily on memory for data storage. These are designed to attain minimal response time by eliminating the need to access disks. They are ideal for applications that require microsecond response times and can have large spikes in traffic. For example: Memorystore
• Wide-column databases: Use the tabular format but allow a wide variance in how data is named and formatted in each row, even in the same table. They have some basic structure while preserving a lot of flexibility. For example: Bigtable
• Graph databases: Use graph structures to define the relationships between stored data points; useful for identifying patterns in unstructured and semi-structured information. For example: JanusGraph

There are three non-relational databases in Google Cloud:

• Firestore: Is a serverless document database which scales on demand and acts as a backend-as-a-service. It is DBaaS that increases the speed of building applications. It is perfect for all general purpose uses cases such as ecommerce, gaming, IoT and real time dashboards. With Firestore users can interact with and collaborate on live and offline data making it great for real-time application and mobile apps.  
• Cloud Bigtable: Cloud Bigtable is a sparsely populated table that can scale to billions of rows and thousands of columns, enabling you to store terabytes or even petabytes of data. It is ideal for storing very large amounts of single-keyed data with very low latency. It supports high read and write throughput at sub-millisecond latency, and it is an ideal data source for MapReduce operations. It also supports the open-source HBase API standard to easily integrate with the Apache ecosystem including HBase, Beam, Hadoop and Spark along with Google Cloud ecosystem.
• Memorystore: Memorystore is a fully managed in-memory data store service for Redis and Memcached at Google Cloud. It is best for in-memory and transient data stores and automates the complex tasks of provisioning, replication, failover, and patching so you can spend more time coding. Because it offers extremely low latency and high performance, Memorystore is great for web and mobile, gaming, leaderboard, social, chat, and news feed applications.

Conclusion

Choosing a relational or a non-relational database largely depends on the use case. Broadly, if your application requires ACID transactions and your data structure is not going to change much, select a relational database. 

In Google Cloud use Cloud SQL for any general-purpose SQL database and Cloud Spanner for large-scale globally scalable, strongly consistent use cases. In general, if your data structure may change later and if scale and availability is a bigger requirement than consistency then a non-relational database is a preferable choice.  Google Cloud offers Firestore, Memorystore, and Cloud Bigtable to support a variety of use cases across the document, key-value, and wide column database spectrum.

For more comparison resources on each database check out the overview. For more hands-on experience with Bigtable, check out our on-demand training here and learn about migrating databases to managed services check out this whitepaper.  

https://youtube.com/watch?v=2TZXSnCTd7E%3Fenablejsapi%3D1%26

For more #GCPSketchnote, follow the GitHub repo. For similar cloud content follow me on Twitter @pvergadia and keep an eye out on thecloudgirl.dev.

May is a big month for the security industry. It’s been over a year since we gathered for RSA in San Francisco for one of 2020’s last major in-person events. While we likely won’t be together in person this year, it’s an important time for the security community to come together and reflect on many accomplishments, and to consider the challenges still ahead of us. As the world focuses on security incidents and all the risks that still need resolving, it is important to stand back, on occasion, and also note that immense progress has been made by large numbers of small, medium and large enterprises to protect themselves and their customers against increased threats. What is also amazing is to see organizations do this while accelerating their digital transformations, supporting and protecting customers and managing ongoing remote working challenges. We are privileged to play our part in supporting those great teams. 

It’s also been a busy month for us here at Google Cloud since our inaugural CISO perspectives blog post in April. Today, I’ll recap our cloud security and industry highlights, a sneak peak of what’s ahead from Google at RSA and more. 

Thoughts from around the industry 

• Risk Governance of Digital Transformation in the Cloud – In our latest Office of the CISO whitepaper, we shared guidance on both the challenges and opportunities of cloud transformation for Chief Risk Officers, Chief Compliance Officers, Heads of Internal Audit and their teams. A misconception we sometimes see among these executives is that moving to the cloud creates more risk to manage. Having held these leadership positions in previous roles, I believe that the cloud is as much a means of managing security, resilience and other risks as it is a risk in its own right. The whitepaper dives deep into considerations for each of these leadership functions as their organization embarks on a digital transformation journey. 
• The importance of meeting global compliance requirements – Compliance is critical for building trust with customers in regulated industries, especially the public sector. It is worth remembering that in any critical industry, where there can be material impact from incidents, strong industry practices and standards to protect customers are vital (I wrote about this last summer). At Google Cloud, we’re regularly adding new compliance and security certifications to meet our customers’ needs globally. Recently, we expanded our list of FedRAMP High-certified products to include Cloud DNS, and helped our customers in the Asia-Pacific region address various compliance requirements to meet new government regulations for security and data protections. Google Cloud was also the only cloud service provider to complete an annual pooled audit with the Collaborative Cloud Audit Group (CCAG), which is a syndicate of 39 leading European financial institutions and insurance companies who depend on cloud infrastructure and technologies to deliver innovative solutions and experiences for their customers. Having spent most of my career in the financial services industry, I know firsthand the importance of managing risk assessments for outsourced vendors to provide the necessary assurances customers need from their cloud providers. 

RSA 2021 

We have a great lineup of speaking sessions and keynotes from Googlers at RSA this year. Below are the highlights you don’t want to miss: 

• I’ll be doing a session on May 20 about supply chain resilience, where a panel of experts will dive into how we can adjust risk and security initiatives to handle the next “punch to the supply chain.” Additionally, on May 18 I’ll join many of my esteemed CISO leaders from various industries and governments for a keynote discussion on our top security insights, lessons learned and best practices for how we move forward as an industry to address the next wave of challenges. 
• Google’s Senior Director of Information Security Heather Adkins will deliver a session on how to build secure and reliable systems at scale, which will cover principles from Google’s Site Reliability Engineering book with the same title (available for free download here). I’m most looking forward to Heather’s advice for how we as an industry can reshape our security thinking, based on modern architectures and technologies that can help organizations design scalable and reliable systems that are fundamentally secure.
• Nelly Porter, Senior Product Manager at Google Cloud Security, will participate in a panel discussion with security experts on the importance of Confidential Computing technology, how it’s changing the security landscape and where it’s headed. Google Cloud has made great progress in delivering a Confidential Computing portfolio for our customers in regulated industries over the past year, and we’re excited for new milestones in 2021. 

Google cloud security highlights

• Infrastructure and SRE spotlight – Before I joined Google Cloud, I always admired the infrastructure and benefits this organization delivers that are uniquely Google – from the subsea cable innovations to SRE inventions and principles. Security and resiliency are baked into every layer of our infrastructure. Many of the Googlers who build and support our platform have sat in the same seat as our customers, so they understand those needs intimately. Over the last few months it’s been amazing to watch our technical infrastructure team grow, and the direct reliability, operational resilience and security benefits that team brings to our customers. For example, we’ve opened a new region in Poland, announced the first subsea cable that will directly connect the U.S. to Singapore with fiber pairs over an express route, and released an SRE book focused on how organizations can complete a successful cloud migration.
• New security foundations blueprint guide – As part of our mission to deliver the industry’s most trusted cloud, we strive to operate in a shared-fate model for risk management in conjunction with our customers. This includes sharing opinionated step-by-step guidance with key decision points and focus areas for how our customers deploy workloads in Google Cloud. This is why we’ve updated our Google Cloud security foundations guide and corresponding Terraform blueprint scripts. These blueprints are tremendously helpful to many stakeholders within an enterprise, like a CISO that needs to understand our key principles for cloud security, or a C-Suite business leader that needs to quickly identify the skills their teams need to meet an organization’s security, risk, and compliance needs on Google Cloud. 

When we think about the types of features to build into products, we have many principles we follow. But the two that I keep coming back to as crucial are:

1. The need for secure products not just security products. All products should have security built in and while we do build great security products our security and other teams remain focused on constantly enhancing the base levels of security and the security features in all our products. 
2. Defense in Depth. We don’t just focus on defense in depth from attacks – for ourselves and our customers. We also prioritize defense in depth from configuration errors or other hazards. 

As you see below in some of the highlights of new features and products, these represent our commitment to secure products and all forms of defense in depth. 

• Workload identity federation – Service account keys are powerful credentials, and can represent a security risk if they are not managed correctly. A safer approach is to use workload identity federation, using IAM to grant external identities IAM roles, including the ability to impersonate service accounts. This lets you access resources directly and eliminates the maintenance and security burden associated with service account keys. We also offered related overall guidance on the best way to use and authenticate service accounts on Google Cloud.
• VPC-SC Directional Policies – With VPC Service Controls (VPC-SC), admins can define a security perimeter around Google-managed services to control communication to and between those services. Using VPC-SC, you can isolate your production GCP resources from unauthorized VPC networks or the internet. But what if you need to transfer data between isolated environments that you’ve set up? VPC-SC directional policies is a new secure data exchange feature that allows you to configure efficient, private, and secure data exchange between isolated environments. 
• Anthos service mesh supports VMs as well as clusters – Most enterprise compute resources are still in VMs and many will remain there for a long time to come. In Anthos 1.7,  your VM-based workloads can now take advantage of the same mesh functionality as your container-based workloads.
• Cloud Spanner CMEK and Access Approvals – Cloud Spanner is Google Cloud’s fully managed relational database that offers unlimited scale, high performance, strong consistency across regions and high availability. Spanner now supports customer-managed encryption keys (CMEK) and Access Approval, Google Cloud’s industry-leading controls to require approval before access to your content by Google support and engineering teams.
• External Key Manager enhancements – In early 2020 we launched Cloud External Key Manager (Cloud EKM), the industry’s leading Hold-Your-Own-Key (HYOK) product. Using Cloud EKM, the keys used to protect your data stored and processed in Google Cloud are completely hosted and managed outside of Google Cloud infrastructure. Cloud EKM initially launched with support for BigQuery and GCE/PD;  we expanded support for Cloud SQL, GKE, Dataflow Shuffle, and Secret Manager, with CMEK support currently in beta. We also provided in-depth documentation on the functionality, architecture and use cases for Cloud EKM in a new whitepaper.
• Web App and API Protection solution –  Web applications and public APIs are increasingly important to how organizations interface with their customers and partners, and we’ve seen increased investment in tools to protect these resources from fraud and abuse. Google Cloud’s new Web App and API protection solution is based on the same technology Google uses to protect its public-facing services against web application exploits, DDoS attacks, fraudulent bot activity, and API targeted threats. It provides protection across clouds and on-premises environments.
• Threat Intel for Chronicle – Most threat intelligence feeds require security teams to do the implementation and legwork. With our new Threat Intel for Chronicle offering, however, our intelligence insights are applied automatically across your security telemetry to present unique observations within your environment. Threat Intel for Chronicle is exclusively curated for enterprise customers by Uppercase, Google Cloud’s intelligence research and applications team to provide our perspective on threats across the internet and surface them as relevant alerts.

That wraps up another month of thoughts and highlights. If you’d like to have this Cloud CISO Perspectives post delivered every month to your inbox, click here to sign-up, and we’ll see you in June!

About FNM Group

FNM Group is the main private, integrated group in transportation and mobility in the North of Italy.

Industries: Other

Location: Italy

About Innext

Innext, one of Italy’s select Google Cloud Premier Partners, helps companies turn objectives into tangible actions, and change into a competitive advantage.

Industries:

Location:

Founded in 1877, the FNM Group is Italy’s second largest railway company, operating in the Lombardy region. With 700,000 passengers a day, the core business remains with railway operations, but in recent years the FNM Group has redefined itself as a mobility service provider rather than just a public transport operator, with transport solutions like car-sharing platforms in parts of the country disconnected from the main railway lines. As part of this evolution, in 2016, FNM Group initiated a new strategy to overhaul its technology and build a platform that could support its ambitious, new direction.

“We needed an IT system that could rapidly adapt to the new market requirements we are planning to address,” says Augusto de Castro, Director of Human Resources, Organization, and IT at FNM Group. “To achieve this goal, we decided to modernize our information system and move to a cloud-based infrastructure, so we started testing various solutions. For us, the best came from Google.”

Collaborative office productivity

Since 2005, the FNM Group had used SAP Business Suite for its platform, using on-premises servers to manage its technology and database solutions. By 2016, when the group expanded its horizons, the on-premises infrastructure was showing its age and limitations. In addition, FNM Group office productivity tools limited workers’ mobility and flexibility. The company decided that a cloud-based solution was the most economical and effective infrastructure for its new innovation-focused strategy, but also wanted to continue the high-quality service it received from SAP.

In April 2017, FNM Group teamed up with Innext, a leading Italian consultancy and Google Cloud Premier Partner, to help transition to the cloud. After taking the time to assess specific needs at FNM Group, Innext supported the company’s decision to start the project with a migration to G Suite, replacing the existing email and office productivity platform. With help from Innext, FNM Group provided its workers with new tools such as Gmail, cloud-based storage on Google Drive, and fast, effective collaboration with Google Hangouts. As well as technical help, Innext brought a whole change management program designed to help FMN workers take full advantage of the new platform. “One of our top goals in any project is the adoption of the G Suite platform,” says Andrea Servili, Co-founder and Partner at Innext.

Cloud-based enterprise solutions with SAP HANA and SAP S/4HANA

While defining the G Suite migration, Innext and FNM Group began looking for an infrastructure solution. The timing was perfect. Earlier in the year, Google announced that SAP HANA and SAP S/4HANA, the latest software from SAP, would be able to run on Google Cloud Platform (GCP). Running on Google Compute Engine instances, Google Cloud Storage for backups, and Google Virtual Private Cloud as a networking solution, FNM Group can continue to use the enterprise platform of its choice along with all the advantages of Google cloud technology. By July, 2018, FNM Group plans to have fully migrated to SAP HANA and SAP S/4HANA on GCP and have its 1,200 employees using G Suite.

“SAP HANA and SAP S/4HANA on Google Cloud Platform and G Suite will make our employees much more mobile,” says Augusto. “They can access the system through a web browser if they have to, so they’re not tied to a local physical interface.”

Building a platform to last

With Innext and SAP HANA and SAP S/4HANA on GCP, FNM Group is building an enterprise infrastructure that can handle the demands of its traditional public transport operations and adapt to the company’s more innovative activities, such as car-sharing or energy management. By the time the migration is complete in 2018, FNM Group will save on infrastructure costs by only paying for the resources it uses with flexible pricing from Google. G Suite lets the company’s workers stay mobile and effective, and with Google Compute Engine, FNM Group can easily scale its infrastructure up or down to meet whatever challenges it faces in the future. Meanwhile, Google Cloud Storage helps ensure that FNM Group will have a highly secure, reliable backup solution. Once the migration is complete, FNM Group can start exploring other ways in which Google can help its business, especially with data analytic products like Google BigQuery or Google Cloud AI machine learning tools. Whatever the challenge, FNM Group knows that it can rely on Innext and Google to provide the right solution.

“Our main priority is to shut down our on-premises infrastructure and take advantage of the benefits of the cloud,” says Augusto. “Google Cloud Platform and SAP HANA and SAP S/4HANA will help us reduce our maintenance costs and improve our flexibility. I think we will spend less time continually upgrading our infrastructure and more time improving our productivity.”

Casper Labs announced a collaboration with Google Cloud that will allow developers to launch public and/or private Casper nodes directly from Google Cloud. This enables a much more seamless and highly secure process for the millions of developers who want to build in blockchain environments without having to learn new, highly specialized programming languages. Additionally, Google Cloud will provide its scalable and reliable infrastructure to developers building on the Casper Protocol. 

Blockchain technology is maturing 

As blockchain technology matures, a growing number of businesses are embracing it as a key way to drive new efficiencies and realize cost savings. 

According to a recent Casper Labs study, 87% of executives polled in the United States, United Kingdom and China reported plans to invest in a blockchain solution in 2023. This is due in no small part due to recent innovations that help organizations overcome the so-called Blockchain Adoption Trilemma, which previously held that it was impossible for any blockchain to be simultaneously a) decentralized, b) scalable, and c) secure.

Thanks to the rise of proof-of-stake blockchains like Casper, new models have emerged that enable a more scalable and secure architecture that no longer forces a compromise on decentralization. 

Another trend facilitating these growing adoption rates is the rise of WebAssembly (WASM) as a baseline technology for newer blockchains, including Casper. WASM (created by W3C) makes application development in blockchain environments far more accessible and interoperable to the millions of developers worldwide who specialize in languages like Java, Javascript, C++ and Rust. Previously, any blockchain-based build required a high degree of specialized developer knowledge, which made it a much more challenging option for most organizations. 

Meet Casper

Casper is a permissionless, decentralized public blockchain based on WASM that was built explicitly to foster enterprise adoption of blockchain technology. Beyond its more accessible model, Casper is the first and only blockchain to offer native upgradable smart contracts. This means that organizations can have the option to securely and consistently update software code even after it is running on Casper. This gives organizations the control and flexibility to use industry best practices, such as continuous deployment and continuous integration, which are already in use in their IT departments. Casper is also highly configurable and allows organizations to support public, private, and/or hybrid deployments. 

Casper is also noteworthy for the presence of Casper Labs, a software development and professional services firm that supports organizations building on the Casper network. Unlike most blockchains that follow a more traditional open-source project, Casper Labs provides around-the-clock support and bespoke software development for enterprise organizations. Recently, Casper Labs helped patent management company IPwe execute the largest-ever blockchain deployment, featuring more than 25 million patents being added as custom NFTs to the Casper Blockchain. 

How to get started with Casper on Google Cloud

Developers who want to start building on Casper can find a comprehensive series of tutorials here.

The Casper Association also recently announced a $25 million grant program to support projects and developers building on Casper. Interested participants can apply here.