To help improve your security posture, improve the reliability of your applications, and reduce configuration drift in your environment, we’re excited to announce a new Anthos reference architecture.

Written in collaboration across our product, engineering, support, and field teams, this new reference architecture helps you plan, deploy, and configure the required components for Anthos hybrid environments.

Anthos hybrid environments give you the flexibility to deploy on-premises components that run container-based workloads and VMs using Anthos clusters on VMware and Anthos clusters on bare metal. You can continue to utilize existing investments in your on-premises infrastructure, and start to add components like Anthos Config Management. When you’re ready to bring everything together, you can add additional Google Cloud-based services like Artifact Registry, Cloud Monitoring, and Identity and Access Management (IAM).

The following sneak peek covers some of our best practices for architecting an Anthos hybrid environment. For more detailed guidance and planning information, see the full Anthos hybrid environment reference architecture.

When you design and deploy an Anthos hybrid environment, we recommend that you use two or more on-premises computing customer sites and two or more Google Cloud regions. In your sites, run multiple clusters. This approach is recommended for several reasons, such as:

• Disaster recovery. If one cluster or site fails, you can continue to run workloads.
• Multiple environments, like production and staging, to test infrastructure changes.
• Different cluster types in each environment: admin clusters and user clusters. This approach separates administrative resources, which is a security best-practice.

The following diagram shows an example of an Anthos hybrid environment that’s spread across customer sites and regions, with different clusters for admin and user workloads and for production and staging:

In each site, you can use Anthos clusters on VMware or Anthos clusters on bare metal. For both products, we recommend the following:

• Use a highly available (HA) control plane with three members for continued control plane availability concurrently with operating system upgrades, control plane software updates, or single-machine hardware or kernel failures.
• Deploy two admin clusters so that admin cluster configuration changes and updates can be tested in the staging environment first.

The following diagram shows an example of Anthos clusters on bare metal with control plane and worker nodes spread across physical machines. With Anthos clusters on VMware, the control plane and worker nodes are spread across VMware VMs:

Configure your on-premises clusters and applications to send logging and monitoring data back to Google Cloud for analysis and review. Different personas should only be granted access to the environments they need. The following diagram shows how application developers and application or platform operators can then view logging and monitoring data in Google Cloud:

Use Anthos Config Management to manage Kubernetes objects in your clusters. Anthos Config Management is a GitOps-style tool that uses a Git repository or Open Container Initiative (OCI) as its storage mechanism and source of truth. Git provider workflows allow multiple stakeholders to participate in review of changes.

As shown in the following diagram, a common Anthos Config Management deployment uses one folder containing configuration for all clusters. Use separate additional folders to hold configuration data, one for application:

Plan and implement a way to secure the network traffic in your Anthos hybrid environment. The following services help with authentication, connectivity, and communication in a cluster:

• Anthos Identity Service connects clusters to on-site identity providers to authenticate local access.
• Connect gateway and workforce identity federation can provide secure cloud-mediated access to mobile workforce clusters without using a VPN.
• Workload Identity provides on-premises workloads with managed short-lifetime credentials for access to cloud resources.
• Anthos Service Mesh encrypts and controls communication between services in the same cluster.

The following diagram shows how Anthos Service Mesh can control the flow of traffic between services within your clusters:

You don’t have to implement all these cloud-based services as part of your initial on-premises deployments. As you become more comfortable and want to expand your capabilities, you can add in some of these hybrid offerings. But, we hope that this blog post has given you some ideas to think about when you start to plan and design your own Anthos hybrid environments.

For more detailed guidance and planning information, see the full Anthos hybrid environment reference architecture. Let us know what you think!

You have something in your pocket right now that possesses more computing power than spacecraft fifteen billion miles away from Earth. No, not your mobile phone — the key fob for your car! Voyager 1 and Voyager 2 launched in 1977, and forty five years later, these little rascals still work and send data back to Earth while moving through cold, empty space at over thirty thousand MPH. Think your key fob will still be going strong in 2068? It’s also amazing that the Romans created a concrete so long-lasting that two thousand years later, structures built with it are still in use. Meanwhile, the roads outside my house develop potholes if a rabbit sneezes on them. 

I’m a fan of durability because it allows you to think outside the box. If I know that my foundations are safe, I’ll be more inclined to push the boundaries to innovate outside my comfort zone because the environment can handle it. Durability needs to be a cornerstone of the application modernization conversation, as each application and environment we create for our customers is the first step of their next-generation technology and business strategy. An entire organization’s goals and dreams rests on the shoulders of the technology architectures we are building, and we need to ensure that the foundation has the strength to withstand everything the future can throw at it — and then some.

When it comes to selecting the appropriate foundational technologies for modern applications, it’s crucial to consider the longevity and reliability of the technology. Will an exciting new open-source large language model be around in a year? Or that interesting new database from PayPal? How do I choose something from the ever-growing CNCF landscape diagram? This is where the strength of open-source communities and vendor backing comes into play. For instance, Kubernetes, PostgreSQL, and Java have stood the test of time, thanks to their robust feature sets, dedicated communities, and strong vendor support.

Kubernetes provides a scalable solution for managing and deploying applications, with Google’s strong backing: As of July, we’ve made over 1,000,000 contributions to the k8s project — 2.3 times more than any other contributor. PostgreSQL, meanwhile, is one of the world’s most advanced open-source databases and offers a comprehensive set of features that cater to a wide range of data processing needs, with a vibrant community constantly enhancing its capabilities. And Java, a general-purpose programming language, has been a staple in the development community for decades, providing a reliable platform for building robust applications.

Choosing an established technology not only brings the benefit of a mature feature set but also the assurance of continuity. A new database or language model may be promising, but they lack the track record of these tested technologies. The risk of adopting such new technologies is their potential discontinuation or lack of support, which could jeopardize your application’s stability and longevity. 

The Voyager team’s use of aluminum foil is a great illustration of this principle. They chose a simple, reliable, and available solution to protect sensitive instruments during their mission. The choice of aluminum foil might not have been the most cutting-edge or exciting, but it was practical, reliable, and ultimately successful. Similarly, when choosing foundational technologies for your modern applications, sometimes the “boring” choice is the best one. It’s not about chasing the latest trends; it’s about choosing what works and stands the test of time.

Vendor backing is another critical consideration when choosing foundational technologies. A reliable platform provider that runs these technologies ensures a high-uptime Service Level Agreement (SLA). For example, Google Kubernetes Engine (GKE) offers a 99.95% uptime SLA, while Bigtable “just works,” and Cloud Storage doesn’t lose data thanks to a design that supports 99.999999999% annual durability.

Boring doesn’t mean blah

That’s not to say we shouldn’t experiment with new technologies and encourage our customers to do the same. Everyone needs an innovation strategy. The concept of an ‘innovation spectrum’ is helpful here. This spectrum represents different degrees of technological innovation that companies can employ based on their specific needs and capabilities. On one end of the spectrum, there’s incremental innovation, which involves making small improvements or extensions to existing products, services, or processes. On the other end, there’s radical or disruptive innovation, which involves creating entirely new products or services that can potentially disrupt entire industries.

A classic example of balancing cutting-edge technology with “boring” or legacy technology is seen in many financial institutions. They might use AI and ML for fraud detection or predictive analytics while still relying on tried and true technologies for their core banking systems. This blend of newer and older technologies allows them to benefit from the latest advancements without jeopardizing the stability and reliability of their critical operations. However, as the banking industry is finding out, that can also come at a risk of stifling innovation and can cause customers to look elsewhere.

For developers, Google Cloud’s Kubernetes-based platforms present a similar balance between innovation and stability. For example, researchers can leverage cutting-edge GPU sharing in GKE to explore the origins of the universe, while the BBC uses Cloud Run serverless containers to keep up with demands of a very busy news day. 

Adopting best practices like platform engineering can provide a robust foundation for rolling out new technologies. Platform engineering focuses on creating a stable, scalable, and secure platform that allows for rapid deployment of applications. GitOps is another important practice that involves using Git as a single source of truth for declarative infrastructure and applications. With Git at the center of the delivery pipelines, developers can use familiar tools to make pull requests. Changes can be rolled out or rolled back easily, making the process of adopting new technologies smoother.

When it comes to modern application development, developers need to be able to trust that the foundational technologies they choose will be reliable and durable. Without this assurance, developers may be hesitant to take risks or explore creative solutions. To give them the confidence they need, an effective platform engineering strategy can provide a strong foundation for rolling out new technologies while ensuring stability and security.

Boring can be beautiful, especially if you’re building for the long haul. Regardless of what you’re developing, from roads and rocketships to microservices or network architecture, the fundamental structure needs to withstand everything the conceivable future can throw at it. A solid, durable foundation offers developers the capabilities they need to push the boundaries, and the reliability they need so their brainchild is still humming along, 15 billion miles away.

Today, more and more applications are being deployed in containers on Kubernetes—so much so that we’ve heard Kubernetes called the Linux of the cloud.

Despite all that growth on the application layer, the data layer hasn’t gotten as much traction with containerization. That’s not surprising, since containerized workloads inherently have to be resilient to restarts, scale-out, virtualization, and other constraints. So handling things like state (the database), availability to other layers of the application, and redundancy for a database can have very specific requirements. That makes it challenging to run a database in a distributed environment. 

However, the data layer is getting more attention, since many developers want to treat data infrastructure the same as application stacks.

Operators want to use the same tools for databases and applications, and get the same benefits as the application layer in the data layer: rapid spin-up and repeatability across environments. In this blog, we’ll explore when and what types of databases can be effectively run on Kubernetes.

Before we dive into the considerations for running a database on Kubernetes, let’s briefly review our options for running databases on Google Cloud Platform (GCP) and what they’re best used for.

• Fully managed databases. This includes Cloud Spanner, Cloud Bigtable and Cloud SQL, among others. This is the low-ops choice, since Google Cloud handles many of the maintenance tasks, like backups, patching and scaling. As a developer or operator, you don’t need to mess with them. You just create a database, build your app, and let Google Cloud scale it for you. This also means you might not have access to the exact version of a database, extension, or the exact flavor of database that you want.
• Do-it-yourself on a VM. This might best be described as the full-ops option, where you take full responsibility for building your database, scaling it, managing reliability, setting up backups, and more. All of that can be a lot of work, but you have all the features and database flavors at your disposal.
• Run it on Kubernetes. Running a database on Kubernetes is closer to the full-ops option, but you do get some benefits in terms of the automation Kubernetes provides to keep the database application running. That said, it is important to remember that pods (the database application containers) are transient, so the likelihood of database application restarts or failovers is higher. Also, some of the more database-specific administrative tasks—backups, scaling, tuning, etc.—are different due to the added abstractions that come with containerization.

Tips for running your database on Kubernetes

When choosing to go down the Kubernetes route, think about what database you will be running, and how well it will work given the trade-offs previously discussed.

Since pods are mortal, the likelihood of failover events is higher than a traditionally hosted or fully managed database. It will be easier to run a database on Kubernetes if it includes concepts like sharding, failover elections and replication built into its DNA (for example, ElasticSearch, Cassandra, or MongoDB). Some open source projects provide custom resources and operators to help with managing the database.

Next, consider the function that database is performing in the context of your application and business. Databases that are storing more transient and caching layers are better fits for Kubernetes. Data layers of that type typically have more resilience built into the applications, making for a better overall experience.  

Finally, be sure you understand the replication modes available in the database. Asynchronous modes of replication leave room for data loss, because transactions might be committed to the primary database but not to the secondary database(s). So, be sure to understand whether you might incur data loss, and how much of that is acceptable in the context of your application.

After evaluating all of those considerations, you’ll end up with a decision tree looking something like this:

How to deploy a database on Kubernetes

Now, let’s dive into more details on how to deploy a database on Kubernetes using StatefulSets.

With a StatefulSet, your data can be stored on persistent volumes, decoupling the database application from the persistent storage, so when a pod (such as the database application) is recreated, all the data is still there.

Additionally, when a pod is recreated in a StatefulSet, it keeps the same name, so you have a consistent endpoint to connect to. Persistent data and consistent naming are two of the largest benefits of StatefulSets. You can check out the Kubernetes documentation for more details.

If you need to run a database that doesn’t perfectly fit the model of a Kubernetes-friendly database (such as MySQL or PostgreSQL), consider using Kubernetes Operators or projects that wrap those database with additional features. Operators will help you spin up those databases and perform database maintenance tasks like backups and replication. For MySQL in particular, take a look at the Oracle MySQL Operator and Crunchy Data for PostgreSQL. 

Operators use custom resources and controllers to expose application-specific operations through the Kubernetes API. For example, to perform a backup using Crunchy Data, simply execute pgo backup [cluster_name]. To add a Postgres replica, use pgo scale cluster [cluster_name].

There are some other projects out there that you might explore, such as Patroni for PostgreSQL. These projects use Operators, but go one step further. They’ve built many tools around their respective databases to aid their operation inside of Kubernetes. They may include additional features like sharding, leader election, and failover functionality needed to successfully deploy MySQL or PostgreSQL in Kubernetes.

While running a database in Kubernetes is gaining traction, it is still far from an exact science. There is a lot of work being done in this area, so keep an eye out as technologies and tools evolve toward making running databases in Kubernetes much more the norm. 

When you’re ready to get started, check out GCP Marketplace for easy-to-deploy SaaS, VM, and containerized database solutions and operators that can be deployed to GCP or Kubernetes clusters anywhere.