KRM Series Part 5: Learn to Manage and Configure Hosted Resources with Kubernetes
3215
Of your peers have already read this article.
3:00 Minutes
The most insightful time you'll spend today!
This is the fifth and final post in a multi-part series about the Kubernetes Resource Model. Check out parts 1, 2, 3, and 4 to learn more.
In part 2 of this series, we learned how the Kubernetes Resource Model works, and how the Kubernetes control plane takes action to ensure that your desired resource state matches the running state.
Up until now, that “running resource state” has existed inside the world of Kubernetes- Pods, for example, run on Nodes inside a cluster. The exception to this is any core Kubernetes resource that depends on your cloud provider. For instance, GKE Services of type Load Balancer depend on Google Cloud network load balancers, and GKE has a Google Cloud-specific controller that will spin up those resources on your behalf.
But if you’re operating a Kubernetes platform, it’s likely that you have resources that live entirely outside of Kubernetes. You might have CI/CD triggers, IAM policies, firewall rules, databases. The first post of this series introduced the platform diagram below, and asserted that “Kubernetes can be the powerful declarative control plane that manages large swaths” of that platform. Let’s close that loop by exploring how to use the Kubernetes Resource Model to configure and provision resources hosted in Google Cloud.
Why use KRM for hosted resources?
Before diving into the “what” and “how” of using KRM for cloud-hosted resources, let’s first ask “why.” There is already an active ecosystem of infrastructure-as-code tools, including Terraform, that can manage cloud-hosted resources. Why use KRM to manage resources outside of the cluster boundary?
Three big reasons. The first is consistency. The last post explored ways to ensure consistency across multiple Kubernetes clusters- but what about consistency between Kubernetes resources and cloud resources? If you have org-wide policies you’d like to enforce on Kubernetes resources, chances are that you also have policies around hosted resources. So one reason to manage cloud resources with KRM is to standardize your infrastructure toolchain, unifying your Kubernetes and cloud resource configuration into one language (YAML), one Git config repo, one policy enforcement mechanism.
The second reason is continuous reconciliation. One major advantage of Kubernetes is its control-loop architecture. So if you use KRM to deploy a hosted firewall rule, Kubernetes will work constantly to make sure that resource is always deployed to your cloud provider- even if it gets manually deleted.
A third reason to consider using KRM for hosted resources is the ability to integrate tools like kustomize into your hosted resource specs, allowing you to customize resource specifications without templating languages.
These benefits have resulted in a new ecosystem of KRM tools designed to manage cloud-hosted resources, including the Crossplane project, as well as first-party tools from AWS, Azure, and Google Cloud.
Let’s explore how to use Google Cloud Config Connector to manage GCP-hosted resources with KRM.
Introducing Config Connector
Config Connector is a tool designed specifically for managing Google Cloud resources with the Kubernetes Resource Model. It works by installing a set of GCP-specific resource controllers onto your GKE cluster, along with a set of Kubernetes Custom Resources for Google Cloud products, from Cloud DNS to Pub/Sub.
How does it work? Let’s say that a security administrator at Cymbal Bank wants to start working more closely with the platform team to define and test Policy Controller constraints. But they don’t have access to a Linux machine, which is the operating system used by the platform team. The platform team can address this by manually setting up a Google Compute Engine (GCE) Linux instance for the security admin. But with Config Connector, the platform team can instead create a declarative KRM resource for a GCE instance, commit it to the config repo, and Config Connector will spin up the instance on their behalf.
What does this declarative resource look like? A Config Connector resource is just a regular Kubernetes-style YAML file- in this case, a custom resource called Compute Instance. In the resource spec, the platform team can define specific fields, like what GCE machine type to use.
apiVersion: compute.cnrm.cloud.google.com/v1beta1kind: ComputeInstancemetadata:annotations:cnrm.cloud.google.com/allow-stopping-for-update: "true"name: secadmin-debianlabels:created-from: "image"network-type: "subnetwork"spec:machineType: n1-standard-1zone: us-west1-abootDisk:initializeParams:size: 24type: pd-ssdsourceImageRef:external: debian-cloud/debian-9...
Once the platform team commits this resource to the Config Sync repo, Config Sync will deploy the resource to the cymbal-admin GKE cluster, and Config Connector, running on that same cluster, will spin up the GCE resource represented in the file.
This KRM workflow for cloud resources opens the door for powerful automation, like custom UIs to automate resource requests within the Cymbal Bank org.
Integrating Config Connector with Policy Controller
By using Config Connector to manage Google Cloud-hosted resources as KRM, you can adopt Policy Controller to enforce guardrails across your cloud and Kubernetes resources.
Let’s say that the data analytics team at Cymbal Bank is beginning to adopt BigQuery. While the security team is approving production usage of that product, the platform team wants to make sure no real customer data is imported. Together, Config Connector and Policy Controller can set up guardrails for BigQuery usage within Cymbal Bank.
Config Connector supports BigQuery resources, including Jobs, Datasets, and Tables. The platform team can work with the analytics team to define a test dataset, containing mocked data, as KRM, pushing those resources to the Config Sync repo as they did with the GCE instance resource.
apiVersion: bigquery.cnrm.cloud.google.com/v1beta1kind: BigQueryJobmetadata:name: cymbal-mock-load-jobannotations:configsync.gke.io/cluster-name-selector: cymbal-adminspec:location: "US"jobTimeoutMs: "600000"load:sourceUris:- "gs://cymbal-bank-datasets/cymbal-mock-transactions.csv"
From there, the platform team can create a custom Constraint Template for Policy Controller, limiting the allowed Cymbal datasets to only the pre-vetted mock dataset:
rego: |package bigquerydatasetallownameviolation[{"msg": msg}] {input.review.object.kind == "BigQueryDataset"input.review.object.metadata.name != input.parameters.allowedNamemsg := sprintf("The BigQuery dataset name %v is not allowed", [input.review.object.metadata.name])}apiVersion: constraints.gatekeeper.sh/v1beta1
These guardrails, combined with IAM, can allow your organization to adopt new cloud products safely- not only defining who can set up certain resources, but within those resources, what field values are allowed.
Manage existing GCP resources with Config Connector
Another useful feature of Config Connector is that it supports importing existing Google Cloud resources into KRM format, allowing you to bring live-running resources into the management domain of Config Connector.
You can use the config-connector command line tool to do this, exporting specific resource URIs into static files:
config-connector export "//sqladmin.googleapis.com/sql/v1beta4/projects/cymbal-bank/instances/cymbal-dev" \--output cloudsql/
Output:
apiVersion: sql.cnrm.cloud.google.com/v1beta1kind: SQLInstancemetadata:annotations:cnrm.cloud.google.com/project-id: cymbal-bankname: cymbal-devspec:databaseVersion: POSTGRES_12region: us-east1resourceID: cymbal-dev...
From here, we can push these KRM resources to the config repo, and allow Config Sync and Config Controller to start lifecycling the resources on our behalf. The screenshot below shows that the cymbal-dev Cloud SQL database now has the “managed-by-cnrm” label, indicating that it’s now being managed from Config Connector (CNRM = “cloud-native resource management”).
This resource export tool is especially useful for teams looking to try out KRM for hosted resources, without having to invest in writing a new set of YAML files for their existing resources. And if you’re ready to adopt Config Connector for lots of existing resources, the tool has a bulk export option as well.
Overall, while managing hosted resources with KRM is still a newer paradigm, it can provide lots of benefits for resource consistency and policy enforcement. Want to try out Config Connector yourself? Check out the part 5 demo.
This post concludes the Build a Platform with KRM series. Hopefully these posts and demos provided some inspiration on how to build a platform around Kubernetes, with the right abstractions and base-layer tools in mind.
Thanks for reading, and stay tuned for new KRM products and features from Google.
11255
Of your peers have already watched this video.
1:30 Minutes
The most insightful time you'll spend today!
Indian Retailer Figures Optimizes Hyperlocal Delivery to Increase Customer Experience
Anyone who follows the Indian e-commerce scene knows that one of the largest challenges these companies face is hyperlocal delivery.
That was a problem facing Wellness Forever, a retail chain of pharmacies with 150-plus stores across India.
“Exactly a year ago, we started our journey of hyperlocal deliveries. This optimization was a big time challenge for us to understand how to optimize this,” Palani Subbiah, CTO, Wellness Forever.
The problem in front of Wellness Forever was to identify which customer could can be sold from which store, so that a delivery could be made within 90 minutes.
“We handle a large amount of customer data and we wanted to use insights to help and improve the customer satisfaction index,” says Subbiah.
To do that Wellness Forever leveraged Google Big Query to run massive amount of data to come up with the operational insights. They also used Firebase and Google Maps.
“By 2021, we are going to have about 450 stores. Those stores are going to be not only a physical store, which is a digital store.
3591
Of your peers have already downloaded this article
3:30 Minutes
The most insightful time you'll spend today!
Defending the world’s largest network against persistent and constantly evolving cyber threats has driven Google to architect, automate, and develop advanced tools to help keep it ahead. Understanding how Google has built and evolved it’s defenses can help you make smart architectural decisions of your own as you move forward.
- At Google every minute:
- 10 million spam messages are prevented from reaching Gmail customers.
- 694,000 indexed Web pages are scanned for harmful software.
- 7,000 deceitful URLs, executables, and browser extensions that may carry viruses, unwanted content, or phishing attempts are spotted and stopped.
- 6000 instances of unwanted software and nearly 1,000 instances of suspected malware are reported to Chrome users.
- 2 phishing sites and 1 malware site are found and labeled.
Download this e-book to know more about Google’s security at scale.
Google Products Helps HMH’s Healthcare Staff Work from Anywhere Efficiently and Securely!
10258
Of your peers have already read this article.
3:00 Minutes
The most insightful time you'll spend today!
Hackensack Meridian Health (HMH) executive Mark Eimer explains how an ambitiously-timed rollout of a comprehensive suite of Google products helped the entire organization—from doctors to IT staff—achieve better security, cultivate a more equitable work environment, and ultimately, improve patient outcomes.
How does a recently merged, 17-hospital healthcare system fast-track a platform migration and hardware rollout securely and in a way that improves work for everyone, regardless of location or role? These are the questions that kept me up at night in early 2020, when the pandemic demanded a “big bang”—something our legacy laptops and operating systems couldn’t handle.
We began our work with Google in 2020 with the adoption of Chrome as our default browser. As we migrated platforms, keeping patient data safe was of the utmost importance to us, along with providing every staff member with the tools they needed to work virtually. Our staff often experienced issues accessing our web-based applications using Internet Explorer or Edge Browser, a problem that went away when we switched to Chrome. Chrome’s versatile compatibility also made it easier for my team to migrate all of our web-based operations, and Chrome’s security and manageability were key components to making this switch a huge win for the organization.
The success of this migration led us to extend our Google partnership to patient care applications—where Google’s expertise in AI and ML helps scale the use of diagnostics tools and improve other aspects of the patient journey.
Achieving security at every step
Like so many other healthcare organizations, we’ve been concerned about ransomware attacks. This is part of why we moved to Google Workspace and distributed over 3,000 Chrome OS devices in kiosk mode in March of 2020, when many of us went remote due to the pandemic. We were very concerned about team members accessing corporate applications through home devices that were running EOL operating systems (WIN7), as well as a general lack of antivirus and encryption measures.
We were protected by the fact that Google’s software and hardware both had built-in security features that we needed to stave off sophisticated attackers. For example, Chrome OS automatically updates to the latest security update and encrypts data living outside the cloud on the hardware. These features protected us from security-related disruptions, letting us securely move a huge library of file shares and emails across thousands of accounts to Google Chrome OS in just four months.
A year later, in March 2021, we migrated the enterprise over to Google Workspace and saw an immediate reduction in spam by 30% from the inherent built-in AI/ML. This meant staff were less likely to receive (and click through) phishing attempts. My team could connect, create, and collaborate easily and securely—even as more of us were working from home and needed to access sensitive data remotely.
Leveling the playing field
As an organization, we were surprised by how many team members didn’t have personal computers at home. We quickly decided that if we needed team members to work from home, the health network would have to supply hardware. Chromebooks’ lower price tag compared to PCs—on top of their built-in security controls—allowed us to purchase, deploy, and support that initial distribution of 3,000 Chromebooks to team members in less than three weeks, providing devices to every eligible remote employee instead of just a select few. This was vital to reaching our equitable technology goal as part of our diversity and inclusion initiative: everybody has the same tools to do good work.
When all employees have what they need to do their jobs well, we get better patient outcomes. Before we began this cloud adoption journey, patient and staff experiences were different within the hospitals and outside of them.
Now it’s the same wherever our staff is, and we’ve seen efficiency and accessibility benefits extend to the patient side. For example, we built a web-based contact center that supports 80 locations that use Workspace and Chrome OS devices. Since customer service, admin, and providers are all on the same system, it has become a one-stop shop for patients.
Furthermore, through the Grow with Google program, we were able to provide another benefit to employees that drove our equity goals. Google trained 50 non-IT staff members—from environmental services, food and nutrition, and other non-tech areas who were interested in making a career change to IT—on the Google products we were using. They may not have thought about switching to a career in IT before the Grow with Google program came to our organization, but through this partnership, they now have that opportunity.
A strategic, long-term partner
With any large-scale rollout, the work doesn’t end once laptops are in employee hands. Google has shown their commitment to long-term collaboration as they continuously optimize their products for the unique needs of healthcare providers and go the extra mile in tailoring tools to our staff’s workflows.
For example, on the Chrome OS side, the Google team has helped our registration desks and document centers with device integration for hardware like credit card readers and e-signature pads. They’ve also helped us meet security and privacy requirements mandated by state and federal governments around HIPAA, Medicaid, and Medicare reimbursements. Over this next year, we’ll look at a feature roadmap with Google Cloud to deliver further enhancements, iterating on the product itself to meet our needs for the present and the future.
Delivering the future of healthcare
The benefits we’ve seen around security and usability—and the ability to provide all staff with equal access to Google’s technology—are why we’re expanding our partnership with Google to both the administrative and clinical sides of HMH. In addition to further Google rollouts with corporate, next year we’re distributing Chromebooks to all 350 of our ambulatory clinics.
We’re also working with the Google professional services team to create a custom AI model that analyzes 3D mammogram images. This AI model will enable two providers to read mammograms—which adheres to international best practices but is currently rare in the US—without requiring additional time. Conducting double readings of mammograms will yield better health outcomes for our patients, such as a lower patient recall rate and an increased accuracy in detecting breast cancer.
We’re currently building the model using a variety of Google Cloud products, including Cloud Healthcare API. Once complete, this model is expected to be trained, deployed, and maintained in Google’s Vertex AI, allowing our providers to be more productive as they make clinical decisions with AI support. As the model is proven over time, we plan to make the predictive services accessible to other healthcare organizations.
With Google, we’re able to achieve a unified architecture for storing data as well as training and deploying AI models, which enable our staff to work more efficiently and securely from anywhere. While I may not be able to predict the future as accurately as AI can, I foresee our continued partnership with Google as a key part of HMH’s improved provider and patient outcomes.
Israeli Government Chooses Google Cloud to Power its Cloud-based ‘Project Nimbus’
3295
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
Google Cloud today announced that it has been selected by the Israeli government to provide public cloud services to help address the country’s challenges within the public sector, including in healthcare, transportation, and education.
Following a thorough public tender process, Google Cloud was selected for this four-phase project known as “Project Nimbus.” The agreement will deliver cloud services to all government entities from across the state, including ministries, authorities, and government-owned companies. The agreement is also available for higher education, health maintenance organizations and municipalities. The project will run for an initial period of 7 years, and the Israeli government may extend the engagement for up to 23 years in total.
As part of the agreement, Google Cloud will work with the public sector on the formulation of best practices for cloud migration, integration and migration to the cloud, and optimisation of cloud services. Google Cloud will also provide training to the country’s technical government employees and senior leaders to enhance digital skills. Google Cloud announced last month that it will open a Google Cloud region in Israel to make it easier for customers to serve their own users faster, more reliably and securely. The region in Israel will be available to serve not only the government and related entities but also private commercial companies, just like any other Google Cloud region.
The Accountant General of Israel, Mr. Yali Rothenberg, congratulates Google on their winning bid in the first tender of “Project Nimbus”, a multi-year flagship project led by the Israeli Government Procurement Administration, that is intended to provide a comprehensive framework for the provision of cloud services to the Government of Israel.
We are delighted to have been chosen to help digitally transform Israel. This builds on the continued success we are seeing with the public sector globally.
Google Cloud region in Israel
In April, we announced that a new Google Cloud region is coming to Israel to make it easier for customers to serve their own users faster, more reliably and securely. Our global network of Google Cloud regions are the foundation of the cloud infrastructure we’re building to support our customers in the Middle East and around the world. With cloud’s 25 regions (and forthcoming Middle East regions in Qatar and Saudi Arabia) and 76 zones around the world, we deliver high-performance, low-latency services and products for Google Cloud’s enterprise and public sector customers.
7 Fantastic Ways Google Cloud VMWare Engine Stands Out from the Rest for Running VMWare Workloads in the Cloud!
3598
Of your peers have already read this article.
2:00 Minutes
The most insightful time you'll spend today!
Google Cloud VMware Engine delivers an enterprise-grade, cloud-native VMware experience that is built on Google Cloud’s highly performant and scalable infrastructure. By enabling a consistent VMware experience, the service allows customers to adopt Google Cloud rapidly, easily, and with minimal modifications to their vSphere workloads, bringing the best of VMware and Google Cloud together on one platform for a variety of use-cases. These include rapid data center exit, application lift and shift, disaster recovery, virtual desktop infrastructure, or modernization at your own pace.
Here are seven ways VMware Engine outshines alternatives for running your VMware workloads in the cloud, simplify your operations, and help you innovate faster:
- Dedicated 100Gbps east-west networking
Google Cloud VMware Engine nodes come with redundant switching and dedicated 100Gbps east-west networking with no oversubscription of bandwidth, unlike other options where there is generally oversubscription. This is especially important when it comes to running latency-sensitive workloads. - Four 9’s of availability in a single zone
The service offers 99.99% uptime SLA for a cluster in a single Zone with five to 16 nodes and FTT=2 or more without the need for stretched clusters, which is higher than the alternatives. Further, dedicated connectivity for core service functions such as vSAN and vMotion enables better solution stability and availability. This enables the service to support the needs of enterprise workloads that require high availability.
Note: “Cluster” means a deployment of three or more dedicated bare metal nodes running VMware ESXi and associated networking managed via management interfaces.
- Global networking without complex routing
Google Cloud VMware Engine networking is built based on Google Cloud’s powerful networking architecture. With simplified regional and global routing modes—which allow a VPC’s subnets to be deployed in any region where our service is available—you can architect global networks without the need or overhead of creating and connecting regional network designs. You get instant, direct Layer 3 access between them. In alternative cloud environments, you may have to configure special networking between regions, often requiring VPN-based tunnels over the WAN to enable global uniform network communication. This adds to the deployment and operational complexity, in addition to cost. - Integrated multi-VPC networking
Users often have application deployments in different VPC networks, such as separate dev/test and production environments or multiple administrative domains across business units. The service supports “many-to-many” access from VPC networks to Google Cloud VMware Engine networks with multi-VPC networking, allowing you to retain existing deployed architectures and extend them flexibly to your VMware environments. In addition, by providing multi-VPC networking, you can pool their VMware needs—say for QA and dev—to a smaller set of clusters, effectively reducing their costs.
For more information about the end-to-end networking capabilities and services available in Google Cloud VMware Engine, please refer to the Private Cloud Networking for Google Cloud VMware Engine whitepaper. Here, you’ll find details about network flows, configuration options, and the differentiated benefits of running your VMware workloads in Google Cloud.
- Unified, cloud-integrated model
Google Cloud VMware Engine is a fully managed Google first-party service, operated and supported by Google and its world-class team. With fully integrated identities, billing, and access control, you have a simpler end-to-end experience that is different from other services. You access Google Cloud VMware Engine service via the Google Cloud console, like any other Google Cloud service. You can also access other native Google Cloud services privately from your VMware private cloud running in Google Cloud VMware Engine over local connections. - Flexibility in third-party ecosystem compatibility
With Google Cloud VMware Engine, you can set up existing VMware on-premises third-party tools or products that require additional privileges by using a solution user account. This uniquely enables operational consistency, ensuring that the tools you have invested in and used over the years work on Google Cloud VMware Engine. Furthermore, in key areas such as vSAN data encryption, you have the choice of not only using Google Cloud Key Management Service (KMS)—which is turned on by default on vSAN datastores—but also external KMS providers such as HyTrust, Thales, and Fortanix. - Dense nodes with high storage:core and memory:core ratios and fast provisioning
Google Cloud VMware Engine nodes are dense. Each node is powered by Intel® Xeon® Scalable Processors and comes with 36 cores, 72 hyperthreaded cores, 768 GB memory, 19.2 TB NVMe data and 3.2 TB NVMe cache storage. This, along with oversubscription, leads to high consolidation ratios and compelling storage:core per dollar and memory:core per dollar. In addition, you can rapidly spin up these nodes in a VMware private cloud often in under an hour, enabling on-demand, VMware-consistent capacity in Google Cloud for your needs.
These are just a few examples of customer-centric innovation that set Google Cloud VMware Engine infrastructure apart. In addition, migrating to Google cloud can save you up to 38% in TCO. Get started by learning about Google Cloud VMware Engine and your options for migration, or talk to our sales team to join the customers who have embarked upon this journey.
The authors would like to thank the Google Cloud VMware Engine product team for their contributions on this blog.
More Relevant Stories for Your Company
Google Cloud and StartEd Join Forces to Boost EdTech Startups
Over the past few years, as the education sector was going through transformative change, EdTech startups rose to meet the demand of learners and help address some of the biggest challenges in education. At Google Cloud, we're inspired by how EdTech startups continue to solve challenges with agility, innovative technology,
Google Cloud Connector for SAP LaMa Helps Make Most of the Multi & Hybrid-cloud Strategies
As an SAP user, you may already be familiar with SAP Landscape Management (SAP LaMa) — a specialized product that supports centralized management of your SAP landscape. SAP LaMa is useful for a wide range of SAP customers and use cases, but it's especially important to large enterprises that manage
Canadian Bank’s SAP Workload Moved to BigQuery Helps Unlock New Business Opportunities
When ATB Financial decided to migrate its vast SAP landscape to the cloud, the primary goal was to focus on things that matter to customers as opposed to IT infrastructure. Based in Alberta, Canada, ATB Financial serves over 800,000 customers through hundreds of branches as well as digital banking options. To
Cloud as an Innovation Platform in Capital Markets
Public cloud, big data, and AI technologies offer competitive advantages and cost savings for capital markets firms ready to make the transition. This paper discusses the three phases capital markets firms go through in transitioning to public cloud, and the workloads, benefits, and cultural changes that characterize the three phases:
