Founded in 1930 and headquartered in Jordan, Arab Bank is one of the oldest banks in the Middle East. Operating out of 28 countries, we’ve earned our customers’ trust with a prudent approach to operations and respect for the cultures and customs in the region. 

With a few exceptions where cloud providers have hosted their datacenter in a Middle Eastern or North African country, the banking sector, in general, in the region has been slow to adopt cloud technology for a number of reasons, including concern about data security, maturity and security controls of cloud services (PaaS and SaaS), and regulations in place. But on the other hand, we saw the opportunity to accelerate our development and testing using the cloud, as well as to partner with the fintech community and digital service providers to integrate their solution in the banking ecosystem. We needed more flexibility to connect with the outside world, and a more open architecture to help us drive our internal innovation at a faster rate with the help of the fintech industry. By collaborating with Google Cloud, we reached those goals and accelerated app development and testing through products like Apigee and Anthos. We’re now offering innovative apps and services to our customers and employees that leverage new technological capabilities to give more agility and flexibility, and to optimize our workloads.

Embracing the cloud in a regulated industry 

To get started with the cloud, we needed to create internal awareness about cloud technology, the API layer, containers and their benefits amongst our leaders and staff. Google helped us educate and get buy-in from key functions by organizing open technology demonstration sessions and discussion panels. When considering potential cloud providers we had four decision criteria: maturity of security controls, ease of use, cost, and scalability / agility for new deployments and continuous innovation. This last factor was critical, and we were impressed by Google Cloud’s innovation roadmap, both via direct conversations and at Google’s Next conference, where we met a lot of people passionate about technology, innovation and building something new.  

Going back to our journey, given the above-mentioned regional limitations, we started to develop a hybrid cloud approach. This helped us continue to operate on-premises for a number of services in production, particularly those that have personally identifiable information (PII) or other sensitive data attached, and to leverage the cloud for development, testing and production workloads that don’t contain customer data. 

In the short term, we didn’t anticipate that our many jurisdictions would allow data to be transported to other countries. But cloud tools will allow us to tokenize or anonymize customer data while maintaining customer data on-premises. This applies to many digital journeys such as customer onboarding, credit facility online applications, or marketplace navigation. In the coming years, we predict our API integration with partners will accelerate and enrich the overall digital value proposition of our business segments, namely consumer banking, small and medium-size businesses and large corporate and institutional clients. 

Building connections and cornerstones in the cloud

The first move in our digital transformation was to implement Apigee, Google Cloud’s API management platform, to connect to the world’s digital banking ecosystem. Apigee provides the security, sharing, mediation policies, and developer portal capabilities for us to successfully meet Open Banking standards while focusing on innovation.  

On the back of the Apigee implementation, we created an accelerator program to incubate Fintech ideas that can, in turn, integrate into our digital platforms and be offered to our customers. We also developed various banking APIs, all designed and documented in accordance with PSD2 and Open Banking regulations, and made them available to our partners. These APIs exposed on our API development portal offer the needed code structure for fintech companies to design creative solutions around them.

Next, we adopted Anthos, Google Cloud’s managed application platform. Anthos has become a cornerstone of our operations because it works across hybrid cloud, offering integration of microservice containers and fueling collaborative opportunities with external parties. Our current Anthos infrastructure includes several hundreds of  microservices now running on containers in Google Kubernetes Engine (GKE) and on-premises. We now use the cloud for collaboration, development and testing, but not for production, which is done on-premises. 

Along the way, Google Cloud’s Professional Services Organization (PSO) helped us through the entire cloud setup process, and with the adoption of Anthos. We originally built on the cloud tools through an iterative process, learning from our successes and errors along the way. Now that we have a better sense of how Anthos operates, we’re building a fresh infrastructure atop a sound, stable, and resilient foundation that will let us scale easily as we work to transform Arab Bank into a digital-first enterprise, that is our ambition. 

Currently products running on Anthos include customer acquisition and onboarding via mobile apps, and our Arabi-Pay app, which allows customers to instantly pay each other via WhatsApp or other messaging platforms. Leveraging Anthos, our instant loan service for Arab Bank salaried employees can grant and disburse loans up to $7,000 in less than seven minutes.

In addition, we’ve built a number of digital journeys for our Small and Medium Enterprise (SME) customers, such as our SME client digital onboarding process and paperless SME lending platform.

While some may think that digital adoption in this part of the world can be slow, as customer contact remains anchored in our customs, the recent COVID-19 pandemic has accelerated the adoption of digital banking services and electronic payments, inspiring more confidence to buy and pay online. Thanks to our rich and user-friendly banking app that relies on Apigee and Anthos for critical customer journeys, over 90% of new-to-bank customers are using our mobile apps. Within the next 18 months, we predict that number will be closer to 100%. 

Of course, with higher customer adoption comes the challenge of potential service interruptions. A single moment of downtime can be highly visible to many digital customers. But Google Cloud’s Anthos and Apigee give us the flexibility to resume processes at a fast rate, so any interruptions are almost invisible to our customers. In fact, when the COVID-19 pandemic hit, though our branches could be open only for limited hours each day, our consumer clients in particular were able to take advantage of our digital services in a very self-sufficient manner. Being well positioned with Google Cloud, we could also keep our internal teams and external partners connected and productive. Without Google Cloud, continuing the digital transformation of the bank at the pace we wanted would have been a big challenge. 

Collaborating across borders and time zones

With Google Cloud, our ability to collaborate and partner has transformed significantly. We operate 24/7 now because our developers are scattered across multiple geographies and different time zones. Because testing and deployment can run around the clock, including on weekends, we currently deploy a new digital journey in a few weeks, faster than ever before. This has given our organization a spirited mindset that prioritizes innovation, and raised the bar in terms of our operating model.  

Another consideration about Google Cloud tools is the elimination of inefficient processes typically seen in a software development lifecycle. We now build in a completely agile manner, from design squads until deployment in production. Compared to where we were two years ago, when we had an annual maximum of two systems releases in production, we now have close to monthly releases of our digital packages. In addition, we also  supplement those monthly releases with additional ad-hoc releases and fixes in between. As a result, we have removed internal silos and improved tremendously the collaboration between the product and sales teams, operations, IT Dev Factory and Infrastructure, as well as our supporting functions.

Through the agile process facilitated by APIs, we introduced Design Thinking workshops involving external customers and prospects early on to understand their true pain points better and emotions during the existing journeys and how to make new digital journeys frictionless. As a result, the relevance of our products for various customer personas has improved tremendously. 

Transcending banking 

With Google Cloud, we can offer our customers so much more than just banking. We’ve become more digitally relevant to their lives. For example, we recently launched a mortgage app that helps customers all the way from home selection through mortgage negotiations and closing, and even to getting the home decorated. It’s an end-to-end journey in which API integration with key regional players was a cornerstone to our success. 

For other digital products, we have an extensive roadmap of lifestyle-based solutions relevant to each segment and age group. We’ve only scratched the surface of the services we can provide, and we see the cloud as the future for everything we want to do.

Read more about Google Cloud’s Open Banking solution to learn how you can simplify and accelerate the process of delivering open banking as required by PSD2. You can also view our video on Open Banking, powered by Apigee API Management.

The software development process requires complex, cross-functional collaboration while continuously improving products and services. Our customers who build software say that they value Google Workspace for its ability to drive innovation and collaboration throughout the entire software development life cycle. Developers can hold standups and scrums in Google Chat, Meet, and Spaces, create and collaborate on requirements documentation in Google Docs and Sheets, build team presentations in Google Slides, and manage their focus time and availability with Google Calendar.

Development teams also use many other tools to get work done, like tracking issues and tasks in Atlassian’s Jira, managing workloads with Asana, and incident management in PagerDuty. One of the benefits of Google Workspace is that it’s an open platform tailored to improve the performance of your tools by seamlessly integrating them together. We’re constantly expanding our ecosystem and improving Google Workspace, giving you the power to push your software development even further.

Make software development more agile

Google Workspace gives you real-time visibility into project progress and decisions to help you ship quality code fast and stay connected with your stakeholders, all without switching tools and tabs. By leveraging applications from our partners, you can pull valuable information out of silos, making collaborating on requirements, code reviews, bug triage, deployment updates, and monitoring operations easy for the whole team. This allows your teams to stay focused on their priorities while keeping everyone aligned, ensuring collaborators are always in the loop.

Plan and execute together

When combined with integrations, Google Workspace makes the software development planning process more collaborative and efficient. For example, many organizations use Asana—a leading work management platform—to coordinate and manage everything from daily tasks to cross-functional strategic initiatives. To make the experience more seamless, Asana built integrations so users can always have access to their tasks and projects with Google Drive, Gmail, and Chat. With these integrations for Google Workspace, you can turn your conversations into action and create new tasks in Asana—all without leaving Google Workspace.

“We’ve seen exceptional, heavy adoption of tasks being created from within the Gmail add-on. Our customers and community have also shown very strong interest in future development work, which is something we’ll continue to prioritize.” Strand Sylvester, Product Manager, Asana

To date, users have installed the Asana for Gmail add-on over 2.5 million times, as well as over 3.8 million installs of the Asana for Google Workspace add-on for Google Drive.

Start coding quickly

Google Workspace makes it easy for product managers, UX designers, and engineers to agree on what they’re building and why. By bringing all stakeholders, decisions, and requirements into one place—whether it’s a Gmail or Google Chat conversation, or a document in Google Docs, Sheets, or Slides—Google Workspace removes friction, helping your teams finalize product specifications and get started right away.

Integrations like GitHub for Google Chat make the entire development process fit easily into a developer’s workflow. With this integration, teams can quickly push new commits, make pull requests, do code reviews, and provide real-time feedback that improves the quality of their code—all from Google Chat.

Speed up testing

Integrations like Jira for Google Chat accelerate the entire QA process in the development workflow. The app acts as a team member in the conversation, sending new issues and contextual updates as they are reported to improve the quality of your code and keep everyone informed on your Jira projects.

Ship code faster

Developers use Jenkins—a popular open-source continuous integration and continuous delivery tool—to build and test products continuously. Along with other cloud-native tools, Jenkins supports strong DevOps practices by letting you continuously integrate changes into the software build.

With Jenkins for Google Chat, development and operations teams can connect into their Jenkins pipeline and stay up to date by receiving software build notifications directly in Google Chat.

Proactively monitor your services

Improving the customer experience requires capturing and monitoring data sources to improve application and infrastructure observability. Google Workspace supports DevOps teams and organizations by helping stakeholders collaborate and troubleshoot more effectively. When you integrate Datadog with Google Chat, monitoring data becomes part of your team’s discussion, and you can efficiently collaborate to resolve issues as soon as they arise.

The integration makes it easy to start a discussion with all the relevant teams by sharing a snapshot of a graph in any of your Chat spaces. When an alert notification is triggered, it allows you to notify each Chat space independently, precisely targeting your communication to the right teams.

Improve service reliability

Orchestrating business-wide responses to interruptions is a cross-functional effort. When revenue and brand reputation depends on customer satisfaction, it’s important to proactively manage service-impacting events. Google Workspace supports response teams by ensuring that urgent alerts reach the right people by providing teams with a central space to discover incidents, find the root cause, and resolve them quickly.

PagerDuty for Google Chat empowers developers, DevOps, IT operations, and business leaders to prevent and resolve business-impacting incidents for an exceptional customer experience—all from Google Chat. See and share details with link previews, and perform actions by creating or updating incidents. By keeping all conversations in a central space, new responders can get up to speed and solve issues faster without interrupting others.

Accelerate developer productivity

Integrating your DevOps tools with Google Workspace allows your development teams to centralize their work, stay focused on what’s important—like managing their work—build code quickly, ship quality products, and communicate better during service impacting incidents. For more apps and solutions that help centralize your work so you and your teams can connect, create, and get things done, check out Google Workspace Marketplace, where you’ll find more than 5,300 public applications that integrate directly into Google Workspace.

How many times have you closed an application when you encounter the “spinning wheel of death” (a melodramatic way of saying an application that is taking too long to load)? In today’s digital economy where many organizations rely on applications as a primary source of revenue, that spinning wheel of death (or poor application performance) can mean lost users or revenue. And just about every modern application relies on APIs as the nervous system across distributed systems, third-party services and microservice architectures. While meeting the need for rapid release cycles and frequent API updates, it is also imperative for IT teams to ensure that your APIs are meeting SLOs, performance requirements, and proactively mitigating issues.

Why are synthetic monitoring tools not enough?

When thousands or even millions of users are making multiple requests to your APIs, just relying on synthetic monitoring tools (that rely on sampling or limited API availability information) is not enough for precision diagnoses or useful forensics. At the same time, monitoring every single aspect just increases your overhead and mean time to diagnosis. Apropos, API monitoring has become absolutely critical — and a fusion of art and science — for operations teams to make sure all APIs are running and performing as intended. If you are worried about monitoring blind spots or overheads, let us look at 3 key practices you can follow to stop dreading your sev1 alerts.

#1 Prioritize alerts for critical events requiring immediate investigation

• Ask any engineer on-call for a critical service and they will tell you about the overheads created by incorrectly prioritized alerts. For example let us imagine a distributed application with 20 APIs. Even if you set up basic alert monitors across latency, error, and traffic for these APIs, you end up monitoring and maintaining 60 alert definitions – which is a lot. To balance avoiding monitoring blindspots and alert fatigue at the same time, operations teams must develop a clear understanding of all events and prioritize configuring alerts for events supporting critical traffic.

• Consider quality over quantity while defining new alert conditions where each new condition is urgent, actionable, and actively or imminently user-visible. Every alert condition created should also contain intelligence that requires active engagement from a user as opposed to a mere robotic response. Apigee’s API monitoring allows creation of alert conditions based on metrics or logs while providing actionable information (Ex: status code, rate etc.,) and playbooks for diagnosi
• In today’s multilayered systems, one team’s symptom (“what’s broken?”) is another downstream system’s cause (“Why?”). Even if some events are not suitable for actionable alerts, a failure needs to create an informational broadcast to a downstream system to mitigate the impact of the upstream dependency. In such cases, investing in automating alerts, grouping multiple incidents in notification channels, and incident tracking. For example, Apigee lets you integrate and group your alert notifications in channels like Slack, Pagerduty, webhooks etc.,
• Modern production systems are ever-evolving where an alert that’s currently rare might become frequent and automatable. Analogous to ticket backlog grooming, alert policies need to be reviewed periodically to make sure new conditions are identified and existing alerts are refined with new thresholds, prioritization and correlation. Controls like Advanced API Ops leverage AI and ML to detect anomalous traffic differentiated from random fluctuations to help define accurate alert definitions

Check out some examples of alerts here

#2 Isolate problem areas quickly with dashboards

Google’s Site Reliability Engineering book presents the case for efficient diagnosis by building dashboards that answer basic questions about every service, normally including some form of the four golden signals — latency, traffic, errors and saturation. But at the same time, capturing just these golden metrics at different levels of granularity can quickly add up. Like all software systems, monitoring can become an endless pit of complexity, complicated to change and a burden to maintain. In the same book, the most effective direction to create a well-functioning standalone system is to collect and aggregate basic metrics, paired with alerting and dashboards.

If you are running a large scale API program with a dedicated team to monitor your APIs, you can leverage the out-of-the-box monitoring dashboards in your API Management solution (like Apigee’s API monitoring) to gather real-time insights into your API performance, availability, latency, and errors. In other cases, you can use solutions like Cloud Monitoring that can provide visibility across your full application stack where individual metrics, events, and metadata can be visualized in a rich query language for rapid analysis. Leveraging a single system for your application stack provides observability in context and can reduce your time spent navigating between systems (Apigee customers can use Cloud Monitoring by default or integrate with other systems using Cloud Monitoring API)

Even after you collect and aggregate the metrics, it is important to have impactful data visualizations to quickly understand the issue and identify correlations during diagnosis. In data visualizations as well, focusing on too many dashboards creates a steep learning curve and increases mean time for every diagnosis. For example, Apigee API Monitoring provides the following visualizations as a standard to balance simplicity and efficiency:

• Timeline view to diagnose traffic (in 1 min intervals), error rates (4xx and 5xx across traffic) and latencies (50th, 90th, 95th and 99th percentiles)
• Pivot tables of metrics and attributes for all API traffic, to help compare activity across different metrics
• Treemaps of recent API traffic by proxy to get a snapshot of incidents, error rates and latencies

#3 Incorporate distributed tracing into your end to end observability strategy

Modern application development accelerated the adoption of technologies and practices like cloud, containers, APIs, microservice architectures, DevOps, SRE etc. While this increases release velocity, it also introduces complexity and more points of failure in an application stack. For example, a slow response to a customer request spans across multiple micro services owned (and monitored) by various teams who might not observe any individual performance issues. Without an end-to-end contextual view of a request, it is nearly impossible to isolate the point of high latency.

In such cases, distributed tracing is the best way for DevOps, Operations and SREs to get answers to questions such as service health, root cause of defects, or performance bottlenecks in a distributed system. Organizations should invest in instrumenting their distributed applications using open source standards such as OpenCensus and Zipkin. Using tools like Cloud Trace with a broad platform, language and environment support can help easily ingest data from any source — open instrumentation or proprietary agents.

While distributed tracing helps in narrowing the issue to a given service, in some cases you might need further context to pinpoint the root cause. For example: Even if you have isolated the source of a performance issue to an API proxy, it is still a tedious process to identify the right bottleneck among multiple policies being executed. Tools like Apigee Debug enable you to zoom into an API proxy flow and probe the details of each step to see internal details like policy executions, performance issues, and routing etc.,

As soon as a request starts to span across a handful of microservices, tools like distributed tracing and Debug will become crucial elements of your monitoring strategy. When every service in a distributed system emits a trace, the amount of data can quickly become overwhelming leading to the classic “needle in the haystack” problem. In such scenarios, it becomes vital to ask the right questions and choose between head-based sampling (randomly selecting which traces will be sampled for analysis) and tail-based sampling (observe all trace information and sample the traces with unusual latency or errors) based on application complexity

Implement effective API monitoring in Apigee

Apigee’s API monitoring (based on metrics exposed by the internals of the system) capabilities work with your existing monitoring infrastructure to help reduce mean time to diagnosis and increase application resiliency. Specifically, operations teams can leverage

• Monitoring dashboards to gain in-depth insights into API availability and performance metrics.
• Debug to precisely diagnose with deeper insights into an API proxy flow without toggling multiple tools.
• Alerts and notifications to gather contextual Insights, facilitate customization, and grouping with first class integrations to various tools (like Slack, PagerDuty, email and support for webhooks).
• Best of Google technologies such as Data Flow, Pub/Sub, Stackdriver, Bigtable and BigQuery to handle massive volumes and complex metrics at scale

Using Apigee’s API monitoring will help you maintain high application resiliency with comprehensive controls to reduce mean time to diagnosis and resolution. Get started with Apigee today or explore Apigee’s API monitoring for free here. Check out our documentation for additional information on API monitoring.

Once upon a time, all Google Kubernetes Engine (GKE) clusters used public IP addressing for communication between nodes and the control plane. Subsequently, we heard your security concerns and introduced private clusters enabled by VPC peering.

To consolidate the connectivity types, starting in March 2022, we began using Google Cloud’s Private Service Connect (PSC) for new public clusters’ communication between the GKE cluster control plane and nodes, which has profound implications for how you can configure your GKE environment. Today, we’re presenting a new consistent PSC-based framework for GKE control plane connectivity from cluster nodes. Additionally, we’re excited to announce a new feature set which includes cluster isolation at the control plane and node pool levels to enable more scalable, secure — and cheaper! — GKE clusters.

New architecture

Starting with GKE version 1.23 and later, all new public clusters created on or after March 15th, 2022 began using Google Cloud’s PSC infrastructure to communicate between the GKE cluster control plane and nodes. PSC provides a consistent framework that helps connect different networks through a service networking approach, and allows service producers and consumers to communicate using private IP addresses internal to a VPC.

The biggest benefit of this change is to set the stage for using PSC-enabled features for GKE clusters.

The new set of cluster isolation capabilities we’re presenting here is part of the evolution to a more scalable and secure GKE cluster posture. Previously, private GKE clusters were enabled with VPC peering, introducing specific network architectures. With this feature set, you now have the ability to:

• Update the GKE cluster control plane to only allow access to a private endpoint
• Create or update a GKE cluster node pool with public or private nodes
• Enable or disable GKE cluster control plane access from Google-owned IPs.

In addition, the new PSC infrastructure can provide cost savings. Traditionally, control plane communication is treated as normal egress and is charged for public clusters as a normal public IP charge. This is also true if you’re running kubectl for provisioning or other operational reasons. With PSC infrastructure, we have eliminated the cost of communication between the control plane and your cluster nodes, resulting in one less network egress charge to worry about.

Now, let’s take a look at how this feature set enables these new capabilities.

Allow access to the control plane only via a private endpoint

Private cluster users have long had the ability to create the control plane with both public and private endpoints. We now extend the same flexibility to public GKE clusters based on PSC. With this, if you want private-only access to your GKE control plane but want all your node pools to be public, you can do so.

This model provides a tighter security posture for the control plane, while leaving you to choose what kind of cluster node you need, based on your deployment.

To enable access only to a private endpoint on the control plane, use the following gcloud command:

gcloud container clusters update CLUSTER_NAME \
--enable-private-endpoint

Allow toggling and mixed-mode clusters with public and private node pools

All cloud providers with managed Kubernetes offerings offer both public and private clusters. Whether a cluster is public or private is enforced at the cluster level, and cannot be changed once it is created. Now you have the ability to toggle a node pool to have private or public IP addressing.

You may also want a mix of private and public node pools. For example, you may be running a mix of workloads in your cluster in which some require internet access and some don’t. Instead of setting up NAT rules, you can deploy a workload on a node pool with public IP addressing to ensure that only such node pool deployments are publicly accessible.

To enable private-only IP addressing on existing node pools, use the following gcloud command:

gcloud container node-pools update POOL_NAME \
--cluster CLUSTER_NAME \
--enable-private-nodes

To enable private-only IP addressing at node pool creation time, use the following gcloud command:

gcloud container node-pools create POOL_NAME \
--cluster CLUSTER_NAME \
--enable-private-nodes

Configure access from Google Cloud

In some scenarios, users have identified workloads outside of their GKE cluster, for example, applications running in Cloud Run or any GCP VMs sourced with Google Cloud public IPs were allowed to reach the cluster control plane. To mitigate potential security concerns, we have introduced a feature that allows you to toggle access to your cluster control plane from such sources.

To remove access from Google Cloud public IPs to the control plane, use the following gcloud command:

gcloud container clusters update CLUSTER_NAME \
--no-enable-google-cloud-access

Similarly, you can use this flag at cluster creation time.

Choose your private endpoint address

Many customers like to map IPs to a stack for easier troubleshooting and to track usage. For example — IP block x for Infrastructure, IP block y for Services, IP block z for the GKE control plane, etc. By default, the private IP address for the control plane in PSC-based GKE clusters comes from the node subnet. However, some customers treat node subnets as infrastructure and apply security policies against it. To differentiate between infrastructure and the GKE control plane, you can now create a new custom subnet and assign it to your cluster control plane.

gcloud container clusters create CLUSTER_NAME \
--private-endpoint-subnetwork=SUBNET_NAME

What can you do with this new GKE architecture?

With this new set of features, you can basically remove all public IP communication for your GKE clusters! This, in essence, means you can make your GKE clusters completely private.

You currently need to create the cluster as public to ensure that it uses PSC, but you can then update your cluster using gcloud with the --enable-private-endpoint flag, or the UI, to configure access via only a private endpoint on the control plane or create new private node pools.

Alternatively, you can control access at cluster creation time with the --master-authorized-networks and --no-enable-google-cloud-access flags to prevent access from public addressing to the control plane.

Furthermore, you can use the REST API or Terraform Providers to actually build a new PSC-based GKE cluster with the default (thus first) node pools to have private nodes. This can be done by setting the enablePrivateNodes field to true (instead of leveraging the public GKE cluster defaults and then updating afterwards, as currently required with gcloud and UI operations).

Lastly, the aforementioned features extend not only to Standard GKE clusters, but also to GKE Autopilot clusters.

When evaluating if you’re ready to move these PSC-based GKE cluster types to take advantage of private cluster isolation, keep in mind that the control plane’s private endpoint has the following limitations:

• Private addresses in URLs for new or existing webhooks that you configure are not supported. To mitigate this incompatibility and assign an internal IP address to the URL for webhooks, set up a webhook to a private address by URL, create a headless service without a selector and a corresponding endpoint for the required destination.
• The control plane private endpoint is not currently accessible from on-premises systems.
• The control plane private endpoint is not currently globally accessible: Client VMs from different regions than the cluster region cannot connect to the control plane’s private endpoint.

All public clusters on version 1.25 and later that are not yet PSC-based are currently being migrated to the new PSC infrastructure; therefore, your clusters might already be using PSC to communicate with the control plane.

To learn more about GKE clusters with PSC-based control plane communication, check out these references:

• GKE Concept page for public clusters with PSC
• How-to: Change Cluster Isolation page
• How-to: GKE node pool creation page with isolation feature flag
• How-to: Schedule Pods on GKE Autopilot private nodes
• gcloud reference to create a cluster with a custom private subnet
• Terraform Providers Google: release v4.45.0 page
• Google Cloud Private Services Connect page.

Here are the more specific features in the latest Terraform Provider, handy to integrate into your automation pipeline:

Terraform Providers Google: release v4.45.0

• gcp_public_cidrs_access_enabled
• enable_private_endpoint
• private_endpoint_subnetwork
• enable_private_nodes