As CentOS 7 reaches end of life, many enterprises are considering their options for an enterprise-grade, downstream Linux distribution on which to run their production applications. Rocky Linux has emerged as a strong alternative that, like CentOS, is 100% compatible with Red Hat Enterprise Linux.

In April 2022, we announced a customer support partnership with CIQ, the official support and services partner and sponsor of Rocky Linux, as the first step in providing a best-in-class enterprise-grade supported experience for Rocky Linux on Google Cloud. Today we’re excited to announce the general availability of Rocky Linux Optimized for Google Cloud. We developed this collection of Compute Engine virtual machine images in close collaboration with CIQ so that you get optimal performance when using Rocky Linux on Compute Engine to run your CentOS workloads.

These new images contain customized variants of the Rocky Linux kernel and modules that optimize networking performance on Compute Engine infrastructure, while retaining bug-for-bug compatibility with Community Rocky Linux and Red Hat Enterprise Linux. The high bandwidth networking enabled by these customizations will be beneficial to virtually any workload, and are especially valuable for clustered workloads such as HPC (see this page for more details on configuring a VM with high bandwidth).

Going forward, we’ll collaborate with CIQ to publish both the community and Optimized for Google Cloud editions of Rocky Linux for every major release, and both sets of images will receive the latest kernel and security updates provided by CIQ and the Rocky Linux community. And of course, we’ll offer support with CIQ for both these images, per our partnership.

Rocky Linux Optimized for Google Cloud lets you take advantage of everything Compute Engine has to offer, including day-one support for our latest VM families, GPUs, and high-bandwidth networking. And for customers building for a multi-cloud deployment environment, the community Rocky images have you covered.

Starting today, Rocky Linux 8 Optimized for Google Cloud is available for all x86-based Compute Engine VM families (and soon for the new Arm-based Tau T2A), with version 9 soon to follow. Give it a try and let us know what you think.

Most large enterprises started with their own data centers and developed in-house solutions to meet their specific business needs, regulations, and industry specifications. These solutions were based on traditional applications from legacy vendors like Oracle and Microsoft — or even mainframes — and from key technology providers like SAP and VMware. During the last decade, organizations started their digital transformation journey by migrating their IT infrastructure, applications, and data to the cloud to reduce operating costs and gain efficiencies as their needs scaled.

The next step in the journey was to take advantage of cloud-based data analytics, services and scalability to drive business growth and revenue. Rather than offer generic “industry clouds” which require multi-year IT deployments and significant customization, Google Cloud has responded to customer needs by developing production-ready industry solutions that address specific use cases with repeatability across organizations.

These industry solutions are grounded in Google Cloud’s differentiated capabilities, including AI, ML, and data analytics. With these, you can dramatically reduce implementation time and realize value from the cloud more quickly with deep insights into your customers and more efficient interactions with your suppliers and partners.

Here are some highlights from the That Digital Show Podcast where Umesh Vemuri, VP of Global Strategic Customers and Industries at Google Cloud, discusses his industry strategy.

Why Google Cloud solutions?

“We are uniquely positioned to help enterprises with solutions based on our deep understanding of customer needs that are supported by our technologies. There are three examples that come to mind.
The first is our understanding of the consumer from the standpoint of running an ecommerce platform. We have the solutions to run digital platforms at a very large scale, including the engineering practices we bring to support organizations during important high traffic events like Black Friday and Cyber Monday.

The second example deals with media and entertainment as a whole. We operate the world’s largest streaming service and support the largest digital platforms currently out there. That means that whatever problems our media customers are experiencing, there’s a good chance we’ve already seen them and know how to deal with them.

Finally, there is our experience with AI and big data. Our leadership in deploying our AI and big data healthcare technologies has given us the experience to solve complex problems — such as techniques to support evidence-based selection, drug therapy, and molecular profiling.”

B to B to C

“Google is a B to B organization, I always like to add that we are also B to B to C, because the concern the consumer ultimately has is their experiences and the journey that they’re on. Connecting the dots between the business and the consumer experience is really critical and a real big differentiator in how we could better serve our customers.

For example, take Ford Motors. Really think about the challenges they have — traditional problems around manufacturing, core I. T. modernization and information, and how to remove costs. But then when you really think about the core of the business, how do you actually make this incredible experience for drivers of Ford vehicles? What do you want to do from an infotainment perspective? What do you want to do in speech-to-text conversion?

And suddenly your business is really a direct-to-consumer experience. Ultimately, all the infrastructure and technology is designed to provide the consumer — who’s ultimately buying that vehicle — with an amazing experience that will maintain their loyalty.

From this example, we can see this kind of linkage in every industry: from retail and e-commerce, media, and direct streaming to healthcare with direct telehealth.”

Solutions that transform the consumer experience

“The consumer’s expectations are constantly shifting and we have to be able to provide the technologies, the structure, and the solutions to our customers to be able to meet those changing expectations at that consumer level.

So first, we want to be very clear on the industry segments that we’re going to focus on and what we believe our clear differentiation will be. So we’ve focussed on ten industries including: retail, financial services, manufacturing, telecommunications, media and entertainment, healthcare and life sciences, education, government, supply chain and logistics and gaming.

Second, we really have to be very prescriptive about the solution pillars in the areas where our customers tell us we have challenges. We want to build solutions that solve not only today’s issues, but the problems of the future.

And third, in those pillars, we have to be very clear on what are the specific use cases that we think have high value for our customers. Then make these available as a catalog of actual products and production-ready solutions that we and our partners in the ecosystem provide.”

Google Cloud industry solutions focus on our top ten industries where we can provide differentiated value to organizations. Whether it’s discovery in retail, AI-enabled call centers, or automotive tools for connected cars, we are delivering production-ready solutions that are ready to implement with minimum customization.

The rapidly evolving landscape of distributed computing demands efficient and scalable frameworks. Ray.io is an open-source framework to easily scale up Python applications across multiple nodes in a cluster. Ray provides a simple API for building distributed, parallelized applications, especially for deep learning applications.

Google Kubernetes Engine (GKE) is a managed container orchestration service that makes it easy to deploy and manage containerized applications. GKE provides a scalable and flexible platform that abstracts away the underlying infrastructure.

KubeRay enables Ray to be deployed on Kubernetes. You get the wonderful Pythonic unified experience delivered by Ray, and the enterprise reliability and scale of GKE managed Kubernetes. Together, they offer scalability, fault tolerance, and ease of use for building, deploying, and managing distributed applications.

In this blog post, we share a solution template to get you started easily with Ray on GKE. We discuss the components of the solution and showcase an inference example using Ray Serve for Stable Diffusion.

Overview of the Solution

In this solution template we use KubeRay, an OSS solution for managing Ray clusters on Kubernetes, as the operator for provisioning our workloads. Follow the step-by-step instructions in the README file to get started. The solution contains two groups of resources: platform-level and user-level.

Platform-level resources are expected to be deployed once for each development environment by the system administrator. These include the common infrastructure and GCP service integrations that are shared by all users.

• GKE cluster and node pool. Configurations can be changed in the main.tf file. This module deploys a GKE cluster with a GPU node pool, including required Nvidia drivers for GPUs. You can replace these with other machine types.
• Kubernetes system namespace and service accounts, along with the necessary IAM policy bindings. This allows the platform administrator to provide fine-grained user access control and quota policies for Ray cluster resources.
• KubeRay operator. The operator is responsible for watching for changes in KubeRay resources and reconciling the state of the KubeRay clusters.  
• Logging. The `logging_config` section enables logs from system components and workloads to write logs to Cloud logging.
• Monitoring. The `monitoring_config` section enables Managed Prometheus integration. This allows the deployment to automatically scrape system-level metrics and writes them to the managed metrics service.
• Workload identity. This enables your workloads to authenticate with other GCP services using Google IAM service accounts.

User-level resources are expected to be deployed once by each user in the development environment.

• KubeRay cluster. This is the actual Ray cluster that we will be used for your workloads. It is configured to use a Workload Identity pool and a IAM-binded service account that provides fine-grained access to GCP services. You can customize the Ray cluster settings by editing the kuberay-values.yaml file.

• Logging. The solution adds a side car container deployed alongside each KubeRay worker node. This uses fluentbit to forward Ray logs from the head node to Cloud logging. You can edit the fluentbit-config file to change how the logging container filters and flushes logs.
• Monitoring. This module provides a PodMonitoring resource that scrapes metrics from the user’s Ray cluster and uploads data points to Google Managed Prometheus. An optional installation for Grafana dashboard is included and can be accessed through a web browser.
• JupyterHub server. This module installs a JupyterHub notebook server in the user namespace, enabling users to interact directly with their Ray clusters.

Run a Workload on Your Ray Cluster

Let’s try running the provided example with Ray Serve to deploy Stable Diffusion. This example was originally taken from the Ray Serve documentations here. To open the example in Jupyter notebook, go to the external IP for proxy-public in your browser (instructions to get the IP). And then click on File -> Open from URL, and input the raw URL of the notebook to open it.

Since the notebook runs in the same Kubernetes cluster as the Ray cluster, it is able to talk directly to the latter using its cluster-internal service endpoint – thus there is no need to expose the Ray cluster to public internet traffic. For production workloads, you should secure your endpoints with GCP account credentials. Google Cloud Identity Aware Proxy (IAP) can be used to enable fine-grained access control to user resources, such as our Ray cluster, to protect your GCP resources from unnecessary exposure. A full tutorial on how to enable IAP on your GKE cluster can be found here.

The notebook contains code for deploying a pre-trained model to a live endpoint. The last cell makes a call to the created service endpoint:prompt = “a cute cat is dancing on the grass.”input = “%20″.join(prompt.split(” “))resp = requests.get(f”http://example-cluster-kuberay-head-svc:8000/imagine?prompt={input}”)with open(“output.png”, ‘wb’) as f:   f.write(resp.content)

Executing the notebook will generate a file with a unique picture of a cute cat. Here is an example we got:

Congratulations! You have now deployed a large model for image generation on GKE.

Logging and Monitoring

As mentioned earlier, this solution enables logging and monitoring automatically. Let’s find those logs.

In your Cloud Console, open up Logging -> Log Explorer. In the query text box, enter the following:resource.type=”k8s_container”resource.labels.cluster_name=%CLUSTER_NAME%resource.labels.pod_name=%RAY_HEAD_POD_NAME%resource.labels.container_name=”fluentbit”

You should see the Ray logs from your cluster forwarded here.

To see your monitoring metrics, go to Metrics Explorer in the Cloud Console. Under the menu for “Target”, select “Prometheus Target” and then “Ray”. Select the metric that you want to see, for instance `prometheus/ray_component_cpu_percentage/gauge`:

The deployment also comes with a Grafana deployment. Follow this guide to open it up and view your Ray cluster’s metrics.

Conclusion

The combination of Ray and GKE offers a simple and powerful solution for building, deploying, and managing distributed applications. Ray’s simplicity makes it an attractive choice for data and model developers while GKE’s scalability and reliability is the defacto choice for enterprise platforms. The solution template presented in this blog post offers a convenient way to get started quickly with KubeRay, the recommended approach to deploy Ray on GKE.

If you have any questions for building Ray on Kubernetes and GKE, you can contact us directly at ray-on-gke@google.com or comment in GitHub. Learn more about building AI Platforms with GKE by visiting our User Guide.

When I reflect back on the past year and the pandemic, I’m struck by how the reliance on remote work and operations has changed the fundamentals of business forever. For the financial services industry, this rings particularly true. Many conversations I’m having right now with organizations revolve around embracing a transformation cloud, and thinking of cloud computing not just as an infrastructure decision, but also as the locus for transformation throughout the company. 

Today, as we welcome the industry to our Financial Services Summit, we’ll demonstrate just how Google Cloud accelerates a financial organization’s digital transformation through app and infrastructure modernization, data democratization, people connections, and trusted transactions. We hope you’ll join us.

How we’re helping financial services firms build their transformation clouds

At Google Cloud, we continue to focus on areas where we can bring the best of our capabilities to banking, capital markets, insurance, and payments customers around the world. Our work with financial services industry customers has given us a deep understanding of the real-world, specific use cases that matter most to them. This groundwork led us to engineer products and solutions that are open and flexible, not ones that force them to rip out existing investments in ERP or other early IaaS cloud implementations. 

It’s why we’ve engineered solutions such as Lending DocAI, Open Banking with Apigee, and Datashare for financial services to help transform the industry. These solutions were created with our customers’ security and compliance top-of-mind and are built at the intersection of user-optimized experience, technology, and sovereignty.

At their core, financial institutions want to drive growth, reduce costs, mitigate risk, stay compliant, and increase efficiency. As a result, when we partner with them on their transformation journeys, we consider three essential focus areas: 

• Enabling the human experience and connected interactions
• Building an open and intelligent data foundation for better insights
• Providing the most trusted and secure cloud in the industry

Enabling humans and connected interactions

A company’s transformation is about more than technology; people and culture ultimately drive change. HSBC, for example, recognized its business users would benefit from guided answers to common questions around risk policy compliance, and turned to Google Cloud to leverage AI and machine learning bots to assist employees, ease the burden on policy experts, and improve the user experience. Using Dialogflow, a core component of Contact Center AI, HSBC was able to build a conversational platform that quickly and accurately addresses user needs at scale. 

Another example is Equifax, which used Google Workspace to support collaboration not only internally between employees, but also externally with customers. Customers can use Google Cloud solutions for financial services to build these sorts of technology-enabled human interactions quickly and easily—supporting organizational change at scale.

Building an open and intelligent data foundation for smarter, faster insights

The real impact of Google Cloud solutions for financial services comes when the whole company has access to the right information at the right time, and can act more intelligently on that data. Our solutions help businesses safely leverage their data and get a complete 360-degree view of their customers’ information, which can often be scattered across multiple systems (CRM, lending, credit, etc.). This helps financial institutions improve the overall customer experience—and sometimes even develop new products quickly. 

Indeed, all financial institutions are looking for ways to grow revenue and reduce expenses, and data can be a critical ingredient to doing both effectively. As daily transactions rise, so does the volume and complexity of data. But to implement new customer experience innovations (and new revenue streams), financial institutions must first capture data effectively. This is why AXA Switzerland, for example, uses real-time analytics on Google Cloud to gain cross-industry insights about future trends and customer preferences.

Financial services organizations also need the confidence of building on a platform that provides choice, flexibility, and agility to move fast. It’s why we have an open cloud approach that allows Google Cloud services to run in different physical locations such as on-premises, other public clouds, and the edge. Customers can also harness the power of data and AI through our open APIs, machine-learning services, and analytics engines on any major cloud platform. This is why companies like Macquarie Bank are taking advantage of Google Cloud’s open, hybrid architecture to modernize and empower its developers.

Compliant and secure to address risk and regulatory needs

As a highly regulated industry, financial services is focused on security and compliance, risk and regulations, and fraud detection and prevention. Google Cloud offers unique capabilities to earn customers’ trust as part of our continuing work to be the most trusted cloud in the industry. Google Cloud provides a secure foundation that you can verify and independently control. Our cloud technology reduces risk and data loss, because it is built on comprehensive zero-trust architecture. Finally, we offer a shared-fate model built on best practices in risk management via automation, guidance, and insurance. This is why customers like BBVA have confidence anywhere their systems may operate. 

On the regulatory front, global legislators and regulators continue to focus on the stability of the industry that only a decade ago went through one of the biggest liquidity crises in history. With this oversight comes strong expectations of risk mitigation. Google Cloud offers a single, global set of controls, reviewed by financial institutions and regulators around the world, and verified in collaborative audits—making compliance simpler and less costly for our customers.

Finally, Google Cloud allows financial services firms to operate confidently with advanced security tools that help protect data, applications, and infrastructure, as well as their customers from fraudulent activity, spam, and abuse. We help protect your data against threats, using the same infrastructure and security services we use for our own operations, ensuring you never have to trade-off between ease of use and security. Google Cloud encrypts data at-rest and in-transit. And we now also offer the ability to encrypt data-in use, while it’s being processed for customer VM and container workloads.

Let us help you with your transformation cloud journey

We’ve seen leading financial services companies embrace Google Cloud to help them move beyond infrastructure toward the next phase of their cloud evolution. This is an era where no company is better positioned to lead than Google Cloud, and we’re excited to help you with your journey.

Learn more about Google Cloud for financial services.

Most enterprise companies enforce strict requirements for how APIs should be exposed to the public internet that require centralized handling of the authentication credentials, the collection of metrics metrics and other classic API management capabilities. Before the introduction of sidecars in Cloud Run, developers either had to implement these requirements within the service itself or add a full-lifecycle API management platform in front of their service as described in a previous blog post.

In this post we want to demonstrate a new way to fulfill the requirement for API management in Cloud Run. Specifically we want to look at how the recently announced multi-container feature in Cloud Run enables a sidecar pattern that can be used by developers of Cloud Run services to add pre-packaged API management capabilities. This includes self-service developer onboarding in a developer portal, credential validation and quota enforcements. As an additional operational benefit the described solution also adds centralized analytics and metrics for APIs that are exposed via Cloud Run.

Our solution for adding API management capabilities for Cloud Run is provided by the following three components:

• The Cloud Run service that hosts a traditional RESTful web application and is fronted by a vanilla Envoy proxy.
• An Apigee Envoy Adapter aka. Remote service that runs in GKE Autopilot and acts as a policy decision point PDP for Envoy’s external authorization filter and is responsible for accepting or rejecting calls to the Cloud Run service.
• An Apigee API Platform that is used to manage the API lifecycle of the Cloud Run service. Apigee also offers a turn-key developer portal where developers can obtain access credentials to access an API.

The user journey of an incoming request to our new Cloud Run Service with API management looks as follows:

1. An API Developer self-registers in the Apigee Developer Portal and obtains access credentials for the API
2. They call the Cloud Run endpoint and provide their credential for authentication
3. In Cloud Run service the Envoy proxy container intercepts the request and initiates a gRPC call to the Apigee Envoy adapter to authenticate the client.
4. The Apigee Envoy adapter verifies the request’s credentials and identifies the corresponding API product as defined in Apigee. The Envoy adapter also verifies the call quota associated with the client and sends the analytics data and access logs back to the control plane.
5. If the credentials are valid and the client hasn’t exhausted their call quota the Apigee Envoy adapter forwards the request to its co-located container in the Cloud Run service.

The step by step instructions for how to configure the architecture above can be found in this blog post. 

Starting from a pre-existing Apigee installation the Apigee Envoy adapter is used to connect back to the Apigee runtime and control plane for accessing the API product definitions to link the incoming requests to the issued credentials and quotas. The Apigee remote service component can be replicated for high availability and is shared between multiple Cloud Run services that require a gRPC target for the sidecar’s external authorization filter.

The Cloud Run service connects to the Apigee Remote Service via an Envoy proxy that acts as a sidecar to the main application. The configuration for the envoy proxy can either be embedded within a customized Envoy image or mounted via the secret manager integration of cloud run as shown in the diagram above. Externalizing the configuration simplifies the maintenance and upgrade of the sidecar container as it can just pull the latest patch version of Envoy regularly.

The newly added API management capability is transparent for the primary application container within the Cloud Run service and does not require any changes in the application source code. Once the Cloud Run service is re-deployed with the sidecar in place, consuming applications can start to use credentials that they obtained via the Apigee API Management platform or the API developer portal to consume the Cloud Run service. At an operations level the platform operators will start to see requests to the Cloud Run service popping up in Apigee’s analytics dashboards and be able to track consumption and exposure at an API product level.

Next Steps

If you are interested in trying the multi-container support for Cloud Run yourself, check out the release announcement with many more use case descriptions. For another example and a detailed walkthrough on how to use sidecars in Cloud Run to report custom metrics to Google Cloud managed service for Prometheus you can head over to this tutorial in the Cloud Run documentation. Lastly, if you’re interested in the broader picture of how the latest features in Cloud Run are moving serverless forward, then make sure you check out this video.