On March 15th, the Go team announced Go 1.18 GA, the latest release of the Go programming language. The culmination of over a decade of design delivers the features our developers demanded: generics, fuzzing, and module workspaces. With this release, Go becomes the first major language to integrate fuzz testing into its core toolchain without using third-party support, further establishing Go as a preferred language for developing secure applications.

Go was created at Google in 2007, designed to help developers build fast, reliable, and secure software. Unlike traditional languages, Go was built for the modern multi-core computing world. Go has emerged as a modern language for developing cloud applications, services, and infrastructure. Today Go powers several of Google’s largest products, and is used by many customers to scale their businesses. Organizations big and small love Go and the community of Go developers, known as “gophers” has grown into a global network with over 2 million users worldwide.

Using the power of Go in the Cloud

When looking at the public repos, over 75% of CNCF projects including Kubernetes are written in Go and 10% of developers are writing in Go worldwide (as of May 2021). Google delivers high performance infrastructure to run key, cloud native, Open Source projects. Our modern cloud infrastructure is based on Kubernetes at its core and our strong support for Istio and Knative have formed the base of some of our leading services like Google Kubernetes Engine (GKE), our managed application platform with Anthos, Cloud Functions, and Cloud Run. Google uses Go extensively for a wide range of applications from our indexing platform that powers Google Search, to the server side optimizations that power Chrome’s 1B+ users, to the infrastructure on which Google cloud is built.

Release Highlights

With this new release of Go 1.18, Generics are the biggest change to Go since the language was created. Go developers told us that they feel that Go lacks critical features, with generics being the main missing piece. With Go 1.18, new and existing Go developers can take advantage of the productivity, performance, and maintenance benefits that generics can bring. We’ve already begun to see the new kinds of libraries and projects gophers are building with generics in its short beta period, and expect this creativity to grow as time goes on.

This Go release also brings native support for fuzzing. Fuzzing is a type of vulnerability testing that throws arbitrary data at a piece of software to expose unknown errors and is emerging as a common testing scheme in enterprise development. Go is now the first major language to provide fuzzing support with no third-party integrations necessary, allowing developers to start building secure software with minimal additional cost. Go’s innovative approach to fuzzing can provide not only security for the current code but also ongoing protection as code and dependencies evolve. With attacks on software becoming more common and complex, vulnerability detection can be a critical part of the enterprise development lifecycle, and Go’s fuzzing capabilities catch vulnerabilities earlier in the lifecycle.

Build securely using Go

At Google we are helping to make Open Source software secure. Open source software is a connective tissue for much of the online world. At Google, we’ve been working to raise awareness of the state of open source security and are committed to helping secure the software supply chain for organizations. Go has been designed to create secure applications, helping to minimize risk as much as possible. Go applications compile down to a single binary without local dependencies. It’s not uncommon to see an application built using only the standard library, or only a couple well-vetted Go dependencies. Go’s dependency management uses tamper-evident transparency log, with built in tooling that helps ensure your dependencies are what you can expect. Go has native encryption, which is used across much of the internet, including key components of Google. Go even supports distroless containers, where there are zero local dependencies to worry about. Google Cloud products like Cloud Build, for CI/CDand Artifact Registry, for container management, and have direct access to Go’s vulnerability database and can provide you instant warnings about security threats.

“At Google we are committed to helping to secure the online infrastructure and applications upon which the world depends. A critical aspect of this mission is being able to understand and verify the security of open source dependency chains. The 1.18 release of Go is an important step towards helping to ensure that developers are able to build secure applications, understand risk when vulnerabilities are discovered, and reduce the impact of cybersecurity attacks” said Eric Brewer, VP Infrastructure, Google Fellow

This launch is a significant milestone for Go that helps developers from around the world build more performant and secure applications that run on any infrastructure. For more information on this release and how to get started with Go, please visit.

The rapidly evolving landscape of distributed computing demands efficient and scalable frameworks. Ray.io is an open-source framework to easily scale up Python applications across multiple nodes in a cluster. Ray provides a simple API for building distributed, parallelized applications, especially for deep learning applications.

Google Kubernetes Engine (GKE) is a managed container orchestration service that makes it easy to deploy and manage containerized applications. GKE provides a scalable and flexible platform that abstracts away the underlying infrastructure.

KubeRay enables Ray to be deployed on Kubernetes. You get the wonderful Pythonic unified experience delivered by Ray, and the enterprise reliability and scale of GKE managed Kubernetes. Together, they offer scalability, fault tolerance, and ease of use for building, deploying, and managing distributed applications.

In this blog post, we share a solution template to get you started easily with Ray on GKE. We discuss the components of the solution and showcase an inference example using Ray Serve for Stable Diffusion.

Overview of the Solution

In this solution template we use KubeRay, an OSS solution for managing Ray clusters on Kubernetes, as the operator for provisioning our workloads. Follow the step-by-step instructions in the README file to get started. The solution contains two groups of resources: platform-level and user-level.

Platform-level resources are expected to be deployed once for each development environment by the system administrator. These include the common infrastructure and GCP service integrations that are shared by all users.

• GKE cluster and node pool. Configurations can be changed in the main.tf file. This module deploys a GKE cluster with a GPU node pool, including required Nvidia drivers for GPUs. You can replace these with other machine types.
• Kubernetes system namespace and service accounts, along with the necessary IAM policy bindings. This allows the platform administrator to provide fine-grained user access control and quota policies for Ray cluster resources.
• KubeRay operator. The operator is responsible for watching for changes in KubeRay resources and reconciling the state of the KubeRay clusters.  
• Logging. The `logging_config` section enables logs from system components and workloads to write logs to Cloud logging.
• Monitoring. The `monitoring_config` section enables Managed Prometheus integration. This allows the deployment to automatically scrape system-level metrics and writes them to the managed metrics service.
• Workload identity. This enables your workloads to authenticate with other GCP services using Google IAM service accounts.

User-level resources are expected to be deployed once by each user in the development environment.

• KubeRay cluster. This is the actual Ray cluster that we will be used for your workloads. It is configured to use a Workload Identity pool and a IAM-binded service account that provides fine-grained access to GCP services. You can customize the Ray cluster settings by editing the kuberay-values.yaml file.

• Logging. The solution adds a side car container deployed alongside each KubeRay worker node. This uses fluentbit to forward Ray logs from the head node to Cloud logging. You can edit the fluentbit-config file to change how the logging container filters and flushes logs.
• Monitoring. This module provides a PodMonitoring resource that scrapes metrics from the user’s Ray cluster and uploads data points to Google Managed Prometheus. An optional installation for Grafana dashboard is included and can be accessed through a web browser.
• JupyterHub server. This module installs a JupyterHub notebook server in the user namespace, enabling users to interact directly with their Ray clusters.

Run a Workload on Your Ray Cluster

Let’s try running the provided example with Ray Serve to deploy Stable Diffusion. This example was originally taken from the Ray Serve documentations here. To open the example in Jupyter notebook, go to the external IP for proxy-public in your browser (instructions to get the IP). And then click on File -> Open from URL, and input the raw URL of the notebook to open it.

Since the notebook runs in the same Kubernetes cluster as the Ray cluster, it is able to talk directly to the latter using its cluster-internal service endpoint – thus there is no need to expose the Ray cluster to public internet traffic. For production workloads, you should secure your endpoints with GCP account credentials. Google Cloud Identity Aware Proxy (IAP) can be used to enable fine-grained access control to user resources, such as our Ray cluster, to protect your GCP resources from unnecessary exposure. A full tutorial on how to enable IAP on your GKE cluster can be found here.

The notebook contains code for deploying a pre-trained model to a live endpoint. The last cell makes a call to the created service endpoint:prompt = “a cute cat is dancing on the grass.”input = “%20″.join(prompt.split(” “))resp = requests.get(f”http://example-cluster-kuberay-head-svc:8000/imagine?prompt={input}”)with open(“output.png”, ‘wb’) as f:   f.write(resp.content)

Executing the notebook will generate a file with a unique picture of a cute cat. Here is an example we got:

Congratulations! You have now deployed a large model for image generation on GKE.

Logging and Monitoring

As mentioned earlier, this solution enables logging and monitoring automatically. Let’s find those logs.

In your Cloud Console, open up Logging -> Log Explorer. In the query text box, enter the following:resource.type=”k8s_container”resource.labels.cluster_name=%CLUSTER_NAME%resource.labels.pod_name=%RAY_HEAD_POD_NAME%resource.labels.container_name=”fluentbit”

You should see the Ray logs from your cluster forwarded here.

To see your monitoring metrics, go to Metrics Explorer in the Cloud Console. Under the menu for “Target”, select “Prometheus Target” and then “Ray”. Select the metric that you want to see, for instance `prometheus/ray_component_cpu_percentage/gauge`:

The deployment also comes with a Grafana deployment. Follow this guide to open it up and view your Ray cluster’s metrics.

Conclusion

The combination of Ray and GKE offers a simple and powerful solution for building, deploying, and managing distributed applications. Ray’s simplicity makes it an attractive choice for data and model developers while GKE’s scalability and reliability is the defacto choice for enterprise platforms. The solution template presented in this blog post offers a convenient way to get started quickly with KubeRay, the recommended approach to deploy Ray on GKE.

If you have any questions for building Ray on Kubernetes and GKE, you can contact us directly at ray-on-gke@google.com or comment in GitHub. Learn more about building AI Platforms with GKE by visiting our User Guide.

ANZ Bank is one of Australia’s top four banks and the largest bank in New Zealand by market capitalization. Headquartered in Melbourne, Victoria, we operate in more than 32 markets across Australia, New Zealand, Asia, Pacific, Europe, America and the Middle East. Our Payment team deals with all payment transaction types by providing highly secure, mission critical payment services to retail, institutional and international customers. The core payment platform is based on microservices architecture to support discreet payment processing requirements such as continuous volume growth, industry service level agreement (SLA) and complex payment orchestration and more. The API platform leverages APIs to perform specific business functions under the payment orchestration layer, which need to be resilient, scalable and heavy on security controls.

Simplifying our technology landscape

At ANZ, our mission is to improve the financial wellbeing and sustainability of our customers through reliable payment services. Over the past 12 months, we looked to simplify our technology landscape, to free up time so we could focus on implementing more customer-focused banking services. We needed an API management solution that would align to our API-first strategy, while also maintaining our high performance, security, and regulatory standards.

Specifically, the solution needed to:

• Simplify channel interactions with our platform;
• Improve developer experience and enable self-service;
• Ensure resilient fault tolerance;
• Improve our payment API security posture;
• Enable API-first digital enablement for both our internal and external payments customers;
• Have a scalable and transparent pricing model that ensured sustainable API programme growth.
  

Core focus on security and compliance

To meet our security and regulatory compliance standards an API solution would need to offer functionality including:

• Providing coarse-grained and fine-grained authorization and domain specific entitlements for API requests;
• Bundling APIs based on the nature of the channel;
• Integrating easily with established identity providers;
• Establishing patterns for connecting trusted upstream and downstream systems;
• Providing support for the industry standard OAuth 2.0 authorization protocol as per enterprise security standards.

We reviewed Apigee against other API management solutions and gateways, and determined it was the best fit for our needs. Not only did Apigee deliver on all our mission-critical requirements, but it also provided strong ease of use, feature-completeness and support for multiple coding languages.

Enabling smooth developer onboarding, processes and troubleshooting

Using Apigee has greatly improved developer onboarding and reduced the tedious steps involved in knowledge transfer and upskilling. The platform is easy for developers to use, and is backed by great documentation and video resources. These resources provide clear guidance about how to execute certain processes and troubleshoot as needed.

Overall, Apigee has delivered the features we need to manage the publication and consumption of APIs efficiently. We are now running eighteen APIs in production, including two APIs that enable payments through connection to our transaction database, and others that enable supporting services in the payment services platform team.

Our engineers and testers have found it easy to use the interface and Apigee API Management to deploy and test proxies. In addition, out of the box policies allow for fine-grained access control to handle and build proxies of varying degrees of complexity. These policies allow us to control security, manage traffic, mediate transformations and implement custom functionality via scripts.

We can now implement our fine-grained entitlement-based access control requirements, onboard new customers to the cloud and streamline payment channel onboarding. With Apigee, we are also integrating with other API management tools at the bank, in part to enable self-service for API providers and consumers by building a developer portal.

Thanks to Apigee, we are well positioned to adopt a decentralized API team model that will see different teams within payments and beyond create APIs, and ramp up to full production of our API-centric model in the near future. This is a key component of the bank’s broader cloud and API-first strategy that is designed to help ANZ become a more agile and adaptive technology organization. With the right components in place, can we help ANZ create opportunities and propositions that colleagues and customers love, ultimately helping ANZ realize its vision across Australia, New Zealand and international markets.