The rapidly evolving landscape of distributed computing demands efficient and scalable frameworks. Ray.io is an open-source framework to easily scale up Python applications across multiple nodes in a cluster. Ray provides a simple API for building distributed, parallelized applications, especially for deep learning applications.

Google Kubernetes Engine (GKE) is a managed container orchestration service that makes it easy to deploy and manage containerized applications. GKE provides a scalable and flexible platform that abstracts away the underlying infrastructure.

KubeRay enables Ray to be deployed on Kubernetes. You get the wonderful Pythonic unified experience delivered by Ray, and the enterprise reliability and scale of GKE managed Kubernetes. Together, they offer scalability, fault tolerance, and ease of use for building, deploying, and managing distributed applications.

In this blog post, we share a solution template to get you started easily with Ray on GKE. We discuss the components of the solution and showcase an inference example using Ray Serve for Stable Diffusion.

Overview of the Solution

In this solution template we use KubeRay, an OSS solution for managing Ray clusters on Kubernetes, as the operator for provisioning our workloads. Follow the step-by-step instructions in the README file to get started. The solution contains two groups of resources: platform-level and user-level.

Platform-level resources are expected to be deployed once for each development environment by the system administrator. These include the common infrastructure and GCP service integrations that are shared by all users.

• GKE cluster and node pool. Configurations can be changed in the main.tf file. This module deploys a GKE cluster with a GPU node pool, including required Nvidia drivers for GPUs. You can replace these with other machine types.
• Kubernetes system namespace and service accounts, along with the necessary IAM policy bindings. This allows the platform administrator to provide fine-grained user access control and quota policies for Ray cluster resources.
• KubeRay operator. The operator is responsible for watching for changes in KubeRay resources and reconciling the state of the KubeRay clusters.  
• Logging. The `logging_config` section enables logs from system components and workloads to write logs to Cloud logging.
• Monitoring. The `monitoring_config` section enables Managed Prometheus integration. This allows the deployment to automatically scrape system-level metrics and writes them to the managed metrics service.
• Workload identity. This enables your workloads to authenticate with other GCP services using Google IAM service accounts.

User-level resources are expected to be deployed once by each user in the development environment.

• KubeRay cluster. This is the actual Ray cluster that we will be used for your workloads. It is configured to use a Workload Identity pool and a IAM-binded service account that provides fine-grained access to GCP services. You can customize the Ray cluster settings by editing the kuberay-values.yaml file.

• Logging. The solution adds a side car container deployed alongside each KubeRay worker node. This uses fluentbit to forward Ray logs from the head node to Cloud logging. You can edit the fluentbit-config file to change how the logging container filters and flushes logs.
• Monitoring. This module provides a PodMonitoring resource that scrapes metrics from the user’s Ray cluster and uploads data points to Google Managed Prometheus. An optional installation for Grafana dashboard is included and can be accessed through a web browser.
• JupyterHub server. This module installs a JupyterHub notebook server in the user namespace, enabling users to interact directly with their Ray clusters.

Run a Workload on Your Ray Cluster

Let’s try running the provided example with Ray Serve to deploy Stable Diffusion. This example was originally taken from the Ray Serve documentations here. To open the example in Jupyter notebook, go to the external IP for proxy-public in your browser (instructions to get the IP). And then click on File -> Open from URL, and input the raw URL of the notebook to open it.

Since the notebook runs in the same Kubernetes cluster as the Ray cluster, it is able to talk directly to the latter using its cluster-internal service endpoint – thus there is no need to expose the Ray cluster to public internet traffic. For production workloads, you should secure your endpoints with GCP account credentials. Google Cloud Identity Aware Proxy (IAP) can be used to enable fine-grained access control to user resources, such as our Ray cluster, to protect your GCP resources from unnecessary exposure. A full tutorial on how to enable IAP on your GKE cluster can be found here.

The notebook contains code for deploying a pre-trained model to a live endpoint. The last cell makes a call to the created service endpoint:prompt = “a cute cat is dancing on the grass.”input = “%20″.join(prompt.split(” “))resp = requests.get(f”http://example-cluster-kuberay-head-svc:8000/imagine?prompt={input}”)with open(“output.png”, ‘wb’) as f:   f.write(resp.content)

Executing the notebook will generate a file with a unique picture of a cute cat. Here is an example we got:

Congratulations! You have now deployed a large model for image generation on GKE.

Logging and Monitoring

As mentioned earlier, this solution enables logging and monitoring automatically. Let’s find those logs.

In your Cloud Console, open up Logging -> Log Explorer. In the query text box, enter the following:resource.type=”k8s_container”resource.labels.cluster_name=%CLUSTER_NAME%resource.labels.pod_name=%RAY_HEAD_POD_NAME%resource.labels.container_name=”fluentbit”

You should see the Ray logs from your cluster forwarded here.

To see your monitoring metrics, go to Metrics Explorer in the Cloud Console. Under the menu for “Target”, select “Prometheus Target” and then “Ray”. Select the metric that you want to see, for instance `prometheus/ray_component_cpu_percentage/gauge`:

The deployment also comes with a Grafana deployment. Follow this guide to open it up and view your Ray cluster’s metrics.

Conclusion

The combination of Ray and GKE offers a simple and powerful solution for building, deploying, and managing distributed applications. Ray’s simplicity makes it an attractive choice for data and model developers while GKE’s scalability and reliability is the defacto choice for enterprise platforms. The solution template presented in this blog post offers a convenient way to get started quickly with KubeRay, the recommended approach to deploy Ray on GKE.

If you have any questions for building Ray on Kubernetes and GKE, you can contact us directly at ray-on-gke@google.com or comment in GitHub. Learn more about building AI Platforms with GKE by visiting our User Guide.

Google Cloud and the DORA research team are excited to announce our eight sponsors for the 2021 State of DevOps report. We recently launched the 2021 State of DevOps survey, a 25-min survey for the DevOps community to share how they are using DevOps to improve software delivery performance. So if you haven’t taken the survey yet, this is your chance! 

For those unfamiliar with the State Of DevOps report, it is the largest and longest running research of its kind. It provides an independent view into how teams and companies can drive powerful business outcomes, no matter what stage of the DevOps journey

With the State of DevOps reports we aim to capture how teams and companies are driving excellence in technology delivery through the implementation of DevOps practices, no matter what stage of the DevOps journey your team is in. If you’re wondering how your team measures up in your industry take our DevOps Quick Check and discover which capabilities you should focus on to improve your performance.

To capture a diverse array of information on how those in the DevOps community are performing, our eight sponsors of the 2021 State of DevOps survey are focussed on helping organizations of all sizes and industries to develop and deliver software faster across the whole DevOps lifecycle. Captured below, you’ll find more information on our sponsors for this year and why they chose to sponsor the State Of the DevOps 2021 Report.

Armory

Armory enables enterprise companies to ship better software, faster through trusted, reliable, safe, and secure deployments — at its core, Armory is powered by Spinnaker OSS.

“Armory exists to unlock innovation through software. To achieve this, we help enterprises reliably deploy software at scale, naturally aligning with DevOps practices to improve software delivery performance. Given this, we applaud efforts like the State of DevOps Report that provides an independent view into the practices and capabilities that organizations can employ to drive better performance.” 

Carl Timm, Senior Director of Product Marketing at Armory

CircleCI

The world’s best software teams deliver quality code, confidently, with CircleCI. The world’s best software teams use CircleCI to go from next-up to feature shipped, at the speed ambitious businesses require. 

“Though the DevOps space is only over a decade old, it moves incredibly quickly. Google’s State of DevOps report is both a reflection and projection of the industry, capturing how DevOps culture drives business results and where DevOps practitioners can look to improve. At CircleCI, we rely heavily on this survey data to glean valuable insights into our market and our customers overall. We also build upon these insights to guide our own research into how engineering teams move code through pipelines in our annual State of Software Delivery report. Taken together, this research highlights teams’ reported and actual behavior and paints a vibrant picture of how technology-driven organizations drive for success.”

Emma Webb, VP, Corporate Communications, CircleCI 

Continuous Delivery Foundation

CD Foundation is an open-source community improving the world’s ability to deliver software with security and speed.

“Good decision-making is based on good data. Open source is a critical piece of the technology roadmap for DevOps, and the 2021 State of DevOps Report will provide actionable information for high performing teams in organizations of all types and sizes. The report will show where open source and DevOps intersect, and in a space that is changing so quickly, new relevant data is critical,” said Tracy Miranda, Continuous Delivery Foundation Executive Director. “CD Foundation members are deeply involved with producing this annual report, with over 10 years of historic research. Thank you to Google and our other members who have worked so hard on it. This report is a tangible result of working together.”

Tracy Miranda, Continuous Delivery Foundation Executive Director

Deloitte

Deloitte provides audit and assurance, tax, consulting, and risk and financial advisory services to a broad cross-section of the largest corporations and governmental agencies. At Deloitte, they are continuously evolving how they work and how they look at marketplace challenges so they can continually deliver measurable, sustainable results for their clients and communities.

“Software delivery is evolving rapidly and we know our customers need unique and compelling insights to make the right decisions. State of DevOps is a widely used report and considered as an Industry standard to understand the drivers for excellence in Software Development and Delivery. Deloitte is excited to help the team at DORA and Google Cloud in publishing this report through our sponsoring since we believe the insights from this report will help make software delivery better.” 

Manoj Mishra, Consulting Managing Director, Deloitte Consulting LLP

GitLab

GitLab is the open DevOps platform built from the ground up as a single application for all stages of the DevOps lifecycle enabling Product, Development, QA, Security, and Operations teams to work concurrently on the same project.

“We’re happy to sponsor the DORA Report and the vital work behind it. The more we understand the DevOps journey, the better we and our GitLab community can contribute to its future. We’re particularly interested to see this year’s results and the impact COVID-19 and remote work have had on DevOps. We appreciate the chance to be part of this exciting, informative process.”

Brendon O’Leary, Senior Developer Evangelist

Liquibase

Liquibase helps millions of developers easily manage database schema changes by enabling teams to track, version, and deploy database changes by delivering on the promise of CI/CD for the database.

“With their rigorous methodology, the DORA research team delivers actionable information with the simple goal of helping organizations of any size accelerate the development and delivery of software. Liquibase is honored to sponsor the 2021 State of DevOps Report and its role in helping so many organizations build value for their customers and shareholders”

Matt Geise, VP of Marketing at Liquibase

PagerDuty

PagerDuty is a digital operations management platform that empowers the right action, when seconds matter.

“As a leader in digital operations management, PagerDuty is proud to sponsor this year’s report as it aligns with our dedication to helping DevOps professionals make better decisions. This report will inform tech and business leaders about the trends/challenges developers are facing and the opportunities there are to accelerate their own DevOps transformation.” 

Carolyn Guss, VP of Corporate Marketing

SysDig

Sysdig is driving the secure DevOps movement, empowering organizations to confidently secure containers, Kubernetes and cloud.

“There is a learning curve with all new technology, cloud is no exception. Learning from mistakes and sharing best practices is how we will ultimately ship secure applications, faster. For seven years, DORA and Google have partnered to understand the State of DevOps to help vendors and cloud practitioners to learn from each other and implement standards for best practices,”

Loris Degioanni, CTO and founder of Sysdig

Google Cloud, the DORA team, and our sponsors are very excited about this year’s report. We look forward to hearing from you, your colleagues, and networks about how DevOps is integrated into your workflow and ways we can help to further improve your performance. Please share your experience with software delivery by completing our survey that will be used to foster the next generation of DevOps best practices. To provide ample time for the DevOps community to contribute to this industry wide report we have extended the deadline for the survey until July 2nd.

Thank you to everyone who has participated so far, and the Google Cloud DORA team looks forward to hearing from even more of you soon!

This year at Google I/O, we announced the general availability of Cloud-based maps styling for the Maps JavaScript API. In an effort to provide you with more options and more control to help create the best experience for your users, today we’re releasing new features to Cloud-based maps styling. You may already be familiar with these features from the consumer Google Maps web and mobile apps—Landmarks and Building Footprints. We’re also releasing updates to our industry optimized map styles to provide even more map details while providing the flexibility to craft the best experience for your users. Let’s take a look.

Help users quickly scan and orient themselves with Landmarks

You may have noticed some enhancements for prominent places in the consumer Google Maps web and mobile apps, these landmarks help show your users points of interest that help them orient and navigate cities they are exploring or visiting.

You now have the ability to bring this same experience to your users by creating maps using Cloud-based maps styling. This feature is available in 100 cities globally including New York, Dubai, Paris, Mumbai, and Singapore. To enable landmarks for your map, log into the Cloud console and in our style editor navigate to the Points of interest feature type and select ‘Illustrated’ under Marker Style.

Simplify maps features by switching to Building Footprints

Sometimes less is more. In dense, highly vertical cities, showing 3D building heights can add cognitive load for users. Now, in addition to 3D buildings, we offer building footprints as an option in the style editor. Building footprints can provide a strikingly different basemap balance and composition to better support use cases that may not benefit from the added complexity that 3D buildings can present.

Fill and stroke geometries can also be styled independently to support various color themes. To enable Building Footprints, log into the Cloud console and in our style editor navigate to Buildings and choose ‘Footprints’ under building style.

Industry Optimized Map Styles now include Landmarks and Building Footprints, plus Detailed Street Maps

In January of this year we launched Industry Optimized Map Styles for the travel, real estate, retail, and logistics industries, providing customers with pre-styled map configurations, available via Cloud-based maps styling. Landmarks are now included in all of our Industry Optimized Map Styles and we have turned on Building Footprints in the travel style map.  If you are already an Industry Optimized Map Styles user, these new features will be applied to your map with no action needed from you. If you would like to disable these changes, you can use the style editor to turn off these features.
For Industry Optimized Map Styles only, we are also excited to enable Detailed Street Maps. You may have seen these features in our consumer products at Google I/O, released back in August of 2020 for the consumer Google Maps web and mobile apps. Detailed Street Maps are available in San Francisco, New York, London, and Tokyo, and we are targeting expansion to 50 new cities by the end of 2021.

Detailed Street Maps are on by default for all Industry Optimized Map Styles and we created a new settings menu to change the visibility, as needed. We are working on bringing the full styling capability for Detailed Street Maps features to all Cloud-based maps styles in the future.

Landmarks and Building Footprints as well as the updates to Industry Optimized Map Styles are only available via Cloud-based map styling in the Google Cloud Console and are included in Google Maps Platform pricing. Learn more about how to use Landmarks and Building Footprints and Industry Optimized Map Styles. To get started with Cloud-based map styling, check out our documentation for JavaScript.

For more information on Google Maps Platform, visit our website.

On March 15th, the Go team announced Go 1.18 GA, the latest release of the Go programming language. The culmination of over a decade of design delivers the features our developers demanded: generics, fuzzing, and module workspaces. With this release, Go becomes the first major language to integrate fuzz testing into its core toolchain without using third-party support, further establishing Go as a preferred language for developing secure applications.

Go was created at Google in 2007, designed to help developers build fast, reliable, and secure software. Unlike traditional languages, Go was built for the modern multi-core computing world. Go has emerged as a modern language for developing cloud applications, services, and infrastructure. Today Go powers several of Google’s largest products, and is used by many customers to scale their businesses. Organizations big and small love Go and the community of Go developers, known as “gophers” has grown into a global network with over 2 million users worldwide.

Using the power of Go in the Cloud

When looking at the public repos, over 75% of CNCF projects including Kubernetes are written in Go and 10% of developers are writing in Go worldwide (as of May 2021). Google delivers high performance infrastructure to run key, cloud native, Open Source projects. Our modern cloud infrastructure is based on Kubernetes at its core and our strong support for Istio and Knative have formed the base of some of our leading services like Google Kubernetes Engine (GKE), our managed application platform with Anthos, Cloud Functions, and Cloud Run. Google uses Go extensively for a wide range of applications from our indexing platform that powers Google Search, to the server side optimizations that power Chrome’s 1B+ users, to the infrastructure on which Google cloud is built.

Release Highlights

With this new release of Go 1.18, Generics are the biggest change to Go since the language was created. Go developers told us that they feel that Go lacks critical features, with generics being the main missing piece. With Go 1.18, new and existing Go developers can take advantage of the productivity, performance, and maintenance benefits that generics can bring. We’ve already begun to see the new kinds of libraries and projects gophers are building with generics in its short beta period, and expect this creativity to grow as time goes on.

This Go release also brings native support for fuzzing. Fuzzing is a type of vulnerability testing that throws arbitrary data at a piece of software to expose unknown errors and is emerging as a common testing scheme in enterprise development. Go is now the first major language to provide fuzzing support with no third-party integrations necessary, allowing developers to start building secure software with minimal additional cost. Go’s innovative approach to fuzzing can provide not only security for the current code but also ongoing protection as code and dependencies evolve. With attacks on software becoming more common and complex, vulnerability detection can be a critical part of the enterprise development lifecycle, and Go’s fuzzing capabilities catch vulnerabilities earlier in the lifecycle.

Build securely using Go

At Google we are helping to make Open Source software secure. Open source software is a connective tissue for much of the online world. At Google, we’ve been working to raise awareness of the state of open source security and are committed to helping secure the software supply chain for organizations. Go has been designed to create secure applications, helping to minimize risk as much as possible. Go applications compile down to a single binary without local dependencies. It’s not uncommon to see an application built using only the standard library, or only a couple well-vetted Go dependencies. Go’s dependency management uses tamper-evident transparency log, with built in tooling that helps ensure your dependencies are what you can expect. Go has native encryption, which is used across much of the internet, including key components of Google. Go even supports distroless containers, where there are zero local dependencies to worry about. Google Cloud products like Cloud Build, for CI/CDand Artifact Registry, for container management, and have direct access to Go’s vulnerability database and can provide you instant warnings about security threats.

“At Google we are committed to helping to secure the online infrastructure and applications upon which the world depends. A critical aspect of this mission is being able to understand and verify the security of open source dependency chains. The 1.18 release of Go is an important step towards helping to ensure that developers are able to build secure applications, understand risk when vulnerabilities are discovered, and reduce the impact of cybersecurity attacks” said Eric Brewer, VP Infrastructure, Google Fellow

This launch is a significant milestone for Go that helps developers from around the world build more performant and secure applications that run on any infrastructure. For more information on this release and how to get started with Go, please visit.