Google Cloud Functions provides a simple and intuitive developer experience to execute code from Google Cloud, Firebase, Google Assistant, or any web, mobile, or backend application. Oftentimes that code needs secrets—like API keys, passwords, or certificates—to authenticate to or invoke upstream APIs and services.

While Google Secret Manager is a fully-managed, secure, and convenient storage system for such secrets, developers have historically leveraged environment variables or the filesystem for managing secrets in Cloud Functions. This was largely because integrating with Secret Manager required developers to write custom code… until now. We listened to customer feedback and today we are announcing Cloud Functions has a native integration with Secret Manager!

This native integration has many key benefits including:

• Zero required code changes. Cloud functions that already consume secrets via environment variables or files bundled with the source upload simply require an additional flag during deployment. The Cloud Functions service resolves and injects the secrets at runtime and the plaintext values are only visible inside the process.
• Easy environment separation. It’s easy to use the same codebase across multiple environments, e.g., dev, staging, and prod, because the secrets are decoupled from the code and are resolved at runtime.
• Supports the 12-factor app pattern. Because secrets can be injected into environment variables at runtime, the native integration supports the 12-factor pattern while providing stronger security guarantees.
• Centralized secret storage, access, and auditing. Leveraging Secret Manager as the centralized secrets management solution enables easy management of access controls, auditing, and access logs.

Cloud Functions’ native integration with Secret Manager is available in preview to all Google Cloud customers today. Let’s take a deeper dive into this new integration.

Example

Suppose the following cloud function invoked via HTTP uses a secret token to invoke an upstream API:

  const https = require('https');
const token = process.env.TOKEN;

exports.secretDemo = (req, res) => {
  https.get(`https://upstream-api.example.com?token=${token}`, (innerRes) => {
    innerRes.on('end', () => { res.send('OK') });
  }).on('error', (err) => {
    res.send(`Error: ${err}`);
  });
}

Without Cloud Functions’ native integration with Secret Manager, this function is deployed via:

  $ gcloud functions deploy "secretDemo" \
    --runtime "nodejs14" \
    --trigger-http \
    --allow-unauthenticated \
    --set-env-vars "TOKEN=abcd1234"

This approach has a number of drawbacks, the biggest of which being that anyone with viewer permissions on the Google Cloud project can see the environment variables set on a cloud function.

To improve the security of this code, migrate the secret to Secret Manager in the same project:

  $ gcloud secrets create "token" \
    --replication-policy "automatic" \
    --data-file - <<< "abcd1234"

Finally, without changing any code in the function re-deploy with slightly different flags:

  $ gcloud beta functions deploy "secretDemo" \
    --runtime "nodejs14" \
    --trigger-http \
    --allow-unauthenticated \
    --set-secrets "TOKEN=my_token:latest"

It’s truly that easy to migrate from hard-coded secrets to using secure secret storage with Secret Manager! To add even more layers of security, consider running each cloud function with a dedicated service account and practice the principle of least privilege. Learn more in the Secret Manager Best Practices guide.

Making security easy

Security and proper secrets management are core pillars of modern software development, and we’re excited to provide customers a way to improve the security of their cloud functions. To learn more about the new native integration, check out the Cloud Functions documentation. You can also learn more about Cloud Functions or learn more about Secret Manager.

Cloud Deploy is a fully managed continuous delivery platform that automates the delivery of your application. Recently, we’ve heard from users that they want Cloud Deploy to also perform self-defined pre- and post-deployment operations to reduce the amount of additional toil required for each rollout. Examples of such operations include database schema updates, application infrastructure deployment, and network configuration adjustment. 

Today, we are pleased to announce the public preview availability of Cloud Deploy deploy hooks. With this launch, Cloud Deploy now offers an easy way to define and execute pre- and post-deployment operations as part of a rollout.

Deploy hooks

The ability to run user defined actions immediately before and after deployment is a powerful feature that allows DevOps organizations to build out complete CI/CD pipelines to meet their specific deployment needs.

With the introduction of deploy hooks in Cloud Deploy, you can now take advantage of this capability by configuring your delivery pipeline to include pre-deploy and post-deploy actions. These actions take advantage of the custom actions defined through Skaffold. When enabled, two new jobs — predeploy and postdeploy — will be performed when deploying a rollout into a target. 

Deploy hooks are supported for all target types (GKE, Cloud Run, Anthos, and Multi-Target) and can be used with all deployment strategies, such as canary deployments.

As enabled, deploy hooks can be used in a variety of ways, such as:

• Performing a database migration prior to deployment
• Performing infrastructure deployment prior to application deployment
• Interacting with third-party platforms post deployment, such as sending out an email or updating an open task tracker

Like other rollout jobs, Cloud Deploy provides the same observability and control for predeploy and postdeploy jobs. This includes the ability to view logs and perform control over individual jobs, such as retry, terminate, or ignore — all within the same rollout details interface.

Interested in exploring deploy hooks for yourself? Get started now with our quickstart and documentation!

The future

Comprehensive, easy-to-use, and cost-effective DevOps tools are key to building an efficient software delivery capability, and it’s our hope that Cloud Deploy will help you implement complete CI/CD pipelines. Stay tuned as we introduce exciting new capabilities and features to Cloud Deploy in the months to come. 

In the meantime, check out the product page, documentation, quickstarts, and tutorials. Finally, If you have feedback on Cloud Deploy, you can join the conversation. We look forward to hearing from you!

The rapidly evolving landscape of distributed computing demands efficient and scalable frameworks. Ray.io is an open-source framework to easily scale up Python applications across multiple nodes in a cluster. Ray provides a simple API for building distributed, parallelized applications, especially for deep learning applications.

Google Kubernetes Engine (GKE) is a managed container orchestration service that makes it easy to deploy and manage containerized applications. GKE provides a scalable and flexible platform that abstracts away the underlying infrastructure.

KubeRay enables Ray to be deployed on Kubernetes. You get the wonderful Pythonic unified experience delivered by Ray, and the enterprise reliability and scale of GKE managed Kubernetes. Together, they offer scalability, fault tolerance, and ease of use for building, deploying, and managing distributed applications.

In this blog post, we share a solution template to get you started easily with Ray on GKE. We discuss the components of the solution and showcase an inference example using Ray Serve for Stable Diffusion.

Overview of the Solution

In this solution template we use KubeRay, an OSS solution for managing Ray clusters on Kubernetes, as the operator for provisioning our workloads. Follow the step-by-step instructions in the README file to get started. The solution contains two groups of resources: platform-level and user-level.

Platform-level resources are expected to be deployed once for each development environment by the system administrator. These include the common infrastructure and GCP service integrations that are shared by all users.

• GKE cluster and node pool. Configurations can be changed in the main.tf file. This module deploys a GKE cluster with a GPU node pool, including required Nvidia drivers for GPUs. You can replace these with other machine types.
• Kubernetes system namespace and service accounts, along with the necessary IAM policy bindings. This allows the platform administrator to provide fine-grained user access control and quota policies for Ray cluster resources.
• KubeRay operator. The operator is responsible for watching for changes in KubeRay resources and reconciling the state of the KubeRay clusters.  
• Logging. The `logging_config` section enables logs from system components and workloads to write logs to Cloud logging.
• Monitoring. The `monitoring_config` section enables Managed Prometheus integration. This allows the deployment to automatically scrape system-level metrics and writes them to the managed metrics service.
• Workload identity. This enables your workloads to authenticate with other GCP services using Google IAM service accounts.

User-level resources are expected to be deployed once by each user in the development environment.

• KubeRay cluster. This is the actual Ray cluster that we will be used for your workloads. It is configured to use a Workload Identity pool and a IAM-binded service account that provides fine-grained access to GCP services. You can customize the Ray cluster settings by editing the kuberay-values.yaml file.

• Logging. The solution adds a side car container deployed alongside each KubeRay worker node. This uses fluentbit to forward Ray logs from the head node to Cloud logging. You can edit the fluentbit-config file to change how the logging container filters and flushes logs.
• Monitoring. This module provides a PodMonitoring resource that scrapes metrics from the user’s Ray cluster and uploads data points to Google Managed Prometheus. An optional installation for Grafana dashboard is included and can be accessed through a web browser.
• JupyterHub server. This module installs a JupyterHub notebook server in the user namespace, enabling users to interact directly with their Ray clusters.

Run a Workload on Your Ray Cluster

Let’s try running the provided example with Ray Serve to deploy Stable Diffusion. This example was originally taken from the Ray Serve documentations here. To open the example in Jupyter notebook, go to the external IP for proxy-public in your browser (instructions to get the IP). And then click on File -> Open from URL, and input the raw URL of the notebook to open it.

Since the notebook runs in the same Kubernetes cluster as the Ray cluster, it is able to talk directly to the latter using its cluster-internal service endpoint – thus there is no need to expose the Ray cluster to public internet traffic. For production workloads, you should secure your endpoints with GCP account credentials. Google Cloud Identity Aware Proxy (IAP) can be used to enable fine-grained access control to user resources, such as our Ray cluster, to protect your GCP resources from unnecessary exposure. A full tutorial on how to enable IAP on your GKE cluster can be found here.

The notebook contains code for deploying a pre-trained model to a live endpoint. The last cell makes a call to the created service endpoint:prompt = “a cute cat is dancing on the grass.”input = “%20″.join(prompt.split(” “))resp = requests.get(f”http://example-cluster-kuberay-head-svc:8000/imagine?prompt={input}”)with open(“output.png”, ‘wb’) as f:   f.write(resp.content)

Executing the notebook will generate a file with a unique picture of a cute cat. Here is an example we got:

Congratulations! You have now deployed a large model for image generation on GKE.

Logging and Monitoring

As mentioned earlier, this solution enables logging and monitoring automatically. Let’s find those logs.

In your Cloud Console, open up Logging -> Log Explorer. In the query text box, enter the following:resource.type=”k8s_container”resource.labels.cluster_name=%CLUSTER_NAME%resource.labels.pod_name=%RAY_HEAD_POD_NAME%resource.labels.container_name=”fluentbit”

You should see the Ray logs from your cluster forwarded here.

To see your monitoring metrics, go to Metrics Explorer in the Cloud Console. Under the menu for “Target”, select “Prometheus Target” and then “Ray”. Select the metric that you want to see, for instance `prometheus/ray_component_cpu_percentage/gauge`:

The deployment also comes with a Grafana deployment. Follow this guide to open it up and view your Ray cluster’s metrics.

Conclusion

The combination of Ray and GKE offers a simple and powerful solution for building, deploying, and managing distributed applications. Ray’s simplicity makes it an attractive choice for data and model developers while GKE’s scalability and reliability is the defacto choice for enterprise platforms. The solution template presented in this blog post offers a convenient way to get started quickly with KubeRay, the recommended approach to deploy Ray on GKE.

If you have any questions for building Ray on Kubernetes and GKE, you can contact us directly at ray-on-gke@google.com or comment in GitHub. Learn more about building AI Platforms with GKE by visiting our User Guide.