In fast-moving organizations, it’s not uncommon for cloud resources, including entire projects, to occasionally be forgotten about. Not only such unattended resources can be difficult to identify, but they also tend to create a lot of headaches for product teams down the road, including unnecessary waste and security risks. 

To help you prune your idle cloud resources, we’re excited to introduce Unattended Project Recommender. It’s a new feature of Active Assist that provides you with a one-stop shop for discovering, reclaiming, and shutting down unattended projects. With actionable and automatic recommendations, you no longer have to worry about wasting money or mitigating security risks presented by your idle resources. Unattended Project Recommender uses machine learning to identify, with a high degree of confidence, projects that are likely abandoned based on API and networking activity, billing, usage of cloud services, and other signals. This feature is available via the Recommender API today, making it easy for you to integrate with your company’s existing workflow management and communication tools, or export results to a BigQuery table for custom analysis.

Thousands of projects can be unattended in large organizations, presenting major security risks

Your cloud projects can go abandoned or unattended for a number of reasons — ranging from a test environment that’s no longer needed, to project cancellation, to project owner switching jobs, and more. Not only can such projects contribute to your cloud bill (waste) but they may contain security issues such as open firewalls or privileged service account keys that attackers can exploit to get a hold of your cloud resources for cryptocurrency mining or, worse, compromise your company’s sensitive data. These security risks tend to grow over time because the latest best practices and patches are usually not applied to unattended projects. 

We experience this issue here at Google, too… In fact, it has been on Google’s internal security team’s radar for some time now, so we joined forces and looked into this problem together, starting with our very own “google.com” organization cloud projects. We quickly found some projects that were unattended, but remediating this issue was easier said than done due to challenges in several areas:

• Detection: With lots of signals available to you via sources like Cloud Monitoring, what are the right ones you should look at (e.g. API, networking, user activity)? How can you tell the difference between an unattended project and a project that has a low level of activity by design (e.g. a “shell” project that holds an auth token)?
• Remediation: Once you have identified a project that seems abandoned, how do you go about ensuring that it’s indeed an unattended project? How do you reduce the risk of deleting something that might be essential to a production workload, causing irreversible data loss? How do you solve this at the scale of your entire organization, beyond a one-time cleanup? 

Over the course of 2021 we built and tested a Google-internal prototype first, cleaning up many of our internal unattended projects, and then worked with a number of Google Cloud customers to build and tune this feature based on real-life data (thank you to all of our early adopters for working with us and your generous feedback that helped us shape this feature!) It was not uncommon for us to come across organizations with thousands of unattended projects, and we’re very excited to bring Unattended Project Recommender to all customers, in public preview.

Discovering and acting on unattended project recommendations

Unattended Project Recommender analyzes usage activity across all projects under your organization, including the following data:

• API activity (e.g. service accounts with authentication activity, API calls consumed)
• Networking activity (ingress and egress)
• Billing activity (e.g. services with billable usage)
• User activity (e.g. active project owners)
• Cloud services usage (e.g. number of active VMs, BigQuery jobs, storage requests)

Based on these signals, it can generate recommendations to clean up projects that have low usage activity (where “low usage” is defined using a machine learning model that ranks projects in your organization by level of usage), or recommendations to reclaim projects that have high usage activity but no active project owners. Here’s what an example post-processed summary list of recommendations can look like for the “foobar” organization that has 3 projects:

  Project ID: demo-project-307815
Recommendation: CLEANUP_PROJECT

Project ID: new-project
Recommendation: N/A

Project ID: bobs-playground-project
Recommendation: RECLAIM_PROJECT

In addition to the recommendations, you can also examine the underlying project activity insights that the recommendations are based upon. The insights provide additional information that can be useful for integration with your organization’s existing workflows and automation (e.g. send an auto-generated email or chat message to project owners based on the list provided by the owners field). Here’s an example insight payload:

  content:
  activeAppengineInstanceDailyCount: 0
  activeCloudsqlInstanceDailyCount: 0
  activeGceInstanceDailyCount: 3
  activeServiceAccountDailyCount: 1
  apiClientDailyCount: 18922           // Daily average API calls produced
  bigqueryInflightJobDailyCount: 0
  bigqueryInflightQueryDailyCount: 0
  bigqueryStorageDailyBytes: 0
  bigqueryTableDailyCount: 0
  consumedApiDailyCount: 0             // Daily average API calls consumed
  datastoreApiDailyCount: 0
  gcsObjectDailyCount: 11
  gcsRequestDailyCount: 0
  gcsStorageDailyBytes: 2663548
  hasActiveOauthTokens: false          // OAuth tokens used in the last 180 days
  hasBillingAccount: true
  numActiveUserOwners: 1
  owners:                              // List of project owners
  - activeOnProject: false
    member: user:user1@example.com
  - activeOnProject: true
    member: user:user2@example.com
  serviceWithBillableUsage:
  – Cloud Storage
  - Compute Engine
  vpcEgressDailyBytes: 264456938       // Daily average VPC egress bytes
  vpcIngressDailyBytes: 392435047      // Daily average VPC ingress bytes
  usagePercentile: 20                  // Level of usage relative to other projects

GCP projects are used in many different ways and for many different purposes. In case you get a recommendation to delete a project that’s being used in a way that’s out of the scope for this feature, you can dismiss the recommendation and it will stop showing up for the given project. 

Restoring deleted projects

When you choose to shut down a project using the projects.delete() method, it gets marked for deletion. After a project is marked for deletion, it becomes unusable, all resources within that project are shut down, and a 30-day wait period for the project and all of its data to get fully deleted begins.

In case a useful project is accidentally shut down, you have the option to restore the project within that 30-day wait period. Since restoring allows you to recover most but not necessarily all of your project data and resources, we recommend carefully examining the utilization insights associated with a project and considering any additional utilization signals that may not be captured by the Unattended Project Recommender before taking the cleanup action.

Early customer success stories

A number of enterprise customers are already using Unattended Project Recommender to keep their organizations clean of unattended projects and resources.

Decathlon, a French sporting goods retailer, is excited for the insight Unattended Project Recommender will bring to their environment, and are already deploying it as a part of their latest cloud security initiatives.

“After a thorough test of this feature and the validation of our CISO, we ended up deleting our first 775 projects, and no one complained! A great help to improve our security. The next step for us will be to operationalize it at scale, and implement a company wide policy for unattended resource management.” —Adeline Villette, Cloud Security Officer

For Veolia, one of the world’s largest water, waste and energy management companies, not only does this feature reduce security risks and waste, but also helps drive cultural shift and alignment with its ecological transformation strategy.

“This feature allows us to reduce our costs and security debt on assets that are no longer in use, and is also fully in line with Veolia’s philosophy of limiting its carbon footprint. After having tested Unattended Project Recommender on more than 3,000 projects throughout our organization, we are looking to bring it as proactive alerts to our project owners at scale.”—Thomas Meriadec, Product Manager

Box, a secure cloud content management provider, views it as a foundation for building a repeatable process to remediate unused resources.

“Unattended Project Recommender is a great fit for us. It gives us a unified view of project usage across our entire organization and enables us to address security risks of legacy projects in a systematic and organized manner, ensuring an even safer environment.” —Matt Bowes, Staff Security Engineer

Getting started with the Unattended Project Recommender

To help you get started, we’ve prepared a Cloud Shell tutorial (source code) that you can use to find unattended project recommendations within your own Projects/Folders/Organization. Click this button to clone the tutorial from GitHub and run in your Cloud Shell environment:

As you can see, listing recommendations for your projects only takes a few clicks with the tutorial (special thanks to Lanre Ogunmola, Security & Compliance Specialist, for making this look so easy)! For additional detail on using the gcloud CLI or API to discover unattended project recommendations, please refer to the documentation page.

You can also automatically export all recommendations from your Organization to BigQuery and then investigate the recommendations with DataStudio or Looker, or use Connected Sheets that let you use Google Workspace Sheets to interact with the data stored in BigQuery without having to write SQL queries.

As with any other Recommender, you can choose to opt out of data processing at any time by disabling the appropriate data groups in the Transparency & control tab under Privacy & Security settings.

We hope that you can leverage Unattended Project Recommender to improve your cloud security posture and reduce cost, and can’t wait to hear your feedback and thoughts about this feature! Please feel free to reach us at active-assist-feedback@google.com and we also invite you to sign up for our Active Assist Trusted Tester Group if you would like to get early access to the newest features as they are developed.

Google Cloud’s industry-leading controls, contractual commitments, and accountability tools have helped organizations across Europe meet stringent data protection regulatory requirements for years. This commitment to supporting the  compliance efforts of European companies has earned us the trust of businesses like retailers, manufacturers and financial services providers.

As part of our continued efforts to uphold that trust, Google Cloud was one of the first cloud providers to support and adopt the EU GDPR Cloud Code of Conduct (CoC). The CoC is a mechanism for cloud providers to demonstrate how they offer sufficient guarantees to implement appropriate technical and organizational measures as data processors under the GDPR.  

Today the Belgian Data Protection Authority, based on a positive opinion by the European Data Protection Board (EDPB), approved the CoC, a product of years of constructive collaboration between the cloud computing community, the European Commission, and European data protection authorities. We are proud to say that Google Cloud Platform and  Google Workspace already adhere to these provisions. This is the first European code approved under the GDPR; it is excellent news for the industry to have a new transparency and accountability tool that helps promote trust in the cloud. 

In addition to the CoC, Google Cloud has already been certified against internationally-recognized privacy standards such as ISO/IEC 27001, ISO/IEC 27017, ISO/IEC 27018 and ISO/IEC 27701. These certifications provide independent validation of our ongoing dedication to world-class security and privacy.

This initiative reaffirms Google Cloud’s commitment to help our customers navigate their compliance journey when using our services. To learn more about how Google Cloud can help organizations with their compliance efforts, visit our Cloud Compliance resource center.

One of the biggest challenges when serving machine learning models is delivering predictions in near real-time. Whether you’re a retailer generating recommendations for users shopping on your site, or a food service company estimating delivery time, being able to serve results with low latency is crucial. That’s why we’re excited to announce Private Endpoints on Vertex AI, a new feature in Vertex Predictions. Through VPC Peering, you can set up a private connection to talk to your endpoint without your data ever traversing the public internet, resulting in increased security and lower latency for online predictions. 

Configuring VPC Network Peering

Before you make use of a Private Endpoint, you’ll first need to create connections between your VPC (Virtual Private Cloud) network and Vertex AI.  A VPC network is a global resource that consists of regional virtual subnetworks, known as subnets, in data centers, all connected by a global network. You can think of a VPC network the same way you’d think of a physical network, except that it’s virtualized within GCP. If you’re new to cloud networking and would like to learn more, check out this introductory video on VPCs.

With VPC Network Peering, you can connect internal IP addresses across two VPC networks, regardless of whether they belong to the same project or the same organization. As a result, all traffic stays within Google’s network.

Deploying Models with Vertex Predictions

Vertex Predictions is a serverless way to serve machine learning models. You can host your model in the cloud and make predictions through a REST API. If your use case requires online predictions, you’ll need to deploy your model to an endpoint. Deploying a model to an endpoint associates physical resources with the model so it can serve predictions with low latency. 

When deploying a model to an endpoint, you can specify details such as the machine type, and parameters for autoscaling. Additionally, you now have the option to create a Private Endpoint. Because your data never traverses the public internet, Private Endpoints offer security benefits in addition to reducing the time your system takes to serve the prediction when it receives the request. The overhead introduced by Private Endpoints is minimal, achieving performance nearly identical to DIY serving on GKE or GCE.  There is also no payload size limit for models deployed on the private endpoint.

Creating a Private Endpoint on Vertex AI is simple.

In the Models section of the Cloud console, select the model resource you want to deploy.

Next, select DEPLOY TO ENDPOINT

In the window on the right hand side of the console, navigate to the Access section and select Private. You’ll need to add the full name of the VPC network for which your deployment should be peered.

Note that many other managed services on GCP support VPC peering, such as Vertex Training, Cloud SQL, and Firestore. Endpoints is the latest to join that list.

What’s Next?

Now you know the basics of VPC Peering and how to use Private Endpoints on Vertex AI. If you want to learn more about configuring VPCs, check out this overview guide. And if you’re interested to learn more about how to use Vertex AI to support your ML workflow, check out this introductory video. Now it’s time for you to deploy your own ML model to a Private Endpoint for super speedy predictions!

Security operations teams are facing a “perfect storm” of challenges from nation-state actors turning their attention to financial crime, to rising uncertainty and potential complexities because of rapidly advancing cloud migration and IoT adoption, to the long-lamented skills shortage. 

Now imagine having to face this trifecta without visibility into your IT and security infrastructure because scale and cost concerns have impeded your ability to ingest and monitor critical telemetry. To help organizations rethink threat detection and broader security team efforts, we showcased at this month’s Google Cloud Security Talks a keynote on why DEI is a cybersecurity imperative, Mandiant’s review of lessons for organizations two years after SolarWinds, how Google Chrome already helps protect your business (and what more you can make do with it,) how best to encourage your organization to pursue Zero Trust, and three key ways in which Chronicle Security Operations can help your organization scale threat detection and actionable outcomes. These include:

1. Operationalizing threat intelligence

Harnessing the power of threat intelligence to acquire context, prioritize risk, and act on detections requires more than just ingesting a set of indicator of compromise (IOC) feeds. Threat intelligence must have breadth and depth of coverage to provide custom details on the tools, tactics, and procedures of threat actors. Explore why part of the “magic” behind Google Cloud’s security is the sheer scale of threat intelligence we can share with our customers. 

2. Driving detection as code

We want to enable organizations that are more advanced in their cybersecurity maturity to build their own custom detections and rules, which means offering our customers a powerful detection-authoring platform. Chronicle Security Operations can help analysts build specific detections around, for example, known bad files or a specific registry key change. We also demonstrated how to create more complex detections, such as flagging process execution patterns or crafting a suspicious-behavior trigger from a specific activity sequence.

3. Leveraging curated detections

With curated detections, which we announced in August, we are putting the power of Google Cloud’s intelligence in the hands of security operations teams everywhere. Our detections offer actionable, ready-to-use threat detection content curated, built, and maintained by Google Cloud Threat Intelligence (GCTI) researchers. These detection sets cover a wide variety of threats for your network and beyond, including attacks such as ransomware, remote-access tools (RAT), infostealers, data exfiltration, and suspicious activity.

You can watch all nine of our Security Talks recorded sessions here, and keep an eye out for our next Security Talks, coming in March 2023.

As one of the most trusted cloud platform providers, Google is committed to providing our clients secure and reliable environments for their workloads. Google believes the future of computing will increasingly shift to private, encrypted services where users can be confident that their data is not being exposed to cloud providers or their own insiders. Confidential Computing helps make this future possible by keeping data encrypted in memory, and elsewhere outside the CPU, while it is being processed.

Since our first Confidential Computing offering in 2018, Google has been a pioneer in making the technology widely available through our cloud, while also constantly adding new features that allow a wide range of organizations to benefit from it. 

Expanding the Confidential Computing portfolio

Today at the RSA Conference 2023, we are announcing several exciting updates to our Confidential Computing portfolio.

We are expanding our portfolio with the next generation of our Confidential Computing VM instances that utilize AMD Infinity Guard technology, specifically AMD SEV-SNP (Secure Encrypted Virtualization-Secure Nested Paging). SEV-SNP adds additional new hardware-based security protections such as strong memory integrity protection and hardware rooted attestation. Confidential VMs on general purpose N2D VMs with SEV-SNP enabled are now in Private Preview. Learn more in our deep dive here.

“Advanced, hardware-based security features – such as AMD Infinity Guard built into AMD EPYC processors – are a crucial part of cloud offerings and the further expansion of cloud computing,” said Ram Peddibhotla, corporate vice president, Cloud Business, AMD. “Our work with Google Cloud delivers on the leadership security features our joint customers demand with nearly effortless implementation and minimal performance impact as more mission critical and business critical workloads move to the cloud.”

Google is committed to ensuring Confidential Computing technology is as secure as possible before releasing products to customers. We constantly evaluate various attack vectors to make certain that Google Cloud Confidential Computing environments are protected against a wide range of attacks. 

Throughout last year, Google and Intel collaborated on a research project to identify potential security vulnerabilities in Intel’s new Confidential Computing technology, Trust Domain Extensions (TDX). We’ve recently released the full report and made it available in our blog here.

“We want to make it such that people don’t worry about the security and trustworthiness of their data,” said Anil Rao, vice president and general manager of systems architecture and engineering in the Office of the CTO at Intel. “Organizations use confidential computing to control their data and provide access to trusted parties in a manner that is verifiable, revocable and time sensitive – we have an obligation to make sure the technology is secure. Our early effort with Google solidifies our commitment to perform thorough analysis to address all potential vulnerabilities.”

Confidential Space is now in general availability. Confidential Space builds on Confidential Computing and provides a secure enclave, also known as a Trusted Execution Environment (TEE). Google Cloud customers can leverage the TEE for various privacy-focused use cases such as joint data analysis and machine learning (ML) model training, with trust guarantees that data can stay protected — including hardened protection against cloud service provider access. 

Confidential Space allows for new privacy-preserving technologies to take shape, and one such technology is The Privacy Sandbox. This is a crucial step forward as Privacy Sandbox for the Web will phase out third-party cookies and limit covert tracking. By supporting a choice of trusted execution environments, including Confidential Space, the Privacy Sandbox will provide ad-techs with safer alternatives to existing technology, so they can continue building digital businesses while keeping user data private.

“Trusted Execution Environments (TEEs) are already in use across industries such as digital advertising to enable privacy preserving analytics, and we are excited to leverage Google Cloud’s built-in TEE called Confidential Space,” said Gaurav Bhaya, vice president and general manager, Google Ads Measurement.

Building on our history of innovation with Confidential Computing

Google has worked diligently with internal and external partners to validate, audit and publish the security assessment of the hardware, firmware, and software that underpins Confidential Computing. 

Last year, a joint security effort between Google Project Zero, Google Cloud, and AMD led to more secure technology for the entire industry, regardless of what service provider you choose for your Confidential Computing deployment. You can learn more in our full report here.

Since then, we have continued to expand the portfolio of Confidential Computing products and services, including Confidential Virtual Machines (CVMs), Confidential GKE, Confidential Dataproc, and Confidential Space. These offerings give our customers more choice in their deployments. 

Currently, our Confidential Computing services are widely available across 80% of Google Cloud regions and growing, and we recently added support for Compute-Optimized C2D VMs for Confidential GKE.

Organizations are now leveraging Confidential Computing for joint data analysis and machine learning (ML) model training with trust guarantees that the data they own stays protected and across various other industries by organizations, including AstraZeneca, Bullish, HashiCorp, Matrixx Software, MonetaGo and Yellowdog.

Confidential Computing has also proven to be a helpful additional control for organizations implementing digital sovereignty strategies, providing an encryption capability, and protection for data-in-use where encryption keys are not accessible by the cloud provider.

We can’t wait to see the possibilities this technology will open up for your organization. Check out our Confidential Computing page to learn more.