Today, we published a new Google research report on software supply chain security because we’ve seen a sharp rise in software supply chain attacks across almost every sector —and expect these trends to continue for the foreseeable future. We urge all organizations to act now to improve their software supply chain security.

Among the report’s conclusions, there are two key findings we want to highlight. First, the lessons we’ve learned from various security events call for a more holistic approach to strengthen defenses against software supply chain attacks. Second, we have worked with the security community to develop and deploy a common Supply-chain Levels for Software Artifacts (SLSA) framework that can mitigate threats across the entire software supply chain ecosystem. These frameworks can help organizations securely build and verify the integrity of software. You can find more information on the report’s conclusions here.

We know that modern day software supply chains continue to grow deeper, wider, and more complex. That complexity can make it challenging for customers to even know where to begin analyzing their supply chains for security issues. Our research shows that organizations must deal with these same complex issues regardless of which environments they operate in.

At Google Cloud, we’re deeply committed to working with our customers to help ensure that they have the support they need to evaluate their security posture, resiliency, and hygiene. Below, we suggest five steps to protect software across processes and systems, and tap into relevant Google Cloud products and services. These recommendations can enable customers to benefit from Google’s extensive security experience and reduce their need to develop, maintain, and operate complex processes to secure their open source dependencies.

Implementing best practices with Google Cloud

Customers who are interested in improving their software supply chain security can take immediate steps to implement best practices.

1. Enhance your existing Google Cloud security features with the Google Cloud security foundation guide. The guide can help you weigh important considerations including organizational structure, authentication and authorization, resource hierarchy, networking, logging, and detective controls. You can further engage Mandiant experts to assess your readiness.

You can also view centralized information about vulnerabilities and possible risks using Google Cloud services like Security Command Center, and get information about your service usage with Recommender, including recommendations that can help you to reduce risk. For example, you can identify IAM principals with excess permissions or unattended Google Cloud projects. You can also find additional resources from the Google Cybersecurity Action Team (GCAT), our premier security advisory team, here.

2. Explore fast software delivery and reliable and secure software with Google Cloud’s DevOps capabilities. You also should review foundational practices for designing, developing, and testing code that apply to most programming languages.

We strongly recommend you evaluate how you distribute software and the terms of software licenses in all of your dependencies. For more information on Google’s approach to helping organizations address vulnerabilities in open source software, see Appendix B in the research report.

3. Document the policies for your organization and incorporate validation of policies into your development, build, and deployment processes as you implement best practices. For example, your organization’s policies might include criteria for deployment that you implement with Binary Authorization. GCAT has published additional information on security policies and other cloud security transformation tips for CISOs here.

You can also explore Minimum Viable Secure Product, a security checklist of controls to establish a baseline security posture for a product. You can use the checklist to establish your minimum security control requirements and to evaluate software by third-party vendors.

Tapping into new Google product and service offerings

At Google Cloud, we continue to focus on delivering new and innovative security capabilities to help customers address the latest security threats. From the attack on SolarWinds to the community response to open source vulnerabilities such as Log4j, we’re seeing a spike in demand from customers on what we can do to help them manage software supply chain risk. We’ve made several recent announcements on that front that can help customers get started with Google Cloud today.

4. Use Google Cloud’s Software Delivery Shield. It provides a fully managed software supply chain security solution that offers a modular set of capabilities to help equip developers, DevOps, and security teams with the tools they need to build secure cloud applications. Software Delivery Shield spans across a family of Google Cloud services from developer tooling to runtimes including GKE, Cloud Code, Cloud Build, Cloud Deploy, Artifact Registry, and Binary Authorization. To learn more about Software Delivery Shield, check out the solution page, or watch this Google Cloud Next session to get a quick overview of Software Delivery Shield.
5. Enable our Assured Open Source Software (OSS) service, which can help enterprise and public sector open source software users to easily incorporate the same OSS packages that we use at Google into their own developer workflows. Packages curated by the Assured OSS service:

• are regularly scanned, analyzed, and fuzz-tested for vulnerabilities;
• have corresponding enriched metadata incorporating Container/Artifact Analysis data;
• are built with Cloud Build including evidence of verifiable Supply chain Levels for Software Artifacts (SLSA)-compliance;
• are verifiably signed by Google;
• and are distributed from an Artifact Registry that is secured and protected by Google.

If you are interested in learning more about software supply chain security in general, please contact us or reach out to your sales representative to schedule a software supply chain security workshop.

In fast-moving organizations, it’s not uncommon for cloud resources, including entire projects, to occasionally be forgotten about. Not only such unattended resources can be difficult to identify, but they also tend to create a lot of headaches for product teams down the road, including unnecessary waste and security risks. 

To help you prune your idle cloud resources, we’re excited to introduce Unattended Project Recommender. It’s a new feature of Active Assist that provides you with a one-stop shop for discovering, reclaiming, and shutting down unattended projects. With actionable and automatic recommendations, you no longer have to worry about wasting money or mitigating security risks presented by your idle resources. Unattended Project Recommender uses machine learning to identify, with a high degree of confidence, projects that are likely abandoned based on API and networking activity, billing, usage of cloud services, and other signals. This feature is available via the Recommender API today, making it easy for you to integrate with your company’s existing workflow management and communication tools, or export results to a BigQuery table for custom analysis.

Thousands of projects can be unattended in large organizations, presenting major security risks

Your cloud projects can go abandoned or unattended for a number of reasons — ranging from a test environment that’s no longer needed, to project cancellation, to project owner switching jobs, and more. Not only can such projects contribute to your cloud bill (waste) but they may contain security issues such as open firewalls or privileged service account keys that attackers can exploit to get a hold of your cloud resources for cryptocurrency mining or, worse, compromise your company’s sensitive data. These security risks tend to grow over time because the latest best practices and patches are usually not applied to unattended projects. 

We experience this issue here at Google, too… In fact, it has been on Google’s internal security team’s radar for some time now, so we joined forces and looked into this problem together, starting with our very own “google.com” organization cloud projects. We quickly found some projects that were unattended, but remediating this issue was easier said than done due to challenges in several areas:

• Detection: With lots of signals available to you via sources like Cloud Monitoring, what are the right ones you should look at (e.g. API, networking, user activity)? How can you tell the difference between an unattended project and a project that has a low level of activity by design (e.g. a “shell” project that holds an auth token)?
• Remediation: Once you have identified a project that seems abandoned, how do you go about ensuring that it’s indeed an unattended project? How do you reduce the risk of deleting something that might be essential to a production workload, causing irreversible data loss? How do you solve this at the scale of your entire organization, beyond a one-time cleanup? 

Over the course of 2021 we built and tested a Google-internal prototype first, cleaning up many of our internal unattended projects, and then worked with a number of Google Cloud customers to build and tune this feature based on real-life data (thank you to all of our early adopters for working with us and your generous feedback that helped us shape this feature!) It was not uncommon for us to come across organizations with thousands of unattended projects, and we’re very excited to bring Unattended Project Recommender to all customers, in public preview.

Discovering and acting on unattended project recommendations

Unattended Project Recommender analyzes usage activity across all projects under your organization, including the following data:

• API activity (e.g. service accounts with authentication activity, API calls consumed)
• Networking activity (ingress and egress)
• Billing activity (e.g. services with billable usage)
• User activity (e.g. active project owners)
• Cloud services usage (e.g. number of active VMs, BigQuery jobs, storage requests)

Based on these signals, it can generate recommendations to clean up projects that have low usage activity (where “low usage” is defined using a machine learning model that ranks projects in your organization by level of usage), or recommendations to reclaim projects that have high usage activity but no active project owners. Here’s what an example post-processed summary list of recommendations can look like for the “foobar” organization that has 3 projects:

  Project ID: demo-project-307815
Recommendation: CLEANUP_PROJECT

Project ID: new-project
Recommendation: N/A

Project ID: bobs-playground-project
Recommendation: RECLAIM_PROJECT

In addition to the recommendations, you can also examine the underlying project activity insights that the recommendations are based upon. The insights provide additional information that can be useful for integration with your organization’s existing workflows and automation (e.g. send an auto-generated email or chat message to project owners based on the list provided by the owners field). Here’s an example insight payload:

  content:
  activeAppengineInstanceDailyCount: 0
  activeCloudsqlInstanceDailyCount: 0
  activeGceInstanceDailyCount: 3
  activeServiceAccountDailyCount: 1
  apiClientDailyCount: 18922           // Daily average API calls produced
  bigqueryInflightJobDailyCount: 0
  bigqueryInflightQueryDailyCount: 0
  bigqueryStorageDailyBytes: 0
  bigqueryTableDailyCount: 0
  consumedApiDailyCount: 0             // Daily average API calls consumed
  datastoreApiDailyCount: 0
  gcsObjectDailyCount: 11
  gcsRequestDailyCount: 0
  gcsStorageDailyBytes: 2663548
  hasActiveOauthTokens: false          // OAuth tokens used in the last 180 days
  hasBillingAccount: true
  numActiveUserOwners: 1
  owners:                              // List of project owners
  - activeOnProject: false
    member: user:user1@example.com
  - activeOnProject: true
    member: user:user2@example.com
  serviceWithBillableUsage:
  – Cloud Storage
  - Compute Engine
  vpcEgressDailyBytes: 264456938       // Daily average VPC egress bytes
  vpcIngressDailyBytes: 392435047      // Daily average VPC ingress bytes
  usagePercentile: 20                  // Level of usage relative to other projects

GCP projects are used in many different ways and for many different purposes. In case you get a recommendation to delete a project that’s being used in a way that’s out of the scope for this feature, you can dismiss the recommendation and it will stop showing up for the given project. 

Restoring deleted projects

When you choose to shut down a project using the projects.delete() method, it gets marked for deletion. After a project is marked for deletion, it becomes unusable, all resources within that project are shut down, and a 30-day wait period for the project and all of its data to get fully deleted begins.

In case a useful project is accidentally shut down, you have the option to restore the project within that 30-day wait period. Since restoring allows you to recover most but not necessarily all of your project data and resources, we recommend carefully examining the utilization insights associated with a project and considering any additional utilization signals that may not be captured by the Unattended Project Recommender before taking the cleanup action.

Early customer success stories

A number of enterprise customers are already using Unattended Project Recommender to keep their organizations clean of unattended projects and resources.

Decathlon, a French sporting goods retailer, is excited for the insight Unattended Project Recommender will bring to their environment, and are already deploying it as a part of their latest cloud security initiatives.

“After a thorough test of this feature and the validation of our CISO, we ended up deleting our first 775 projects, and no one complained! A great help to improve our security. The next step for us will be to operationalize it at scale, and implement a company wide policy for unattended resource management.” —Adeline Villette, Cloud Security Officer

For Veolia, one of the world’s largest water, waste and energy management companies, not only does this feature reduce security risks and waste, but also helps drive cultural shift and alignment with its ecological transformation strategy.

“This feature allows us to reduce our costs and security debt on assets that are no longer in use, and is also fully in line with Veolia’s philosophy of limiting its carbon footprint. After having tested Unattended Project Recommender on more than 3,000 projects throughout our organization, we are looking to bring it as proactive alerts to our project owners at scale.”—Thomas Meriadec, Product Manager

Box, a secure cloud content management provider, views it as a foundation for building a repeatable process to remediate unused resources.

“Unattended Project Recommender is a great fit for us. It gives us a unified view of project usage across our entire organization and enables us to address security risks of legacy projects in a systematic and organized manner, ensuring an even safer environment.” —Matt Bowes, Staff Security Engineer

Getting started with the Unattended Project Recommender

To help you get started, we’ve prepared a Cloud Shell tutorial (source code) that you can use to find unattended project recommendations within your own Projects/Folders/Organization. Click this button to clone the tutorial from GitHub and run in your Cloud Shell environment:

As you can see, listing recommendations for your projects only takes a few clicks with the tutorial (special thanks to Lanre Ogunmola, Security & Compliance Specialist, for making this look so easy)! For additional detail on using the gcloud CLI or API to discover unattended project recommendations, please refer to the documentation page.

You can also automatically export all recommendations from your Organization to BigQuery and then investigate the recommendations with DataStudio or Looker, or use Connected Sheets that let you use Google Workspace Sheets to interact with the data stored in BigQuery without having to write SQL queries.

As with any other Recommender, you can choose to opt out of data processing at any time by disabling the appropriate data groups in the Transparency & control tab under Privacy & Security settings.

We hope that you can leverage Unattended Project Recommender to improve your cloud security posture and reduce cost, and can’t wait to hear your feedback and thoughts about this feature! Please feel free to reach us at active-assist-feedback@google.com and we also invite you to sign up for our Active Assist Trusted Tester Group if you would like to get early access to the newest features as they are developed.

The success of the modern security operations center, despite the infusion of automation, machine learning, and artificial intelligence, remains heavily dependent on people. This is largely due to the vast amounts of data a security operations center must ingest—a product of an ever-expanding attack surface and the borderless enterprise brought on by the rapid rise of cloud adoption.

All those alerts coming in mean proactive and reactive human decision making remains critical.

Perhaps it should come as no surprise that the information security analyst now ranks as No. 1 News’ 100 Best Jobs Rankings, “determined by identifying careers with the largest projected number and percentage of openings through 2030, according to the U.S. Bureau of Labor Statistics.” Security, and specifically detection and response, is not only a business imperative—it is arguably the top worry on the minds of CEOs.

However, the security analyst is also one of the most likely professionals to want to leave their jobs, according to a newly released “Voice of the SOC Analyst” study conducted by Tines.

What gives? Turnover woes are attributable to several key SecOps challenges that never seem to budge.

1) Alert fatigue and false positives: Have you ever received so much spam or junk mail that you end up ignoring your new messages entirely, leading you to miss an important one? The same can happen for alerts. Too much noise is unsustainable and can lead to the real threats being missed, especially as perimeters expand and cloud adoption increases.

2) Disparate tools: Already in the company of too many point-detection tools, security operations professionals are saying hello to a few more in the era of remote work and increased cloud demands. The latest count is north of 75 security tools that need to be managed by the average enterprise.

3) Manual processes: Use case procedures that result in inconsistent, unrepeatable processes can bottleneck response times and frustrate SecOps teams. Not everything in the SOC needs to—or should be—automated, but much can be, which then frees up analysts and engineers to concentrate on higher-order tasks and be able to more easily train new employees.

4) Talent shortage: Death, taxes, and the cybersecurity skills shortage. As sure as the sun will rise tomorrow, so will the need for skilled individuals to wage the cybersecurity fight. But what happens when not enough talent is filling the seats? Teams must compensate to fill the gap.

5) Lack of visibility: Security operations metrics are critical for improving productivity and attracting executive buy-in and support, but SecOps success can be difficult to track, as reports can require a significant amount of work to pull together.

The caveat of course is that it would be rare to find a SecOps team working without the above challenges. As such, some of the immediate steps you can take to push back against these constraints focus on people-powered processes and technologies to remedy the issues.

According to a recent paper co-authored by Google Cloud and Deloitte:

Humans are—and will be—needed to both perform final triage on the most obtuse security signals (similar to conventional SOC Level 3+) and to conduct a form of threat hunting (i.e. looking for what didn’t trigger that alert).

Machines will be needed to deliver better data to humans, both in a more organized form (stories made of alerts) and in improved quality detections using rules and algorithms— all while covering more emerging IT environments.

Both humans and machines will need to work together on mixed manual and automated workflows.

So, what does this ultimately mean you must do to improve your security operations? Here are five practical suggestions:

Detect Threats More Efficiently

Efficiencies within the SOC can be realized from a SIEM solution that automatically detects threats in real-time and at scale. The right platform will support massive data ingestion and storage, relieve traditional cost and scaling limitations, and broaden the lens for anomaly and machine learning/AI-based detection. With data stored and analyzed in one place, security teams can investigate and detect threats more effectively.

Respond to Threats Automatically

SOAR can be a game-changer in terms of caseload reduction and faster (and smarter, especially when integrated with threat intelligence) response times. But before rushing headfirst into automation, you should consider your processes, review outcomes you are trying to achieve (such as reduced MTTD)–and then decide exactly what you want to automate (which can be a lot with SOAR). Once clear processes are determined where automation can contribute, SOC personnel are freed up to be more creative in their work.

Prioritize Logs

Many teams lack a strategy for collecting, analyzing and prioritizing logs, despite the fact that these sources of insight often hold the clues of an ongoing attack. To help, here are two cheat sheets featuring essential logs to monitor.

Outsource What You Can’t Do Yourself

Process improvements may help you compensate for perceived personnel shortages (for example, perhaps fixing a misconfigured monitoring tool will reduce alert noise). Of course, many organizations need additional human hands to help them perform tasks like round-the-clock monitoring and more specialized functions like threat hunting. Here is where a managed security services provider or managed detection provider can be helpful. Be realistic about your budget, however, as you may be able to introduce some solutions in-house.

Institute Career Models

Lack of management support is cited as the fourth-biggest obstacle to a fully functioning SOC model, according to the 2022 SANS Security Operations Center Survey. To overcome this, SecOps leaders must help improve workflow processes, protect innovation, keep teams absorbed in inspiring and impactful work versus mundane tasks, remain flexible with staff, and endorse training and career development. Because at the end of the day, the SOC is still distinctly human–and that is who will be the difference maker between success and failure.