Google has long looked to Israel for globally impactful technologies including popular Search features, Waze, Live Caption, Duplex and flood forecasting. At our Decode with Google 15RAEL event last week, we celebrated 15 years of Google innovation in Israel and our longstanding support of the country’s vibrant startup ecosystem. 

Over the years, we’ve expanded our enterprise investments in the country, too. In addition to our over a decade long investment in the space, Google has acquired Israeli-based companies like Alooma, Elastifile and Velostrata, and Uri Frank joined Google Cloud last month to lead our server chip design team from our offices in Tel Aviv and Haifa. 

As we continue to meet growing demand for cloud services in Israel, we’re excited to announce that a new Google Cloud region is coming to Israel to make it easier for customers to serve their own users faster, more reliably and securely.

Our global network of Google Cloud regions are the foundation of the cloud infrastructure we’re building to support our customers. With cloud’s 25 regions and 76 zones around the world, we deliver high-performance, low-latency services and products for Google Cloud’s enterprise and public sector customers. With each new Google Cloud region, customers get access to secure infrastructure, smarter analytics tools, an open platform and the cleanest cloud in the industry. 

Having a region in Israel will help accelerate innovation for customers of all sizes, including PayBox, a digital wallet application owned by Discount Bank, one of Israel’s largest banks. “When we acquired PayBox, our goal was to improve the security and the user experience for its products, but we also wanted to keep the startup’s agility and innovation. Google Cloud has enabled us to do just that,” said Sarit Beck-Barkai, Managing Director of PayBox at Discount Bank.

“We are very excited that leading vendors like Google are investing and launching a local cloud region in Israel. This will make a significant change in the technology landscape of the public-sector, enterprise and SMB markets in Israel. Matrix is proud to be a major part of the transition to the cloud,” said Moti Gutman, CEO at Matrix, technology services company and Google Cloud partner. 

“In the last year, Panorays more than tripled its customer base and scaled its infrastructure, practically at the click of a button. Google Cloud made it easy for us to scale without worrying about DevOps, which meant that our engineers could focus on developing new and better features for our customers. The new region launching in Israel will allow us to serve our local customer base even better, as we’ll be able to experience higher availability and deploy resources in specific regions, thus reducing latency.” said Demi Ben-Ari, Co-founder and CTO, Panorays, a third-party security platform and Google Cloud customer.

“This new cloud region will provide even better access and growth potential for our mutual customers with tech hubs in the region. We are serving hyper growth companies who need Google Cloud’s services and will benefit greatly from this regional presence,” said Yoav Toussia-Cohen, CEO of DoiT International.

When it launches, the Israel region will deliver a comprehensive portfolio of Google Cloud products to private and public sector organizations locally. We look forward to welcoming you to the Israel region, and we’re excited to support your growing business on our platform. 

Learn more about our global cloud infrastructure, including new and upcoming regions, here.

At Google, we are constantly looking to improve our products, services, and contracts so that we can better serve our customers. To this end, we are pleased to announce that we have updated and merged our data processing terms for Google Cloud, Google Workspace (including Workspace for Education), and Cloud Identity (when purchased separately) into one combined Cloud Data Processing Addendum (the “CDPA”).

The CDPA maintains the benefits of the previously separate Data Processing and Security Terms for Google Cloud customers and Data Processing Amendment for Google Workspace and Cloud Identity customers, while streamlining and strengthening Google’s data processing commitments. A corresponding new CDPA (Partners) offers equivalent commitments to Google Cloud partners.

As part of this update, we have also incorporated the new international data transfer addendum issued by the U.K. Information Commissioner (“U.K. Addendum”). The U.K. Addendum allows the EU Standard Contractual Clauses (“SCCs”) to be used for transfers of personal data under the U.K. GDPR, replacing the separate U.K. SCCs that previously formed part of our terms. For an overview of the European legal rules for data transfers and our approach to implementing the EU SCCs and U.K. Addendum, please see our updated whitepaper. You can view our SCCs here.

While our data processing terms have been renamed, consolidated, and updated, our commitment to protecting the data of all Google Cloud, Workspace and Cloud Identity customers and all Google Cloud partners, and to enabling their compliance with data transfer and other regulatory requirements, remains unchanged.

For more information about our privacy commitments for Google Cloud, Google Workspace, and Cloud Identity, please see our Privacy Resource Center.

Content production is increasingly happening in the cloud. While moving a creative process that spans three centuries into the modern era brings challenges and risks, MovieLabs is one organization that takes digital transformation seriously.

A nonprofit research and development group founded by a consortium of the major Hollywood studios, MovieLabs describes their mission as “exploring innovative solutions to industry challenges shared by both our member studios and the broader production, post-production and distribution ecosystem.”

In more than 50 pages of densely-packed ideas and principles, MovieLab’s The Evolution of Media Production lays out an ambitious and revelatory vision for streamlining media production in the cloud. The key themes are summarized in 10 declarative statements, which envision a new modality for media production by the year 2030:

1. All assets are created or ingested straight into the cloud and do not need to be moved.
2. Applications come to the media.
3. Propagation and distribution of assets is a “publish” function.
4. Archives are deep libraries with access policies matching speed, availability and security to the economics of the cloud.
5. Preservation of digital assets includes the future means to access them and edit them.
6. Every individual on a project is identified and verified, and their access permissions are efficiently and consistently managed.
7. All media creation happens in a highly secure environment that adapts rapidly to changing threats.
8. Individual media elements are referenced, accessed, tracked and interrelated using a universal linking system.
9. Media workflows are non-destructive and dynamically created using common interfaces, underlying data formats and metadata.
10. Workflows are designed around real-time iteration and feedback.

The four bolded statements above are further explored in a follow-up whitepaper from MovieLabs, The Evolution of Production Security, which likewise offers six statements about what secure production in the cloud should look like:

1. Security is Intrinsic and does not Inhibit Creative Processes
2. The Security Architecture Addresses Challenges Specific to Cloud Workflows
3. Production Workflows, Processes, and Assets are Secure, even on Untrusted Infrastructure
4. The Content Owner Controls Security and Workflow Integrity
5. The Security Can Be Scaled to Appropriate Levels and Can Integrate
6. The Security Architecture Limits the Spread of any Breach and is Adaptable

The key to understanding these two papers is repeated in their titles: Evolution. Neither document is meant to outline solely what’s possible today; rather they are beacons guiding us toward a future in which the movie business is truly transformed by the cloud operating model. That means a future where media productions run much more like software supply chains: with speed, agility, and scale, and without sacrificing availability or security.

The good news is that it is possible to deploy secure production workloads in the cloud today. Using some of the built-in security features of Google Cloud, content producers can achieve Common Security Architecture for Production (CSAP) “Level 100” security for their assets (using the CSAP scale L100-L300). 

However, we think that “evolution” word is important. As many companies have learned, operating an infrastructure cloud can be costly and often presents new threat vectors for attackers. Our mission at Google Cloud is to provide scalable, reliable services for traditional IT functions and to accelerate the abilities of every organization to digitally transform its business. 

We believe the era of the transformation cloud is dawning, and companies who have been successfully operating in the cloud for years are now looking for ways to reduce costs while maintaining the cloud-scale advantages of agility and global reach. This aligns perfectly with the evolutionary vision of the MovieLabs papers, which envision media workflows changing radically to embrace cloud dynamism without all the unnecessary asset movement and complexity.

An important part of that vision from Google Cloud is that cloud service providers should remain open and interoperable with other clouds and infrastructures.

Here is a quickstart guide to mapping the L100 CSAP requirements to Google Cloud:

As you can see in the diagram, external identity providers (IdPs) such as Okta or Active Directory can serve as core security components of a CSAP L100 architecture. Alternatively, Google Cloud services such as Cloud Identity or Managed AD could be substituted.

Next, Google Cloud’s Identity-Aware Proxy (IAP) can be used in combination with Access Context Manager (ACM) to act as the first Policy Enforcement Point. This is an optional step, since it is somewhat redundant with the routing and authorization function performed by the native HP/Teradici Connection Brokers. However, IAP and ACM are core components of Google Cloud’s BeyondCorp architecture, on which CSAP is based. Additionally, adopting IAP with ACM offers productions the ability to swap out the Teradici PCoIP protocol for an alternative, such as Unity’s Parsec, without sacrificing security.

The Teradici Connection Broker is also a Policy Enforcement Point, since groups and role membership can be used to enforce access to specific workstations. 

Finally, the identity context for each user is passed through to the storage layer. For some productions, Google Cloud Storage (GCS) buckets may offer enough functionality; GCS offers an S3-compatible API which should make it easy to immediately adopt OSS tools built for S3. Other options for this include the managed NetApp Cloud Volumes and Dell PowerScale for Google Cloud services. Both of these are managed shared storage services which offer file-level access controls using standard SMB/NFS permissions. There are also a variety of third-party filers available in the Google Cloud Marketplace, such as Nasuni. You can read more about those here.

For added security, both NetApp and PowerScale should be configured to use the private services access model in Google Cloud. This effectively limits any external access to these resources by only allowing traffic from user-managed VPCs into the Google-managed and peered VPC.

Following this architecture, you can see how to achieve CSAP L100 compliance by using either only Google Cloud services or a mix of managed and cloud-first services. In either scenario, access is managed through a unified control plane and based on both user identity and role. Zero Trust principles such as context-aware access and dynamic rulesets could further be applied to the Authentication step using either Google Cloud IAP or the equivalent services from Okta, Active Directory, or others.

At this weekend’s National Association of Broadcasters conference, Google experts will be talking about the cloud and content production, and the intersection of AI and media. The journey to 2030 is only just beginning.

In fast-moving organizations, it’s not uncommon for cloud resources, including entire projects, to occasionally be forgotten about. Not only such unattended resources can be difficult to identify, but they also tend to create a lot of headaches for product teams down the road, including unnecessary waste and security risks. 

To help you prune your idle cloud resources, we’re excited to introduce Unattended Project Recommender. It’s a new feature of Active Assist that provides you with a one-stop shop for discovering, reclaiming, and shutting down unattended projects. With actionable and automatic recommendations, you no longer have to worry about wasting money or mitigating security risks presented by your idle resources. Unattended Project Recommender uses machine learning to identify, with a high degree of confidence, projects that are likely abandoned based on API and networking activity, billing, usage of cloud services, and other signals. This feature is available via the Recommender API today, making it easy for you to integrate with your company’s existing workflow management and communication tools, or export results to a BigQuery table for custom analysis.

Thousands of projects can be unattended in large organizations, presenting major security risks

Your cloud projects can go abandoned or unattended for a number of reasons — ranging from a test environment that’s no longer needed, to project cancellation, to project owner switching jobs, and more. Not only can such projects contribute to your cloud bill (waste) but they may contain security issues such as open firewalls or privileged service account keys that attackers can exploit to get a hold of your cloud resources for cryptocurrency mining or, worse, compromise your company’s sensitive data. These security risks tend to grow over time because the latest best practices and patches are usually not applied to unattended projects. 

We experience this issue here at Google, too… In fact, it has been on Google’s internal security team’s radar for some time now, so we joined forces and looked into this problem together, starting with our very own “google.com” organization cloud projects. We quickly found some projects that were unattended, but remediating this issue was easier said than done due to challenges in several areas:

• Detection: With lots of signals available to you via sources like Cloud Monitoring, what are the right ones you should look at (e.g. API, networking, user activity)? How can you tell the difference between an unattended project and a project that has a low level of activity by design (e.g. a “shell” project that holds an auth token)?
• Remediation: Once you have identified a project that seems abandoned, how do you go about ensuring that it’s indeed an unattended project? How do you reduce the risk of deleting something that might be essential to a production workload, causing irreversible data loss? How do you solve this at the scale of your entire organization, beyond a one-time cleanup? 

Over the course of 2021 we built and tested a Google-internal prototype first, cleaning up many of our internal unattended projects, and then worked with a number of Google Cloud customers to build and tune this feature based on real-life data (thank you to all of our early adopters for working with us and your generous feedback that helped us shape this feature!) It was not uncommon for us to come across organizations with thousands of unattended projects, and we’re very excited to bring Unattended Project Recommender to all customers, in public preview.

Discovering and acting on unattended project recommendations

Unattended Project Recommender analyzes usage activity across all projects under your organization, including the following data:

• API activity (e.g. service accounts with authentication activity, API calls consumed)
• Networking activity (ingress and egress)
• Billing activity (e.g. services with billable usage)
• User activity (e.g. active project owners)
• Cloud services usage (e.g. number of active VMs, BigQuery jobs, storage requests)

Based on these signals, it can generate recommendations to clean up projects that have low usage activity (where “low usage” is defined using a machine learning model that ranks projects in your organization by level of usage), or recommendations to reclaim projects that have high usage activity but no active project owners. Here’s what an example post-processed summary list of recommendations can look like for the “foobar” organization that has 3 projects:

  Project ID: demo-project-307815
Recommendation: CLEANUP_PROJECT

Project ID: new-project
Recommendation: N/A

Project ID: bobs-playground-project
Recommendation: RECLAIM_PROJECT

In addition to the recommendations, you can also examine the underlying project activity insights that the recommendations are based upon. The insights provide additional information that can be useful for integration with your organization’s existing workflows and automation (e.g. send an auto-generated email or chat message to project owners based on the list provided by the owners field). Here’s an example insight payload:

  content:
  activeAppengineInstanceDailyCount: 0
  activeCloudsqlInstanceDailyCount: 0
  activeGceInstanceDailyCount: 3
  activeServiceAccountDailyCount: 1
  apiClientDailyCount: 18922           // Daily average API calls produced
  bigqueryInflightJobDailyCount: 0
  bigqueryInflightQueryDailyCount: 0
  bigqueryStorageDailyBytes: 0
  bigqueryTableDailyCount: 0
  consumedApiDailyCount: 0             // Daily average API calls consumed
  datastoreApiDailyCount: 0
  gcsObjectDailyCount: 11
  gcsRequestDailyCount: 0
  gcsStorageDailyBytes: 2663548
  hasActiveOauthTokens: false          // OAuth tokens used in the last 180 days
  hasBillingAccount: true
  numActiveUserOwners: 1
  owners:                              // List of project owners
  - activeOnProject: false
    member: user:user1@example.com
  - activeOnProject: true
    member: user:user2@example.com
  serviceWithBillableUsage:
  – Cloud Storage
  - Compute Engine
  vpcEgressDailyBytes: 264456938       // Daily average VPC egress bytes
  vpcIngressDailyBytes: 392435047      // Daily average VPC ingress bytes
  usagePercentile: 20                  // Level of usage relative to other projects

GCP projects are used in many different ways and for many different purposes. In case you get a recommendation to delete a project that’s being used in a way that’s out of the scope for this feature, you can dismiss the recommendation and it will stop showing up for the given project. 

Restoring deleted projects

When you choose to shut down a project using the projects.delete() method, it gets marked for deletion. After a project is marked for deletion, it becomes unusable, all resources within that project are shut down, and a 30-day wait period for the project and all of its data to get fully deleted begins.

In case a useful project is accidentally shut down, you have the option to restore the project within that 30-day wait period. Since restoring allows you to recover most but not necessarily all of your project data and resources, we recommend carefully examining the utilization insights associated with a project and considering any additional utilization signals that may not be captured by the Unattended Project Recommender before taking the cleanup action.

Early customer success stories

A number of enterprise customers are already using Unattended Project Recommender to keep their organizations clean of unattended projects and resources.

Decathlon, a French sporting goods retailer, is excited for the insight Unattended Project Recommender will bring to their environment, and are already deploying it as a part of their latest cloud security initiatives.

“After a thorough test of this feature and the validation of our CISO, we ended up deleting our first 775 projects, and no one complained! A great help to improve our security. The next step for us will be to operationalize it at scale, and implement a company wide policy for unattended resource management.” —Adeline Villette, Cloud Security Officer

For Veolia, one of the world’s largest water, waste and energy management companies, not only does this feature reduce security risks and waste, but also helps drive cultural shift and alignment with its ecological transformation strategy.

“This feature allows us to reduce our costs and security debt on assets that are no longer in use, and is also fully in line with Veolia’s philosophy of limiting its carbon footprint. After having tested Unattended Project Recommender on more than 3,000 projects throughout our organization, we are looking to bring it as proactive alerts to our project owners at scale.”—Thomas Meriadec, Product Manager

Box, a secure cloud content management provider, views it as a foundation for building a repeatable process to remediate unused resources.

“Unattended Project Recommender is a great fit for us. It gives us a unified view of project usage across our entire organization and enables us to address security risks of legacy projects in a systematic and organized manner, ensuring an even safer environment.” —Matt Bowes, Staff Security Engineer

Getting started with the Unattended Project Recommender

To help you get started, we’ve prepared a Cloud Shell tutorial (source code) that you can use to find unattended project recommendations within your own Projects/Folders/Organization. Click this button to clone the tutorial from GitHub and run in your Cloud Shell environment:

As you can see, listing recommendations for your projects only takes a few clicks with the tutorial (special thanks to Lanre Ogunmola, Security & Compliance Specialist, for making this look so easy)! For additional detail on using the gcloud CLI or API to discover unattended project recommendations, please refer to the documentation page.

You can also automatically export all recommendations from your Organization to BigQuery and then investigate the recommendations with DataStudio or Looker, or use Connected Sheets that let you use Google Workspace Sheets to interact with the data stored in BigQuery without having to write SQL queries.

As with any other Recommender, you can choose to opt out of data processing at any time by disabling the appropriate data groups in the Transparency & control tab under Privacy & Security settings.

We hope that you can leverage Unattended Project Recommender to improve your cloud security posture and reduce cost, and can’t wait to hear your feedback and thoughts about this feature! Please feel free to reach us at active-assist-feedback@google.com and we also invite you to sign up for our Active Assist Trusted Tester Group if you would like to get early access to the newest features as they are developed.