A Look Back on Google Cloud’s Data Analytics Development Efforts from June

5965
Of your peers have already read this article.
4:00 Minutes
The most insightful time you'll spend today!
June is the month that holds the summer solstice, and some of us in the northern hemisphere get to enjoy the longest days of sunshine out of the entire year. We used all the hours we could in June to deliver a flurry of new features across BigQuery, Dataflow, Data Fusion, and more. Let’s take a look!
Simple, Sophisticated, and Secure
Usability is a key tenant of our data analytics development efforts. Our new user-friendly BigQuery improvements this month include:
- Flexible data type casting
- Formatting to change column descriptions
- GRANT/REVOKE access control commands using SQL
We hope this will delight data analysts, data scientists, DBAs, and SQL-enthusiasts who can find out more details in our blog here.
Beyond simplifying commands, we also recognize that it’s equally important to have more sophistication when dealing with transactions. That’s why we introduced multi-statement transactions in BigQuery.
As you probably know, BigQuery has long supported single-statement transactions through DML statements, such as INSERT, UPDATE, DELETE, MERGE and TRUNCATE, applied to one table per transaction. With multi-statement transactions, you can now use multiple SQL statements, including DML, spanning multiple tables in a single transaction.
This means that any data changes across multiple tables associated with all statements in a given transaction are committed atomically (all at once) if successful—or all rolled back atomically in the event of a failure.

We also know that organizations need to control access to data, down to the granular level and that, with the complexity of data platforms increasing day by day, it’s become even more critical to identify and monitor who has access to sensitive data.
To help address these needs, we announced the general availability of BigQuery row-level security. This capability gives customers a way to control access to subsets of data in the same table for different groups of users. Row-level security in BigQuery enables different user personas access to subsets of data in the same table and can easily be created, updated, and dropped using DDL statements. To learn more, check out the documentation and best practices.

Simple, Safe, and Smart
Beyond building a simpler, more sophisticated and more secure data platform for customers, our team has been focused on providing solutions powered by built-in intelligence. One of our core beliefs is that for machine learning to be adopted and useful at scale, it must be easy to use and deploy.
BigQuery ML, our embedded machine learning capabilities, have been adopted by 80% of our top customers around the globe and it has become a cornerstone of their data to value journey.
As part of our efforts, we announced the general availability of AutoML tables in BigQuery ML. This no-code solution lets customers automatically build and deploy state-of-the-art machine learning models on structured data. With easy integration with Vertex AI, AutoML in BQML makes it simple to achieve machine learning magic in the background. From preprocessing data to feature engineering and model tuning all the way to cross validation, AutoML will “automagically” select and ensemble models so everyone—even non-data scientists—can use it.
Want to take this feature for a test drive? Try it today on BigQuery’s NYC Taxi public dataset following the instructions in this blog!
Speaking of public datasets, we also introduced the availability of Google Trends data in BigQuery to enable customers to measure interest in a topic or search term across Google Search. This new dataset will soon be available in Analytics Hub and will be anonymized, indexed, normalized, and aggregated prior to publication.
Want to ensure your end-cap displays are relevant to your local audience? You can take signals from what people are looking for in your market area to inform what items to place. Want to understand what new features could be incorporated into an existing product based on what people are searching for? Terms that appear in these datasets could be an indicator of what you should be paying attention to.
All this data and technology can be put to use to deploy critical solutions to grow and protect your business. For example, it can be difficult to know how to define anomalies during detection. If you have labeled data with known anomalies, then you can choose from a variety of supervised machine learning model types that are already supported in BigQuery ML.
But what if you don’t know what kind of anomaly to expect, and you don’t have labeled data? Unlike typical predictive techniques that leverage supervised learning, organizations may need to be able to detect anomalies in the absence of labeled data.
That’s why, we were particularly excited to announce the public preview of new anomaly detection capabilities in BigQuery ML that leverage unsupervised machine learning to help you detect anomalies without needing labeled data.
Our team has been working with a large number of enterprises who leverage machine learning for better anomaly detection. In financial services for example, customers have used our technology to detect machine-learned anomalies in real-time foreign exchange data.
To make it easier for you to take advantage of their best practices, we teamed up with Kasna to develop sample code, architecture guidance, and a data synthesizer that generates data so you can test these innovations right away.
Simple, Scalable, and Speedy
Capturing, processing and analyzing data in motion has become an important component of our customer architecture choices. Along with batch processing, many of you need the flexibility to stream records into BigQuery so they can become available for query as they are written.
Our new BigQuery Storage Write API combines the functionality of streaming ingestion and batch loading into a single API. You can use it to stream records into BigQuery or even batch process an arbitrarily large number of records and commit them in a single atomic operation.
Flexible systems that can do batch and real-time in the same environment is in our DNA: Dataflow, our serverless, data processing service for streaming and batch data was built with flexibility in mind.
This principle applies not just to what Dataflow does but also how you can leverage it—whether you prefer using Dataflow SQL right from the BigQuery web UI, Vertex AI notebooks from the Dataflow interface, or the vast collection of pre-built templates to develop streaming pipelines.
Dataflow has been in the news quite a bit recently. You might have noted the recent introduction of Dataflow Prime, a new no-ops, auto-tuning functionality that optimizes resource utilization and further simplifies big data processing. You might have also read that Google Dataflow is a Leader in The 2021 Forrester Wave™: Streaming Analytics, giving Dataflow a score of 5 out of 5 across 12 different criteria.
We couldn’t be more excited about the support the community has provided to this platform. The scalability of Dataflow is unparalleled and as you set your company up for more scale, more speed, and “streaming that screams”, we suggest you take a look at what leaders at Sky, RVU or Palo Alto Networks have already accomplished.
If you’re new to Dataflow, you’re in for a treat: this past month, Priyanka Vergadia (AKA CloudGirl) released a great set of resources to get you started. Read her blog here and watch her introduction video below!
https://youtube.com/watch?v=WRspZRG9e90%3Fenablejsapi%3D1%26
Simple structure that sticks together
We thrive to be the partner of choice for your transformation journey, regardless where your data comes from and how you choose to unify your data stack.
Our partners at Tata Consultancy Services (TCS) recently released research that highlights the importance of a unifying digital fabric and how data integration services like Google Cloud Data Fusion can enable their clients to achieve this vision.
We also announced SAP Integration with Cloud Data Fusion, Google Cloud’s native data integration platform, to seamlessly move data out of SAP Business Suite, SAP ERP and S4/HANA. To date, we provide more than 50 pipelines in Cloud Data Fusion to rapidly onboard SAP data.
This past month, we introduced our SAP Accelerator for Order to Cash. This accelerator is a sample implementation of the SAP Table Batch Source feature in Cloud Data Fusion and will help you get started with your end-to-end order to cash process and analytics.
It includes sample Cloud Data Fusion pipelines that you can configure to connect to your SAP data source, perform transformations, store data in BigQuery, and set up analytics in Looker. It also comes with LookML dashboards which you can access on Github.
Countless great organizations have chosen to work with Google for their SAP data. In June, we wrote about ATB Financial’s journey and how the company uses data to better serve over 800,000 customers, save over CA$2.24 million in productivity, and realize more than CA$4 million in operating revenue through “D.E.E.P”, a data exposure enablement platform built around BigQuery.
Finally, if you are an application developer looking for a unified platform that brings together data from Firebase Crashlytics, Google Analytics, Cloud Firestore, and third party datasets, we have good news!
This past month, we released a unified analytics platform that combines Firebase, BigQuery, Google Looker and FiveTran to easily integrate disparate data sources, and infuse data into operational workflows for greater product development insights and increased customer experience. This resource comes with sample code, a reference guide and a great blog! We hope you enjoy it. See you all next month!
https://youtube.com/watch?v=L25Vfzr2Ciw%3Fenablejsapi%3D1%26
Secrets Management Risk Analysis: Mitigating Threats with Secret Manager

917
Of your peers have already read this article.
2:30 Minutes
The most insightful time you'll spend today!
Can you keep a secret? Many people struggle to keep secrets. So do organizations. An organization’s digital credentials — its passwords, API keys, tokens, and encryption keys baked into its software code — are its vital secrets, used to authenticate access to protected resources and services.
The security of these secrets represent one of the biggest risks that enterprises must manage when securing application infrastructure and environments. Fortunately, we have an important tool to help our customers better manage their secrets: Google Cloud Secret Manager.
Secret Manager is a secure and efficient tool to centrally store, access, manage, and audit your organization’s secrets. As we noted in our April 2023 Threat Horizons Report, insufficient management of credentials and keys is now one of the top security issues facing cloud customers today. In particular, we have observed service account keys being used for everything from insider threats to cross-cloud breaches to intellectual property espionage.
Along with the increasing popularity of DevOps, secrets are used throughout development. In some cases, secrets are coded as plaintext into configuration files or directly into functions, where they can end up exposed in code repositories. If compromised, secrets can be used to gain unauthorized privileged access to enterprise data, infiltrate critical systems, and launch attacks.
Secret Manager can help organizations better manage their secrets even as concerns over potential security breaches grow. Organizations must consider the following four points during their secrets management risk analysis:
- Secrets sprawl: As organizations move more workloads to the cloud, the number of secrets they need to manage increases exponentially. This can lead to secrets sprawl, where secrets are stored in a variety of different locations, making it difficult to track and manage them.
- Human error: Humans are fallible, and this is especially true when it comes to managing secrets. Mistakes, such as accidentally sharing a secret with the wrong person or the misconfiguration of a secret management tool, can have serious consequences.
- Compliance violations: Organizations that handle sensitive data are subject to a variety of compliance regulations, including PCI-DSS, SOC 2, HIPAA, and GDPR. Without proper management, secrets could lead to potential compliance violations.
- Downtime: If secrets are not properly managed or rotated, it could lead to downtime for applications and services that rely on them.
Secret Manager can enable IT teams to keep track of secrets by providing a centralized place to store and manage them. You can access secrets directly from the Secret Manager console, or use the Secret Manager API or SDKs. Secret Manager is also designed to scale with your application. You can store as many secrets as you need without performance issues, and its API-first design lets you extend and integrate into other existing systems.
Secret Manager integrations in Google Cloud
In many cases, developers save secret configuration details as plaintext in code for ease of use. There are two main risks associated with this approach:
- Anyone with access to secrets can see and copy them. There is no way to enforce any auditing or access controls practices here.
- It becomes difficult to move code between dev, test, and production environments.
Cloud-centered integrations supported by Secret Manager with other Google Cloud services in the software supply chain make it easier and safer to store and access sensitive information.
The tool is about the secret information being made securely available in build or runtime.
Secret Manager integrations also can play a role in securing sensitive information in the software supply chain:
1. Integration with CI/CD systems used to build and deploy your software
Cloud Code integration can help developers build more secure applications by preventing hard-coding of sensitive configuration data such as passwords, credentials, and tokens in the codebase. It replaces them with secrets which can be programmatically fetched as required. You can create, access, and modify secrets directly from within your favorite IDE without having to navigate away from it. Integration is available for VS Code, IntelliJ, and Cloud Shell Editor.
Cloud Functions integration lets you access and expose secrets as environment variables or as a volume via the filesystem.
Cloud Build: To handle sensitive data in builds, you can store them in Secret Manager and then configure the build steps to access the information directly. In the build steps, you can associate the value of a secret to an environment and you can access this value via an environment variable from scripts or processes. Two common uses for fetching sensitive dependencies during build time are:
- Storing Docker authentication credentials for pull/push of images to Docker hub.
- Creating GitHub pull requests in response to builds.
2. Integration with runtime engines
Cloud Run: Native integration makes it easy to mount secrets in cloud run services. Using this integration, secrets can be made available to containers either as volume mounts or as env variables. Sensitive dependencies can be stored in environment variables, by referencing a secret name stored in Secret Manager instead of storing it as a plain text environment variable. This secret access operation can be done directly from the Cloud Run console without changing a single line of source code.
If code reads secrets from a file instead of an env variable, the “mounted as volume” option can be used to make secrets look like files to the env vars. During service deployment, the service account used to run the containers are checked to see if they have the required permissions or not, preventing any unwanted access.
Kubernetes and GKE: The CSI driver–provider solution is an open-source work that provides integration with K8s. The driver lets customers centrally store their secrets in Google Secret Manager and consume them in a kubernetes native manner, as files mounted in K8s pods. The solution also comes with additional syncing capabilities. This method of consuming secrets also allows users to take advantage of SM’s capabilities like versioning, managed rotation, default/CMEK encryption and cloud-focused integrations with other Google Cloud services. Work is currently ongoing to provide official support to the driver solution as a GKE component.
Is Secret Manager right for my organization?
Developers carry a big responsibility to protect the sensitive data they are entrusted to work with, and keep secrets out of their code. However, organizations also have a responsibility to provide the tools and solutions needed to help their teams deliver on these expectations. Secret Manager makes it straightforward for developers to store and manage secrets while improving security.
To learn more about Secret Manager, please review these resources:
Read the previous Best Kept Security Secrets blogs here:
Five Ways Cloud Computing Helps Companies Optimize Operations and Lower IT Costs

4802
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
There is no one way to recover. The past year has seen unprecedented challenges for business across the world, with social distancing and quarantine measures forcing many organizations to quickly adapt to remote working. A new report from BCG Platinion, titled “Finding a New Normal in the Cloud”, points out that while companies have done well to survive so far, the overall business environment is still a tough one with contracted economies and lowered revenues. CIOs have to find the right balance between technological innovation and lowering their cash burn rate.
The report identifies five key ways in which companies can use cloud computing to optimize their operations and reduce overall IT costs by as much as 10%.
1. Go beyond VPN
The BCG Platinion report states that companies need “rapid, efficient, highly scalable, and device agnostic solutions”. Traditionally, when employees had to work offsite, companies provided them with VPNs. But the sheer scale of the COVID-19 pandemic has shown these solutions to be expensive, slow, inconvenient, and hard to manage when entire workforces are working remotely.
Instead, BCG Platinion emphasizes the need for cloud-based solutions such as BeyondCorp Enterprise for more secure, more effective, lower cost remote access at scale. BeyondCorp Enterprise offers customers a zero trust platform for simple and secure access with continuous end-to-end protection that can be used on any device at any time. Deliveroo, a global food delivery company headquartered in the UK, uses BeyondCorp Enterprise to bring the zero trust model to its distributed workforce.
“Having secure access to applications and associated data is critical for our business,” says Vaughn Washington, VP of Engineering at Deliveroo. “With BeyondCorp Enterprise, we manage security at the app level, which removes the need for traditional VPNs and associated risks.”1
With a low cost cloud subscription model, BeyondCorp Enterprise eliminates the need for hardware, operating, and maintenance costs that come with VPN solutions and can also enable organizations to offer protection to the extended workforce at a fraction of the cost. BCG Platinion estimates that solutions like BeyondCorp Enterprise can save companies as much as 50% versus the cost of a traditional VPN.
2. Use SaaS to keep productivity up
With so much of the workforce fragmented due to social distancing measures, the need for seamless, efficient collaboration is greater than ever. According to BCG Platinion, a fully functional Software-as-a-Service productivity solution such as Google Workspace helps to “alleviate the traditional costs and burdens around availability, backup, and maintenance of on-premise collaboration infrastructure.” BCG Platinion analysts estimate that adopting a SaaS solution can lower computing costs for end users by up to 35%.
But working in the cloud does more than lower costs. The BCG Platinion report cites a 2020 study from Forrester that found adopting Google Workspace boosted revenue growth by 1.5%, reduced the need for on-demand tech support by 20%, and cut the risk of data breaches by more than 95%. Over the last year, companies of all sizes have looked on the conditions of the pandemic as an opportunity to change the way they work.
“Airbus has spent the past year thinking about what it actually means to return to work and we’re looking to support greater flexibility with Google Workspace in a leading role,” says Andrew Plunkett, Airbus Vice President Digital Workplace. “In 2020, we held 5.6M Google Meet sessions and we now have more than 70,000 shared Drives where people collaborate. Google Workspace has changed the way people work at Airbus and that will continue as the solution empowers the hybrid work reality.”2
3. Reduce IT overhead and management costs with cloud-first devices
Working in the cloud can be made even more effective with devices specifically designed for the cloud, says the BCG Platinion report. “Cloud-native devices such as Google’s Chromebooks and Chromeboxes are cost-effective, easy to deploy, simple to use, and highly secure,” says BCG Platinion. Additionally, with “thin client” devices, companies can save on hardware costs compared with traditional laptops and desktops.The report suggests that a thin client approach can produce savings of up to 25% in end-user technology and support.
Organizations and businesses of all sizes have found that Chrome OS and devices have greatly enhanced their capacity to work together in even the most challenging circumstances. Chrome OS provides employees with a modern experience and devices that stay fast, have built-in security, deploy quickly, and reduce the total cost of ownership. “Chromebooks are simple-to-use and cost-effective devices that do everything that our staff need them to do, which is mainly accessing Google Workspace online,” says Henry Lewis, Head of Platform for the London Borough of Hackney. “As soon as the Grab and Go Chromebooks were available, they were well used every day.”3
4. Lift and shift for easier transitions
While migrating to the cloud is a priority for many organizations to succeed, it does not have to happen all at once and in the same way. A total cloud migration can be a huge undertaking, involving redesigns of architecture and refactoring of applications. For many organizations, this process can take several months, even years.
But in times like these, CIOs need to make decisions quickly and reduce cash burn as much as possible. When resources are stretched and time is tight, BCG Platinion recommends a “lift-and-shift” approach for minimal disruption. With Google Compute Engine, for example, organizations can simply rehost their existing workloads on virtual machines without transformation. BCG Platinion reports that moving non-critical workloads to the cloud with lift-and-shift approaches can reduce IT spend by as much as 4% in just three months. Additionally, a quick, effective migration paves the way for more advanced IT infrastructure changes, creating effects that ripple long into the future.
5. Make data work for you with the cloud
Data is central to any industry and making the most out of it is more important now than ever before. With business as usual upended by the pandemic, organizations have turned to data for urgent tasks like demand forecasting or predicting supply-chain disruptions. A McKinsey report from last year points out that while many organisations had already started to engage with analytics and AI technologies, their progress was dramatically accelerated by the urgency of the pandemic: “Analytics capabilities that once might have taken these organizations months or years to build came to life in a matter of weeks.”
With the right setup a company can use data to drive efficiencies, respond quickly to its customers and open new markets. But big wins require huge amounts of information and powerful analytics, which means that effective data handling can be very difficult and expensive to do with on-premises architecture.
Moving to a cloud-based infrastructure minimizes infrastructure costs while opening up cutting edge techniques like machine learning for greater insight. The BCG Platinion report found that using a cloud-based data platform was not only cheaper and more efficient for businesses, but could result in a 70% increase in “effectiveness” which it defined as increased sales, lower costs of procured goods, and reduced inventory holding costs. “Moving to cloud provides competitive advantage as you scale innovation, accelerating the creation of new services to keep ahead of the competition,” explains Norbert Faure, Managing Director, Platinion Western Europe at Boston Consulting Group (BCG).
The key mission of any data platform is to make sure that the right people have access to the right information at the right time. Google Cloud helps to unify data across the entire business, increase agility, and innovate faster with a range of products. BigQuery runs complex analytics at infinite scale while helping organizations save up to 34% on the total cost of ownership compared with alternative cloud-based data warehouse solutions.4 Cloud Spanner provides a fully managed relational database that operates at 99.999% availability. Meanwhile, with Vertex AI, businesses can access the same groundbreaking machine learning tools that Google uses itself for unprecedented insight on a unified AI platform. “The whole Google Cloud suite of products accelerates the process of getting established and up and running, so we can perform the ‘bread-and-butter’ work of data science,” says David Herberich, VP, Head of Data at fintech startup Branch.5
Minimize costs today, innovate for tomorrow
As the world recovers from the pandemic, continued success depends on staying nimble and adaptive, argues the BCG Platinion report. Cloud computing can make companies more resilient by helping them to keep costs low, reducing overall IT spend by as much as 10% overall according to BCG Platinion. “For CIOs, cloud migrations can offer important near-term savings and benefits, while also reinvigorating progress toward the long-term goal of digital transformation.“
To learn more, read the full report from BCG Platinion.
1. BeyondCorp Enterprise: Introducing a safer era of computing
2. Building the future of work with Google Workspace
3. Hackney Council: Empowering 4,000 staff to keep serving their community from home
4. The Economic Advantages of Google BigQuery versus Alternative Cloud-based EDW Solutions
5. Fintech startup, Branch makes data analytics easy with BigQuery
End Security Risks with the Unattended Projects Recommender Feature

6126
Of your peers have already read this article.
3:00 Minutes
The most insightful time you'll spend today!
In fast-moving organizations, it’s not uncommon for cloud resources, including entire projects, to occasionally be forgotten about. Not only such unattended resources can be difficult to identify, but they also tend to create a lot of headaches for product teams down the road, including unnecessary waste and security risks.
To help you prune your idle cloud resources, we’re excited to introduce Unattended Project Recommender. It’s a new feature of Active Assist that provides you with a one-stop shop for discovering, reclaiming, and shutting down unattended projects. With actionable and automatic recommendations, you no longer have to worry about wasting money or mitigating security risks presented by your idle resources. Unattended Project Recommender uses machine learning to identify, with a high degree of confidence, projects that are likely abandoned based on API and networking activity, billing, usage of cloud services, and other signals. This feature is available via the Recommender API today, making it easy for you to integrate with your company’s existing workflow management and communication tools, or export results to a BigQuery table for custom analysis.
Thousands of projects can be unattended in large organizations, presenting major security risks
Your cloud projects can go abandoned or unattended for a number of reasons — ranging from a test environment that’s no longer needed, to project cancellation, to project owner switching jobs, and more. Not only can such projects contribute to your cloud bill (waste) but they may contain security issues such as open firewalls or privileged service account keys that attackers can exploit to get a hold of your cloud resources for cryptocurrency mining or, worse, compromise your company’s sensitive data. These security risks tend to grow over time because the latest best practices and patches are usually not applied to unattended projects.
We experience this issue here at Google, too… In fact, it has been on Google’s internal security team’s radar for some time now, so we joined forces and looked into this problem together, starting with our very own “google.com” organization cloud projects. We quickly found some projects that were unattended, but remediating this issue was easier said than done due to challenges in several areas:
- Detection: With lots of signals available to you via sources like Cloud Monitoring, what are the right ones you should look at (e.g. API, networking, user activity)? How can you tell the difference between an unattended project and a project that has a low level of activity by design (e.g. a “shell” project that holds an auth token)?
- Remediation: Once you have identified a project that seems abandoned, how do you go about ensuring that it’s indeed an unattended project? How do you reduce the risk of deleting something that might be essential to a production workload, causing irreversible data loss? How do you solve this at the scale of your entire organization, beyond a one-time cleanup?
Over the course of 2021 we built and tested a Google-internal prototype first, cleaning up many of our internal unattended projects, and then worked with a number of Google Cloud customers to build and tune this feature based on real-life data (thank you to all of our early adopters for working with us and your generous feedback that helped us shape this feature!) It was not uncommon for us to come across organizations with thousands of unattended projects, and we’re very excited to bring Unattended Project Recommender to all customers, in public preview.
Discovering and acting on unattended project recommendations
Unattended Project Recommender analyzes usage activity across all projects under your organization, including the following data:
- API activity (e.g. service accounts with authentication activity, API calls consumed)
- Networking activity (ingress and egress)
- Billing activity (e.g. services with billable usage)
- User activity (e.g. active project owners)
- Cloud services usage (e.g. number of active VMs, BigQuery jobs, storage requests)
Based on these signals, it can generate recommendations to clean up projects that have low usage activity (where “low usage” is defined using a machine learning model that ranks projects in your organization by level of usage), or recommendations to reclaim projects that have high usage activity but no active project owners. Here’s what an example post-processed summary list of recommendations can look like for the “foobar” organization that has 3 projects:
Project ID: demo-project-307815Recommendation: CLEANUP_PROJECTProject ID: new-projectRecommendation: N/AProject ID: bobs-playground-projectRecommendation: RECLAIM_PROJECT
In addition to the recommendations, you can also examine the underlying project activity insights that the recommendations are based upon. The insights provide additional information that can be useful for integration with your organization’s existing workflows and automation (e.g. send an auto-generated email or chat message to project owners based on the list provided by the owners field). Here’s an example insight payload:
content:activeAppengineInstanceDailyCount: 0activeCloudsqlInstanceDailyCount: 0activeGceInstanceDailyCount: 3activeServiceAccountDailyCount: 1apiClientDailyCount: 18922 // Daily average API calls producedbigqueryInflightJobDailyCount: 0bigqueryInflightQueryDailyCount: 0bigqueryStorageDailyBytes: 0bigqueryTableDailyCount: 0consumedApiDailyCount: 0 // Daily average API calls consumeddatastoreApiDailyCount: 0gcsObjectDailyCount: 11gcsRequestDailyCount: 0gcsStorageDailyBytes: 2663548hasActiveOauthTokens: false // OAuth tokens used in the last 180 dayshasBillingAccount: truenumActiveUserOwners: 1owners: // List of project owners- activeOnProject: falsemember: user:user1@example.com- activeOnProject: truemember: user:user2@example.comserviceWithBillableUsage:– Cloud Storage- Compute EnginevpcEgressDailyBytes: 264456938 // Daily average VPC egress bytesvpcIngressDailyBytes: 392435047 // Daily average VPC ingress bytesusagePercentile: 20 // Level of usage relative to other projects
GCP projects are used in many different ways and for many different purposes. In case you get a recommendation to delete a project that’s being used in a way that’s out of the scope for this feature, you can dismiss the recommendation and it will stop showing up for the given project.
Restoring deleted projects
When you choose to shut down a project using the projects.delete() method, it gets marked for deletion. After a project is marked for deletion, it becomes unusable, all resources within that project are shut down, and a 30-day wait period for the project and all of its data to get fully deleted begins.
In case a useful project is accidentally shut down, you have the option to restore the project within that 30-day wait period. Since restoring allows you to recover most but not necessarily all of your project data and resources, we recommend carefully examining the utilization insights associated with a project and considering any additional utilization signals that may not be captured by the Unattended Project Recommender before taking the cleanup action.
Early customer success stories
A number of enterprise customers are already using Unattended Project Recommender to keep their organizations clean of unattended projects and resources.
Decathlon, a French sporting goods retailer, is excited for the insight Unattended Project Recommender will bring to their environment, and are already deploying it as a part of their latest cloud security initiatives.
“After a thorough test of this feature and the validation of our CISO, we ended up deleting our first 775 projects, and no one complained! A great help to improve our security. The next step for us will be to operationalize it at scale, and implement a company wide policy for unattended resource management.” —Adeline Villette, Cloud Security Officer
For Veolia, one of the world’s largest water, waste and energy management companies, not only does this feature reduce security risks and waste, but also helps drive cultural shift and alignment with its ecological transformation strategy.
“This feature allows us to reduce our costs and security debt on assets that are no longer in use, and is also fully in line with Veolia’s philosophy of limiting its carbon footprint. After having tested Unattended Project Recommender on more than 3,000 projects throughout our organization, we are looking to bring it as proactive alerts to our project owners at scale.”—Thomas Meriadec, Product Manager
Box, a secure cloud content management provider, views it as a foundation for building a repeatable process to remediate unused resources.
“Unattended Project Recommender is a great fit for us. It gives us a unified view of project usage across our entire organization and enables us to address security risks of legacy projects in a systematic and organized manner, ensuring an even safer environment.” —Matt Bowes, Staff Security Engineer
Getting started with the Unattended Project Recommender
To help you get started, we’ve prepared a Cloud Shell tutorial (source code) that you can use to find unattended project recommendations within your own Projects/Folders/Organization. Click this button to clone the tutorial from GitHub and run in your Cloud Shell environment:

As you can see, listing recommendations for your projects only takes a few clicks with the tutorial (special thanks to Lanre Ogunmola, Security & Compliance Specialist, for making this look so easy)! For additional detail on using the gcloud CLI or API to discover unattended project recommendations, please refer to the documentation page.
You can also automatically export all recommendations from your Organization to BigQuery and then investigate the recommendations with DataStudio or Looker, or use Connected Sheets that let you use Google Workspace Sheets to interact with the data stored in BigQuery without having to write SQL queries.
As with any other Recommender, you can choose to opt out of data processing at any time by disabling the appropriate data groups in the Transparency & control tab under Privacy & Security settings.
We hope that you can leverage Unattended Project Recommender to improve your cloud security posture and reduce cost, and can’t wait to hear your feedback and thoughts about this feature! Please feel free to reach us at active-assist-feedback@google.com and we also invite you to sign up for our Active Assist Trusted Tester Group if you would like to get early access to the newest features as they are developed.
Google Cloud Leads the Landscape for Unstructured Data Security Platform: Forrester

7049
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
As organizations expand their use of cloud computing services, more of their sensitive data inevitably moves to and lives in the cloud. Much of this sensitive data is unstructured and can be challenging to secure. Despite this potential challenge, the usefulness of cloud for data storage and processing is too big for most organizations to ignore and has in turn led to data sprawl, where their sensitive data is spread over many resources, both in the cloud and on-premise. Addressing data sprawl requires solutions that can discover, manage, and secure sensitive data, especially unstructured data, as it spreads.
To help organizations confidently move their sensitive data to the cloud, Google Cloud works diligently to earn and maintain customer trust. Control and transparency are pillars of our approach to offering a trusted cloud. Therefore, we’ve been expanding our capabilities to act on unstructured data as sprawl increases.
Given the importance of these capabilities to our strategy, we are happy to announce today that Forrester Research has named Google Cloud a Leader in The Forrester Wave™: Unstructured Data Security Platforms, Q2 2021 report, and rated Google Cloud highest in the current offering category among the providers evaluated.

The report evaluates the 11 most significant providers with platform solutions to secure and protect unstructured data, spanning from cloud providers to data security-focused vendors. The report notes that “Google offers breadth and depth with built-in data security in the cloud. Google Cloud Platform, Google Workspace, and BeyondCorp Enterprise have underlying data security products and features for protecting customer data.”
Google Cloud tools focused on protecting unstructured data were developed and battle-tested internally at Google to alleviate some of our own data security challenges. This brings the best of Google security to the organizations utilizing Google Cloud and our security tools. The report highlights that “Google productizes capabilities originally developed to secure its own business, and brings a disciplined approach to product enhancements for enterprise requirements. It serves a wide range of enterprise and mid-market, with a focus on emphasizing data protection needs by industry. ”
Google Cloud’s data security strategy focuses on meeting customers wherever they are in their cloud migration journey. The report highlights that “Google further enables a Zero Trust approach with third-party integrations through its BeyondCorp Alliance of partners in device management, endpoint security and gateways.”
Google Cloud received the highest possible score in sixteen criteria, in total receiving the most 5 out of 5 ratings among all vendors assessed. These criteria include: Data Intelligence, Access Control, Deletion, Obfuscation-Scope, Obfuscation-Key Management, Deployment, Security and Risk, APIs and Integration, Data Security Platform Vision, Data Security Execution Roadmap, Performance, Planned Enhancements, Zero Trust Enabling Partner Ecosystem, Diversity, Equity and Inclusion, Installed Base, and Revenue.
Notably, Google Cloud received the highest possible score in the Obfuscation criteria. Obfuscation can help protect sensitive data, like personally identifiable information (PII), which is critical to many enterprise workflows. Cloud DLP helps customers inspect and mask this sensitive data with techniques like redaction, bucketing, and tokenization, which help strike the balance between risk and utility. This is especially crucial when dealing with unstructured or free-text workloads, in which it can be challenging to know what data to redact. More than 150 detectors combine to power Cloud DLP’s masking, which can be deployed in data migrations and business workloads like real-time data collection and processing. For Obfuscation specifically, the report mentioned that Google “takes a broad view of DLP, which includes in-line redaction of sensitive elements in unstructured data and DLP APIs that extend support to additional data types like images or other media.”
We are honored to be a Leader in The Forrester Wave™ Unstructured Data Security Platforms Q2 2021 report, and look forward to continuing to innovate and partner with you on ways to make your digital transformation journey safer as we work to become your most trusted Cloud.
A copy of the full report can be viewed here.
How Google Meet Keeps Your Video Conferences Secure

6811
Of your peers have already read this article.
4:30 Minutes
The most insightful time you'll spend today!
All over the world, businesses, schools and users depend on G Suite to help them stay connected and get work done. Google designs, builds, and operates our products on a secure foundation, aimed at thwarting attacks and providing the protections needed to keep you safe. G Suite and Google Meet are no exception.
Google Meet’s security controls are turned on by default, so that in most cases, organizations and users won’t have to do a thing to ensure the right protections are in place. Here, we’ll summarize the key capabilities of Google Meet that help protect you.
Proactive protections to combat abuse and block hijacking attempts
Google Meet employs an array of counter-abuse protections to keep your meetings safe. These include anti-hijacking measures for both web meetings and dial-ins.
Google Meet makes it difficult to programatically brute force meeting IDs (this is when a malicious individual attempts to guess the ID of a meeting and make an unauthorized attempt to join it) by using codes that are 10 characters long, with 25 characters in the set. We limit the ability of external participants to join a meeting more than 15 minutes in advance, reducing the window in which a brute force attack can even be attempted. External participants cannot join meetings unless they’re on the calendar invite or have been invited by in-domain participants. Otherwise, they must request to join the meeting, and their request must be accepted by a member of the host organization.
In addition, we’re rolling out several features to help schools keep meetings safe and improve the remote learning experiences for teachers and students, including:
- Only meeting creators and calendar owners can mute or remove other participants. This ensures that instructors can’t be removed or muted by student participants.
- Only meeting creators and calendar owners can approve requests to join made by external participants. This means that students can’t allow external participants to join via video, and that external participants can’t join before the instructor.
- Meeting participants can’t rejoin nicknamed meetings once the final participant has left. This means if the instructor is the last person to leave a nicknamed meeting, students can’t join later without the instructor present.

Secure deployment and access controls for admins and end-users
To limit the attack surface and eliminate the need to push out frequent security patches, Google Meet works entirely in your browser. This means we do not require or ask for any plugins or software to be installed if you use Chrome, Firefox, Safari, or Microsoft Edge. On mobile, we recommend that you install the Google Meet app.
To help ensure that only authorized users administer and access Meet services, we support multiple 2-Step Verification options for accounts that are secure and convenient. These include hardware and phone-based security keys and Google prompt. Additionally, Google Meet users can enroll their account in our Advanced Protection Program (APP), which provides our strongest protections available against phishing and account hijacking and is specifically designed for the highest-risk accounts.
For G Suite Enterprise and G Suite for Education customers, we offer Access Transparency, which logs any Google access to Google Meet recordings stored in Drive, along with the reason for the access (support team actions that you might have requested, for example). Customers can also use data regions functionality to store select/covered data of Google Meet recordings in specific regions (i.e. US or Europe).
Secure, compliant, and reliable meeting infrastructure
In Google Meet, all data is encrypted in transit by default between the client and Google for video meetings on a web browser, on the Android and iOS apps, and in meeting rooms with Google meeting room hardware. Meet adheres to IETF security standards for Datagram Transport Layer Security (DTLS) and Secure Real-time Transport Protocol (SRTP). For every person and for every meeting, Meet generates a unique encryption key, which only lives as long as the meeting, is never stored to disk, and is transmitted in an encrypted and secured RPC (remote procedure call) during the meeting setup.
Security is an integral part of all our operations at Google. Our team of full-time security and privacy professionals supports our software engineering and operations to ensure that security is always a part of how we build and run our services. All of our Google Cloud and G Suite customers benefit from these capabilities, including:
- Secure-by-design infrastructure: Google Meet benefits from Google Cloud’s defense-in-depth approach to security, which utilizes the built-in protections and global-private network that Google uses to secure your information and safeguard your privacy.
- Compliance certifications: Our Google Cloud products, including Google Meet, regularly undergo independent verification of their security, privacy, and compliance controls, including validation against standards such as SOC, ISO/IEC 27001/17/18, HITRUST, and FedRAMP. We support your compliance requirements around regulations such as GDPR and HIPAA, as well as COPPA and FERPA for education.
- Incident management: We have a rigorous process for managing data and security incidents that specifies actions, escalations, mitigation, resolution, and notification of any potential incidents impacting customer data.
- Reliability: Google’s network is engineered to accommodate peak demand and handle future growth. Our network is resilient and engineered to accommodate the increased activity we’ve seen on Google Meet.
- Transparency: At Google Cloud, we’re clear about our commitments regarding customer data: we process customer data according to your instructions; we never use customer data for advertising purposes; and we publish the locations of our Google data centers, which are highly available, resilient, and secure.
During COVID-19 and beyond, we will continue to protect Google Meet users and their data, and keep innovating with new features to make our tools helpful, secure, and safe.
More Relevant Stories for Your Company
Google Cloud Leads the Pack Among Data Security Vendors: Forrester
In its 25-criterion evaluation of data security portfolio providers, Forrester identified the 13 most significant ones and researched, analyzed, and scored them. Here's a summary of what Forrester said about Google Cloud Google puts cloud and cloud security at the center of its strategy. Capabilities from Google Cloud Platform, G Suite,
The Rise of Cloud-Based Security Analytics and Operations Technologies
Security analytics and operations can be complex, requiring highly skilled professionals and detailed processes. To overcome these issues, security teams tend to deploy an array of security analytics tools and technologies to collect, process, analyze, and act upon growing volumes of security telemetry. Despite this investment, however, many organizations continue

What Users and System Designers Must Learn from Google’s Best Practices for Password Management
Updated for 2021: This post includes updated best practices including the latest from Google's Best Practices for Password Management whitepapers for both users and system designers. Account management, authentication and password management can be tricky. Often, account management is a dark corner that isn't a top priority for developers or product managers. The resulting experience

How Air Asia’s CIO Ensures Its 22,000 Employees Service Customers Efficiently and Preserve Security
At AirAsia, we operate a fleet of more than 270 aircraft across 23 markets, fly to more than 150 destinations and carry 100m guests each year. We’ve also been named the world’s best low-cost carrier for 11 years running. To accomplish all of this, we rely heavily on our 22,000 Allstars (employees).








