Data security is an ongoing concern for anyone managing a data warehouse. Organizations need to control access to data, down to the granular level, for secure access to data both internally and externally. With the complexity of data platforms increasing day by day, it’s become even more critical to identify and monitor access to sensitive data. In many cases, sensitive data is co-mingled with non-sensitive data, and access restrictions to sensitive data need to be enabled based on factors like data location or presence of financial information. There may also be nuances where data is sensitive for some groups of users, while for others, it is not. 

Today, we’re pleased to announce the general availability of BigQuery row-level security, which gives customers a way to control access to subsets of data in the same table for different groups of users. Row-level security (RLS) extends the principle of least privilege access and enables fine-grained access control policies in BigQuery tables. BigQuery currently supports access controls at the project-, dataset-, table- and column-level. Adding RLS to the portfolio of access controls now enables customers to filter and define access to specific rows in a table based on qualifying user conditions—providing much needed peace of mind for data professionals. 

“Our digital transformation and migration of data to the cloud magnifies the business value we can extract from our information assets. However, granular data access control is essential to comply with international regulatory and contractual requirements. BigQuery row-level security helps us comply with data residency and export restrictions,” says Jarrett Garcia, Iron Mountain’s Enterprise Data Platform Senior Director. “It enables us to manage fine-grained access controls without replicating data. What used to take months for approval and access provisioning can now be done more efficiently and effectively. We are looking forward to implementing additional data security capabilities on the BigQuery roadmap to address other critical business use cases.”

How BigQuery row-level security works

Row-level security in BigQuery enables different user personas access to subsets of data in the same table. Customers who are currently using authorized views to enable these use cases can leverage RLS for ease of management. To express the concept of RLS, we have introduced a new entity in BigQuery called row access policy. Row access policies map a group of user principals to the rows that they can see, defined by a SQL filter predicate. 

Secure logic rules created by data owners and administrators determines which user can see which rows through the creation of a row-level access policy. The row-level access policies created on a target table by administrators or data owners are applied when a query is run on the table. One table can have multiple policies applied to it.

Below is an example, where row-level access policies have been created to filter data based on users’ “region”.

In the illustrated scenario above, row-level access policies have been created to verify a querying user’s region and to give them access only to the subset of data relevant to that region. Access policies are granted to a grantee list which support all types of IAM principles such as individual users, groups, domains or service accounts. In this example, when a user queries the table, row-level access policies are evaluated to assess which, if any, policies are applicable to that user. The group ‘sales-apac’ is granted access to view a subset of rows where region = ‘APAC’ whereas the group ‘sales-us’ is granted access to view a subset of rows where the region = ’US’. Likewise, users in both groups will see rows in both regions, and users in neither group will not see any rows.

Row-level access policies can also be created using the SESSION_USER() function to restrict access only to rows that belong to the user running the query. If none of the row access policies are applicable to the querying user, the user will have no access to the data in the table.

When a user queries a table with a row-level access policy, BigQuery displays a banner notice indicating that their results may be filtered by a row-level access policy. This notice displays even if the user is a member of the `grantee_list`.

When to put BigQuery row-level security to work

Row-level access policies are useful when you have a need to limit access to data based on filter conditions. The row-access policies’ filter predicate supports arbitrary SQL, and is conceptually similar to the WHERE clause of a SQL query. Filter predicates support the SESSION_USER() function to restrict access only to rows that belong to the user running the query. If none of the row access policies are applicable to the querying user, the user will have no access to the data in the table. Currently, the column used for filtering must be in the table, but we anticipate adding support for subqueries in the filter expression, opening up access to use cases where data is filtered based on lookup tables and calculated values. Row-level access policies can be created, updated and dropped using DDL statements. You will be able to see the list of row-level access policies applied to a table using the BigQuery schema pane in the Cloud Console,  which simplifies the management of policies per table, or by using the bq command-line tool.

Row-level security is compatible with other BigQuery security features, and can be used along with column-level security for further granularity.  Since row-level access policies are applied on the source tables, any actions performed on the table will inherit the table’s associated access policies, to ensure access to secure data is protected. Row-level access policies are applicable to every method used to access BigQuery data (API, Views, etc). 

Try it out

We’re always working to enhance BigQuery’s (and Google Cloud’s) data governance capabilities, to provide more controls around managing your data. With row-level security, we are adding deeper protections for your data. You can learn more about BigQuery row-level security in our documentation and best practices.

While cloud security skeptics might believe that data in the cloud is just one access configuration mistake away from a breach, the reality is that a well-designed set of defense in depth controls can help minimize the risk of configuration mistakes and other security issues. Our Virtual Private Cloud (VPC) Service Controls can play a vital role in creating an additional layer of security while also making it easier to manage your data in a way that most cloud services can’t do today.

Organizations across industries and business models use cloud services for activities such as processing their data, performing analytics, and deploying systems. VPC Service Controls can empower an organization when deciding how users and data can cross the perimeter of the supported cloud services, if at all. While VPC Service Controls are designed to help stop attackers, they can also enable contextual trusted data sharing (similar to how Zero Trust allows contextual access).

What are VPC Service Controls

VPC Service Controls help administrators define a security perimeter around Google-managed services, which can control communication to and between those services. The Service Controls isolate your Google Cloud resources from unauthorized networks, including the internet. For example, this can help you keep a clear separation between services that are allowed to run in production and services that are not.

VPC Service Controls can help you prevent mistakes that lead to costly data breaches because they control access to your data at a granular level. They add context-aware access controls on these services, and can help you achieve your organization’s Zero Trust access goals.

Like wearing two layers of clothing made from different fabrics to protect you from winter weather, VPC Service Controls may appear similar to Identity and Access Management (IAM) but they come from a different approach to implementing security. IAM enables granular identity based access control; VPC Service Controls create a security perimeter that protects your cloud resources and sets up private connectivity to Google Cloud’s APIs and services. While it’s recommended to use both, VPC Service Controls have an added bonus: They can support blocking data theft during a breach.

The additional layer of security that VPC Service Controls offer customers is challenging to achieve with on-premise systems or even with other cloud providers. You can think of it as an firewall for APIs that also adds a logical security control around three paths that data can take:

• From the public internet to your resources
• Inside your VPC and the cloud service perimeter
• For service-to-service communication (for example, denying access to someone who wants to load data to BigQuery or exfiltrate data from a BigQuery instance.)

How VPC Service Controls can help stop attackers

VPC Service Controls are used to enforce a security perimeter. They can help isolate resources of multi-tenant Google Cloud services, which can help reduce the risk of data exfiltration or a data breach.

For example, a bank that migrated financial data processing to Google Cloud can use VPC Service Controls to isolate their processing pipeline from public access (or any unauthorized access) by defining a trusted service perimeter.

How VPC Service Controls can enable trusted sharing

VPC Service Controls are used to securely share data across service perimeters with full control over what resource can connect to other resources, or outside the perimeter. This can help mitigate data exfiltration risks stemming from stolen identities, IAM policy misconfigurations, some insider threats, and compromised virtual machines.

Returning to our bank example, that same bank using VPC Service Controls may securely share or access data across Service Perimeters and Organizations. They may allow access to specific partners and for specific operations.

How VPC Service Controls support Zero Trust access

VPC Service Controls deliver Zero Trust access to multi-tenant Google Cloud services. Clients can restrict access to authorized IPs, client context, user identity, and device parameters while connecting to multi-tenant services from the internet and other services.

A bank can use moving its services to the public cloud as an opportunity to abandon outdated access management approaches and adopt Zero Trust access. VPC Service Controls let them create granular access control policies in Access Context Manager based on attributes such as user location and IP address. For example, it would allow an analyst to only access Big Query from a corporate device on the corporate network during business hours. These policies can help ensure the appropriate security controls are in place when granting access to cloud resources from the Internet.

Next steps with VPC

Check out these pages to learn more about VPC Service Controls for your sensitive cloud deployments, especially for regulated workloads. This blog is the third in our Best Kept Security Secrets series, which includes how to tap into the power of Organization Policy Service and how Cloud EKM can help resolve the cloud trust paradox.

We’re pleased to announce that Confidential Space, our new solution that allows you to control access to your sensitive data and securely collaborate in ways not previously possible, is now available in public Preview. First announced at Google Cloud Next, Confidential Space can offer many benefits to securely manage data from financial institutions, healthcare and pharmaceutical companies, and Web3 assets. Today, we will explore some security properties of the Confidential Space system that makes these solutions possible.

Confidential Space uses a trusted execution environment (TEE), which allows data contributors to have control over how their data is used and which workloads are authorized to act on the data. An attestation process and hardened operating system image helps to protect the workload and the data that the workload processes from an untrusted operator.

The Confidential Space system has three core components:

1. The workload is a containerized image with a hardened OS that runs in a cloud-based TEE. You can use Confidential Computing as the TEE that offers hardware isolation and remote attestation capabilities.
2. The attestation service, which is an OpenID Connect (OIDC) token provider. This service verifies the attestation quotes for TEE and releases authentication tokens. The tokens contain identification attributes for the workload.
3. A managed cloud protected resource, such as a Cloud Key Management Service key or Cloud Storage bucket. The resource is protected by an allow policy that grants access to authorized federated identity tokens.

The system can help ensure that access to protected resources is granted only to authorized workloads. Confidential Space also can help protect the workload from inspection and tampering, before and after attestation.

In our published Confidential Space Security Overview research paper, we explore several potential attack vectors against a Confidential Space system and how it can mitigate those threats. Notably, the research notes how Confidential Space can protect against malicious workload operators and administrators, and malicious outside adversaries, who are attempting to create rogue workload attestations.

Through these protections, Confidential Space establishes confidence that only the agreed upon workloads will be able to access sensitive data. The research also highlights some of the extensive security reviews and tests executed to identify potential weak points in the system, including domain expert reviews, meticulous security audits, and functional and fuzz testing.

We asked the NCC Group for an independent security assessment of Confidential Space to analyze its architecture and implementation. NCC Group leveraged their experience reviewing other Google Cloud products to dig deep into Confidential Space.

The NCC Group’s extensive review, which included penetration testing and automated security scanning, found zero security vulnerabilities. In their report, the architecture review highlights how the security properties are achieved through the coordination of measured boot with vTPM attestation, reduced attack surface with constricted administrator controls and access, workload measurement and enforced launch policy, and resource protection policy based on attested workload runtime properties.

The combination of these attributes creates powerful security properties, gating release of data on runtime measurements of the actual workload code and environment instead of just user and service account credentials. Confidential Space provides a platform that includes:

• A dependable workload attestation, including workload code measurement, arguments and environment, and operating environment claims
• A fully-managed attestation verification service that validates expected environmental attestation claims
• A policy engine allowing for arbitrarily complex (or extremely simple) policy to be created around those claims
• A mechanism to attach those policies to Google Cloud resources

Together, the platform provides a mechanism where one can ensure that their data is only ever released into trusted workloads that will not abuse that data.

Take a look at our documentation and codelab and take it for a spin. We hope that Confidential Space can inspire organizations to solve their use cases around multi-party collaboration with sensitive data; please contact your Google Cloud sales representative if you have any questions.

Grocery shopping has changed for good and Ocado Group has played a major role in this transformation. We started as an online supermarket, applying technology and automation to revolutionise the online grocery space. Today, after two decades of innovation, we are a global technology company providing state-of-the-art software, robotics, and AI solutions for online grocery. We created the Ocado Smart Platform, which powers the online operations of some of the world’s most forward-thinking grocery retailers, from Kroger in the U.S. to Coles in Australia.

With the global penetration of the Ocado Smart Platform and the increasing complexity of our operations, we’re paying close attention to our security estate. To proactively identify and tackle any security vulnerabilities, we decided to introduce Google Cloud’s Security Command Center (SCC) Premium as our centralized vulnerability and threat reporting service.

Gaining consolidated visibility into Ocado’s cloud assets

From the start, we were impressed with the speed of deployment and security findings surfaced with SCC. Where it would take several weeks in the past with other software vendors, we were able to quickly set up SCC in our environment and we could immediately start identifying our most vulnerable assets.

Today, we use SCC to detect misconfigurations and vulnerabilities across hundreds of projects throughout our organization and we use it to get an aggregated view of our security health findings. We filter the findings and then use Pub/Sub or Cloud Functions to send alerts directly to the tools each division is working with, such as Splunk or JIRA. This way, each of our teams can discover and respond to the security findings in their own environment, with SCC acting as the single source of truth for our security-related issues.

Driving autonomy by delegating security findings

Autonomy fuels innovation at Ocado Technology, which is why we want to make our teams as self-sufficient as possible. SCC helps to make our divisions more autonomous from the central organization. It delivers all the security insights technology teams need to make smart decisions on their own and at pace.

Here’s where SCC’s delegation features providing folder and project level access control come in. The platform’s fine-grained access control capabilities enable us to delegate SCC findings to specific teams, without having to give them a view of the entire Ocado Technology organization. Business units no longer need to contact us in the security team to track down vulnerabilities, they can do it themselves in a compliant and secure manner. It makes our work more efficient and autonomous, allowing everyone to focus on their own areas of expertise and environments.

Identifying and remediating multiple medium and high vulnerabilities

SCC’s findings are very rich and don’t end with the identification of the potential misconfigurations and vulnerabilities. It goes beyond this, recommending solutions to resolve any issues and providing clear guidelines on next steps. That’s why the feedback from our users across the organization has been so good.

SCC delivers on both quality and quantity. Since implementation, it has helped us identify and remove hundreds of medium and high vulnerabilities from our Google Cloud estate. The number of security related findings have also gone down each quarter, indicating real and tangible improvements in our security posture. SCC is so useful in maintaining our security posture as once we know where the issues are, tackling them is easy.

From 8-hour security scans to instant insights

One particular issue we’ve been able to handle well with SCC are vulnerabilities targeting the Apache logging system Log4j. SCC informed us about attempted compromises, active compromises, or the vulnerability exposure of our Dataproc images. During Log4j response, all these would have been otherwise very hard to track down, especially with limited resources. With SCC, we were able to leverage the security expertise of Google Cloud to identify the latest vulnerabilities, based on the most up-to-date security trends, and act on them quickly.

Obviously, speed is of the essence when it comes to threat mitigation and SCC has enabled us to fix issues faster, making us less exposed to outside threats. In the past, just scanning everything once could take up to eight hours. SCC sped things up from the start and findings have been nearly instantaneous since it rolled out real-time Security Health Analytics.

Strengthening compliance and demonstrating standards to stakeholders

SCC helps us to achieve better compliance standards, and demonstrate these standards to our stakeholders. We recently ran an internal audit exercise across the Ocado Technology organization, for example, where we identified the projects with the most numerous and severe security-related findings. Without the reports from SCC, this would have been extremely hard or even impossible.

We also use the Security Health Analytics information from SCC to visualize the data per project, creating a kind of heat map of security across the organization. This helps us assign our resources to the right projects and prioritize our efforts accordingly, informing our strategic decisions.

From top-down to a developer-led security

There’s been a paradigm shift in security operations, and things are moving from a top-down approach to a more developer-led and autonomous process. SCC helps drive that change at Ocado Technology. It enables us to place the responsibility for security-related issues closer to the resource owners. By making sure that the teams most impacted by a potential problem are the ones who get to fix it, we empower teams to resolve issues proactively and efficiently.

Looking forward, we can’t wait to see SCC evolve further. One of the features we’re most excited about is the ability to create custom findings (currently in preview) and additional integration capabilities that enable automation. We’re still not using everything SCC has to offer, but it is already a vital tool for our security team.

At Ocado Technology, we’re pioneering the future of online grocery shopping, and this future needs a strong security foundation. SCC helps us to strengthen and maintain that foundation, making profitable, scalable, and secure online grocery shopping possible for even more businesses around the world.

As enterprises accelerate adoption of cloud-based services and products, they face a common challenge: How can they effectively secure and govern rapidly expanding volumes of their most sensitive data in new environments? 

Today, Google Cloud released an architectural whitepaper and accompanying source code for a solution which successfully completed an assessment facilitated by KPMG LLP and has been accredited by the Enterprise Data Management Council (EDM Council) as a Cloud Data Management Capabilities (CDMC) Certified Cloud Solution. The architecture, which includes Google Cloud’s BigQuery and Dataplex Data Catalog, has been validated against CDMC’s control framework and can be used by new or existing Google Cloud clients wanting to migrate their sensitive data to the cloud with greater confidence.

“This is an exciting milestone for Google Cloud and the CDMC framework,” said John Bottega, president of EDM Council. “Google is advancing cloud adoption across all industries. Now as a CDMC Certified Cloud Solution, their clients can have the added certainty that Google’s architecture has the key controls and accepted best practices in place to protect their data in the cloud.”  

Introducing CDMC

The CDMC framework, developed and published by EDM Council, provides a set of data management capabilities, standards, and best practices for securing their cloud implementations. Contributors to the framework include large global enterprises across regulated industries, major cloud service providers, technology service organizations, and advisory firms. 

The EDM Council also provides a data management assessment framework which includes a framework composed of 14 key controls and corresponding assessment procedures, which are used to validate that sensitive data is being managed effectively. To become a CDMC Certified Cloud Solution, the Google Cloud team designed, implemented, and documented a reference architecture which met the control requirements and was then assessed by KPMG LLP.

Using the solution, customer data assets are automatically classified and tagged with business metadata, their lifecycle is managed, and a reporting engine that proactively scans and publishes findings provides security administrators information about misconfigurations and security issues via a single dashboard.

Who should use this framework?

• CDOs and CISOs: The CDMC framework provides a systematic approach to handling sensitive data in the cloud and is a good starting point for chief digital officer-level discussions around what governance and security controls should be applied to different types of sensitive data. The Google Cloud CDMC assets can help with determining practical scope and efforts needed to implement the CDMC controls within your organization.
• Data platform architects and engineers: The Google Cloud reference architecture provides reusable patterns that can be adopted and customized to suit the particular needs of your organization, as well as your existing architecture and technology stack. 
• Data owners and data stewards: Once implemented, the resulting classification, automation, findings, and reports should reduce the manual burden of effectively governing sensitive data at scale.

Architecture

The architecture combines a number of Google Cloud’s services with built-in platform capabilities and data-protection services to protect data at scale. 

These services include:

• BigQuery, which can fully manage a serverless data warehouse that enables scalable analysis for petabytes of data.
• Dataplex’s Data Catalog, which can fully manage highly scalable data discovery and metadata management with built-in data lineage features.
• Cloud Data Loss Prevention, which provides tools to classify and mask sensitive data.
• VPC Service Controls, which defines a security perimeter around Google Cloud resources to mitigate data exfiltration risks.
• Looker Studio, which provides a sample dashboard to visualize the overall health of the environment.

Additionally, the architecture leverages: 

• Secure Data Warehouse Blueprint, which adheres to practices for data governance when creating, deploying, and operating a data warehouse in Google Cloud.
• Tag Engine, which automates the process of creating and populating metadata tags in Data Catalog.

The diagram below highlights the Google Cloud services used and how they mapped to the 14 CDMC key controls:

Further information

To get started, Google Cloud customers can: 

• Read our whitepaper that describes the architecture and controls
• Download and deploy the solution via our GitHub repository
• Read more about CDMC and its Key Controls Framework
• Read more about the EDM Council