Data security is an ongoing concern for anyone managing a data warehouse. Organizations need to control access to data, down to the granular level, for secure access to data both internally and externally. With the complexity of data platforms increasing day by day, it’s become even more critical to identify and monitor access to sensitive data. In many cases, sensitive data is co-mingled with non-sensitive data, and access restrictions to sensitive data need to be enabled based on factors like data location or presence of financial information. There may also be nuances where data is sensitive for some groups of users, while for others, it is not. 

Today, we’re pleased to announce the general availability of BigQuery row-level security, which gives customers a way to control access to subsets of data in the same table for different groups of users. Row-level security (RLS) extends the principle of least privilege access and enables fine-grained access control policies in BigQuery tables. BigQuery currently supports access controls at the project-, dataset-, table- and column-level. Adding RLS to the portfolio of access controls now enables customers to filter and define access to specific rows in a table based on qualifying user conditions—providing much needed peace of mind for data professionals. 

“Our digital transformation and migration of data to the cloud magnifies the business value we can extract from our information assets. However, granular data access control is essential to comply with international regulatory and contractual requirements. BigQuery row-level security helps us comply with data residency and export restrictions,” says Jarrett Garcia, Iron Mountain’s Enterprise Data Platform Senior Director. “It enables us to manage fine-grained access controls without replicating data. What used to take months for approval and access provisioning can now be done more efficiently and effectively. We are looking forward to implementing additional data security capabilities on the BigQuery roadmap to address other critical business use cases.”

How BigQuery row-level security works

Row-level security in BigQuery enables different user personas access to subsets of data in the same table. Customers who are currently using authorized views to enable these use cases can leverage RLS for ease of management. To express the concept of RLS, we have introduced a new entity in BigQuery called row access policy. Row access policies map a group of user principals to the rows that they can see, defined by a SQL filter predicate. 

Secure logic rules created by data owners and administrators determines which user can see which rows through the creation of a row-level access policy. The row-level access policies created on a target table by administrators or data owners are applied when a query is run on the table. One table can have multiple policies applied to it.

Below is an example, where row-level access policies have been created to filter data based on users’ “region”.

In the illustrated scenario above, row-level access policies have been created to verify a querying user’s region and to give them access only to the subset of data relevant to that region. Access policies are granted to a grantee list which support all types of IAM principles such as individual users, groups, domains or service accounts. In this example, when a user queries the table, row-level access policies are evaluated to assess which, if any, policies are applicable to that user. The group ‘sales-apac’ is granted access to view a subset of rows where region = ‘APAC’ whereas the group ‘sales-us’ is granted access to view a subset of rows where the region = ’US’. Likewise, users in both groups will see rows in both regions, and users in neither group will not see any rows.

Row-level access policies can also be created using the SESSION_USER() function to restrict access only to rows that belong to the user running the query. If none of the row access policies are applicable to the querying user, the user will have no access to the data in the table.

When a user queries a table with a row-level access policy, BigQuery displays a banner notice indicating that their results may be filtered by a row-level access policy. This notice displays even if the user is a member of the `grantee_list`.

When to put BigQuery row-level security to work

Row-level access policies are useful when you have a need to limit access to data based on filter conditions. The row-access policies’ filter predicate supports arbitrary SQL, and is conceptually similar to the WHERE clause of a SQL query. Filter predicates support the SESSION_USER() function to restrict access only to rows that belong to the user running the query. If none of the row access policies are applicable to the querying user, the user will have no access to the data in the table. Currently, the column used for filtering must be in the table, but we anticipate adding support for subqueries in the filter expression, opening up access to use cases where data is filtered based on lookup tables and calculated values. Row-level access policies can be created, updated and dropped using DDL statements. You will be able to see the list of row-level access policies applied to a table using the BigQuery schema pane in the Cloud Console,  which simplifies the management of policies per table, or by using the bq command-line tool.

Row-level security is compatible with other BigQuery security features, and can be used along with column-level security for further granularity.  Since row-level access policies are applied on the source tables, any actions performed on the table will inherit the table’s associated access policies, to ensure access to secure data is protected. Row-level access policies are applicable to every method used to access BigQuery data (API, Views, etc). 

Try it out

We’re always working to enhance BigQuery’s (and Google Cloud’s) data governance capabilities, to provide more controls around managing your data. With row-level security, we are adding deeper protections for your data. You can learn more about BigQuery row-level security in our documentation and best practices.

I’ve been at Google Cloud just a few weeks, following years of experience as an AWS Hero and building on other clouds. So last week’s Google Cloud Next–my first!—was a bit of a culture shock. On the GCP podcast, I used the word “intentionality” to describe what I’m seeing: a thoughtful, holistic approach that informs so much of how Google Cloud is put together. Not just in the headline-grabbing new announcements like Google Distributed Cloud, but in the everyday things too. 

Things like IAM and project setup. Step 1 of any cloud project is to provision access to an environment, and that’s why I always found it so frustrating in my past life when I had to deal with outdated or clunky stuff like:

• Homebrewed, sketchy SSO tooling and config files
• No centralized identity—I was a different person in every cloud account
• Mysterious logouts, redirects, and missing project context within the cloud console
• Account organization features that were “bolted-on” rather than designed the right way from the beginning

In contrast, I recently shared a Twitter thread about how shockingly right Google Cloud gets identity and environments. It’s probably my favorite thing about Google Cloud so far, and so in this post I want to expand on what I’ve learned. If you’re searching for a better way to access and organize your cloud, let me make you one of today’s lucky 10,000.

Nine things to love about Google Cloud identity and environments

1. You are YOU!
Every user is just a Google account (personal or corporate) that works across projects. For beginners, this lowers the barrier to entry and makes cloud feel like an extension of things you already know. For experts, it reduces the friction of having to juggle a bunch of unrelated identities. I love that you can permit any Google account into a cloud project as a collaborator—even a contributor from outside your organization! 

2. No non-IAM root accounts
Google Cloud has been designed from the ground up to avoid the chicken/egg problem of requiring a manually configured superuser that sits outside the rest of the identity management infrastructure. In the Google world, humans use Google accounts, and services use IAM-based service accounts —it’s as straightforward as that. (Even non-Google services can be IAM—yay, workload identity federation!)  

3. Project discovery for humans
Project, folder, and organization discovery are baked into the console, like browsing a file system scoped to your access level. This hardly even feels like a feature, it’s so subtle and yet absolutely fundamental. But once you see it, you can’t imagine going back to a world where environments exist in a vacuum with no contextual awareness of each other. 

The hierarchical organization model also means that project-per-application-per-environment best practices are the path of least resistance; if anything, I’ve erred on the side of setting up *too many* logical groupings. It’s just too much fun to play with projects and folders!

4. Billing that protects you from yourself
The project context gives you a logical container for the cost of the resources contained within it. My favorite part of this is that your billing entity is managed separately from the project itself. So you can delete a project and feel sure that all associated resources are gone and no longer racking up charges … without also trashing your ability to pay for future projects you might spin up. 

(Related: the free tier does not charge you money unless you click a big button that basically says “YES, IT’S OK TO CHARGE ME MONEY.” This guarantee, combined with the familiarity of Google Accounts for access, are the main reasons I now recommend Google Cloud to beginners in my network who are looking for a safe place to learn and explore cloud.)

5. Organizational structure != billing structure
For organizations, billing is decoupled from the organization root. So permissions inheritance is a separate design decision from chargeback, as it should be. This keeps your Google Cloud footprint from converging toward Conway’s Law.

6. SSO that just works
Want to use the CLI? You get SSO out of the box with your Google Account—no corporate organization required, and no manual shuffling with config files and access keys. Or, better yet, you can use Cloud Shell to run gcloud commands right in your browser, even (especially?) on the docs pages. (Random trivia: I think Cloud Shell is the only native cloud service that has the same name across AWS, Azure, and Google Cloud–but Google’s version has been around the longest and as far as I can tell is the most fully-featured.) 

7. One group to rule them all
Remember how user entities are just Google accounts? Guess what: you can use Google Groups to manage group access to IAM roles! That’s right: one set of users with permissions across docs, email, and cloud. It’s one reason why Google Workspace makes sense as a core piece of Google Cloud; it really does function like just another cloud service from an identity standpoint. 

8. Never lose your place
In other clouds, I’ve experienced a problem I call the Timeout of Doom: when your console session expires, you’re left on a generic error screen and it’s up to you to figure out how to rebuild your context from scratch–starting with remembering what account you used in the first place. Imagine my delight to realize that reaching your Google Cloud console is as easy as bookmarking a single URL. console.cloud.google.com works and remembers who you are (or, at least, suggests the set of people you might be)—no mystery logouts or redirects.

9. Progressive complexity FTW
In my experience it’s been common for cloud providers to design most of their account features for organizations: if you’re an independent developer, you get more exposure to dangerous bills, less access to helpful SSO features, and generally must fend for yourself in a world that wasn’t really created with you in mind.

I love that Google Cloud has found a way to work with enterprises while still maintaining their roots as a cloud that developers love to use. Sign in with your personal Google account, attach it to an organization when-and-if you’re ready, and in the meantime you get the same thoughtfulness around SSO and billing as the giant shop down the street. 

I’m not going to tell you my experience has been seamless; there are footguns here (every Google Workspace integration creates a new project?), and I’m still learning. But it’s that “intentionality” thing again. The Google Cloud identity and environment experience feels like it was designed, not just accreted; there’s an elegant simplicity to it that makes cloud feel fresh and exciting to me all over again. I can’t wait to see what’s next.

In the meantime, I highly encourage you to do what I did and spin up a free trial to try things out for yourself. Then hit me up on Twitter with your favorite Google Cloud identity or environment feature!

Last July, Google Cloud launched our ambassador partnership with the Health Information Sharing and Analysis Center (Health-ISAC) and committed to working with industry leaders to better protect our healthcare ecosystem. Securing healthcare technology and data is a global challenge, and to meet it security professionals need to have better channels for sharing information and effective security practices. 

To that end, we’re pleased to announce that our relationship with Health-ISAC is now expanding to include CISOs and security leaders in Europe, the Middle East, and Africa (EMEA).  

On May 23, 2023, we’ll join the Health-ISAC on a 17-city tour across the region, starting in Zurich, Switzerland, as part of its European Healthcare Threat Landscape Tour. In each session, we will bring together experts from across the region with cybersecurity practitioners from the Health-ISAC, Google Cloud, law enforcement, and regulatory bodies.  

Our goals is to focus the discussions on several topics:

• How the threat landscape is unique to each country we visit;
• How we can better share threat intelligence; 
• What are the emerging European laws and regulations  that can affect healthcare and cybersecurity;
• How ransomware continues to evolve and the threat that poses to healthcare; organizations;
• What we can do to improve cyber incident response;
• What impact digital sovereignty can have on European healthcare;
• And which initiatives Health-ISAC and Google Cloud are launching to better protect Europe’s healthcare system.

The timing for the tour could not come at a more critical time. Ransomware and digital extortion threatens healthcare and its subsectors, with national health systems in Ireland, UK, Germany, and Spain all recently experiencing extended outages because of ransomware. Threat actors continue to target proprietary data at biotechnology and pharmaceutical firms. Meanwhile, existing regulations such as the Digital Markets Act and GDPR, and new regulations such as the NIS2 Directive (Directive (EU) 2022/2555), which seek to protect EU citizens and businesses from risks posed by the digital world, are raising the bar that CISOs and security leaders must surpass to ensure their businesses can stay compliant and able compete in the global digital economy.

“The European Healthcare Threat Landscape Tour is one of many efforts we have undertaken to build a closer community of trust around threats and cybersecurity practices that keep health organization’s safe,” said Denise Anderson, executive director, Health-ISAC.

“We all rely on these organizations to take care of us and produce therapies that keep us healthy. Delivering these engagements alongside Google Cloud helps us achieve global scale and shows off both organizations’ commitments to building a sustainable, resilient healthcare ecosystem for entire societies.” 

Each stop in the tour will provide opportunities to dive deep into these topics, grow an understanding of these challenges, create a space for collaboration, and build closer, trusted partnerships. The meetings are open to member and non-member organizations working in healthcare. Interested participants can contact the Health-ISAC through their contact form or register on their website for the nearest event.

“The Google Cybersecurity Action Team’s mission is to secure customers, secure the cloud, and secure the planet (and beyond). Working with industry groups like Health-ISAC helps us achieve this mission while respecting each other and the unique challenges European health systems face. Myself and our EMEA-based Google Cloud and Mandiant security teams are looking forward to another great collaboration,” said Phil Venables, chief information security officer, Google Cloud.

At Google Cloud, we’re committed to helping build a secure and resilient healthcare ecosystem for everyone. As we hit the road with the Health-ISAC this summer, we’ll be sure to send updates on what we learn through this collaboration and how we can all work together to secure smarter in 2023 and beyond.

Editor’s note: Today we are hearing from Jono MacDougall , Principal Software Engineer at Ravelin. Ravelin delivers market-leading online fraud detection and payment acceptance solutions for online retailers. To help us meet the scaling, throughput, and latency demands of our growing roster of large-scale clients, we migrated to Google Cloud and its suite of managed services, including Cloud Bigtable, the scalable NoSQL database for large workloads.

As a fraud detection company for online retailers, each new client brings new data that must be kept in a secure manner and new financial transactions to analyze. This means our data infrastructure must be highly scalable and constantly maintain low latency. Our goal is to bring these new organizations on quickly without interrupting their business. We help our clients with checkout flows, so we need latencies that won’t interrupt that process—a critical concern in the booming online retail sector. 

We like Cloud Bigtable because it can quickly and securely ingest and process a high volume of data. Our software accesses data in Bigtable every time it makes a fraud decision. When a client’s customer places an order, we need to process their full history and as much data as possible about that customer in order to detect fraud, all while keeping their data secure. Bigtable excels at accessing and processing that data in a short time window. With a customer key, we can quickly access data, bring it into our feature extraction process, and generate features for our models and rules. The data stays encrypted at rest in Bigtable, which keeps us and our customers safe.

Bigtable also lets us present customer profiles in our dashboard to our client, so that if we make a fraud decision, our clients can confirm the fraud using the same data source we use.

We have configured our bigtable clusters to only be accessible within our private network and have restricted our pods access to it using targeted service accounts. This way the majority of our code does not have access to bigtable and only the bits that do the reading and writing have those privileges.

We also use Bigtable for debugging, logging, and tracing, because we have spare capacity and it’s a fast, convenient location. 

We conduct load testings against Bigtable.  We started at a low rate of ~10 Bigtable requests per second and we peaked at ~167000 mixed read and write requests per second  at absolute peak. The only intervention that was done to achieve this was pressing a single button to increase the number of nodes in the database. No other changes were made.

In terms of real traffic to our production system, we have seen ~22,000 req/s (combined read/write) on Bigtable in our live environment as a peak within the last 6 weeks.

Migrating seamlessly to Google Cloud 

Like many startups, we started with Postgres, since it was easy and it was what we knew, but we quickly realized that scaling would be a challenge, and we didn’t want to manage enormous Postgres instances. We looked for a kind of key value store, because we weren’t doing crazy JOINS or complex WHERE clauses. We wanted to provide a customer ID and get everything we knew about it, and that’s where key value really shines.  

I used Cassandra at a previous company, but we had to hire several people just for that chore. At Ravelin we wanted to move to managed services and save ourselves that headache. We were already heavy users and fans of BigQuery, Google Cloud’s serverless, scalable data warehouse, and we also wanted to start using Kubernetes. This was five years ago, and though quite a few providers offer Kubernetes services now, we still see Google Cloud at the top of that stack with Google Kubernetes Engine (GKE). We also like Bigtable’s versioning capability that helped with a use case involving upserts. All of these features helped us choose Bigtable.

Migrations can be intimidating, especially in retail where downtime isn’t an option. We were migrating not just from Postgres to Bigtable, but also from AWS to Google Cloud. To prepare, we ran in AWS like always, but at the same time we set up a queue at our API level to mirror every request over to Google Cloud. We looked at those requests to see if any were failing, and confirmed if the results and response times were the same as in AWS. We did that for a month, fine tuning along the way. 

Then we took the big step and flipped a config flag and it was 100% over to Google Cloud. At the exact same time, we flipped the queue over to AWS so that we could still send traffic into our legacy environment. That way, if anything went wrong, we could fail back without missing data. We ran like that for about a month, and we never had to fail back. In the end, we pulled off a seamless, issue-free online migration to Google Cloud.

Flexing Bigtable’s features

For our database structure, we originally had everything spread across rows, and we’d use a hash of a customer ID as a prefix. Then we could scan each record of history, such as orders or transactions. But eventually we got customers that were too big, where the scanning wasn’t fast enough. So we switched and put all of the customer data into one row and the history into columns. Then each cell was a different record, order, payment method, or transaction. Now, we can quickly look up the one row and get all the necessary details of that customer. Some of our clients send us test customers who place an order, say, every minute, and that quickly becomes problematic if you want to pull out enormous amounts of data without any limits on your row size. The garbage collection feature makes it easy to clean up big customers.  

We also use Bigtable replication to increase reliability, atomicity, and consistency. We need strong consistency guarantees within the context of a single request to our API since we make multiple bigtable requests within that scope. So within a request we always hit the same replica of Bigtable and if we have a failure, we retry the whole request. That allows us to make use of the replica and some of the consistency guarantees, a nice little trade-off where we can choose where we want our consistency to live.https://www.youtube.com/embed/0-eH5u7rrQQ?enablejsapi=1&

We also use BigQuery with Bigtable for training on customer records or queries with complicated WHERE clauses. We put the data in Bigtable, and also asynchronously in BigQuery using streaming inserts, which allows our data scientists to query it in every way you can imagine, build models, and investigate patterns and not worry about query engine limitations. Since our Bigtable production cluster is completely separate, doing a query on BigQuery has no impact on our response times. When we were on Postgres many years ago, it was used for both analysis and real time traffic and it was not the optimal solution for us. We also use Elasticsearch for powering text searches for our dashboard.

If you’re using Bigtable, we recommend three features:

• Key visualizer. If we get latency or errors coming back from Bigtable, we look at the key visualizer first. We may have a hotkey or a wide row, and the visualizer will alert us and provide the exact key range where the key lives, or the row in question. Then we can go in and fix it at that level. It’s useful to know how your data is hitting Bigtable and if you’re using any anti-patterns or if your clients have changed their traffic pattern that exacerbated some issue.
• Garbage collection. We can prevent big row issues by putting size limits in place with the garbage collection policies.  
• Cell versioning. Bigtable has a 3d array, with rows, columns, and cells, which are all the different versions. You can make use of the versioning to get history of a particular value or to build a time series within one row. Getting a single row is very fast in Bigtable so as long as you can keep the data volume in check for that row, making use of cell versions is a very powerful and fast option. There are patterns in the docs that are quite useful and not immediately obvious. For example, one trick is to reverse your timestamps (MAXINT64 – now) so instead of the latest version, you can get the oldest version effectively reversing the cell version sorting if you need it.

Google Cloud and Bigtable help us meet the low-latency demands of the growing online retail sector, with speed and easy integration with other Google Cloud services like BigQuery. With their managed services, we freed up time to focus on innovations and meet the needs of bigger and bigger customers. 

Learn more about Ravelin and Bigtable, and check out our recent blog, How BIG is Cloud Bigtable?

Hackensack Meridian Health (HMH) executive Mark Eimer explains how an ambitiously-timed rollout of a comprehensive suite of Google products helped the entire organization—from doctors to IT staff—achieve better security, cultivate a more equitable work environment, and ultimately, improve patient outcomes.

How does a recently merged, 17-hospital healthcare system fast-track a platform migration and hardware rollout securely and in a way that improves work for everyone, regardless of location or role? These are the questions that kept me up at night in early 2020, when the pandemic demanded a “big bang”—something our legacy laptops and operating systems couldn’t handle.

We began our work with Google in 2020 with the adoption of Chrome as our default browser. As we migrated platforms, keeping patient data safe was of the utmost importance to us, along with providing every staff member with the tools they needed to work virtually. Our staff often experienced issues accessing our web-based applications using Internet Explorer or Edge Browser, a problem that went away when we switched to Chrome. Chrome’s versatile compatibility also made it easier for my team to migrate all of our web-based operations, and Chrome’s security and manageability were key components to making this switch a huge win for the organization.

The success of this migration led us to extend our Google partnership to patient care applications—where Google’s expertise in AI and ML helps scale the use of diagnostics tools and improve other aspects of the patient journey.

Achieving security at every step 

Like so many other healthcare organizations, we’ve been concerned about ransomware attacks. This is part of why we moved to Google Workspace and distributed over 3,000 Chrome OS devices in kiosk mode in March of 2020, when many of us went remote due to the pandemic. We were very concerned about team members accessing corporate applications through home devices that were running EOL operating systems (WIN7), as well as a general lack of antivirus and encryption measures.

We were protected by the fact that Google’s software and hardware both had built-in security features that we needed to stave off sophisticated attackers. For example, Chrome OS automatically updates to the latest security update and encrypts data living outside the cloud on the hardware. These features protected us from security-related disruptions, letting us securely move a huge library of file shares and emails across thousands of accounts to Google Chrome OS in just four months.

A year later, in March 2021, we migrated the enterprise over to Google Workspace and saw an immediate reduction in spam by 30% from the inherent built-in AI/ML. This meant staff were less likely to receive (and click through) phishing attempts. My team could connect, create, and collaborate easily and securely—even as more of us were working from home and needed to access sensitive data remotely. 

Leveling the playing field

As an organization, we were surprised by how many team members didn’t have personal computers at home. We quickly decided that if we needed team members to work from home, the health network would have to supply hardware. Chromebooks’ lower price tag compared to PCs—on top of their built-in security controls—allowed us to purchase, deploy, and support that initial distribution of 3,000 Chromebooks to team members in less than three weeks, providing devices to every eligible remote employee instead of just a select few. This was vital to reaching our equitable technology goal as part of our diversity and inclusion initiative: everybody has the same tools to do good work.

When all employees have what they need to do their jobs well, we get better patient outcomes. Before we began this cloud adoption journey, patient and staff experiences were different within the hospitals and outside of them. 

Now it’s the same wherever our staff is, and we’ve seen efficiency and accessibility benefits extend to the patient side. For example, we built a web-based contact center that supports 80 locations that use Workspace and Chrome OS devices. Since customer service, admin, and providers are all on the same system, it has become a one-stop shop for patients.

Furthermore, through the Grow with Google program, we were able to provide another benefit to employees that drove our equity goals. Google trained 50 non-IT staff members—from environmental services, food and nutrition, and other non-tech areas who were interested in making a career change to IT—on the Google products we were using. They may not have thought about switching to a career in IT before the Grow with Google program came to our organization, but through this partnership, they now have that opportunity.

A strategic, long-term partner

With any large-scale rollout, the work doesn’t end once laptops are in employee hands. Google has shown their commitment to long-term collaboration as they continuously optimize their products for the unique needs of healthcare providers and go the extra mile in tailoring tools to our staff’s workflows. 

For example, on the Chrome OS side, the Google team has helped our registration desks and document centers with device integration for hardware like credit card readers and e-signature pads. They’ve also helped us meet security and privacy requirements mandated by state and federal governments around HIPAA, Medicaid, and Medicare reimbursements. Over this next year, we’ll look at a feature roadmap with Google Cloud to deliver further enhancements, iterating on the product itself to meet our needs for the present and the future.

Delivering the future of healthcare

The benefits we’ve seen around security and usability—and the ability to provide all staff with equal access to Google’s technology—are why we’re expanding our partnership with Google to both the administrative and clinical sides of HMH. In addition to further Google rollouts with corporate, next year we’re distributing Chromebooks to all 350 of our ambulatory clinics.

We’re also working with the Google professional services team to create a custom AI model that analyzes 3D mammogram images. This AI model will enable two providers to read mammograms—which adheres to international best practices but is currently rare in the US—without requiring additional time. Conducting double readings of mammograms will yield better health outcomes for our patients, such as a lower patient recall rate and an increased accuracy in detecting breast cancer.

We’re currently building the model using a variety of Google Cloud products, including Cloud Healthcare API. Once complete, this model is expected to be trained, deployed, and maintained in Google’s Vertex AI, allowing our providers to be more productive as they make clinical decisions with AI support. As the model is proven over time, we plan to make the predictive services accessible to other healthcare organizations.

With Google, we’re able to achieve a unified architecture for storing data as well as training and deploying AI models, which enable our staff to work more efficiently and securely from anywhere. While I may not be able to predict the future as accurately as AI can, I foresee our continued partnership with Google as a key part of HMH’s improved provider and patient outcomes.