Data security is an ongoing concern for anyone managing a data warehouse. Organizations need to control access to data, down to the granular level, for secure access to data both internally and externally. With the complexity of data platforms increasing day by day, it’s become even more critical to identify and monitor access to sensitive data. In many cases, sensitive data is co-mingled with non-sensitive data, and access restrictions to sensitive data need to be enabled based on factors like data location or presence of financial information. There may also be nuances where data is sensitive for some groups of users, while for others, it is not. 

Today, we’re pleased to announce the general availability of BigQuery row-level security, which gives customers a way to control access to subsets of data in the same table for different groups of users. Row-level security (RLS) extends the principle of least privilege access and enables fine-grained access control policies in BigQuery tables. BigQuery currently supports access controls at the project-, dataset-, table- and column-level. Adding RLS to the portfolio of access controls now enables customers to filter and define access to specific rows in a table based on qualifying user conditions—providing much needed peace of mind for data professionals. 

“Our digital transformation and migration of data to the cloud magnifies the business value we can extract from our information assets. However, granular data access control is essential to comply with international regulatory and contractual requirements. BigQuery row-level security helps us comply with data residency and export restrictions,” says Jarrett Garcia, Iron Mountain’s Enterprise Data Platform Senior Director. “It enables us to manage fine-grained access controls without replicating data. What used to take months for approval and access provisioning can now be done more efficiently and effectively. We are looking forward to implementing additional data security capabilities on the BigQuery roadmap to address other critical business use cases.”

How BigQuery row-level security works

Row-level security in BigQuery enables different user personas access to subsets of data in the same table. Customers who are currently using authorized views to enable these use cases can leverage RLS for ease of management. To express the concept of RLS, we have introduced a new entity in BigQuery called row access policy. Row access policies map a group of user principals to the rows that they can see, defined by a SQL filter predicate. 

Secure logic rules created by data owners and administrators determines which user can see which rows through the creation of a row-level access policy. The row-level access policies created on a target table by administrators or data owners are applied when a query is run on the table. One table can have multiple policies applied to it.

Below is an example, where row-level access policies have been created to filter data based on users’ “region”.

In the illustrated scenario above, row-level access policies have been created to verify a querying user’s region and to give them access only to the subset of data relevant to that region. Access policies are granted to a grantee list which support all types of IAM principles such as individual users, groups, domains or service accounts. In this example, when a user queries the table, row-level access policies are evaluated to assess which, if any, policies are applicable to that user. The group ‘sales-apac’ is granted access to view a subset of rows where region = ‘APAC’ whereas the group ‘sales-us’ is granted access to view a subset of rows where the region = ’US’. Likewise, users in both groups will see rows in both regions, and users in neither group will not see any rows.

Row-level access policies can also be created using the SESSION_USER() function to restrict access only to rows that belong to the user running the query. If none of the row access policies are applicable to the querying user, the user will have no access to the data in the table.

When a user queries a table with a row-level access policy, BigQuery displays a banner notice indicating that their results may be filtered by a row-level access policy. This notice displays even if the user is a member of the `grantee_list`.

When to put BigQuery row-level security to work

Row-level access policies are useful when you have a need to limit access to data based on filter conditions. The row-access policies’ filter predicate supports arbitrary SQL, and is conceptually similar to the WHERE clause of a SQL query. Filter predicates support the SESSION_USER() function to restrict access only to rows that belong to the user running the query. If none of the row access policies are applicable to the querying user, the user will have no access to the data in the table. Currently, the column used for filtering must be in the table, but we anticipate adding support for subqueries in the filter expression, opening up access to use cases where data is filtered based on lookup tables and calculated values. Row-level access policies can be created, updated and dropped using DDL statements. You will be able to see the list of row-level access policies applied to a table using the BigQuery schema pane in the Cloud Console,  which simplifies the management of policies per table, or by using the bq command-line tool.

Row-level security is compatible with other BigQuery security features, and can be used along with column-level security for further granularity.  Since row-level access policies are applied on the source tables, any actions performed on the table will inherit the table’s associated access policies, to ensure access to secure data is protected. Row-level access policies are applicable to every method used to access BigQuery data (API, Views, etc). 

Try it out

We’re always working to enhance BigQuery’s (and Google Cloud’s) data governance capabilities, to provide more controls around managing your data. With row-level security, we are adding deeper protections for your data. You can learn more about BigQuery row-level security in our documentation and best practices.

As we engage with enterprises across the globe, one thing is becoming clear: Today’s businesses are solving complex business problems that are data-intensive.

But often, their data platform infrastructure is holding them back. Data platform architectures that were designed in the 1990s are not ready to solve business problems for 2020. We don’t have to tell you about the explosive data growth that’s going on for businesses around the world. If you’re managing data infrastructure today, you already know plenty about that data growth.

Ever faster and larger data streams, global business needs, and tech-savvy users are all putting the pressure on IT teams to move faster, with more agility.

Despite all these changes, it is often the legacy, traditional data warehouse where most of the data analytics tasks take place, and they’re underprepared for those demands.

When we talk to people working in IT today, we hear a lot about the constraints that come with operating legacy technology while trying to build a modern data strategy. Those legacy data warehouses likely aren’t cutting it anymore. Here’s why—and here’s what you can do about it.  

1. Business agility is hard to achieve with legacy tools. 
Business agility is the main goal as organizations move toward completely digital operations. Think of online banking, or retailers staying ahead of always-on e-commerce needs in a competitive environment. All these great, cutting-edge innovations reflect cultural and technical change, where flexibility is essential. A business has to be able to manage and analyze data quickly to understand how to better serve customers, and allow its internal teams to do their best work with the best data available.

We hear that lots of data warehouses running today are operating at 95% or 100%, maxing out what they can provide to the business.

Whether it’s on-premises or an existing data warehouse infrastructure moved wholesale to cloud, those warehouses aren’t keeping up with all the data requests users have. Managing and preventing these issues can take up a lot of IT time, and the problems often compound over time. Hitting capacity limits slows down users and ties up database administrators too.

From a data infrastructure perspective, separating the compute and storage layers is essential to achieve business agility. When a data warehouse can handle your scalability needs and self-manage performance, that’s when you can really start being proactive. 

2. Legacy data warehouses require a disproportionate degree of management. 
Most of the reports and queries your business runs are probably time-sensitive, and that sense of urgency is only increasing as users and teams see the possibilities of data analytics.

In our engagements with customers, we often observe that they are spending a majority of the time on systems engineering, so that only about 15% of the time is spent analyzing data. That’s a lot of time spent on maintenance work.

Because legacy infrastructure is complex, we often hear that businesses continue to invest in hiring people to manage those outdated systems, even though they’re not advancing data strategy or agility.

To cut time on managing a data warehouse, it helps to automate the system engineering work away from the analytics work, like BigQuery enables.

Once those functions are separated, the analytics work can take center stage and let users become less dependent on administrators. BigQuery also helps remove the user access issues that are common with legacy data warehouses. Once that happens, users can focus on building reports, exploring datasets, and sharing trusted results easily. 

3. Legacy data warehouse costs make it harder to invest in strategy. 
Like other on-prem systems, data warehouses adhere to the old-school model of paying for technology, with the associated hardware and licensing costs and ongoing systems engineering.

This kind of inefficient architecture drives more inefficiency. When the business is moving toward becoming data-driven, they’ll continue to ask your team for more data. But responding to those needs means you’ll run out of money pretty quickly. 

Cloud offers much more cost flexibility, meaning you’re not paying for, or managing, the entire underlying infrastructure stack. Of course, it’s possible to simply port an inefficient legacy architecture into the public cloud.

To avoid that, we like to talk about total cost of ownership (TCO) for data warehouses, because it captures the full picture of how legacy technology costs and business agility aren’t matching up.

Moving to BigQuery isn’t just moving to cloud—it’s moving to a new cost model, where you’re cutting out that underlying infrastructure and systems engineering. You can get more detail on cloud data warehouse TCO comparisons from ESG.

4. A legacy data warehouse can’t flexibly meet business needs. 
While overnight data operations used to be the norm, the global opportunities for businesses mean that a data warehouse now has to load streaming and batch data while also supporting simultaneous queries. Hardware is the main constraint for legacy systems as they struggle to keep up.

Moving your existing architecture into the cloud usually means moving your existing issues into the cloud, and we hear from businesses that doing so still doesn’t allow for real-time streaming.

That’s a key component for data analysts and users. Using a platform like BigQuery means you’re essentially moving your computational capabilities into the data warehouse itself, so it scales as more and more users are accessing analytics.

Unlimited compute is a pretty good way to help your business become digital. Instead of playing catch-up with user requests, you can focus on developing new features. Cloud brings added security, too, with cloud data warehouses able to do things like automatically replicate, restore and back up data, and offer ways to classify and redact sensitive data. 

5. Legacy data warehouses lack built-in, mature predictive analytics solutions. 
Legacy data warehouses are usually struggling to keep up with daily data needs, like providing reports to departments like finance or sales. It can be hard to imagine having the time and resources to start doing predictive analytics when provisioning and compute limits are holding your teams back.

We hear from customers that many of them are tasked with simplifying infrastructure and adding modern capabilities like AI, ML and self-service analytics for business users.

The best stories about digital transformation are those where the technology changes and business or cultural changes happen at the same time. One customer told us that because BigQuery uses a familiar SQL interface, they were actually able to shift the work of data analytics away from a small, overworked group of data scientists into the hands of many more workers.

Doing so also eliminated a lot of the siloed data lakes that had sprung up as data scientists extracted data one project at a time into various repositories to train ML models.

These large-scale computational possibilities save time and overhead, but also let businesses explore new avenues of growth. AI and ML are already changing the face of industries like retail, where predictive analytics can provide forecasting and other tasks to help the business make better decisions. BigQuery lets you take on sophisticated machine learning tasks without moving data or using a third-party tool. 

We designed BigQuery so that our engineers deploy the resources needed for you to scale. It means your focus can change entirely toward meeting the needs the business has put forth, and bringing a lot more flexibility.

BigQuery is fully serverless and runs on underlying Google infrastructure, so it integrates with our ecosystem of data and analytics partner tools. This architecture means you’re continually getting the most up-to-date software stack—analytics that scale, real-time insights, and cutting-edge functionality that includes geospatial and machine learning right from the SQL interface. 

Streamline your path to data warehouse modernization with BigQuery by learning about Google Cloud’s proven migration methodology and get started with your data warehouse by applying for our migration offer.

Pierogies might just be the ultimate comfort food. But when Mrs. T’s Pierogies — the leading manufacturer of frozen pierogies in the US — learned it needed to transition its existing on-premises SAP ECC to S/4HANA, the company sought a little comfort for itself. 

Founded in 1952, Mrs. T’s Pierogies now produces more than 650 million pierogies a year. Moving that many pierogies requires a powerful ERP system — and an equally powerful IT infrastructure on which to run it. Mrs. T’s realized that the SAP-mandated transition of its ERP solution to SAP S/4HANA was an opportunity to move its SAP systems to Google Cloud and gain real-time analytics capabilities for faster sales forecasting, more effective trade promotions, and more sophisticated planning.

From necessity to opportunity

Mrs. T’s successfully ran its SAP ECC solution on its own on-premises servers for years. But after SAP decided to sunset the product, Mrs. T’s realized that it faced multiple challenges, including: 

• Migrating its SAP ECC 6.0 system from an on-premises leagcy OS to a cloud-based Linux environment
• Moving data from its SAP DB2 database to HANA
• Transitioning from ECC to SAP S/4HANA

That complex transition needed to take place with little or no downtime, since nearly all of the company’s invoices, warehouse movements, and transfer orders used the Electronic Data Interchange (EDI) protocol. Missing even a few hours of EDI transactions would put significant revenue at stake. Adding to the challenge: Some Mrs. T’s customers would not accept an invoice past five days, which left little room for error. Mrs. T’s chose Rackspace Technology, a longtime Google Cloud partner, to oversee the move. 

Mrs. T’s could have chosen to run S/4HANA on its legacy hardware but saw migration to Google Cloud as an opportunity to improve key aspects of its business, in particular data analytics. Historically, sales planning and forecasting involved time-consuming manual processes. But the speed, availability, and scalability of Google Cloud meant that Mrs. T’s could take advantage of S/4HANA’s embedded analytics capabilities. Migrating could also open the door to leveraging Google Cloud’s native integration of SAP data to power Google tools such as BigQuery, Google Cloud AI Building Blocks, and more. 

The move to Google Cloud also gave Mrs. T’s an opportunity to update its disaster recovery process. Previously, the company backed up to tape. So, if its SAP systems went down, a member of the IT department would have to drive the most recent tape backups an hour to its cold site, where the disaster recovery partner would load the SAP backup tape, boot the system up, switch network connections to that site, and cross their fingers. Not only would downtime be significant, but restored data would be limited to the periodic tape backup. 

“Everything just worked”

Once Mrs. T’s decided to migrate to Google Cloud, the company worked with Rackspace Technology to implement the system. The first phase focused on moving the SAP production environment from on-premises infrastructure to Google Cloud and updating its database to HANA, which took place over 12 weeks. The switchover occurred over a weekend and was all but invisible to users. “We came in on Monday and everything just worked,” recalls Timothy Coyle, Director of Information Systems & Technology. In a second four-month phase, the company transitioned from ECC to S/4. 

The move to Google Cloud paid dividends immediately. Batch transactions occurred twice as fast and on-screen end-user transactions rendered instantly. The upgrade also gave the finance team access to embedded analytics and monitoring for the first time. With everything now in the cloud, disaster recovery could be dynamic and nearly instantaneous, with a worst-case scenario of just 5 to 10 minutes of downtime. 

“Mrs. T’s needed a skilled and experienced partner that could move its SAP environment to Google Cloud with no negative impacts to its business. We knew this migration was a key initiative in Mrs. T’s digital transformation journey,” says Chuck Britton, Google Partner Development Manager at Rackspace. “We also knew that running SAP on Google Cloud would give the business the fast and flexible analytical capabilities it needed for its SAP data.” 

From ideation to production, Mrs. T’s migrated from its legacy on-premises infrastructure to a modern SAP S/4HANA solution on Google Cloud in only seven months, with minimal downtime and zero disruptions. Now that the migration is complete, Mrs. T’s has a flexible, highly scalable environment to run the SAP applications and data that fuel the business. Says Coyle, “Our strategic intent for IT is to make processes simpler, people more productive, and infrastructure more secure. This project fits right square in the middle of that strategic philosophy.”

Learn more about ways in which Google Cloud can transform your SAP experience and about Rackspace Google Cloud solutions for SAP customers.