The COVID-19 pandemic continues to push the healthcare and life sciences industry in entirely new ways. In record time, we’ve witnessed public health officials, vaccine developers, equipment manufacturers, and essential workers take life saving actions—regularly putting their own lives at risk—to respond to the exceptional challenges of our time. 

Yet after all the turmoil and uncertainty of the pandemic, record breaking levels of investment continue to come into the market to fuel innovations. 

Vaccine development is now measured in weeks rather than years; providers are leveraging telehealth technologies to improve the physician and patient experience; and individuals have embraced a variety of devices to assume greater ownership and control of their personal health. 

With that in mind, here are five of the many innovations I see driving healthcare and life sciences for at least the next 12 months: 

1. Unleashing the power of healthcare and life sciences data

People have arguably never had as much access and understanding to their personal health data than they can in 2022. They have the ability to understand their genetic makeup, medical history, family history, and activity levels to ensure they are living a healthy lifestyle. Taken together, this longitudinal profile can become the basis for more personalized medicine. Add wearable devices to the mix and patients gain real- or near-real-time updates on their health status. 

Further, global regulatory agencies have continued to mandate the need for providers to maintain a longitudinal health record that provides a holistic view of the patient across all the encounters they have had with a health system. Physician surveys conducted by the The Harris Poll and Google Cloud show that a 360-degree view of the patient, across all provider encounters, leads to faster, more accurate diagnosis, and better outcomes. 

If secure data access and interoperability can begin to include insurers, researchers, public health officials, and others in the field, the powerful network effects benefiting patients will only grow. When combined with those longitudinal phenome profiles, the opportunities for personalized and preventative medicine enter a whole new era.

2. Healthcare and life sciences M&A boom continues

Although the pandemic initially slowed activity on mergers and acquisitions in the early part of 2020, the healthcare and life sciences industry has seen a rapid rebound and acceleration of deals ever since. The success of COVID-19 vaccines, the importance of telehealth, and the focus on molecular modeling and genomic-based drug development have all boosted investment as organizations look to enter new markets, develop new therapies, and leverage low interest rates while they last. 

In 2021, the total funding of US digital health startups surpassed $29 billion across 729 deals, according to advisory Rock Health. That’s almost double the levels of 2020, which itself set records. Analysts at PWC meanwhile estimate M&A investments in biopharmaceutical and life sciences could approach $400 billion this year across all sub-sectors

Clearly, the pandemic has been a primary driver of investment as the focus on healthcare has dramatically increased. Yet the increased activity also reflects changing business models and emerging technologies that are now required to compete in the rapidly evolving space. For organizations to capitalize on these investments, it will take not only great vision and intellectual property but also the right technologies—like cloud—and the right data interoperability models, to make partnerships and acquisitions more scalable, feasible, and seamless.

3. Transforming the patient experience at a new rate

The pandemic has shined a spotlight on the inefficiencies and complexities that exist in healthcare markets across the globe. As wave after wave has surged, global healthcare systems remain overwhelmed on most every aspect of patients’ treatment journeys. Even before COVID-19, healthcare was already one of the largest spend areas for governments around the world. The pandemic has only exacerbated the known issues. 

Clearly, administrators, regulators, physicians, nurses, and patients would agree that the processes and models need to change. There’s a need to maintain this momentum and even increase the tempo to achieve lasting change.

Take telehealth. Within months of the start of the pandemic, providers moved to provide more remote capabilities so physicians could still meet with patients virtually to ensure health and safety on all sides. Payers recognized the importance of telehealth and began to update reimbursement rules. And organizations are now reimagining policies in areas such as prior authorization, submission, and adjudication to reduce complexity and bureaucracy while improving responsiveness.

Looking beyond the system, organizations are also recognizing and deepening their understanding of the structural and social determinants of health that impact patient care and health outcomes, especially for historically underserved communities.  Private and public sectors are learning from, and increasingly partnering with, the social sciences, public health, biomedical informatics, computer science, public policy and community groups around how to build a mI’ore equitable and inclusive consumer products and Health IT strategies. 

The newfound levels of transparency, visibility, and accountability that patients, caregivers, and organizations are achieving will ultimately increase competition and provide a more effective, equitable, efficient and, above all, healthier marketplace for all patients. As we move past the worst of the pandemic, regulators and organizations should keep fighting for progress over business as usual.

4. AI is now a core competency for Drug Development

The ability of organizations like Pfizer, Moderna, Johnson & Johnson, and Astrazeneca to develop COVID-19 vaccines has been a remarkable accomplishment—particularly the historic speed with which they were created and deployed. This innovation acceleration was largely enabled by the use of new drug development platforms that allow researchers to use artificial intelligence and machine learning to model protein and cellular interactions to rapidly advance the science.

No longer must researchers rely on traditional laboratory testing (and retesting). With their improved understanding of the molecular and genetic structure of a patient and, for example, their tumor, researchers can use AI to enable simulations on computers rather than testing in live conditions. This technology can process thousands, even millions of simulations to help  identify high-potential candidates for treatment consideration and subsequent analysis. 

AI-enabled drug discovery models can eliminate months and years from the research process, which can reduce the time to develop a drug and accelerate the time to treatment for an individual patient. As just one example, consider the work on AlphaFold2 by Google’s DeepMind unit who leverages AI to predict effective protein shapes for new drugs. Healthcare and life sciences organizations already recognize the potential of AI. Now comes the investments to leverage this rapidly evolving technology to support their efforts now and in the future.   

5. Ecosystem partnerships tackle complexity and spur innovation

As the importance and growth of the healthcare and life sciences industry continues, we will see even more new players and partnerships emerging to address old problems in new ways. This trend will touch all aspects of the healthcare value chain and will, increasingly, see three- and four-player partnerships emerge to address the complex challenges of today’s healthcare marketplace.

Technology will continue to play a key role as capabilities and platforms will transform all aspects of the marketplace. Cell phones, wearable devices, and other technologies will provide real-time updates and notifications to patients on everything from glucose levels to payments for healthcare services. Voice recognition software will document physician and patient discussions to reduce the burden of record keeping. Real-world data will be used to simplify and confidentially recruit patients for participation in clinical trials. 

New players will continue to enter the market to improve health outcomes and reduce costs. Major retailers are among the companies extending their pharmacies to provide additional diagnostic and concierge services, saving patients from additional appointments while boosting prevention. Community organizations are emerging to help identify and care for underserved communities whose health outcomes are significantly lower than the average patient. 

For all the exhausting and heart-wrenching challenges of the past two years, the opportunities the pandemic has laid bare cannot be overlooked. We owe it to those who have worked and fought so hard for every life to forge even more new partnerships—and make it easier to do so—so that the next crisis, when it does arise, will never be as bad as the one we’re now conquering. Technology can be the enabler in this effort and help bring us together to continue to conquer the challenges that lie ahead.

At Precognitive, we were able to start with a blank technology slate to support our fraud detection software products. When we started building the initial version of our platform in 2017, we had some decisions to make: What coding language to use? What cloud infrastructure provider to choose? What database to use? The majority of the decisions were straightforward, but we struggled to decide upon a database. We had plenty of collective experience with relational databases, but not with a wide-column database like Cloud Bigtable—which we knew we’d need to scale our behavior and device workloads. At launch, our products were supported by a self-managed database, but we quickly migrated to Cloud Bigtable, and we love it.  

To efficiently support our bursty, real-time fraud detection workloads, we needed a cloud database that could satisfy the following key requirements:

• Stability to keep up with increased adoption of our products
• Intelligent scaling that avoids bottlenecks
• Native integrations with BigQuery and Cloud Dataproc
• Managed services that free up our engineers’ time to work on our products

Adding Cloud Bigtable as our performance database

As we scaled our services and added customers, our data collection services for our Device Intelligence and Behavioral Analytics products were seeing thousands of events per second. Cloud Bigtable provided a stable managed database that could handle the volume we were receiving during peak hours. We weren’t always able to handle this scale, as an early version of our product utilized a self-managed database.

Every month, two or three engineers spent hours managing the database instances. Whenever the instances crashed, it would cost at least one engineer a day or two of productivity attempting to restore the instances and recovering any data from our backup database. Managing this database internally was taking precious time away from product development.

We circled back to Cloud Bigtable. After two weeks of R&D, we decided to switch the Device Intelligence and Behavioral Analytics services to Cloud Bigtable.

Cloud Bigtable solved our scaling issues. Cloud Bigtable had been attractive to us from the start because it was fully managed, and offered regional replication and other features we were lacking in our own managed instances. Cloud Bigtable provides horizontal scaling and automatically rebalances row keys (equivalent to a shard key) over time to prevent “hot” nodes. In addition, Cloud Bigtable provides a connector to BigQuery and Cloud Dataproc that allows us to analyze the terabytes of data we are processing and use that data for unsupervised machine learning.

The perks of using Cloud Bigtable

After the migration to Cloud Bigtable, we noticed a number of additional benefits: improved I/O performance, a significant cost reduction, and a sizable decrease in hours spent on database maintenance.

We measured some of our typical metrics before and after implementing Cloud Bigtable. Our request latency dropped by about 30 ms on average (to sub-10 ms) for API requests. Prior to the change, we were seeing latencies of 40+ ms on average. This latency drop on our Behavioral Analytics and Device Intelligence products allowed us to trim about an additional 10 to 15 ms off our average response time across all dependent services.

Before we moved to Cloud Bigtable, we had to scale our database instances every time a new customer was onboarded. We were over-scaling in an attempt to avoid constantly resizing our database servers. By sunsetting our self-managed database and switching to Cloud Bigtable, we cut database infrastructure costs by approximately 35% and can now scale as needed, with a couple of clicks, during onboarding.

We have spent zero hours managing a Cloud Bigtable database since launch, and we put the time we are saving every month toward product development.

Moving forward with Cloud Bigtable

As an engineering team, we love working with Cloud Bigtable. We are not only seeing improved developer experience and reduced latency, which keeps the engineers happy, but also reduced costs, which keeps the business happy. We’re able to build more product, too, with the time we’ve saved by switching to Cloud Bigtable. Stay tuned to our engineering blog for more on the lessons we’ve learned and our contributions to the wider Cloud Bigtable community.

Editor’s note:  With Google Cloud’s datasets solution, you can access an ever-expanding resource of the newest datasets to support and empower your analyses and ML models, as well as frequently updated best practices on how to get the most out of any of our datasets. We will be regularly updating this blog with new datasets and announcements, so be sure to bookmark this link and check back often.

August 2021

New dataset: Google Cloud Release Notes

• Looking for a new way to access Google Cloud Release Notes besides the docs, the XML feed, and Cloud Console? Check out the Google Cloud Release Notes dataset. With up-to-date release notes for all generally available Google Cloud products, this dataset allows you to use BigQuery to programmatically analyze release notes across all products, exploring security bulletin updates, fixes, changes, and the latest feature releases. 
• Access the dataset

July 2021

Best practice: Use Google Trends data for common business needs

• The Google Trends dataset represents the first time we’re adding Google-owned Search data into Datasets for Google Cloud. The Trends data allows users to measure interest in a particular topic or search term across Google Search, from around the United States, down to the city-level. You can learn more about the dataset here, and check out the Looker dashboard here! These tables are super valuable in their own right, but when you blend them with other actionable data you can unlock whole new areas of opportunity for your team. To learn how to make informed decisions with Google Trends data, keep reading.
• Access the dataset

New dataset: COVID-19 Vaccination Search Insights

• With COVID-19 vaccinations being a topic of interest around the United States, this dataset shows aggregated, anonymized trends in searches related to COVID-19 vaccination and is intended to help public health officials design, target, and evaluate public education campaigns. Check out this interactive dashboard to explore searches for COVID-19 vaccination topics by region.
• Access the dataset

June 2021

New dataset: Google Diversity Annual Report 2021

• Since 2014, Google has disclosed data on the diversity of its workforce in an effort to bring candid transparency to the challenges technology companies like Google face in recruitment and retention of underrepresented communities. In an effort to make this data more accessible and useful, we’ve loaded it into BigQuery for the first time ever. To view Google’s Diversity Annual Report and learn more, check it out.
• Access the dataset

New dataset: Google Trends Top 25 Search terms

• The most popular and surging Google Search terms are now available in BigQuery as a public dataset. View the Top 25 and Top 25 rising queries from Google Trends from the past 30-days, including 5 years of historical data across the 210 Designated Market Areas (DMAs) in the US. Keep reading.
• Access the dataset

New dataset: COVID-19 Vaccination Access

• With metrics quantifying travel times to COVID-19 vaccination sites, this dataset is intended to help Public Health officials, researchers, and Healthcare Providers to identify areas with insufficient access, deploy interventions, and research these issues. Check out how this data is being used in a number of new tools.
• Access the dataset

Best practice: Leveraging BigQuery Public Boundaries datasets for geospatial analytics 

• Geospatial data is a critical component for a comprehensive analytics strategy. Whether you are trying to visualize data using geospatial parameters or do deeper analysis or modeling on customer distribution or proximity, most organizations have some type of geospatial data they would like to use – whether it be customer zipcodes, store locations, or shipping addresses. However, converting geographic data into the correct format for analysis and aggregation at different levels can be difficult. In this post, we’ll walk through some examples of how you can leverage the Google Cloud platform alongside Google Cloud Public Datasets to perform robust analytics on geographic data. Keep reading.
• Access the dataset

Get the metadata and try BigQuery sandbox 

When you’ve learned about many of our datasets and pre-built solutions from across Google, you may be ready to start querying them. Check out the full dataset directory and read all the metadata at g.co/cloud/marketplace-datasets, then dig into the data with our free-to-use BigQuery sandbox account, or $300 in credits with our Google Cloud free trial.

In the last blog post, we discussed why legacy data warehouses are not cutting it any more and why organizations are moving their data warehouses to cloud.

At GCP, we often hear that customers feel that migration is an uphill battle because the migration strategy was not deliberately considered. 

Migrating to a modern data warehouse from a legacy environment can require a massive up-front investment in time and resources. There’s a lot to think about before and during the process, so your organization has to take a strategic approach to streamline the process.

At Google Cloud, we work with enterprises shifting data to our BigQuery data warehouse, and we’ve helped companies of all kinds successfully migrate to cloud. Here are some of the questions we frequently hear around migrating a data warehouse to the cloud:

• How do we minimize any migration risks or security challenges?
• How much will it cost?
• How do we migrate our data to the target data warehouse?
• How quickly will we see equal or better performance?

These are big, important questions to ask—and have answered—when you’re starting your migration. Let’s take them in order.

How do we minimize any migration risks or security challenges?
It’s easy to consider an on-premises data warehouse secure because, well, it’s on-site and you can manage its data protection. But if scaling up an on-prem data warehouse is difficult, so is securing it as your business scales. 

We’ve built in multiple features to secure BigQuery. For enterprise users, Cloud Identity and Access Management (Cloud IAM) is key to setting appropriate role-based user access to data.

You can also take advantage of SQL’s security views within BigQuery. And all BigQuery data is encrypted at rest and in transit.

You can add the protection of customer-managed encryption keys to establish even stronger security measures. Using virtual private cloud (VPC) security controls can secure your migration path, since it helps reduce data exfiltration risks. 

How much will it cost?
The cost of a cloud data warehouse has a different structure from what you’re likely used to with a legacy data warehouse. An on-prem system like Teradata may depend on your IT team paying every three years for the hardware, then paying for licenses for users who need to access the system. Capacity increases come at an additional cost outside of that hardware budget.

With cloud, you’ve got a lot more options for cost and scale. Instead of a fixed set of costs, you’re now working on a price-utility gradient, where if you want to get more out of your data warehouse, you can spend more to do so immediately, or vice versa. While cloud data warehouses help reduce or eliminate capital and fixed costs, they are not all the same.

You’ll find varying levels of simplicity and cost savings across vendors, so it’s important to check out the operational costs of each data warehouse in relation to its performance. 

With a cloud data warehouse like BigQuery, TCO becomes an important metric for customers when they’ve migrated to BigQuery (check out ESG’s report on that), and Google Cloud’s flexibility makes it easy to optimize costs.

How do we migrate all of our data to the target data warehouse?
This question encompasses both migrating your extract, transform, load (ETL) jobs and SAS/BI application workloads to the target data warehouse, as well as migrating all your queries, stored procedures, and other extract, load, transform (ELT) jobs.

Actually getting all of a company’s data into the cloud can seem daunting at the outset of the migration journey. We know that most businesses have a lot of siloed data. That might be multiple data lakes set up over the years for various teams, or systems acquired through acquisition that handle just one or two crucial applications. You may be moving data from an on-prem or cloud data warehouse to BigQuery and type systems or representations don’t match up.

One big step you can take to prepare for a successful migration is to do some workload and use case discovery.

That might involve auditing which use cases exist today and whether those use cases are part of a bigger workload, as well as identifying which datasets, tables, and schemas underpin each use case.

Use cases will vary by industry and by job role. So, for example, a retail pricing analyst may want to analyze past product price changes to calculate future pricing. Use cases may include the need to ingest data from a transactional database, transforming data into a single time series per product, storing the results in a data warehouse table, and more. 

After the preparation and discovery phase, you should assess the current state of your legacy environment to plan for your migration. This includes cataloging and prioritizing your use cases, auditing data to decide what will be moved and what won’t, and evaluating data formats across your organization to decide what you’ll need to convert or rewrite.

Once that’s decided, choose your ingest and pipeline methods. All of these tasks take both technology and people management, and require some organizational consensus on what success will look like once the migration is complete. 

How quickly will we see equal or better performance?
Managing a legacy data warehouse isn’t usually synonymous with speed. Performance often comes at the cost of capacity, so users can’t do the analysis they need till other queries have finished running.

Reporting and other analytics functions may take hours or days, which is especially true for running large reports with a lot of data, like an end-of-quarter sales calculation. As the amount of data and number of users rapidly grows, performance begins to melt down and organizations often face disruptive outages.

However, with a modern cloud data warehouse like BigQuery, compute and storage are decoupled, so you can scale immediately without facing capital infrastructure constraints. 

BigQuery helps you modernize because it uses a familiar SQL interface, so users can run queries in seconds and share insights right away. Home Depot is an example of a customer that migrated their warehouse and reduced eight-hour workloads to five minutes. 

Moving to cloud may seem daunting, especially when you’re migrating an entrenched legacy system. But it brings the benefits of adopting technology that lets the business grow, rather than simply adopting a tool. It’s likely you’ve already seen that the business demand exists. Now it’s time to stop standing in the way of that demand and instead make way for growth.