NVIDIA and Google Cloud Pave the Way for Single-cell Genomic Analysis - Build What's Next
Blog

NVIDIA and Google Cloud Pave the Way for Single-cell Genomic Analysis

3005

Of your peers have already read this article.

2:00 Minutes

The most insightful time you'll spend today!

Google Cloud's Dataproc and NVIDIA GPUs accelerate single-cell analysis by addressing common obstacles to scientific analysis such as performance, scalability and automation. Learn how this paves way for single-cell genomic analysis in minutes.

In the past decade, the Healthcare and Life Sciences industry has enjoyed a boon in technological and scientific advancement. New insights and possibilities are revealed almost daily. At Google Cloud, driving innovation in cloud computing is in our DNA. Our team is dedicated to sharing ways Google Cloud can be used to accelerate scientific discovery. For example, the recent announcement of AlphaFold2 showcases a scientific breakthrough, powered by Google Cloud, that will promote a quantum leap in the field of proteomics. In this blog, we’ll review another omics use case, single-cell analysis, and how Google Cloud’s Dataproc and NVIDIA GPUs can help accelerate that analysis.

The Need for Performance in Scientific Analysis

The ability to understand the causal relationship between genotypes and phenotypes is one of the long-standing challenges in biology and medicine. Understanding and drawing insights from the complexity of biological systems abounds from the actual code of life (DNA) through to expression of genes (RNA) to translation of gene transcripts into proteins that function in different pathways, cells, and tissues within an organism. Even the smallest of changes in our DNA can have large impacts on protein expression, structure, and function, which ultimately drives development and response – at both cellular and organism levels. And, as the omics space becomes increasingly data- and compute-intensive, research requires an adequate informatics infrastructure. An infrastructure that scales with growing data demands, enables a diverse range of resource-intensive computational activities, and is affordable and efficient – reducing data bottlenecks and enabling researchers to maximize insight. 

But where do all these data and compute challenges come from and what makes scientific study so arduous? The layers of biological complexity begin to be made apparent immediately when looking at not just the genes themselves, but their expression. Although all the cells in our body share nearly identical genotypes, our many diverse cell types (e.g. hepatocytes versus melanocytes) express a unique subset of genes necessary for specific functions, making transcriptomics a more powerful method of analysis by allowing researchers to map gene expression to observable traits. Studies have shown that gene expression is heterogeneous, even in similar cell types. Yet, conventional sequencing methods require DNA or RNA extracted from a cell population. The development of single-cell sequencing was pivotal to the omics field. Single-cell RNA sequencing has been critical in allowing scientists to study transcriptomes across large numbers of individual cells. 

Despite its potential, and the increasing availability of single-cell sequencing technology, there are several obstacles: an ever increasing volume of high-dimensionality data, the need to integrate data across different types of measurements (e.g. genetic variants, transcript and protein expression, epigenetics) and across samples or conditions, as well as varying levels of resolution and the granularity needed to map specific cell types or states. These challenges present themselves in a number of ways including background noise, signal dropouts requiring imputation, and limited bioinformatics pipelines that lack statistical flexibility. These and other challenges result in analysis workflows that are very slow, prohibiting the iterative, visual, and interactive analysis required to detect differential gene activity. 

Accelerating Performance

Cloud computing can help not only with data challenges, but with some of the biggest obstacles: scalability, performance, and automation of analysis. To address several of the data and infrastructure challenges facing single-cell analysis, NVIDIA developed end-to-end accelerated single-cell RNA sequencing workflows that can be paired with Google Cloud Dataproc, a fully-managed service for running open source frameworks like Spark, Hadoop, and RAPIDS. The Jupyter notebooks that power these workflows include examples using samples like human lung cells and mouse brains cells and demonstrate acceleration between CPU-based processing compared to GPU-based workflows. 

Google Cloud Dataproc powers the NVIDIA GPU-based approach and demonstrates data processing capabilities and acceleration, which in turn have the potential of delivering considerable performance gains.  When paired with RAPIDS, practitioners can accelerate data science pipelines on NVIDIA GPUs, reducing operations like data loading, processing, and training from hours to seconds. RAPIDS abstracts the complexities of accelerated data science by building upon popular Python and Java libraries effortlessly. When applying RAPIDS and NVIDIA accelerated compute to single-cell genomics use cases, practitioners can churn through analysis of a million cells in only a few minutes.

Give it a Try

The journey to realizing the full potential of omics is long; but through collaboration with industry experts, customers, and partners like NVIDIA, Google Cloud is here to help shine a light on the road ahead. To learn more about the notebook provided for single-cell genomic analysis, please take a look at NVIDIA’s walkthrough. To give this pattern a try on Dataproc, please visit our technical reference guide.

Case Study

How Ather Energy is leveraging the Cloud to build and scale smart mobility solutions for India

8397

Of your peers have already read this article.

8:30 Minutes

The most insightful time you'll spend today!

Ather Energy, India’s first-ever electric scooter, turns to Google Cloud to support the smooth running of its vehicles, lower costs, improve time to market, and create a great customer experience.

In 2013, long before the world was discussing clean energy and sustainable practices, two IIT Madras graduates — Swapnil Jain and Tarun Mehta — had an idea to develop India’s first-ever electrical scooter.

This was at a time when auto manufacturers were still focusing on fossil-fuel-driven vehicles and ‘eco-friendly’ mobility solutions were more a trendy alternative catering to a niche market.

The duo founded Ather Energy in 2013 and launched their first fully-electric scooter, the Ather S340, in Bengaluru in 2016. Since then, the company has released several new models into the market and is planning to expand to eight more cities by the end of the year.

To support the smooth running of their vehicles, lower costs, improve time to market, and create great customer experience, Ather turned to Google Cloud.

Read the Full Story on YourStory

Case Study

This Chart, from Home Depot, Dramatically Demonstrates the Power of a Cloud Data Warehouse

7615

Of your peers have already read this article.

2:30 Minutes

The most insightful time you'll spend today!

When Home Depot moved it's gigantic enterprise data warehouse to Google Cloud, it could not have imagined how much faster it could crunch data--for a variety of uses cases.

The Home Depot (THD) is the world’s largest home-improvement chain, growing to more than 2,200 stores and 700,000 products in four decades. Much of that success was driven through the analysis of data. This included developing sales forecasts, replenishing inventory through the supply chain network, and providing timely performance scorecards.

However, to compete in today’s business world, THD has taken this data-driven approach to an entirely new level of success on Google Cloud, providing capabilities not practical on legacy technologies.

The Home Depot BigQuery installation performance table
Percent reduction in time that specific workloads took using BigQuery versus on-premises data warehousing.

The pressures of contemporary growth that drove much of the work are familiar to many businesses. In addition to everything it was doing, THD needed to better integrate the complexities in its related businesses, like tool rental and home services. It needed to better empower teams, including a fast-growing data analysis staff and store associates with mobile computing devices. It wanted to better use online commerce and artificial intelligence to meet customer needs, while maintaining better security.

Even before addressing these new challenges, THD’s existing on-premises data warehouse was under stress as more data was required for analytics and data analysts were utilizing the data with increasingly complex use cases. This drove rapid growth of the data warehouse, but also created constant challenges for the team in managing priorities, performance, and cost.

In order to add capacity to the environment, it was a major planning, architecture, and testing effort. In one case, adding on-premises capacity took six months of planning and a three-day service outage. Within a year, capacity was again scarce, impacting performance and ability to execute all the reporting and analytics workloads required. The capacity refresh cycles were shrinking, and the expecations for data were growing. There had to be a better way.

Still, THD did not take its move to the cloud lightly. A large-scale enterprise data warehouse migration involves tremendous effort among people, process, and technology. After careful consideration, THD chose Google Cloud’s BigQuery for its cloud enterprise data warehouse.

BigQuery, a scalable serverless data warehouse, was better on cost, infrastructure agility, and analytics capability, driving better insights with improved performance. There are no service interruptions when capacity is added, and that capacity can be added within a week (and soon same day). It doesn’t require complex system administration, and its standard SQL support means people can easily ramp up quickly. Valuable BigQuery products like Identity and Access Management meant THD could create many separate Google Cloud projects, while ensuring that different teams weren’t interfering with each other or accessing protected data.

THD also utilizes BigQuery’s flat-rate monthly pricing model that allows teams to budget their capacity based on need and provides billing predictability. The capacity not being used by a given project is available for enterprise use. This ensures no surprises when the monthly bill arrives and provides all analytical users access to significant computing power.

While THD’s legacy data warehouse contained 450 terabytes of data, the BigQuery enterprise data warehouse has over 15 petabytes. That means better decision-making by utilizing new datasets like website clickstream data and by analyzing additional years of data.

As for performance, look at this chart:

With the cloud EDW migration complete, and the legacy on-premises data warehouse retired, analysts now execute more complex and demanding workloads that they would not have been able to complete before, such as utilizing Datalab for orchestrating analytics through Python Notebooks, utilizing BigQuery ML for machine learning directly against the BigQuery data (no movement of large datasets), and AutoML to help determine the best model for predictions.

Additionally, engineers at THD have adapted BigQuery to monitor, analyze, and act on application performance data across all its stores and warehouses in real time, something that was not practical in the on-premises system.

With over 600 projects that THD now has on Google Cloud, the BigQuery story is just one of the many ways that Google Cloud is working with THD to deliver meaningful business results, every day.

2996

Of your peers have already watched this video.

34:30 Minutes

The most insightful time you'll spend today!

Explainer

Reinventing geospatial analytics as only Google can do: Intelligent platform

Geospatial analytics is undergoing a fundamental change with the application of cloud computing products. Every day, data analysts need to understand the geospatial impact on their data and GIS professionals are finding themselves constrained by legacy technology that can’t keep pace with increasing data volume.

Google Cloud’s geospatial analytics platform brings the full suite of Google’s Geo offerings to both sets of users to unblock workloads and free them from legacy capacity constraints.

Listen to Joe Bettridge, Consulting Lead, TELUS Insights, TELUS and Chad Jennings, Product Manager and GIS Lead, Google Cloud, and learn about Google Cloud’s uniquely powerful, full–platform solution for geospatial analytics and how organizations are using it to reduce query times from hours, days, or months to minutes. These workloads can span both structured and imagery data.

In this session, they will cover:

  • An overview of BigQuery GIS
  • Some use cases
  • Examples of algorithms TELUS is using

10526

Of your peers have already watched this video.

2:15 Minutes

The most insightful time you'll spend today!

Case Study

The True Story of How HotStar Broke a World-Record–Thanks to Firebase and Google BigQuery

Hotstar, India’s largest video streaming platform with 150 million monthly active users around the world, provides live-streaming of TV shows, movies, sports, and news on the go.

By using a combination of Firebase products together, Hotstar safely rolled out new features to its watch screen during a major live-streaming event without disrupting users, sacrificing stability, or releasing a new build. They also used Firebase with BigQuery to analyze their event data and reduce app startup time.

“We have an ambitious mission, but our engineering team is only a fraction of the size of most of our competitors. But we are still keeping up, and we are doing it with the help of Firebase,” says Ayushi Gupta, Android Engineer, Hotstar.

3634

Of your peers have already watched this video.

5:00 Minutes

The most insightful time you'll spend today!

How-to

Quick Demo: How to Push Data to PubSub, Dataflow, and BigQuery to Improve Customer Experience

How can you use big data to raise your business’ customer experience levels? It’s a question many organizations ask themselves. Some of the more forward-looking ones already know which data points can be leveraged—they just don’t know how to put together the different pieces of the big data puzzle together.

Imagine for minute that you run an airline. You already collect huge amounts of customer data that you would like to leverage to improve both revenue and customer satisfaction.

Here’s an example: Customers taking a Monday morning flights from certain hubs would indicate that they are business users. These customers would appreciate being offered in-flight Wifi. Such a targetted offer would increase both customer experience and revenues.

In this five-minute demo, you’ll learn how easy it is to query data sets to reveal, for example, which passengers have connecting flights—and also have checked-in luggage—and would therefore appreciate an upgrade. The walkthrough will show you how to leverage Cloud IoT Core to push airline data to PubSub, Dataflow, and BigQuery.

Watch it now!

More Relevant Stories for Your Company

Blog

Transform ‘Dark Data’ from Documents with Document AI, Cloud Functions and Workflows

At enterprises across industries, documents are at the center of core business processes. Documents store a treasure trove of valuable information whether it's a company's invoices, HR documents, tax forms and much more. However, the unstructured nature of documents make them difficult to work with as a data source. We

Blog

Google and AI Researchers Work towards Building Data-centric AI

AI researchers and engineers need better data to enable better AI solutions. The quality of an AI solution is determined by both the learning algorithm (such as a deep-neural network model) and the datasets used to train and evaluate that algorithm. Historically, AI research has focused much more on algorithms

Blog

Partnering with Google Cloud is the Key Behind Recent Healthcare Innovations

It’s simply amazing to witness how some of our systems integrators employ Google Cloud solutions to drive innovation in ways we at Google may never have considered—especially in healthcare.  According to analyst firm MarketsandMarkets, the market for the Cloud in healthcare is projected to grow 43% between 2020 and 2025

How-to

Why and How to Migrate to Google BigQuery

Over the past few decades, organizations have mastered the science of data warehousing. They have increasingly applied descriptive analytics to large quantities of stored data, gaining insight into their core business operations. Conventional Business Intelligence (BI), which focuses on querying, reporting, and Online Analytical Processing, might have been a differentiating factor

SHOW MORE STORIES