Whether with the cloud or within their own data centers, enterprises have undergone a period of remarkable consolidation and centralization of their compute resources. But with the rise of ever more powerful mobile devices, and increasingly capable cellular networks, application architects are starting to think beyond the confines of the data center, and looking out to the edge. 

What exactly do we mean by edge? Think of the edge as distributed compute happening on a wide variety of non-traditional devices — mobile phones of course, but also equipment sensors in factories, industrial equipment, or even temperature and reaction monitoring in a remote lab. Edge devices are also connected devices, and can communicate back to the mothership over wireless or cellular networks. 

Equipped with increasingly powerful processors, these edge devices are being called upon to perform tasks that have thus far been outside the scope of traditional IT. For enterprises, this could mean pre-processing incoming telemetry in a vehicle, collecting video in kiosks at a mall, gathering quality control data with cameras in a warehouse, or delivering interactive media to retail stores. Enterprises are also relying on edge to ingest data from outposts or devices that have even more intermittent connectivity, e.g., oil rigs or farm equipment, filtering that data to improve quality, reducing it to right-size information load, and processing it in the cloud. New data and models are then pushed back to the edge; in addition, we can also push configuration, software, and media updates and decentralize processing workload.

Edge isn’t all about enabling new use cases – it’s also about right-sizing environments and improving resource utilization. For example, adopting an edge model can also relieve load on existing data centers. 

But while edge computing is full of promise for enterprises, there are many pieces that are still works in progress. Further, developing edge workloads is very different from developing traditional applications, which enjoy the benefits of persistent data connections and run on well-resourced hardware platforms. As such, cloud architects are still in the early days of figuring out how to use and implement edge for their organizations. 

Fortunately, there are tools you can use to help ease the transition to edge computing — and that likely fit into your organization’s existing computing systems. Kubernetes, of course, but also higher level management tools like Anthos, which provides a consistent control plane across cloud, private data center and edge locations. Other parts of the Anthos family – Anthos Config Management and Anthos Service Mesh — go one step further and provide consistent, centralized management to your edge and cloud deployments. And there’s more to come.

For the remainder of this blog post, we’ll dive deeper into the past and current state of edge computing, and the benefits that architects and developers can expect to see from edge computing. In a next post, we’ll take a deeper look at some of the challenges that designing for edge introduces, and some of the advantages the average enterprise has in adopting the edge model. Finally, we’ll look at the Google Cloud tools that are available today to help you build out your edge environment, and look at some early customer examples that highlight what’s possible today — and that will spark your imagination for what to do tomorrow. 

The evolution of edge computing 

The edge is not a new concept. In fact, it’s been around for the last two decades, spanning many use cases that are prevalent today. One of the first applications for edge was to use content delivery networks (CDN) to cache and serve daily static website pages near clients, for example, web servers in California data centers serving financial data to European customers. 

As connectivity has improved and software evolved, the edge has evolved too, and the focus has shifted towards using edge to distribute services. First, simple services expanded from static HTML to javascript libraries or image repositories. Common functions like image transformation, credit and address validation support services followed. Soon, organizations were deploying more complex cloudlet and clustered microservices installations, as well as distributed and replicated datasets. The term “endpoint” became ubiquitous, and APIs profilerated. 

In parallel, there’s been an explosion of creativity in hardware, microcontrollers and dedicated edge devices. Fit-for-purpose products were deployed globally. Services like Google Cloud IoT Core extended our ability to manage and securely connect these dispersed devices, allowing platform managers to register tools and leverage managed services like Pub/Sub and Dataflow for data ingestion. And with Kubernetes, large remote clusters — mini private clouds in and of themselves — operate as self-healing, autoscaling services across the broader internet, opening the door to new models for applications and architectural patterns. In short, both distributed asynchronous systems and economies have blossomed.

What does this mean for enterprises? For the purposes of this series, edge means you can now go beyond the corporate network, beyond cloud VPCs, and beyond hybrid. The modern edge is not sitting at a major remote data center, nor is it a CDN, cloud provider, or in a corporate data center rack — it’s just as likely to look like 100 of these attached to a thousand sensors.

Edge, in short, is about having hardware and devices installed at remote locations that can process and communicate back the information they collect and generate. The edge management challenge, meanwhile, is being able to push configuration and software/model/media updates to these remote locations when they are connected.

Enable new use cases

Today, we have reached a new threshold for edge computing — one where micro-data-processing centers are deployed as the edge of a fractal arm, as it were. Together, they form a broad, geographically distributed, always-on framework for streaming, collecting, processing and serving asynchronous data. This big, loosely coupled application system lives, breathes and grows. Always changing, always learning from the data it collects — and always pushing out updated models when the tendrils are connected. 

Right now, the rise of 5G is pushing the limits of edge even further. Devices enabled with 5G can transmit using a mobile network — no ISP required — enabling connectivity anywhere within reach of a cell tower. Granted, these networks have lower bandwidth, but they are often more than adequate for certain types of data, for example fire sensors in forests bordering remote towns that emit temperature or carbon monoxide data periodically. Recently, Google Cloud partnered with AT&T to enhance business use of 5G edge technology but there is so much more that can be done. 

Reduce data center investments

In addition to enabling the digitization of a broad range of new use cases, adopting edge can also benefit your existing data center.

Let’s face it: data centers are expensive to maintain. Moving some data center load to edge locations can reduce your data center infrastructure investment, as well as compute time spent there. Edge services tend to have much lower service level objectives (SLOs) than data center services, driving lower levels of hardware investment. Edge installations also tend to tolerate disconnectedness, and thus function perfectly well with lower SLOs — and lower costs. 

Let’s look at an example of where edge can really reduce costs: big data. Back in the day, we used to build monolithic serial processors — state machines — that had to keep track of where they were in processing in case of failure. But time and again, we’ve seen that smaller, more distributed processing can break down big, expensive problems into smaller, more cost-effective chunks. 

Starting with the explosion of MapReduce almost 20 years ago, big-data workloads were parallelized across clusters on a network, and state management was simplified with intermediate output to share, wait for, or restart processing from checkpoints. Those monolithic systems were replaced by cheaper, smarter, networked clusters and data repositories where parallel work could be executed and rendered into workable datasets. 

Flash forward to today, and we are seeing those same concepts applied and distributed to edge data-collection points. In this evolution of big data processing, we are scaling up and out to the point where observation data is so massive that it must first be prefiltered, and then preprocessed down to a manageable size and still be actionable. Only then should it be written back to the main data repositories for more resource-intensive processing and model building.

In short, data collection, cleanup, and potentially initial aggregation happens at the edge location, which reduces the amount of junk data sitting in costly data stores. This increases performance of the core data warehouse, and reduces the size and cost of network transfers and storage! 

The edge is a huge opportunity for today’s enterprises. But designing environments that can make effective use of the edge isn’t without its challenges. Stick around for part two of this series, where we look at some of the architectural challenges typically encountered while designing for the edge and how we begin to address them.

Financial institutions across the globe rely on real-time indices to inform real-time portfolio valuations, to provide benchmarks for other investments, and as a basis for passive investment instruments including exchange-traded products (ETPs). This reliance is growing—the index industry dramatically expanded in 2020, reaching revenues of $4.08 billion.

Today, indices are calculated and distributed by index providers with proximity and access to underlying asset data, and with differentiating real-time data processing capabilities. These providers offer subscriptions to real-time feeds of index prices and publish the constituents, calculation methodology, and update frequency for each index.

But as new assets, markets, and data sources have proliferated, financial institutions have developed new requirements. Financial institutions will need to quickly create bespoke and frequently updating indices that represent a specific actual or theoretical portfolio, with its unique constituents and weightings.

In other words, existing index providers and other financial institutions alike will need mechanisms for rapid creation of real-time indices. This blog post’s focus—an index publication pipeline collaboratively developed by CME Group and Google Cloud—is an example of such a mechanism. 

The pipeline closely approximates a particular CME Group index benchmark, but with far greater frequency (in near real time vs. daily) than its official counterpart. It does so by leveraging open-source models such as Apache Beam and cloud-based technologies such as Dataflow, which automatically scales pipelines based on inbound data volume.

Machine learning’s production problem

In the past decade, advances in AI toolchains have enabled faster ML model training—and yet a majority of ML models are still not making it into production. As organizations endeavor to develop their ML capabilities, they soon realize that a real-world ML system is comprised of a small amount of ML code embedded in a network of complex and large ancillary components. Each component brings its own development and operational challenges, which are met by bringing a DevOps methodology to the ML system, commonly referred to as MLOps (Machine Learning Operations). To apply ML to business problems, a firm must develop continuous delivery and automation pipelines for ML.

This index publication collaboration is instructive because it demonstrates MLOps best practices for just such a pipeline. One Apache Beam pipeline, suited for operating on both batch and streaming data, extracts insights and packages them for downstream consumers. These consumers may include ML pipelines that, thanks to Apache Beam, require only one code path for inference across batch and real-time data sources. The pipeline is run inside Google Cloud’s Dataflow execution engine, greatly simplifying management of underlying compute resources. 

But the collaboration’s value is not constrained to the ML and data science realm. The project shows that consumers of the Apache Beam pipeline’s insights may also include traditional business intelligence dashboards and reporting tools. It also demonstrates the simplicity and economy of cloud-based time series data such as CME Smart Stream, which is metered by the hour, quickly and automatically provisioned, and consumable at a per-product-code (not per-feed) level.

A focus on real-time processing for financial services

To illustrate the above points, the collaboration applies data engineering and MLOps best practices to a financial services problem. We chose the financial services domain because many financial institutions do not yet have real-time market data processing or MLOps capabilities today, owing to a significant gap on either side of their ML/AI objectives.

Upstream from ML/AI models, financial institutions often experience a data engineering gap. For many financial institutions, batch processes have sufficiently addressed business requirements. As a result, the temporal nature of the time series data underlying these processes is deemphasized. For example, the original purpose of most trade booking systems was to capture a trade and ensure that it found its way to the middle and back office for settlement. It was not built with ML/AI in mind, and its underlying data therefore has not been packaged for consumption by ML/AI processes. 

And downstream from ML/AI models, financial institutions often encounter the aforementioned “ML production problem.”

As ML/AI becomes ever more strategic, these two gaps have left many financial institutions in a conundrum—unable to train ML models for lack of properly packaged time series data, and unmotivated to package time series data for lack of ML models. By recreating a key energy market index using open-source libraries and cloud-based tools, this collaboration demonstrates that for the financial services domain a solution to this conundrum is more accessible today than ever. 

Creating a new index

We modeled our new index after one of CME Group’s many index benchmarks. The particular index expresses the value of a basket of three New York Mercantile Exchange—listed energy futures as a single price. Today, CME Group publishes the index at the end of the day by calculating the settlement price of each underlying futures contract, and then weighing and summing these values. 

While CME Group does not currently publish this index in real time, this collaboration aims to create a near real-time solution leveraging Google Cloud capabilities and CME Group market data delivered via CME Smart Stream. However, in order to publish the value so frequently—every five seconds, with 40-second publish latency—this collaboration’s pipeline has to solve a number of challenges in near-real time.

First, the pipeline must process sparse data from three separate trades feeds in memory to create open-high-low-close (OHLC) bars. More specifically, for five-second windows for each of the three front-month (and sometimes second-month) energy contracts, a bar must be produced. This is solved by using the Apache Beam library to implement functions which, when executed on Dataflow, automatically scale out as input load increases. The bars must be time-aligned across the underlying feeds, which is greatly simplified by Beam’s watermark feature. And for intervals in which no tick data is observed, the Beam library is used to pull forward the last value received, yielding perfect gap-free bars for downstream processors.

Second, the pipeline must calculate volume-weighted average price (VWAP) in near real-time for each front-month contract. The VWAP calculations are also written using the Beam API and executed on Dataflow. Each of these functions requires visibility of each element in the time window, so the functions cannot be arbitrarily scaled out. Nonetheless, this is tractable because their input—OHLC bars—is manageably small.

Third, the pipeline must replicate CME Group’s specific settlement price methodology for each contract. The rules specify whether to use VWAP or another source as price, depending on certain conditions. They also specify how to weigh combinations of monthly contracts during a roll period. The pipeline again encapsulates these requirements as an Apache Beam class, and joins the separate price streams at the correct time boundary.

The end result is a new stream publishing bespoke index data to a Google Cloud Pub/Sub topic thousands of times daily, enabling AI models as well as traditional industry index usage, dashboards, and other tools to assist real-time decision making. The stream’s pipeline uses open source libraries that solve common time series problems out-of-the box, and cloud-based services to reduce the user’s operational and scaling burden.

The importance of cloud-based data

The promise of cloud-based pipeline execution services cannot be realized using legacy data access patterns, which often require market data users to colocate and configure servers and network gear. Such patterns inject expense and scaling complexity into the pipeline’s overall operation, diverting resources from the adoption of MLOps best practices. Instead, a newer, cloud-based access pattern—in which resources subscribe to data streams inexpensively, rapidly and programatically—is necessary.

In 2018, CME Group identified the customer need for accessible futures and options market data. CME Group collaborated with Google Cloud to launch CME Smart Stream, which distributes CME Group’s real-time market data across Google Cloud’s global infrastructure with sub-second latency. Any customer with a CME Group data usage license and a Google Cloud project can consume this data for an hourly usage fee, without purchasing and configuring servers and network gear.

CME Smart Stream met this index pipeline’s requirements for cost-effective, cloud-based streaming data, but this is just one use case. Since the launch of a CME Smart Stream offering on Google Cloud, globally dispersed firms have adopted the solution. For example, Coin Metrics has been using the offering to better inform its customers in the crypto markets. According to CME Group, Smart Stream has become popular with new customers as the fastest, simplest way to access CME Group’s market data from anywhere in the world. 

Adapt the design pattern to your needs

By combining cloud-based data, open-source libraries, and cloud-based pipeline execution services, we created a real-time index using the same constituents as its end-of-day counterpart. Additionally, financial institutions will find this approach addresses many other challenges—real-time valuation of a large set of portfolios; benchmark creation for new ETPs; or external publication of new indices.

Give it a try

This approach is available to help you meet your organization’s needs. Please review our user guide, whose Tutorials section provides a step-by-step guide to constructing a simple Apache Beam pipeline to generate metrics on streaming data in real-time, and connecting a new data source to the pipeline. We’ll be discussing this topic in CME Group’s webinar End-to-End Market Data Solutions in the Cloud at 10:30 am ET on June 16th.

At Quantum Metric, we’re in the business of bringing our customers business insights that are based on customer experience data and analytics for mid-market and Fortune 500 companies.

Our software, powered by big data, machine intelligence, and Google Cloud, helps our customers identify, quantify, prioritize and measure opportunities to improve digital experiences.

As companies move to a more agile product lifecycle, including continuous deployment and continuous integration, they’re finding that it’s critical to receive perpetual quantified feedback and insights from their data in real time to understand where the largest opportunities exist. 

Each year, billions of customer interactions are captured through browsers or mobile apps on PCs, tablets, and mobile devices. This data, fed into the Quantum Metric platform, can show if a customer had a password problem they couldn’t solve or struggled when trying to purchase something and abandoned their cart.

It also can show if the customer tried in vain to complete an online change to their service provider’s subscription, to reach tech support, or couldn’t find the size or color they were looking for while shopping online.

Most importantly, the Quantum Metric platform quantifies the business value of the issue, helping organizations prioritize where they can make the largest impact to their business.

Success overwhelms our initial architecture
Initially, the Quantum Metric experience analytics software ran on a MySQL open source relational database management system (RDBMS). The MySQL RDBMS worked great for simple queries, when there was a specific question to ask of the data.

Soon, though, we knew we needed to offer more advanced data science capabilities. Our bigger customers wanted to ask questions across very large data sets—days, weeks, months, and years worth of data. They wanted to pose iterative questions using complex filters to answer their most challenging business questions. 

With more complex queries across more data, response times from our RDBMS went from 100 to 500 milliseconds to as long as 20 minutes.

That delay was slowing down our ability and time to insights, which also reduced the value we could provide to our customers, since iterative exploration and analysis requires real-time query responses. Because of the need for real-time responses, there were certain questions that we just weren’t able to ask of the data. It became clear that we needed a much more robust data warehouse solution. 

There were also operational challenges with MySQL and massive-scale data ingestion. We spent a lot of time into the wee hours of the night and morning handling errors and recovering databases.

We tried to address these challenges by sharding, partitioning, and indexing the data to optimize for the types of questions customers were asking. But the problems were escalating and happening more often, from once a month across the customer base to monthly for at least 20 different customers.

We could tune the platform for today and tomorrow’s workload, with good guesses at where indexes could be used, but we simply couldn’t continue to horizontally scale MySQL in a cost-efficient and operationally efficient manner. 

Speed breeds innovation
Once we started exploring options that could better scale with our business, we looked at NoSQL technologies like Cassandra (a partitioned row-store database), MySQL’s Column Store (a columnar store database), and Vertica (a columnar store database)—each with unique ways of handling data storage and accessibility.

But with high volumes of complex queries across large data stores, all of these solutions began to fail, bogged down with multiple, simultaneous users. We could have solved the problems with more raw compute and storage, but it would have been prohibitively expensive to run and require a large team to operate. 

We then decided to try BigQuery, and it was transformative.

We connected our front end to BigQuery via APIs. Once data is 15 minutes old, it is automatically extracted, loaded, and transformed (ETL) to BigQuery.

We continuously update the legacy MySQL RDBMS so its data is integrated with BigQuery data when queries require real-time data. Most query response times are within 100-200 milliseconds, matching what we initially experienced with MySQL.

When traffic from our customers scales up, we can now scale on-demand to accommodate it, thanks to BigQuery’s hundreds of thousands of CPUs. Our customers no longer run into slow response times, and we’ve gained confidence that we can offer them—and their users—advanced insights and better experiences without delay.

More importantly, with this scale of query power, we were able to build data science algorithms into the platform, which iteratively query BigQuery based on the results, and help quantify the impact of a specific issue to a specific segment of users. Adding these capabilities was possible because of the massive scale of BigQuery. 

In addition to new insights and fast response times, we wanted our customers to be able to ask complex questions using very simple language.

For example: “Show me high-loyalty customers, located in specific geographic areas, who visited the web site at least five times, based on specific campaigns, and never booked a seat on a flight.”

This was exactly the kind of query that was used by a major U.S. airline to understand the multi-million dollar impact of a failure affecting their most valuable customers: their high-loyalty members.

And this was all done while maintaining the highest standard of care of customer data and privacy by default, using multiple layers of encryption of data in transit, at rest, and a unique military-grade encryption approach. This approach encrypts PII, including even session cookies, with a RSA-2048 key available only to a select few and used for use cases such as fraud analysis.   

It’s no exaggeration to say that BigQuery has totally transformed our business. It provides the petabyte scale and speed we were missing, in addition to taking care of operational maintenance, a task that was burying our team with MySQL.

We’re now able to support some of the largest companies in the world that require real-time, petabyte-scale analytics. That lets them serve more customers faster with higher quality, and take advantage of BigQuery’s power and scale to innovate.

There are other cloud solutions that can address petabyte analytics, but the most unique value proposition of BigQuery was its on-demand scaling and operational management, with extremely cost-effective pay-as-you-go billing. While today we are at a scale where we have round-the-clock querying needs, our early days had very sporadic query loads where we needed instant scale, then a long lull of nothing. The unique business model of BigQuery’s pay-by-bytes-scanned allowed us to have access to a massive-scale querying platform without breaking the bank. 

Using BigQuery powers better customer experience and reduces purchasing friction
Among the many features of Quantum Metric is the ability to replay online customer sessions. In the example below from a mobile e-commerce site, each action is displayed chronologically. Why did this customer’s transaction fail?

Diving deeper, the session replay shows that the user tried to change the item quantity in the checkout cart, which resulted in a failed API call. Powered by BigQuery, Quantum Metric can then show how many other end users had this issue, with a simple click of “Show More Errors Like This.”

With BigQuery’s massive scale, Quantum Metric will then quantify the impact of that issue, so companies can prioritize which issues need attention immediately. If this is the issue that’s impacting the business the most, our customer can use a single click to open a Jira ticket, forwarding the discovery to their product and engineering teams. Those teams can then re-engineer the experience in near-real time, addressing the failed API call and cutting out the frustrating time it takes for engineers to reproduce the issue.

Once we had a powerful back end in BigQuery, we realized that Quantum Metric’s platform could be used to ask complex questions from vast datasets. We built some of the processes that data scientists use to formulate those queries right into our product.  

For example, we added one series of processes to our platform to help customers understand whether a suspect issue is really impacting end-user experience. Is it something that should be prioritized and fixed? Does this really affect the user experience? Does it have financial impact? These and other questions can be pre-defined as a complex query in Quantum Metric to let our customer quickly gain insight on how an issue is impacting the business. Customers were blown away when they heard this was possible. It was the holy grail for what they were looking for in data science. It really sets us apart from our competition.

Today, with every company heavily dependent on data, those companies that can uncover and act on insights fastest are the ones that will succeed. BigQuery gives us the data warehouse platform we need to provide our customers with fast, reliable technology tools. It frees us from having to deal with the minutiae of technology infrastructure operations, so we can focus on finding and extracting the magic in customer data. With the power and scale of BigQuery, combined with the real-time capture of every user experience with 100% fidelity, we’re able to offer a self-service analytics platform that provides insights into digital journey friction points and acts as the indisputable arbitrator of truth. 

The COVID-19 pandemic continues to push the healthcare and life sciences industry in entirely new ways. In record time, we’ve witnessed public health officials, vaccine developers, equipment manufacturers, and essential workers take life saving actions—regularly putting their own lives at risk—to respond to the exceptional challenges of our time. 

Yet after all the turmoil and uncertainty of the pandemic, record breaking levels of investment continue to come into the market to fuel innovations. 

Vaccine development is now measured in weeks rather than years; providers are leveraging telehealth technologies to improve the physician and patient experience; and individuals have embraced a variety of devices to assume greater ownership and control of their personal health. 

With that in mind, here are five of the many innovations I see driving healthcare and life sciences for at least the next 12 months: 

1. Unleashing the power of healthcare and life sciences data

People have arguably never had as much access and understanding to their personal health data than they can in 2022. They have the ability to understand their genetic makeup, medical history, family history, and activity levels to ensure they are living a healthy lifestyle. Taken together, this longitudinal profile can become the basis for more personalized medicine. Add wearable devices to the mix and patients gain real- or near-real-time updates on their health status. 

Further, global regulatory agencies have continued to mandate the need for providers to maintain a longitudinal health record that provides a holistic view of the patient across all the encounters they have had with a health system. Physician surveys conducted by the The Harris Poll and Google Cloud show that a 360-degree view of the patient, across all provider encounters, leads to faster, more accurate diagnosis, and better outcomes. 

If secure data access and interoperability can begin to include insurers, researchers, public health officials, and others in the field, the powerful network effects benefiting patients will only grow. When combined with those longitudinal phenome profiles, the opportunities for personalized and preventative medicine enter a whole new era.

2. Healthcare and life sciences M&A boom continues

Although the pandemic initially slowed activity on mergers and acquisitions in the early part of 2020, the healthcare and life sciences industry has seen a rapid rebound and acceleration of deals ever since. The success of COVID-19 vaccines, the importance of telehealth, and the focus on molecular modeling and genomic-based drug development have all boosted investment as organizations look to enter new markets, develop new therapies, and leverage low interest rates while they last. 

In 2021, the total funding of US digital health startups surpassed $29 billion across 729 deals, according to advisory Rock Health. That’s almost double the levels of 2020, which itself set records. Analysts at PWC meanwhile estimate M&A investments in biopharmaceutical and life sciences could approach $400 billion this year across all sub-sectors

Clearly, the pandemic has been a primary driver of investment as the focus on healthcare has dramatically increased. Yet the increased activity also reflects changing business models and emerging technologies that are now required to compete in the rapidly evolving space. For organizations to capitalize on these investments, it will take not only great vision and intellectual property but also the right technologies—like cloud—and the right data interoperability models, to make partnerships and acquisitions more scalable, feasible, and seamless.

3. Transforming the patient experience at a new rate

The pandemic has shined a spotlight on the inefficiencies and complexities that exist in healthcare markets across the globe. As wave after wave has surged, global healthcare systems remain overwhelmed on most every aspect of patients’ treatment journeys. Even before COVID-19, healthcare was already one of the largest spend areas for governments around the world. The pandemic has only exacerbated the known issues. 

Clearly, administrators, regulators, physicians, nurses, and patients would agree that the processes and models need to change. There’s a need to maintain this momentum and even increase the tempo to achieve lasting change.

Take telehealth. Within months of the start of the pandemic, providers moved to provide more remote capabilities so physicians could still meet with patients virtually to ensure health and safety on all sides. Payers recognized the importance of telehealth and began to update reimbursement rules. And organizations are now reimagining policies in areas such as prior authorization, submission, and adjudication to reduce complexity and bureaucracy while improving responsiveness.

Looking beyond the system, organizations are also recognizing and deepening their understanding of the structural and social determinants of health that impact patient care and health outcomes, especially for historically underserved communities.  Private and public sectors are learning from, and increasingly partnering with, the social sciences, public health, biomedical informatics, computer science, public policy and community groups around how to build a mI’ore equitable and inclusive consumer products and Health IT strategies. 

The newfound levels of transparency, visibility, and accountability that patients, caregivers, and organizations are achieving will ultimately increase competition and provide a more effective, equitable, efficient and, above all, healthier marketplace for all patients. As we move past the worst of the pandemic, regulators and organizations should keep fighting for progress over business as usual.

4. AI is now a core competency for Drug Development

The ability of organizations like Pfizer, Moderna, Johnson & Johnson, and Astrazeneca to develop COVID-19 vaccines has been a remarkable accomplishment—particularly the historic speed with which they were created and deployed. This innovation acceleration was largely enabled by the use of new drug development platforms that allow researchers to use artificial intelligence and machine learning to model protein and cellular interactions to rapidly advance the science.

No longer must researchers rely on traditional laboratory testing (and retesting). With their improved understanding of the molecular and genetic structure of a patient and, for example, their tumor, researchers can use AI to enable simulations on computers rather than testing in live conditions. This technology can process thousands, even millions of simulations to help  identify high-potential candidates for treatment consideration and subsequent analysis. 

AI-enabled drug discovery models can eliminate months and years from the research process, which can reduce the time to develop a drug and accelerate the time to treatment for an individual patient. As just one example, consider the work on AlphaFold2 by Google’s DeepMind unit who leverages AI to predict effective protein shapes for new drugs. Healthcare and life sciences organizations already recognize the potential of AI. Now comes the investments to leverage this rapidly evolving technology to support their efforts now and in the future.   

5. Ecosystem partnerships tackle complexity and spur innovation

As the importance and growth of the healthcare and life sciences industry continues, we will see even more new players and partnerships emerging to address old problems in new ways. This trend will touch all aspects of the healthcare value chain and will, increasingly, see three- and four-player partnerships emerge to address the complex challenges of today’s healthcare marketplace.

Technology will continue to play a key role as capabilities and platforms will transform all aspects of the marketplace. Cell phones, wearable devices, and other technologies will provide real-time updates and notifications to patients on everything from glucose levels to payments for healthcare services. Voice recognition software will document physician and patient discussions to reduce the burden of record keeping. Real-world data will be used to simplify and confidentially recruit patients for participation in clinical trials. 

New players will continue to enter the market to improve health outcomes and reduce costs. Major retailers are among the companies extending their pharmacies to provide additional diagnostic and concierge services, saving patients from additional appointments while boosting prevention. Community organizations are emerging to help identify and care for underserved communities whose health outcomes are significantly lower than the average patient. 

For all the exhausting and heart-wrenching challenges of the past two years, the opportunities the pandemic has laid bare cannot be overlooked. We owe it to those who have worked and fought so hard for every life to forge even more new partnerships—and make it easier to do so—so that the next crisis, when it does arise, will never be as bad as the one we’re now conquering. Technology can be the enabler in this effort and help bring us together to continue to conquer the challenges that lie ahead.