Alphabet CEO Sundar Pichai has compared the potential impact of artificial intelligence (AI) to the impact of electricity—so it may be no surprise that at Google Cloud, we expect to see increased AI and machine learning (ML) momentum across the spectrum of users and use cases.

Some of the momentum is more foundational, such as the hundreds of academic citations that Google AI researchers earn each year, or products like Google Cloud Vertex AI accelerating ML development and experimentation by 5x, with 80% fewer lines of code required. Some are more concrete, like mortgage servicer Mr. Cooper using Google Cloud Document AI to process documents 75% faster with 40% cost savings; Ford leveraging Google Cloud AI services for predictive maintenance and other manufacturing modernizations; and customers across a wide range of industries deploying ML platforms atop Google Cloud.

Together, these proof points reflect our belief that AI is for everyone, and that it should be easy to harness in workflows of all kinds and for people of all levels of technical expertise. We see our customers’ accomplishments as validation of this philosophy and a sign that we are taking away the right things from our conversations with business leaders. Likewise, we see validation in recognition from analysts, which recently includes Google being named a Leader by

Gartner® in the 2022 Magic Quadrant™ for Cloud AI Developer Services report

Forrester in the Forrester Wave™: AI Infrastructure, Q4 2021 report, the Forrester Wave™: Document-Oriented Text Analytics Platforms, Q2 2022 report, and The Forrester Wave™: People-Oriented Text Analytics Platforms, Q2 2022 report

In June, we talked about four pillars that guide our approach to creating products for MLOps and to accelerate development of ML models and their deployment into product. In this article, we’ll look more broadly at our AI and ML philosophy, and what it means to create “AI for everyone.”

AI should be for everyone

One of the pillars we discussed in June was “meeting users where they are,” and this idea extends far beyond products for data scientists. Technical expertise should not be a barrier to implementing AI—otherwise, use cases where AI can help will languish without modernization, and enterprises without well-developed AI practices will risk falling behind their competitors.

To this end, we focus on creating AI and ML services for all kinds of users, e.g.:

• DocumentAI, Contact Center AI, and other solutions that inject AI and ML into business workflows without imposing heavy technical requirements or retraining on users;
• Pre-trained APIs, ranging from Speech to Fleet Optimization, that let developers leverage pre-trained ML models and free them from having to develop core AI technologies from scratch;
• BigQuery ML to unite data analysis tasks with ML;
• AutoML for abstracted and low-code ML production without requiring ML expertise;
• Vertex AI to speed up ML experimentation and deployment, with every tool you need to build deploy and the lifecycle of ML projects
• AI Infrastructure options for training deep learning and machine learning models cost effectively. Including Deep Learning VMs optimized for data science and machine learning tasks and AI accelerators for every use case, from low-cost inference to high-performance training.

It’s important to provide not only leading tools for advanced AI practitioners, but also leading AI services for users of all kinds. Some of this involves abstracting or automating parts of the ML workflow to meet the needs of the job and technical aptitude of the user. Some of it involves integrating our AI and ML services with our broader range of enterprise products, whether that means smarter language models invisibly integrated into Google Docs or BigQuery making ML easily accessible to data analysts. Regardless of any particular angle, AI is turning into a multi-faceted, pervasive technology for businesses and users the world over, so we feel technology providers should reflect this by building platforms that help users harness the power of AI by meeting them wherever they are.

How we’re powering the next generation of AI

Creating products that help bring AI to everyone requires large research investments, including in areas where the path to productization may not be clear for years. We feel a foundation in research combines with our focus on business needs and users to inform sustainable AI products that are in keeping with our AI principles and encourages responsible use of AI.

Many of our recent updates to our AI and ML platforms began as Google research projects. Just consider how DeepMind’s breakthrough AlphaFold project has led to the ability to run protein prediction models in Vertex AI. Or how research into neural networks helped create Vertex AI NAS, which lets data science teams train models more accurately with lower latency and power requirements.

Research is crucial, but also only one way of validating an AI strategy. Products have to speak for themselves when they reach customers, and customers need to see their feedback reflected as products are iterated and updated. This reinforces the importance of seeing customer adoption and success across a range of industries, use cases, and user types. In this regard, we feel very fortunate to work with so many great customers, and very proud of the work we help them accomplish.

I’ve already mentioned Ford and Mr. Cooper, but those are just a small sampling. For example, Vodafone Commercial’s “AI Booster” platform uses the latest Google technology to enable cutting-edge AI use cases such as optimizing customer experiences, customer loyalty, and product recommendations. Our conversational AI technologies are used by companies ranging from Embodied, whose Moxie robot helps children overcome developmental challenges, to HubSpot connecting meeting notes to CRM data. Across our products and across industries around the world, customer stories grow by the day.

We also see validation in our partner network. As we noted in the pillars discussed in June, partners like Nvidia help us to ensure customers have freedom of choice when building their AI stacks, and partners like Neo4j help our customers to expand our services into areas like graph structures. Partners support our mission to bring AI to everyone, helping more customers use our services for new and expanded use cases.

Accelerating the momentum

Overall, to create products that reflect AI’s potential and likely future ubiquity, we have to take all of the preceding factors, from research to customer and analyst conversations to working with partners, and turn them into products and product updates. We’ve been very active over the last year, from the launch of Call Center AI Platform in March, to the new Speech model we released in May, to a range of announcements at the Google Cloud Applied ML Summit in June. We have much more planned in coming months, and we’re excited to work with customers not just to maintain the pace of AI momentum, but to accelerate it. To learn more about Google Cloud’s AI and ML services, visit this link or browse recent AI and ML articles on the Google Cloud Blog.

GARTNER and MAGIC QUADRANT are registered trademarks and service marks of Gartner, Inc. and/or its affiliates in the U.S. and internationally and are used herein with permission. All rights reserved. Gartner does not endorse any vendor, product or service depicted in its research publications and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s Research & Advisory organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.

For over two decades, Google has helped lead the invention of modern cloud systems—defining, designing and deploying warehouse-scale computing as the foundation for reliable, performant, and secure global-scale information services delivered to billions of users around the world. This leadership involves significant innovation across a broad range of systems technologies, including distributed systems, storage systems, databases, analytics, operating systems, wide area and data center networking, cluster computing, ML, video acceleration and more.

Today, we are announcing a significant step in continuing Google’s tradition of innovation and charting its path into the future: the formation of SystemsResearch@Google (SRG). SRG will be a new research team, positioned in the heart of Google’s Cloud and Infrastructure engineering organization, with the mission of shaping the future of hyperscaler systems design for Google and its ecosystem. It is focused on inventing, incubating, and infusing new concepts, designs, and technologies into Google’s applications, systems, and data centers. The team’s position will allow seamless engagement with engineering and product teams, enabling joint exploration in concert with transformative workloads. Beyond Google, the SRG team will look to forge strong relationships with external research communities working on the most pressing systems-research problems.

Critical research at a pivotal time

We are at a time of enormous transition and opportunity, as nearly all large-scale computing is moving to cloud infrastructure, classical technology trends are hitting limits, new programming paradigms and usage patterns are taking hold, and most levels of systems design are being restructured. We are seeing wholesale change with the introduction of new applications around ML training and real-time inference to massive-scale data analytics and processing workloads fed by globally connected edge and cellular devices. This is all happening while the performance and efficiency gains we’ve relied on for decades are slowing dramatically from generation to generation. And while reliability is more important than ever as we deploy societally-critical infrastructure, we are challenged by increasing hardware entropy as underlying components approach angstrom scale manufacturing processes and trillions of transistors.

In the last twenty years, much of the world’s population has gained real-time access to the world’s information and to one another in ways that were previously the stuff of science fiction. The next decade will see computing and associated capabilities undergo an even more profound transformation, bringing real-time insights, sensing, and actuation to trillions of network-connected devices spanning all of the world’s population. Doing so will require fundamental advances in security, reliability, programming models, data analysis, systems for machine learning, networking, storage systems, hardware architecture, and software systems.

SystemsResearch@Google will be co-led by David Culler and Hank Levy, who bring a combination of academic and industrial experience, plus a long history of successful and impactful research in computer systems. Culler is the former Chair of EECS at UC Berkeley, where he worked to create the Division of Data Sciences and became its founding Dean. His research has focused on parallel architectures, clusters, embedded wireless networks, planetary-scale internet services, and sustainability design. He was the founding faculty director of Intel Research Berkeley, co-founded two startups, and worked with Sun Microsystems for a decade. Levy is the former Chair of Computer Science & Engineering at University of Washington, where he worked to create the Paul G. Allen School and became its founding Director. His research has focused on operating systems, distributed systems, computer architecture, and hardware multithreading. Before UW, Levy spent a decade at Digital Equipment Corporation (DEC), where he worked on operating systems and early-generation clustered computer systems; he has also co-founded two startups. Culler and Levy are both Members of the National Academy of Engineering and Fellows of the IEEE and the ACM.

SRG will be located across sites in Google’s Bay Area and Seattle facilities. We are currently building the SRG team, bringing together leading networked systems thinkers from around the world and inside Google. If you are interested in learning more please reach out to us at systemsresearch@google.com.

Clinical Decision Support System (CDSS) is an important technology for the healthcare industry that analyzes data to help healthcare professionals make decisions related to patient care. The market size for the global clinical decision support system appears poised for expansion, with one study predicting a compound annual growth rate (CAGR) of 10.4%, from 2022 to 2030, to $10.7 billion.

For any health organization that wants to build a CDSS system, one key block is to locate and extract the medical entities that are present in the clinical notes, medical journals, discharge summaries, etc. Along with entity extraction, the other key components of the CDSS system are capturing the temporal relationships, subjects, and certainty assessments.

At Google Cloud, we know how critical it is for the healthcare industry to build CDSS systems, so we worked with Apollo 24|7, the largest multi-channel digital healthcare platform in India, to build the key blocks of their CDSS solution.

We helped them to parse the discharge summaries and prescriptions to extract the medical entities. These entities can then be used to build a recommendation engine that would help doctors with the “Next Best Action” recommendation for medicines, lab tests, etc.

Let’s take a sneak peek at Apollo 24|7’s entity extraction solutions, and the various Google AI technologies that were tested to form the technology stack.

Datasets Used

To perform our experiments on entity extraction, we used two types of datasets.

1. i2b2 Dataset – i2b2 is an open-source clinical data warehousing and analytics research platform that provides annotated deidentified patient discharge summaries made available to the community for research purposes. This dataset was primarily used for training and validation of the models.
2. Apollo 24|7’s Dataset – De-identified doctor’s notes from Apollo24|7 were used for testing. Doctors annotated them to label the entities and offset values.

Experimentation and choosing the right approach — Four models put to test

For entity extraction, both Google Cloud products and open-source approaches were explored. Below are the details:

1. Healthcare Natural Language API: This is a no-code approach that provides machine learning solutions for deriving insights from medical text. Using this, we parsed unstructured medical text and then generated a structured data representation of the medical knowledge entities stored in the data for downstream analysis and automation. The process includes:

• Extract information about medical concepts like diseases, medications, medical devices, procedures, and their clinically relevant attributes;
• Map medical concepts to standard medical vocabularies such as RxNorm, ICD-10, MeSH, and SNOMED CT (US users only);
• Derive medical insights from text and integrate them with data analytics products in Google Cloud.

The advantage of using this approach is that it not only extracts a wide range of entity types like MED_DOSE, MED_DURATION, LAB_UNIT, LAB_VALUE, etc, but also captures functional features such as temporal relationships, subjects, and certainty assessments, along with the confidence scores. Since it is available on Google Cloud, this offers long-term product support. It is also the only fully-managed NLP service among all the approaches tested and hence, it requires the least effort to implement and manage.

But one thing to keep in mind is that since the Healthcare NL API offers natural language models that are pre-trained, it currently cannot be used for custom entity extraction models trained using custom annotated medical text or to extract custom entities. This has to be done via AutoML Entity Extraction for Healthcare, another Google Cloud service for custom model development. Custom model development is important for adapting the pre-trained models to new languages or region-specific natural language processing, such as medical terms whose use may be more prevalent in India than in other regions

2. Vertex AutoML Entity Extraction for Healthcare: This is a low-code approach that’s already available on Google Cloud. We used AutoML Entity Extraction to build and deploy custom machine learning models that analyzed documents, categorized them, and identified entities within them. This custom machine learning model was trained on the annotated dataset provided by the Apollo 24|7 team.

The advantage of AutoML Entity Extraction is that it gives the option to train on a new dataset. However, one of the prerequisites to keep in mind is that it needs a little pre-processing to capture the input data in the required JSONL format. Since this is an AutoML model just for Entity Extraction, it does not extract relationships, certainty assessments, etc.

3. BERT-based Models on Vertex AI: Vertex AI is Google Cloud’s fully managed unified AI platform to build, deploy, and scale ML models faster, with pre-trained and custom tooling. We experimented with multiple custom approaches based on pre-trained BERT-based models, which have shown state-of-the-art performance in many natural language tasks. To gain better contextual understanding of medical terms and procedures, these BERT-based approaches are explicitly trained on medical domain data. Our experiments were based on BioClinical BERT, BioLink BERT, Blue BERT trained on Pubmed dataset, and Blue BERT trained on Pubmed + MIMIC datasets.

The major advantage of these BERT-based models is that they can be finetuned on any Entity Recognition task with minimal efforts.

However, since this is a custom approach, it requires some technical expertise. Additionally, it does not extract relationships, certainty assessments, etc. This is one of the main limitations of using BERT-based models.

4. ScispaCy on Vertex AI: We used Vertex AI to perform experiments based on ScispaCy, which is a Python package containing spaCy models for processing biomedical, scientific or clinical text.

Along with Entity Extraction, Scispacy on Vertex AI provides additional components like Abbreviation Detector, Entity Linking, etc. However, when compared to other models, it was less precise, with too many junk phrases, like “Admission Date,” captured as entities.

“Exploring multiple approaches and understanding the pros/cons of each approach helped us to decide the one that would fit our business requirements.” according to Abdussamad M, Engineering Lead at Apollo 24|7.

Evaluation Strategy

In order to match the parsed entity with the test data labels, we used extensive matching logic that comprised of the below four methods:

1. Exact Match – Exact match captures entities where the model output and the entities in the test dataset match. Here, the offset values of the entities have also been considered. For example, the entity “gastrointestinal infection” that is present as-is in both the model output and the test label will be considered an “Exact Match.”
2. Match-Score Logic – We used a scoring logic for matching the entities. For each word in the test data labels, every word in the model output is matched along with the offset. A score is calculated between the entities and based on the threshold, it is considered as a match.
3. Partial Match – In this matching logic, entities like “hypertension” and “hypertensive” are matched based on the Fuzzy logic.
4. UMLS Abbreviation Lookup – We also observed that the medical text had some abbreviations, like AP meaning abdominal pain. These were first expanded by doing a lookup on the respective UMLS (Unified Medical Language System) tables and then passed to the individual entity extraction models.

Performance Metrics

We used precision and recall metrics to compare the outcomes of different models/experiments.

Precision (also called positive predictive value) is the fraction of relevant instances among the retrieved instances, while recall (also known as sensitivity) is the fraction of relevant instances that were retrieved.

The below example shows how to calculate these metrics for a given sample.

Example sample: “Krish has fever, headache and feels uncomfortable”

Expected Entities: [“fever”, “headache”]

Model Output: [“fever”, “feels”, “uncomfortable”]

Thus,

Experimentation Results
The following table captures the results of the above experiments on Apollo24|7’s internal datasets.

Finally, the Blue BERT model trained on the Pubmed dataset had the best performance metrics with a 81% improvement on Apollo 24|7’s baseline mode with the Healthcare Natural Language API providing the context, relationships, and codes. This performance could be further improved by implementing an ensemble of these two models.

“With the Blue BERT model giving the best performance for entity extraction on Vertex AI and the Healthcare NL API being able to extract the relationships, certainty assessments etc, we finally decided to go with an ensemble of these 2 approaches,“ Abdussamad added.

Fast track end-to-end deployment with Google Cloud AI Services (AIS)

Google AIS (Professional Services Organization) helped Apollo24|7 to build the key blocks of the CDSS system.

The partnership between Google Cloud and Apollo 24|7 is just one of the latest examples of how we’re providing AI-powered solutions to solve complex problems to help organizations drive the desired outcomes. To learn more about Google Cloud’s AI services, visit our AI & ML Products page, and to learn more about Google Cloud solutions for health care, explore our Google Cloud Healthcare Data Engine page.

Acknowledgements

We’d like to give special thanks to Nitin Aggarwal, Gopala Dhar and Kartik Chaudhary for their support and guidance throughout the project. We are also thankful to Manisha Yadav, Santosh Gadgei and Vasantha Kumar for implementing the GCP infrastructure. We are grateful to the Apollo team (Chaitanya Bharadwaj, Abdussamad GM, Lavish M, Dinesh Singamsetty, Anmol Singh and Prithwiraj) and our partner team from HCL/Wipro (Durga Tulluru and Praful Turanur) who partnered with us in delivering this successful project. Special thanks to the Cloud Healthcare NLP API team (Donny Cheung, Amirhossein Simjour, and Kalyan Pamarthy).