Scaling Virtual Support in the Pandemic Era: The AI Connection

Since the early days of the pandemic, we’ve partnered with government organizations and academic institutions to serve communities at scale with Contact Center Artificial Intelligence (CCAI). I sat down with Bill MacKenzie, IT liaison for the Upper Grand School District in Ontario, and Marco Palermo, director of digital government and modernization, to discuss how they embraced CCAI to introduce scalable service delivery to residents and students alike. I’m sharing more on their stories below, and for the full overview, check out our Google Cloud Public Sector Summit session, Scaling Virtual Support in the Pandemic Era: The AI Connection.

Upper Grand School District: Answering questions with speed and accuracy

Bill MacKenzie described the Upper Grand School District’s struggles at the beginning of the pandemic, particularly helping parents with IT issues. The staff-oriented help desk was not equipped to assist parents trying to securely login for students as young as kindergarten. Without sufficient support for parents, the District was struggling to handle thousands of phone calls a day.

To alleviate the manual strain, they turned to Quantiphi, a Google Cloud partner, to implement Google Cloud Dialogflow. They were fully functional within a few weeks. The new website provided real-time responses as well as clear documentation to help parents get up and running quickly.

“In the first 10 days, we had over 5,000 hits, and the accuracy rate was 92%,” MacKenzie said. 

The district doesn’t know what the future holds, but now that they have been through the process, they are confident they now understand how to create their own bots to meet critical needs.

City of Toronto: Getting critical information to the community

Marco Palermo explained how the city of Toronto was facing a very rapid and fluid situation at the beginning of the pandemic. Getting information to constituents was extremely important, and they needed alternative channels to deliver that information.

Toronto has been committed to workforce equity and inclusivity in order to best represent the diversity of its residents. As a result, solutions had to be accessible to all. The bot handled 25,330 unique users and addressed 20,174 total questions with an 80% accurate response rate in the first four months. It’s been a huge success for the city, with plans to expand its capabilities.

Personalized response to the pandemic challenge

I noted that even before the pandemic, government leaders were asking for a way to provide more flexible personal experiences and better support outside of normal business hours. Around 60% of constituents want more self-service options and 75% would happily interact with digital resources if they could get the answers they needed.

Google Cloud CCAI  addresses these needs with a single core of intelligence that provides a consistently high-quality conversational experience—human or virtual—across all channels and platforms. It can be deployed on legacy infrastructure without upgrades in an average of two weeks.

CCAI operates with a conversational core that centralizes the ability to talk, understand, and interact, orchestrating high-quality conversations at scale. It provides services in three different ways:

• Virtual Agent AI allows natural conversation with customers to identify and address their issues effectively
• Agent Assist AI helps human agents by providing real-time turn-by-turn guidance so they can better serve constituents
• Insight AI determines metrics and trends in real-time to enable faster and more accurate insights

CCAI provides consistency across every application. It can go off-script when answering complex questions, adjusting human conversation with the capacity to handle unexpected stops and starts, odd word choices, or implied meanings. It can also handle multiple use cases for the customer, such as taking payments, updating information, providing information, and more. By allowing agencies to automate routine tasks and reduce the amount of time employees need to dedicate to answering calls, the solution created cost savings for the agency.

Explore more on this session by visiting the Public Sector Summit on-demand video.

NVIDIA GPU-powered instances on Google Cloud provide an optimal platform for organizations to develop their AI applications on the latest hardware and software stack, then seamlessly deploy those applications at scale in production.

Simplifying Workflows to Speedup AI Developments

NVIDIA recently announced the One Click Deploy feature on the NVIDIA NGC catalog, the hub for GPU-optimized AI software. Developed in collaboration with Google Cloud, this feature simplifies the deployment of AI software, to a single click from the NGC catalog.

This allows data scientists to deploy frameworks, software development kits and Jupyter Notebooks directly to Google Cloud’s Vertex AI Workbench, a new managed Jupyter Notebook service on top of Vertex AI, Google’s service for machine learning operations.

Under the hood, this feature launches the JupyterLab instance on Google Cloud Vertex AI Workbench with optimal instance configuration, preloads the software dependencies, and downloads the NGC notebook in one go.

NGC Catalog – One Stop for AI Software

NVIDIA is expanding the rich trove of NVIDIA AI software in the catalog to ensure AI practitioners have everything they need to get started — from frameworks to models.

All of the AI models in the catalog come with credentials. They’re like resumes that show the model’s skills, the dataset that trained it, how to use the model and how it’s expected to perform.

These model credentials provide transparency, which gives developers the confidence in picking the right model for their use case.

The NGC catalog also hosts Jupyter Notebooks tailored for the most popular AI/ML applications. Examples include:

Computer Vision – A collection of models for detecting human actions, gestures and more.

Automatic Speech Recognition – An end-to-end workflow for text-to-speech training.

Recommendation – A collection of example notebooks to help build end-to-end recommendation services.

Serve Robotics uses Vertex AI and the simple easy-to-use interface that the NGC One Click Deploy delivers.

“NGC catalog allows our ML research engineers to launch environments for experiments on Vertex AI with a single click. This saves us the efforts on ML infra setup and lets researchers focus on the ML problem in the computer vision and robotics space more efficiently.”—Kaiwen Yuan, Director of ML/Head of Perceptions & Predictions at Serve Robotics

Accelerate ML Deployments

Explore hundreds of Jupyter notebook examples for speech, computer vision and recommenders and, if you’re just getting started with AI, browse NVIDIA’s collection of Jupyter notebook examples and run it using the One Click Deploy feature on Google Cloud Vertex AI.

Lung cancer is the leading cause of cancer death in the United States and like any cancer, early detection is crucial to survival. Screening at-risk populations is an important part of reducing mortality, and if concerning nodules are found on imaging, further testing may be required. Today, we’ll share how Northwell Health uses Google Cloud products such as Cloud Healthcare APIs and BigQuery to increase caregiver productivity and deliver better care for patients with findings that indicate potential development of lung cancer.

Northwell is New York’s largest healthcare provider

Northwell Health is New York’s largest healthcare provider with 23 hospitals and nearly 800 outpatient facilities. Northwell’s nearly 4,000 doctors care for millions of patients each year, and at this scale, there is an immense amount of healthcare data to manage. To better manage and leverage this data, Northwell Health partnered with Google Cloud starting in 2018.

Enabling caregivers to spend more time with patients

Nic Lorenzen, the lead developer of Northwell Emerging Technology and Innovation team, has a mission to put together data for caregivers in a way that makes sense. It is no secret that inefficient electronic health records systems have a negative impact on a physician’s ability to deliver quality care. Traditional EHRs have information distributed across many tabs, which forces caregivers to spend considerable time at the computer trying to find information. Moreover, speed of care matters. If care is delayed, patients may have to spend more time in the hospital and may suffer worse health outcomes.

To solve this problem, Nic’s team focused on giving caregivers the most relevant pieces of data at the right time by developing an intelligent clinician rounding app. The data needed to derive these insights can depend on the caregiver’s role–a nurse cares about different things than a cardiologist. This system aggregates multiple data sources, and provides patient-specific insights to caregivers.

This system would not have been possible before with traditional EHRs and data warehouses that have proprietary data models and rarely sync data in real time. Now with data easily accessible through Google Cloud’s Healthcare solutions, Nic’s team can deliver the right clinical information to the right people instantly. These days, Nic says, “instead of spending 75% of our time dealing with architecting the underlying platforms, we spend 75% of our time focused on  higher value use cases for clinicians and patients. Google Cloud’s Healthcare solutions have greatly improved our developer productivity and time to value.”

Caregivers have found this new system to be a game changer.Before the implementation of this system, caregivers would spend, on average, seven to nine minutes finding the data needed to make medical decisions for one patient. Now, that aggregated information is delivered to a caregiver’s mobile device in less than a second.

Ensuring patients get the right care with the power of AI

There are a number of reasons why patients might not get the care that they need. For example, patients today can go to multiple hospitals and clinics settings, and coordinating care across multiple facilities is complex. Regional hospitals and clinics have their own siloed view of their data, so pertinent information gathered by one clinic might not be seen by another. These gaps in clinical data lead to gaps in patient care.

When a patient gets radiologic imaging, they may have findings unrelated to the reason they initially got the imaging. For example, a chest CT for a car accident might reveal an incidental lung nodule that could be cancerous. Unfortunately, research shows that a large portion of patients do not get follow up for these incidental findings because it isn’t the primary reason why the patient is seeing a doctor. Moreover, social determinants of health are a factor that affects which patients receive follow-up care. Identifying these patients and providing the necessary follow up care prevents adverse events related to delayed detection of cancer.

With Cloud Healthcare solutions, Northwell built an AI model to identify these patients so that oncologists can appropriately follow up with patients who have findings suspicious for lung cancer. The AI model detects incidental pulmonary nodules in radiology reports so that doctors can then contact the patients that need follow-up care. Nic says his team was able to build this system in a week: “Google Cloud did a lot of heavy-lifting for us and allowed us to get to the AI applications much faster. It allowed us to build a platform that just works.” 

Healthcare systems can now rapidly generate healthcare insights with one end-to-end solution, Google Cloud Healthcare Data Engine. It builds on and extends the core capabilities of the Google Cloud Healthcare API to make healthcare data more immediately useful by enabling an interoperable, longitudinal record of patient data. Northwell Health uses Google Cloud as the core of their platform, enabling their developers to create solutions to the most pressing healthcare problems.

Special thanks to Kalyan Pamarthy, Product Management Lead on Cloud Healthcare and Natural Language APIs for contributing to this blog post.

As I type this blog in a Google Doc, I can’t help but think about how much we rely on digital documents to communicate, collaborate, and do business. Yet most of the data in documents remains un-analyzed. Even when documents are part of customer-facing workflows like processing a mortgage application or a contract, businesses frequently struggle with the data those documents contain. Often, even finding the right document in a haystack of thousands across the organization can be challenging. To resolve these difficulties, most organizations rely on manual, time-consuming, and resource-intensive processes—all of which are incredibly frustrating for employees at the forefront of document workflows.

Because of challenges like these, in 2020, we launched Document AI, an AI agent that lets organizations apply machine learning (ML) to their hardest document automation problems. Since then, we have introduced specialized models to extract data for industry-specific use cases such as mortgage processing and procurement. With the launch of Document AI Workbench and Document AI Warehouse at Google Cloud Next ‘22, we’ve continued to take significant steps in our mission to help organizations simplify and automate document processing. Let’s double click on each of these announcements.

Custom document processing with Document AI Workbench

With Document AI Workbench, organizations can process documents by creating custom ML models that are specific to their business needs and extract unstructured data with a high degree of accuracy. Thanks to the user-friendly interface, even business users who do not have extensive ML skills can get started training or uptraining models.

Moreover, if an organization wants to transfer learning from pretrained models and enhance a model further to, say, include new fields, users can now do so by what we call “uptraining.” The uptraining feature is especially valuable for the most common yet complex use cases because it helps to save time and resources, so businesses don’t have to start from scratch. Uptraining for the invoice, purchase order (PO), contracts, W2, 1099-R, payslip, and 1040 pre-trained models unlocks new possibilities for improving accuracy, adding new language support, and schema customization.

We’re continuing to invest in these pretrained models. At Next’22, we announced an update to our invoice and expense pre-trained models with improvements to normalization and line item entities detection, as well as new ID proofing capabilities via a flexible API designed to spot fake, altered, or doctored ID documents. We’ve also added support for five new languages across invoice and expense models, in addition to the 12 previously-supported languages, and expanded availability in Canada and Australia regions, in addition to previously-supported US, EU, and Singapore regions.

According to Daan De Groodt, Managing Director, Deloitte Consulting LLP, Document AI Workbench “is poised to be a game changer, because we can now uptrain various text documents and forms utilizing powerful Google Machine Learning models to get the desired accuracy creating greater time and resource efficiencies for our clients.”

And customers are already seeing benefits. Libeo used Document AI to uptrain an invoice parser with 1,600 documents and increase its testing accuracy from 75.6% to 83.9%. “Thanks to uptraining, the Document AI results now beat the results of a competitor and will help Libeo save ~20% on the overall cost for model training over the long run,” said Libeo chief technology officer, Pierre-Antoine Glandier.

Google-powered document search with Document AI Warehouse

With Document AI Warehouse we are bringing the best of Google’s semantic search to documents. Document AI Warehouse lets enterprises search, store, govern and manage documents and their AI-extracted data and metadata in a single platform. With Document AI Warehouse’s simple and intuitive web accessible user interface, users can explore, view, bulk update and organize documents into folders. Document AI Warehouse offers robust enterprise control and governance so you can control who has access at the document and folder levels and assign users and groups permissions to view, edit, manage (share, delete) documents. You can migrate, sync, or federate documents from other repositories, such as Microsoft SharePoint, Amazon S3, and IBM FileNet. Or if that’s not an option we simply index the content and any extracted/tagged metadata).

We also will consolidate a number of next-generation product enhancements on Document AI OCR and Form Parser by the end of this year – including deeper insights into document quality & semantics, a unified document OCR experience, expanded language coverage for Form Parser, and advanced tooling for model lifecycle management. Google’s DeepMind team developed a new method that allows the creation of document parsing ML models for utility bills and purchase orders with 50%-70% less training data than what was previously needed for Document AI. We’re working on integrating this method into Document AI Workbench in the coming months.

Getting started

I’m very excited about what the future holds for Document AI as a platform for businesses to simplify document automation. Learn more about all these exciting developments in my session at Next’22 or try out one of our offerings today.