We are excited to announce that Google has been named a Leader in The Forrester Wave™: Streaming Analytics, Q2 2021 report. Thank you to our strong community of customers and partners for working with us to deliver a customer focused product. We believe Forrester’s recognition is an acknowledgement of our leadership across an integrated set of capabilities that rely on data to drive transformation. We were also honored to be named a leader in The Forrester Wave™: Cloud Data Warehouse, Q1 2021. 

Forrester gave Dataflow a score of 5 out of 5 across 12 different criteria and according to the report: “Google Cloud Dataflow has strengths in data sequencing, advanced analytics, performance, and high-availability. Google Dataflow’s sweet spot is for enterprises that have a preponderance of real-time data generated on Google Cloud Platform or wish to simplify all data processing by using a single platform that unifies both streaming and batch jobs.” 

Harnessing the power of real-time data

The speed with which businesses are able to respond to change is the difference between those that successfully navigate the future and those that get left behind. In order to accelerate their digital transformation, reimagine their business and leverage the power of real-time data,  today’s data leaders require a streaming analytics platform that provides both depth and breadth.  

Cloud Pub/Sub and Cloud Dataflow, based on more than a decade of experience in internet scale systems for Google’s own needs, provide customers with a reliable, scalable, performant platform. In addition, we’ve designed these products for ease of use to make streaming analytics accessible to more users, which is why customers such as Sky and others from across all industries use Dataflow to run streaming analytics workloads.

5 out of 5 across key streaming analytics criteria

While Forrester gave Dataflow a score of 5 out of 5 in 12 criteria, the product achieved the highest possible scores in areas that are top of mind for our customers.

We continue to be focused on solving problems that matter to you. For example, just in the last month we announced Dataflow Prime and  Auto Sharding for BigQuery – two new auto tuning capabilities that bring efficiency and simplicity to your streaming pipelines. 

Dataflow achieves highest score possible in strategy 

With Google, organizations gain an industry leading product and a partner that has the vision and strategy to help you tackle new business challenges and provide delightful experiences to your customers.

In summary, we are honored to be a Leader in The Forrester Wave™, Streaming Analytics, and look forward to continuing to innovate and partner with you on your digital transformation journey. 

Download the full report: The Forrester Wave™: Streaming Analytics, Q2 2021 and check out these smart analytics reference patterns. To learn more about Dataflow, visit our website and get to know the product by taking an interactive tutorial. You can also watch recordings from the Data Cloud Summit event (May 2021), where we provided an in-depth view of new product innovations in Dataflow and other data analytics products.

Let’s face it: in the globalized world, which is now more than ever a digital demand world, you need to scale and reach your customers right where they’re at. Translation is a critical piece of that, whether you’re translating a website in multiple languages or releasing a document, a piece of software, or training materials.

Manual translation does not scale, which is why machine translation, powered by machine learning (ML), is becoming more important to our customers.  Machine translation has historically been challenging because of the sheer volume and breadth of content that can add value when translated into multiple languages. Companies acquire and share content in many languages and formats, and scaling translation to meet needs is a tall order due to multiple document formats, integrations with optical character recognition (OCR), and the need to correct for domain-specific terminology.

Our goal is to simplify translation services, while enabling flexibility and control for our customers’ unique needs across industries. Read on to learn more about recent features and updates. 

Formatting matters: Document Translation is now GA 

In many cases, the layout of a document dictates how it should be interpreted—e.g., readers navigate text and discern meaning based on formatting, like bold or italicized text, or markups for headers, paragraphs, and columns. Previously, to automate translation of documents, text needed to be separated from these layout attributes, meaning the document’s structure was either lost or needed to be recreated later in the developer pipeline, after the text had been translated. This required translation teams to do a lot of extra work and maintain a lot of additional code. But now, those steps are unnecessary. Formatting can be retained throughout the translation process, handled directly by the Translation API Advanced. 

This feature lets customers translate documents in 100+ languages and supports document types such as Docx, PPTx, XLSx, and PDF while preserving document formatting.

And if your needs go beyond Document Translation, we can help you translate audio as well. For real-time streaming translation, check out the Media Translation API, and for offline transcription translation, combine the Translation API with the Video Intelligence API.   

Real-Time translation when you need it, Batch when you don’t

One of the biggest differentiators for Translation API Advanced’s document translation capabilities is the ability to do real-time, synchronous processing for a single file. 

For example, if you are translating a business document such as HR documentation, online translation provides flexibility for smaller files and provides faster results. You can easily integrate with our APIs via REST or gRPC with mobile or browser applications, with instant access to 100+ language pairs so that content can be understandable in any supported language. 

Meanwhile, batch translation allows customers to translate multiple files into multiple languages in a single request. For each request, customers can send up to 100 files with a total content size of up to 1 GB or 100 million Unicode codepoints, whichever limit is hit first.

State of the Art (SOTA) accuracy, with flexibility for customization

In order to achieve the highest level of accuracy for your translation, we now support multiple options:

• Use Google’s SOTA translation models: Each year, Google heavily invests to improve the quality of our translations across Apps, Cloud APIs, and Chrome, as well to enable multilanguage answers in Search. A popular metric for automatic quality evaluation of Machine translation systems is the BLEU score, which is based on the similarity between machine translation and the reference translations that were generated by people. While we push out incremental improvements for individual models on a monthly cadence, there are also times where we make significant leaps. In the releases since 2019, we have improved our average BLEU score by 5pts on average across 100+ languages and 7pts on low resource languages.
• Leverage glossaries for specific terms and phrases: Glossary is our terminology control feature. It allows you to import source content to define preferred translations, such as product names or department names. Then, when calling the glossary in the API request, your preferred translations will be enforced. This will work for words as well as phrase translation.
• Pick a pre-trained model with model selection: If you create custom models for machine translation, we don’t think you should have multiple client libraries and multiple APIs to maintain in order for you to use the best model for your needs. Translation API Advanced now supports Model Selection. Pick your pretrained model or pick your custom ML model built on AutoML for any language pair you’ve created and use the same API and the same client library. 
• Build custom translation models with AutoML: AutoML Translation is a suite of ML products that enable you to build high quality models for your own use case or data, with limited-to-no ML expertise or coding required. Bring your past human-validated translations to improve translation specificity for your domain.

Keep localization local with Regional Endpoints

If you are a customer operating in the EU, we recently launched an endpoint specifically for EU regionalization. This is a configurable endpoint for customers to store and perform machine translation processing of customer data only in the EU multi region. For now, this only supports our pretrained translation models and glossary, but batch translations will be coming soon.

How Eli Lilly uses Cloud Translation to translate content globally

Historically, translations at Eli Lilly have been complicated: numerous translation vendors have been needed for different languages and organizations, all with their own processes and expectations. On top of that, translations have been costly and slow. 

To solve this, Eli Lilly took a codified approach to enable users and systems to spend less time and resources to safely generate quality translations. 

Learn more, and even catch a demo, from Thomas Griffin, Translation Tech Lead & Global Regulatory Architect for Eli Lilly.

Learn more 

• To get started using Cloud Translation – Advanced, complete the setup and then try the Translate text (Advanced edition) quickstart.
• Document Translation is priced per page. For more information, see pricing.

Clinical Decision Support System (CDSS) is an important technology for the healthcare industry that analyzes data to help healthcare professionals make decisions related to patient care. The market size for the global clinical decision support system appears poised for expansion, with one study predicting a compound annual growth rate (CAGR) of 10.4%, from 2022 to 2030, to $10.7 billion.

For any health organization that wants to build a CDSS system, one key block is to locate and extract the medical entities that are present in the clinical notes, medical journals, discharge summaries, etc. Along with entity extraction, the other key components of the CDSS system are capturing the temporal relationships, subjects, and certainty assessments.

At Google Cloud, we know how critical it is for the healthcare industry to build CDSS systems, so we worked with Apollo 24|7, the largest multi-channel digital healthcare platform in India, to build the key blocks of their CDSS solution.

We helped them to parse the discharge summaries and prescriptions to extract the medical entities. These entities can then be used to build a recommendation engine that would help doctors with the “Next Best Action” recommendation for medicines, lab tests, etc.

Let’s take a sneak peek at Apollo 24|7’s entity extraction solutions, and the various Google AI technologies that were tested to form the technology stack.

Datasets Used

To perform our experiments on entity extraction, we used two types of datasets.

1. i2b2 Dataset – i2b2 is an open-source clinical data warehousing and analytics research platform that provides annotated deidentified patient discharge summaries made available to the community for research purposes. This dataset was primarily used for training and validation of the models.
2. Apollo 24|7’s Dataset – De-identified doctor’s notes from Apollo24|7 were used for testing. Doctors annotated them to label the entities and offset values.

Experimentation and choosing the right approach — Four models put to test

For entity extraction, both Google Cloud products and open-source approaches were explored. Below are the details:

1. Healthcare Natural Language API: This is a no-code approach that provides machine learning solutions for deriving insights from medical text. Using this, we parsed unstructured medical text and then generated a structured data representation of the medical knowledge entities stored in the data for downstream analysis and automation. The process includes:

• Extract information about medical concepts like diseases, medications, medical devices, procedures, and their clinically relevant attributes;
• Map medical concepts to standard medical vocabularies such as RxNorm, ICD-10, MeSH, and SNOMED CT (US users only);
• Derive medical insights from text and integrate them with data analytics products in Google Cloud.

The advantage of using this approach is that it not only extracts a wide range of entity types like MED_DOSE, MED_DURATION, LAB_UNIT, LAB_VALUE, etc, but also captures functional features such as temporal relationships, subjects, and certainty assessments, along with the confidence scores. Since it is available on Google Cloud, this offers long-term product support. It is also the only fully-managed NLP service among all the approaches tested and hence, it requires the least effort to implement and manage.

But one thing to keep in mind is that since the Healthcare NL API offers natural language models that are pre-trained, it currently cannot be used for custom entity extraction models trained using custom annotated medical text or to extract custom entities. This has to be done via AutoML Entity Extraction for Healthcare, another Google Cloud service for custom model development. Custom model development is important for adapting the pre-trained models to new languages or region-specific natural language processing, such as medical terms whose use may be more prevalent in India than in other regions

2. Vertex AutoML Entity Extraction for Healthcare: This is a low-code approach that’s already available on Google Cloud. We used AutoML Entity Extraction to build and deploy custom machine learning models that analyzed documents, categorized them, and identified entities within them. This custom machine learning model was trained on the annotated dataset provided by the Apollo 24|7 team.

The advantage of AutoML Entity Extraction is that it gives the option to train on a new dataset. However, one of the prerequisites to keep in mind is that it needs a little pre-processing to capture the input data in the required JSONL format. Since this is an AutoML model just for Entity Extraction, it does not extract relationships, certainty assessments, etc.

3. BERT-based Models on Vertex AI: Vertex AI is Google Cloud’s fully managed unified AI platform to build, deploy, and scale ML models faster, with pre-trained and custom tooling. We experimented with multiple custom approaches based on pre-trained BERT-based models, which have shown state-of-the-art performance in many natural language tasks. To gain better contextual understanding of medical terms and procedures, these BERT-based approaches are explicitly trained on medical domain data. Our experiments were based on BioClinical BERT, BioLink BERT, Blue BERT trained on Pubmed dataset, and Blue BERT trained on Pubmed + MIMIC datasets.

The major advantage of these BERT-based models is that they can be finetuned on any Entity Recognition task with minimal efforts.

However, since this is a custom approach, it requires some technical expertise. Additionally, it does not extract relationships, certainty assessments, etc. This is one of the main limitations of using BERT-based models.

4. ScispaCy on Vertex AI: We used Vertex AI to perform experiments based on ScispaCy, which is a Python package containing spaCy models for processing biomedical, scientific or clinical text.

Along with Entity Extraction, Scispacy on Vertex AI provides additional components like Abbreviation Detector, Entity Linking, etc. However, when compared to other models, it was less precise, with too many junk phrases, like “Admission Date,” captured as entities.

“Exploring multiple approaches and understanding the pros/cons of each approach helped us to decide the one that would fit our business requirements.” according to Abdussamad M, Engineering Lead at Apollo 24|7.

Evaluation Strategy

In order to match the parsed entity with the test data labels, we used extensive matching logic that comprised of the below four methods:

1. Exact Match – Exact match captures entities where the model output and the entities in the test dataset match. Here, the offset values of the entities have also been considered. For example, the entity “gastrointestinal infection” that is present as-is in both the model output and the test label will be considered an “Exact Match.”
2. Match-Score Logic – We used a scoring logic for matching the entities. For each word in the test data labels, every word in the model output is matched along with the offset. A score is calculated between the entities and based on the threshold, it is considered as a match.
3. Partial Match – In this matching logic, entities like “hypertension” and “hypertensive” are matched based on the Fuzzy logic.
4. UMLS Abbreviation Lookup – We also observed that the medical text had some abbreviations, like AP meaning abdominal pain. These were first expanded by doing a lookup on the respective UMLS (Unified Medical Language System) tables and then passed to the individual entity extraction models.

Performance Metrics

We used precision and recall metrics to compare the outcomes of different models/experiments.

Precision (also called positive predictive value) is the fraction of relevant instances among the retrieved instances, while recall (also known as sensitivity) is the fraction of relevant instances that were retrieved.

The below example shows how to calculate these metrics for a given sample.

Example sample: “Krish has fever, headache and feels uncomfortable”

Expected Entities: [“fever”, “headache”]

Model Output: [“fever”, “feels”, “uncomfortable”]

Thus,

Experimentation Results
The following table captures the results of the above experiments on Apollo24|7’s internal datasets.

Finally, the Blue BERT model trained on the Pubmed dataset had the best performance metrics with a 81% improvement on Apollo 24|7’s baseline mode with the Healthcare Natural Language API providing the context, relationships, and codes. This performance could be further improved by implementing an ensemble of these two models.

“With the Blue BERT model giving the best performance for entity extraction on Vertex AI and the Healthcare NL API being able to extract the relationships, certainty assessments etc, we finally decided to go with an ensemble of these 2 approaches,“ Abdussamad added.

Fast track end-to-end deployment with Google Cloud AI Services (AIS)

Google AIS (Professional Services Organization) helped Apollo24|7 to build the key blocks of the CDSS system.

The partnership between Google Cloud and Apollo 24|7 is just one of the latest examples of how we’re providing AI-powered solutions to solve complex problems to help organizations drive the desired outcomes. To learn more about Google Cloud’s AI services, visit our AI & ML Products page, and to learn more about Google Cloud solutions for health care, explore our Google Cloud Healthcare Data Engine page.

Acknowledgements

We’d like to give special thanks to Nitin Aggarwal, Gopala Dhar and Kartik Chaudhary for their support and guidance throughout the project. We are also thankful to Manisha Yadav, Santosh Gadgei and Vasantha Kumar for implementing the GCP infrastructure. We are grateful to the Apollo team (Chaitanya Bharadwaj, Abdussamad GM, Lavish M, Dinesh Singamsetty, Anmol Singh and Prithwiraj) and our partner team from HCL/Wipro (Durga Tulluru and Praful Turanur) who partnered with us in delivering this successful project. Special thanks to the Cloud Healthcare NLP API team (Donny Cheung, Amirhossein Simjour, and Kalyan Pamarthy).

Contact Center AI (CCAI) brings Google’s innovation in conversational AI to solve the most challenging customer service needs while lowering operational costs. More than a thousand customers have deployed CCAI and are steadily turning it on to power their production contact centers.

Today, we’re excited to announce that we’ve made CCAI even stronger with Agent Assist for Chat, now in public preview.

Agent Assist provides your human agents with continuous support during their calls and now chats by identifying the customers’ intent and providing them with real-time recommendations such as articles and FAQs as well as responses to customer messages to more effectively resolve the conversation.

Customers using Agent Assist for Chat have been able to manage up to 28% more conversations concurrently, while also driving up customer satisfaction by 10%. Additionally, we’ve seen them respond up to 15% faster to chats, reducing chat abandonment rates and solving more customer problems.

Agent Assist provides two key components to help agents manage conversations better: 

• Smart Reply provides response suggestions to agents so they can quickly and appropriately respond to customer messages. These suggestions can be taken from your top performing agents as well as modified even further to ensure suggestions properly reflect the tone and voice of your brand. Agent Assist learns when and what recommendations to make by building a custom model that’s trained on your (and only your) data.
• Knowledge Assist unlocks the power of your knowledge base to provide articles and FAQ suggestions to agents in real-time as the conversation progresses. When using Knowledge Assist, agents no longer need to make the customer wait while they navigate multiple applications and data to find the resolution to the customer’s issue — the answer is delivered right to them.  

“We’ve been very impressed by the chat capabilities of Agent Assist,” said Chris Smith, Vice President of Digital Service at Optus, one of the largest telecommunications companies in Australia

Optus has been using CCAI Dialogflow CX to send queries to virtual agents and sees great potential to use Agent Assist to provide recommendations to their customer support representatives. They expect Agent Assist to help minimize repetitive tasks by providing response and typeahead suggestions, helping improve the efficiency of their agents and the quality and consistency of service they provide.

Another customer, LoveHolidays, is using Agent Assist to support their agents and customers in the travel industry. 

“Agent Assist has been a beneficial aid to agents and our customers alike… It gives us the power to flex our contact center staff levels in hours not weeks,” said Eugene Neale, Director of CX Engineering & Business IT at LoveHolidays

Analysts say online chat is becoming one of the most popular ways to reach out to businesses for customer support. IDC research finds that single-function contact centers worldwide are increasingly rare — in 2020, although phone/voice is still responsible for most interactions (at around 18%); email is responsible for around 13% of interactions, and live chat (without automation) is responsible for around 8% of interactions, according to IDC, Toward the AI-Powered Contact Center, Doc # EUR147017320, December 2020.

Deploying CCAI with Agent Assist for Chat

As part of Google’s Contact Center AI suite, Agent Assist provides a seamless handoff from chats managed by your Dialogflow CX virtual agents. If a conversation or customer requires a live agent, Agent Assist will help your team pick it up quickly and drive it to a satisfying resolution. 

Historically, when managers saw contact center volumes increase they had two choices: allow customers to wait longer to speak to someone (lowering customer satisfaction) or bring on more agents (increasing cost to serve).  Deploying CCAI provides contact center leaders with a third choice: equip agents with tools like, Agent Assist for Chat, to efficiently manage customer interactions while maintaining high quality service.

Global CCAI partners support Agent Assist for Chat

Agent Assist for Chat is a set of public APIs that your engineering team can integrate directly into an agent desktop to control the agent experience from end-to-end. For a more out-of-the-box solution, we have partnered with LivePerson and 247.ai to build Agent Assist directly into their agent desktops.

“Integrating our Conversational Cloud directly with Agent Assist means agents can leverage cutting-edge productivity AI to build even further on the massive ROI of conversational commerce, from reduced agent effort and time-to-respond to increased customer satisfaction and revenue,” said Alex Spinelli, CTO of LivePerson. 

More Agent Assist resources

To learn more, check out the Agent Assist webpage. Give Agent Assist a try by training a model and then testing it using the Agent Assist simulator.