There’s a lot we can learn from combining technology with science to help support the development of amazing discoveries. By using an AI system to predict protein shapes, we have the potential to accelerate research in every field of biology. Inside every cell in your body, billions of tiny molecular machines are hard at work. They are what allow your eyes to detect light, your neurons to fire, and the ‘instructions’ in your DNA to be read.

These intricate machines are known as proteins.

The protein folding puzzle

Protein folding is something that occurs naturally so that proteins become biologically functional, but it’s a complex process that sometimes fails. For decades, scientists have been trying to find a method to reliably predict a protein’s structure from its sequence of amino acids so we can better understand how proteins work.

The challenge? There are over 200 million known distinct proteins. Each one has a unique 3D shape that determines how it works and what it does. Because there are so many sequences and determining their 3-D structure experimentally is so time-consuming and expensive, scientists only know the exact structure of a tiny fraction of the proteins. And these experimental methods still fall far short of reliable statistical accuracy.

Deepmind’s gigantic leap

In 2020, Alphabet’s artificial intelligence research arm, DeepMind, made a massive breakthrough in predicting protein structures using a deep learning model called AlphaFold.

AlphaFold is trained on publicly available data consisting of about 170,000 protein structures, and is the first computational method that can regularly predict the 3D shape of a protein, at scale with a high degree of accuracy.

AlphaFold has already sent waves throughout the scientific community and has demonstrated the potential for AI to aid fundamental scientific discovery. Recently, Deepmind has made AlphaFold predictions available and open source to anyone. To date, more than 500,000 researchers from 190 countries have accessed the AlphaFold protein structure database to get closer to finding life-saving cures for diseases like Leishmaniasis and Chagas.

And now Deepmind has expanded the set of available predictions by more than 200 times (from nearly 1 million to nearly 214 million) to cover almost all cataloged proteins found in nature.

Open source predictions available on Google Cloud

Together, Google Cloud and Deepmind have released this dataset of predicted protein structures for plants, bacteria, animals, and other organisms as part of the Google Cloud Public Dataset program to enable bulk downloads at no cost. That means you can also create custom queries of the dataset using BigQuery!

Running AlphaFold on Google Cloud Vertex AI

Let’s say you want to run AlphaFold on your own in order to get protein structure predictions against your own set of data. There are a few challenges to keep in mind:

• You need to set up feature engineering against genetic sequence databases
• Preprocess data
• And run those inputs against pre-trained models

All of this requires allocating CPUs or GPUs, hosting a notebook environment, and scaling up for larger experiments. It’s hard to build and configure an on-premise system or cloud server to use AlphaFold whether you just want to try it out or run it at scale as a large organization.

That’s why we’re excited to share a deep integration between Google Cloud and Deepmind. On top of the Public Datasets program we have created end-to-end code samples for AlphaFold on Vertex AI, a managed end-to-end ML platform, to help address these challenges and speed up deployment. With AlphaFold on Vertex AI, you can manage a data science or machine learning workflow in a single development environment. You get access to pre-configured compute, storage, and end-to-end production notebooks. We have removed the heavy lifting needed to set up new ML environments, automate orchestration, and manage large clusters.

The AlphaFold inference workflow can be simplified with Vertex AI: from data preparation to feature engineering and deployment. Unlike the manual set up, the orchestrator makes it possible to parallelize steps, get predictions faster, and with better tracking.

Try it out first using Vertex AI Workbench

For those of you who want to try out a simplified version of AlphaFold, we have a Colab notebook that uses no templates (homologous structures) and a selected portion of the BFD database. You can deploy right on Vertex AI Workbench, which lets you specify a custom container image that we’ve already created for you. You’ll be able to:

1. Configure access to genetic databases
2. Configure GPU acceleration
3. Search against genetic databases
4. Use the pre-processed results as inputs to the AlphaFold model locally

In a little over an hour you can harness the power of AlphaFold to generate 3-D protein structures from amino acid sequences.

Run hundreds of experiments reliably using Vertex AI Pipelines

For organizations that want to run a full blown version of AlphaFold for many protein folding experiments a week, you’ll want an ML pipeline orchestrator. The AlphaFold Batch Inference solution is a set of code samples that uses Vertex AI Pipelines to support hundreds of concurrent inference pipelines with higher throughput to help you run experiments at scale. The solution uses Vertex AI Pipelines as an orchestrator and runtime, Vertex ML Metadata for metadata and artifacts, and Cloud Filestore to manage databases.

Because it’s built on Vertex AI Pipelines, you can automate, monitor, and experiment with interdependent parts of an ML workflow. The minimized inference elapsed times mean what normally would take you days, can now take you hours.

The solution includes two example pipelines:

1. The universal pipeline solution mirrors the exact logic in DeepMind’s open source inference script but decoupled into discrete tasks so you can run the same experiments faster, more efficiently, and with better tracking.

2. The customized pipeline solution shows you how to further optimize the inference workflow by parallelizing feature engineering steps so you can plug in your own database sources.

You get example components, pipelines, and notebooks to start, analyze, and recompile pipelines on different GPUs.

The AlphaFold Vertex AI Workbench solution is great for experimental use, while the AlphaFold Batch Inference solution on Vertex AI Pipelines is great for doing protein folding at scale with a strong process for reproducibility and tracking.

Now go forth and save the world!

Okay maybe that’s a bit hyperbolic, but this is inspiring stuff! What started as a 50 year challenge, to the discovery of AlphaFold, to being able to run it on Google Cloud, researchers, developers, and science enthusiasts now have access to one of the most pivotal advancements in the medical world. Even a non-specialist can easily use a Vertex AI notebook to exercise a simplified version of AlphaFold. The next answers to the mysteries of life and discovery of disease treatments have never felt more attainable. With these no-cost solutions to run AlphaFold on Vertex AI and the Public Dataset, you can help propel us in this worldwide endeavor.

Learn more about healthcare and life sciences solutions on Google Cloud here.

If you have feedback or want to share your experience with me, reach out to me at @stephr_wong.

In the United States, Tax Season descends upon the country every April, requiring millions of Americans to spend hours deciphering cryptic documents and performing complex math just to figure out what they owe. Wouldn’t it be grand if there was a way for a computer to take all the relevant documents and extract out exactly what the IRS is looking for? Lending Document AI from Google Cloud supports common document types used for Income Tax Filing, such as W-2s and 1099s. These advancements in machine learning technology now makes it possible to alleviate some anxiety leading up to April 15th.

Lending Document AI is a Document Understanding solution that allows for classification and parsing of documents commonly used in the mortgage lending industry. The data in these unstructured files is then converted into a structured format, which can be stored in a database or used for analysis and calculations. You can read more about the product in the announcement blog post. For this tax filing use case, we will focus on automatically classifying and parsing the 2020 editions of the following forms:

W-2

1099-DIV

1099-INT

1099-MISC

1099-NEC

This sample application creates an automated pipeline where the user can bulk upload a collection of PDFs, the Lending Document Splitter & Classifier will classify each document and send each PDF to the appropriate specialized parser to extract the data, which can then be used to calculate an individual tax return and fill out a 1040 Form.

Overview

Let’s explore how this application works. You can check out the sample code in this GitHub Repository.

Here is an outline of the architecture of this application. As you can see, it utilizes Cloud Run and Firestore in Native Mode for the web application in addition to Document AI.

The User uploads multiple PDF files to the web application, hosted on Cloud Run.

An API call is made to the Lending Document Splitter & Classifier for each PDF file.

The output of the classifier (e.g. W-2, 1099-MISC, etc.) is then mapped to an appropriate specialized parser in the Google Cloud Project.

Each document file is sent to the appropriate specialized parser that matches the document type.

The entities are extracted by the parser processor and the data is written to Firestore.

The raw data is now retrieved from Firestore and displayed to the User showing the file classification and extracted values from each form.

The data values from all the forms are used together to calculate an individual income tax return.

The Calculated Tax Rates/Incomes/Deductions are displayed to the User in a Tabular Format matching the IRS Form 1040. The app also displays which form data was used for each field. (Some output fields use values from multiple forms, such as line 25b.)

Step-by-Step directions

Want to try this out for yourself? Here’s how you can deploy and run this application using a Google Cloud Project. You can run this in Cloud Shell (Quickstart) or on your local machine.

NOTE: The Lending Processors in this Demo are in Limited GA as of March 2022. If you have a business use case for these processors, you can fill out and submit the Document AI limited access customer request form.

Install dependencies

1. Clone the GitHub Repository to get the sample code.

git clone https://github.com/GoogleCloudPlatform/document-ai-samples.git

2. Enter the directory for the tax pipeline demo

cd document-ai-samples/tax-processing-pipeline-python

3. Install Python and the Google Cloud SDK if they aren’t already installed.
4. Install the python libraries:

pip install -r requirements.txt

5. Create a new Google Cloud project, and enable billing if you don’t already have one.
6. Enable the Document AI API:

gcloud services enable documentai.googleapis.com

7. Setup application default credentials:
   gcloud auth application-default login

Deploy demo application

1. Edit the config.yaml file, adding your own Project Details docai_processor_location: us # Document AI Processor Location (us OR eu)
   docai_project_id: YOUR_PROJECT_ID # Project ID for Document AI Processors
   firestore:
   collection: tax_demo_documents # Set with your preferred Firestore Collection Name
   project_id: YOUR_PROJECT_ID # Project ID for Firestore Database
2. Run setup scripts to create the processors and Cloud Run app in your project. python3 setup.py
   gcloud run deploy tax-demo –source .
3. Visit the Deployed Web Page (You should get a link from the deployment command)

Calculate Tax Returns Homepage

4. Upload Documents. I created some sample documents you can download from the sample-docs folder of the repository.

This demo currently supports the following Document Types (2020 Editions)

W-2
1099-DIV
1099-INT
1099-MISC
1099-NEC

5. Click “Upload” Button, wait for processing to complete.
   The page will display the steps completed for each document file. These are also written to stdout for troubleshooting purposes.

6. View the extracted values from each file.

7. Click “Calculate Taxes” to see the tax calculation output

Conclusion

Warning: This is NOT financial advice, for educational purposes only.

Congratulations! You now have a fully functional tax processing application that can also be modified for use with other workflows that require data from multiple specialized documents.

The Document AI API is flexible and modular enough that most of the code in this example can be reused for any specialized processor.

Now tax returns can be filed with minimal manual effort!

If you want to learn more about Document AI, check out the Cloud Documentation and these videos:

• Getting started with the Document AI platform
• Process billions of pages and cut operational costs with DocAI

And if you want more hands-on experience, I recommend following these step-by-step codelabs to get started with the key features of Document AI:

• Optical Character Recognition (OCR) with Document AI (Python)
• Form Parsing with Document AI (Python)
• Specialized Processors with Document AI (Python)
• Managing Document AI processors with Python

Conversational AI is opening up a new world of possibilities in areas like customer experience, user engagement, and access to content. 

In Cloud AI, we’ve taken Google’s groundbreaking machine learning models in speech and natural language processing and applied them to the contact center space, radically improving the customer experience while also driving down operational costs. It’s shifting the focus of contact centers from the backroom, out of sight (and mind) to the boardroom and the strategic heart of the business.

Our solution, called Contact Center AI (CCAI), is an accelerator of digital transformation as organizations all over the world figure out how to support their customers during these challenging times. One customer that’s redefined the possibilities of AI-powered conversation using CCAI, is Verizon.

Whether through voice calls or chat, Verizon customers will no longer need to go through menu prompts or option trees; they simply say or type their question, and CCAI’s natural-language recognition feature finds the best way to assist them. No stilted speech or robot-like commands.

“These customer service enhancements, powered by the Verizon collaboration with Google Cloud, offer a faster and more personalized digital experience for our customers while empowering our customer support agents to provide a higher level of service,” said Shankar Arumugavelu, SVP & CIO at Verizon. 

CCAI is also driving cost savings without cutting corners on customer service. In the past, to improve customer satisfaction (CSAT), you had to hire more agents, increasing operating costs. Conversely, if you reduced the number of agents, your CSAT scores went down. Contact Center AI (CCAI) lets you have both.  

CCAI does this by expediting customer requests using virtual agents, assisting live agents, and providing insights on customer interactions to drive improvements back into the system. The result is a better experience for your customers—they get answers faster—and you get lower operating costs. It allows human agents to focus on higher value tasks in their customer interactions.

New products and features add richer customer experience 

We are continuing to invest in CCAI and today we are excited to announce the following new capabilities:

Dialogflow CX is the latest version of Dialogflow, a development suite used by over a million developers for building natural, rich conversational experiences into mobile and web applications, smart devices, bots, interactive voice response systems, popular messaging platforms and more. This new version of Dialogflow is optimized for large contact centers that deal with complex (multi-turn) conversations and it is truly omnichannel – you build it once and deploy it everywhere – in your contact centers and digital channels. Dialogflow CX features a new visual builder to create, build and manage virtual agents. It’s available now, in beta.

Dialogflow CX brings conversation state management to a whole new level

Lukasz Rewerenda
Principal Solutions Architect, Randstad (Netherlands)

Agent Assist for Chat is a new module for Agent Assist that provides agents with continuous support over “chat” in addition to voice calls, by identifying intent and providing real-time, step-by-step assistance. Agent Assist enables agents to be more agile and efficient and spend more time on difficult conversations, giving both the customer and the agent a better experience. It transcribes calls in real time, identifies customer intent, provides real-time, step by step assistance (recommended articles, workflows, etc.), and automates call dispositions.

For customers in regulated industries, Agent Assist can remove the risk of agents providing inaccurate information (which can happen due to high agent turnover and limited training). Agent Assist can also surface the latest discount information, deals and special offers, which can be hard for agents to keep track of as this information changes frequently. 

Custom Voice, available in beta, is a new capability for CCAI and our Text-to-Speech API that lets you create a unique voice to represent your brand across all your customer touchpoints, instead of using a common voice shared with other organizations. By taking advantage of the custom Text-to-Speech model created with Custom Voice, you can define and choose the voice profile that suits your business and adjust to changes without scheduling studio time with voice actors to record new phrases.

At Google Cloud, we are committed to ensuring that our products and features are in alignment with our AI Principles. If you are interested in Custom Voice, there is a review process to ensure that your use case is aligned with our AI principles. Learn more about Custom Voice and how to sign up here. 

Bring Google’s groundbreaking speech models on-prem

For many customers, an all-in approach to public cloud is not an option, which is why we’re extending our AI capabilities to run on-prem. Last week we announced Speech-to-Text On-Prem, the first of our hybrid AI offerings, now generally available. Speech-to-Text On-Prem gives you full control over speech data and since it runs in your own data center, it’s easy to comply with your data residency and compliance requirements. At the same time, Speech-to-Text On-Prem uses state-of-the-art speech models from Google researchers that are more accurate, smaller, and require less computing resources to run than existing solutions.

Take a look at how businesses saved costs and improved customer experience with CCAI in the new commissioned study conducted by Forrester Consulting: “New Technology: The Projected Total Economic Impact™ Of Google Cloud Contact Center AI.” To learn more about CCAI and conversational AI, see how our customers are using the solution and check out these sessions at Next OnAir:

• CCAI State of the Union (keynote)
• Measuring and Improving Speech-to-Text Accuracy (breakout)
• Transform customer service with CCAI (demo)
• See the next generation of 3-1-1 in action (demo)