The game of baseball has no shortage of statistics — from batting average to exit velocity, strikeouts to wins above replacement. Among all sports, Major League Baseball (MLB) arguably contains the most analytical and data-driven participants and fan base. Subconsciously or viscerally, players and managers on the field and those following from anywhere are constantly assessing and making decisions based off of game play trends and expectations — whether a batter will come through with a hit in an important situation, when a pitcher should be pulled. Less analyzed, however, is what leads fans to become engaged with certain players or teams, and what factors drive their love of the game. This is the motivation behind the problem being posed by Major League Baseball in their Kaggle competition for Player Digital Engagement Forecasting. Can you use machine learning to deconstruct baseball fandom?

This competition asks you to predict measures of digital engagement for each active player on a daily basis during the MLB season. So, how large was the surge in fan interest after Joe Musgrove threw the first no-hitter in Padres history? Is Shohei Ohtani’s engagement higher when he pitches well, when he hits a monster home run…or when he does both? You’re provided a wealth of game, team and player information – detailed stats, awards, rosters, and transaction information – as well as social and digital engagement data as your inputs. Data scientists will recognize this as an exciting forecasting problem with both traditional regression and time series components, where having this input data just prior to the prediction date is critical to determining which players will receive the most engagement. 

With so many variables in the game, there are an endless number of vectors which could possibly influence fan engagement. Eleven-time All-Star Miguel Cabrera delighted fans by hitting the first home run of the season – in the snow! Occasionally a lesser-known player like Musgrove or Carlos Rodón “wins the day” with an unlikely no-hitter. And sometimes just getting traded to an iconic franchise like the Yankees generates a ton of fan interest, like it did for Rougned Odor in early April.

As these examples show, a player’s digital engagement can be pretty dynamic during the season, with many different potential contributors to who is “trending” on a given day. How can you use data to uncover which factors are the most influential of engagement with each player’s digital content?

Ready to play ball? Check out the competition on Kaggle for all the details. $50,000 in prizes is up for grabs in two prize categories. The code competition puts your machine learning skills to the test, to see who can build the most accurate forecasting models to predict daily digital engagement for every active player. You’ll have until July 31st to build your models and then be evaluated on a future time frame, which will determine the winners. For data visualization and exploration experts out there, the explainability prizes give you an opportunity to analyze more broadly which factors, even those outside of what we’re providing directly, most influence digital engagement. You’ll be evaluated on how well you can use what the data is telling you to support your findings.

And if you’re looking to get started, we’ve provided an introductory video and some notebook tutorials, including a starting point for harnessing the power of Vertex AI through tools including Cloud Notebooks, Explainable AI, and Vizier. 

With the second half of the season upon us, it’s an exciting time to be an MLB fan. With this Kaggle competition, it’s also a perfect opportunity to use data science to help understand baseball fandom and potentially earn some of your own accolades in the process. Step up to the plate!

^{Major League Baseball trademarks and copyrights are used with permission of Major League Baseball. Visit MLB.com.}

Customers reach out to contact centers for help in moments of urgent need, but due to increasing demands, new channels, peak times, and operational pressures, contact centers often struggle to provide timely help. To bridge this gap, enterprises are increasingly investing in AI-driven solutions that balance addressing customer expectations with operational efficiency.

But building and generating value from such solutions can be complicated and challenging. Google Cloud built Contact Center AI (CCAI) to streamline and shorten this time to value, and CCAI Platform, our newest addition, takes a crucial step in this effort by introducing end-to-end call center capabilities. After debuting these new capabilities in March, we are excited to announce that CCAI Platform is now generally available across the US, Canada, UK, Germany, France, Italy and Spain—with more markets soon to come.

Delivering world-class customer experiences and accelerating time-to-value with CCAI

CCAI encompasses a comprehensive set of offerings to address the top pain points of the three main user groups in the contact center: contact center owners, their agents, and the customers they serve.

Dialogflow lets the contact center manager scale their operations while balancing cost and customer satisfaction, including reducing painful, long waiting times endured by end users. Using Dialogflow, contact center managers can build complex chat and voice virtual agents—a proven, cost-effective way to scale contact centers while continuing to provide great customer experiences. Available 24/7, without any waiting queue, these virtual agents can converse naturally with customers, identify their issues, and address them effectively.

Agent Assist reduces overall handling time and coaches human agents to become more effective and helpful. The service uses AI to “listen” to the voice and chat conversations between the human representative and the customer, then provides real-time guidance and recommendations to the agent, based on historical conversations, knowledge bases, and best practices of experienced agents. It also automates post-call actions such as transcription and call summarization, saving significant time and overhead at the end of every call.

CCAI Insights stores and analyzes all the customer conversations in the contact center, whether with human or virtual agents, to provide leaders with real-time, actionable data points on customer queries, agent performance, sentiment trends, and opportunities for automation.

At the heart of these technologies is our conversational AI brain. It uses Google Research’s technology to talk, understand, and interact, enabling and orchestrating high-quality conversational experiences at scale.

CCAI Platform: a modern CCaaS and the shortest path to CCAI value

While the value of the CCAI offerings is clear to our customers, we also hear from them that integrating these solutions with legacy infrastructure takes too long.

To minimize these integration difficulties, accelerate time-to-value using the CCAI offerings, and help businesses provide outstanding customer experiences, we’re pleased to announce the general availability of CCAI Platform, the Contact Center as a Service (CCaaS) solution from Google Cloud built in partnership with UJET.

CCAI Platform is a modern, turnkey Contact Center as a service solution, designed with user-first, AI-first, and mobile-first principles. It offers:

• Turnkey core Contact Center capabilities out-of-the-box, for faster time to production, lower implementation overhead, and custom development needed
• AI-powered experiences, from routing to better handling customer interactions
• Deep integration with CCAI’s offerings, to provide a unified end-to-end experience for contact center transformation
• Mobile-first design that enables interactions in line with the way people expect to communicate across channels
• CRM-centered design with automated updates, so agents can focus on the customer
• Deployment flexibility, with customer data residing in their CRM and the flexibility to bring their own telephony carrier to minimize cost

All of this is available without the typical need to integrate complex technologies from multiple providers.

“With Google Cloud and CCAI Platform, we will quickly move our contact center to the cloud, supporting both our customers and agents with industry-leading CX innovations, all while streamlining operations through more efficient customer care operations,” said Dean Kontul, Division CIO of KeyBank.

For customers looking to change platforms for a cloud-native CCaaS with deep Google AI integrations, CCAI Platform offers end-to-end capabilities that accelerate call center transformations. We also remain strongly committed to customer choice, and customers will continue to have the option to integrate our latest and greatest CCAI offerings through our existing OEM partners.

The Contact Center conversation is just beginning

This launch is part of a broader effort to deliver more value, faster, to more CCAI customers. As companies replace interactive voice response (IVR) with intelligent virtual agents (IVA) and begin to collect and analyze data, use cases are likely to grow more sophisticated—which is one reason Google Cloud is continuing to invest in technologies to make our CCAI offerings even more useful, as well as best practices like the following:

• CCAI Agent Assist and Insights are a great first step in AI transformation. They let contact center owners enable call transcription and use Topic Modeling to identify conversation themes that demand attention. Human agents can automatically generate high-quality conversation summaries to reduce call wrap-up time, and the associated costs, while improving business insights. We are working to make these features available both in CCAI Platform and with our partner ISVs.
• Chat and call steering are the first step for IVA automation. Another area of broad impact is conversational chat or call steering, in which friction is reduced by routing customers to the correct virtual or live agent experience. Many call centers rely on IVR systems in which customers have to use a keypad to select an option. Enterprise leaders tell us that attrition is very high throughout this process: some customers angrily hang up without resolution and, just as bad, many simply pound a single key in hopes of reaching a human agent, leading to the customer reaching the wrong person because their issue was never correctly identified or routed. Using Dialogflow’s natural language understanding (NLU) capabilities can sweep away such problems, with the customer more likely to not only reach the appropriate resources, but also share conversational data from which insights can be gleaned. It’s an approach that can pay dividends right away, and a quick first step to IVA automation.

In coming months, we will continue to work on these and other capabilities that are targeted to deliver higher and quicker value to our customers. We plan to release pre-built components to help companies tackle call center use cases in specific industries, for example. We’ll also continue to partner with companies that share our vision of transforming customer experiences with AI, such as TTEC, a provider of customer experience technology and software.

“TTEC Digital and Google Cloud have a shared vision for transforming global CX delivery through artificial intelligence, digital innovation, and operational excellence,” said Sam Thepvongs, VP of TTEC Digital. “With CCAI Platform, we can offer our largest enterprise customers a strategic blueprint for moving to the cloud while adopting a leading, AI-powered contact center platform. We couldn’t be more excited about this evolution of Google Cloud’s groundbreaking CCAI portfolio, and the opportunity to help our customers digitally transform their CX through this partnership.”

To get started with CCAI Platform, visit our solutions page or checkout our new omni channel demo video—and don’t forget to join us at Google Cloud Next ’22, where I’ll be sharing exciting new updates for CCAI in my session, “Delight customers in every interaction with Contact Center AI.”

Businesses rely on processing an inflow of documents to drive processes and make decisions. Many such documents are combined into a single file. For example, a loan application may have a driver’s license, paystub, W2, bank statement, and other document types within a single file. The complexity of handling many document types within a single file makes it difficult for businesses to manage at scale. 

At Google Cloud, we’re committed to solving these challenges with continued investment in our Document AI solutions suite which offers machine learning products for document processing and insights. Document AI Workbench helps users quickly build ML models with world-class accuracy, trained for their specific use cases. In February 2023, we launched the Custom Document Extractor (CDE) in General Availability (GA) to help users extract structured data from documents in production use cases. In March 2023, we launched the Custom Document Classifier (CDC) in GA to help automatically classify document types. Today, we announce the newest feature of Document AI Workbench, Custom Document Splitter (CDS) in GA to help users automatically split and classify multiple documents within a single file. 

CDS provides tangible business value to customers by helping them sort and classify documents. For example, businesses can validate if they have all the needed documents from an applicant. Furthermore, individually classified documents enable businesses to better automate downstream processes, including selecting the proper storage, analysis, or processing steps based on the document type. The efficiencies enabled by CDS helps businesses lower their document processing time and cost.

Benefits of splitting and classification models in Document AI Workbench 

Document AI Workbench can save time and money by simplifying model training, from dataset management, to testing, to deployment. CDS helps businesses achieve higher automation rates to scale processes while lowering costs.

Sean Earley, VP of Delivery Services at Zencore said, “We completed a project for a large bank using Document AI Workbench to split, classify, and extract data from documents to automate Home Mortgage Disclosure Act reporting. Given the accuracy of the models we built, our client estimated increasing loan reporting coverage from 20% to 100% while eliminating thousands of errors per year, drastically reducing the operational cost of the bank’s compliance reporting procedures.”  

Fabian Beckmann, Manager Artificial Intelligence & Data at Deloitte Consulting GmbH said, “By leveraging Document AI’s Custom Document Splitter, our client, Commerzbank, a large european bank, can effortlessly segment customer submissions tailored to their back-office requirements, significantly diminishing the need for extra manual sorting or routing. This integration paves the way towards seamless automation within the Document AI pipeline, delivering substantial business benefits.“

According to Kaïs Albichari – ML Tribe Tech Lead, G Cloud at IT services firm Devoteam, “Custom Document Splitter (CDS) has helped one of our clients in the financial services industry save significant time and improve data accuracy. By identifying which parts of documents they can discard and which they retain for entity extraction, CDS has helped the company automate its document processing tasks. The implementation resulted in a more efficient and streamlined workflow, freeing employees to focus on other tasks. Devoteam’s G Cloud team helped the company implement CDS and achieve these benefits.”

Frank Neugebauer, a Google Cloud Insurance Solutions Consultant, worked with a Fortune 100 insurance company and used CDS to create a model to split and classify millions of insurance documents with up to 98% accuracy. With this information, the insurer can better understand the nature of their unstructured data to inform business strategy, including volume for specific document types to inform extraction work. The customer considers this level of insight unprecedented in their 200+ year history.  

How to use Custom Document Splitter

You can leverage a simple interface in the Google Cloud Console and a set of public APIs to prepare training data, create and evaluate models, deploy a model into production, and call an API endpoint to split and classify document types. You can follow the documentation for instructions to create, train, evaluate, deploy, and run predictions with models.

Import and prepare training data

To get started, import and label documents to train and evaluate an ML model. 

To quickly build a training dataset, import single documents, one document per file, and bulk label them with the relevant document type. You can import one folder or multiple folders at once and choose the correct document type per folder. As shown in the next image, one import could have a folder with 200 bank statements, another folder with 200 W2s, another folder with 200 paystubs, etc., all of which are labeled at once while imported. Up to 30,000 documents and 100,000 pages can be inputted for training. This way, you can build a training dataset with hundreds of labeled documents per class in minutes. As always, if documents are already labeled using other tools, simply import labels with JSON in the Document format.

You can initiate training with a click of a button. Once you have trained a model, you can use it  to automatically label documents added to your dataset, letting you quickly build robust test and training datasets to evaluate and improve model performance.

To accurately evaluate a CDS model, import files which contain multiple document types within the same file and assign them to the test dataset. Then, use a simple interface to define document boundaries and types.

The ground truth you label in the test dataset is used to evaluate splitting and classification predictions from the CDS model.

Going into production

Once a model meets accuracy targets, it’s time to deploy into production and call the API endpoint to split and classify document types.

Getting started with Document AI Workbench 

Custom Document Splitter is publicly available in GA and ready to help customers automate document splitting and classification. Learn more via our Document AI Workbench web page, Document AI Workbench documentation or try it out in the Google Cloud Console.

Powered by Vertex AI (Google Cloud’s platform for accelerating development and deployment of machine learning models into production), SAVI (Semi Automated Vision Inspection)1 is transforming surgical instrument identification and cataloging, leading to fewer canceled surgeries and easing pressure on surgery waitlists.

Max Kelsen, an analytics and software agency that specializes in machine learning, has worked closely with Google Cloud and Johnson & Johnson MedTech to create a system that can manage tens of thousands of individual devices, their characteristics, and how they apply to each set or tray used by a surgeon. SAVI does this while delivering a one in 10,000 real-world error rate, much faster and more accurately than manual processes currently in use across the industry. Implementing SAVI can also unlock end-to-end visibility and traceability across the surgical set supply chain and provide advanced analytics and insights.

Eliminating time-consuming manual processes

Surgeons need a large number of specialist instruments and devices to complete complex, delicate procedures. Because each tray of these instruments can typically cost more than $350,000, and having every type of set on shelf at every surgical facility is not feasible, manufacturers generally loan them to hospitals for procedures, such as inserting one of the manufacturers’ implants into a patient’s knee. Once a procedure is complete, the hospital returns the instrument tray to the manufacturer for storage and re-distribution to other hospitals as needed.

Each time a hospital returns a tray, the manufacturer needs to check that each instrument is there, correctly placed, cleaned, and fit for the purpose of the next procedure. As each set may hold more than 400 instruments, completing this process manually is complex and time-consuming. While each tray is checked before and after surgery at the hospital, and again when it arrives and leaves the manufacturer’s facility, Max Kelsen finds that 5% of surgeries can still be affected by missing, broken or bent instruments. This has a severe downstream impact on private hospitals in particular, directly affecting patient safety and outcomes; in Australia, for example, around 60% of surgeries are performed in private hospitals.

Johnson & Johnson MedTech has 60,000 surgical trays across the Asia-Pacific, and loans these trays out about 100,000 times per month. The manufacturer approached Max Kelsen to help design and develop a solution to make the supply chain more efficient, and to give more visibility into asset movement. As a Google Cloud Partner specializing in applying machine learning at scale in healthcare contexts, Max Kelsen had the expertise and track record to meet Johnson & Johnson MedTech’s need for a globally scalable solution that was engineered for quality and performance.

The first step was to establish a baseline for the project by determining how long the manufacturer’s team took to process each tray, and to set an efficiency number. We then spent six months determining and evaluating how to deliver a robust, accurate solution that outperformed current manual and labor-intensive methods in processing instruments and trays, globally. Our work included extensive technical feasibility research involving a representative sample for the variety and complexity of sets, trays, and devices needed for different types of surgery, including orthopedics, spinal trauma, and maxillofacial groups.

Working with Google Cloud to accelerate and de-risk the project

This is a familiar problem that is industry-wide. The issue has been widely explored and tried with a number of technologies over several years without producing the scalability and performance results required to make this an appropriate and feasible solution. Google Cloud partnered with Max Kelsen to accelerate and de-risk this large and strategic project for a mutual customer.

Technical feasibility took four months, prior to a year-long production pilot of SAVl in a distribution center in Queensland that services over 100 hospitals. After obtaining enough real-world data and experience to validate that the solution was as scalable and as accurate as needed, an Asia-Pacific rollout of the system commenced. SAVI is now live across Johnson & Johnson MedTech’s operations in Australia, New Zealand, and Japan, garnering recognition with a JAISA excellence award.

Google Cloud machine learning is integral to SAVI. Google Cloud’s technologies were a big differentiator for Max Kelsen’s engineering team in delivering the breakthroughs needed at scale, and in production, to meet Johnson & Johnson MedTech’s needs.

Reducing checking and documentation time

Running SAVI in Google Cloud has reduced the time Johnson & Johnson MedTech needs to check and document inspections of these surgical instrument sets by over 40%. The application also delivers consistent measurable quality that is often hard to measure at scale when using manual processes. During the pandemic, the application enabled Johnson & Johnson MedTech to operate with a lower headcount for the same volume output, enabling the organization to quickly service a backlog of waiting list surgeries.

In addition, the automation delivered with SAVI has reduced the time required to bring technicians up to speed on quality control processes, from eight to 12 months down to just three months, enhancing productivity and performance while delivering a more robust workforce.

So how does SAVI work in a real-world context? SAVI is deployed via a tablet and a web-based application incorporates an API to photograph the medical device trays, as shown below. Max Kelsen captures the photograph and sends it to a range of different services, via an API endpoint hosted on Google Cloud:

• Image information is stored in Cloud Storage
• Data relating to the trays is stored in Cloud SQL for PostgreSQL
• APIs and web UI components run in CloudRun
• Analytics data is stored within BigQuery

Once this tray and device onboarding stage is completed, the next step is to perform inferences from the images and data. By hosting online models with Kubeflow model serving on GKE, we enable a model to identify all the instruments in a tray at low latency.

Vertex AI Workbench notebooks are used for data exploration and modeling. Kubeflow training pipelines hosted on GKE are executed to produce machine learning models for specific surgical instrument sets. Several hundred machine learning models are then hosted with Kubeflow model serving on GKE, with state and analytics managed using Firebase. Using machine learning to infer from images whether any devices are incorrectly placed, dirty, or otherwise not fit for purpose, the data is then returned to the tablet for the user to respond accordingly.

Based on our success to date with SAVI, it is now available on Google Cloud Marketplace to help healthcare organizations achieve machine learning-powered efficiencies across a range of use cases, and ultimately improve patient safety and outcomes.

Not to be confused with the usage of Visual Inspection Model (Assembly) available in Vertex AI Vision