Verve Group is an ecosystem of demand and supply technologies fusing data, media, and technology to deliver results and growth to both advertisers and publishers – no matter the screen or location, no matter who, what, or where a customer is. Classifying massive amounts of this unstructured data at scale is the first step in helping to surface relevant, high-quality content to users—and that’s where natural language processing (NLP) comes in.

Verve Group uses the NLP API from Google Cloud’s Vertex AI to fetch data for their internal content classification quality verification and as an additional source for building categorization models. By leveraging the NLP API’s Content Classification models, which are now generally available and offer Google’s latest large language model (LLM) technology, Verve Group powers classification through an updated and expanded training data set with over 1,000 labels and support for 11 languages (Chinese, French, German, Italian, Japanese, Korean, Portuguese, Russia, Spanish, and Dutch join previously-available English). 

Verve Group has been using Google Cloud’s NLP API since day one, because of both the ease of implementation and the quality compared to competing NLP products. With documentation that is “comprehensive and self-explanatory,” the NLP API “allows for fast adoption and implementation, from test models all the way to production,” said Rami Alanko, GM of Verve Group.

Leveraging the NLP API has facilitated Verve Group’s fast go-to-market motions by enabling its customers to quickly discover and classify new content. “Operating on a global level with tens of different regions and languages, we have still been able to maintain high quality for our product and high retention rates with our clients,” Rami shared. “In a recent client case, we achieved 82% improvement in CTR when optimized with content quality measurements enabled by the API. In another client case, we drove brand safety risk down to 0.16% from 4% thanks to classification quality. Along with the new functionalities of the Google NLP, I can only see this trend continuing to strengthen.”

Verve Group is excited to further expand their NLP use cases by leveraging the new Content Classification models, which have already helped them expand their classification inventory, improve the quality and performance of their quality verification for customers, and unlock new use cases for NLP. “Our classification model accuracy improved 41% using Google NLP as a verification partner,” said Rami.  

Additionally, Verve Group is now using the API for metadata analysis on a  large image database. “We browse the database and run the image metadata via our classification. This flow enables us to classify images reliably aligned with our standard classification.  We pretty much use the same data flow for our runtime in-app textual content analysis, therefore allowing for close to real-time consumer engagement,” Rami added.

To learn more about how companies are leveraging NLP API from Google Cloud Vertex AI, click here, and to learn more about Google Cloud’s work with foundation models and generative AI, read The Prompt on Transform with Google Cloud.

One of the biggest challenges when serving machine learning models is delivering predictions in near real-time. Whether you’re a retailer generating recommendations for users shopping on your site, or a food service company estimating delivery time, being able to serve results with low latency is crucial. That’s why we’re excited to announce Private Endpoints on Vertex AI, a new feature in Vertex Predictions. Through VPC Peering, you can set up a private connection to talk to your endpoint without your data ever traversing the public internet, resulting in increased security and lower latency for online predictions. 

Configuring VPC Network Peering

Before you make use of a Private Endpoint, you’ll first need to create connections between your VPC (Virtual Private Cloud) network and Vertex AI.  A VPC network is a global resource that consists of regional virtual subnetworks, known as subnets, in data centers, all connected by a global network. You can think of a VPC network the same way you’d think of a physical network, except that it’s virtualized within GCP. If you’re new to cloud networking and would like to learn more, check out this introductory video on VPCs.

With VPC Network Peering, you can connect internal IP addresses across two VPC networks, regardless of whether they belong to the same project or the same organization. As a result, all traffic stays within Google’s network.

Deploying Models with Vertex Predictions

Vertex Predictions is a serverless way to serve machine learning models. You can host your model in the cloud and make predictions through a REST API. If your use case requires online predictions, you’ll need to deploy your model to an endpoint. Deploying a model to an endpoint associates physical resources with the model so it can serve predictions with low latency. 

When deploying a model to an endpoint, you can specify details such as the machine type, and parameters for autoscaling. Additionally, you now have the option to create a Private Endpoint. Because your data never traverses the public internet, Private Endpoints offer security benefits in addition to reducing the time your system takes to serve the prediction when it receives the request. The overhead introduced by Private Endpoints is minimal, achieving performance nearly identical to DIY serving on GKE or GCE.  There is also no payload size limit for models deployed on the private endpoint.

Creating a Private Endpoint on Vertex AI is simple.

In the Models section of the Cloud console, select the model resource you want to deploy.

Next, select DEPLOY TO ENDPOINT

In the window on the right hand side of the console, navigate to the Access section and select Private. You’ll need to add the full name of the VPC network for which your deployment should be peered.

Note that many other managed services on GCP support VPC peering, such as Vertex Training, Cloud SQL, and Firestore. Endpoints is the latest to join that list.

What’s Next?

Now you know the basics of VPC Peering and how to use Private Endpoints on Vertex AI. If you want to learn more about configuring VPCs, check out this overview guide. And if you’re interested to learn more about how to use Vertex AI to support your ML workflow, check out this introductory video. Now it’s time for you to deploy your own ML model to a Private Endpoint for super speedy predictions!

Investing in Artificial Intelligence (AI) can bring a competitive advantage to your organization. If you’re in charge of an AI or Data Science team, you’ll want to measure and maximize the value that you’re providing. Here is some advice from our years of experience in the field. 

A checklist to embark on a project: 

As you embark on projects we’ve found it’s good to have the following areas covered: 

• Have a customer. It’s important to have a customer for your work, and that they  agree with what you’re trying to achieve. Be sure to know what value you’re delivering to them. 
• Have a business case.  This will rely on estimates and assumptions, and may take no more than a few minute’s work.  You should revise this, but always know what justifies your team’s effort, and what you (and your customer) expect to get in return. 
• Know what process you will change or create. You’ll want to put your work in production, so you have to be clear about what business operations are changing or created around your work and who needs to be involved to make it happen
• Have a measurement plan. You’ll want to show that ongoing work is impacting some relevant business indicator. Measure and show incremental value. The goal of these measurements is to establish what has changed because of your project that would otherwise not have changed. Be sure to account for other factors like seasonality or other business changes that may affect your measurements.
• Use all the above to get your organization’s support for your team and your work. 

What measures to use?

As you start the work, what measures and indicators can you use to show that your team’s work is useful for your organization?

How many decisions you make. A major function of ML is to automate and optimize decisions: which product to recommend, which route to follow, etc. Use logs to track how many decisions your systems are making. 

Changes to revenue or costs. Better and quicker decisions often lead to increased revenue or savings. If possible, measure it directly, otherwise estimate it (for example fuel costs saved from less distance traveled, or increased purchases from personalized offers). 

As an example, the Illinois Department of Employment Security is using Contact Center AI to rapidly deploy virtual agents to help more than 1 million citizens file unemployment claims. To measure success the team tracked the two outcomes:  (1) the number of web inquiries and voice calls they were able to handle, and (2) the overall cost of the call center after the implementation. Post implementation, they were able to observe more than 140,000 phone and web inquiries per day and over 40,000 after-hours calls per night. They  also anticipate an estimated annual cost savings of $100M based on an initial analysis of IDES’s virtual agent data (see more in the link to case study).

Implementation costs. The other side of increased revenue or savings, is to put your achievements in the context of how much they cost. Show the technology costs that your team incurs and, ideally, how you can deliver more value, more efficiently. 

How much time was saved.  If the team built a routing system then it saved travel time, if it built an email classifier then it saved reading time, etc. Quantify how many hours were given back to the organization thanks to the efficiency of your system. 

In the medical field, quicker diagnostics matter. Johns Hopkins University’s Brain Injury Outcomes (BIOS) Division has focused on studying brain hemorrhage aiming to improve medical outcomes. The team identified the time to insights as a key metric in measuring business success. They experimented with a range of cloud computing solutions like Dataflow, Cloud Healthcare API, Compute Engine, and AI Platform for distributed training to accelerate iterations. As a result, in their recent work they were able to accelerate insights from scans from approximately 500 patients from 2,500 hours to 90 minutes.

How many applications your team supports. Some of your organization’s operations don’t use ML (say reconciling financial ledgers) but others do. Know how many parts of your organization benefit from the optimization and automation your team builds.

User experience. You may be able to measure your customer’s experience: fewer complaints, better reviews, reduced latency, more interactions, etc. This is valid both for internal and external stakeholders. At Google we measure usage and regularly ask for feedback on any internal system or process.

One of our customers, The City of Memphis, is using VisionAI and ML to tackle a common but very challenging issue: identifying and addressing potholes.  The implementation team identified the percentage increase of potholes identified as one of the key metrics along with accuracy and cost savings. The solution captures video footage from it’s public vehicles and leverages Google Cloud capabilities like Compute Engine, AI Platform, and BigQuery to automate the review of videos.  The project increased  pothole detection by 75% with over 90% accuracy. By measuring and demonstrating these outcomes, the team proved the viability of a cost-effective, cloud-based machine learning model and is looking into new applications of AI and ML that will further improve city services and help it build a better future for its 652,000 residents. 

Acknowledgements

Filipe and Payam would like to thank our colleague and co-author Mona Mona (AI/ML Customer Engineer, Healthcare and lifesciences) who contributed equally to the writing.