Improved TabNet on Vertex AI: High-performance, scalable Tabular Deep Learning
2516
Of your peers have already read this article.
2:30 Minutes
The most insightful time you'll spend today!
Data scientists choose models based on various tradeoffs when solving machine learning (ML) problems that involve tabular (i.e., structured) data, the most common data type within enterprises. Among such models, decision trees are popular because they are easy to interpret, fast to train, and can obtain high accuracy quickly from small-scale datasets. On the other hand, deep neural networks offer superior accuracy on larger datasets, as well as the benefits of end-to-end learning, but are black-box and difficult to interpret. TabNet, an interpretable deep learning architecture developed by Google AI, combines the best of both worlds: it is explainable, like simpler tree-based models, and can achieve the high accuracy of complex black-box models and ensembles.
We’re excited to announce that TabNet is now available in Vertex AI Tabular Workflows! Tabular Workflows provides fully managed, optimized, and scalable pipelines, making it easier to use TabNet without worrying about implementation details, and to deploy TabNet with the MLOps capabilities of Vertex. TabNet on Vertex AI Tabular Workflows is optimized for efficient scaling to massive tabular datasets. Moreover, TabNet on Vertex AI Tabular Workflows come with machine learning improvements on top of the original TabNet, yielding better accuracy for real-world data challenges.
TabNet on Vertex AI is well-suited for a wide range of tabular data tasks where model explainability is just as important as accuracy, such as financial asset price prediction, fraud/cyberattack/crime detection, retail demand forecasting, user modeling, credit/risk scoring, diagnosis from healthcare records, and product recommendations.
Overview of Tabnet
TabNet has a specially-designed architecture (overviewed in Fig. 1), based on sequential attention, that selects which model features to reason from at each step. This mechanism makes it possible to explain how the model arrives at its predictions and the thoughtful design helps with superior accuracy. TabNet not only outperforms alternative models (including neural networks and decision trees) but also provides interpretable feature attributions. More details, including results on academic benchmarks, can be found in our AAAI 2021 paper.
Since its publication, TabNet has received significant traction from various enterprises across different industries and a variety of high-value tabular data applications (most of which include the ones for which deep learning was not even used a priori). It has been used by numerous enterprises like Microsoft, Ludwig, Ravelin, and Determined. Given high customer interest on TabNet, we’ve worked on making it available on Vertex given the real-world deep learning development and productionization needs, as well as improving its performance and efficiency.
Highlights of TabNet on Vertex AI Tabular Workflows
Scaling to Very Large Datasets
Fueled by the advances in cloud technologies like BigQuery, enterprises are increasingly collecting more tabular data, and datasets with billions of samples and hundreds/thousands of features are becoming the norm. In general, deep learning models get better learning from more data samples, and more features, with the optimal methods as they can better learn the complex patterns that drive the predictions. The computational challenges become significant though when model development on massive datasets is considered. This results in high cost or very long model development times, constituting a bottleneck for most customers to fully take advantage of their large datasets. With TabNet on Tabular Workflows, we’re making it more efficient to scale to very large tabular datasets.
Key Implementation Aspects: The TabNet architecture has unique advantages for scaling: it is composed mainly of tensor algebra operations, it utilizes very large batch sizes, and it has high compute intensity (i.e., the architecture employs a high number of operations for each data byte transmitted). These open a path to efficient distributed training on many GPUs, utilized to scale TabNet training in our improved implementation.
In TabNet on Vertex AI Tabular Workflows, we have carefully engineered the data and training pipelines to maximize hardware utilization so that users can get the best return for their Vertex AI spending. The following features enable scale with TabNet on Tabular workflows:
- Parallel data reading with multiple CPUs in a pipeline optimized to maximize GPU utilization for distributed training, reflecting best practices from Tensorflow.
- Training on multiple GPUs that can provide significant speedups on large datasets with high compute requirements. Users can specify any available machine on GCP with multiple GPUs, and the model will automatically run on them with distributed training.
- For efficient data parallelism with distributed learning, we use Tensorflow mirrored distribution strategy to support data parallelism across many GPUs. Our results demonstrate >80% utilization with several GPUs on billion-scale datasets with 100s-1000s of features.
Standard implementations of deep learning models could yield a low GPU utilization, and thus inefficient use of resources. With our implementation, TabNet on Vertex, users can get the maximal return on their compute spend on large-scale datasets.
Examples on real-world customer data: We have benchmarked the training time specifically for enterprise use cases where large datasets are being used and fast training is crucial. In one representative example, we used 1 NVIDIA_TESLA_V100 GPU to achieve state-of-the-art performance in ~1 hour on a dataset with ~5 million samples. In another example, we used 4 NVIDIA_TESLA_V100 GPUs to achieve state-of-the-art performance in ~14 hours on a dataset with ~1.4 billion samples.
Improving Accuracy given Real-World Data Challenges
Compared to its original version, TabNet on Vertex AI Tabular Workflows has improved machine learning capabilities. We have specifically focused on the common real-world tabular data challenges. One common challenge for real-world tabular data is numerical columns having skewed distributions, for which we productionized learnable preprocessing layers (e.g. including parametrized power transform families and quantile transformations) that improve the TabNet learning. Another common challenge is the high number of categories for categorical data, for which we adopted tunable high-dimensional embeddings. Another one is imbalance of label distribution, for which we added various loss function families (e.g. focal loss and differentiable AUC variants). We have observed that such additions can provide a noticeable performance boost in some cases.
Case studies with real-world customer data: We have worked with large customers to replace legacy algorithms with TabNet for a wide range of use cases, including recommendation, rankings, fraud detection, and estimated arrival time predictions. In one representative example, TabNet was stacked against a sophisticated model ensemble for a large customer. It outperformed the ensemble in most cases, leading to a nearly 10% error reduction on some of the key tasks. This is an impressive result, given that each percentage improvement on this model resulted in multi-million savings for the customer!
Out-of-the-box Explainability
In addition to high accuracy, another core benefit of TabNet is that, unlike conventional deep neural network (DNN) models such as multi-layer perceptrons, its architecture includes explainability out of the box. This new launch on Vertex Tabular Workflows makes it very convenient to visualize explanations of the trained TabNet models, so that the users can quickly gain insights on how the TabNet models arrive at its decisions. TabNet provides feature importance output via its learned masks, which indicate whether a feature is selected at a given decision step in the model. Below is the visualization of the local and global feature importance based on the mask values. The higher the value of the mask for a particular sample, the more important the corresponding feature is for that sample. Explainability of TabNet has fundamental benefits over post-hoc methods like Shapley values that are computationally-expensive to estimate, while TabNet’s explanations are readily available from the model’s intermediate layers. Furthermore, post-hoc explanations are based on approximations to nonlinear black-box functions while TabNet’s explanations are based on what the actual decision making is based on.
Explainability example: To illustrate what is achievable with this kind of explainability, Figure 2 below shows the feature importance for the Census dataset. The figure indicates that education, occupation, and number of hours per week are the most important features to predict whether a person can earn more than $50K/year (the color of corresponding columns are lighter). The explainability capability is sample-wise, which means that we can get the feature importance for each sample separately.
Benefits as a Fully-Managed Vertex Pipeline
TabNet on Vertex Tabular Workflows makes the model development and deployments tasks much simpler – without writing any code, one can obtain the trained TabNet model, deploy it in their application, and use the MLOps capabilities enabled by Vertex Managed Pipelines! Some of these benefits are highlighted as:
- Compatibility with Vertex AI ML Ops for implementing automated ML at scale including products like Vertex AI Pipelines and Vertex AI Experiments.
- Deployment convenience: Vertex AI prediction services, both in batch and online mode, are supported out-of-the-box.
- Customizable feature engineering to enable the best utilization of the domain knowledge of users.
- Using Google’s state-of-the-art search algorithms, automatic tuning to identify the best-performing hyperparameters, with automatic selection of the appropriate hyper-parameter search space based on dataset size, prediction type, and training budget.
- Tracking the deployed model and convenient evaluation tools.
- Easiness in comparative benchmarking with other models (such as AutoML and Wide & Deep Networks) as the user journey would be unified.
- Multi-region availability to better address international workloads.
More details
If you’re interested in trying TabNet on Vertex AI on your tabular datasets, please check out Tabular Workflow on Vertex AI and fill out this form.
Acknowledgements: We’d like to thank Nate Yoder, Yihe Dong, Dawei Jia, Alex Martin, Helin Wang, Henry Tappen and Tomas Pfister for their contributions to this blog.
Google Cloud’s Recommendation AI Helps Bazaarvoice with 60 Percent increase in CTR
4507
Of your peers have already read this article.
2:00 Minutes
The most insightful time you'll spend today!
Not long ago, building AI into recommendation engines was a daunting, expensive task that could take years to get off the ground. But as Bazaarvoice has shown, with the help of cloud services, the time from AI investment to business outcomes is shorter than ever.
Bazaarvoice is the leading provider of product reviews and user-generated content (UGC) solutions that help brands and retailers understand and better serve customers. Its 2019 acquisition of Influenster.com, a community of consumer reviewers 6.5 million strong, expanded the Bazaarvoice portfolio with a platform where consumers can share their candid opinions — and share they have, over 54 million times.
After the acquisition, Bazaarvoice expanded the site’s product diversity by 53%, to more than 5.4 million unique products. To keep user engagement high, Influenster must be seen as both a source of trusted, transparent reviews and a place for customers to discover useful, relevant products for the first time. By introducing shoppers to new products Influenster not only provides value to customers but also helps brands collect consumer insights.
Influenster started out as a place where people gathered to share their honest thoughts on beauty products but quickly expanded to nearly every category, from Art to Wearables. Because of the much smaller scope, the site started and flourished under a rules-based recommendation engine. However, as Influenster expanded its scope under Bazaarvoice, a more robust recommendation system became necessary. In its earliest days, Influenster was successful because of the human perspective it offered: For every product there was a litany of reviews and images that made users feel as if they were getting an endorsement on a product from a friend.
The Bazaarvoice engineering team asked themselves how they could keep that same feeling of personalization with an ever-growing catalog of items and categories. They needed recommendations that could scale with the site, rather than requiring more rules be constructed each time a new product category was introduced. They also needed to ensure the Influenster experience would remain performant even towards unknown members.
Bazaarvoice tested out several recommendation engines, benchmarking each against their current rules-based system. In the end they decided on Google Cloud’s Recommendations AI because of its transparent billing, ease of integration and setup, and naturally, its proven results.
Transparent Billing
“Part of what the engineers loved was they knew exactly what it was going to cost as it scaled” says Nick Shiftan, SVP, Content Acquisition Services Product Unit for Influenster. The goal was to build once and innovate rather than leave a wake of technical debt only to be tackled when costs grew unexpectedly out of control. Google Cloud’s straightforward and pay-as-you-go billing allowed them to anticipate how costs would grow as user interactions did and plan accordingly.
Ease of integration
“I’m positively surprised how Google packed such a complex system in a very easy-to-use API” remarks Eralp Bayraktar, the Software Engineering team lead overseeing the project. Because the original team was made of just one full-time engineer the ease of integration became an even more critical feature. Not only does Recommendations AI pull from years of suggestion expertise in Google Search and YouTube, but in combination with Ad’s Merchant Center, it also creates a streamlined process for importing product metadata. From there, creating a model becomes a matter of picking the preferred recommendation type and then the business objective to optimize for. Once the model is created and the API integrated into the website, the code is already deployed at the global scale: There are no further architectural considerations to ensure recommendations are available to users worldwide. For Bazaarvoice, this meant going from ideation to production in one month.
Proven Results
“We have used it for product recommendations and off-loaded our DB-tiring business logic to Recommendations AI, which resulted in overall faster response times and much better recommendations as proven by our A/B tests,” Eralp continues.
Bazaarvoice began by A/B testing Recommendations AI against their rules-based system. Early on in the experimental phase they noticed a clear and consistent 60% increase in the click-through rate over their original recommendation system.
Even more impressive was the performance on Unknown Members. For every person that signs up for an account on Influenster.com there are many other visitors that come to the website and leave without fully registering. This is typically referred to as the “cold start” problem in the industry — how do you figure out what to recommend to those people without their history, behavior, or preferences? Recommendations AI gives you the option to input and train on unknown users, and by providing metadata on products, it can provide high-quality suggestions to registered members and first-time users alike.
With a mind to the future, Eralp concludes his thoughts on Bazaarvoice’s experience: “It enables discovery by adding an adjustable percentage of cross-category products [for] healthier [traffic distribution] across all our catalog. We are investing in data science and having the Recommendations AI as the baseline is a good challenge for us to thrive.”
To learn more about Recommendations AI and how it can help your organization thrive, check out our recently published 4 part guide which kicks off with an overview on “How to get better retail recommendations with Recommendations AI.” This series also covers data ingestion, modeling, as well as serving predictions & evaluating Recommendations AI. You can also easily get started with our Quickstart Guide.
5658
Of your peers have already downloaded this article
5:30 Minutes
The most insightful time you'll spend today!
Public cloud, big data, and AI technologies offer competitive advantages and cost savings for capital markets firms ready to make the transition. This paper discusses the three phases capital markets firms go through in transitioning to public cloud, and the workloads, benefits, and cultural changes that characterize the three phases:
Infrastructure Optimizers: The first step on the public cloud journey, where firms focus on migrating specific workloads to save costs.
Cautious Strategists: Firms build on the success of their first public cloud migrations, and begin to change the way they develop technology to increase cost savings and start taking advantage of capabilities only available on public cloud.
Transformative Innovators: Firms shift to a fully public cloud-enabled mentality, and fully leverage the flexibility and agility of the public cloud to build industry-changing solutions and attract top IT talent.
Additionally, we reveal the five things that capital markets innovators who have advanced to the transformation phase do well in their adoption of cloud, big data, and AI technologies across the front, middle, and back office functions.
redBus: Mastering Big Data with Google BigQuery
5403
Of your peers have already read this article.
2:30 Minutes
The most insightful time you'll spend today!
In 2006, online travel agency redBus introduced internet bus ticketing in India, unifying tens of thousands of bus schedules into a single booking operation. (Think of it as Expedia for bus booking.) Using BigQuery, redBus crunches terabytes of booking and inventory data in mere seconds and at a fraction of the cost of other big data services. BigQuery also helps engineers fix glitches quickly, minimize lost sales, and improve customer service.
Challenge
Executives at the Bangalore-based redBus needed a powerful tool to analyze booking and inventory data across their system of hundreds of bus operators serving more than 10,000 routes. They considered using clusters of Hadoop servers to process the data but decided the system would take too much time to set up and would require a specialized staff to maintain it. It also would not provide the lightning-fast analysis they needed.
“It would have taken at least a couple of hours to analyze anything,” says Pradeep Kumar, a technical architect at redBus. “Crunching very large data sets would have been a day’s job. We needed something more powerful to get the real-time analysis we were looking for.”
Solution
Kumar and his colleagues learned about BigQuery and realized it was the right match for their data processing needs. The web-based service, which enables companies to analyze massive datasets using Google’s data processing infrastructure, is easy to set up and manage since its simple, SQL-like query language doesn’t require complex technology or specialized personnel. It also has low overhead costs.
The redBus team uses BigQuery as part of an intricate data collection and analysis process. Applications hosted on a range of servers continually pump information related to customer searches, seat inventory, and bookings into a centralized data collection system. Engineers upload the data to BigQuery, which provides answers to complex queries within seconds. For example, BigQuery helps redBus staff:
- Learn how many times customers searched for seats and found none or very few available, indicating more seats should be added to a route
- Investigate decreases in bookings and notify engineers if a technical problem is the cause
- Identify server problems by quickly analyzing data related to server activity
Results
BigQuery provides near real-time data analysis capabilities at 20% of the cost of maintaining a complex Hadoop infrastructure. Queries that would have required a day to analyze on a Hadoop framework take less than 30 seconds using Google’s web-based service.
“We explored several data analytics solutions. Nothing comes remotely close to the sheer power of Google BigQuery,” Kumar says. “It made large-scale data collection and crunching possible with little effort, which has translated to a significant business advantage.”
Google Cloud Platform results
- Analyzes data sets as large as 2 terabytes in less than 30 seconds using a simple, SQL-like language
- Saves time analyzing technical problems and customer booking trends
- Spends 80% less than they would have on a Hadoop infrastructure and avoids setting up and maintaining a complex infrastructure in-house
- Strengthens the company by improving customer service and engineering quality
The fast insights gained through BigQuery are also making redBus a stronger company. By minimizing the time it takes staff members to solve technical problems, BigQuery has helped improve customer service and reduce lost sales.
“Getting to the root of problems used to be really time-consuming,” Kumar says. “By the time we figured it out, customers might have given up. Now if there are booking problems, BigQuery helps us understand the reason right away. Choosing Google BigQuery was the right decision for our company.”
3318
Of your peers have already watched this video.
44:14 Minutes
The most insightful time you'll spend today!
Supercharge Marketing with Ready-Made, Centralized Smart Analytics
Your company has lots of valuable marketing data, but it’s spread across many systems and teams. That’s a problem. In fact, only 13% of organisations feel like they’re making the most out of their available customer data today.
And the reason is that data–marketing and customer data–is spread across many different places.
With Cloud, you can bring this data together in one place so you can better analyze and activate it.
Find out how new GCP tools for marketers and hear directly from brands, including Rituals and Waitrose, who are using them to solve use cases like audience segmentation, purchase prediction, and measuring lifetime value.
You’ll learn how Google Cloud is helping IT and marketing teams accelerate digital transformation within their organizations, to be data-driven, and customer-centric.
You’ll walk away with an understanding of how to adopt technology that transforms the way your company delivers marketing campaigns and customer experiences.
VCP Peering and Private Endpoints on Vertex AI to Better Security and Predictions in Near Real-time
4497
Of your peers have already read this article.
2:00 Minutes
The most insightful time you'll spend today!
One of the biggest challenges when serving machine learning models is delivering predictions in near real-time. Whether you’re a retailer generating recommendations for users shopping on your site, or a food service company estimating delivery time, being able to serve results with low latency is crucial. That’s why we’re excited to announce Private Endpoints on Vertex AI, a new feature in Vertex Predictions. Through VPC Peering, you can set up a private connection to talk to your endpoint without your data ever traversing the public internet, resulting in increased security and lower latency for online predictions.
Configuring VPC Network Peering
Before you make use of a Private Endpoint, you’ll first need to create connections between your VPC (Virtual Private Cloud) network and Vertex AI. A VPC network is a global resource that consists of regional virtual subnetworks, known as subnets, in data centers, all connected by a global network. You can think of a VPC network the same way you’d think of a physical network, except that it’s virtualized within GCP. If you’re new to cloud networking and would like to learn more, check out this introductory video on VPCs.
With VPC Network Peering, you can connect internal IP addresses across two VPC networks, regardless of whether they belong to the same project or the same organization. As a result, all traffic stays within Google’s network.
Deploying Models with Vertex Predictions
Vertex Predictions is a serverless way to serve machine learning models. You can host your model in the cloud and make predictions through a REST API. If your use case requires online predictions, you’ll need to deploy your model to an endpoint. Deploying a model to an endpoint associates physical resources with the model so it can serve predictions with low latency.
When deploying a model to an endpoint, you can specify details such as the machine type, and parameters for autoscaling. Additionally, you now have the option to create a Private Endpoint. Because your data never traverses the public internet, Private Endpoints offer security benefits in addition to reducing the time your system takes to serve the prediction when it receives the request. The overhead introduced by Private Endpoints is minimal, achieving performance nearly identical to DIY serving on GKE or GCE. There is also no payload size limit for models deployed on the private endpoint.
Creating a Private Endpoint on Vertex AI is simple.
In the Models section of the Cloud console, select the model resource you want to deploy.
Next, select DEPLOY TO ENDPOINT
In the window on the right hand side of the console, navigate to the Access section and select Private. You’ll need to add the full name of the VPC network for which your deployment should be peered.
Note that many other managed services on GCP support VPC peering, such as Vertex Training, Cloud SQL, and Firestore. Endpoints is the latest to join that list.
What’s Next?
Now you know the basics of VPC Peering and how to use Private Endpoints on Vertex AI. If you want to learn more about configuring VPCs, check out this overview guide. And if you’re interested to learn more about how to use Vertex AI to support your ML workflow, check out this introductory video. Now it’s time for you to deploy your own ML model to a Private Endpoint for super speedy predictions!
More Relevant Stories for Your Company
How AI and Analytics in EdTech are Living Upto the Hype
Over the last year, COVID-19 presented unforeseen challenges for practically every type of business and organization—including schools, colleges, and universities. For educational institutions, the pandemic was an unapologetic agent of acceleration, shifting one billion learners from in-person to online learning within two months. The rapid transition to online learning exposed
Enhancing SAP Build Process Automation with Google Document AI and Google Workspace
SAP Build Process Automation is designed to optimize business processes and boost efficiency. The platform helps both business users and developers alike digitize core workflows and incorporate artificial intelligence (AI) into time consuming and error-prone manual tasks. All digital paths can benefit from automation. The pandemic, supply chain shortages, and
AI Features in Apigee X Helps Build and Manage APIs at Scale
APIs are the backbone of digital transformation. Via APIs, you can securely share data and functionality with developers both inside and outside of your organizational boundaries, letting you build applications faster, seamlessly connect and interact with partners, and drive new business revenue. Because APIs encompass business-critical information, any downtime or
NLP in Healthcare Can Unlock Clinical Insights beyond Typical Data Format Barriers
Aches and pains. What, if anything, is the difference between those? And do you know a “myocardial infarction” from a “heart attack”? What about an “MI”? Is that shorthand, or part of the address of a hospital in Michigan? For people, it can be easy to understand the distinctions and
