UKG Ready: Meeting the Needs of Complex Machine Learning Models and Distributed Data Sets
3981
Of your peers have already read this article.
2:30 Minutes
The most insightful time you'll spend today!
Business Problem
UKG Ready primarily operates in the Small and Medium Business (SMB) space, so inherently many customers are forced to operate and make key business decisions with less Workforce Management (WFM) / Human Capital Management (HCM) data. In addition to volume, SMB lacks the variety of data needed to create a dynamic and agile organization. This puts SMB at a major disadvantage compared to larger segments.
Project Goals
People Insights module is committed to surfacing insights to customers in the context of their day-to-day duties and aid in decision making. With the SMB customer data limitations mentioned above, the goal of this project was to create a global dataset that augments individual customer data to bring light to less obvious, yet important information.
Challenges
UKG Ready is a highly configurable application that gives customers the opportunity to build solutions on a platform that meets their specific business needs. High configurability gives high flexibility to customers in their usage of the software. However, it becomes nearly impossible to create a global dataset for machine learning and data insights. UKG Ready manages just under 4 million of the US workforce and some 30,000+ customers. Despite the large employee dataset size, machine learning models that are specific to customers are starved for data because the individual customers have a relatively small employee population. Does that mean we cannot support our SMB customers’ decision making with ML?
Result
Partnering with Google, we were able to develop an approach that allowed us to standardize various domain entities (pay categories, time off codes, job titles, etc.) so that we could build a global dataset to augment SMB customer data. Using machine learning we were able to build a common vocabulary across our customer base. This common vocabulary encapsulates the nuances of how our customers manage their business and yet is generalized and standardized such that the data can be aggregated over the variety of customer configurations. This allows us to serve up practical insights to customers through various use cases. Our partnership allowed us to leverage Google Cloud Services to meet the needs of our complex machine learning models, distributed data sets and CI/CD processes.
How
UKG Ready decided to partner with Google for an end-to-end solution for the analytics offering. This allowed us to focus on our core business logic without having to worry about the platform, environment configurations, performance and scalability of the entire solution. We make use of various Google Cloud services such as Cloud Triggers, Cloud Storage, Cloud Functions, Cloud Composer, Cloud Dataflow, Big Query, Vertex AI, Cloud Pub/Sub… to host our analytics solution. Jenkins manages the entire CI/CD pipelines and cloud environments are configured and deployed using Terraform.
The standardization of business entities problem was solved in three distinct steps:
Step 1: Collecting aggregated data
We needed an approach to collect aggregated data from our highly distributed, sharded, multi-tenant data sources. We developed a custom solution that allows us to extract data aggregated at source for PII and GDPR considerations and transfer to Google Cloud Storage in the fastest manner possible. Data is then transformed and stored in Big Query. Services used: GCS, Cloud Functions, DataFlow, Cloud Composer and Big Query. All processes are orchestrated using Cloud Composer and detailed logging is available in Cloud Logging (Stackdriver).
Step 2: Applying NLP (Natural Language Processing)
Once we had the variety of customer configurations or the business entities available, we then applied NLP algorithms to categorize and standardize these in buckets. This approach assumes that customers use natural language for configurations like job titles, pay codes etc.
String Preparation
The input data for string preparation process is an entity string or several strings, that describe one entity object (like name-description pair or code-name pair). The output represents set of tokens that may be used to run a classification/clustering model. The process of string preparation tokenizes strings, replaces shortcuts, handles abbreviations, translates tokens, handles grammatical errors and mistypes
ML Models
Statistical
The idea of the model is to use defined target classes (clusters) and assign several tokens (anchors) to each of them an entity that has any of those tokens would be “attracted” to appropriate class. All other tokens are weighted according to frequencies of usage of theses tokens in the entities with anchor tokens:
Using anchor tokens, we are building kind-of Word2Vec - dimensionality of vector is equal to number of target classes. The higher the specific dimension (cluster) value, the higher the probability of entity to be included in appropriate cluster. Final prediction for entity tokens list for specific class is sum of weights of all the tokens included. Predicted cluster is a cluster that has maximal prediction score.
Lexical Model
We managed to generate reasonable amount of labeled data during statistical model implementation and testing. That opens a possibility to build “classical” NLP model that uses labeled data to train classification neural network using pretrained layers to produce token embeddings or even string embeddings. We started experimentation with pre-trained models like GloVe and got good results with single words and bi-grams but started getting issues in handling of n-grams. Our Google account team came to our rescue and recommended some white papers that helped formulate our strategy. We now use Tensorflow nnlm-en-dim128 model to produce string embeddings – it was trained on 200B records English Google News corpus and produces for each input string 128-dimensional vector. After that we use several Dense and Dropout layers to build a classification model.
Ensembling
To perform ensembling all the model results for each class are cast to probabilities using softmax transformation with scale normalization. Final predicted probability is maximal average score of both models among all the classes scores – appropriate class is predicted class.
The machine learning models are deployed on Vertex AI and are used in batch predictions. Model performance is captured at every prediction boundary and monitored for quality in production.
Step 3: Making available common vocabulary
Having the standardized vocabulary, we then needed a mechanism to have the results be available in UKG Ready reports and customer specific models like Flight Risk and Fatigue. For this we again used Google Services for orchestration, data transformation and data storage.
Once the modeling is complete, we made the customer specific models leveraging the above architecture be available in Reports. We utilized our proven existing technology choices in GCP for orchestration, data transformation and data storage
Results
We are able to build a common vocabulary of our customers’ business entities with good confidence. And be an expert advisor to our SMB customers in their decision-making using machine learning. With the advice of our Google account team and using Google services we can add value to our product in a relatively short amount of time. And we are not done! We continue to use this platform for new use cases, complex business problems and innovative machine learning solutions.
Sample result:
Special thanks to Kanchana Patlolla , AI Specialist, Google for the collaboration in bringing this to light
Google Migration and BigQuery Brings PedidosYa Closer towards its Goal of Becoming Data-driven
7250
Of your peers have already read this article.
2:00 Minutes
The most insightful time you'll spend today!
Editor’s note: PedidosYa is the market leader for online food ordering in Latin America, serving 15 markets and over 400 cities. It’s also one of the largest brands within the German multinational company Delivery Hero SE. With over 20 million app downloads, PedidosYa provides the best online delivery experience through 71,000+ online partners, including restaurants, shops, drugstores, and specialized markets.
Having constant access to fresh customer data is a key requirement for PedidosYa to improve and innovate our customer’s experience. Our internal stakeholders also require faster insights to drive agile business decisions. Back in early 2020, PedidosYa’s leadership tasked the data team to make the impossible possible. Our team’s mission was to democratize data by providing universal and secure access while creating a comprehensive information ecosystem across PedidosYa. We also had to achieve this goal while keeping costs under control— even during the migration stage and removing operational bottlenecks.
Challenges with legacy cloud infrastructure
PedidosYa first built its data platform on top of AWS. Our data warehouse ran on Redshift, and our data lake was in S3. We used Presto and Hue as the user interfaces for our data analysts. However, maintaining this infrastructure was a daunting task. Our legacy platform couldn’t keep up with the increasing analytics demands. For example, the data stored on S3 complemented by Presto/Hue required high operational overhead. This was because Presto and our IAM (identity access management) didn’t integrate well in our legacy ecosystem. Managing individual users and mapping IAM roles with groups and Kerberos was operationally time-consuming and costly. Further, sharding access on the S3 files was far too complicated to enable seamless ACLs (access control lists).
There were also challenges with workload management. Our data warehouse had batch data loaded overnight. If one analyst scheduled a query to run during the overnight ETL (extract, transform, load) workload, it would disrupt the current ETL task. This could stop the entire data pipeline. We’d have to wait until data engineers intervened with a manual fix.
It was also difficult to understand whether a query error was due to performance issues or platform resource exhaustion. This lack of clarity affected our data analysts’ ability to autonomously improve querying efficiency. Data team members needed to manually inspect personal queries looking for performance issues. Also, the current architecture was prone to a ‘tragedy of the commons’ situation; it was seen as an unlimited and free resource. As a result, it was impossible to disentangle the infrastructure from different stakeholder teams, as all had very different needs.
The decision to modernize our data warehouse
Given the growing challenges from our legacy platform, our tech team decided to transform our analytics environment with a modern data warehouse. They required the following key criteria from their next data platform:
- Scalability – The ability to grow with elastic infrastructure.
- Cost control – Cost management and transparency. These factors promote efficiency and ownership—both key aspects of data democratization.
- Metadata management – Intuitive data platform focusing on users’ previous SQL knowledge. Plus, being able to enrich the informational ecosystem with metadata, to diminish data gatekeepers.
- Ease of management – The team needed to reduce operational costs with a serverless solution. Data engineers wanted to focus on their key roles rather than acting as database administrators and infrastructure engineers. The team also wanted much higher availability, and to reduce the impact of maintenance windows and vacuum/analysis.
- Data governance and access rights – With a growing employee base with varying data access requirements, the team needed a simple yet comprehensive solution to understand and track user access to data.
Migrating to Google Cloud
After exploring other alternatives, we concluded Google Cloud had an answer to each of our decision drivers. Google Cloud’s serverless, managed, and integrated data platform, coupled with its seamless integration across open-source solutions, was the perfect answer for our organization. In particular, the natural integration with Airflow as a job orchestrator and Kubernetes for flexible on-demand infrastructure was key.
We used Dataflow together with Pub/Sub and Cloud Functions for our data ingestion requirements, which has made our deployment process with Terraform seamless. Because we set up everything in our environment programmatically, operation time has diminished. Google Cloud reduced the deployment process from about 16 hours in our legacy platform to 4 hours. This is partly due to the friendliness of automating the deployment (such as schema check, load test, table creation, build.) process with Terraform, Cloud Functions, Pub/Sub, Dataflow, and BigQuery on GCP. Input messages processed with Dataflow allow us to abstract and plan the schema changes according to the needs of the functional team. For example, schema changes raise an alarm, and then we can modify the raw layer table schema. By doing this, we ensure that backend modifications that we don’t control do not affect upper layers.
A key reason why we picked Google Cloud was because of its advanced cost and workload management coupled with its transparent log analytics. This information gives us a complete view into any query performance issues to make improvements on the fly. Further, we achieved a significant amount of cost savings by consolidating multiple tools to BigQuery.With BigQuery, we’ve been able to reduce our total cost per query by 5x.
This was due to a number of reasons:
- Automating pipeline deployment made it much simpler to maintain the data processing processes.
- Analysts are conscious about what queries they’re running, resulting in running better, more optimized queries.
- Analysts use a Data Studio dashboard to see their queries and all the associated costs. As a result, there’s a lot more transparency for each persona.
With these changes, we can easily manage and assign costs associated with each workload with their own cost centers using specific Google Cloud projects.
Change management is always challenging. However, BigQuery is intuitive and doesn’t have a steep learning curve from Hue/Hive on SQL basics. BigQuery also allowed the team to expand its capabilities and enabled them to properly work with nested structures, avoiding unnecessary joins and improving query efficiency. Additionally, we now use Data Catalog as our unique point of truth for metadata management. This allows our team to break the data access barriers and enable federation of data across the organization. By using Airflow to orchestrate everything, we keep track of every data stream. With this information, each end user can see their regularly used data entities’ status via the dashboard. This also adds transparency to our everyday data processes.
Finally, with Google Cloud’s IAM rules applied across the different products, data sharing and access is close to a noOps experience. We have programmatically implemented access according to roles and level access within the company. This allows certain pre-validated roles to view more sensitive information. These solutions help drive a more automated data governance experience.
Up next: Google Cloud AI/ML
The new stack based on BigQuery has created significant productivity gains. Freed from the burden of operational management, PedidosYa’s data team can now focus on adding value through data tools and products.
- Our data engineers are better equipped to integrate constantly changing transactional and operational data.
- The dataOps team can automate the infrastructure and provide autonomy to the end user.
- Our data quality team can focus on bringing added value to data stakeholders.
- Data scientists and data analytics can spend more time analyzing data and less time asking data gatekeepers for data access.
PedidosYa can now democratize data access with a well-governed architecture. We are still at the beginning of our journey, but we are closer to achieving our vision of building a data-driven organization. Up next: expanding our artificial intelligence and machine learning capabilities.
Tune in to Google Cloud’s Applied ML Summit on June 10th, 2021, or listen on-demand later, to learn how to apply groundbreaking machine learning technology in your projects.
Anthos for Manufacturing: Tackle DevOps Complexities and Drive Digital Transformation
DOWNLOAD E-BOOK6270
Of your peers have already downloaded this article
2:00 Minutes
The most insightful time you'll spend today!
5778
Of your peers have already watched this video.
4:50 Minutes
The most insightful time you'll spend today!
Google had Enough of Flooding in India. Here’s What It Did About It
Over the last century, floods have become the most common and deadly natural disaster on the planet.
Many countries currently lack effective early warning systems and alerts. Today, 20 percent of flood fatalities occur in India.
To help, Google sent a team to study the Ghaghara River, near Patna.
“In India, where we’re running our first pilot program, the government has thousands of people who measure water levels, every hour, in rivers across India, with what is effectively very long measuring sticks. They are called stream gauges. This allows them to know whether the river will overflow and flood. But it doesn’t yet allow them to understand exactly what areas are going to be affected, what neighborhoods, or even what villages,” says Sella Nevo, Tech Lead, Google Flood Forecasting.
“The big technical question was: Do we have enough information to try to do forecasting that would be accurate enough to make a difference?” says Yossi Matias, VP of Engineering and Crisis Response Lead, Google.
This is their incredible story. It includes collecting thousands of satellite images and generating hundreds of thousands of simulations of how a river could possibly behave.
3190
Of your peers have already watched this video.
27:30 Minutes
The most insightful time you'll spend today!
Generating Value with AI
Hear how enterprises leveraging the Google Cloud–across industries–are using AI to navigate uncertain times, and how they are innovating with AI to generate value moving forward.
In this video, you’ll uncover how to start using innovations from Google Cloud AI in your business today, and how customers deploy AI to transform their organizations.
Accuracy and Real-time Updates with Google Maps’ On-Demand Rides and Delivery Solution Impacts CX
5652
Of your peers have already read this article.
3:00 Minutes
The most insightful time you'll spend today!
Last year we launched our on-demand rides and delivery solution to help businesses improve operations as well as transform the driver and customer journey from booking to arrival or delivery. When it comes to on-demand rides and deliveries, every minute matters. When users book a ride or order food, they want a seamless experience and real-time accurate updates. Today, we’re taking a closer look into data quality improvements for location, time and distance accuracy, and motorbike routes.
Machine learning helps drive location accuracy
Location accuracy stands at the base of our customers’ operation. The location signals that are coming from mobile devices can sometimes be off for various reasons and a driver’s location can get stuck, or jump around.
Recently we developed mechanisms in our fleet management product that can take in multiple location signals and determine the most reliable location to use for a given vehicle. With that, we noticed drastic improvements:
- Eliminated long periods of location ‘stuckness’ almost completely; a vehicle is considered ‘stuck’ when we think it’s moving but the measured location is not
- Reduced the jumpiness of the location signal by 52%-86%: ‘jumpiness’ is when a vehicle shows a sudden and usually drastic change in location. A jump is determined to exist when the speed the vehicle had to go at in order to cover the distance it did is unrealistic
- Reduced the average jump distance by 44%-86% : ‘jump distance’ is the distance between two consecutive location pings when we determined a ‘location jump’ has occurred.
Dunzo, a local e-commerce platform in India, explains how integrating the order tracking capability within Google Maps Platform’s On Demand Rides and Delivery solution has helped reduce support calls by 90%. The out-of-the-box solution helped Dunzo’s motorbike delivery partners with updating location sync to reduce stuckness and jumpiness as well as deliver premium user experiences.
When the vehicle location is more reliable, the dispatch decision is of higher quality, meaning there is a higher chance you will be able to make the optimal decision. This can lead to less wait time for consumers, increased driver happiness and fewer cancellations.
Improvements in ETA accuracy in motorbike routes
In many geographic areas, road space is limited and car ownership is prohibitively expensive so motorbike is a prominent transportation mode. Motorbike mapping requires unique routing, ETA models, and navigation capabilities. Improvements in motorbike routing and ETA estimation have enabled customers like Gojek to offer better overall services, even in geographies with poor wifi or missing roads.
Recently, we further improved ETA outcomes by developing new machine learning models trained specifically for motorbikes. These models help our systems account for differences in congestion and traffic flow that arise in different regions and scenarios.
Globally, we measured an ~8% improvement in ETA accuracy for riders in the general public, and a ~6% improvement in on-demand rides and deliveries ETA accuracy. In this on-demand economy, where consumers are accustomed to real-time trip and order progress, these improvements will significantly improve their experience.
We will continue to innovate on both our car and motorbike on-demand rides and deliveries capabilities based on customer demand and requirements. We are committed to the success of our customers by building a seamless experience for all parties—consumers, drivers, and fleet operators—in the rides and deliveries journey.
For more information on Google Maps Platform, visit our website.
More Relevant Stories for Your Company
An Out-of-the-box, End-to-end Solution for Contact Centers!
Providing best-in-class customer service is crucial for the success of your business. Contact centers are a critical touch point, as they have to balance between representing your brand and prioritizing customer care. When your customers seek help and support, they expect efficient service that is accessible through modern voice and
Trend 2: Google Research on Machine Learning Themes for 2022 and Beyond!
Trend 2: Continued Efficiency Improvements for MLImprovements in efficiency — arising from advances in computer hardware design as well as ML algorithms and meta-learning research — are driving greater capabilities in ML models. Many aspects of the ML pipeline, from the hardware on which a model is trained and executed
Elevating Voice Solutions: Google Cloud Introduces Speech-to-Text v2 API and Chirp for Businesses
As one of the most innate and ubiquitous forms of expression, speech is a fundamental pillar of human interaction. It comes as no surprise, then, that Google Cloud’s Speech API has become a crucial tool for enterprise customers, launched to general availability (GA) over six years ago and, now, processing over 1
What Swiggy and You Can Learn From This Company’s Use of ML to Engage Customers
The app economy has enabled a huge range of unique business models to flourish. One such model is online food ordering and delivery services, in which apps leverage geo-location data to aggregate local food choices and offer personalized options to consumers. A leading company in this space is Just Eat.
