Advertising agencies are faced with the challenge of providing the precision data that marketers require to make better decisions at a time when customers’ digital footprints are rapidly changing. They need to transform customer information and real-time data into actionable insights to inform clients what to execute to ensure the highest campaign performance.

In this post, we’ll explore how two of our advertising agency customers are turning to Google BigQuery to innovate, succeed, and meet the next generation of digital advertising head on. 

Net Conversion eliminated legacy toil to reach new heights

Paid marketing and comprehensive analytics agency Net Conversion has made a name for itself with its relentless attitude and data-driven mindset. But like many agencies, Net Conversion felt limited by traditional data management and reporting practices. 

A few years ago, Net Conversion was still using legacy data servers to mine and process data across the organization, and analysts relied heavily on Microsoft Excel spreadsheets to generate reports. The process was lengthy, fragmented, and slow—especially when spreadsheets exceeded the million-row limit.

To transform, Net Conversion built Conversionomics, a serverless platform that leverages BigQuery, Google Cloud’s enterprise data warehouse, to centralize all of its data and handle all of its data transformation and ETL processes. BigQuery was selected for its serverless architecture, high scalability, and integration with tools that analysts were already using daily, such as Google Ads, Google Analytics, and Data Hub. After moving to BigQuery, Net Conversion discovered surprising benefits that streamlined reporting processes beyond initial expectations. For instance, many analysts had started using Google Sheets for reports, and BigQuery’s native integration with Connected Sheets gave them the power to analyze billions of rows of data and generate visualizations right where they were already working.If you’re still sending Excel files that are larger than 1MB, you should explore Google Cloud.Kenneth Eisinger
Manager of Paid Media Analytics at Net Conversion

Since modernizing their data analytics stack, Net Conversion has saved countless hours of time that can now be spent on taking insights to the next level. Plus, BigQuery’s advanced data analytics capabilities and robust integrations have opened up new roads to offer more dynamic insights that help clients better understand their audience.   

For instance, Net Conversion recently helped a large grocery retailer launch a more targeted campaign that significantly increased downloads of their mobile application. The agency was able to better understand and predict their customers’ needs by analyzing buyer behavior across the website, mobile application, and their purchase history. Net Conversion analyzed website data in real-time with BigQuery, ran analytics on their mobile app data through the Firebase’s integration with BigQuery, and enriched these insights with sales information from the grocery retailer’s CRM to generate propensity behavior models that  accurately predicted which customers would most likely install their mobile app.

WITHIN helped companies weather the COVID storm

WITHIN is a performance branding company, focused on helping brands maximize growth by fusing marketing and business goals together in a single funnel. During the COVID-19 health crisis, WITHIN became an innovator in the ad agency world by sharing real-time trends and insights with customers through its Marketing Pulse Dashboard. This dashboard was part of the company’s path to adopting BigQuery for data analytics transformation. 

Prior to using BigQuery, WITHIN used a PostgreSQL database to house its data and manual reporting. Not only was the team responsible for managing and maintaining the server, which took focus away from the data analytics, but query latency issues often slowed them down. 

BigQuery’s serverless architecture, blazing-fast compute, and rich ecosystem of integrations with other Google Cloud and partner solutions made it possible to rapidly query, automate reporting, and completely get rid of CSV files. 

Using BigQuery, WITHIN is able to run Customer Lifetime Value (LTV) analytics and quickly share the insights with their clients in a collaborative Google Sheet. In order to improve the effectiveness of their campaigns across their marketing channels, WITHIN further segments the data into high and low LTV cohorts and shares the predictive insights with their clients for in-platform optimizations.

By distilling these types of LTV insights from BigQuery, WITHIN has been able to use those to empower their campaigns on Google Ads with a few notable success stories.

• WITHIN worked with a pet food company to analyze historical transactional data to model predicted LTV of new customers. They found significant differences between product category and autoship vs single order customers, and they implemented LTV-based optimization. As a result, they saw a 400% increase in average customer LTV. 
• WITHIN helped a coffee brand increase their customer base by 560%, with the projected 12-month LTV of newly acquired customers jumping a staggering 1280%.

Through integration with Google AI Platform Notebooks, BigQuery also advanced WITHIN’s ability to use machine learning (ML) models. Today, the team can build and deploy models to predict dedicated campaign impact across channels without moving the data.  The integration of clients’ LTV data through Google Ads has also impacted how WITHIN structures their clients’ accounts and how they make performance optimization decisions.

Now, WITHIN can capitalize on the entire data lifecycle: ingesting data from multiple sources into BigQuery, running data analytics, and empowering people with data by automatically visualizing data right in Google Data Studio or Google Sheets.A year ago, we delivered client reporting once a week. Now, it’s daily. Customers can view real-time campaign performance in Data Studio — all they have to do is refresh.Evan Vaughan
Head of Data Science at WITHIN

Having a consistent nomenclature and being able to stitch together a unified code name has allowed WITHIN to scale their analytics. Today, WITHIN is able to create an internal Media Mix Modeling (MMM) tool with the help of Google Cloud that they’re trialing with their clients.

The overall unseen benefit of BigQuery was that it put WITHIN in a position to remain nimble and spot trends before other agencies when COVID-19 hit. This aggregated view of data allowed WITHIN to provide unique insights to serve their customers better and advise them on rapidly evolving conditions.

Ready to modernize your data analytics? Learn more about how Google BigQuery unlocks the insights hidden in your data.

Generative AI is a rapidly expanding technology with a wide range of potential applications. Google Cloud Learning is thrilled to offer a new, no-cost Generative AI Fundamentals skill badge. This skill badge is designed for anyone eager to learn about the power of generative AI. No technical skills or prior knowledge required!

For those new to digital credentials, Google Cloud skill badges are digital credentials issued by Google Cloud in recognition of your knowledge of Google Cloud products and services. Individuals can earn skill badges on Google Cloud Skills Boost, and can share their skill badge to their social media profile and resume.

Watch the short videos in the generative AI courses and complete the final quiz to earn the Generative AI Fundamentals skill badge pictured below. In as little as 120 minutes, you will learn the basics of how generative AI works, how Google Cloud AI technology can be used by businesses and individuals, and how the principles of responsible AI lead to ethical decisions about the use of generative AI. 

By earning the skill badge, you will demonstrate your understanding of foundational concepts in generative AI.

The topics covered in the courses include:

1. Introduction to Generative AI 

• Explain how generative AI works
• Describe generative AI model types
• Describe generative AI applications

2. Introduction to Large Language Models

• Define large language models (LLMs)
• Describe LLM use cases
• Explain prompt tuning
• Describe Google’s generative AI development tools

3. Introduction to Responsible AI

• Identify the need for a responsible AI practice within an organization
• Recognize that decisions made at all stages of a project make an impact in Responsible AI
• Recognize that organizations can design an AI infrastructure to fit their own business needs and values

Earn the skill badge and show off your generative AI knowledge today! And for more content to help you stay up to date with generative AI, check out “The Prompt” and our generative AI primer for executives on Transform with Google Cloud.

Business Problem
UKG Ready primarily operates in the Small and Medium Business (SMB) space, so inherently many customers are forced to operate and make key business decisions with less Workforce Management (WFM) / Human Capital Management (HCM) data. In addition to volume, SMB lacks the variety of data needed to create a dynamic and agile organization. This puts SMB at a major disadvantage compared to larger segments.

Project Goals
People Insights module is committed to surfacing insights to customers in the context of their day-to-day duties and aid in decision making. With the SMB customer data limitations mentioned above, the goal of this project was to create a global dataset that augments individual customer data to bring light to less obvious, yet important information.

Challenges
UKG Ready is a highly configurable application that gives customers the opportunity to build solutions on a platform that meets their specific business needs. High configurability gives high flexibility to customers in their usage of the software. However, it becomes nearly impossible to create a global dataset for machine learning and data insights. UKG Ready manages just under 4 million of the US workforce and some 30,000+ customers. Despite the large employee dataset size, machine learning models that are specific to customers are starved for data because the individual customers have a relatively small employee population. Does that mean we cannot support our SMB customers’ decision making with ML?

Result
Partnering with Google, we were able to develop an approach that allowed us to standardize various domain entities (pay categories, time off codes, job titles, etc.) so that we could build a global dataset to augment SMB customer data. Using machine learning we were able to build a common vocabulary across our customer base. This common vocabulary encapsulates the nuances of how our customers manage their business and yet is generalized and standardized such that the data can be aggregated over the variety of customer configurations. This allows us to serve up practical insights to customers through various use cases. Our partnership allowed us to leverage Google Cloud Services to meet the needs of our complex machine learning models, distributed data sets and CI/CD processes.

How
UKG Ready decided to partner with Google for an end-to-end solution for the analytics offering. This allowed us to focus on our core business logic without having to worry about the platform, environment configurations, performance and scalability of the entire solution. We make use of various Google Cloud services such as Cloud Triggers, Cloud Storage, Cloud Functions, Cloud Composer, Cloud Dataflow, Big Query, Vertex AI, Cloud Pub/Sub… to host our analytics solution. Jenkins manages the entire CI/CD pipelines and cloud environments are configured and deployed using Terraform.

The standardization of business entities problem was solved in three distinct steps:

Step 1: Collecting aggregated data
We needed an approach to collect aggregated data from our highly distributed, sharded, multi-tenant data sources. We developed a custom solution that allows us to extract data aggregated at source for PII and GDPR considerations and transfer to Google Cloud Storage in the fastest manner possible. Data is then transformed and stored in Big Query. Services used: GCS, Cloud Functions, DataFlow, Cloud Composer and Big Query. All processes are orchestrated using Cloud Composer and detailed logging is available in Cloud Logging (Stackdriver).

Step 2: Applying NLP (Natural Language Processing)
Once we had the variety of customer configurations or the business entities available, we then applied NLP algorithms to categorize and standardize these in buckets. This approach assumes that customers use natural language for configurations like job titles, pay codes etc.

String Preparation
The input data for string preparation process is an entity string or several strings, that describe one entity object (like name-description pair or code-name pair). The output represents set of tokens that may be used to run a classification/clustering model. The process of string preparation tokenizes strings, replaces shortcuts, handles abbreviations, translates tokens, handles grammatical errors and mistypes

ML Models

Statistical
The idea of the model is to use defined target classes (clusters) and assign several tokens (anchors) to each of them an entity that has any of those tokens would be “attracted” to appropriate class. All other tokens are weighted according to frequencies of usage of theses tokens in the entities with anchor tokens:

Using anchor tokens, we are building kind-of Word2Vec - dimensionality of vector is equal to number of target classes. The higher the specific dimension (cluster) value, the higher the probability of entity to be included in appropriate cluster. Final prediction for entity tokens list for specific class is sum of weights of all the tokens included. Predicted cluster is a cluster that has maximal prediction score.

Lexical Model
We managed to generate reasonable amount of labeled data during statistical model implementation and testing. That opens a possibility to build “classical” NLP model that uses labeled data to train classification neural network using pretrained layers to produce token embeddings or even string embeddings. We started experimentation with pre-trained models like GloVe and got good results with single words and bi-grams but started getting issues in handling of n-grams. Our Google account team came to our rescue and recommended some white papers that helped formulate our strategy. We now use Tensorflow nnlm-en-dim128 model to produce string embeddings – it was trained on 200B records English Google News corpus and produces for each input string 128-dimensional vector. After that we use several Dense and Dropout layers to build a classification model.

Ensembling
To perform ensembling all the model results for each class are cast to probabilities using softmax transformation with scale normalization. Final predicted probability is maximal average score of both models among all the classes scores – appropriate class is predicted class.

The machine learning models are deployed on Vertex AI and are used in batch predictions. Model performance is captured at every prediction boundary and monitored for quality in production.

Step 3: Making available common vocabulary
Having the standardized vocabulary, we then needed a mechanism to have the results be available in UKG Ready reports and customer specific models like Flight Risk and Fatigue. For this we again used Google Services for orchestration, data transformation and data storage.

Once the modeling is complete, we made the customer specific models leveraging the above architecture be available in Reports. We utilized our proven existing technology choices in GCP for orchestration, data transformation and data storage

Results
We are able to build a common vocabulary of our customers’ business entities with good confidence. And be an expert advisor to our SMB customers in their decision-making using machine learning. With the advice of our Google account team and using Google services we can add value to our product in a relatively short amount of time. And we are not done! We continue to use this platform for new use cases, complex business problems and innovative machine learning solutions.

Sample result:

Special thanks to Kanchana Patlolla , AI Specialist, Google for the collaboration in bringing this to light

Practical machine learning (ML) is a trial and error process. ML practitioners compare different performance metrics by running ML experiments till you find the best model with a given set of parameters. Because of the experimental nature of ML, there are many reasons for tracking ML experiments and making them reproducible including debugging and compliance.

But tracking experiments is challenging: you need to organize experiments so that other team members can quickly understand, reproduce and compare them. That adds overhead that you don’t need.

We are happy to announce Vertex AI Experiments autologging, a solution which provides automated experiment tracking for your models, which streamlines your ML experimentation

With Vertex AI Experiments autologging, you can now log parameters, performance metrics and lineage artifacts by adding one line of code to your training script without needing to explicitly call any other logging methods.

How to use Vertex AI autologging

As a data scientist or ML practitioner, you conduct your experiment in a notebook environment such as Colab or Vertex AI Workbench. To enable Vertex AI Experiments autologging, you call aiplatform.autolog() in your Vertex AI Experiment session. After that call, any parameters, metrics and artifacts associated with model training are automatically logged and then accessible within the Vertex AI Experiment console. 

Here’s  how to enable autologging in your training session with a Scikit-learn model.

# Enable autologging
aiplatform.autolog()

# Build training pipeline
ml_pipeline = Pipeline(...)

# Train model
ml_pipeline.fit(x_train, y_train)

This video shows parameters and training/post-training metrics in the Vertex AI Experiment console.

Vertex AI SDK autologging uses MLFlow’s autologging in its implementation and it supports several frameworks including XGBoost, Keras and Pytorch Lighting. See documentation for all supported frameworks. 

Vertex AI Experiments autologging automatically logs model time series metrics when you train models along multiple epochs. That’s because of the integration between Vertex AI Experiments autologging and Vertex AI Tensorboard. 

Furthermore, you can adapt Vertex AI Experiments autologging to your needs. For example, let’s say your team has a specific experiment naming convention. By default, Vertex AI Experiments autologging automatically creates Experiment Runs for you without requiring you to call `aiplatform.start_run()` or `aiplatform.end_run()`. If you’d like to specify your own Experiment Run names for autologging, you can manually initialize a specific run within the experiment using aiplatform.start_run() and aiplatform.end_run() after autologging has been enabled. 

What’s next

You can access Vertex AI Experiments autologging with the latest version of Vertex AI SDK for Python. To learn more, check out these resources :

• Documentation: Autolog data to an experiment run
• Github:  Get started with Vertex AI Experiments autologging

While I’m thinking about the next blog post, let me know if there is Vertex AI content you’d like to see on Linkedin or Twitter.

Patients about to undergo a healthcare procedure understandably want the best medical professionals they can get. But how can they know which doctors have had the most experience and the best outcomes with that particular procedure? How can they make an informed decision about which doctor to select when the information they have is limited to the doctor’s practice area and subjective reviews from other patients?

MD Insider is working to solve that problem using machine learning (ML) to objectively analyze doctor performance. By analyzing data from thousands of institutions and millions of doctor-patient interactions and medical events, MD Insider identifies physician performance insights based on their experience and outcomes. Insights are then integrated into a triage engine, that enables consumers to search for and schedule appointments with providers who meet their clinical criteria and convenience preferences, such as insurances accepted, office hours, locations, and language.

MD Insider also offers robust data APIs to help health systems, health plans, and employers reduce costs and improve quality of care. Payers use the APIs to curate high-quality provider networks and manage provider directories. Examples of MD Insider’s data APIs include Provider Experience and Share of Practice metrics, Provider Quality and Outcomes, Network Modeling, Expert Clinical Search Taxonomy, Find a Provider, Acute-Care Hospital Quality, and Provider and Facility Metadata.

“We use Google Cloud Platform the way the cloud was supposed to be used. By comparison, other cloud providers feel like you’re renting somebody else’s data center.”

—Ed Holsinger, Lead Data Engineer and Head of Data Science, MD Insider

MD Insider is continuously ingesting the latest performance data about physicians and analyzing billions of rows of data. Requiring constant scalability, the company was born in the cloud; however, it had difficulty configuring server instances for the optimal balance of memory and CPU, and its Hadoop cluster had to be kept running 24/7. Network performance was often slow for no apparent reason. As a result, failure rates from node timeouts increased, and costs grew along with the data. MD Insider had to estimate its usage and pay up front, and received little financial benefit from sustained use commitments.

Knowing that data would continue to grow, MD Insider decided to move its data services — the most demanding and complex portion of its infrastructure — to Google Cloud Platform (GCP), and took advantage of GCP managed services for container management and big data analytics.

“One of the reasons we decided to move to Google Cloud Platform is because it feels like a unified, well-designed cloud architecture and pricing model,” says Ed Holsinger, Lead Data Engineer and Head of Data Science at MD Insider. “We use Google Cloud Platform the way the cloud was supposed to be used. By comparison, other cloud providers feel like you’re renting somebody else’s data center.”

“Moving to Google Cloud Platform and using Kubernetes Engine gave us 5x the processing power for analyzing physician performance at 75% less cost. Our data scientists have more power than ever before to generate insights for our customers.”

—Ed Holsinger, Lead Data Engineer and Head of Data Science, MD Insider

5x the performance, 75% less cost

MD Insider now uses Kubernetes Engine to automate container management and deploy clusters in minutes with just a few clicks. When hundreds of machines are required to analyze a large dataset, automation in Kubernetes Engine deploys ML models as containers, each of which manages the full lifecycle of its task, including scaling up resources, deploying results, and scaling back down when the task is finished. It’s easy for MD Insider to specify exactly how much CPU and memory each container needs, helping maximize performance while reducing costs.

“Moving to Google Cloud Platform and Kubernetes Engine gave us 5x the processing power for analyzing physician performance at 75% less cost,” says Ed. “Our data scientists have more power than ever before to generate insights for our customers. Data scoring jobs that used to take three business days now take four hours.”

Adds Galen Meurer, Senior Software Engineer at MD Insider: “Even if all Google Cloud Platform had to offer was Kubernetes Engine, I would still want to use it. Previously we spent up to 30% of our time managing our container infrastructure, which we can now use for product development.”

A foundation for data science

MD Insider was happy to find that GCP offers a wide variety of managed services. For example, the company is supplementing its Kubernetes Engine clusters with BigQuery for its big data masters and selection jobs, enabling scientists to analyze new and different types of data as well as analyze larger datasets in less time. MD Insider also uses Cloud Storage for big data staging and Cloud Dataproc to run managed Apache Spark clusters for data processing.

“Google Cloud Platform gives us an incredibly powerful cloud architecture for data engineering and data science,” says Eric Wilson, CEO of MD Insider. “That gives our scientists independence, they can do what they need to do without waiting and with no contention between them.”

A developer-friendly platform

Migrating its data services was such a success that MD Insider decided to move the rest of its infrastructure to GCP, including the front end for its web application. Since the migration, MD Insider has experienced no unplanned downtime on GCP, allowing it to easily meet the 99.5% uptime SLA it promises to customers. It’s also taking advantage of Build Triggers in Kubernetes Engine to automate container builds and reduce build times by more than 40%. Production code can be updated in seconds, with no impact to end users other than making new features available.

“GCP has simplified our workflow in so many ways, from intelligent load balancing to content delivery and automating builds,” says Matthew Frey, Software Engineer. “Everything on GCP is cohesive and developer friendly, with a superior UI and better network performance than other cloud providers.”

Ryan Beaini, Senior Software Engineer at MD Insider, agrees: “Since we moved to GCP, our developers are definitely happier. The pain and the headaches we experienced because of the limitations of our previous toolset all went away.”

“We’re a small company, but what we’re doing is incredibly important. We’re helping people make decisions about healthcare providers that could impact their lives and even be life-saving. Google Cloud Platform is helping us make our mission bigger, better, and brighter.”

—Eric Wilson, CEO, MD Insider

Securing billions of rows of clinical and non-clinical healthcare data

As a healthcare technology company, MD Insider processes billions of rows of clinical and non-clinical healthcare data. To control user access to GCP, it uses Identity & Access Management (IAM) along with Yubico YubiKeys for hardware-based two-factor authentication when logging into Google Workspace. MD Insider takes comfort that GCP encrypts data at rest by default, and encrypts and authenticates data in transit when data moves outside physical boundaries not controlled by Google or on behalf of Google.

“On GCP, everything that we need to be encrypted for compliance purposes is encrypted, which is fantastic,” says Eric. “When I tell our potential clients and partners about the resources that Google has dedicated to security, it gives them the confidence that their data will be protected.”

Transforming how teams work

As a growing company, MD Insider must collaborate seamlessly between offices in California, Colorado, and Illinois. It relies on Google Workspace for communication and productivity, using Docs, Sheets, and Slides to drive the business. Employee and team files are stored in Drive, and meetings are conducted via Google Meet with Chromebox for Meetings videoconferencing hardware kits. Google Workspace also helps MD Insider maintain information security by authenticating email domains with digital signatures in Gmail and scanning outgoing email using Gmail Data Loss Prevention (DLP).

“I use Google Workspace every day, and everyone else here does too,” says Eric. “Team Drives are a big time saver for us. We’ve let our previous office software expire, because there’s no need to pay for those licenses anymore.”

Promoting healthcare transparency

With GCP helping MD Insider increase velocity and momentum, the company is making exciting progress. For example, it has entered into a strategic partnership with Zelis Healthcare, which will use MD Insider’s API to provide insights for a next-gen analytics platform that will give health plans unprecedented transparency around physician performance.

“We’re a small company, but what we’re doing is incredibly important,” says Eric. “We’re helping people make decisions about healthcare providers that could impact their lives and even be life-saving. Google Cloud Platform is helping us make our mission bigger, better, and brighter.”

*Google Workspace was formerly known as G Suite prior to Oct. 6, 2020.