Zeotap Uses Big Data to Enable Personalized, Precision Marketing at Scale
3356
Of your peers have already read this article.
5:30 Minutes
The most insightful time you'll spend today!
Customers today expect brands to know what they want. In fact, Forbes cited that 71% of customers feel frustrated when the shopping experience is impersonal. From interests to purchase habits, companies need to target customers effectively in order to stand out from the plethora of competitors in the online marketplace.
“Our mission is to help brands engage customers with the most appropriate marketing messages at the right place and at the right time while respecting consumer privacy and ensuring regulatory compliance. The experience of Google Cloud in working with large-scale datasets makes the platform an excellent fit for us.”
—Projjol Banerjea, co-founder and Chief Product Officer, Zeotap
Leveraging first-, second-, and third-party data, Zeotap’s Customer Intelligence Platform (CIP)—a CDP with additional identity resolution and third-party data enrichment capabilities—allows brands to engage with their known and unknown users with tailored marketing messages and next best offers or actions. It also offers out-of-box algorithms for standard marketing use cases such as promoting a specific product that will resonate with audiences and custom algorithms for very niche and specific use cases.
“Our mission is to help brands engage customers with the most appropriate marketing messages at the right place and at the right time while respecting consumer privacy and ensuring regulatory compliance. The experience of Google Cloud in working with large-scale datasets makes the platform an excellent fit for us,” says Projjol Banerjea, co-founder and Chief Product Officer at Zeotap.
“With BigQuery, we are able to operate more efficiently, removing 80% of the operation load that we would otherwise have to manage on a different cloud provider. As a result, we get to build and deliver products faster.”
—Sathish K S, VP Engineering, Zeotap
Improving data processing to deliver products faster
The cloud-native company migrated from its previous provider to Google Cloud in October 2019 because it was attracted to the benefits of BigQuery as a serverless cloud data warehouse. This was a feature that wasn’t available with its previous provider.
Sathish K S, Engineering Vice-President at Zeotap says, “With BigQuery, we are able to operate more efficiently, removing 80% of the operation load that we would otherwise have to manage on a different cloud provider. As a result, we get to build and deliver products faster.” Since coming onboard BigQuery, Zeotap has been able to increase the speed of its product delivery by 40%.
BigQuery also offers a much higher query limit, which is helpful because Zeotap can converge all of its previous operation load across different platforms into one data warehouse without worrying about it being overloaded.
One of the key services Zeotap offers is customer relationship management (CRM) platform integration. Using Dataflow, Zeotap can easily stream first-party data into BigQuery at scale.
The data integration capabilities of Cloud Data Fusion also complements Dataflow to help Zeotap enhance its API offerings. To illustrate, an ecommerce company may use Zeotap’s internal API to understand their customer’s cart and check-out habits and add Zeotap’s extra layer of data intelligence to it, so they can improve their marketing efforts. Cloud Data Fusion enables a seamless flow of real-time data so that the company can deliver the relevant ads or suggest products just before they make a purchase, which increases the likelihood of them buying more.
Zeotap uses Dataproc and Pub/Sub to manage the large flow of data that goes in and out of its database. Because many of its customers are pushing data in real time, Pub/Sub acts as a queuing engine that feeds them into Dataflow. Likewise, the data from Dataflow is also being channeled back smoothly to the customer endpoints using Pub/Sub.
Being able to automate all of its processes benefitted the DevOps team, as they now have time to focus on developing and improving Zeotap’s product offerings instead of spending time and energy on operations.
Managing a full migration seamlessly
Today, all of Zeotap’s data is hosted on Google Cloud. The company began its lift and shift migration in October 2019 and was complete by April 2020. “We have close to 405 data pipelines, and even after the migration, we had to verify the right datasets. We had terabytes of data to move, and despite this, our operations were not affected,” says Aditya Chandra, Vice-President of Infrastructure and Security at Zeotap.
Apart from the BigQuery capabilities, there were other major things Zeotap looked out for in its decision to move to a new cloud provider. The functional parity of Google Cloud was important so that it did not have to rebuild many of its existing structures. Zeotap also didn’t want any drop in performance during and after the move to a new cloud provider. Finally, as a multi-region company, it used to have to spot instance problems and run them on demand, which had cost implications. With the move, it no longer had to face such issues.
“Whatever Spark code that was already running on our existing system had to be lifted and shifted into Dataproc without any library changes because 60% of our codebase is out of there. We’re glad that this was possible,” says Ameya Agnihotri, Chief Technology Officer.
Managing business continuity with a trusted partner
Zeotap now works with Rackspace Technology (Rackspace) as it continues to optimize its infrastructure operations. As a marketing and technology company, Zeotap needs niche expertise to manage big data processing issues.
Ameya Agnihotri shares, “Given its global expertise in dealing with on-premises as well as cloud solutions, Rackspace can really add value because they are better equipped to troubleshoot and handle complex data operations.” Zeotap has plans to explore more of Google Cloud’s artificial intelligence (AI) and machine learning (ML) capabilities, so having a partner in this journey will be valuable to the team.
With so much data on hand, security is paramount to Zeotap. It does a lot of internal security assessments daily. This is one of the areas where the team works with Rackspace to ensure that it adopts best practices and stays on top of the latest updates. With its ability to handle big data operations, Rackspace also acts as an extension of the Zeotap team that supports the infrastructure side of the business.
“The level of engagement from Google Cloud has been excellent, particularly the ability to bring in expertise from different parts of the organization. The personal touch meant we could reach product owners and managers of specific products. This kind of support is what we really value and appreciate.”
—Ameya Agnihotri , Chief Technology Officer, Zeotap
Future plans and collaboration with Google Cloud
Zeotap is currently exploring Cloud Bigtable and API Gateway as it continues to grow the business and explore new types of services to offer to its customers. Although it has a software-as-a-service (SaaS) offering for customers to self-serve, some may prefer to do a server-to-server integration so that they do not have to worry about the back end at all. For this reason, Zeotap is exploring the possibility of integrating these customers’ APIs in the back end of its API, using Google Cloud’s API Gateway features that are available out of the box. The company is also looking forward to the new features being released on BigQuery, as well as exploring more of the alerting and monitoring capabilities of Google Cloud.
“The level of engagement from Google Cloud has been excellent, particularly the ability to bring in expertise from different parts of the organization,” says Ameya. He shares that the team has had many fortnightly calls when there were issues to address, and the Google Cloud team would help to fix the problems. “The personal touch meant we could reach product owners and managers of specific products. This kind of support is what we really value and appreciate.”
With big plans ahead for Zeotap that include becoming a Google Cloud Technology Partner in the near future, Swapnasarit Sahu, Chief Data and Analytics Officer, is confident that the company has chosen the right cloud solution. “We see Google as one of the frontrunners in the space of AI and ML solutions, which can really help data scientists in developing new and more innovative products and solutions, and we look forward to building great products with them,” he says.
Customer Voices: How Firms from Across Industries Leverage Google Cloud
DOWNLOAD CASE STUDY14112
Of your peers have already downloaded this article
1:30 Minutes
The most insightful time you'll spend today!
From powering everyday operations and accelerating application innovation, to providing tools for specific business needs and executing on big ideas, to advancing the security of technology solutions, companies from across industries have leveraged Google Cloud for business benefits.
Companies from across industries have turned to Google Cloud for transforming their business, modernizing their infrastructure, and gleaning intelligence from data. For instance:
- Johnson & Johnson achieved a 41% increase in search results from high-quality job applicants, significantly improving the company’s ability to quickly hire top talent.
- Sony Network Communications now processes 10 billion monthly queries faster, which advances data analysis.
- University College Dublin saw significant 6-figure savings by eliminating legacy hardware, software, and maintenance.
And there are many such examples. Read the collection of case studies to find out how companies from across industries and geographies leveraged Google Cloud for measurable business benefits and for solving complex problems.
3424
Of your peers have already watched this video.
19:30 Minutes
The most insightful time you'll spend today!
Data vs. COVID-19: How public data is helping flatten the curve
The outbreak of COVID-19 has changed the reality of millions of lives around the world. Communities around the world began social distancing, events were cancelled, and healthcare efforts redirected to fight the pandemic.
Hear how public data and the Google Cloud COVID-19 Public Datasets Program is helping combat the pandemic and informing individual decision-making to help everyone make informed decisions about the spread and risks of the virus, and how cooperative efforts could help flatten the curve.
In this presentation, Javier de la Torre, Founder and Chief Strategy Officer of Carto, and Shane Glass, Developer Advocate, Google Cloud demonstrate how the Carto platform is contributing to spatial analysis and understanding for how the outbreak evolved over time as government policies and social distancing measures were put into place.
They also show how to access these public datasets, what is included in the COVID-19 Public Datasets Program, and how you can use these datasets to develop maps and dashboards that will help us continue to combat the pandemic.
3140
Of your peers have already watched this video.
24:30 Minutes
The most insightful time you'll spend today!
How to Build a Global Marketing Data Hub
Data-driven marketing ensures that the right message is reaching the right customer at the right time through the right channel. However if an organization’s data resides in multiple systems and platforms it hampers its ability to deliver truly global data-driven campaigns, attribution analysis, and reporting.
See how Cloud Data Fusion with its growing plugins ecosystem can be leveraged to integrate multiple disparate data sources to build a comprehensive Marketing Data Warehouse. Learn how Cloud Data Fusion helped global CPG organizations build their Marketing DW.
Prateek Duble, Data Analytics Specialist, Google Cloud and Bhooshan Mogal, Product Manager, Cloud Data Fusion, Google Cloud, cover:
- How enterprises can build a robust global marketing data warehouse with Google Cloud including the business needs and technical challenge they encounter while building the holistic marketing data warehouses.
- How Cloud Data Fusion, Google Cloud Platform’s data ingestion and integration product, delivers critical capabilities to accelerate an enterprises’ journey to building marketing data warehouse.
- A reference architecture with Google Cloud products that shows how easily and quickly you can build marketing data warehouse in Google Cloud.
- A recent case study of a CPG business who built their marketing data warehouse with Google Cloud Platform.
This presentation also includes recent product launch announcements.
4329
Of your peers have already watched this video.
20:00 Minutes
The most insightful time you'll spend today!
The New and Upcoming Infrastructure for Google Cloud’s AI and ML Solutions
How does Google manage to provide its customers a differentiated compute platform experience and define ways to fully leverage its infrastructure supporting its cutting-edge AI and ML offerings? Easy-to-use, scalable and ability to create innovative products and services to end-users at low cost of ownership is the narrative behind Google Cloud’s AI and ML solutions. Explore Google Cloud’s ML infrastructure and accelerator innovation for 2021.
Watch the video to find out how Google Cloud’s leadership in AI through Google research, Deep Mind and also practical application of AI within Google Products drive innovative platforms and offerings that cater to customers’ AI and ML use cases!
How Eventrac and Workflows Integration Helps Implement Hybrid Architecture in Google Cloud
6543
Of your peers have already read this article.
3:00 Minutes
The most insightful time you'll spend today!
I previously talked about Eventarc for choreographed (event-driven) Cloud Run services and introduced Workflows for orchestrated services.
Eventarc and Workflows are very useful in strictly choreographed or orchestrated architectures. However, you sometimes need a hybrid architecture that combines choreography and orchestration.
For example, imagine a use case where a message to a Pub/Sub topic triggers an automated infrastructure workflow or where a file upload to a Cloud Storage bucket triggers an image processing workflow. In these use cases, the trigger is an event but the actual work is done as an orchestrated workflow.
How do you implement these hybrid architectures in Google Cloud? The answer lies in Eventarc and Workflows integration.
Eventarc triggers
To recap, an Eventarc trigger enables you to read events from Google Cloud sources via Audit Logs and custom sources via Pub/Sub and direct them to Cloud Run services:
One limitation of Eventarc is that it currently only supports Cloud Run as targets. This will change in the future with more supported event targets. It’d be nice to have a future Eventarc trigger to route events from different sources to Workflows directly.
In absence of such a Workflows enabled trigger today, you need to do a little bit of work to connect Eventarc to Workflows. Specifically, you need to use a Cloud Run service as a proxy in the middle to execute the workflow.
Let’s take a look at a couple of concrete examples.
Eventarc Pub/Sub + Workflows integration
In the first example, imagine you want a Pub/Sub message to trigger a workflow.
Define and deploy a workflow
First, define a workflow that you want to execute. Here’s a sample workflows.yaml that simply decodes and logs the Pub/Sub message body:
main:params: [args]steps:- init:assign:- headers: ${args.headers}- body: ${args.body}...- pubSubMessageStep:call: sys.logargs:text: ${"Decoded Pub/Sub message data is " + text.decode(base64.decode(args.body.message.data))}severity: INFODeploy the workflow with a single command:gcloud workflows deploy ${WORKFLOW_NAME} --source=workflow.yaml --location=${REGION}
Deploy a Cloud Run service to execute the workflow
Next, you need a Cloud Run service to execute this workflow. Workflows has an execution API and client libraries that you can use for your favorite language. Here’s an example of the execution code from a Node app.js file. It simply passes the received HTTP request headers and body to the workflow and executes it:
const execResponse = await client.createExecution({parent: client.workflowPath(GOOGLE_CLOUD_PROJECT, WORKFLOW_REGION, WORKFLOW_NAME),execution: {argument: JSON.stringify({headers: req.headers, body: req.body})}});
Deploy the Cloud Run service with the Workflows name and region passed as environment variables:
gcloud run deploy ${SERVICE_NAME} \--image gcr.io/${PROJECT_ID}/${SERVICE_NAME} \--region=${REGION} \--allow-unauthenticated \--update-env-vars GOOGLE_CLOUD_PROJECT=${PROJECT_ID},WORKFLOW_REGION=${REGION},WORKFLOW_NAME=${WORKFLOW_NAME}
Connect a Pub/Sub topic to the Cloud Run service
With Cloud Run and Workflows connected, the next step is to connect a Pub/Sub topic to the Cloud Run service by creating an Eventarc Pub/Sub trigger:
gcloud eventarc triggers create ${SERVICE_NAME} \--destination-run-service=${SERVICE_NAME} \--destination-run-region=${REGION} \--location=${REGION} \--event-filters="type=google.cloud.pubsub.topic.v1.messagePublished"
This creates a Pub/Sub topic under the covers that you can access with:
export TOPIC_ID=$(basename $(gcloud eventarc triggers describe ${SERVICE_NAME} --format='value(transport.pubsub.topic)'))
Trigger the workflow
Now that all the wiring is done, you can trigger the workflow by simply sending a Pub/Sub message to the topic created by Eventarc:
gcloud pubsub topics publish ${TOPIC_ID} --message="Hello there"
In a few seconds, you should see the message in Workflows logs, confirming that the Pub/Sub message triggered the execution of the workflow:
Eventarc Audit Log-Storage + Workflows integration
In the second example, imagine you want a file creation event in a Cloud Storage bucket to trigger a workflow. The steps are similar to the Pub/Sub example with a few differences.
Define and deploy a workflow
As an example, you can use this workflow.yaml that logs the bucket and file names:
main:params: [args]steps:...- log:call: sys.logargs:text: ${"Workflows received event from bucket " + bucket + " for file " + file}severity: INFO
Deploy a Cloud Run service to execute the workflow
In the Cloud Run service, you read the CloudEvent from Eventarc and extract the bucket and file name in app.js using the CloudEvent SDK and the Google Event library:
const cloudEvent = HTTP.toEvent({ headers: req.headers, body: req.body });//"protoPayload" : {"resourceName":"projects/_/buckets/events-atamel-images-input/objects/atamel.jpg}";const logEntryData = toLogEntryData(cloudEvent.data);const tokens = logEntryData.protoPayload.resourceName.split('/');const bucket = tokens[3]
Executing the workflow is similar to the Pub/Sub example, except you don’t pass in the whole HTTP request but rather just the bucket and file name to the workflow:
const execResponse = await client.createExecution({parent: client.workflowPath(GOOGLE_CLOUD_PROJECT, WORKFLOW_REGION, WORKFLOW_NAME),execution: {argument: JSON.stringify({bucket: bucket, file: file})}});
Connect Cloud Storage events to the Cloud Run service
To connect Cloud Storage events to the Cloud Run service, create an Eventarc Audit Logs trigger with the service and method names for Cloud Storage:
gcloud eventarc triggers create ${SERVICE_NAME} \--destination-run-service=${SERVICE_NAME} \--destination-run-region=${REGION} \--location=${REGION} \--event-filters="type=google.cloud.audit.log.v1.written" \--event-filters="serviceName=storage.googleapis.com" \--event-filters="methodName=storage.objects.create" \--service-account=${PROJECT_NUMBER}-compute@developer.gserviceaccount.com
Trigger the workflow
Finally, you can trigger the workflow by creating and uploading a file to the bucket:
echo "Hello World" > random.txtgsutil cp random.txt gs://${BUCKET}/random.txt
In a few seconds, you should see the workflow log the bucket and object name.
Conclusion
In this blog post, I showed you how to trigger a workflow with two different event types from Eventarc. It’s certainly possible to do the opposite, namely, trigger a Cloud Run service via Eventarc with a Pub/Sub message (see connector_publish_pubsub.workflows.yaml) from Workflows or a file upload to a bucket from Workflows.
All the code mentioned in this blog post is in eventarc-workflows-integration. Feel free to reach out to me on Twitter @meteatamel for any questions or feedback.
More Relevant Stories for Your Company
The 40 Most Astounding Stories of Tech-Enabled Innovation
These business and technology innovators are leveraging cloud computing, machine learning, APIs, collaborative platforms, among others to advance healthcare, financial services, retail, media and entertainment, manufacturing, and government. They include some of the biggest brands in the world, as well as some you've probably never heard of. But they all
“It’s an Astonishing Difference”: What Data Operation Execs say About Google Cloud’s Data Warehouse
The popularity of meal kit delivery services has surged in recent years as consumer attitudes toward home cooking and grocery shopping have shifted. As a pioneer in the category, Blue Apron helps its customers create incredible home cooking experiences by sending culinary-driven recipes with high-quality ingredients and step-by-step instructions straight to customers’
Google Cloud Celebrates Journey of 3 Inspiring Founders for the Asian Pacific American Heritage Month
May is Asian Pacific American Heritage Month —a time for us to come together to celebrate and remember the important people and history of Asian and Pacific Island heritage. This feature highlights three AAPI founders from the Google For Startups community. Read on to learn how these three founders built
Lead Your Company Towards Data-Powered Innovation
In this talk, Lak Lakshmanan, Head of Analytics and AI Solutions, Google Cloud, presents a prescriptive, proven approach to foster the ability of your organization to innovate with data. This approach has helped other Google Cloud customers handle key challenges around data size, speed, and a (lack of) structure. This
