Customers today expect brands to know what they want. In fact, Forbes cited that 71% of customers feel frustrated when the shopping experience is impersonal. From interests to purchase habits, companies need to target customers effectively in order to stand out from the plethora of competitors in the online marketplace.

“Our mission is to help brands engage customers with the most appropriate marketing messages at the right place and at the right time while respecting consumer privacy and ensuring regulatory compliance. The experience of Google Cloud in working with large-scale datasets makes the platform an excellent fit for us.”

—Projjol Banerjea, co-founder and Chief Product Officer, Zeotap

Leveraging first-, second-, and third-party data, Zeotap’s Customer Intelligence Platform (CIP)—a CDP with additional identity resolution and third-party data enrichment capabilities—allows brands to engage with their known and unknown users with tailored marketing messages and next best offers or actions. It also offers out-of-box algorithms for standard marketing use cases such as promoting a specific product that will resonate with audiences and custom algorithms for very niche and specific use cases.

“Our mission is to help brands engage customers with the most appropriate marketing messages at the right place and at the right time while respecting consumer privacy and ensuring regulatory compliance. The experience of Google Cloud in working with large-scale datasets makes the platform an excellent fit for us,” says Projjol Banerjea, co-founder and Chief Product Officer at Zeotap.

“With BigQuery, we are able to operate more efficiently, removing 80% of the operation load that we would otherwise have to manage on a different cloud provider. As a result, we get to build and deliver products faster.”

—Sathish K S, VP Engineering, Zeotap

Improving data processing to deliver products faster

The cloud-native company migrated from its previous provider to Google Cloud in October 2019 because it was attracted to the benefits of BigQuery as a serverless cloud data warehouse. This was a feature that wasn’t available with its previous provider.

Sathish K S, Engineering Vice-President at Zeotap says, “With BigQuery, we are able to operate more efficiently, removing 80% of the operation load that we would otherwise have to manage on a different cloud provider. As a result, we get to build and deliver products faster.” Since coming onboard BigQuery, Zeotap has been able to increase the speed of its product delivery by 40%.

BigQuery also offers a much higher query limit, which is helpful because Zeotap can converge all of its previous operation load across different platforms into one data warehouse without worrying about it being overloaded.

One of the key services Zeotap offers is customer relationship management (CRM) platform integration. Using Dataflow, Zeotap can easily stream first-party data into BigQuery at scale.

The data integration capabilities of Cloud Data Fusion also complements Dataflow to help Zeotap enhance its API offerings. To illustrate, an ecommerce company may use Zeotap’s internal API to understand their customer’s cart and check-out habits and add Zeotap’s extra layer of data intelligence to it, so they can improve their marketing efforts. Cloud Data Fusion enables a seamless flow of real-time data so that the company can deliver the relevant ads or suggest products just before they make a purchase, which increases the likelihood of them buying more.

Zeotap uses Dataproc and Pub/Sub to manage the large flow of data that goes in and out of its database. Because many of its customers are pushing data in real time, Pub/Sub acts as a queuing engine that feeds them into Dataflow. Likewise, the data from Dataflow is also being channeled back smoothly to the customer endpoints using Pub/Sub.

Being able to automate all of its processes benefitted the DevOps team, as they now have time to focus on developing and improving Zeotap’s product offerings instead of spending time and energy on operations.

Managing a full migration seamlessly

Today, all of Zeotap’s data is hosted on Google Cloud. The company began its lift and shift migration in October 2019 and was complete by April 2020. “We have close to 405 data pipelines, and even after the migration, we had to verify the right datasets. We had terabytes of data to move, and despite this, our operations were not affected,” says Aditya Chandra, Vice-President of Infrastructure and Security at Zeotap.

Apart from the BigQuery capabilities, there were other major things Zeotap looked out for in its decision to move to a new cloud provider. The functional parity of Google Cloud was important so that it did not have to rebuild many of its existing structures. Zeotap also didn’t want any drop in performance during and after the move to a new cloud provider. Finally, as a multi-region company, it used to have to spot instance problems and run them on demand, which had cost implications. With the move, it no longer had to face such issues.

“Whatever Spark code that was already running on our existing system had to be lifted and shifted into Dataproc without any library changes because 60% of our codebase is out of there. We’re glad that this was possible,” says Ameya Agnihotri, Chief Technology Officer.

Managing business continuity with a trusted partner

Zeotap now works with Rackspace Technology (Rackspace) as it continues to optimize its infrastructure operations. As a marketing and technology company, Zeotap needs niche expertise to manage big data processing issues.

Ameya Agnihotri shares, “Given its global expertise in dealing with on-premises as well as cloud solutions, Rackspace can really add value because they are better equipped to troubleshoot and handle complex data operations.” Zeotap has plans to explore more of Google Cloud’s artificial intelligence (AI) and machine learning (ML) capabilities, so having a partner in this journey will be valuable to the team.

With so much data on hand, security is paramount to Zeotap. It does a lot of internal security assessments daily. This is one of the areas where the team works with Rackspace to ensure that it adopts best practices and stays on top of the latest updates. With its ability to handle big data operations, Rackspace also acts as an extension of the Zeotap team that supports the infrastructure side of the business.

“The level of engagement from Google Cloud has been excellent, particularly the ability to bring in expertise from different parts of the organization. The personal touch meant we could reach product owners and managers of specific products. This kind of support is what we really value and appreciate.”

—Ameya Agnihotri , Chief Technology Officer, Zeotap

Future plans and collaboration with Google Cloud

Zeotap is currently exploring Cloud Bigtable and API Gateway as it continues to grow the business and explore new types of services to offer to its customers. Although it has a software-as-a-service (SaaS) offering for customers to self-serve, some may prefer to do a server-to-server integration so that they do not have to worry about the back end at all. For this reason, Zeotap is exploring the possibility of integrating these customers’ APIs in the back end of its API, using Google Cloud’s API Gateway features that are available out of the box. The company is also looking forward to the new features being released on BigQuery, as well as exploring more of the alerting and monitoring capabilities of Google Cloud.

“The level of engagement from Google Cloud has been excellent, particularly the ability to bring in expertise from different parts of the organization,” says Ameya. He shares that the team has had many fortnightly calls when there were issues to address, and the Google Cloud team would help to fix the problems. “The personal touch meant we could reach product owners and managers of specific products. This kind of support is what we really value and appreciate.”

With big plans ahead for Zeotap that include becoming a Google Cloud Technology Partner in the near future, Swapnasarit Sahu, Chief Data and Analytics Officer, is confident that the company has chosen the right cloud solution. “We see Google as one of the frontrunners in the space of AI and ML solutions, which can really help data scientists in developing new and more innovative products and solutions, and we look forward to building great products with them,” he says.

Today at Google Cloud Next we are announcing innovations that will enable data teams to simplify how they work with data and derive value from it faster. These new solutions will help organizations build modern data architectures with real-time analytics to power innovative, mission-critical, data-driven applications. 

Too often, even the best minds in data are constrained by ineffective systems and technologies. A recent study showed that only 32% of companies surveyed gained value from their data investments. Previous approaches have resulted in difficult to access, slow, unreliable, complex, and fragmented systems. 

At Google Cloud, we are committed to changing this reality by helping customers simplify their approach to data to build their data clouds. Google Cloud’s data platform is simply unmatched for speed, scale, security, and reliability for any size organization with built-in, industry-leading machine learning (ML) and artificial intelligence (AI), and an open standards-based approach.

Vertex AI and data platform services unlock rapid ML modeling 

With the launch of Vertex AI in May 2021, we empowered data scientists and engineers to build reliable, standardized AI pipelines that take advantage of the power of Google Cloud’s data pipelines. Today, we are taking this a step further with the launch of Vertex AI Workbench, a unified user experience to build and deploy ML models faster, accelerating time-to-value for data scientists and their organizations. We’ve integrated data engineering capabilities directly into the data science environment, which lets you ingest and analyze data, and deploy and manage ML models, all from a single interface.

Data scientists can now build and train models 5X faster on Vertex AI than on traditional notebooks. This is primarily enabled by integrations across data services (like Dataproc, BigQuery, Dataplex, and Looker), which significantly reduce context switching. The unified experience of Vertex AI let’s data scientists coordinate, transform, secure and monitor Machine Learning Operations (MLOps) from within a single interface, for their long-running, self-improving, and safely-managed AI services.

“As per IDC’s AI StrategiesView 2021, model development duration, scalable deployment, and model management are three of the top five challenges in scaling AI initiatives,” said Ritu Jyoti, Group Vice President, AI and Automation Research Practice at IDC. “Vertex AI Workbench provides a collaborative development environment for the entire ML workflow – connecting data services such as BigQuery and Spark on Google Cloud, to Vertex AI and MLOps services. As such, data scientists and engineers will be able to deploy and manage more models, more easily and quickly, from within one interface.”

Ecommerce company, Wayfair, has transformed its merchandising capabilities with data and AI services. “At Wayfair, data is at the center of our business. With more than 22 million products from more than 16,000 suppliers, the process of helping customers find the exact right item for their needs across our vast ecosystem presents exciting challenges,” said Matt Ferrari, Head of Ad Tech, Customer Intelligence, and Machine Learning; Engineering and Product at Wayfair. “From managing our online catalog and inventory, to building a strong logistics network, to making it easier to share product data with suppliers, we rely on services including BigQuery to ensure that we are able to access high-performance, low-maintenance data at scale. Vertex AI Workbench and Vertex AI Training accelerate our adoption of highly scalable model development and training capabilities.”

BigQuery Omni: Breaking data silos with cross-cloud analytics and governance

Businesses across a variety of industries are choosing Google Cloud to develop their data cloud strategies and better predict business outcomes — BigQuery is a key part of that solution portfolio. To address complex data management across hybrid and multicloud environments, this month we are announcing the general availability of BigQuery Omni, which allows customers to analyze data across Google Cloud, AWS, and Azure. Healthcare provider, Johnson and Johnson was able to combine data in Google Cloud and AWS S3 with BigQuery Omni without needing data to migrate. 

This flexible, fully-managed, cross-cloud analytics solution allows you to cost-effectively and securely answer questions and share results from a single pane of glass across your datasets, wherever you are. In addition to these multicloud capabilities, Dataplex will be generally available this quarter to provide an intelligent data fabric that enables you to keep your data distributed while making it securely accessible to all your analytics tools.

Spark on Google Cloud simplifies data engineering 

To help make data engineering even easier, we are announcing the general availability of Spark on Google Cloud, the world’s first autoscaling and serverless Spark service for the Google Cloud data platform. This allows data engineers, data scientists, and data analysts to use Spark from their preferred interfaces without data replication or custom integrations. Using this capability, developers can write applications and pipelines that autoscale without any manual infrastructure provisioning or tuning. This new service makes Spark a first class citizen on Google Cloud, and enables customers to get started in seconds and scale infinitely, regardless if you start in BigQuery, Dataproc, Dataplex, or Vertex AI.

Spanner meets PostgreSQL: global, relational scale with a popular interface

We’re continuing to make Cloud Spanner, our fully managed, globally scalable, relational database, available to more customers now with a PostgreSQL interface, now in preview. With this new PostgreSQL interface, enterprises can take advantage of Spanner’s unmatched global scale, 99.999% availability, and strong consistency using skills and tools from the popular PostgreSQL ecosystem. 

This interface supports Spanner’s rich feature set that uses the most popular PostgreSQL data types and SQL features to reduce the barrier to entry for building transformational applications. Using the tools and skills they already have, developer teams gain flexibility and peace of mind because the schemas and queries they build against the PostgreSQL interface can be easily ported to another Postgres environment. Complete this form to request access to the preview.

Our commitment to the PostgreSQL ecosystem has been long standing. Customers choose Cloud SQL for the flexibility to run PostgreSQL, MySQL and SQL Server workloads. Cloud SQL provides a rich extension collection, configuration flags, and open ecosystem, without the hassle of database provisioning, storage capacity management, or other time-consuming tasks.

Auto Trader has migrated approximately 65% of their Oracle footprint to Cloud SQL, which remains a strategic priority for the company. Using Cloud SQL, BigQuery, and Looker to facilitate access to data for their users, and with Cloud SQL’s fully managed services, Auto Trader’s release cadence has improved by over 140% (year-over-year), enabling an impressive peak of 458 releases to production in a single day.

Looker integrations make augmented analytics a reality

We are announcing a new integration between Tableau and Looker that will allow customers to operationalize analytics and more effectively scale their deployments with trusted, real-time data, and less maintenance for developers and administrators. Tableau customers will soon be able to leverage Looker’s semantic model, enabling new levels of data governance while democratizing access to data. They will also be able to pair their enterprise semantic layer with Tableau’s leading analytics platform. The future might be uncertain, but together with our partners we can help you plan for it. 

We remain committed to developing new ways to help organizations go beyond traditional business intelligence with Looker. In addition to innovating within Looker, we’re continuing to integrate within other parts of Google Cloud. Today, we are sharing new ways to help customers deliver trusted data experiences and leverage augmented analytics to take intelligent action. 

First, we’re enabling you to democratize access to trusted data in tools where you are already familiar. Connected Sheets already allows you to interactively explore BigQuery data in a familiar spreadsheet interface and will soon be able to leverage the governed data and business metrics in Looker’s semantic model. It will be available in preview by the end of this year. 

Another integration we’re announcing is Looker’s Solution for Contact Center AI, which helps you gain a deeper understanding and appreciation of your customers’ full journey by unlocking insights from all of your company’s first-party data, such as contextualizing support calls to make sure your most valuable customers receive the best service. 

We’re also sharing the new Looker Block for Healthcare NLP API, which provides simplified access to intelligent insights from unstructured medical text. Compatible with Fast Healthcare Interoperability Resources (FHIR), healthcare providers, payers, and pharma companies can quickly understand the context and relationships of medical concepts within the text, and in turn, can begin to link this to other clinical data sources for additional AI and ML actions. 

Bringing the best of Google together with Google Earth Engine and Google Cloud

We are thrilled to announce the preview of Google Earth Engine on Google Cloud. This launch makes Google Earth Engine’s 50+ petabyte catalog of satellite imagery and geospatial data sets available for planetary-scale analysis. Google Cloud customers will be able to integrate Earth Engine with BigQuery, Google Cloud’s ML technologies, and Google Maps Platform. This gives data teams a way to better understand how the world is changing and what actions they can take — from sustainable sourcing, to saving energy and materials costs, to understanding business risks, to serving new customer needs. 

For over a decade, Earth Engine has supported the work of researchers and NGOs from around the world, and this new integration brings the best of Google and Google Cloud together to empower enterprises to create a sustainable future for our planet and for your business.

At Google Cloud, we are deeply grateful to work with companies of all sizes, and across industries, to build their data clouds. Join my keynote session to hear how organizations are leveraging the full power of data, from databases to analytics that support decision making to AI and ML that predict and automate the future. We’ll also highlight our latest product innovations for BigQuery, Spanner, Looker, and Vertex AI.

I can’t wait to hear how you will turn data into intelligence and look forward to connecting with you.

With access to more data than ever before, companies have never been better positioned to adopt precision marketing methods and target the right customers at the right time. Emarsys, a digital marketing platform, enables its clients to collect, analyze, and act on a wide variety of data. From websites to mobile apps to emails, Emarsys’ customers can handle data from all its digital channels on a single, easy-to-use platform. Emarsys also makes sure that customers receive the highest quality data possible, making for smarter decisions and better business practices.

“We were close to the limits of our internal data warehouse, scalability-wise. We didn’t want to get to the point where we’d have to delete data or cancel projects. In Google Cloud we saw a platform that could scale with our ambitions and be optimized for AI and real-time solutions.”

—Levente Otti, Head of Data, Emarsys

Since launching as an email solutions provider in 2000, Emarsys has grown into the world’s largest independent digital marketing platform, with more than 2,500 clients worldwide and reaching more than 1.4 billion people. By 2016, the company felt that its existing data warehouse platform was close to its limits, affecting not just day-to-day operations but also important strategic goals.

“We were close to the limits of our internal data warehouse, scalability-wise. We didn’t want to get to the point where we’d have to delete data or cancel projects,” says Levente Otti, Head of Data at Emarsys. “In Google Cloud, we saw a platform that could scale with our ambitions and be optimized for AI and real-time solutions.”

Minimal maintenance, unlimited scale with Google Cloud

Digital marketing is a highly competitive environment. Emarsys works alongside big players with a huge market share on the one hand and smaller, specialist companies on the other. It has thrived by successfully combining the all-inclusive offerings of the former with the agility of the latter, constantly looking for ways to innovate and improve. In recent years, the company had started to feel that the ability to handle large quantities of data was no longer enough. The next challenge was speed. “We truly believe that in the future, everything will be done in real time, including data processing, analytics, and AI predictive models,” Levente says.

At the start of 2016, Emarsys’ existing data warehouse was a software-as-a-service solution running on-premises, which required hardware and software maintenance in order to keep up with the company’s growing appetite for data-heavy use cases such as prediction and analytics. The existing platform had proven its worth processing large amounts of data in batches, but its real-time capabilities were limited. Moreover, Emarsys had begun to experiment with AI technology, but found that its data warehouse couldn’t scale to accommodate some of the more resource-intensive processes, such as training the predictive models. The company decided that it needed a new, cloud-based data platform.

After evaluating some of the leading cloud providers, Emarsys chose Google Cloud for its mature AI capabilities and its ease of use. “With the other solutions, we still had to rent virtual machines and hardware and be responsible for maintenance. At the time, Google Cloud was the only provider that could take that management overhead away from us, while keeping customers accounted for on every query level,” Levente says.

To implement its new data platform, Emarsys teamed up with Google Cloud Partner Aliz. Over a series of meetings, workshops, and architecture reviews, Aliz helped Emarsys navigate the Google Cloud ecosystem to find the right products for the solution it was looking for. “Aliz really helped us set off in the right direction,” explains Levente.

With Google BigQuery, we can run queries which process terabytes of data, in seconds. We can also develop our own user-defined functions, incorporating Bayesian statistics into our predictive algorithms. That means we can take into account historical data, resulting in much more accurate predictions in a scalable way within seconds.”

—Levente Otti, Head of Data, Emarsys

Emarsys’ new data platform would actually be two: one platform for batch processing data and one for real-time analysis and interactions. Firstly, a proprietary publishing component gathered all the data points from Emarsys’ various channels including the website, mobile, emails, and custom events. With Cloud Pub/Sub and Cloud Dataflow, Emarsys transported and processed the data into BigQuery, which allows for further work and reviews that take into account errors or delayed events. After this, the data was exported to the main batch processing platform, which ran on BigQuery. For the real-time analytics, Emarsys used Cloud Bigtable to access data and Cloud Dataflow to pipeline it into the real-time platform, which could communicate with AI components or interaction components via an API to deliver real-time interactions with customers.

On top of the overall data infrastructure, Emarsys built a new AI platform with Google Cloud components. Training the predictive models had been an issue in the past due to the large number of resources required, so Emarsys chose to use Google Kubernetes Engine clusters, which can scale up and down on demand, without the need for hardware configuration or management. The trained models were held securely in Cloud Storage. From here, they were integrated with BigQuery for power and flexibility, allowing Emarsys to improve not just the speed of its AI predictions but also the quality.

“With Google BigQuery, we can run queries which process terabytes of data, in seconds,” shares Levente. “We can also develop our own user-defined functions incorporating Bayesian statistics into our predictive algorithms. That means we can take into account historical data, resulting in much more accurate predictions in a scalable way within seconds.”

Real-time insight, long-term satisfaction

Google Cloud enabled Emarsys to build a scalable data and AI platform that delivers powerful, actionable insights in real time. According to Levente, the company wanted to spend less time managing overload and more time considering how it should handle data. An immediate result of the new platform has been that data is now available in a scalable way, without hardware additions and management.

“With Google Cloud, we’ve been able to build a truly real-time data platform. The norm used to be daily batch processing of data. Now, if an event happens, marketing actions can be executed within seconds, and customers can react immediately. That makes us very competitive in our market.”

—Levente Otti, Head of Data, Emarsys

The clear and innovative pricing schemes of Google Cloud have also brought a new level of accountability to Emarsys’ costs in a way that wasn’t possible with its on-premises infrastructure. “Now that we only pay for what we use, we can assign costs to specific customers or queries, which has a huge impact on our pricing and product development strategies,” says Levente.

Thanks to the power of BigQuery and the scale at which it can handle data, Emarsys can now apply its analytics and AI tools to their full potential. “It’s very important to enable our clients to create the best possible experience for customers,” says Levente. At the same time, the company has cut its AI platform costs by 70% with Kubernetes while increasing scalability compared to the previous solution. The whole data platform was built to be scalable, and its first big test came during the retail peak of Black Friday, when it comfortably handled 250,000 events per second. “Perhaps the biggest impact on the business came with the real-time nature of the new platform,” says Levente.

“With Google Cloud, we’ve been able to build a truly real-time data platform,” he explains. “The norm used to be daily batch processing of data. Now, if an event happens, marketing actions can be executed within seconds, and customers can react immediately. That makes us very competitive in our market.”

Since implementing the new platform, Emarsys has continued to innovate with it and is about to release a new Real-Time Decision Framework, which will provide customers with even more real-time products and tools. The company continues to work with Aliz and Google Cloud, exploring other products such as Google BigQuery ML and TensorFlow to improve its AI processes. “We had a problem that we wanted to tackle now, and for us, Google Cloud was the best way of doing that,” says Levente. “But it was also about looking ahead. We felt that Google Cloud offered us the best way of future-proofing our platform.”