With access to more data than ever before, companies have never been better positioned to adopt precision marketing methods and target the right customers at the right time. Emarsys, a digital marketing platform, enables its clients to collect, analyze, and act on a wide variety of data. From websites to mobile apps to emails, Emarsys’ customers can handle data from all its digital channels on a single, easy-to-use platform. Emarsys also makes sure that customers receive the highest quality data possible, making for smarter decisions and better business practices.

“We were close to the limits of our internal data warehouse, scalability-wise. We didn’t want to get to the point where we’d have to delete data or cancel projects. In Google Cloud we saw a platform that could scale with our ambitions and be optimized for AI and real-time solutions.”

—Levente Otti, Head of Data, Emarsys

Since launching as an email solutions provider in 2000, Emarsys has grown into the world’s largest independent digital marketing platform, with more than 2,500 clients worldwide and reaching more than 1.4 billion people. By 2016, the company felt that its existing data warehouse platform was close to its limits, affecting not just day-to-day operations but also important strategic goals.

“We were close to the limits of our internal data warehouse, scalability-wise. We didn’t want to get to the point where we’d have to delete data or cancel projects,” says Levente Otti, Head of Data at Emarsys. “In Google Cloud, we saw a platform that could scale with our ambitions and be optimized for AI and real-time solutions.”

Minimal maintenance, unlimited scale with Google Cloud

Digital marketing is a highly competitive environment. Emarsys works alongside big players with a huge market share on the one hand and smaller, specialist companies on the other. It has thrived by successfully combining the all-inclusive offerings of the former with the agility of the latter, constantly looking for ways to innovate and improve. In recent years, the company had started to feel that the ability to handle large quantities of data was no longer enough. The next challenge was speed. “We truly believe that in the future, everything will be done in real time, including data processing, analytics, and AI predictive models,” Levente says.

At the start of 2016, Emarsys’ existing data warehouse was a software-as-a-service solution running on-premises, which required hardware and software maintenance in order to keep up with the company’s growing appetite for data-heavy use cases such as prediction and analytics. The existing platform had proven its worth processing large amounts of data in batches, but its real-time capabilities were limited. Moreover, Emarsys had begun to experiment with AI technology, but found that its data warehouse couldn’t scale to accommodate some of the more resource-intensive processes, such as training the predictive models. The company decided that it needed a new, cloud-based data platform.

After evaluating some of the leading cloud providers, Emarsys chose Google Cloud for its mature AI capabilities and its ease of use. “With the other solutions, we still had to rent virtual machines and hardware and be responsible for maintenance. At the time, Google Cloud was the only provider that could take that management overhead away from us, while keeping customers accounted for on every query level,” Levente says.

To implement its new data platform, Emarsys teamed up with Google Cloud Partner Aliz. Over a series of meetings, workshops, and architecture reviews, Aliz helped Emarsys navigate the Google Cloud ecosystem to find the right products for the solution it was looking for. “Aliz really helped us set off in the right direction,” explains Levente.

With Google BigQuery, we can run queries which process terabytes of data, in seconds. We can also develop our own user-defined functions, incorporating Bayesian statistics into our predictive algorithms. That means we can take into account historical data, resulting in much more accurate predictions in a scalable way within seconds.”

—Levente Otti, Head of Data, Emarsys

Emarsys’ new data platform would actually be two: one platform for batch processing data and one for real-time analysis and interactions. Firstly, a proprietary publishing component gathered all the data points from Emarsys’ various channels including the website, mobile, emails, and custom events. With Cloud Pub/Sub and Cloud Dataflow, Emarsys transported and processed the data into BigQuery, which allows for further work and reviews that take into account errors or delayed events. After this, the data was exported to the main batch processing platform, which ran on BigQuery. For the real-time analytics, Emarsys used Cloud Bigtable to access data and Cloud Dataflow to pipeline it into the real-time platform, which could communicate with AI components or interaction components via an API to deliver real-time interactions with customers.

On top of the overall data infrastructure, Emarsys built a new AI platform with Google Cloud components. Training the predictive models had been an issue in the past due to the large number of resources required, so Emarsys chose to use Google Kubernetes Engine clusters, which can scale up and down on demand, without the need for hardware configuration or management. The trained models were held securely in Cloud Storage. From here, they were integrated with BigQuery for power and flexibility, allowing Emarsys to improve not just the speed of its AI predictions but also the quality.

“With Google BigQuery, we can run queries which process terabytes of data, in seconds,” shares Levente. “We can also develop our own user-defined functions incorporating Bayesian statistics into our predictive algorithms. That means we can take into account historical data, resulting in much more accurate predictions in a scalable way within seconds.”

Real-time insight, long-term satisfaction

Google Cloud enabled Emarsys to build a scalable data and AI platform that delivers powerful, actionable insights in real time. According to Levente, the company wanted to spend less time managing overload and more time considering how it should handle data. An immediate result of the new platform has been that data is now available in a scalable way, without hardware additions and management.

“With Google Cloud, we’ve been able to build a truly real-time data platform. The norm used to be daily batch processing of data. Now, if an event happens, marketing actions can be executed within seconds, and customers can react immediately. That makes us very competitive in our market.”

—Levente Otti, Head of Data, Emarsys

The clear and innovative pricing schemes of Google Cloud have also brought a new level of accountability to Emarsys’ costs in a way that wasn’t possible with its on-premises infrastructure. “Now that we only pay for what we use, we can assign costs to specific customers or queries, which has a huge impact on our pricing and product development strategies,” says Levente.

Thanks to the power of BigQuery and the scale at which it can handle data, Emarsys can now apply its analytics and AI tools to their full potential. “It’s very important to enable our clients to create the best possible experience for customers,” says Levente. At the same time, the company has cut its AI platform costs by 70% with Kubernetes while increasing scalability compared to the previous solution. The whole data platform was built to be scalable, and its first big test came during the retail peak of Black Friday, when it comfortably handled 250,000 events per second. “Perhaps the biggest impact on the business came with the real-time nature of the new platform,” says Levente.

“With Google Cloud, we’ve been able to build a truly real-time data platform,” he explains. “The norm used to be daily batch processing of data. Now, if an event happens, marketing actions can be executed within seconds, and customers can react immediately. That makes us very competitive in our market.”

Since implementing the new platform, Emarsys has continued to innovate with it and is about to release a new Real-Time Decision Framework, which will provide customers with even more real-time products and tools. The company continues to work with Aliz and Google Cloud, exploring other products such as Google BigQuery ML and TensorFlow to improve its AI processes. “We had a problem that we wanted to tackle now, and for us, Google Cloud was the best way of doing that,” says Levente. “But it was also about looking ahead. We felt that Google Cloud offered us the best way of future-proofing our platform.”

At Google Cloud, we believe that the protection of our customers’ sensitive data is paramount, and encryption is a powerful mechanism to help achieve this goal. For years, we have supported encryption in transit when our customers ingest their data to bring it to the cloud. We’ve also long supported encryption at rest, for all customer content stored in Google Cloud.

To complete the full data protection lifecycle, we can protect customer data when it’s processed through our Confidential Computing portfolio. Confidential Computing products from Google Cloud protect data in use by performing computation in a hardware isolated environment that is encrypted with keys managed by the processor and unavailable to the operator. These isolated environments help prevent unauthorized access or modification of applications and data while in use, thereby increasing the security assurances for organizations that manage sensitive and regulated data in public cloud infrastructure.

Secure isolation has always been a critical component of our cloud infrastructure; with Confidential Computing, this isolation is cryptographically reinforced. Google Cloud’s Confidential Computing products leverage security components in AMD EPYC™ processors including AMD Secure Encrypted Virtualization (SEV) technology.

Building trust in Confidential Computing through industry collaboration

Part of our mission to bring Confidential Computing technology to more cloud workloads and services is to make sure that the hardware and software used to build these technologies is continuously reviewed and tested. We evaluate different attack vectors to help ensure Google Cloud Confidential Computing environments are protected against a broad range of attacks. As part of this evaluation, we recognize that the secure use of our services and the Internet ecosystem as a whole depends on interactions with applications, hardware, software, and services that Google doesn’t own or operate.

The Google Cloud Security team, Google Project Zero, and the AMD firmware and product security teams collaborated for several months to conduct a detailed review of the technology and firmware that powers AMD Confidential Computing technology. This review covered both Secure Encrypted Virtualization (SEV) capable CPUs, and the next generation of Secure Nested Paging (SEV-SNP) capable CPUs which protect confidential VMs against the hypervisor itself. The goal of this review was to work together and analyze the firmware and technologies AMD uses to help build Google Cloud’s Confidential Computing services to further build trust in these technologies.

This in-depth review focused on the implementation of the AMD secure processor in the third generation AMD EPYC processor family delivering SEV-SNP. SNP further improves the posture of confidential computing using technology that removes the hypervisor from the trust boundary of the guest, allowing customers to treat the Cloud Service Provider as another untrusted party. The review covered several AMD secure processor components and evaluated multiple different attack vectors. The collective group reviewed the design and source code implementation of SEV, wrote custom test code, and ran hardware security tests, attempting to identify any potential vulnerabilities that could affect this environment.

Working on this review, the security teams identified and confirmed potential issues of varying severity. AMD was diligent in fixing all applicable issues and now offers updated firmware through its OEM channels. Google Cloud’s AMD-based Confidential Computing solutions now include all the mitigations implemented during the security review.

“At Google, we believe that investing in security research outside of our own platforms is a critical step in keeping organizations across the broader ecosystem safe,” said Royal Hansen, vice president of Security Engineering at Google. “At the end of the day, we all benefit from a secure ecosystem that organizations rely on for their technology needs and that is why we’re incredibly appreciative of our strong collaboration with AMD on these efforts.”

“Together, AMD and Google Cloud are continuing to advance Confidential Computing, helping enterprises to move sensitive workloads to the cloud with high levels of privacy and security, without compromising performance,” said Mark Papermaster, AMD’s executive vice president and chief technology officer. ”Continuously investing in the security of these technologies through collaboration with the industry is critical to providing customer transformation through Confidential Computing. We’re thankful to have partnered with Google Cloud and the Google Security teams to advance our security technology and help shape future Confidential Computing innovations to come.”

Reviewing trusted execution environments for security is difficult given the closed-source firmware and proprietary hardware components. This is why research and collaborations such as this are critical to improve the security of foundational components that support the broader Internet ecosystem. AMD and Google believe that transparency helps provide further assurance to customers adopting Confidential Computing, and to that end AMD is working toward a model of open source security firmware.

With the analysis now complete and the vulnerabilities addressed, the AMD and Google security teams agree that the AMD firmware which enables Confidential Computing solutions meets an elevated security bar for customers, as the firmware design updates mitigate several bug classes and offer a way to recover from vulnerabilities. More importantly, the review also found that Confidential VMs are protected against a broad range of attacks described in the review.

Google Cloud’s Confidential Computing portfolio

The Google Cloud Confidential VMs, Dataproc Confidential Compute, and Confidential GKE Nodes have enabled high levels of security and privacy to address our customers’ data protection needs without compromising usability, performance, and scale. Our mission is to make this technology ubiquitous across the cloud. Confidential VMs run on hosts with AMD EPYC processors which feature AMD Secure Encrypted Virtualization (SEV). Incorporating SEV into Confidential VMs provide benefits and features including:

Isolation: Memory encryption keys are generated by the AMD Secure Processor during VM creation and reside solely within the AMD Secure Processor. Other VM encryption keys such as for disk encryption can be generated and managed by an external key manager or in Google Cloud HSM. Both sets of these keys are not accessible by Google Cloud, offering strong isolation.

Attestation: Confidential VMs use Virtual Trusted Platform Module (vTPM) attestation. Every time a Confidential VM boots, a launch attestation report event is generated and posted to customer cloud logging, which gives administrators the opportunity to act as necessary.

Performance: Confidential Computing offers high performance for demanding computational tasks. Enabling Confidential VM has little or no impact on most workloads.

The future of Confidential Computing and secure platforms

While there are no absolutes in computer security, collaborative research efforts help uncover security vulnerabilities that can emerge in complex environments and help to prevent Confidential Computing solutions from threats today and into the future. Ultimately, this helps us increase levels of trust for customers.

We believe Confidential Computing is an industry-wide effort that is critical for securing sensitive workloads in the cloud and are grateful to AMD for their continued collaboration on this journey.

To read the full security review, visit this page.

Acknowledgments 

We thank the many Google security team members who contributed to this ongoing security collaboration and review, including James Forshaw, Jann Horn and Mark Brand.

We are grateful for the open collaboration with AMD engineers, and wish to thank David Kaplan, Richard Relph and Nathan Nadarajah for their commitment to product security. We would also like to thank AMD leadership: Ab Nacef, Prabhu Jayanna, Hugo Romero, Andrej Zdravkovic and Mark Papermaster for their support of this joint effort.

Commercial trucks often travel partially empty because many shippers don’t have enough cargo to fill an entire container or trailer. Although offering available space to other shippers helps minimize carbon emissions and reduce operating costs, most trucking companies can’t efficiently schedule, track, or deliver multiple freight loads.

Companies have always struggled to ship over-the-road freight efficiently. However,recent economic events have created an unprecedented logistics and transportation crisis that continues to disrupt supply chains, delay deliveries, and significantly raise the price of basic goods. Since some stores can’t keep their shelves fully stocked, many people across the country are finding it more difficult than ever to buy the things they need at an affordable price.

Although exacerbated by the pandemic, many of these supply chain issues have existed for decades. That’s why, in 2015, Flock Freight was started with the mission of reducing waste and inefficiency from the supply chain by reimagining the way freight moves. First to market with advanced algorithms that enable pooling shipments at scale, we create a new standard of service for shippers, increase revenue for carriers and reduce the impact of carbon emissions through shared truckload (STL) service.

Our technology helps lower prices compared to full truckload (FTL) by enabling shippers to only pay for the space they need—and maintain full control over pickup and delivery dates. Flock Freight also optimizes travel routes to speed up deliveries compared to traditional less than truckload (LTL), while eliminating unnecessary shipping hub transfers to minimize damage to cargo.

Today, thousands of shippers and trucking companies across the U.S. use Flock Freight to schedule shared truckloads, lower shipping costs, quickly deliver and track goods, and reduce their carbon footprint by up to 40%. Flock Freight further offsets carbon emissions by buying carbon credits for every FlockDirect™ guaranteed shared truckload shipment—at no extra cost to shippers.

Moving Flock Freight to Google Cloud

We founded Flock Freight with a small team based in southern California. We soon realized we needed a more scalable and affordable technology stack to support our rapidly growing platform and team. After joining the Google for Startups Cloud Program and consulting with dedicated Google startup experts, we decided to move all our data and applications to Google Cloud.

The highly secure-by-design infrastructure of Google Cloud now enables thousands of Flock Freight customers to move their freight faster, cheaper, and with less damage than traditional shipping methods. Specifically, we rely on Google Kubernetes Engine (GKE) to support the combinatorial optimization and machine learning (ML) algorithms and services that identify, pool, and schedule shared truckloads. We also leverage GKE to rapidly develop, deploy, and manage new applications and services.

In addition, we leverage Cloud SQL to automate database provisioning, storage capacity management, and other time-consuming tasks. Cloud SQL easily integrates with existing apps and Google Cloud services such as GKE and Pub/Sub. Lastly, we use Compute Engine to create and run virtual machines, optimize resource utilization, and lower computing costs by up to 91%. These cost savings allow us to shift more resources to R&D and rapidly develop new solutions and services for our customers.

Building a greener, more resilient, and responsive supply chain

The Google for Startups Cloud Program and dedicated Google startup experts were instrumental in helping us manage cloud infrastructure cost and maintaining very high SLAs, helping Flock Freight to focus on developing a comprehensive shipping platform that powers shared truckloads and drives positive industry change.

We especially want to highlight the Google Cloud research credits we relied on to launch Flock Freight and make rapid progress toward transforming the shipping industry. To this day, we continue to work with Google Cloud Managed Services partner DoiT International International to further scale and optimize operations on Google Cloud.

We’re proud of the results we’re delivering for our customers. For example, a home improvement importer now enjoys faster, safer, and easier shipping with 99.9% damage-free service and a 97.5% on-time delivery rate. A packaging supplier continues to maintain a 99% on-time delivery streak and decrease carbon emissions by 37%, while a mineral water company consistently reduces delivery expenses upwards of 50%.

Nationwide demand for shared truckloads continues to increase as the shipping industry works to lower costs and alleviate supply chain disruptions. With the Flock Freight platform, companies are building a more sustainable and resilient supply chain by efficiently combining multiple shipments into shared truckloads.

If you want to learn more about how Google Cloud can help your startup, visit our page here to get more information about our program, and sign up for our communications to get a look at our community activities, digital events, special offers, and more.