Cloud Bigtable Helps Fraud-detection Company Meet Scalability Demands and Secure Customer Data

7131
Of your peers have already read this article.
3:00 Minutes
The most insightful time you'll spend today!
Editor’s note: Today we are hearing from Jono MacDougall , Principal Software Engineer at Ravelin. Ravelin delivers market-leading online fraud detection and payment acceptance solutions for online retailers. To help us meet the scaling, throughput, and latency demands of our growing roster of large-scale clients, we migrated to Google Cloud and its suite of managed services, including Cloud Bigtable, the scalable NoSQL database for large workloads.
As a fraud detection company for online retailers, each new client brings new data that must be kept in a secure manner and new financial transactions to analyze. This means our data infrastructure must be highly scalable and constantly maintain low latency. Our goal is to bring these new organizations on quickly without interrupting their business. We help our clients with checkout flows, so we need latencies that won’t interrupt that process—a critical concern in the booming online retail sector.
We like Cloud Bigtable because it can quickly and securely ingest and process a high volume of data. Our software accesses data in Bigtable every time it makes a fraud decision. When a client’s customer places an order, we need to process their full history and as much data as possible about that customer in order to detect fraud, all while keeping their data secure. Bigtable excels at accessing and processing that data in a short time window. With a customer key, we can quickly access data, bring it into our feature extraction process, and generate features for our models and rules. The data stays encrypted at rest in Bigtable, which keeps us and our customers safe.
Bigtable also lets us present customer profiles in our dashboard to our client, so that if we make a fraud decision, our clients can confirm the fraud using the same data source we use.

We have configured our bigtable clusters to only be accessible within our private network and have restricted our pods access to it using targeted service accounts. This way the majority of our code does not have access to bigtable and only the bits that do the reading and writing have those privileges.
We also use Bigtable for debugging, logging, and tracing, because we have spare capacity and it’s a fast, convenient location.
We conduct load testings against Bigtable. We started at a low rate of ~10 Bigtable requests per second and we peaked at ~167000 mixed read and write requests per second at absolute peak. The only intervention that was done to achieve this was pressing a single button to increase the number of nodes in the database. No other changes were made.
In terms of real traffic to our production system, we have seen ~22,000 req/s (combined read/write) on Bigtable in our live environment as a peak within the last 6 weeks.
Migrating seamlessly to Google Cloud
Like many startups, we started with Postgres, since it was easy and it was what we knew, but we quickly realized that scaling would be a challenge, and we didn’t want to manage enormous Postgres instances. We looked for a kind of key value store, because we weren’t doing crazy JOINS or complex WHERE clauses. We wanted to provide a customer ID and get everything we knew about it, and that’s where key value really shines.
I used Cassandra at a previous company, but we had to hire several people just for that chore. At Ravelin we wanted to move to managed services and save ourselves that headache. We were already heavy users and fans of BigQuery, Google Cloud’s serverless, scalable data warehouse, and we also wanted to start using Kubernetes. This was five years ago, and though quite a few providers offer Kubernetes services now, we still see Google Cloud at the top of that stack with Google Kubernetes Engine (GKE). We also like Bigtable’s versioning capability that helped with a use case involving upserts. All of these features helped us choose Bigtable.
Migrations can be intimidating, especially in retail where downtime isn’t an option. We were migrating not just from Postgres to Bigtable, but also from AWS to Google Cloud. To prepare, we ran in AWS like always, but at the same time we set up a queue at our API level to mirror every request over to Google Cloud. We looked at those requests to see if any were failing, and confirmed if the results and response times were the same as in AWS. We did that for a month, fine tuning along the way.
Then we took the big step and flipped a config flag and it was 100% over to Google Cloud. At the exact same time, we flipped the queue over to AWS so that we could still send traffic into our legacy environment. That way, if anything went wrong, we could fail back without missing data. We ran like that for about a month, and we never had to fail back. In the end, we pulled off a seamless, issue-free online migration to Google Cloud.
Flexing Bigtable’s features
For our database structure, we originally had everything spread across rows, and we’d use a hash of a customer ID as a prefix. Then we could scan each record of history, such as orders or transactions. But eventually we got customers that were too big, where the scanning wasn’t fast enough. So we switched and put all of the customer data into one row and the history into columns. Then each cell was a different record, order, payment method, or transaction. Now, we can quickly look up the one row and get all the necessary details of that customer. Some of our clients send us test customers who place an order, say, every minute, and that quickly becomes problematic if you want to pull out enormous amounts of data without any limits on your row size. The garbage collection feature makes it easy to clean up big customers.
We also use Bigtable replication to increase reliability, atomicity, and consistency. We need strong consistency guarantees within the context of a single request to our API since we make multiple bigtable requests within that scope. So within a request we always hit the same replica of Bigtable and if we have a failure, we retry the whole request. That allows us to make use of the replica and some of the consistency guarantees, a nice little trade-off where we can choose where we want our consistency to live.https://www.youtube.com/embed/0-eH5u7rrQQ?enablejsapi=1&
We also use BigQuery with Bigtable for training on customer records or queries with complicated WHERE clauses. We put the data in Bigtable, and also asynchronously in BigQuery using streaming inserts, which allows our data scientists to query it in every way you can imagine, build models, and investigate patterns and not worry about query engine limitations. Since our Bigtable production cluster is completely separate, doing a query on BigQuery has no impact on our response times. When we were on Postgres many years ago, it was used for both analysis and real time traffic and it was not the optimal solution for us. We also use Elasticsearch for powering text searches for our dashboard.
If you’re using Bigtable, we recommend three features:
- Key visualizer. If we get latency or errors coming back from Bigtable, we look at the key visualizer first. We may have a hotkey or a wide row, and the visualizer will alert us and provide the exact key range where the key lives, or the row in question. Then we can go in and fix it at that level. It’s useful to know how your data is hitting Bigtable and if you’re using any anti-patterns or if your clients have changed their traffic pattern that exacerbated some issue.
- Garbage collection. We can prevent big row issues by putting size limits in place with the garbage collection policies.
- Cell versioning. Bigtable has a 3d array, with rows, columns, and cells, which are all the different versions. You can make use of the versioning to get history of a particular value or to build a time series within one row. Getting a single row is very fast in Bigtable so as long as you can keep the data volume in check for that row, making use of cell versions is a very powerful and fast option. There are patterns in the docs that are quite useful and not immediately obvious. For example, one trick is to reverse your timestamps (MAXINT64 – now) so instead of the latest version, you can get the oldest version effectively reversing the cell version sorting if you need it.
Google Cloud and Bigtable help us meet the low-latency demands of the growing online retail sector, with speed and easy integration with other Google Cloud services like BigQuery. With their managed services, we freed up time to focus on innovations and meet the needs of bigger and bigger customers.
Learn more about Ravelin and Bigtable, and check out our recent blog, How BIG is Cloud Bigtable?
The Ultimate Guide to VPC Service Controls: Strengthening Your Security Posture

2515
Of your peers have already read this article.
2:30 Minutes
The most insightful time you'll spend today!
While cloud security skeptics might believe that data in the cloud is just one access configuration mistake away from a breach, the reality is that a well-designed set of defense in depth controls can help minimize the risk of configuration mistakes and other security issues. Our Virtual Private Cloud (VPC) Service Controls can play a vital role in creating an additional layer of security while also making it easier to manage your data in a way that most cloud services can’t do today.
Organizations across industries and business models use cloud services for activities such as processing their data, performing analytics, and deploying systems. VPC Service Controls can empower an organization when deciding how users and data can cross the perimeter of the supported cloud services, if at all. While VPC Service Controls are designed to help stop attackers, they can also enable contextual trusted data sharing (similar to how Zero Trust allows contextual access).
What are VPC Service Controls
VPC Service Controls help administrators define a security perimeter around Google-managed services, which can control communication to and between those services. The Service Controls isolate your Google Cloud resources from unauthorized networks, including the internet. For example, this can help you keep a clear separation between services that are allowed to run in production and services that are not.
VPC Service Controls can help you prevent mistakes that lead to costly data breaches because they control access to your data at a granular level. They add context-aware access controls on these services, and can help you achieve your organization’s Zero Trust access goals.

Example of the fine-grained policies based on access context that can be implemented with VPC Service Controls.
Like wearing two layers of clothing made from different fabrics to protect you from winter weather, VPC Service Controls may appear similar to Identity and Access Management (IAM) but they come from a different approach to implementing security. IAM enables granular identity based access control; VPC Service Controls create a security perimeter that protects your cloud resources and sets up private connectivity to Google Cloud’s APIs and services. While it’s recommended to use both, VPC Service Controls have an added bonus: They can support blocking data theft during a breach.
The additional layer of security that VPC Service Controls offer customers is challenging to achieve with on-premise systems or even with other cloud providers. You can think of it as an firewall for APIs that also adds a logical security control around three paths that data can take:
- From the public internet to your resources
- Inside your VPC and the cloud service perimeter
- For service-to-service communication (for example, denying access to someone who wants to load data to BigQuery or exfiltrate data from a BigQuery instance.)
How VPC Service Controls can help stop attackers
VPC Service Controls are used to enforce a security perimeter. They can help isolate resources of multi-tenant Google Cloud services, which can help reduce the risk of data exfiltration or a data breach.
For example, a bank that migrated financial data processing to Google Cloud can use VPC Service Controls to isolate their processing pipeline from public access (or any unauthorized access) by defining a trusted service perimeter.
How VPC Service Controls can enable trusted sharing
VPC Service Controls are used to securely share data across service perimeters with full control over what resource can connect to other resources, or outside the perimeter. This can help mitigate data exfiltration risks stemming from stolen identities, IAM policy misconfigurations, some insider threats, and compromised virtual machines.
Returning to our bank example, that same bank using VPC Service Controls may securely share or access data across Service Perimeters and Organizations. They may allow access to specific partners and for specific operations.

Example of allowing an authorized device plus authorized access.
How VPC Service Controls support Zero Trust access
VPC Service Controls deliver Zero Trust access to multi-tenant Google Cloud services. Clients can restrict access to authorized IPs, client context, user identity, and device parameters while connecting to multi-tenant services from the internet and other services.
A bank can use moving its services to the public cloud as an opportunity to abandon outdated access management approaches and adopt Zero Trust access. VPC Service Controls let them create granular access control policies in Access Context Manager based on attributes such as user location and IP address. For example, it would allow an analyst to only access Big Query from a corporate device on the corporate network during business hours. These policies can help ensure the appropriate security controls are in place when granting access to cloud resources from the Internet.
Next steps with VPC
Check out these pages to learn more about VPC Service Controls for your sensitive cloud deployments, especially for regulated workloads. This blog is the third in our Best Kept Security Secrets series, which includes how to tap into the power of Organization Policy Service and how Cloud EKM can help resolve the cloud trust paradox.
Latest News: Secure Digital Infrastructure Services with Apigee Advanced API Security for Google Cloud

4485
Of your peers have already read this article.
3:30 Minutes
The most insightful time you'll spend today!
Organizations in every region and industry are developing APIs to enable easier and more standardized delivery of services and data for digital experiences. This increasing shift to digital experiences has grown API usage and traffic volumes. However, as malicious API attacks also have grown, API security has become an important battleground over business risk.
To help customers more easily address their growing API security needs, Google Cloud is announcing today the Preview of Advanced API Security, a comprehensive set of API security capabilities built on Apigee, our API management platform. Advanced API Security enables organizations to more easily detect security threats. Here’s a closer look at the two key functionality included in this launch: identifying API misconfigurations and detecting bots.
Identify API misconfigurations
Misconfigured APIs are one of the leading reasons for API security incidents. In 2017, Gartner® predicted that by 2022 API abuses will be the most frequent attack vector resulting in data breaches for enterprise web applications. Today, our customers tell us application API security is one of their top concerns, which is supported by an independent study from 2021 by Fugue and Sonatype. The report found that misconfigurations are the number one cause of data breaches, and that “too many cloud APIs and interfaces to adequately govern” are frequently the main point of attack in cyberattacks.
While identifying and resolving API misconfigurations is a top priority for many organizations, the configuration management process can be time consuming and require considerable resources.
Advanced API Security can make it easier for API teams to identify API proxies that do not conform to security standards. To help identify APIs that are misconfigured or experiencing abuse, Advanced API Security regularly assesses managed APIs and provides API teams with a recommended action when configuration issues are detected.

Advanced API Security identifies misconfigured API proxies, including the missing CORS policy.
APIs form an integral part of the digital connective tissue that make modern medicine run smoothly for patients and healthcare staff. One common healthcare API use case occurs when a healthcare organization inputs a patient’s medical coverage information into a system that works with insurance companies. Almost instantly, that system determines the patient’s coverage for a specific medication or procedure, a process which is enabled by APIs. Because of the often-sensitive personal healthcare data being transmitted, it is important that the required authentication and authorization policies are implemented so that only authorized users, such as an insurance company, can access the API.
Advanced API Security can detect if those required policies have not been applied, an alert which can help reduce the surface area of API security risks. By leveraging Advanced API Security, API teams at healthcare organizations can more easily detect misconfiguration issues and can reduce security risks to sensitive information.
Detect Bots
Because of the increasing volume of API traffic, there is also an increase in cybercrime in the form of API bot attacks—the automated software programs deployed over the Internet for malicious purposes like identity theft.
Advanced API Security uses pre-configured rules to help provide API teams an easier way to identify malicious bots within API traffic. Each rule represents a different type of unusual traffic from a single IP address. If an API traffic pattern meets any of the rules, Advanced API Security reports it as a bot.
Additionally, Advanced API Security can speed up the process of identifying data breaches by identifying bots that successfully resulted in the HTTP 200 OK success status response code.

Financial services APIs are frequently the target of malicious bot attacks due to the high-value data that is processed. A bank that has adopted open banking standards by making APIs accessible to customers and partners can use Advanced API Security to make it easier to analyze traffic patterns and identify the sources of malicious traffic. You may experience this when your bank allows you to access your data with a third-party application. While a malicious hacker could try to use a bot to access this information, Advanced API Security can help the bank’s API team to identify and stop malicious bot activity in API traffic.
API Security at Equinix
Equinix powers the world’s digital leaders, bringing together and interconnecting infrastructure to fast-track digital advantage. Operating a global network of more than 240 data centers with a 99.999% or greater uptime, Equinix simplifies global interconnections for organizations, saving customers time and effort with the Apigee API management platform.
“A key enabler of our success is Google’s Apigee, delivering digital infrastructure services securely and quickly to our customers and partners,” said Yun Freund, senior vice president of Platform at Equinix. “Security is a key pillar to our API-first strategy and Apigee has been instrumental in enabling our customers to securely bridge the connections they need for their businesses to easily identify potential security risks and mitigate threats in a timely fashion. As our API traffic has grown, so has the amount of time and effort required to secure our APIs. Having a bundled solution in one managed platform gives us a differentiated high-performing solution.”
Getting started
To learn more, check out the documentation or contact us to request access to get started with Advanced API Security.
To learn more about API security best practices, please register to attend our Cloud OnAir webcast on Thursday, July 28th, 2:00 pm PT.
Chrome OS’s Hybrid Work Model Powers Google’s Return to Work Strategy

7510
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
The pandemic continues to deeply affect our lives around the globe. In some places, new cases are surging and returning to work is the last thing on people’s minds. In other areas, conditions are improving and companies are starting to think about transitioning their workforce back to the office.
Exactly when and how to do this remains complex and varies by country, industry, and company. What’s certain is that hybrid work will become an essential part of the business world moving forward. And finding solutions that bridge the gap between “in-person” and “somewhere else” are crucial.
At Google, we’ve been focused on what the hybrid transformation means for us. To prepare for hybrid work, using modern solutions is a key enabler.
Chrome OS: Supporting Google’s return to office strategy
At Google, we’ll move to a hybrid work week where most Googlers spend approximately three days in the office and two days wherever they work best. It’s no surprise that as the modern, secure, cloud-first platform, Chrome OS is playing a key role in our transition to a hybrid work model. Because Chrome OS devices can easily be shared, more flexible working models and spaces are now possible. And with user profiles stored in the cloud and collaboration solutions like Google Workspace, employees can log in to any Chrome OS device, access what they need and pick up where they left off.
Here are just a few ways we are using Chrome OS and its tools to support the return to the office:
- In select office locations, Googlers can reserve desks through an internal booking tool set up with a high-performance Chromebox, keyboard, mouse, and monitor. Employees can log in to the Chromebox which syncs their cloud profile, and start working with the same environment they have on all their Chrome OS devices.
- We’re announcing new docking stations that are designed for Chrome OS devices and allow employees to bring in their Chromebook from home, connect to the dock with one USB-C cable, and use a monitor, keyboard, and mouse for a full desktop experience.
- Every Chrome OS device enables a zero-trust security working model with BeyondCorp Enterprise providing our workforce with simple and secure access to applications while providing additional security controls for IT.
- We’ve deployed the new Chrome OS Readiness Tool to our extended workforce to identify employees that are able to switch to Chrome OS. This allows us to expand the latest security, deployment, and manageability benefits to more of the workforce.
Additional ways Chrome OS is helping organizations with return to office
We aren’t the only ones supporting our return to office and hybrid work strategy with Chrome OS. We’ve heard more ways our customers are using Chrome OS to make the transition as smooth as possible. These include:
- Streamlining deployment and management of Chrome OS devices using zero-touch enrollment which allows devices to automatically enroll into a corporate domain without IT configuration.
Grab & Go Chromebooks being used for frontline and hybrid information workers, allowing employees to grab a Chromebook from a cart and get to work right away. - Parallels Desktop for Chrome OS being deployed to allow employees to access Windows or legacy apps locally on their Chrome OS devices.
- Existing Windows and Mac devices being modernized and repurposed to run a Chrome OS experience using CloudReady. (Google is currently offering a CloudReady promotion. Learn more here.)
While the Chrome OS team has been working towards making remote working as seamless as possible for IT, we’ve also made advancements in supporting traditional technology that’s required in the office.
- In October, we announced Chrome Enterprise Recommended: a collection of identity, printing, productivity, communications, and virtualization solutions that are verified to run great on Chrome OS.
- With the increased usage of video conferencing on Chrome OS devices, we’ve made improvements to Google Meet and Zoom performance including camera and video improvements to reduce any unnecessary processing and features that intelligently adapt to your device, your network, and what you are working on.
- For improved access to Windows and legacy apps, VMware Horizon introduced multi-monitor support and USB redirection and Citrix Workspace released a tech preview with webcam enhancements and Microsoft Teams optimizations.
- We integrated with the Okta Workflows platform, so IT administrators can include Chrome OS in it’s access logic and deploy quickly without code. Recently, we’ve added the ability to require users to reauthenticate on their Chrome OS device once Okta has detected that the user has changed their password. Try out this new capability by signing up for our Trusted Tester program.
- We’ve increased our support for direct IP printing with additional printer models since the beginning of 2020. In addition, we have made improvements to management features, including support for multiple print servers and launched policy APIs to provide a better IT admin experience.
Join us for a digital event with Modern Computing Alliance and its newest member HP
Last year we announced the launch of the Modern Computing Alliance—a collaboration of industry leaders including Box, Chrome Enterprise, Citrix, Dell, Google Workspace, Imprivata, Intel, Okta, RingCentral, Slack, VMware, and Zoom aim to create the pioneering solutions that businesses need. We are thrilled to introduce HP, who brings their innovative hardware and enterprise hardware expertise to the Modern Computing Alliance and has been working closely with the alliance to ensure true silicon-to-cloud innovation.
As an alliance, we’ve been deeply engaged in the hybrid work shift. We invite you to join us for a digital event where we discuss questions about returning to work. Like how product design can encourage participation and collaboration regardless of where employees are, important security issues to keep in mind, and what factors businesses should consider to support a safe, working environment.
Home. Heading back. Hybrid.
Hear from the experts on hybrid work and return to office.
Date: May 20th, 2021
Register here.
If you are ready to try Chrome OS today, it’s easy to get started. You can contact us to get connected to a partner, sign up for a free 30-day trial of Chrome Enterprise Upgrade to start managing Chrome OS devices, or deploy the Chrome OS Readiness Tool to identify employees that are ready to switch to Chrome OS.POSTED IN:
Ubuntu Pro Images Now Available on Google Cloud

4839
Of your peers have already read this article.
2:00 Minutes
The most insightful time you'll spend today!
Today, we’re pleased to announce the general availability of Ubuntu Pro images on Google Cloud, providing customers with an improved Ubuntu experience, expanded security coverage, and integration with critical Google Cloud features. In partnership with Canonical, we’re making it even easier for customers that have fully embraced open source to ensure security and compliance for their most mission-critical and enterprise workloads.
With Ubuntu Pro on Google Cloud, you now have access to features like:
- 10-year lifetime security updates – Canonical backs Ubuntu Pro for 10 years with security updates and a guaranteed upgrade path.
- FIPS & CC-EAL2 certification – Ubuntu Pro includes components that meet requirements from entities like FedRAMP, HIPAA, ISO, and PCI.
- Open-source security coverage – Protect your most important open-source workloads including MongoDB, Apache Kafka, Redis, NGINX, and PostgreSQL.
- Multi-version availability – Pro images are available for the three most popular Ubuntu Server distributions: 16.04 LTS, 18.04 LTS, and 20.04 LTS.
- Kernel Livepatch – Kernel patches are delivered immediately without having to reboot your VMs.
- Optional CIS and DISA STIG profiles – Choose from two leading profiles to harden your environment according to industry benchmarks.
- Cloud-based pricing – Ubuntu Pro does not require a contract, and pricing tracks with the underlying compute cost depending on the instance type.
Extended Security Maintenance (ESM) for Ubuntu 16.04 LTS with Ubuntu Pro
Availability of Ubuntu Pro images is especially important if you’re an Ubuntu 16.04 LTS customer and want extended security maintenance (ESM) for your virtual machines but don’t want to upgrade to Ubuntu 18.04 LTS or Ubuntu 20.04 LTS versions immediately. ESM is included with Ubuntu Pro 16.04. You can move your workloads from Ubuntu 16.04 LTS VM instances to Ubuntu Pro 16.04 instances to continue receiving ESM and all the above-mentioned benefits, without having to test your applications on a new version of the OS.

Gojek has evolved from offering just ride-hailing to a suite of more than 20 services today, serving everyday solutions for millions of users across Southeast Asia.
“We needed more time to comprehensively test and migrate our Ubuntu 16.04 LTS workloads to Ubuntu 20.04 LTS, which would mean stretching beyond the standard maintenance timelines for Ubuntu 16.04 LTS. With Ubuntu Pro on Google Cloud, we now have the ability to postpone this, and in moving our 16.04 workloads to Ubuntu Pro, we benefit from its live kernel patching and improved security coverage for our key open source components.”—Kartik Gupta, Engineering Manager for CI/CD & FinOps at Gojek
“With the launch of Ubuntu Pro on Google Cloud, we build on our joint investments with Google to optimize Ubuntu performance on Google Cloud, and add comprehensive security patching and Long Term Support for another 30,000 open source packages—the widest range of security-maintained open source on the planet,” said Mark Shuttleworth, CEO of Canonical. “As the world moves to open source for everything, Canonical offers the safety net of security maintenance that enterprises count on to unleash their developers.”
Getting started
Getting started with Ubuntu Pro on Google Cloud is simple. You can now purchase these premium images directly from Google Cloud by selecting Ubuntu Pro as the operating system straight from the Google Cloud Console.
To learn more about Ubuntu Pro on Google Cloud, please visit the documentation page and read the announcement from Canonical.
Vodafone Leverages Google Cloud to Aid COVID-19 Frontline with Anonymized Insights on Population Mobility

11215
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
Editor’s note: When Europe’s largest mobile communications company, Vodafone, was asked by the European Commission to help understand population movement across the European Union and the UK to help fight COVID-19, it was able to provide anonymized mobile network-based insights to answer the call. Here’s how Vodafone, with the support of Google Cloud, rapidly mobilized the COVID-19 frontline, while respecting its customers’ privacy.
With the emergence of COVID-19 in early 2020, the European Commission—the executive branch of the European Union (EU)—knew that technology would be instrumental in its fight to control the pandemic. With various lockdowns imposed across its member states, the Commission was keen to predict and prevent the spread of COVID-19 and to manage the related social, political and financial impacts.
Mobile network data helps track COVID-19 across the EU
Mobile networks produce location data, which can be turned into useful anonymous insights to understand population movement within a geographic area. The European Commission, working with mobile industry association GSMA (Groupe Speciale Mobile Association), asked Europe’s major mobile phone operators for help in producing insights to support the fight against COVID-19. As the largest mobile network operator within the EU, Vodafone saw this as a critical opportunity to participate.
Vodafone had previous experience of using mobile network data to support pandemic research. For example, in 2019, Vodafone provided mobility pattern analysis to help track the spread of Malaria in Mozambique. And, during the early stages of the COVID-19 pandemic (prior to working with the European Commission), Vodafone assisted the Italian and Spanish governments in understanding their citizens’ mobility patterns. Vodafone had also previously offered anonymized and aggregated population mobility insights to support public transport and tourism authorities and retail organizations in a number of countries. Consequently, Vodafone was perfectly placed to play a greater role in supporting the European Commission’s response to the pandemic.
When asked to assist the European Commission, Vodafone first considered how it could safely share its data with the governing body without providing details on the individual movements of its customers. It realized it could achieve this through an elaborate set of anonymization and aggregation techniques. Insights are aggregated from a minimum of 50 users and Vodafone only shared these anonymous insights and never the actual raw data with the Commission. As specified by the EU, these insights are then presented onto a large geographical region, typically a city or a county with thousands of people living in that area.
These insights illustrate how people move, helping to determine how lockdowns and self-isolation measures were impacting behaviors.
Using Google Cloud to collate and store population mobility data
In April 2020, Vodafone began migrating its operations, including its mobile data, to Google Cloud on servers in Europe and the UK with elaborate security safeguards, including encryption, building on a previous partnership.
With the data residing in EU and UK data centers and not the United States, Vodafone could then retrieve anonymous insights from Google Cloud Storage instantaneously. Before supplying any information to the European Commission, however, Vodafone used Dataflow to validate the data and run a series of tests to ensure the database had accurate data, before ingesting and archiving the relevant metrics. For instant access, the data was then made available to the European Commission using a Redis database on Google Kubernetes Engine.
To ensure aggregate Vodafone customer data was always safe, secure, and anonymous, all entry points to the front-end were protected behind Google Cloud Armor, where only specific IP addresses were allowed. Using these tools, seamless data pipelines fed in predefined key performance indicators from each specified European market. While data quality measures ensured the definitions for metrics across markets were consistent and could be accurately compared.
The architecture (pictured below) shows how Vodafone integrated and anonymized its data on Google Cloud.

Live interactive dashboard shows population mobility in real-time
With its data integrated on Google Cloud, Vodafone created a live, interactive dashboard to track mobility patterns and share relevant information with the European Commission in real-time.
The European Commission Joint Research Center (JRC) was able to gather valuable information from these insights, which enabled them to see where population mobility was aiding the spread of the disease, when cross-referenced with health data. It could also assess the implications of lockdowns on different populations and forecast cross-country spreading.
Mobile data aids disease modeling for multiple stakeholders
The Vodafone data became instrumental in modeling the likely course of the disease too. For example, the University of Southampton in the UK used it to predict the outcome of different coordinated COVID-19 exit strategies across Europe. This research was published in Science Magazine in September 2020.
The Vodafone data dashboard continues to be used by individual governments, NGOs and organizations to further investigate the impacts of the pandemic and to measure the effectiveness of response strategies alongside the rollout of vaccination programs. The project also helped Vodafone win a DataIQ award for most effective stakeholder engagement.
Using the learnings from this project, Vodafone has been able to adapt its own B2B solution, called Vodafone Analytics, by adaptIng and migrating the code to work in Google Cloud Platform. This solution has been rolled out across Germany, Greece, Portugal and South Africa, and new countries are being onboarded every day. Vodafone Analytics already has more than 100 customers leveraging it for a variety of use cases—Italian fashion retailer OVS, uses it for its smart retail operation, while global real estate company, JLL, uses it to understand the footfall passing through its properties.
Working together, Vodafone and Google Cloud continue to help a range of organizations, governments, and NGOs navigate through the ongoing pandemic, optimize their operations, and help the greater good, without infringing individuals’ fundamental rights to privacy.
To learn more about Google Cloud and Vodafone, watch our full interview here.
More Relevant Stories for Your Company

Best Practices for Cost Optimization in the Cloud
When customers migrate to Google Cloud Platform (GCP), their first step is often to adopt Compute Engine, which makes it easy to procure and set up virtual machines (VMs) in the cloud that provide large amounts of computing power. Launched in 2012, Compute Engine offers multiple machine types, many innovative features, and is available in

5 ways a SOAR solution improves SOC analyst onboarding
Editor's note: This blog was originally published by Siemplify on Feb. 19, 2021. The number of unfilled cybersecurity jobs stretches into the millions, and a critical part of the problem is the length of time it takes to backfill a position. Industry group ISACA has found that the average cybersecurity

How AI-powered ML Models Helps Run Unemployment Claims Verification at Scale
With unemployment application submissions reaching record numbers over the past year, state and local agencies in the United States have faced the challenge of processing unprecedented numbers of claims per week. The digital infrastructure most agencies have in place is unable to handle this volume, resulting in constituents waiting longer,

Announcing New, Faster Search and Investigative Experience in Chronicle Security Operations
In cybersecurity, speed matters. Whether a security analyst is trying to understand the details of an alert that was triggered by an indicator of compromise (IoC), or find additional context for a suspicious asset, speed is often the critical factor that will help thwart a cyberattack before threat actors are






