Meesho’s Zero Downtime Success: Cloud CDN Migration Made Easy

1982
Of your peers have already read this article.
4:30 Minutes
The most insightful time you'll spend today!
Meesho is an Indian online marketplace that serves millions of customers every day. Recently, the company decided to adopt a multi-cloud strategy, leveraging Google Cloud’s scalable and reliable infrastructure to drive operational efficiency, modernize and scale for growth. To do so, they needed to migrate billions of static files and images to Google Cloud, to render the static content that serves their web and mobile applications. But with over a petabyte of data in their object storage system, and 10 billion requests per day, Meesho needed to perform this gigantic migration gradually, with zero downtime — a huge challenge.
In this blog post, we look at how Meesho did this using Storage Transfer Service, Cloud Storage and Cloud CDN. We also look at how it saved on storage capacity by resizing static images as needed on the fly, using Cloud Run.
CDN migration requirements
Migrating from one cloud to another isn’t easy. To pull it off, Meesho identified the following requirements:
- Petabyte-scale data transfer: Meesho needed to migrate billions of image files from their existing object storage server to Cloud Storage.
- Dynamic image resizing: To save on storage costs, Meesho wanted the ability to dynamically resize the images based on the end user platform and store the smaller images in the Cloud CDN cache.
- High-throughput data transfer: To support consumer demand, Meesho needed images to be served at a throughput of thousands of requests per second.
- Zero downtime: Since any downtime involves potential loss of revenue, Meesho needed to perform the migration without taking any systems offline.
Migration architecture

The above figure depicts the CDN migration architecture implemented in Meesho. The existing DNS server points to both the source load balancer as well as Google External HTTP Load Balancer with weighted distribution. The source load balancer points to the source object storage. Images were transferred from the source object storage to Google Cloud Storage.
The Google External HTTP Load Balancer was deployed with Cloud CDN to serve static images that are stored in the CDN cache to users. The Google Load Balancer public IP is configured as an end point on their existing DNS server. The Load Balancer is connected to Cloud Run, which talks to the Cloud Storage bucket. When a request reaches the Load Balancer in the edge, it first checks if the content is available in Cloud CDN, and returns the object from the closest edge network. If the image is not available in the Cloud CDN cache, the request is sent to Cloud Run which obtains the image from the Cloud Storage bucket and performs dynamic resizing of the image if necessary.
Data transfer
Meesho used Google Cloud’s Storage Transfer Service to transfer data from their current object storage to Cloud storage bucket over the internet. Since the number of files and total size of the data to be transferred was huge, Meesho executed multiple parallel transfers by specifying folders and subfolders as prefixes in a Storage Transfer Service job.
Dynamic image resizing
Meesho delivers static images to multiple end user platforms — mobile, laptop — at multiple resolutions. Rather than store each image at multiple image resolutions, Meesho opted to store a single high-resolution mezzanine image. It then attached Cloud Run as a serverless network endpoint group to a Cloud Load Balancer. Application requests for images specify the name of the object, the format of the image, and its resolution (for example, abc.jpeg with 750*450 resolution). If the specific image exists for the requested resolution, then it is returned from the Cloud Storage bucket to the end user and stored in the Cloud CDN cache. If an image for a specified resolution and/or format is not found, the mezzanine image (in our example, abc.jpeg) is resized to the specified resolution and format, then stored in Cloud Storage bucket and returned to the end user. The dynamic resizing and formatting is only performed the first time for a specific resolution.
In this architecture, it is important to configure Cloud Run to scale appropriately as it handles a bulk of “CDN cache-miss” requests. Meesho performed the following configuration steps:
- Configured the number of concurrent requests that a single instance of Cloud Run can handle
- Ensured a sufficient minimum of Cloud Run instances were available to serve user traffic to avoid cold-start latency
- Reviewed limits of Cloud Run maximum instance size for the region and increased the limits if necessary to handle peak load
- Set up smaller start-up times for Cloud Run containers, so that the application could quickly autoscale to handle a surge in traffic
- Optimized the memory and CPU configuration to handle processing requirements
CDN configuration
Cloud CDN was configured to ensure a high cache hit ratio > 99 %. This not only sped up the rendering of the images, but also reduced the load on Cloud Run, saving cost and improving performance.
Achieving zero downtime
Meesho followed well-established DevOps principles to achieve a zero-downtime migration:
- Metrics and alerts were configured in Cloud Monitoring to oversee the load balancer.
- The DNS server was configured to point to Cloud Load Balancer IP addresses in addition to their current load balancer, which served status assets.
- Weight-based DNS load balancing was employed to gradually shift the traffic to Google Cloud, while monitoring application performance and HTTP response codes.
- The initial migration process distributed .1% of traffic during non-peak hours. The metrics, end user performance and response codes were continuously monitored.
- Traffic was gradually incremented over a two-week period by increasing the weight of the Google Cloud Load balancer in DNS. By gradually shifting traffic, Meesho ensured a healthy cache-hit ratio, allowing Cloud Run to learn traffic patterns gradually and scale seamlessly.
Meesho learned a lot through this experience, and has the following advice for anyone undertaking a similar migration:
- While transferring data using Storage Transfer Service of Google Cloud, split the transfer process into multiple transfers.
- Ensure that applications do not pin certificates, which could create problems while migrating to the newer certificates in Google Cloud.
- Plan a gradual migration process to gradually increase the traffic to Google Cloud.
Summary
When all is said and done, Meesho considers its migration to Google Cloud a big success. After migrating the static images to Cloud CDN, Meesho held two major sales that each had three times the normal peak traffic, all with no issues. The CDN migration helped Meesho reduce its costs, improve performance and reduce load balancer errors when fetching static images. To learn more about Cloud CDN and how you can use it in your environment, check out the documentation.
Predictive Model Built on Google Cloud Helps You Get a 7-day Mosquito Forecast Report!

4763
Of your peers have already read this article.
3:00 Minutes
The most insightful time you'll spend today!
Mosquitoes aren’t just the peskiest creatures on Earth; they infect more than 700 million people a year with dangerous diseases like Zika, Malaria, Dengue Fever, and Yellow Fever. Prevention is the best protection, and stopping mosquito bites before they happen is a critical step.
SC Johnson—a leading developer and manufacturer of pest control products, consumer packaged goods, and other professional products—has an outsized impact in reducing the transmission of mosquito-borne diseases. That’s why Google Cloud was honored to team up with one of the company’s leading pest control brands, OFF!®, to develop a new publicly available, predictive model of when and where mosquito populations are emerging nationwide.
As the planet warms and weather changes, OFF! noticed month-to-month and year-to-year fluctuations in consumer habits at a regional level, due to changes in mosquito populations. Because of these rapid changes, it’s difficult for people to know when to protect themselves. The OFF!Cast Mosquito Forecast™, built on Google Cloud and available today, will predict mosquito outbreaks across the United States, helping communities protect themselves from both the nuisance of mosquitoes and the dangers of mosquito-borne diseases—with the goal of expanding to other markets, like Brazil and Mexico, in the near future.

With the OFF!Cast Mosquito Forecast™, anyone can get their local mosquito prediction as easily as a daily weather update. Powered by Google Cloud’s geospatial and data analytics technologies, OFF!Cast Mosquito Forecast is the world’s first public technology platform that predicts and shares mosquito abundance information. By applying data that is informed by the science of mosquito biology, OFF!Cast accurately predicts mosquito behavior and mosquito populations in specific geographical locations.
Starting today, anyone can easily explore OFF!Cast on a desktop or mobile device and get their local seven-day mosquito forecast for any zip code in the continental United States. People can also sign up to receive a weekly forecast. To make this forecasting tool as helpful as possible, OFF! modeled its user interface after popular weather apps, a familiar frame of reference for consumers.

The technology behind the OFF!Cast Mosquito Forecast
To create this first-of-its-kind forecast, OFF! stood up a secure and production-scale Google Cloud Platform environment and tapped into Google Earth Engine, our cloud-based geospatial analysis platform that combines satellite imagery and geospatial data with powerful computing to help people and organizations understand how the planet is changing.
The OFF!Cast Mosquito Forecast is the result of multiple data sources coming together to provide consumers with an accurate view of mosquito activity. First, Google Earth Engine extracts billions of individual weather data points. Then, a scientific algorithm co-developed by the SC Johnson Center for Insect Science and Family Health and Climate Engine experts translates that weather data into relevant mosquito information. Finally, the collected information is put into the model and distilled into a color-coded, seven-day forecast of mosquito populations. The model is applied to the lifecycle of a mosquito, starting from when it lays eggs to when it could bite a human.
The SC Johnson Center for Insect Science and Family Health is one the world’s leading entomology research centers, studying advanced science of insect biology, insect-borne disease prevention and effective product solutions for consumer use. The science behind brands like OFF! is grounded in knowledge from world-class entomologists who have devoted their careers to SC Johnson’s mission of eradicating diseases like Malaria and Zika.
“We are putting the power in consumers’ hands in providing them with a tool to help predict their exposure and prevent mosquito bites,” said Maude Meier, SC Johnson entomologist. “It’s an exciting time to be working in the field of insect science as we find new opportunities to combine science and technology, like Google Earth Engine, to be a force for good in our mission to prevent the spread of insect-borne diseases.”
It takes an ecosystem to battle mosquitos
Over the past decade, academics, scientists and NGOs have used Google Earth Engine and its earth observation data to make meaningful progress on climate research, natural resource protection, carbon emissions reduction and other sustainability goals. It has made it possible for organizations to monitor global forest loss in near real-time and has helped more than 160 countries map and protect freshwater ecosystems. Google Earth Engine is now available in preview with Google Cloud for commercial use.
Our partner, Climate Engine, was a key player in helping make the OFF!Cast Mosquito Forecast a reality. Climate Engine is a scientist-led company that works with Google Cloud and our customers to accelerate and scale the use of Google Earth Engine, in addition to those of Google Cloud Storage and BigQuery, among other tools. With Climate Engine, OFF! integrated insect data from VectorBase, an organization that collects and counts mosquitoes and is funded by the U.S. National Institute of Allergy and Infectious Diseases.
The model powering the OFF!Cast Mosquito Forecast combines three inputs—knowledge of a mosquito’s lifecycle, detailed climate data inputs, and mosquito population counts from more than 5,000 locations provided by VectorBase. The model’s accuracy was validated against precise mosquito population data collected over six years from more than 33 million mosquitoes across 141 different species at more than 5,000 unique trapping locations.
A better understanding of entomology, especially things like degree days and how they affect mosquito populations, and helping communities take action is critically important to improving public health. Learn more about OFF!Cast Mosquito Forecast and see here to learn more about Google Earth Engine on Google Cloud.
6187
Of your peers have already watched this video.
13:00 Minutes
The most insightful time you'll spend today!
Google Cloud’s 2021 Data Analytics Launches
Google Cloud announced closed to 15 services and programs spanning database, analytics, business intelligence and AI to help businesses gain value out of their data. Here’s a rundown of announcements throughout 2021 on analytics solutions and services such as Dataplex, an intelligent data fabric solution, Datastream, a serverless change data capture and replication service and Google Cloud analytics hub. Watch the video to get started with Google Cloud’s Data Analytics offerings.
6065
Of your peers have already watched this video.
1:30 Minutes
The most insightful time you'll spend today!
Explore the Innovations and Architecture Powering Spanner and BigQuery
Previously, databases had architectures with tightly coupled storage and compute. This resulted in higher latency, and with faster networks these constraints no longer surface. With Google Cloud’s BigQuery and CloudSpanner, the storage and compute architecture have been separated, allowing for better scalability and availability to address businesses’ high throughput data needs.
Watch the video to understand how these database and analysis products leverage Google’s distributed storage system, in-house custom network hardware and software, internal cluster management system and more!

Google Leads the Pack in Big Data NoSQL Market: Forrester Research
DOWNLOAD RESEARCH REPORTS4145
Of your peers have already downloaded this article
2:30 Minutes
The most insightful time you'll spend today!
NoSQL has become critical for all businesses to support modern business applications. It has gone from supporting simple schemaless apps to becoming a mission-critical data platform for large Fortune 1000 companies. It has already disrupted the database market, which was dominated for decades by relational database vendors.
Today, half of global data and analytics technology decision makers either have implemented or are implementing NoSQL platforms, taking advantage of the benefits of a flexible database that serves a broad range of use cases.
Analyst firm Forrester Research in its recent report has named Google Cloud a leader in the Big Data NoSQL market as it supports a broader set of use cases, automation, good scalability and performance, and security offerings.
Download this Forrester Research report to understand why enterprises are turning to NoSQL and why Google Cloud is a leader in this space.
Firestore Cheatsheet to Unlock Application Innovation

3728
Of your peers have already read this article.
2:00 Minutes
The most insightful time you'll spend today!
Your product teams might ask – “Why does it take so long to build a feature or application?” Building applications is a heavy lift due to the technical complexity, which includes the complexity of backend services that are used to manage and store data. Every moment focused on this technical complexity is a distraction from delivering on core business value. Firestore alters this by having Google Cloud manage your backend complexity through a complete backend-as-a-service!
Firestore is a serverless, NoSQL document database that unlocks application innovation with simplicity, speed and confidence.
It acts as a glue that intelligently brings together the complete Google Cloud backend ecosystem, in-app services from Firebase and core UI frameworks & OS from Google.

What is Firestore?
Firestore is a serverless, fully managed NoSQL document database that scales from zero to global scale without configuration or downtime. Here’s what makes Firestore unique:
- Ideal for rapid, flexible and scalable web and mobile development with direct connectivity to the database.
- Supports effortless real time data synchronization with changes in your database as they happen.
- Robust support for offline mode, so your users can keep interacting with your app even when the internet isn’t available or is unreliable.
- Fully customizable security and data validation rules to ensure the data is always protected
- Built-in strong consistency, elastic scaling, high performance & best in class 99.999% availability
- Integration with Firebase and Google Cloud services like Cloud Functions and BigQuery, serverless data warehouse.
- In addition to a rich set of Google Cloud service integrations, Firestore also offers deep one-click integrations with a growing set of 3rd party partners via Firebase Extensions to help you even more rapidly build applications.
Document-model database

Firestore is a Document-model database. All of your data is stored in “documents” and then “collections”. You can think of a document as a JSON object. It’s a dictionary with a set of key-value mappings, where the values can be several different supported data types including strings, numbers or binary values..
These documents are stored in collections. Documents can’t directly contain other documents, but they can point to subcollections that contain other documents, which can point to subcollections, and so on. This structure brings with it a number of advantages. For starters, all queries that you make are shallow, meaning that you can grab a document without worrying about grabbing all the data underneath it. And this means that you can structure your data hierarchically in a way that makes sense to you logically, without having to worry about grabbing tons of unnecessary data.
How to use Firestore?
Firestore can be used in two modes:
- Firestore in Native Mode: This mode is differentiated by its ability to directly connect your web & mobile app to Firestore. Native Mode supports up to 10K writes per second, and over a million connections.
- Firestore in Datastore Mode: This mode supports only server-side usage of Firestore, but supports unlimited scaling, including writes.
Conclusion
Whatever your application use case may be, if you want to build a feature or an application quickly using Firestore backend-as-a-service. For a more in-depth look into Firestore check out the documentation. https://www.youtube.com/embed/moglAjmwmUQ?enablejsapi=1&
For more #GCPSketchnote, follow the GitHub repo. For similar cloud content follow me on Twitter @pvergadia and keep an eye out on thecloudgirl.dev.
More Relevant Stories for Your Company

BigQuery Omni: Your Solution for Multi-Cloud Geospatial Analytics
As we become increasingly reliant on technology to make decisions, geospatial data is becoming more critical than ever. It is a powerful resource that can be used to solve a variety of problems, from tracking the movement of goods, identifying interest areas and potential areas of disaster. Geospatial data incorporates

Canadian Bank’s SAP Workload Moved to BigQuery Helps Unlock New Business Opportunities
When ATB Financial decided to migrate its vast SAP landscape to the cloud, the primary goal was to focus on things that matter to customers as opposed to IT infrastructure. Based in Alberta, Canada, ATB Financial serves over 800,000 customers through hundreds of branches as well as digital banking options. To

Home Depot’s Interconnected Retail Experience by Virtue of Google Cloud Migration for SAP Applications
With nearly 2,300 stores, The Home Depot is the world’s largest home-improvement chain — a brand that professional contractors and DIYers alike have come to depend on. The home improvement industry continues to experience unprecedented demand and dramatic increases in online ordering accompanied by expanding consumer expectations for things like

The Right Datawarehouse Helps Fight Climate Change, While Improving Customer Experience
When you think about climate change, you might not consider a daily commute to work or a drive around town as big contributing factors. And yet, transport is the fastest growing source of CO2 emissions from fossil fuel, which in turn is the largest contributor to climate change. This is the key insight






