Editor’s note: Today’s guest post comes from Indian social media platform ShareChat. Here’s the story of how they improved performance, app development, and analytics for serving regional content to millions of users using Google Cloud. 

How do you create a social network when your country has 22 major official languages and countless active regional dialects? At ShareChat, we serve more than 160 million monthly active users who share and view videos, images, GIFs, songs, and more in 15 different Indian languages. We also launched a short video platform in 2020, Moj, which already supports over 80 million monthly active users. 

Connecting with people in the language they understand

As mobile data and smartphones have become more affordable in India, we noticed a large new segment of people, many in rural areas, being welcomed onto the internet. However, many of them didn’t speak English, and when it comes to accessing content and information—language plays a significant role. Instead of joining other social media sites where English reigned supreme, new internet users chose to join language or dialect-specific Whatsapp groups where they felt more comfortable instead.

So, we set out to build a platform where people can share their opinions, document their lives, and make new friends, all in their native language. ShareChat simplifies content and people discovery by using a personalized content newsfeed to deliver language-specific content to the right audience.

Given the high-intensity data and high volume of content and traffic, we rely heavily on IT infrastructure. On top of that, a large number of our users rely on 2G networks to post, like, view, or follow each other. Our platform needs to deliver great experiences to people who are spread out across the country and different networks without any reduction in performance.

The right cloud partner to support future growth

ShareChat was born in the cloud—we already knew how to scale systems to serve a large customer base with our existing cloud provider. But like many companies, we struggled with over-provisioning compute and storage to accommodate unpredictable traffic and avoid running out of storage. With demand rising for local language content and an increase in online interactions in response to the COVID-19 crisis, we realized that we would need a more efficient way to scale dynamically and allocate resources as needed.

Google Cloud was a natural choice for us. We wanted to partner with a technology-first company that would make it easy (and cost-effective) to manage a strong technology portfolio that would allow us to build whatever we wanted. Google is at the forefront of technology innovation and provided everything we needed to build, run, and manage our applications (including creating an efficient DevOps pipeline to fix and release new features quickly). 

We had a few issues in mind at the start of discussions with the Google Cloud team, but over time, as we got information and support from them, we realized that these were the partners we wanted in our corner when it came time to tackle our most challenging problems. In the end, we decided to take our entire infrastructure to Google Cloud.

To support millions of users, we deploy and scale using Google Kubernetes Engine. While we analyze our data using a combination of managed data cloud services, such as Pub/Sub for data pipelines, BigQuery for analytics, Cloud Spanner for real-time app serving workloads, and Cloud Bigtable for less-indexed databases. We also rely on Cloud CDN to help us distribute high-quality and reliable content delivery at low latency to our users. 

We now use just half the total core consumption of our legacy environment to run ShareChat’s existing workloads.

Google Cloud delivers better outcomes at every level 

By moving to Google Cloud, we saw major benefits in several key areas: 

Zero-downtime migration for users

At the time of migration, we had over 70 terabytes of data, consisting of 220 tables—some of which were up to 14 terabytes with nearly 50 billion rows. Due to our data’s interdependencies, moving services over one at a time wasn’t an option for us. 

Even though we were migrating such large volumes of data, we didn’t want to impact any of our customers. Latency spikes for out-of-sync data might affect message delivery. For instance, if a message or notification was delayed, we didn’t want to risk a bad user experience causing someone to abandon ShareChat. 

To prepare for the move, we ran a proof-of-concept cluster for over four months to test database performance in a real-world scenario for handling more than a million queries per second. Using an open-source API gateway, we replicated our legacy data environment into Google Cloud for performance testing and capacity analysis. As soon as we were confident Google Cloud could handle the same traffic as our previous cloud environment, we were ready to execute.

Using wrappers, we were able to migrate without having to change anything in our existing application code. The entire migration of 60 million users to Google Cloud took five hours—without any data loss or downtime. Today, ShareChat has grown to 160 million users, and Google Cloud continues to give us the support we need.

Scaling globally to meet unexpected demand

We rely on real-time data to drive everything on ShareChat by tracking everything that goes on in our app—from messages and new groups to content people like or who they follow. Our users create more than a million posts per day, so it’s critical that our systems can process massive amounts of data efficiently. 

We chose to migrate to Spanner for its global consistency and secondary index. Unlike our legacy NoSQL database, we could scale without having to rethink existing tables or schema definitions and keep our data systems in sync across multiple locations. It’s also cost-effective for us—moving over 120 tables with 17 indexes into Cloud Spanner reduced our costs by 30%.

Spanner also replicates data seamlessly in multiple locations in real time, enabling us to retrieve documents if one region fails. For instance, when our traffic unexpectedly grew by 500% over just a few days, we were able to scale horizontally with zero lines of code change. We were also launching our Moj video app simultaneously, and we were able to move it to another region without a single issue. 

Simplifying development and deployment

On average, we experience about 80,000 requests per second (RPS) –nearly 7 billion RPS per day. That means daily push notifications sent out to the entire user base about daily trending topics can often result in a spike of 130,000 RPS in just a few seconds. 

Instead of over-provisioning, Google Kubernetes Engine (GKE) enables us to pre-scale for traffic spikes around scheduled events, such as holidays like Diwali, when millions of Indians send each other greetings. 

Migrating to GKE has also enabled us to adopt more agile ways of work, such as automating deployment and saving time with writing scripts. Even though we were already using container-based solutions, they lacked transparency and coverage across the entire deployment funnel. 

Kubernetes features, such as sidecar proxy, allows us to attach peripheral tasks like logging into the application without requiring us to make code changes. Kubernetes upgrades are managed by default, so we don’t have to worry about maintenance and stay focused on more valuable work. Clusters and nodes automatically upgrade to run the latest version, minimizing security risks and ensuring we always have access to the latest features.

Low latency and real-time ML predictions

Even though many of our users may be accessing ShareChat outside of metropolitan areas, it doesn’t mean they’re more patient if the app loads slowly or their messages are delayed. We strive to deliver a high-performance experience, regardless of where our users are. 

We use Cloud CDN to cache data in five Google Cloud Point of Presence (PoP) locations at the edge in India, allowing us to bring content as close as possible to people and speeding up load time. Since moving to Cloud CDN, our cache hit ratio has improved from 90% to 98.5%—meaning our cache can handle 98.5% of content requests. 

As we expand globally, we’d like to use machine learning to reach new people with content in different languages. We want to build new algorithms to process real-time datasets in regional languages and accurately predict what people want to see. Google Cloud gives us an infrastructure optimized to handle compute-intensive workloads that will be useful to us both now—and in the future.  

The confidence to build the best platform

Our current system now performs better than before we migrated, but we are continuously building new features on top of it. Google’s data cloud has provided us with an elegant ecosystem of services that allows us to build whatever we want, more easily and faster than ever before. 

Perhaps the biggest advantage of partnering with Google Cloud has been the connection we have with the engineers at Google. If we’re working to solve a specific problem statement and find a specific solution in a library or a piece of code, we have the ability to immediately connect with the team responsible for it. 

As a result, we have experienced a massive boost in our confidence. We know that we can build a really good system because we not only have a good process in place to solve problems—we have the right support behind us.

They say black never goes out of style. It’s something the team at Nuuly, URBN’s digital rental and resale business, know well. And it’s not just true of the company’s garments but their gadgets, too.

“I was having an offhand conversation with a UX designer recently,” Rebecca Sandercock, Nuuly’s strategy and insights manager, recalled in a recent interview from the company’s sunny, South Philly headquarters. “The designer was talking about how we had chosen dark mode for a number of interfaces internally because it actually saves so much on electrical output. They had the data to back that decision up, but more importantly, it’s just the kind of thing everyone is thinking about all the time here.”

Of course most every business is thinking about sustainability in some way these days. What makes Nuuly stand out is how quickly it can act, thanks in large part to the technology platform that’s made the entire enterprise possible.

“We’re kind of sustainable by our very nature,” Kim Gallagher, Nuuly’s director of marketing and customer success, said.

While she meant the rental and resale business, which helps customers buy fewer clothes and sees Nuuly items worn many times by many people—Gallagher could just as well have been referring to the sustainability inherent in, and enabled by, cloud computing.

There are the obvious, and oft-cited, advantages, such as how centralized data centers can operate more efficiently (some have been carbon neutral since the beginning). Yet there are even more subtle yet substantial benefits. In a marketplace and climate that are both changing faster and faster, sustainability requires a certain amount of agility. Such adaptability and scalability are intrinsic to the cloud technology that threads its way throughout Nuuly.

It turns out that being cloud native also means being sustainability native—as well as growth native. Since its 2019 launch, Nuuly’s net sales have risen roughly 6x over the first three fiscal years.

Cloud fits any situation

When URBN was developing Nuuly—launching in just 10 months—it chose to create everything from scratch on Google Cloud. Despite being part of a larger organization with decades of history and expertise, the company recognized the limitations presented by legacy systems and, more importantly, the necessity of building a wholly new platform that could be fully responsive.

The company has to react not only to new fashion trends but, crucially, the changing behaviors of customers. And not just their evolving tastes but also shopping habits, delivery preferences, unexpected customer service requests—is this a pattern, or a stain?—and social media chatter.

The pressure for a successful launch was high. The URBN portfolio, which also includes Urban Outfitters, Anthropologie, and Free People, had to keep evolving to satisfy a new generation of shopper who exists in an increasingly crowded and demanding digital marketplace.

The cloud’s responsiveness has thus proven its worth in creating a financially sustainable business as well as an environmentally sustainable one. Those even go hand in hand, as Gallagher points out: “Every customer who keeps renting is one who isn’t buying more occasion-specific clothes that go unworn most of the time.”

Dr. Alan Rosenwinkel, director of data science at URBN, had heard from his team about one garment that has become an emblem for the power of the platform. “It had been rented 25 different times before someone loved it enough to buy it and keep it forever,” he explained. (It’s also an emblem of the power of the cloud, that they would have the data awareness to track a single item so closely.)

It’s a new way of shopping made possible by a new way of computing. As one writer for Business Insider cheered, Nuuly “completely cured my addiction to fast-fashion.”

It makes for a healthy business, too. Sales for fiscal year 2019 exceeded $8 million and surpassed $24 million in 2020—one of the few URBN segments to grow during a tough year for fashion—and reached $47 million in 2021. The subscriber base had grown to 51,000 at the end of January.

Agility drives sustainability drives agility

For digital retailers to achieve such customer enthusiasm often relies as much on how the clothes get there as how they look. And those deliveries turn out to be a prime example of where Nuuly’s sustainability and technology meet.

For now, all shipping is handled through a state-of-the-art distribution center in Bucks County on the Philadelphia outskirts. Garments are shipped six-at-a-time, in fully reusable packaging, using ground transportation to keep the carbon footprint to a minimum. In certain limited geographic areas, shipments were sometimes taking longer than the two to five days most members would find acceptable.

“If we were an older company or weren’t set up from a technology perspective to be agile, we might have just said, ‘All right, we’re going to just go to three-day shipping for everyone,’” Rosenwinkel said. “That would drive up the cost, and the environmental impact. But we were able to be more strategic and more targeted about it.”

By regularly analyzing customer sentiment and retention data through BigQuery and Cloud Composer, and tying those to historical shipping times, Nuuly has been able to understand how shipping speed impacts its customers. Using a custom-built order management system, deliveries can be automatically adjusted to arrive more quickly, particularly when certain regions or days of the week are proving difficult to reach customers in time. These accelerated deliveries, like all Nuuly shipments, are made via UPS’s Carbon Offset program.

“Because we have the data, and the platforms to analyze it all,” Rosenwinkel said, “we’re delivering faster with the bare minimum impact on cost and emissions.”

It’s just one example of how modern retailers must juggle so many demands from consumers, workers, suppliers, and even regulators. Adding sustainability to that mix could be seen as a burden, but Nuuly shows how the right technology can lead to a holistic approach that makes all those interests work together even better.

And it allows for more opportunities and more kinds of sustainability, reaching from the designer’s atelier to the customer’s doorstep.

Sustainable details at every level

Back at the distribution center, workers can experience sustainability in a different way, as data is leveraged to enhance their well-being.

All workers are equipped with customized Android devices that help guide order tracking and fulfillment, plus cleaning, repairs, and reselling of garments as needs change throughout their lifecycle. Yet the insights go even deeper. The data science team has closely analyzed routes and repetitive motions for workers to keep their strenuous jobs as low-impact as the company’s broader environmental footprint.

“Through machine learning, we estimate we’ll be able to save our workers over 300,000 miles of steps over a five year period,” Rosenwinkel said. That’s enough walking to circumnavigate the globe 12.5 times.

The company has also applied ingenuity to one of the most notorious aspects of digital retail: packaging. Made from 100% post-consumer recycled materials like plastic bottles, Nuuly’s reusable carriers require no disposable bags or hangers to send goods back and forth. And once the packages have reached their end of life, the team is working with designers on ways to repurpose them into items that can then be offered for rental or sale on Nuuly.

“We’re really looking at the circular economy from every angle,” Gallagher said. “It’s built into the business.”

That includes not just the research the team did on-site at the Dry Cleaning & Laundry Institute—”We went to laundry school,” Gallagher jokes—but the digital tools that take those lessons even further. The team is always looking for ways to optimize fabric care, both to cut down on chemicals and water usage and extend the life of a garment. By analyzing the lifespan of every item, Nuuly not only makes them last longer but can identify problems faster. Employees even use custom apps to mark stains and damage so repair teams can more easily identify and fix the issues.

With its meticulous inventory tracking, Nuuly can even take marginal items, like a white gown or jeans with a small stain, and turn them into a custom dye job or an upcycling opportunity with a partner. These reworked items are then inserted back into the Nuuly Rent inventory as part of a growing collection of one-of-a-kind pieces, called Re_Nuuly.

Other services have launched quickly and easily thanks to the company’s cloud-enabled backend. Wanting to encourage more community and more reuse, the company created Nuuly Thrift. Debuting last fall after just a year in development, the team built everything from new interfaces to an evolved point of sale system.

It’s enabled Nuuly to offer many garments for rent or sale simultaneously, with “truly real-time inventories,” Rosenwinkel said. “So when it’s gone on one site, it shows up as gone on every site—no more surprises.”

Except for the good kind.

“I like to think we’re helping our customers think about ownership in a completely new way,” Sandercock said. “Once they run out of a use for a garment, they can offer it back, and sell it, and someone else will get to enjoy it and give it a new life. And Nuuly, we get to keep it in the community and keep it in the ecosystem, which is really cool—we’re taking extended responsibility over what happens to the clothing we create.”

Editor’s note: In this blog post we learn how TapClicks migrated to Google Cloud to offer their marketing customers a unified platform for data management, operations, insights, and analysis.

TapClicks is a smart marketing cloud, powered by data, that unifies our customer’s marketing. By choosing to migrate our core applications last year to Google Cloud, we cut costs, solved data-sharing concerns for our customers, and opened our stack up to a new ecosystem of possibilities. 

The core problem that we’re solving for our customers is how to manage their marketing infrastructures data and operations. Life isn’t easy for marketers now. There are 7,000 different vendors servicing this space today – creating much complexity between digital agencies, media, and brands. Marketers face challenges in navigating all of these systems, logging in and out, understanding pacing goals, and managing the flow of marketing data so they can analyze and report internally as well as to their clients at scale.

We unify omnichannel campaign data (250 API connectors and 6000 Smart Connectors ™ ) from a plethora of marketing sources on an automated data warehousing solution, creating simplicity for organizations. Over 4,000 agencies, media companies, and brands use our Marketing Operations and Data Management Platform, which imports data at scale and creates an automatic data warehouse on Google Cloud. Teams can also leverage TapClicks, like our world class Facebook connector, to import data directly into Google Data Studios.  Beyond importing and storing, we also provide data exporting to other Google solutions like Google Data Studio and Google Sheets.  We also create interactive dashboards that let stakeholders and clients analyze their data, as well as automated, multi-channel reports that go out to clients at specified times. So channel comparisons, optimizations, attribution, and calculations are easily performed.  Some of our customers are able to generate hundreds of thousands of individual reports and dashboards for their clients.

Although we may be best known for our reporting and analytics, we also empower teams managing the marketing operations workflow from customers and internal stakeholders, especially at scale. Our user-friendly, configurable system helps manage their orders and campaigns. Through automation of this process, we deliver tremendous amounts of efficiency, time saving, cost savings, and reduction of errors. The combination of these solutions makes up our unified platform, with additional capabilities like marketing intelligence that offers competitive and brand-level analysis. This is a disruptive solution in use by all leading media companies, agencies and many brands.

Partnering for possibilities

We faced a few challenges with our original tech stack, which included a mix of the leader in web services revenue, leaders in high performance data warehousing, as well as vendors on bare metal servers. 

• One challenge was around costs, which were growing. 
• Second, many of our customers work with multiple brands, and are very hesitant to share their data with the leader in web services, who’s often viewed as their competitor. 
• Third, these vendors are more focused on their own revenue rather than a true long term partnership that would enable their customers to enjoy similar success as they have experienced.

When looking at other cloud providers, Google Cloud emerged for us as the front runner. They were competitive on costs, and their native Kubernetes support was superior— a big selling point for our DevOps team. There’s also a movement in the marketing and advertising industry away from AWS toward Google Cloud because of the data-sharing concern. Finally, most of our customers are already using Google Cloud tools, so there’s brand recognition and familiarity there, and easier integrations with their own systems.

Migrating to Google Cloud

Our migration, which took about five months, involved moving a significant chunk of our infrastructure, including our core applications, using Google Kubernetes Engine (GKE). In our legacy architecture, each of our clients was assigned to one of our virtual machines (VMs), and there was a lot of unused capacity because we had to provision for the max usage. We appreciated GKE’s cloud native capabilities, especially autoscaling, a huge benefit for our web application. We have varying usage patterns during the day, and though our application is mostly used during business hours, there are also days in the month of higher usage, and autoscaling saves us time and costs. GKE also makes deployments much easier, and we anticipate a lot of benefits there for our developer environments. We’ve moved some of our microservices into GKE and plan to move more in the future. All in all, we were able to migrate our core products and the bulk of our AWS spend successfully to Google Cloud. 

We also moved from our other vendors Relational Database Service (RDS) to running MySQL on our own VMs on Google Cloud, which gives us more flexibility in terms of settings and fine tuning. We’re still trying to find the best mix as we’re modernizing our infrastructure, and we took this opportunity to migrate from MySQL 5.7 to 8.0.  

Our next stage is exploring more of the capabilities and services of Google Cloud, including BigQuery, which we’re considering for our own data warehouse. The fact that we could also run Snowflake on Google Cloud, if needed, was another selling point for our migration. 

We’re especially interested in BigQuery ML’s machine learning and natural language processing capabilities, which enabled better predictive insights. Our customers want insights from their campaigns— which are working, which are paying off, where should they invest next? Using our platform, they’re looking not only to generate reporting, but also identify opportunities to improve campaign performance. We plan to use AI and ML to improve those capabilities, so that our customers can seamlessly unlock insight and intelligence from their marketing data and campaigns.

Double-clicking on Google Cloud

For us, being able to deeply leverage and partner with Google Cloud to deliver those solutions on a single stack is critical, and we think our customers will love it. We see TapClicks and Google Cloud partnering at a level beyond what you typically see in a cloud provider relationship. Already, fifty percent of our company is working with various Google Cloud solutions, and we envision TapClicks and Google Cloud as extensions of each other, providing a single, powerful platform solution. 

Google Cloud understands the partnership concept, and their team was able to shine a light on their services and what they could bring to the table. Compared to our previous experiences, dealing with the Google Cloud team has been a true pleasure. Now that we’ve migrated, we’re ready to take our next steps into the services available to us in the Google Cloud ecosystem, and the problems we’ll continue to solve for our customers. Learn more about TapClicks and BigQuery ML.