Kitabisa is shaping the future of fundraising with Google Cloud
3114
Of your peers have already read this article.
3:30 Minutes
The most insightful time you'll spend today!
The name Kitabisa means “we can” in Bahasa Indonesia, the official language of Indonesia, and captures our aspirational ethos as Indonesia’s most popular fundraising platform. Since 2013, Kitabisa has been collecting donations in times of crisis and natural disasters to help millions in need. Pursuing our mission of “channeling kindness at scale,” we deploy AI algorithms to foster Southeast Asia’s philanthropic spirit with simplicity and transparency.
Unlike e-commerce platforms that can predict spikes in demand, such as during Black Friday, Kitabisa’s mission of raising funds when disasters like earthquakes strike is by definition unpredictable. This is why the ability to scale up and down seamlessly is critical to our social enterprise.
In 2020, Indonesia’s COVID-19 outbreak coincided with Ramadan. Even in normal times, this is a peak period, as the holy month inspires charitable activity. But during the pandemic, the crush of donations pushed our system beyond the breaking point. Our platform went down for a few minutes just as Indonesia’s giving spirit was at its height, creating frustrations for users.
A new cloud beginning
That’s when we realized we needed to embark on a new cloud journey, moving from our monolithic system to one based on microservices. This would enable us to scale up for surges in demand, but also scale down when a wave of giving subsides. We also needed a more flexible database that would allow us to ingest and process the vast amounts of data that flood into our system in times of crisis.
These requirements led us to re-architect our entire platform on Google Cloud. Guided by a proactive Google Cloud team, we migrated to Google Kubernetes Engine (GKE) for our overall containerized computing infrastructure, and from Amazon RDS to Cloud SQL for MySQL and PostgreSQL, for our managed database services.
The result surpassed our expectations. During the following year’s Ramadan season, we gained a 50% boost in computing resources to easily handle escalating crowdfunding demands on our system. This was thanks to both the seamless scaling of GKE and recommendations from the Google Cloud Partnership team on deploying and optimizing Cloud SQL instances with ProxySQL to optimize our managed database instances.
A progressive journey to kindness at scale
While Kitabisa’s mission has never wavered, our journey to optimized performance took us through several stages before we ultimately landed on our current architecture on Google Cloud.
Origins on a monolithic provider
Kitabisa was initially hosted on DigitalOcean, which only allowed us to run monolithic applications based on virtual machines (VMs) and a stateful managed database. This meant manually adding one VM at a time, which led to challenges in scaling up VMs and core memory when a disaster triggered a spike in donations.
Conversely, when a fundraising cycle was complete, we could not scale down automatically from the high specs of manually provisioned VMs, which was a strain on manpower and budgetary resources.
Transition to containers
To improve scalability, Kitabisa migrated from DigitalOcean to Amazon Web Services (AWS), where we hoped deploying load balancers would provide sufficient automated scaling to meet our network needs. However, we still found manual configurations to be too costly and labor-intensive.
We then attempted to improve automation by switching to a microservices-based architecture. But on Amazon Elastic Container Service (Amazon ECS) we hit a new pain point: when launching applications, we needed to ensure that they were compatible with CloudFormation in deployment, which reduced the flexibility of our solution building due to vendor locking.
We decided it was “never too late” to migrate to Kubernetes, which is a more agile containerized solution. Given that we were already using AWS, it seemed natural to move our microservices to Amazon Elastics Kubernetes Service (Amazon EKS). But we soon found that provisioning Kubernetes clusters with EKS was still a manual process that required a lot of configuration work for every deployment.
Unlocking automated scalability
At the height of the COVID-19 crisis, faced with mounting demands on our system, we decided it was time to give Google Kubernetes Engine (GKE) a try. Since Kubernetes is a Google-designed solution, it seemed likeliest that GKE would provide the most flexible microservices deployment, alongside better access to new features.
Through a direct comparison with AWS, we discovered that everything from provisioning Kubernetes clusters to deploying new applications became fully automated, with the latest upgrades and minimal manual setups. By switching to GKE, we can now absorb any unexpected surge in donations, and add new services without expanding the size of our engineering team. The transformative value of GKE became apparent when severe flooding hit Sumatra in November 2021, affecting 25,000 people. Our system easily handled the 30% spike in donations.
Moving to Cloud SQL and ProxySQL
Kitabisa was also held back by its monolithic database system, which was prone to crashing under heavy demand. We started to solve the problem by moving from a stateful DigitalOcean database to a stateless Redis one, which freed us from relying on a single server, giving us better agility and scale.
But the strategy left a major pain point because it still required us to self-manage databases. In addition, we were experiencing high database egress costs due to the need to execute data transfers from a non-Google Cloud database into BigQuery.
In December 2021, we migrated our Amazon RDS to Cloud SQL for MySQL, and immediately saved 10% in egress costs per month. But one of the greatest benefits came when the Google Cloud team recommended using the open source proxy for MySQL to improve the scalability and stability of our data pipelines.
Cloud SQL’s compatibility allowed us to use connection pooling tools such as ProxySQL to better load balance our application. Historically, creating a direct connection to a monolithic database was a single point of failure that could end up in a crash. With Cloud SQL plus ProxySQL, we create layers in front of our database instances. It serves as a load balancer that allows us to connect simultaneously to multiple database instances, by creating a primary and a read replica instance. Now, whenever we have a read query, we redirect the query to our read replica instance instead of the primary instance.
This configuration has transformed the stability of our database environment because we can have multiple database instances running at the same time, with the load distributed across all instances. Since switching to Cloud SQL as our managed database, and using ProxySQL, we have experienced zero downtime on our fundraising platform even when a major crisis hits.
We are also saving costs. Rather than having a separate database for each different Kubernetes cluster, we’ve merged multiple database instances into one instance. We now group databases according to business units instead of per service, yielding database cost reductions of 30%.
Streamlining with Terraform deployment
There’s another key way in which Google Cloud managed services have allowed us to optimize our environment: using Terraform as an infrastructure-as-a-code tool to create new applications and upgrades to our platform.
We also managed to automate the deployment of Terraform code into Google Cloud with the help of Cloud Build, and no human intervention. That means our development team can focus on creative tasks, while Cloud Build deploys a continuous stream of new features to Kitabisa.
The combination of seamless scalability, resilient data pipelines, and creative freedom is enabling us to drive the future of our platform, expanding our mission to inspire people to create a kinder world in other Asian regions.
We believe that having Google Cloud as our infrastructure backbone will be a critical part of our future development, which will include adding exciting new insurtech features. Now firmly established on Google Cloud, we can go further in shaping the future of fundraising to overcome turbulent times.
Are You Providing Sufficient Digital Leadership?
4972
Of your peers have already read this article.
5:00 Minutes
The most insightful time you'll spend today!
It has been seven years since Marc Andreesen’s famous article “Why Software is Eating the World” was published in the Wall Street Journal, and given the slow pace of change in many companies, some executives still may not be taking the threat of being “eaten” seriously enough.
Software, and platform business models based on software, have the potential to deliver powerful economic forces into virtually any company or industry. Every company has valuable assets—such as data, expertise or access to certain services or user bases—and most of these assets can be delivered via software. Once an asset is expressed as software in a modern way—that is, as an application programming interface (API)—it can be combined with other software to create new applications and digital experiences.
Benefits of this approach, just to name a few, include near-zero marginal cost to scale up APIs for new users or use cases; global reach for both partners using APIs and end users consuming the digital experiences those APIs power; and network effects triggered as more partners use a given company’s digital assets and spread its services into new markets and use cases.
Disruption by software-powered business models
In the last two decades we’ve seen individual companies and entire industries upended by these kinds of software-powered business models. Examples abound: Amazon and the retail industry, Netflix and movie rentals, Uber and ride hailing, Airbnb and hotels, etc. We’ve reached the point that these companies’ names have become verbs synonymous with being “eaten” by software (e.g., “Amazoned” or “Netflixed”).
The most famous examples of digital disruption involve digital natives, of course, but legacy businesses are leveraging software to evolve too. Brazilian retailer Magazine Luiza—a company I’ve worked with through my employer, Google Cloud’s Apigee team—has enjoyed enormous revenue growth and seen its stock soar, for example, as it has built out its digital platform capabilities and transitioned from a primarily brick-and-mortar model to an omnichannel one. The point is, whether a company has been in business five decades, five years or five months, software remains ravenous and is always looking for new companies and industries to “eat.”
Change in the face of serious threats
Facing this threat, how should corporate leadership respond? There are some excellent examples of CEOs who have galvanized their companies and led them through the massive, gut-wrenching change required to pivot in the face of a serious threat. A few of the biggest examples include:
- In 1995, it became apparent to Microsoft co-founder and then-CEO Bill Gates that the internet was “the most important single development to come along since the IBM PC,” and, if not embraced in haste, a threat to many of Microsoft’s businesses. In May of that year he published the “The Internet Tidal Wave” memo and focused all of Microsoft on adopting and building for the internet. Almost 20 years later, current Microsoft CEO Satya Nadella similarly made the bold decision to redirect the company for a cloud-first world.
- Facebook went public at $38 per share in May of 2012 but within months, stocks could be had for a little over half that. The concern? Facebook was a desktop-optimized website without a polished mobile presence, and by 2012, consumer attention had begun to accelerate towards mobile at a much higher rate than many initially predicted. Facebook CEO Mark Zuckerberg reacted by not only proclaiming Facebook a mobile-first company, but also backing up that proclamation with action.
- Turning to another company I’ve worked with via Apigee, T-Mobile launched its highly visible “Uncarrier” campaign—which offers streamlined, customer-friendly plans and services—while also investing in and executing a new IT vision dedicated to ongoing digital transformation. T-Mobile execs have credited the technology effort, spearheaded by CEO John Legere, with helping the company to introduce new services and better service customers. T-Mobile’s market cap has more than doubled since Legere took over in 2012.
Keeping pace with changing customer needs
In the face of an existential threat, strong executive leadership is required to pivot the company to safety, as these examples attest. Digital transformationisn’t about deploying new technologies just to make an existing approach more efficient or to add a few new apps or features to the status quo; digital transformation is about keeping pace with changing customer needs by leveraging software platforms to continuously evolve how the business operates. This can be akin to turning an enormous ship—and a ship can’t turn very well without her captain, first mate, and other leaders showing the way.
Research supports this. A recent Deloitte survey, for example, found that over “80 percent of respondents from digitally maturing organizations say their leaders have sufficient knowledge and ability to lead the company’s digital strategy,” compared to only “22 percent of early-stage business respondents [who] have the same belief.” Similarly, Gartner research finds that CEOs are seeking a “deeper understanding of digital business” as they shift their focus from growth in general to how technology helps them attain it.
More recently, the onslaught of software devouring the world has been further accelerated by machine learning making everything smarter, voice interfaces changing how people interact with devices, and more. To keep pace, corporate leaders need to galvanize their companies to build and deploy software faster, make systems and data easily accessible inside and outside their companies, and improve digital experiences through not only machine learning but also constant data-driven iteration.
Seven years after Andreesen’s editorial, the pace of digital disruption is still increasing, and so is the need for strong leadership to pivot fully into digital. Over half of the Fortune 500 has been acquired, merged or declared bankruptcy since 2000—and the companies that survive in coming years won’t be those whose leaders treat technology as an IT concern rather than a core part of the business.
5683
Of your peers have already downloaded this article
5:30 Minutes
The most insightful time you'll spend today!
Modernizing apps on the cloud isn’t an “all or nothing” decision. Businesses want the option to modernize on-premises or choose multi-cloud solutions that meet their needs. That’s why we created a new solution for running apps anywhere – simply, flexibly, and securely. Embracing open standards, Anthos lets you run your applications, unmodified, on existing on-prem hardware investments or in the public cloud. So that you write once and deploy anywhere.
Download this report and find out how to:
- Decouple infrastructure and applications with containers and Kubernetes
- Decouple cloud teams from one another so they can work independently
- Meet the challenges of microservice management using service mesh
- Implement a zero-trust security model to enforce more granular controls while maintaining a consistent user experience
3302
Of your peers have already watched this video.
27:00 Minutes
The most insightful time you'll spend today!
Unburden Your Operations and Development Teams with Anthos
VMs are used to run the majority of services in enterprises. Whether due to specific technical requirements, developer preferences, or simply time and budget constraints, many applications are not moving to containers and Kubernetes yet.
Think services first
Microservices architectures present numerous benefits but also introduce challenges like added complexity and fragmentation for different workloads. The Anthos platform unburdens your operations and development teams by simplifying service delivery across the board, from traffic management and mesh telemetry to securing communications between services. Anthos Service Mesh, Google’s fully managed service mesh, lets you easily manage these complex environments and enjoy the benefits they promise.
Learn how to incorporate these existing VM workloads into Anthos Service Mesh. Also, see how to use Compute Engine to simplify the installation and management of the Envoy proxy and how to add VMs to the mesh. Lastly, explore how you can modernize your VMs in place or develop a strategy to migrate your VMs to containers. Regardless of the path chosen, we show how the VMs can participate in the mesh and get all the same benefits of security, policy, and telemetry that Anthos Service Mesh provides to Kubernetes workloads.
5038
Of your peers have already watched this video.
21:00 Minutes
The most insightful time you'll spend today!
Uber’s Story of Scaling Their App with Millions of Concurrent Requests
Uber has millions of concurrent customers who use the platform to book rides and place food delivery orders, generating billions of database transactions per day. In just a click of a button, Uber captures users’ intent which bases their fulfillment model to meet the customers’ demand, and match it to active providers that address their demand with supply.
Uber’s fulfillment platform capability is the lifeline powering every line of business and also allows rapid scaling of new verticals. Hundreds of microservices at Uber depend on this fulfillment platform as a source of truth for all their active booking, delivery or order session. The events generated by this platform are used by hundreds of offline data sets to power business decisions. Additionally over hundreds of developers at Uber extend the platform with APIs, events and codes to build 120+ unique fulfillment flows!
Watch the video to learn about Uber’s decision to move from on-prem to Google Cloud’s Cloud Spanner while still taking live orders at scale and meeting customers’ expectations.
4573
Of your peers have already watched this video.
1:30 Minutes
The most insightful time you'll spend today!
Introduction to Cloud Shell Editor
Watch the video to understand how Google’s Cloud Shell Editor and its powerful features-packed environment can streamline your development workflows.
More Relevant Stories for Your Company
Three Reasons Why Enterprises Must Think Next-gen Serverless
As we reflect on the past year, Heraclitus' phrase “The only constant in life is change” has never rang more true. With the pandemic, companies had to shift operations, launch new products and adapt to extreme demand patterns, sometimes within a matter of weeks. To respond to customer needs faster
How Toyota’s Google Cloud-powered Voice Assistant Gives a Turboboost to Drivers’ Experience
Over the decades, technology has helped us organize large amounts of physical information in ways that are streamlined, efficient, and easily accessible. Rows upon rows of encyclopedias are no longer needed; simply punch in or speak a query into Google Search and find numerous results at your fingertips. There’s no
AI Features in Apigee X Helps Build and Manage APIs at Scale
APIs are the backbone of digital transformation. Via APIs, you can securely share data and functionality with developers both inside and outside of your organizational boundaries, letting you build applications faster, seamlessly connect and interact with partners, and drive new business revenue. Because APIs encompass business-critical information, any downtime or
Speeding up migrations to Google Cloud with migVisor by EPAM
Application modernization is quickly becoming one of the pillars of successful digital transformation and cloud migration initiatives. Many organizations are becoming aware of the dramatic benefits that can be achieved by moving legacy, on-premises apps and databases into cloud native infrastructure and services, such as reduced Total Cost of Ownership
