Optimizing Cloud Load Balancing in Hybrid and Multicloud Architectures - Build What's Next
Blog

Optimizing Cloud Load Balancing in Hybrid and Multicloud Architectures

2769

Of your peers have already read this article.

3:30 Minutes

The most insightful time you'll spend today!

Learn how cloud load balancing can improve performance and availability in hybrid and multicloud environments. We cover key considerations for implementation and best practices in this blog post.

Today’s enterprise applications are often assembled across distributed environments. This includes the integration of services across multi-cloud, multi-SaaS and on-premises environments. While the approach has the advantage of enabling enterprises to choose the best service available to support their applications, it adds the complexity of delivering services across heterogeneous environments. To solve this, Cloud Load Balancing supports an open cloud strategy, which includes:

  • Supporting universal traffic management policies across heterogeneous environment by leveraging open source and open standards
  • Enabling a global front-end so applications can leverage a common set of policies and security postures
  • Providing tools that give your users the highest possible performance and reliability

Universal traffic management with open source and open standards

Kubernetes is a great solution for managing containers across environments. We believe that traffic management policies should also be supported across environments. Cloud Load Balancing creates homogeneous traffic policies across highly distributed heterogeneous environments by supporting standard-based traffic management in a fully managed solution, and allowing open source Envoy Proxy sidecars to be used on-premises or in a multi-cloud environment, using the same traffic management as our fully managed Cloud Load Balancers.

As enterprises start modernizing services and refactor monolithic applications, they require solutions that can provide consistent traffic management across distributed systems at scale. But organizations want to invest their time and resources innovating and building new applications — not on the infrastructure and networking required to deploy and manage these services. Envoy is an open-source high-performance proxy that runs alongside the application to deliver common platform-agnostic networking capabilities, including:

Hybrid Load Balancing across multi-cloud and private clouds

Over the years Google has deployed Load Balancers across 173+ Edge Pop locations, delivering customer applications at massive-scale on Google infrastructure. And now Google Cloud has introduced Hybrid Load Balancing, extending our Load Balancing capabilities beyond Google’s network to on-premises private clouds and multi-cloud solutions. This allows our customers to migrate applications to the cloud iteratively, or build hybrid applications that are assembled from services that are running across heterogeneous environments.

Supporting modern application delivery with HTTP3/QUIC

Cloud Load Balancing is a fully distributed load balancing solution that balances user traffic (HTTP(s), HTTPS/2, HTTPS/3 with gRPC, TCP/SSL, UDP, and QUIC) to multiple backends to avoid congestion, reduce latency, increase security, and reduce costs. It is built on the same frontend-serving infrastructure that powers Google services, supporting millions of queries per second with consistent high performance and low latency.

To serve massive amounts of traffic, Google built the first scaled-out software-defined load balancing, Maglev, which has been serving global traffic since 2008. It has sustained the rapid global growth of Google services, and it also provides network load balancing functions for Google Cloud Platform customers. To accommodate ever-increasing traffic, Maglev is specifically optimized for packet processing performance with Linux Kernel Offload. Maglev is also equipped with consistent hashing and connection tracking features, to minimize the negative impact of unforeseen faults and failures on connection-oriented protocols.

Another key enabler to support this global-scale is that our Cloud Load Balancers are built on top of QUIC(RFC9000), a protocol developed from the original Google QUIC) (gQUIC). HTTP/3 is supported between the External HTTP(S) Load Balancer, Cloud CDN, and end clients. And once enabled, customers typically see dramatic improvements in performance and throughput.

Google Cloud already supports HTTP3 in Cloud Load Balancer. To use HTTP/3 for your applications, you can enable it on your external HTTPS Load Balancers via the Google Cloud Console or the gCloud SDK with a single click.

If your service is sensitive to latency, QUIC will make it faster because it establishes connections with reduced handshakes. When a web client uses TCP and TLS, it requires two to three round trips with a server to establish a secure connection before the browser can send a request. With QUIC, if a client has connected with a given server before, it can start sending data without any round trips, so your web pages will load faster.

QUIC has advantages over legacy TCP as follows.

Summary

Since 2008, Google has been an innovator in software-defined networking, supporting applications running at massive scale. Google Cloud Load Balancers support HTTP3 and QUIC as a next generation web transport protocol, which significantly improves customer traffic latency. Google Load Balancers also have incorporated the Envoy proxy as a foundational technology, providing our customers with advanced traffic management that’s compatible with the open source Envoy ecosystem. This allows our users to have the choice to combine Google’s fully-managed Cloud Load Balancers with open source Envoy Proxies, to enable consistent traffic management capabilities across a multi-cloud distributed environment. And with Hybrid Load Balancing, customers can leverage our 173+ world wide PoPs to seamlessly manage traffic across Google Cloud, on-premises and other cloud providers.

Google Cloud Load Balancers include all these capabilities natively. And when used together, they support globally-scaled applications that run seamlessly across the heterogeneous environments many enterprises deploy today.

Case Study

Swiggy: Delivering Local Food Within 40 Minutes

8825

Of your peers have already read this article.

1:30 Minutes

The most insightful time you'll spend today!

Swiggy needed a scalable mapping platform that covered a wide geographic area and offered tools to help the company to build an efficient mobile app

Founded in 2014, Swiggy started small, delivering food to a few neighborhoods in Bengaluru, India. As the company grew, the team wanted a mapping technology that could help expand the service throughout India.

Swiggy needed a scalable mapping platform that covered a wide geographic area and offered tools to help the company to build an efficient mobile app and website for customers and delivery staff.

Customers find restaurants and order from them using the Android app, iOS app, or the website. Swiggy worked with Google Maps Partner Media Agility and used a variety of Google Maps Platform APIs to develop web and mobile apps that incorporate relevant local restaurant details.

Google Maps Platform Results

  • Built a hyper-local delivery service that is growing throughout India at a rate of 25 percent per month
  • Deliveries are made quickly, resulting in higher customer satisfaction and retention—users have been so satisfied that nearly 80 percent of its orders are from repeat customers
  • Drivers seamlessly handle tens of thousands of orders per day

In order to guarantee fast food delivery, Swiggy returns only restaurants within four to five kilometers of the customer’s location. The Directions API is used by drivers to easily route to restaurants and customers. The customer can track the progress of the delivery and estimated arrival time using a mobile app or the website.

“Google Maps provides the most accurate and reliable data, which is crucial for us because maps and location are central to our business. We also knew Google’s intuitive interface would provide a great customer experience with little to no learning curve… Google Maps’ ability to provide customer location and the distances of nearby restaurants is the backbone of our success, because it ensures a reliable, consistent customer experience,” said Aman Jain, Senior Product Manager, Swiggy.

Blog

How Kubernetes is enabling digital transformation for retailers

2642

Of your peers have already read this article.

3:30 Minutes

The most insightful time you'll spend today!

Kubernetes is revolutionizing the retail industry by enabling digital transformation through its powerful container orchestration capabilities. Discover how retailers are leveraging Kubernetes to modernize their operations and beat the competition.

Retail organizations constantly face financial pressure to increase sales while maintaining profit margins. Digital commerce creates new opportunities and a more competitive landscape for retailers by allowing them to reach a global customer base online, but it also exposes them to competition from larger online retailers. To be successful in this environment, retailers must not only have a strong online presence to keep up with their competition and gain market share, but also uplift the transactional customer experience to a more experiential one.

In today’s data-driven artificial intelligence-inspired business environment, organizations require complex IT infrastructure to support various functions such as prospecting, product development, marketing, and data analytics. Managing and scaling this infrastructure can be challenging, especially as needs evolve. As a result, many organizations are turning to Google Cloud as a solution for meeting their business goals, rather than simply expanding their in-house IT resources with more equipment and personnel. Specifically, retailers across the globe are betting on Kubernetes on Google Cloud to take advantage of secure, reliable, and scalable infrastructure.

Here are a few examples of worldwide retailers adapting to changing customer expectations using intelligent infrastructure solutions including Google Kubernetes Engine (GKE), the most scalable and automated fully managed Kubernetes from Google Cloud.

Haravan is a Vietnamese ecommerce platform that aims to improve the process of buying and selling products, allowing businesses to focus on creating and selling their products.

By using Google Cloud, Haravan helped small and medium-sized enterprises in Vietnam achieve double-digit growth, consistently met its 99.97% uptime commitment to clients, efficiently managed 5 times the normal amount of ecommerce activity, and facilitated the implementation of artificial intelligence-powered expansion plans.

“We have a guaranteed commitment to enable any volume of sales for clients over all channels, be it social networks, marketplaces, livestream, or website. Only Google Cloud, with the flexible autoscaling of GKE, gives us certainty to meet our guarantees even in the most massive Black Friday surges.” —Hung Le, VP of Software Engineering, Haravan

Loblaw is Canada’s food and pharmacy leader and the nation’s largest retailer. The company operates over 2,500 locations, including corporate, franchised, and associate-owned stores, and employs nearly 200,000 full- and part-time employees.

By using Google Cloud, boosted the performance of the online grocery platform, resulting in higher conversion rates and increased revenue, recovered up to 50% of Site Reliability Engineers’ time for innovation, introduced new, real-time personalization features and shopping conveniences for customers, and enhanced resiliency to protect customers and revenue.

“Moving our online grocery site to Google Cloud gave us a 4x performance increase and the capacity to handle up to three times the traffic; and we can scale up at any time.” —Hesham Fahmy, VP Technology, Loblaw

L.L.Bean is a North American retail company known for its boots and mail-order catalog, which dates back to 1912. The company has a strong online presence, with ecommerce accounting for $1 billion of its annual revenues of $1.6 billion. Like many other retailers, L.L.Bean is adopting an omnichannel sales strategy by interacting with customers through various channels including print, physical stores, its website, app, and social media.

By using Google Cloud, L.L.Bean enhanced customers’ online experience through faster page load times and access to transaction history, allowed for a focus on providing value to customers rather than managing infrastructure, and enabled the rapid release of cross-channel services by reducing development cycles.

“GKE has significantly streamlined the process of upgrading nodes and masters. By comparison, upgrading even minor releases of another container solution that L.L.Bean tested resulted in the need to rebuild that solution’s clusters four times.” —Randy Dyer, Enterprise Architect, L.L.Bean

LPP is a Polish fashion retailer established in 1991 by Lubianiec and Piechocki, whose initials make up the company’s name. LPP currently manages five clothing brands that are popular in 38 countries across Europe, Africa, and Asia, and has over 24,000 employees based in its main offices in Central and Eastern Europe. Growing demand for its ecommerce services led LPP to migrate from an on-premises setup to Google Cloud, harnessing automation to ensure great shopping experiences globally.

By using Google Cloud, LPP promoted a DevOps culture among developers through streamlined deployment of new features, provided 90% more time for engineers to work on innovative solutions instead of managing infrastructure, instantly created and updated new VMs, allowing developers to quickly launch new features, ensured a seamless online shopping experience by automatically adjusting capacity to meet demand.

“GKE enables us to deploy new features for our ecommerce sites very quickly. Previously, it took weeks to launch new instances for each brand. Today, it takes seconds: we simply launch a new machine, deploy the code, and changes are reflected automatically across our environment.” —Marek Maciejewski, Head of IT Service Operations, LPP

Noon.com, based in Riyadh, Saudi Arabia, is a local ecommerce marketplace focused on serving the Middle East. The company aims to become the top online retailer in the region, supporting the growth of a digital economy for both consumers and local businesses.

By using Google Cloud, Noon.com built its ecommerce platform to access self-managed services, allowed developers to establish a fully operational staging environment within two weeks, provided uninterrupted service to nearly four times as many daily users during busy seasons using autoscaling on GKE, used real-time data streaming on BigQuery to inform business decisions and personalize the customer experience, and achieved 99.999% availability with no downtime for planned maintenance or schema changes using a fully managed relational database.

“Google Cloud-managed services are playing a major role in enabling Noon.com customers to get their shopping done whenever they need it, without experiencing any delays or glitches, and without us having to lose sleep at night to ensure our platform is functioning as it should.” —Alex Nadalin, SVP of Engineering, Noon.com

In conclusion, the retail industry is constantly evolving and retailers must stay up-to-date with the latest technology and customer preferences to remain competitive. Digital commerce has changed the landscape of retail, allowing businesses to reach a global customer base but also increasing competition. The use of Kubernetes on Google Cloud can help retailers improve the customer experience, streamline internal processes, and make data-driven and AI-inspired decisions. By embracing these changes, retailers can stay ahead in a constantly evolving industry. Get started today with an exclusive workshop, Unlocking efficiency and innovation with Kubernetes on Google Cloud.

Case Study

Wunderkind Leverages Google Cloud to Address the Growing Needs of its Customer Base

5351

Of your peers have already read this article.

1:30 Minutes

The most insightful time you'll spend today!

Performance marketing channel, Wunderkind after having issues with legacy database deployed Google Cloud solutions and Cloud Bigtable for flexibility and scalability to meet the growing needs of their data use cases.

Editor’s note: We’re hearing here how martech provider Wunderkind easily met the scaling demands of their growing customer base on multiple use cases with Cloud Bigtable and other Google Cloud data solutions.

Wunderkind is a performance marketing channel and we mostly have two kinds of customers: online retailers, and publishers like Gizmodo Media Group, Reader’s Digest, The New York Post and more. We help  retailers boost their e-commerce revenue through real-time messaging solutions designed for email, SMS, onsite, and advertising. Brands want to provide a one-to-one experience to more of their customers, and we use our extensive history with best practices in email marketing and technology to help brands reach more customers through targeted messaging and personalized shopping experiences. With  publishers, it’s a different value proposition, we use the same platform to provide a non disruptive and personalized ad experience for their website. For example, if you are on their site and then you left, we might show an ad tailored to you when you come back later – depending on the campaign. 

After running into limitations with our legacy database system, we turned to Cloud Bigtable and Google Cloud, which helped us be more flexible and easily scale for high traffic demand – which can be a stable 40,000 requests per second, and meet the needs of our growing number of data use cases. 

Three different databases power our core product

In our core offering, companies send us user events from their websites. We store these events and later decide (using our secret sauce) if and how to reach out to those users on behalf of our customers. Because many of our customers are retailers, Black Friday and Cyber Monday are big traffic days for us as. On such days, we can get 31 billion events, sometimes as many as  200K events per second. We show 1.6 billion impressions that have seen close to 1 billion pageviews. And at the end of all this, we securely send about 100 million emails. We noticed the same thing for election time; traffic reached the same high volume. We need scalable solutions to support this level of traffic as well as the elasticity to let us pay only for what we use, and that’s where Google Cloud comes in.

So how does this work? Our externally facing APIs, which are running on Google Kubernetes Engine, receive those user events—up to hundreds of thousand per second. All the components in our architecture need to be able to handle this demand. So from our APIs, those events go to Pub/SubDataflow and from there they are written to Bigtable and BigQuery, Google Cloud’s serverless, and highly scalable data warehouse. This business user activity data underpins almost all our products. Events can be things like product views or additions to shopping carts. When we store this data in Bigtable, we use a combination of email address and the customer ID as the Bigtable key and we record the event details in that record. 

What do we do with this information next? It’s important to mention that we also mark the last time we received an event about a user in Memorystore for Redis, Google Cloud’s fully managed Redis service. This is important because we have another service that is periodically checking Memorystore for users that have not been active for a campaign-specific period of time (it can be 30 minutes, for example), then deciding whether to reach out to them.

How we decide when we reach out is an intelligent part of our product offering, based on the channel, message, product, etc. When we do reach out, we use Memorystore for Redis as a rate limiter or token bucket. In order not to overwhelm the email or texting providers we send API requests to, we throttle those requests using Memorystore. (We prefer to preemptively throttle the outgoing API requests as opposed to handling errors later.)

When we do reach out, often we will need details for a specific product—let’s say if the website belongs to a retailer. We usually get that information from the retailer through various channels and we store product information in Cloud SQL for MySQL. We pull that information when we need to send an email with product information, and we use Memorystore for Redis to cache that information, since many of the products are repeatedly called. Our Cloud SQL instance has 16 vCPUs, 60GBs of memory and 0.5TB of disk space and when we perform those product information updates, we have about a thousand write transactions per second. We are also in the process of migrating some tables from a self managed MySQL instance, and we keep those tables synchronized with Cloud SQL using Datastream. 

Our user history database was originally stored in AWS DynamoDB, but we were running into problems with how they structured the data, and we’d often get hot shards but with no way to determine how or why. That led to our decision to migrate to Bigtable. We set up the migration first by writing the data to two locations from Pub/Sub, performed some backfill of data until that was up and running, and then started working on the reading. We performed this over a few short months, then switched everything to Bigtable. 

So, as mentioned, we are using Bigtable for multiple databases. The instance that stores our user events has about 30 TB with about 50 nodes.

Profile management

A second use case for Bigtable is for user profile management, where we track, for example, user attributes based on subscription activity, whether they’ve opted in or out of various lists, and where we apply list-specific rules that determine which targeted emails we send out to users. 

Our very own URL shortener

Our third use case for Bigtable is our URL shortener. When our customers build out campaigns and choose a URL, we append tracking information to the query string of the URLs and they become long. Many times, we are sending them via SMS texts, so the URLs need to be short. We originally used an external solution, but made the determination that they couldn’t support our future demands. Our calls tend to be very bursty in nature, and we needed to plan for a future state of supporting higher throughput. We use a separate table in Bigtable for this shortened URL. We generate the short slug that is 62 bit-encoded and use it as the rowkey. We use the long slug as a Protobuf-encoded data structure in one of the row cells and we also have a cell for counting how many times it was used. We use Bigtable’s atomic increment to increase that counter to track how many times the short slug was used. 

When the user receives a text message on their phone, they click the short URL, which goes through to us, and we expand it to the long slug (from Bigtable) and redirect them to the appropriate site location. Obviously, for the URL shortener use case, we need to make the conversion very quickly. Bigtable’s low latency helps us meet that demand and we can scale it up to meet higher throughput demands.

Meeting the future with Google Cloud

Our business has grown considerably, and as we keep signing up new clients, we need to scale up accordingly, and Bigtable has met our scaling demands easily. With Bigtable and other Google Cloud products powering our data architecture, we’ve met the demand of incredibly high traffic days in the last year, including Black Friday and Cyber Monday. Traffic for these events went much higher than expected, and Bigtable was there, helping us easily scale on demand. 

We are working on leveraging a more cloud native approach and using Google Cloud managed services like GKE, Dataflow, pub/sub, Cloud SQL , Memorystore, BigQuery and more. Google has those 1st party products and we don’t see the value in rolling out or self managing such solutions ourselves..

Thanks to Google Cloud, we now have reliable and flexible data solutions that will help us meet the needs of our growing customer base, and delight their users with fast, responsive, personalized shopping messaging and experiences. 

Learn more about Wunderkind and Cloud Bigtable. Or check out our recent blog exploring the differences between Bigtable and BigQuery.

3296

Of your peers have already watched this video.

24:30 Minutes

The most insightful time you'll spend today!

Case Study

Modernizing 100-Year Old Retail Chain with Anthos

H-E-B, like many enterprises, was moving away from legacy mainframes in favor of microservices and public cloud infrastructure. With hundreds of applications powering their 100+ year-old business, H-E-B needed to be confident that the platform they are building will provide them the agility and security to continue to innovate for their customers.

See how the H-E-B engineering team started breaking down their Curbside and Home Delivery monoliths into microservices, why they chose to make Kubernetes, and why they’re leveraging Anthos as a hybrid cloud platform.

4405

Of your peers have already watched this video.

32:00 Minutes

The most insightful time you'll spend today!

Webinar

On-Demand Webinar: How APIs Help Walgreens Merge Physical and Digital Retail

APIs are how modern businesses rapidly expand into new contexts—making it possible for companies like Walgreens to transform from brick-and-mortar businesses into omnichannel organizations that serve customers in innovative ways.

Headquartered in Deerfield, Illinois, Walgreens is the second-largest pharmacy store chain in the United States. It specializes in filling prescriptions, health and wellness products, health information, and photo services. The company has more than 8,000 stores and operates in 50 states, the District of Columbia, Puerto Rico, and the US Virgin Islands.

Walgreens has built a thriving API program that lets software developers plug into its retail and pharmacy business. As a result, the company now fills one prescription per second via mobile devices.

More than 100 photography apps offer Walgreens photo printing and same-day pickup at 8,000+ locations. Perhaps best of all, Walgreens has found that users who use its mobile app spend more than those who don’t.

Watch Erin Neus-Cheong from Walgreens and Alicia Paterson from Google Cloud explore why treating APIs as products—not projects—can help companies tap into the value of APIs as business accelerators and open up new channels of opportunity.

More Relevant Stories for Your Company

Case Study

Google Cloud Partnership Helps Lowe’s SRE Team Achieve 20X More Releases Per Month

Editor’s note: Today we hear from the Lowe’s SRE team. They share about how they have been able to increase the number of releases they can support by adopting Google’s Site Reliability Engineering (SRE) framework and leveraging their partnership with Google Cloud.  At Lowe’s, we’ve made significant progress in our multiyear technology transformation.

Case Study

Scaling with Breaking News: BBC’s Serverless Infrastructure on Google Cloud

Editors note: Today's post is from Neil Craig at the British Broadcasting Corporation (BBC), the national broadcaster of the United Kingdom. Neil is part of the BBC’s Digital Distribution team which is responsible for building the services such as the public-facing www bbc.co.uk and .com websites and ensuring they are

Blog

Building Unique Customer Experiences with Speed & Scale: Sprinklr & Google Cloud

Enterprises are increasingly seeking out technologies that help them create unique experiences for customers with speed and at scale. At the same time, customers want flexibility when deciding where to manage their enterprise data, particularly when it comes to business-critical applications. That’s why I’m thrilled that Sprinklr, the unified customer

How-to

A Guide to Anthos: the App Modernization Platform for Hybrid and Multi-cloud Deployments

Anthos gives you a consistent platform for all your application deployments, both legacy as well as cloud native, while offering a service-centric view of all your environments. You can build enterprise-grade containerized applications faster with managed Kubernetes on cloud and on-premises environments. Create a fast, scalable software delivery pipeline with

SHOW MORE STORIES