Modern applications need to process large-scale data at millisecond latency to provide experiences like instant gaming leaderboards, fast analysis of streaming data from millions of IoT sensors, or real-time threat detection of malicious websites. In-memory datastores are a critical component to deliver the scale, performance, and availability required by these modern applications. 

Memorystore makes it easy for developers building applications on Google Cloud to leverage the speed and powerful capabilities of the most loved in-memory store: Redis. Memorystore for Redis standard tier instances are a popular choice for applications requiring a highly available Redis instance. Standard tier provides a failover replica across zones for redundancy and provides fast failover with a 99.9% SLA. However, in some cases, your applications will require more read throughput from a standard tier instance. One of the common patterns customers use to scale read queries in Redis is leveraging read replicas.

Introducing Memorystore for Redis Read Replicas

Today we are excited to announce the public preview of Memorystore for Redis Read Replicas, which allows you to seamlessly scale your application’s read requests by 6X with the click of a button.

With read replicas, you can easily add up to five replicas and leverage the read endpoint to automatically load balance read queries across all the available replicas, increasing read performance linearly with each replica added. Additionally, Memorystore’s support for Redis 6 introduced multi-thread I/O, increasing performance significantly for M3 and higher configurations. Combined, you can achieve read requests of more than a million requests per second. 

You will benefit from this new functionality in several ways. You will be able to scale on demand with up to five read replicas and use read endpoint with any redis client to easily load balance read queries across multiple replicas. This new functionality will also improve availability with automatic distribution of replicas across multiple zones. With read replicas, you will also be able to minimize application downtime with fast failover to the replica with the least replication lag. In the future, Memorystore will also easily enable read replicas on existing standard tier instances to increase read throughput. 

You can learn more about how to configure and use read replicas in the Read Replicas Overview. 

Improving performance with Read Replicas and Redis 6

With the launch of read replicas, you can easily increase the read throughput of a Memorystore instance. You can further enhance the read performance by leveraging Redis version 6 along with read replicas.

To understand why combining read replicas and Redis 6 can significantly improve your application’s read performance, let’s look at the various Memorystore configurations that you can use with your applications today. Memorystore Basic and Standard offerings  provide different capacity tiers. The capacity tier determines the single node performance of a basic and standard tier instance. 

The table below outlines the configuration of the various capacity tiers:

Up until version 5, Redis processed commands using a single thread. The processing involved reading the request, parsing the request, processing the request, and writing the response back to the socket. This approach means that all of the processing, which includes writing the response, was sequentially processed by a single vCPU regardless of the number of vCPUs available in the instance.

Redis 6 introduced I\O threading which allows writing the response using parallel threads. This functionality enables Redis 6 to more effectively leverage available vCPUs, thereby increasing the overall throughput compared to lower versions. Memorystore for Redis version 6 leverages I\O threading and automatically configures the optimal number of I\O threads to achieve the best possible performance for the various capacity tiers. We have also improved the overall network throughput for all redis versions by leveraging improvements in the Google Cloud infrastructure. Together these improvements deliver significant performance improvements and come at no additional cost to you. 

So what can you expect from using Redis 6? As outlined in the table, Redis version 5 and lower uses a single thread to process the write requests for all tiers while Redis 6 uses a larger number of I\O threads at higher capacity tiers which provides substantially incremental throughput for higher capacity tiers.

For example, using a Redis 6 standard tier instance with a capacity of 101 GB (M5), we have observed up to a 200% improvement in read/write performance compared to version 5, though the actual value you’ll see is dependent on your workload. You can get the benefits of Redis 6 by upgrading an existing instance or by deploying a new instance.

By enabling read replicas on an instance using Redis 6, you can further improve the read performance. Read throughput scales linearly with the number of replicas so you can increase your read queries on an instance by up to 500% using five read replicas. By combining Redis 6 and read replicas, you can see a 10X increase in read performance compared to a version 5 standard tier instance with no read replicas.

We are excited about the launch of read replicas but this is just one step in our journey to deliver the scale you need at the best price-performance. You can learn more about read replicas pricing on our Memorystore pricing page and get started using the Read Replicas Overview.

Paper currency — which started gaining prominence in the 1600s — changed the face of global economics and ushered in a new era of international monetary regulation. The primary reason currency created such disruption was its ability to standardize the “medium of exchange”. APIs created a similar effect in the world of technology and digitalization by creating a standardized, reusable, and secure way to exchange information.

Modern web APIs took shape in the early 2000s and played a key role in “.com”mercializing every business. APIs started as a connective tissue primarily relegated to a technical context and quickly evolved into a gateway to new business models, revenue streams, and ecosystems. In 2017, McKinsey estimated a total of $1 Trillion in profit could be up for grabs in the API economy. And in 2022, GGV Capital created an index of API-first startups — a generation of stylistically divergent SaaS companies with leaner operating cost structures and organic usage growth. Just as currency is going through an evolution from banknotes to digital wallets, the world of API management is also on the brink of change.

With more than 15 years of experience managing APIs at Google-scale, we’ve got a unique vantage point from which to observe that change. In today’s post, we will spotlight seven API management use cases that we see growing in prominence — and how you can take advantage of these trends to future-proof your architecture.

1 “Shift left” in API security

As a gateway to a wealth of information, APIs have also quickly become the primary attack vector in security incidents. When we surveyed 500 technology leaders, we learned that more than 50% of organizations experienced an API security incident in the last 12 months. Adding to the increasing magnitude of attacks, there are an increasing number of vectors for potential API security incidents like misconfigurations, outdated APIs/data/components, and bots/spam/abuse.

These security issues aren’t just in production APIs, but at every stage in the API lifecycle. Notably, we found that 67% of the issues are discovered during testing as part of the release management process. This trend ushers in the need for forward-thinking organizations to “shift left with security” — moving controls earlier into the production workflow — by bringing security teams and API teams closer. To stay ahead of security threats, many organizations are actively looking for solutions that allow them to be proactive while minimizing the burden on their security teams. According to our research, integrating capabilities that proactively identify security threats (60%) is top of mind for most IT leaders for the next year.

2 “Knowledge graph” for your enterprise APIs

It comes as no surprise that every organization is relying on APIs to expand and even ground their digital ecosystem — a network of partners, developers, and customers facilitated by modern, cloud-first technologies. There is a growing magnitude and variety of middleware assets, contributing to the growth of IT complexity.

As the number of APIs continues to increase, there is a need to simplify consumption for internal and external developers. Even the most objectively useful APIs remain unseen by most of the organization. In turn this results in redundant code, reduced developer productivity, or worse — turned into a potential security attack vector. This complexity is shifting focus towards consolidation of all middleware assets, growing adoption, and improving education (see below) to improve developer efficiency and de-mystify the IT complexity.

This sprawl is a growing problem in the world of APIs, but it has a lot in common with an age-old phenomenon in the world of web pages and content—search. Google was born out of this problem to help organize the world’s information. Similar to Google’s knowledge graph for web pages, there is a need to index, organize, and instantly present API information for developers that need it. Although it is an emerging practice, we see an increasing number of digital leaders and security teams in larger organizations with mature API programs invest in solutions that help consolidate all APIs, organize their information, and manage their lifecycle.

3 The imminent need for “omni” control planes

APIs have taken on such a vital role in the modern application stack that they have slowly become the neural links across the entire enterprise architecture — bridging legacy and modern applications, shifting architectures towards microservices, and enabling operations across heterogeneous environments. To support all these technological decisions without sacrificing speed, organizations adopted multiple API gateways and fragmented API management solutions. However, this led to a lack of universal visibility, consistent governance, comprehensive security, and meaningful analytics across ALL the enterprise APIs (not just the ones within the confines of a given API management solution). And it increases the maintenance costs — fundamentally undercutting the value of APIs. With this evolution there is a growing need for an omni control plane — analogous to the brain in a human body — across all enterprise APIs.

4 API governance rising through the priority ranks

Despite the clear need for governance, there is still no unified understanding on a good (or right) approach to API governance. With the rapid adoption of APIs without appropriate standardization or quality standards, API governance is top of mind for IT leaders, again.

According to our research, 45% of IT leaders identified API governance as a critical component of their API program. The top three components of API security, performance analytics, and governance demonstrate the critical need for visibility, quality, and security across all APIs.

As digital consumers, we have seen this phenomenon across many industries and digital products. For example AirBnB disrupted the short-term rental market by providing standardized listings, detailed information, and high-resolution photos. In fact, the same governance phenomenon is ubiquitous in the world of e-commerce where there is a clear correlation between a high-quality website or product listing and increasing sales.

The same analogy holds true in the world of APIs, as ~90% of developers use APIs in their work there is a direct correlation between the use of APIs and developer productivity. Digital officers and CIOs need to add appropriate governance controls to standardize API design and improve reuse without adding friction to development timelines.

5 Evolution of design patterns with multiple API gateways

Adoption of new API architectural styles and microservices increased the complexity of the modern application stack. Our research found that 54% of organizations use a service mesh and API management in conjunction today to support the API gateway design pattern. In parallel, there is broad adoption of new protocols like GraphQL or AsyncAPI, outpacing the innovation in API gateways. For example, in a recent survey from DZone found that GraphQL accounted for 22.7% of application integrations.

In response to this challenge IT teams are adopting multiple API gateways — by design — which is creating the need for complex communication patterns for future scalability. But the existing design patterns were mostly sufficient when client applications used homogeneous API protocols (Ex: REST). Although patterns like Backend For FrontEnd (BFF) intended to provide specific API interactions that are relevant on a per-client basis, they still did not account for complexities from multiple gateways and protocols. In response to the adoption of new protocols, there is a need to evolve the existing BFF pattern to account for multiple API gateways and protocols.

6 Driving green value chains with digital twins

A digital twin is an effectively indistinguishable virtual representation of a physical object, system, or a process. For example the digital twin of a wind turbine (the object being studied) might be used to capture data like performance, rpm (revolutions per minute), or output captured by various sensors outfitted on the turbine. Digital twin adoption is growing and McKinsey estimates investments in digital twins will reach $58 billion by 2026 with a 58% CAGR. Every digital twin uses APIs to monitor, engage, and possibly control the physical asset. For example, Google created the Digital Buildings project — an open source, Apache-licensed effort to manage applications and analyses between a large heterogeneous portfolio of buildings.

Sustainability is one of the driving forces behind the increased use of digital twins. As the need to reach net zero emissions accelerates, many organizations are tying performance (and in some cases even executive pay) to environmental, social, and governance goals. APIs help connect the dots between digital twins and sustainability. For example, an organization operating a manufacturing process could build a digital twin with APIs to collect behavioral data from sensors, monitoring systems, or other sources — which can eventually be integrated into the organization’s digital platform or applications. These digital twins could be used to analyze and optimize the use of materials and energy, to minimize waste and emissions. Additionally, digital twins could be used to monitor and analyze the performance of systems over time, to identify opportunities for continuous improvement.

Overall, APIs play a valuable role in supporting sustainability efforts by enabling digital twins, effectively driving more efficient operation of systems, and providing insights to improve environmental impact. For further examples, check out this video about driving a green value chain with APIs.

7 Commercializing access to data products

The growing use of data-rich services (like IoT, ML models, remote access services, and web scraping, etc.) coupled with massive ingestion of data everyday is creating massive growth in data delivery paradigms like data lakehouses, data marketplaces, and data streaming systems (global data marketplaces alone are poised to reach $3.5 billion by 2028). Unfortunately, most of these systems are fragmented with almost no relationship or interoperability.

APIs are filling this critical gap for organizations in two critical ways. First, APIs are providing standard and easy access to systems like data lakehouses or analytics hubs. Second, APIs are a key enabler of data products (digital products or services built using data as a core value proposition), a core component of any data sharing system. APIs provide a standardized way for different applications to interact with the data product. For example, an API could be used to allow a mobile app to access data from a weather forecast or a recommendation engine data product. Beyond data products, APIs also provide easy and standardized access various data management platform 

APIs continue to play a critical role in every application, experience, and ecosystem. Robust API strategies help organizations adapt to any architecture, business model, or environment in the face of changing technology landscape. Learn more about how Apigee is driving innovation and helping companies future proof their architectures to stay ahead of the top API trends.