Budget-Friendly Log Management: Four Steps to Cost Optimization in Google Cloud

1218
Of your peers have already read this article.
4:30 Minutes
The most insightful time you'll spend today!
As part of our ongoing series on cost management for observability data in Google Cloud, we’re going to share four steps for getting the most out of your logs while on a budget. While we’ll focus on optimizing your costs within Google Cloud, we’ve found that this works with customers with infrastructure and logs on prem and in other clouds as well.
Step 1: Analyze your current spending on logging tools
To get started, create an itemized list of what volume of data is going where and what it costs. We’ll start with the billing report and the obvious line items including those under Operations Tools/Cloud Logging:
- Log Volume – the cost to write log data to disk once (see our previous blog post for an explanation)
- Log Storage Volume – the cost to retain logs for more than 30 days
If you’re using tools outside Cloud Logging, you’ll also need to include any costs related to these solutions. Here’s a list to get you started:
- Log vendor and hardware costs — what are you paying to observability vendors? If you’re running your own logging solution, you’ll want to include the cost of compute and disk.
- If you export logs within Google Cloud, include Cloud Storage and BigQuery costs
- Processing costs — consider the costs for Kafka, Pub/Sub or Dataflow to process logs. Network egress charges may apply if you’re moving logs outside Google Cloud.
- Engineering resources dedicated to managing your logging tools across your enterprise often are significant too!
Step 2: Eliminate waste — don’t pay for logs you don’t need
While not all costs scale directly with volume, optimizing your log volume is often the best way to reduce spend. Even if you are using a vendor with a contract that locks you into a fixed price for a period of time, you may still have costs in your pipeline that can be reduced by avoiding wasteful logs such as Kafka, Pub/Sub or Dataflow costs.
Finding chatty logs in Google Cloud
The easiest way to understand which sources are generating the highest volume of logs within Google Cloud is to start with our pre-built dashboards in Cloud Monitoring. To access the available dashboards:
- Go to Monitoring -> Dashboards
- Select “Sample Library” -> “Logging”
This blog post has some specific recommendations for optimizing logs for GKE and GCE using prebuilt dashboards.
As a second option, you can use Metrics Explorer and system metrics to analyze the volume of logs. For example, type “log bytes ingested” into the filter. This specific metric corresponds to the Cloud Logging “Log Volume” charge. There are many ways to filter this data. To get a big picture, we often start with grouping by both “resource_type” and “project_id”.
To narrow down the resource type in a particular project, add a “project_id” filter. Select “sum” under the Advanced Options -> Click on Aligner and select “sum”. Sort by volume to see the resources with the highest log volume.

While these rich metrics are great for understanding volumes, you’ll probably want to eventually look at the logs to see whether they’re critical to your observability strategy. In Logs Explorer, the log fields on the left side help you understand volumes and filter logs from a resource type.

Reducing log volume with the Logs Router
Now that we understand what types of logs are expensive, we can use the Log Router and our sink definitions to reduce these volumes. Your strategy will depend on your observability goals, but here are some general tools we’ve found to work well.
The most obvious way to reduce your log volume is not to send the same logs to multiple storage destinations. One common example of this is when a central security team uses an aggregated log sink to centralize their audit logs but individual projects still ingest these logs. Instead, use exclusion filters on the _Default log sink and any other log sinks in each project to avoid these logs. Exclusion filters also work on log sinks to BigQuery, Pub/Sub, or Cloud Storage.

Similarly, if you’re paying to store logs in an external log management tool, you don’t have to save these same logs to Cloud Logging. We recommend keeping a small set of system logs from GCP services such as GKE in Cloud Logging in case you need assistance from GCP support but what you store is up to you, and you can still export them to the destination of your choice!
Another powerful tool to reduce log volume is to sample a percentage of chatty logs. This can be particularly useful with 2XX log balancer logs, for example. This can be a powerful tool, but we recommend you design a sampling strategy based on your usage, security and compliance requirements and document it clearly.
Step 3: Optimize costs over the lifecycle of your logs
Another option to reduce costs is to avoid storing logs for more time than you need them. Cloud Logging charges based on the monthly log volume retained per month. There’s no need to switch between hot and cold storage in Cloud Logging; doubling the default amount of retention only increases the cost by 2%. You can change your custom log retention at any time.
If you are storing your logs outside of Cloud Logging, it is a good idea to compare the cost to retain logs and make a decision.
Step 4: Setup alerts to avoid surprise bills
Once you are confident that the volume of logs being routed through log sinks fit in your budget, set up alerts so that you can detect any spikes before you get a large bill. To alert based on the volume of logs ingested into Cloud Logging:
- Go to the Logs-based metrics page. Scroll down to the bottom of the page and click the three dots on “billing/bytes_ingested” under System-defined metrics.
- Click “ Create alert from metric”
- Add filters (For example: use resource_id or project_id. This is optional).
- Select the logs based metric for the alert policy.
You can also set up similar alerts on the volume for log sinks to Pub/Sub, BigQuery or Cloud Storage.
Conclusion
One final way to stretch your observability budget is to use more Cloud Operations. We’re always working to bring our customers the most value possible for their budget such as our latest feature, Log Analytics, which adds querying capabilities but also makes the same data available for analytics, reducing the need for data silos. Many small customers can operate entirely on our free tier. Larger customers have expressed their appreciation for the scalable Log Router functionality available at no extra charge that would otherwise require an expensive event store to process data. So it’s no surprise that a 2022 IDC report showed that more than half of respondents surveyed stated that managing and monitoring tools from public cloud platforms provide more value compared to third-party tools. Get started with Cloud Logging and Monitoring today.
Google Cloud Celebrates Journey of 3 Inspiring Founders for the Asian Pacific American Heritage Month

8165
Of your peers have already read this article.
1:30 Minutes
The most insightful time you'll spend today!
May is Asian Pacific American Heritage Month —a time for us to come together to celebrate and remember the important people and history of Asian and Pacific Island heritage. This feature highlights three AAPI founders from the Google For Startups community.
Read on to learn how these three founders built their businesses, leverage Google tech, and suggestion they have for aspiring entrepreneurs.

CultivatePeople
Founder: Lola Han
Description: CultivatePeople’s compensation software, Kamsa, provides global market pay rate data and helps companies make data-driven salary decisions so they can attract and retain their most valuable asset: employees.
Why GCP: “CultivatePeople began using GCP when integrating SSO/SAML authentication after our SaaS product became a high priority. We were able to exceed our clients’ expectations and increase their level of trust in our platform’s capabilities. We’re currently investigating additional machine learning products, such as Cloud AutoML, that will allow us to quickly deliver exciting features.”
Note from the Founder: “I grew up with immigrant parents who value stability and are risk-averse. My parents discouraged me from starting my own company because they didn’t want to see me struggle financially or see my health suffer (due to stress). I felt strongly about what CultivatePeople could do, so I started the company as a sole founder in 2017 and watched it double in size year over year since. While the ones I love most may not have cheered me on initially, it was important for me to hang on to the encouraging words of former bosses, executives, and founders to keep me focused on my mission.
My advice for other AAPI founders is to be a “silent assassin” and believe in the mission and values of your organization. Always remember to stop along the way and:
1) Enjoy the journey by celebrating wins and giving yourself credit;
2) Follow your intuition—it’s (almost) always right;
3) Recognize and invest in your people regularly (ie. give increases more than once a year, if warranted);
4) Give regular words of affirmation to employees on even small achievements.”

Swit
Founder: Josh Lee and Max Lim
Description: Swit is a team collaboration platform that seamlessly combines team chat with task management by allowing teams to turn their conversations into trackable tasks and share tasks to chat with simple drag-and-drop functionality, ensuring everyone is on the same page and projects get done faster.
Why GCP: “Swit is a cross-category hybrid work tool for chat and tasks. This functionality requires more complicated and heavier architecture for performance. So, configuring and managing virtual machines was really challenging to scale up our systems, while handling occasional unexpected traffic surges and frequent updates. Eventually we divided our monolithic architecture into 35 microservices when we launched our official product. The migration to GKE took around one month, and it turned out to be well worth the effort—our systems became able to offer high scalability and enough resilience to keep its uptime no matter what happened. Now we’re operating 84 workloads and 252 microservices with high stability with remarkably low downtime – less than 0.00001%/year.”
Note from the Founder: “As an AAPI founder based in Silicon Valley, I feel proud of the work ethics and diligence fellow Korean American entrepreneurs and professionals have long demonstrated here. Especially with K-pop breaking into the mainstream, I feel even more proud of our culture that strives toward an absolute perfection molded through years of training and dedication. The mission-driven culture of Silicon Valley coupled with Google’s edging technology and creativity really helped us build a product that not only encompasses verticals but also transcends cultures. Swit is growing at an unprecedented rate, and we hope to join the long list of successful AAPI entrepreneurs here. Swit’s close network with the AAPI community wouldn’t have been possible without Google support. We are grateful for this collaborative environment, and we hope to become the next-generation ambassador for collaboration after Google.”
Check out more from Swit in their founder story.

WISY
Founder: Min Chen
Description: Wisy develops technology to bring digital efficiency into the physical world, supporting consumer products businesses and making them thrive in the new economy. All of us have a bad experience when we can’t find the product we want to buy. That is a $1.9T problem in the consumer-packaged goods industry that Wisy is solving with AI and analytics to help manufacturers and retailers sell more by reducing out-of-stocks and waste at a global scale.
Why GCP: “GCP has an intuitive, easy to use interface, was lower cost, and offered preemptible instances with flexible compute options. Some of the reasons why Wisy decided to use GCP include instance and payment configurability, privacy and traffic security, cost-efficiency, and Machine Learning.
Wisy has been able to advance quickly with product development, as well as collaborate better and iterate faster in the creation of our AI models, while reducing costs by 40%. At Wisy, we are solving a problem that affects everyone who shops at a store.“
Note from the Founder: “Two years ago, I moved to San Francisco to expand my second startup, Wisy. This is when I learned that my name ‘Min’ stands for ‘minority.’ I was born in China, raised in a Black community in Panama, received scholarships to attend both Carnegie Mellon and UC Berkeley. I worked for 20 years in several countries, but I have never felt so discriminated against due to my race, ethnicity, gender and age than during my time in Silicon Valley. However, this is also the place I learned that my diverse life experience is my competitive advantage. My background enables me to recruit and relate to people in different countries, create scalable and flexible products for multinational customers, and run global operations efficiently.
My recommendation to AAPI founders is to find strength in their multicultural background. Don’t hide what makes you unique, do not limit yourselves, and do not let others limit you. You will lose your edge when trading authenticity for validation. Be proud and own your story.”
If you want to learn more about how Google Cloud can help your startup, visit our Startup Program application page here and sign up for our monthly startup newsletter to get a peek at our community activities, digital events, special offers, and more.To learn more about how you can help #StopAsianHate during Asian Pacific American Heritage Month and beyond, visit their website here.
How APIs Helped PWC Open New Revenue Streams Using Existing Data

4696
Of your peers have already read this article.
5.17 Minutes
The most insightful time you'll spend today!
PwC, one of the “Big Four” accounting firms, is well-known for professional services structured around auditing, insurance, tax, legal, and traditional management consulting. In Australia, the PwC Innovation and Ventures group has taken the global lead in building new, technology-based, turnkey lines of business outside of PwC’s traditional service areas. Applying its insights and knowledge base to the company’s vast amounts of existing data, PwC has uncovered new revenue models, distinct from its traditional, labor-intensive services.
Traditionally, people at PwC connected to critical data in response to scheduled tasks or crises in order to provide independent advice, often after the fact, when there’s little runway to make considered business decisions. The company wanted to move beyond the status quo, where people connected to static data and where benchmarking, deeper insights, and alerts were often an afterthought. Expertise gained from analyzing data and drawing valuable insights often was limited to individuals—it didn’t scale. PwC aimed to leap forward technologically and build utility and value for its customers through the development of a vibrant API-based ecosystem.
Innovation From Down Under
Australia is helping to lead the way at PwC from a software and development perspective. Early on, the Innovation and Ventures group decided to collaborate with PwC New Zealand, which leads the world in cloud general ledger adoption. The group represents the first with over 20% of its customer companies keeping their general ledgers in the cloud (that figure is currently around 35%), and serves as an early example of what can be achieved with APIs based on cloud general ledger data.
Accessing proprietary data (most significantly general ledger data), transforming it, and connecting it to an ecosystem of partners and clients via APIs, has quickly proved a winning formula for PwC, in the form of its Next platform, which combines multiple cloud accounting tools and integrated cloud applications in an open platform. It also includes customizable dashboards that provide a holistic view of a client’s entire portfolio, including business trends, in real time.
“By our very nature, we’re a people and services business, evolving into a data business. The biggest help that Apigee has provided in this transformation is in helping us expose core, rich data so that our people who provide services today can actually demonstrate value in the market tomorrow.”
— Trent Lund, PwC Australia
In developing the firm’s first technology products, PwC Australia’s Head of Innovation and Ventures Trent Lund was adamant that as an accounting firm, PwC never spend a dollar building something that technology professionals had already done better. That credo led Lund to select Google’s Apigee as PwC’s platform of choice for developing productized APIs.
Cloud-first Strategy
Increasingly, the datasets PwC wants to connect with are from public sources and open APIs coming from cloud providers. The Apigee toolset is perfectly positioned for Lund’s team’s focus on data connectivity—especially now that Apigee is part of Google, Lund says.
“The Apigee integration into Google is really helpful for us because we can connect in with that same ecosystem. Our team is relatively small by global standards, so we really don’t have time to try and foster multiple technology relationships,” Lund says. “We need to have deep, trusted relationships where we can get a level of sharing now and into the future, and that’s what we get with Apigee.”
PwC Australia continues to innovate and build the platform, but rather than continuing to pay for development from its local innovation fund, the platform is now funded globally so that it can be accelerated and shared across four countries. Australia, New Zealand, the United Kingdom, and the United States are simultaneously guiding the platform’s requirements and features as PwC develops with an awareness of each country’s individual regulations.
For example, compliance with the European Union’s General Data Protection Regulation (GDPR) and China’s data residency rules would be far more complicated without a single technology partner like Apigee that experiences the same global challenges as part of Google, Lund says.
A CIO’s Guide to the Cloud: Hybrid and Human Solutions to Avoid Trade-offs

3872
Of your peers have already read this article.
2:30 Minutes
The most insightful time you'll spend today!
What do CIOs and CTOs deliver for the company? If you said “technology,” that’s just the beginning. According to their research, McKinsey found that 85% of CIOs and CTOs interviewed in the spring of 2019 said they were essential for at least two of the three most common CEO priorities—revenue acceleration, improved agility and time to market, and cost reduction.
IT modernization – including migrating to the cloud – is key to business growth and agility. Yet, according to a recent McKinsey study, 80% of CIOs report that regardless of their level of cloud migration, they still haven’t reached their projected agility and business benefits. Sometimes, this is because of issues like training and skills gaps in the IT workforce. Surprisingly often though, the barrier to reaching the goals is based on trade-offs that CIOs themselves feel they must make to strike a balance between the perfect and the possible.
But what if you could have it all without the trade-offs? As Will Grannis, Managing Director of the CTO Office at Google, and Arul Elumalai, Partner at McKinsey & Company discussed in our recent digital conference, many of the compromises CIOs make can be avoided with new technology, modern architectures and by encouraging a transformation mindset across the business. In interviews, CIOs explained how they’ve leveraged the best of the cloud without compromising on security, agility, and flexibility. Here’s how these leaders avoid three of the top perceived trade-offs—both with technology and by transforming their operating model.
Trade-off #1: Developer agility vs. control and governance
Moving to the cloud offers new opportunities for speed, but 69% of organizations indicate that stringent security guidelines and code review processes can slow developers significantly. One CISO of a multinational company mentioned that cloud development was so fast that they had to institute manual checks on their developers’ code. So much for agility.
To overcome this trade-off and maintain both speed and security, some respondents found success in DevOps, hiring security-experienced talent and introducing automation for security and quality. Building in security into the CI/CD pipeline and increasing automation don’t just eliminate the tradeoff, they result in higher quality and faster innovation.
At Google Cloud, we’ve also observed that customers with strong DevOps practices have increased speed-to-market and product/service quality. From our own journey, we’ve learned seven critical lessons essential to adopting a DevOps model, ranging from taking up small projects and embracing open source to building an overall DevOps culture.
Trade-off #2: Single-vendor benefits vs. freedom from lock-in
CIOs perceive benefits to using the fewest number of clouds, specifically avoiding introducing multiple systems that require their teams to develop and maintain multiple skillsets. Unfortunately, 83% of the CIOs interviewed said that while they would prefer fewer clouds, the potential financial and technical lock-in drives them to multiple providers.
Successful CIOs said that they can avoid lock-in pitfalls not just with contractual guardrails and executive and board education, but with evolving hybrid cloud technologies that provide additional choices. Hybrid cloud platforms based on containers can further mitigate the risk of using a single cloud vendor. The key to successful hybrid architectures is the infrastructure abstraction and portability that containers create for them, enabling disparate environments to work together.
This notion has been at the heart of our strategy at Google Cloud with Anthos, which provides an abstraction layer and an application modernization platform for hybrid and multi-cloud environments. Enterprises can use Anthos to modernize how they develop, secure, and operate hybrid-cloud environments and enable consistency across cloud environments.
Trade-off #3: Best-of-breed tools vs. standardization and familiarity
Optimizing tool chains for different environments can improve productivity, but many CIOs believe that this means reduced functionality and tools. While 77% of CIOs said they had to standardize to the lowest common denominator, some have found a better solution. Rather than giving up the languages, libraries, and frameworks that their teams prefer, effective leaders said that they found success by investing in training programs to upscale talent and adopting new open and vendor-agnostic solutions. Architectures that are based on open-source components have been the keys that helped remove this tradeoff, and eliminate the notion of a lowest common denominator.
This is why we have built Anthos on open-source components like Kubernetes, Istio and Knative. Anthos gives your business the choice you need. With the ability to create code that works in most environments using the tools, languages, and systems you prefer, you can do more without major changes to how you work.
Regardless of your current cloud adoption level, check out “Unlock business acceleration in a hybrid cloud world” to discover more about McKinsey’s findings, including how CIOs drive agility, methods to make trade-offs unnecessary, and how to prepare your team for the cloud. Then, stay tuned for subsequent posts that take a closer look at how hybrid solutions and strategies can help CIOs drive a transformation mindset across the business—without compromising on security, agility, and flexibility.

Le Figaro Uses Google Firebase to Personalize Experiences and Generates 3X Revenue Results
DOWNLOAD CASE STUDY8314
Of your peers have already downloaded this article
1:45 Minutes
The most insightful time you'll spend today!
Le Figaro, established in 1826, is France’s oldest and largest daily morning newspaper. The company’s mission is to provide timely, digestible and engaging news to their readers. As one of the first in the industry to offer digital content, Le Figaro engages their subscribers across 11 Android, iOS and web apps that cover news, sports, lifestyle and games. Le Figaro has about 22M monthly active users on their mobile and web apps and 120K paid digital subscribers.
The Challenge
In a saturated news app market, Le Figaro was looking to increase paying customers and to retain existing paid subscribers. To do this, Le Figaro’s development team needed to engage readers with personalized content at the right price point, but how could they pull it off with limited time and resources?
The Solution
Le Figaro used a number of Firebase products to retain existing users and increase paid subscriptions. They sent targeted notifications through Firebase Cloud Messaging reminding customers to follow topics and journalists they found interesting. This helped reduce churn by keeping subscribers engaged in content they valued. They also tested different subscription amounts using Firebase A/B testing, which helped Le Figaro identify the price points that led to the highest number of conversions among both Android and iOS users.
“Using Firebase has completely transformed Le Figaro’s digital business by making it easy to rapidly innovate and personalize content for our readers. With Firebase we have seen continuous increases in retention, downloads and screen time in our apps!”
Valentin Paquot, Mobile CTO, Le Figaro
Le Figaro found their biggest increase in paid subscriptions came from embedding real time interactive infographics into their mobile and web app articles. When a user added information into the infographic, it triggered a Cloud Function that accessed data stored in Cloud Firestore and returned a personalized infographic to the user in real time.
For example, in the article “Are you rich?” readers could input their income into the infographic and compare it against different income groups in Paris instantaneously. The infographics was behind a paywall and users had to subscribe to gain access.
According to Le Figaro, this infographic saw 3X the rate of paid subscription sign-ups compared to their other infographics. The team built this interactive infographic system in 3 days instead of their average time of 2-3 weeks using a traditional backend service. Using Cloud Functions and Cloud Firestore, they estimate they were able to reduce development time by 86%.
6 Common Errors to Sidestep in RESTful API Design

3206
Of your peers have already read this article.
2:30 Minutes
The most insightful time you'll spend today!
Imagine ordering a “ready-to-assemble” table online, only to find that the delivery package did not include the assembly instructions. You know what the end product looks like, but have little to no clue how to start assembling the individual pieces to get there. A poorly designed API tends to create a similar experience for a consumer developer. Well designed APIs make it easy for consumer developers to find, explore, access, and use them. In some cases, good quality APIs even spark new ideas and open up new use cases for consumer developers.
There are methods to improve API design — like following RESTful practices. But time and again we are seeing customers unknowingly program minor inconveniences into their APIs. To help you avoid these pitfalls, here are six of the most common mistakes we have seen developers make while creating the API — and guidance on how to get it right.
#1 Thinking inside-out vs outside-in
Being everything for everybody often means that nothing you do is the best it could be, and that is just as true for APIs. When customers turn to APIs, they are looking for specific solutions to make their work easier and more productive. If there is an API that better works to their needs, they will choose that one over yours. This is why it’s so important to know what your customers need to do their work better, and then building to fill those needs. In other words, start thinking Outside-in as opposed to Inside-Out. Specifically,
- Inside-out refers to designing APIs around internal systems or services you would like to expose.
- Outside-in refers to designing APIs around customer experiences you want to create. Read more about the Outside-in perspective in the API product mindset.
The first step to this is learning from your customers — be it internal consumer developers or external customers — and their use cases. Ask them about the apps they are building, their pain points, and what would help streamline or simplify their development. Write down their most significant use cases and create a sample API response that only gives them the exact data they need for each case. As you test this, look for overlap between payloads and adapt your designs to genericize them across common or similar use cases.

If you can’t connect with your customers — because you don’t have direct access, they don’t have time, or they just don’t know what they want — the best approach is to imagine what you would build with your APIs. Think big and think creatively. While you don’t want to design your APIs for vaporware, thinking about the big picture can make it easier to build non-breaking changes in the future. For example the image below showcases APIs offered by Google Maps. Even without diving into the documentation, looking at the names like “Autocomplete” or “Address Validation” clearly outlines the purposes and potential fit for a customer’s use case.

#2 Making your APIs too complex for users
Customers turn to APIs to bypass complicated programming challenges so they can get to the part they know how to do well. If they feel like using your API means learning a whole new system or language, then it isn’t fitting their needs and they will likely look for something else. It’s up to your team to make an API that is strong and smart enough to do what your customer wants, but also simple enough to hide how complicated the tasks your API solves for really are. For example if you know your customers are using your APIs to present information about recently open restaurants and highly rated pizzeria to their consumers, providing them with a simple API call as below would be of great help:
GET /restaurants?location=Austin&category=Pizzeria&open=true&sort=-priority,created_atTo see if your API design is simple enough, pretend you are building the whole system from scratch — or if you have a trusted customer who is willing to help, ask them to test it and report their results. If you can complete the workflow without having to stop to figure something out, then you’re good to go. On the other hand, if you catch rough edges caused by trying to code around system complexity issues, then keep trying to refactor. The API will be ready when you can say that nothing is confusing and that it either meets your customers’ needs or can easily be updated as needs change.
#3 Creating “chatty” APIs with too many calls
Multiple network calls slow down the process and creates higher connection overhead — which means higher operational costs. This is why it’s so important to minimize the number of API calls.
The key to this is outside-in design: simplify. Look for ways to reduce the number of API calls a customer must make in their application’s workflow. If your customers are building mobile applications, for example, they often need to minimize their network traffic to reduce battery drain, and requiring a couple calls instead of a dozen can make a big difference.
Rather than deciding between building distinct, data-driven microservices and streamlining API usage, consider offering both: fine-grained APIs for specific data types, and “experience APIs” (APIs that are designed to power user experiences. Here is a further theoretical discussion on Experience APIs) around common or customer-specific user interfaces. These experience APIs compose multiple smaller domains into a single endpoint; making it much simpler for your customers — especially those building user interfaces — to render their screens easily and quickly.
Another option here is to use something like GraphQL to allow for this type of customizability. Generally you should avoid building a unique endpoint for every possible screen, but common screens like home pages and user account information can make a world of difference to your API consumers.
#4 Not allowing for flexibility
Even if you’ve followed all of the steps above, you may find that there are edge cases that do not fit under your beautifully designed payloads. Maybe your customer needs more data in a single page of results than usual, or the payload has way more data than their app requires. You can’t create a one-size-fits-all solution, but you also don’t want a reputation for building APIs that are limiting. Here are 3 simple options to make your endpoints more flexible.
Filter out response properties: You can either use query parameters for sorting and pagination, or use GraphQL which provides these types of details natively. By giving customers the option to request only the properties they need, it guarantees that they won’t have to sort through tons of unnecessary data to get what they need. For example, if some of your customers only need the title, author, and bestseller ranking, give them the ability to retrieve only that data with a query string parameter.
GET /books?fields=title,author,ranking- Ability to sort with pagination. Generally, you don’t want to guarantee the order of objects in an API response because minor changes in logic or peculiarities in your data source might change the sort order at some point. In some cases, however, your customers may want to sort by a particular field. Giving them that option, combined with a pagination option, will give them a highly efficient API when they only want the top few results. For example Spotify API utilizes a simple offset and limit parameter set to allow pagination. A sample endpoint as shown in the documentation would look like this
$ curl https://api.spotify.com/v1/artists/1vCWHaC5f2uS3yhpwWbIA6/albums?album_type=SINGLE&offset=20&limit=10- Use mature compositions like GraphQL: Since customer data needs can differ, giving them on-the-fly composites lets them build to the combinations of data they need, rather than being restricted to a single data type or a pre-set combination of data fields. Using GraphQL can even bypass the need to build experience APIs, but when this isn’t an option, you can use query string parameter options like “expand” to create these more complex queries. Here is a sample response that demonstrates a collection of company resources with embedded properties included
"data": [
{
"CompanyUid": "27e9cf71-fca4",
"name": "ABCCo",
"status": "Active",
"_embedded": {
"organization": {
"CompanyUid": "27e9cf71-fca4",
"name": "ABCCo",
"type": "Company",
"taxId": "0123",
"city": "Portland",
"notes": ""
}
}
}
]#5 Making design unreadable to humans
“K”eep “I”t “S”imply “S”tupid when you are designing your API. While APIs are meant for computer-to-computer interaction, the first client of an API is always a human, and the API contract is the first piece of documentation. Developers are more apt to study your payload design before they dig into your docs. Observation studies suggest that developers spend more than 51% of their time in editor and client as compared to ~18% on reference.
For example, if you skim through the payload below it takes some time to understand because instead of property names it includes an “id”. Even the property name “data” does not suggest anything meaningful aside from just being an artifact of the JSON design. A few extra bytes in the payload can save a lot of early confusion and accelerate adoption of your API. Notice how user-ids appearing on the left of the colon (in the position where other examples of JSON ideally have property names) creates confusion in reading the payload.
"{id-a}":
{ "data":
[
{
"AirportCode": "LAX",
"AirportName": "Los Angeles",
"From": "LAX",
"To": "Austin",
"departure": "2014-07-15T15:11:25+0000",
"arrival": "2014-07-15T16:31:25+0000"
}
… // More data
]
},We think that JSON like this is more difficult to learn. If you want to eliminate any ambiguity in the words you choose to describe the data, keep the payload simple and if any of those labels could be interpreted in more than one way, adjust them to be more clear. Here is a sample response from Airlines endpoint of aviationstack API. Notice how the property names clearly explain the expected result while maintaining a simple JSON structure.
"data": [
{
"airline_name": "American Airlines",
"iata_code": "AA",
"iata_prefix_accounting": "1",
"icao_code": "AAL",
"callsign": "AMERICAN",
"type": "scheduled",
"status": "active",
"fleet_size": "963",
"fleet_average_age": "10.9",
"date_founded": "1934",
"hub_code": "DFW",
"country_name": "United States",
"country_iso2": "US"
},
[…]
]#6 Know when you can break the RESTful rules
Being true to the RESTful basics — such as using the correct HTTP verbs, status codes, and stateless resource-based interfaces — can make your customers’ lives easier because they don’t need to learn an all new lexicon, but remember that the goal is just to help them get their job done. If you put RESTful design first over user experience, then it doesn’t really serve its purpose.
Your goal should be helping your customers be successful with your data, as quickly and easily as possible. Occasionally, that may mean breaking some “rules” of REST to offer simpler and more elegant interfaces. Just be consistent in your design choices across all of your APIs, and be very clear in your documentation about anything that might be peculiar or nonstandard.
Conclusion
Beyond these common pitfalls, we have also created a comprehensive guide packaging up our rich experience designing and managing APIs at incredible scale with Google Cloud’s API management product, Apigee.
Apigee — Google Cloud’s native API management platform — helps you build, manage, and secure APIs — for any use case, scale or environment. Get started with Apigee today or check out our documentation for additional information.
More Relevant Stories for Your Company

Transforming Media Industry: Three Strategies for Media Leaders to Leverage Generative AI
The digital era turned the traditional formula for media and entertainment success on its head, ushering in new technologies that have changed how content is produced, distributed, experienced, and monetized. Audiences have more choice, flexibility, and power over what they consume, and today’s media companies have to embrace ongoing transformation
Google Cloud named a Leader in API Management Solutions in The Forrester Wave
The right API strategy is a key element of your digital business success, so choosing the best API management solution is critical - but often challenging. Organizations like yours need to address a wide range of criteria to support an effective digital business strategy, and that requires a robust API

API Design Best Practices and Common Pitfalls
The job of an API is to make the application developer as successful as possible. When crafting APIs, the primary design principle should be to maximize application developer productivity and promote adoption. API designers and developers generally understand the importance of adhering to design principles while implementing an interface. No

Google Cloud Offering a Natural Migration Path for Developers Accustomed to Heroku
Modern developers worldwide have grown accustomed to the comfort of writing code, pushing to a remote Git repository and having that code be deployed at an accessible URL without having to worry about how it is deployed. This was a workflow popularized by Heroku years ago which brought joy and







