A client's e-commerce product pages called out to a third-party pricing and availability API on every page load, and that API's response time varied wildly, sometimes taking over two seconds. Since the underlying pricing data only actually changes a handful of times a day, calling it fresh on every request made no sense, and the client's product pages were effectively as slow as their slowest vendor dependency, on every single visit.
Diagnosing where the time was actually going
Before reaching for a cache, we spent a day actually measuring where page load time was going, rather than assuming the third-party API was the whole story. It turned out to be almost the entire story — the pricing call alone accounted for well over half of total page render time on a typical product page, dwarfing every other server-side cost combined. That measurement mattered, because it meant we could be confident a caching layer targeted at exactly this one call would move the needle meaningfully, rather than spending effort on a broader optimization pass that might have shaved milliseconds off a dozen smaller things without addressing the actual bottleneck.
The caching approach
We added a Redis cache in front of the integration with a short TTL — five minutes — keyed by product SKU. The first request for a given product after the cache expires still pays the slow API's cost, but every request after that for the next five minutes is served from Redis in single-digit milliseconds. Average page load time on affected pages dropped by more than half, and the third-party API's own rate limits, which the client had been bumping against during traffic spikes, stopped being a problem entirely, since the vast majority of requests never reach the third-party API at all anymore.
Choosing the TTL
- Five minutes balances freshness against cache-hit rate reasonably well for this client's catalog, where prices genuinely do not change minute to minute
- A shorter TTL would increase how often we pay the slow API's cost without meaningfully improving accuracy, since nothing in the business process updates pricing more frequently than that anyway
- A longer TTL risks showing a shopper a price that has since changed, which is a worse failure mode than a slightly slower page, so we did not push it further even though cache-hit rate would have improved
Handling the third-party API being down, not just slow
We also added a small safeguard for when the third-party API is down or erroring rather than just slow: rather than letting a failed refresh take down the product page, we serve the last successfully cached value past its TTL if a refresh attempt fails, with a short log line noting the fallback so we know it happened. This turned what would previously have been a visible outage on the client's product pages into a non-event during a real third-party outage that occurred a few weeks after launch — customers saw slightly stale pricing for a short window rather than a broken page. We do cap how far past its TTL a stale value can be served before we show a generic "pricing temporarily unavailable" message instead, since serving pricing from hours ago is a worse outcome than admitting it is currently unavailable.
Cache stampede protection
One problem we anticipated and built around from the start: when a popular product's cache entry expires, a burst of near-simultaneous requests could all miss the cache at once and hammer the slow third-party API with redundant calls for the exact same data. We added a short-lived lock in Redis so that only the first request after an expiry actually calls the third-party API, while the others wait briefly and then read the freshly-populated cache instead of triggering their own redundant calls. This is a small addition, but on the client's highest-traffic products it meaningfully reduced load on the third-party API during exactly the moments — a product going viral on social media, for instance — when that API is least able to handle a sudden burst of duplicate requests gracefully.
Extending the pattern elsewhere
Once this cache proved itself, we found two more integrations on the same client's site with a similar shape — a shipping-rate calculator and an inventory feed from a warehouse system — and applied the same pattern to both, with TTLs tuned to how frequently each underlying data source actually changes. Neither needed the same stampede protection, since neither sees the kind of sudden traffic spikes the product pricing endpoint does, which is a useful reminder that the same general pattern does not always need every safeguard applied uniformly.
Results
For any integration where data freshness requirements are looser than "must be real-time," a caching layer like this is usually the highest-leverage fix available, and it is one of the first things we now look for when a client complains about a slow page that turns out to be waiting on a third-party dependency outside our direct control.
Manual cache invalidation for exceptional cases
Occasionally a product's price changes deliberately and needs to reflect immediately — a flash sale starting at a specific minute, for instance — rather than waiting up to five minutes for the cache to expire naturally. We added a small internal endpoint the client's own team can call to invalidate a specific SKU's cached entry on demand, tied into the same admin tool they already use to manage flash sales, so a price change they schedule takes effect the moment it goes live rather than on the cache's own schedule.
What we monitor on the cache itself
- Cache hit rate, broken out by SKU popularity tier, since a low hit rate on a handful of very high-traffic products would matter more than the same low hit rate spread across the long tail of rarely-viewed products
- Redis memory usage and eviction rate, to catch the cache growing beyond what the instance can comfortably hold before it becomes a performance problem rather than after
- The stampede-lock's own contention rate, as a proxy for how often popular products are experiencing simultaneous cache misses
Extending caching into the admin dashboard
The client's own internal admin dashboard, used by their merchandising team to browse and adjust product listings, hit the same slow third-party pricing API on every page load and had never been mentioned in the original complaint that started this project, since internal tools tend to get less attention than customer-facing ones even when they are just as slow. Once the pattern proved itself on the storefront, applying the same cache to the admin dashboard's equivalent calls took under a day and made a tool the merchandising team uses dozens of times a day noticeably more pleasant to work in, which turned out to be a more visible win internally than the original customer-facing improvement, if a less consequential one for the business.
A caveat about caching data that affects promises to customers
One thing we were careful about from the start: availability data, unlike pricing, has a real cost to getting stale in the wrong direction. Showing a shopper a slightly outdated price is a minor annoyance if it is close to correct; showing a shopper that an out-of-stock item is available, only for the order to fail at checkout, is a worse experience that erodes trust more directly. We kept availability data on a shorter, one-minute TTL specifically because of this asymmetry, even though it meant a lower cache-hit rate and more load on the third-party API for that particular field than pricing alone would have required.
A smaller, related fix we almost missed
While instrumenting this cache we noticed the third-party pricing API also supported a batch endpoint capable of returning several SKUs in one call, which nobody on either side had previously used since the integration had been built one product at a time against the single-SKU endpoint. Combining the batch endpoint with the cache-miss path meant a page rendering several related products at once — a category listing page, for instance — now triggers at most one upstream call for all of its uncached SKUs together rather than one call per product, which further reduced both latency and third-party API load beyond what caching alone had already achieved. We plan to apply the same combination of caching and batching to a couple of other slow third-party integrations elsewhere on the site once we have a quiet week to schedule the work, since the pattern is now well understood and mostly a matter of finding the time rather than solving anything new.