# Cache Stampede Prevention: Your TTL Needs Two Clocks

**Short answer:** a cache stampede happens when many requests miss the same cached value at the same time and all rebuild it at once. The usual advice is "add a lock." The more useful fix is to stop letting one TTL do two jobs. Give each hot key a **soft TTL** that starts one background refresh and a **hard TTL** that stops serving, and serve the old value in between. Nobody waits, and the database sees one rebuild instead of hundreds.

The rest of this post is the why, the four ways misses arrive together, and code for Redis, Laravel, nginx and Caffeine.

## The number that predicts a stampede

Your hit rate can be 99% while one key sits at 0% for the half-second that matters. The database doesn't see your hit rate. It sees your misses.

The size of a stampede is roughly:

```text
herd size ≈ request rate for the key × time to rebuild it
```

A key read 2,000 times a second with a 0.4-second rebuild gives about 800 identical queries. That is every request that arrives inside the rebuild window, not just the one at the moment of expiry.

The trap is that rebuild time isn't fixed. Those 800 queries slow the database, the rebuild stretches to 2 seconds, and the next window holds about 4,000.

![Loop diagram: a hot key expires, every reader rebuilds, the database slows, the rebuild takes longer, and more readers land in the miss window](https://miro.medium.com/v2/resize:fit:1600/1*scrn0QCaIc7aJ5u8qCCnpQ.png align="center")

That self-reinforcing loop is why a stampede can keep a system down after the expired key is long gone. It's the same pattern covered in [Metastable Failures: Why Your System Stays Down After the Cause Is Fixed](https://simplyexplained.hashnode.dev/metastable-failure-retry-storm-load-shedding).

## Four ways cache misses arrive together

| Pack | What causes it | What fixes it |
| --- | --- | --- |
| **The hot key** | One popular key expires | One rebuilder + serve the old value |
| **The echo** | Thousands of keys written in the same minute with the same TTL expire in the same minute, then again one TTL later | Jitter on every TTL |
| **The cold cache** | Restart, failover, node leaves the ring, eviction under memory pressure | Warm before traffic, ramp, or shed load |
| **The purge** | "Clear cache" in a deploy, delete-on-write on a hot key, repair logic that runs on every client | Treat clears as deploys; never let every client run the fix |

The purge has a well-documented example. On September 23, 2010, Facebook was down for about two and a half hours after a configuration value in the persistent store was treated as invalid. Every client tried to fix it by querying a database cluster, which was soon overwhelmed. Clients then treated the resulting errors as invalid values and deleted the cache key, so the queries continued even after the original value was corrected. Facebook had to stop all traffic to that cluster, which meant taking the site down. Config changes as an unreviewed path to production are covered in [Nobody Deployed Anything. Production Broke Anyway.](https://simplyexplained.hashnode.dev/nobody-deployed-anything-production-broke-anyway)

## The root cause: one TTL, two jobs

A plain TTL answers two questions with one number:

1. **How old can this value be before showing it is wrong?** (a product question)
2. **When should someone fetch a new one?** (a load question)

When both happen at the same instant, the old value disappears exactly when the rebuild starts, so every caller in the window rebuilds on the request path. Split them:

- **Soft TTL:** the value is past its prime. One caller starts a refresh; everyone keeps getting the old value.
- **Hard TTL:** the value is too old to show. Only reached if refreshes keep failing or the key goes unread.

![Two timelines: a plain TTL where every request misses at once, and a soft/hard TTL where one caller rebuilds and others get the old value](https://miro.medium.com/v2/resize:fit:1600/1*_5VVTbwcIATor2NBuLVx8A.png align="center")

Facebook's 2013 paper *Scaling Memcache at Facebook* used the same idea through leases: one client per key got permission to rebuild (at most once every 10 seconds), and reads could get a value explicitly marked as stale. They report peak database query rate falling from 17K/s to 1.3K/s.

## Two-clock cache in Redis (Python sketch)

This stores the soft deadline inside the value and uses Redis's own expiry as the hard TTL.

```python
import json, random, time, threading, uuid
import redis

r = redis.Redis()

def get_two_clock(key, rebuild, fresh_s=3000, stale_s=600, lock_s=30, fallback=None):
    raw = r.get(key)
    if raw:
        entry = json.loads(raw)
        if time.time() < entry["soft_until"]:
            return entry["value"]                       # fresh
        token = str(uuid.uuid4())
        if r.set(f"lock:{key}", token, nx=True, ex=lock_s):   # one refresher
            threading.Thread(target=_refresh,
                             args=(key, rebuild, fresh_s, stale_s, token),
                             daemon=True).start()
        return entry["value"]                           # everyone gets the old value

    # Cold: nothing to serve. One rebuilds, the rest wait briefly.
    token = str(uuid.uuid4())
    if r.set(f"lock:{key}", token, nx=True, ex=lock_s):
        return _refresh(key, rebuild, fresh_s, stale_s, token)
    for _ in range(10):                                 # bounded wait: 0.5 s max
        time.sleep(0.05)
        raw = r.get(key)
        if raw:
            return json.loads(raw)["value"]
    if fallback:
        return fallback()                               # your cold-start policy
    raise TimeoutError(f"cache cold for {key}")

RELEASE = "if redis.call('get', KEYS[1]) == ARGV[1] then return redis.call('del', KEYS[1]) end return 0"

def _refresh(key, rebuild, fresh_s, stale_s, token):
    try:
        value = rebuild()
        fresh = fresh_s * random.uniform(0.9, 1.1)      # jitter breaks the echo
        entry = {"value": value, "soft_until": time.time() + fresh}
        r.set(key, json.dumps(entry), ex=int(fresh + stale_s))  # hard TTL
        return value
    finally:
        r.eval(RELEASE, 1, f"lock:{key}", token)        # only release our own lock
```

Three decisions are hiding in there, and they're yours to make, not the library's:

- `stale_s` is how stale this value may be shown. For a product description, minutes are fine. For a permission check, it's zero, and you shouldn't use this pattern.
- `lock_s` must be longer than your slowest realistic rebuild, or a second refresher starts while the first is still running.
- `fallback` is what a request gets when the cache is empty and someone else is rebuilding. A degraded default beats a second herd.

## The same pattern in the tools you already use

**Laravel** (11+) ships it as `Cache::flexible`:

```php
// Fresh for 50 minutes; for the next 10, serve stale and refresh after the response
$product = Cache::flexible("product:$id", [3000, 3600],
    fn () => Product::with('variants')->find($id));
```

`Cache::remember` has no stale window: when it expires, every concurrent request runs the closure. And once `flexible`'s second value passes, it behaves like a plain miss again.

**nginx** as a caching proxy:

```nginx
proxy_cache_use_stale updating error timeout http_500 http_502 http_503 http_504;
proxy_cache_background_update on;   # refresh in a subrequest, serve stale meanwhile
proxy_cache_lock on;                # collapse concurrent misses into one upstream request
```

**HTTP responses** (CDNs and caches that support RFC 5861):

```text
Cache-Control: public, s-maxage=60, stale-while-revalidate=300, stale-if-error=86400
```

**Java / Caffeine:**

```java
LoadingCache<String, Product> cache = Caffeine.newBuilder()
    .refreshAfterWrite(Duration.ofMinutes(50))  // soft: refresh in background, keep serving
    .expireAfterWrite(Duration.ofMinutes(60))   // hard: callers wait for a load
    .build(id -> repository.load(id));
```

## Where common fixes fall short

- **Locks with waiters.** Waiting requests hold workers and often database connections. Enough of them can exhaust the pool on their own, which is the failure in [Your Database Is Fine. Your App Is Dying Anyway.](https://simplyexplained.hashnode.dev/connection-pool-exhaustion-explained) If waiters time out and retry, you get a second herd.
- **Per-process dedupe.** Go's `singleflight` or a local mutex gives one rebuild per process. Forty pods means forty rebuilds. Decide if that's acceptable for this query.
- **Jitter alone.** It fixes the echo, not a single hot key.
- **Probabilistic early refresh (XFetch, VLDB 2015).** Coordination-free and good for hot keys. It may never fire for rarely read keys, which is fine because they don't make herds.
- **Stale-serving everywhere.** Some data can't be stale: permissions, balances, stock at checkout. That's the territory of [Why Your Cache Is Serving Stale Data](https://simplyexplained.hashnode.dev/why-your-cache-is-serving-stale-data-and-how-to-actually-fix-it). Choose staleness per value, never by default.

## The Miss Plan

![The Miss Plan: five questions to answer for every hot cached value](https://miro.medium.com/v2/resize:fit:1600/1*FLv0AuxCqVHauo_hfUU95g.png align="center")

For every cached value that's hot or expensive to rebuild:

1. **How stale may this be shown?** → hard TTL (ask the product owner)
2. **When does the refresh start?** → soft TTL, with jitter
3. **Who rebuilds?** → one per fleet or one per process; write the number down
4. **What does everyone else get?** → the old value; if none, a bounded wait, then a fallback
5. **What happens when the cache is empty?** → warm, ramp, or shed

## How to test it: count rebuilds, not misses

- **Log every rebuild** with key and duration. Alert when any key rebuilds more than once inside its own rebuild time.
- **Run the Flush Test.** Replay realistic traffic in staging, flush the cache (or kill one cache node), and watch queries per hot key, pool usage and p99 for the first minute. Target: about one rebuild per hot key, no pool saturation.
- **Look for the echo.** Leave it running for three TTLs and plot rebuilds per minute. Spikes at TTL multiples mean keys born together are dying together.

## Key Takeaways

- A stampede's size is request rate × rebuild time, and the stampede itself makes the rebuild slower.
- Misses arrive together in four ways: hot key, echo, cold cache, purge. Locks address only the first.
- A plain TTL ties "stop trusting this" to "rebuild this." Split them into a soft TTL and a hard TTL.
- Serve the old value while one caller refreshes. Waiting is not free; waiters hold connections.
- Staleness is a per-value product decision. Some data must never be served stale.
- Measure duplicate rebuilds per key, and flush the cache in staging on purpose.

## FAQ

### What is a cache stampede?

A cache stampede (also called a thundering herd or dogpile) is when many requests miss the same cached value at once and all try to rebuild it from the database simultaneously. The duplicate queries can slow or overload the database, which makes the rebuild slower and the stampede larger.

### What is the difference between a soft TTL and a hard TTL?

A soft TTL marks when a cached value should be refreshed; past it, the old value is still served while one caller rebuilds in the background. A hard TTL marks when the value is too old to show at all. Plain caches use a single TTL, which forces both events to happen at the same moment.

### Is a distributed lock enough to prevent a cache stampede?

A lock makes sure only one caller rebuilds, but the other callers still wait. Waiting requests hold workers and often database connections, which can exhaust a connection pool. Locks also do nothing for a cold cache with no old value. Pairing a lock with stale-while-revalidate removes most of the waiting.

### How does TTL jitter help with cache stampedes?

Jitter adds a small random amount to each TTL so keys written at the same time don't expire at the same time. It fixes synchronized expiry after deploys or warm-ups. It doesn't help when a single hot key expires, because that key still expires at one moment.

### What is stale-while-revalidate?

Stale-while-revalidate is a caching pattern where an expired value is still served while a fresh one is fetched in the background. It's defined for HTTP in RFC 5861 and is available in nginx (proxy_cache_use_stale updating with proxy_cache_background_update), Laravel (Cache::flexible), and Caffeine (refreshAfterWrite).

### How do I test whether my cache is vulnerable to a stampede?

Replay realistic traffic in a staging environment, then flush the cache or remove one cache node and count database rebuilds per hot key for the first minute. More than about one rebuild per key, or a saturated connection pool, means the miss path needs a plan. Also plot rebuilds over several TTLs to spot synchronized expiry.

## Related reading

- [Why Your Cache Is Serving Stale Data (And How to Actually Fix It)](https://simplyexplained.hashnode.dev/why-your-cache-is-serving-stale-data-and-how-to-actually-fix-it): the other half of the cache story
- [Metastable Failures: Why Your System Stays Down After the Cause Is Fixed](https://simplyexplained.hashnode.dev/metastable-failure-retry-storm-load-shedding)
- [Your Database Is Fine. Your App Is Dying Anyway.](https://simplyexplained.hashnode.dev/connection-pool-exhaustion-explained)
- [Nobody Deployed Anything. Production Broke Anyway.](https://simplyexplained.hashnode.dev/nobody-deployed-anything-production-broke-anyway)

## The bottom line

Every cached value gets rebuilt eventually. The cache doesn't remove that work from your database; it decides when the work arrives. A plain TTL schedules it for everyone at once. Two clocks and one rebuilder schedule it for one caller, in the background, before anyone notices.

> A cache doesn't remove load from your database. It decides when the load arrives.

*Get one of these in your inbox each week:* [*subscribe to Simply Explained*](https://simplyexplainedbyaj.substack.com/subscribe)*. New here?* [*Start here*](https://simplyexplained.hashnode.dev/start-here-understand-ai)*.*

*Adam Jaber is a software engineer who writes Simply Explained: complex topics, made simple. No jargon, no hype.*
