Interlude · Spirit
The Temple of Eviction
Two different reasons data leaves
Things disappear from a cache for two reasons, and the two have nothing to do with each other.
- Expiration — the copy got old, so you stopped trusting it.
- Eviction — you ran out of room, so something had to go.
That is the whole distinction. It is worth stating that plainly, because the two are confused constantly — including in documentation.
| Trigger | Reason | Controlled by | |
|---|---|---|---|
| Expiration | Time | The copy is too old to trust | TTL |
| Eviction | Memory pressure | You ran out of room | Eviction policy |
Expiration is time-driven. It is about correctness. The value may still be perfectly useful and there may be plenty of memory — you remove it because you no longer believe it.
Eviction is memory-driven. It is about capacity. The value might be perfectly current and useful — you remove it because something else needs the space.
A cache with infinite memory still needs expiration. A cache holding data that never changes still needs eviction. They are independent, and confusing them leads to setting TTLs to manage memory (which makes your data needlessly stale) or sizing memory to manage freshness (which does not work at all).
TTL, used deliberately
You met TTL in Lesson 2. Now use it for two distinct jobs.
TTL as a correctness tool. The TTL is your
statement about how wrong you are willing to be.
TTL = 300 says:
I accept this may be up to five minutes out of date. Pick
it from the business requirement, not from habit.
TTL as a safety net. This one is subtle and important. Even with perfect invalidation (Lesson 11), things go wrong — a message is dropped, a service is down, a code path is missed. TTL bounds the damage. Without it, a failed invalidation means a value is wrong forever. With it, the worst case is bounded by the TTL.
Put a TTL on everything, even data you invalidate explicitly. It is the backstop for the day your invalidation silently fails. It will.
TTL jitter
Set 10,000 keys during a deployment, all with
TTL = 3600.
One hour later, all 10,000 expire in the same second.
Jitter is adding a small random amount to each TTL so they spread out:
ttl = 3600 + random(0, 600) # between 60 and 70 minutes
Now those 10,000 keys expire spread across ten minutes rather than in a single instant. This single line prevents a specific attack in Lesson 12, and it costs you nothing.
Eviction policies
When the cache is full and something new arrives, something old must go. The eviction policy decides what.
LRU — Least Recently Usedoptional read
Remove whatever has not been touched for the longest time.
The idea is simple: recently used items are often used again soon. That makes LRU a useful starting point.
The weakness — and it is severe. LRU is destroyed by a scan: one large pass over data that will not be requested again. A nightly report reading a million products marks all of them “recently used,” pushing out the small hot set that your actual users depend on. The report finishes; your cache is full of things nobody wants; your hit ratio collapses; your database absorbs the difference.
A single batch job can wipe out a perfectly healthy cache. Watch for this.
Where you would use it Almost
everywhere, as the default. It is what Redis
allkeys-lru gives you, what most application cache
libraries do without being asked, and what your CPU does with
its own caches. Session data fits it especially well — a user
who was active a minute ago is the one most likely to be back.
Reach for something else only if you have batch jobs sweeping
the cache, or a hot set stable enough for LFU to protect.
LFU — Least Frequently Usedoptional read
Remove whatever has been requested least often, regardless of when.
The logic: popularity is more meaningful than recency. A page requested 10,000 times matters more than one requested once, even if the once was more recent.
Why it can beat LRU: it survives scans. The report's million products each get one access; the front-page item has thousands. LFU keeps the front page and discards the report data — exactly right.
The weaknesses:
- New items look unpopular. Something added a second ago has a count of 1 and is a prime eviction target, even if it is about to become the most popular item on the site. New content struggles to establish itself.
- Old items overstay. Something enormously popular last month keeps a high count long after people stopped wanting it.
Practical implementations use windowed or decaying LFU, counting recent accesses more heavily, which fixes both. If you choose LFU, choose one of these.
Where you would use it When a small, stable set of items takes most of the traffic — the top 1% of products that account for most views, a CDN holding assets that stay popular for months, reference data that barely changes. It is also the right answer when you have a nightly batch job that would otherwise flush an LRU cache. Avoid it where content turns over quickly: news, social feeds, anything where today's popular item was created this morning.
FIFO — First In, First Outoptional read
Remove whatever was added earliest, regardless of use.
Simple and cheap — no bookkeeping. It also ignores usage entirely, so it will happily evict your most popular item because it happened to be cached first. Rarely the right choice for a cache; fine for a queue.
Where you would use it Honestly, rarely, and almost never for a general cache. Where it does fit is anywhere insertion order is the point rather than an accident — a fixed-size buffer of recent events, a replay window, a rolling log kept in memory. If every entry has roughly equal value and you want zero bookkeeping, FIFO is defensible. If some entries are more valuable than others, it is the wrong tool.
Randomoptional read
Remove a randomly chosen item.
This sounds like giving up, and it is not. It requires no bookkeeping at all, which makes it very fast and memory-efficient — LRU must maintain ordering on every single access, and that costs something on every read.
For workloads without a strong recency pattern, random performs surprisingly close to LRU at a fraction of the overhead. Redis offers it deliberately. Do not dismiss it.
Where you would use it Very
high-throughput caches where LRU's per-read bookkeeping is a
measurable cost, and memory-constrained systems where the
ordering structure itself is too expensive to keep. It also does
well where access is genuinely uniform and no hot set exists to
protect. Redis exposes it as allkeys-random for
exactly these cases.
Choosing
| Policy | Best when | Fails when |
|---|---|---|
| LRU | Access is bursty and recency-driven | A scan sweeps through it |
| LFU | There is a stable hot set | Content turns over quickly |
| FIFO | Almost never for caches | Any time popularity matters |
| Random | Low overhead matters; access is uniform | A clear hot set deserves protecting |
Real caches offer combinations — Redis has allkeys-lru,
volatile-lru (only keys with a TTL set),
allkeys-lfu, volatile-ttl (shortest
remaining life first), and noeviction (reject writes
when full).
volatile-* variants matter more than they look: they
let you mark some entries as never-evictable by giving them no TTL,
protecting data you cannot afford to lose.
Reading the eviction rate
The eviction rate — how many items are being removed per second for lack of space — is one of the most useful signals a cache gives you.
- Near zero: your cache is big enough. Good.
- High and rising: your cache is too small. Items are being thrown out before they can be reused, and your hit ratio is being destroyed by capacity, not by TTL.
- Sudden spike: something just flooded the cache. A batch job, a scan, a bug writing far too much.
A high eviction rate paired with a falling hit ratio is a specific, actionable diagnosis: the cache needs more memory, or you are caching too much. Lesson 13 will show you where to watch this.
At the Storefront
| Part | TTL | Reasoning |
|---|---|---|
| Product details | 1 hour + jitter | Changes rarely; an hour stale is harmless |
| Price | 60 seconds | Must be close to correct |
| Stock count | 10 seconds, or uncached | Being wrong here sells things you do not have |
| Reviews | 10 minutes + jitter | Slow-changing, heavily read |
| Category tree | 5 minutes | Small, hot, rarely edited |
Policy for the Redis instance:
allkeys-lru as the default — and check
for batch jobs that scan products, because those will wreck it. If
you have one, either route it around the cache or move to windowed
LFU.
“LRU is always the right choice.”
LRU is a good default that a single batch job can destroy. LFU survives scans but starves new content. Random is cheaper than both and often nearly as good. Choose from your access pattern, not from habit.
The Latency Lord walks the realm at a quiet hour and does something very simple.
He does not delete anything. He does not break anything. He simply finds every expiry timer in the kingdom and sets them all to the same hour.
Nothing is stolen. Nothing is broken. Everything simply ends at once.