How this lesson works

Use the earlier lessons to work through five attacks. For requests that name nonexistent products, you will also learn an extra defense: Bloom filters.

For each attack:

1. The attack — watch it break the storefront.
2. Your attempt — before reading on, use what you already know to propose a defense.
3. The confirmation — compare your answer with the defense and follow links to earlier lessons when you need a refresher.

Most defenses build on Lessons 1–11. The Bloom-filter section explains a new tool, including when it helps and how to avoid rejecting real products.

Attack 1 — The Stampede

Midnight. The front-page product's cached value expires. In the next 50 milliseconds, 1,000 requests arrive. Every one misses. Every one queries the database.

The database, which was comfortable, receives 1,000 identical queries at once. It slows down. Because it slows down, the reload takes longer. Because the reload takes longer, more requests pile into the gap. Because more requests pile in, the database slows further.

Your database falls over. Not because of a flaw in your cache — because your cache worked exactly as designed. Cache-aside has a gap between “expired” and “reloaded,” and every request arriving in that gap goes to the database.

Also called a dogpile or thundering herd.

Single-flight is the direct answer. Refresh-ahead is the preventive one. Use both on your hottest keys.

Attack 2 — The Penetration

Requests arrive for product:999999999. It does not exist and never has.

Follow it through your realms. The browser has no copy — nothing was ever stored. The CDN has no copy. Redis has no copy. So the request reaches the database, which searches its index and confirms the product does not exist.

Then the same request arrives again. Every step repeats. This implementation caches found products, but forgets missing ones.

Your entire caching architecture is transparent to these requests. Every one goes straight to the database, and an attacker can send thousands per second using randomly generated IDs. Your cache defends nothing, because there is nothing to cache.

Negative caching — cache the fact that something does not exist.

key = "product:999999999"
value = cache.get(key)

if value is NOT_FOUND_MARKER:
    return 404                          # HIT — never touches the database

if value is None:                       # genuinely not in cache
    row = database.query(...)
    if row is None:
        cache.set(key, NOT_FOUND_MARKER, ttl=60)   # ← cache the absence
        return 404
    cache.set(key, row, ttl=300)
    return row

return value                           # ordinary cache hit

Where this comes from: L2 taught that a key maps to a value, and L8 taught cache-aside. The insight is that “there is no such product” is a perfectly valid answer, and valid answers can be cached. Nothing new — just noticing that a negative result is still a result.

Two details:

• Use a short TTL. For this example, cache confirmed absence for 60 seconds. If the product is created, clear its negative entry; the TTL is a fallback if that invalidation is missed.
• Use a distinct marker. You must be able to tell “cached as non-existent” apart from “not in the cache.” Storing plain null usually makes those indistinguishable, and you get a permanent miss instead of a hit.

Also worth applying: input validation. If product IDs are always under 8 digits, reject 999999999 before it reaches any cache or database at all. The cheapest defense is not answering the question.

# After validation and a cache miss.
# Application helper, not a raw Redis command:
verdict = product_filter.check_if_current(product_id)

if verdict == DEFINITELY_ABSENT:
    return 404

# POSSIBLY_PRESENT or UNKNOWN: use the normal lookup.
row = database.find_product(product_id)
if row is None:
    cache.set(key, NOT_FOUND_MARKER, ttl=60)
    return 404
cache.set(key, row, ttl=300)
return row

Attack 3 — The Avalanche

You deployed at 14:00 and warmed the cache — 50,000 keys, all written within a few seconds, all with TTL = 3600.

At 15:00, all 50,000 expire in the same second.

Every request now misses. Not one hot key — everything, simultaneously. Your database receives your entire read traffic with no cache in front of it, for as long as it takes to reload 50,000 entries. It cannot cope, because it was never sized to; the cache was absorbing 95% of the load.

The same thing happens when a cache node restarts, or when a cluster fails over and comes back empty.

The distinction from a stampede: a stampede is many requests, one key. An avalanche is many keys, all at once.

Jitter is the fix, and it is one line of code. This is the cheapest defense in the entire course, and forgetting it is the most common cause of a self-inflicted outage on this list.

Attack 4 — The Hot Key

You partitioned your cache across 10 nodes. Load is evenly spread — except that one key is not like the others.

The front-page item receives 40% of all cache traffic. So one node receives 40% of your traffic while the other nine share the remaining 60%.

That node saturates. Its latency rises. Every request for the popular item — the item most of your revenue depends on — is now slow. The other nine nodes are nearly idle.

Adding more nodes does not help. The key still lives on exactly one node. Sharding distributes keys; it cannot distribute one key.

1. A local shield cache (L5). Keep the hot key in each application server's own memory, in front of the shared cache. Ten app servers each holding a copy means the shared cache receives ten requests instead of a million. A 5-second local TTL is usually plenty. This is the single most effective fix, and it is exactly the point of Lesson 5.

2. Key splitting (L2). Store the same value under several keys and pick one at random per request:

product:42:copy0
product:42:copy1
   ...
product:42:copy9      ← 10 keys, one value, spread across 10 nodes

Reading picks a random copy, so traffic distributes. Writing must update all ten, so this is for read-heavy data only.

3. Replication (L6). Put the hot partition on replicas and spread reads across them.

4. Detection first (L13). You cannot defend a hot key you have not found. Per-key metrics are how you find them — which is next lesson.

This defense combines a local cache with a distributed cache — the two parts of the application realm from Book Three.

Attack 5 — The Poisoning

The worst one, because it is a correctness failure rather than a performance failure — and it can be a security incident.

A logged-in user requests the product page. The response includes their cart badge: “2 items.” Something in your stack caches that response under the key /product/42, without the user in the key.

The next visitor requests /product/42. They get a hit. They see someone else's cart.

That is cache poisoning: incorrect or unauthorised content stored in a cache and served to everyone.

How it happens in practice:

• A key missing a distinguishing input — the Lesson 2 failure, at scale
• Vary not set on a response that varies by language, currency, or authentication
• private not set on a personalised response, so a shared cache stores it
• An error page cached — a 500 error gets a 1-hour TTL and now everyone sees an error page for an hour
• Unnormalised request headers — an attacker sends a header your app reflects into the response, and the poisoned version is cached and served to others

The blanket rule worth memorising:

That is why the cart badge has been marked “never cached” since the very first table in this course. It was not an arbitrary choice. It was this attack, waiting eleven lessons to arrive.

The pattern worth seeing

Look at the five attacks together:

Four of the five are caused by your own key and TTL decisions. Only penetration comes from outside, and even that is a decision about what you chose to cache.

Then the question worth carrying out of this lesson: which of the five would your current design already survive, and which would take you down tonight?