The Last CachebenderBook Three · Earth
Book Three · Earth
Book Three · Earth · Lesson 6 of 14
Lesson Six

The Distributed Cache Tribes

Technique unlocked Stonesplit one cache shared across many application servers

When each server keeps its own copy

You have 10 application servers. Each has its own local cache. This gives you three distinct problems.

1. They disagree

Imagine ten servers running the same code, each with its own cached copy of the price. What happens when the price changes?

The code is identical on every server. What differs is when each one last looked.

09:00   Server A   reads price £25 from the database, caches it
09:00   Server B   reads price £25 from the database, caches it

09:30   Admin      changes the price to £20 in the database
        Server A   handled that write, so it updates its own cache   → £20
        Server B   was not involved. It has no idea anything happened → £25

09:31   Customer   request lands on Server A    → sees £20
09:31   Customer   refreshes, lands on Server B → sees £25

Neither server has a bug. Both are running exactly the code you wrote. They disagree because a cache is a copy, and each server made its own copy at its own moment.

So with ten servers you do not have one cache with a stale entry. You have ten separate caches, each stale in its own way.

2. Your hit ratio collapses

The same product must be loaded and stored ten separate times — once per server. Each server only sees a tenth of the traffic, so each takes far longer to warm up. You are paying ten times the memory for a fraction of the benefit.

3. You cannot invalidate

Change something, and you would need to reach into all ten processes to remove the stale copy. There is no mechanism to do that. As you add servers, it gets worse.

SOLUTION

A distributed cache solves all three: one shared cache that every server reads from and writes to.

Server A ─┐
Server B ─┼─→  Shared cache  ─→  Database
Server C ─┘

One value. One place to update. One warm-up. All ten servers benefit from the first one's miss.

When the cache has more than one node, sharding decides which node owns a key. The same key always goes to the same node; replicas can keep extra copies to make a node failure less disruptive. A popular key can still concentrate load on its owner.

In practice, for most teams, that shared cache is Redis. So the rest of this lesson is about what Redis gives you, when it helps, and what it costs.

What Redis gives you

You met Redis in Lesson 5 as “a separate program that stores key-value data in memory.” Here is what that buys you beyond simply being shared:

  • Every server sees the same value. The reason this lesson exists.
  • It survives your application restarting. Deploys no longer empty your cache.
  • Sub-millisecond reads. Slower than local memory, far faster than a database.
  • TTL is built into every key. You do not implement expiry — you set it and Redis honours it.
  • Eviction is built in. When memory fills, Redis decides what to drop using a policy you choose (Lesson 10).
  • Operations are atomic. Two servers incrementing the same counter cannot overwrite each other, because the increment happens inside Redis rather than in your code.
  • It can pass messages between your servers. Redis can publish a message that every subscribed server receives, which is useful well beyond caching.
  • It stores more than strings. Counters, lists, sets and sorted sets are native, which makes some problems trivial that are awkward anywhere else.

When Redis helps

ScenarioWhy Redis specifically
Session storageEvery server needs the same session. The original use case, and still the most common.
Rate limitingAtomic counters with an expiry — correct under concurrency without you writing locks
Counters and leaderboardsSorted sets do natively what is genuinely awkward in a relational database
Shared cache in front of a databaseThe case this course is about
Simple queuesLists as job handoff between services

Those are the scenarios. Here is what they actually look like to write.

Reading a Redis command

Every command is a verb, a key, and usually a value:

SET  product:42  "{name: Four Realms Atlas, price: 84}"  EX 300
│    │           │                                │
verb key         value                            how long to keep it

Keys are just text. The colons in product:42 mean nothing to Redis — they are a naming habit, the way folders organise files.

The last part is the one that matters most here. Expiry is built into the write. You say how long the value should live at the moment you store it, and Redis deletes it for you when the time is up. There is no cleanup job to write and no cleanup job to forget.

Four of them in practice

A session — signed in for 30 minutes.

SET session:a91f3c "{user_id: 88}" EX 1800

    EX 1800  →  delete this in 1800 seconds, which is 30 minutes

Half an hour later the key is gone and the user is signed out. Nothing had to notice or tidy up.

A rate limit — five attempts a minute.

INCR   ratelimit:user:42:0930        →  1     their first try this minute
EXPIRE ratelimit:user:42:0930 60              throw the counter away after a minute

INCR   ratelimit:user:42:0930        →  2
INCR   ratelimit:user:42:0930        →  3     ... refuse them at 6

INCR adds one and hands back the new total, so you get the count and the increment in a single step. The 0930 on the end is the current minute — 09:30. Each minute quietly gets its own counter, and yesterday's counters have long since deleted themselves.

A leaderboard — the top ten, instantly.

ZADD      leaderboard 4820 "player:17"    record a score of 4820

ZREVRANGE leaderboard 0 9            →  the ten highest scores, in order

A sorted set keeps its members in score order at all times. So "the top ten" is a read, not a sort — Redis already knows the answer. The same thing in a relational database means ordering the table on every request.

A cache — hold a product for five minutes.

SETEX product:42 300 "{name: Four Realms Atlas, price: 84}"

GET   product:42    →  the value, if it is still there
                    →  nil, if it is not

SETEX is just SET with the expiry built into the name. And that nil is worth pausing on: it is your cache miss. It is the moment your code stops asking Redis and goes to the database instead — the exact fork you met in Lesson 2.

Look at the rate-limiting example. Two servers can run INCR on the same key at the same moment and the count is still correct, because Redis performs the increment itself. Doing that with a plain read-modify-write in your own code — read 10, add 1, write 11, twice, concurrently — loses one of the increments.

This is the read-modify-write race, and Redis solves it by doing the work for you.

What belongs in Redis

Things that must be the same on every server, or must survive a restart, or are about one particular person.

  • Login sessions — the request that signs you in might land on server 3 and the next one on server 7. Both have to know who you are.
  • Shopping carts — the same reason, and people expect them to still be there tomorrow.
  • One-time codes and password-reset links — store them with an expiry and Redis deletes them for you when the time is up.
  • Rate limits — “five login attempts a minute” only works if every server counts into the same place.
  • Results fetched from the database — one server's slow query becomes every server's fast answer.
  • Live counters — view counts, or “3 people are looking at this right now.”

Set that against the local-cache list from Lesson 5 and the split becomes clear: local for facts about your application, Redis for facts about your users.

Local cache or Redis?

One question settles most cases.

If two of your servers gave different answers for this, would anything actually break?

No — use a local cache. It is faster and simpler.

Yes — use a shared cache such as Redis, and allow for the network round trip.

The category tree can be five seconds out of date on one server and nobody will ever notice. A shopping cart cannot.

The downsides

Redis is not free, and the costs are worth stating plainly:

  • It is a network hop. Roughly 5,000 times slower than local memory. Do not send it a request you could have answered in-process.
  • It is another thing that can go down. See the rule above.
  • Memory is expensive. RAM costs far more per gigabyte than disk. A cache that holds everything is usually a cache that costs more than the problem.
  • Everything is serialized. Objects go in as bytes and come back as bytes, and that conversion is not free.
  • It is not a database. Redis can persist to disk, but that is not what it is for. Do not store anything there that you cannot afford to lose.
  • One key lives on one node. A single very popular key concentrates all of its traffic on one machine, and adding nodes does not help — the key still lives in exactly one place.

That last one is not a flaw you can configure away. It is a property of how sharding works, and the Latency Lord is about to notice it.

At the Storefront

Move product details, price, and reviews to shared Redis. Choose a key scheme that includes every input that changes the answer. Keep the category tree local — it is small, it is read constantly, and slight staleness is harmless. Local and shared caching, used together.

Common misconception

“A shared cache makes my data consistent.”

It makes your servers agree with each other. All ten can now confidently serve the same stale price. Agreement is not correctness, and the gap between them is Lesson 11.

The Forge

The Latency Lord counts the nodes and studies the traffic flowing to each.

They are evenly spread — except for one. A single key, asked for more often than every other key combined. The front-page item. The one everybody wants.

He does not need to attack every node in the cluster. He needs the one node that holds that key.