Distributed Caching Architecture
A cache stores the result of expensive work (a query, a computation, a render) closer to the consumer so repeated requests skip the work. Distributed caching adds a shared tier (Redis/Memcached) in front of the slow source. The two genuinely hard problems are invalidation and consistency with the source of truth.
Caching trades freshness for speed and load reduction. The design questions are: where does the cache sit (client, CDN, app-local, shared tier), who writes it, and how does stale data get evicted? The famous quip — "there are only two hard things: cache invalidation and naming things" — is about the first. Source: compiled from caching-strategy literature
Read/write strategies
| Strategy | Read path | Write path | Note |
|---|---|---|---|
| Cache-aside (lazy) | App checks cache; miss → load from DB → populate | App writes DB, invalidates cache | Most common; cache only holds requested keys |
| Read-through | Cache loads from DB on miss itself | — | Cache library owns loading |
| Write-through | — | Write cache + DB synchronously | Cache always fresh; slower writes |
| Write-back | — | Write cache, flush to DB async | Fast writes; risk of loss on crash |
Cache-aside is the default; write-through suits read-heavy data that must not be stale; write-back suits write-heavy tolerant data.
Invalidation and eviction
- TTL — expire after a time window; simplest, accepts bounded staleness.
- Explicit invalidation — delete/update the key on write; correct but easy to miss a path.
- Eviction policy — LRU/LFU when memory is full; the cache is bounded, so most keys eventually evict.
Failure modes
- Stampede / thundering herd — a hot key expires and thousands of requests hit the DB at once. Fix with request coalescing (single-flight), early/probabilistic recomputation, or a short lock.
- Penetration — queries for non-existent keys bypass the cache every time; cache negative results or use a Bloom filter.
- Stale reads — invalidation lost ⇒ users see old data; bound with TTL as a backstop.
Design implications
- Pick the consistency you can tolerate. A cache is an eventually-consistent replica (Replication and Consistency Models); decide the acceptable staleness per key class.
- Shard + replicate the cache tier for capacity and HA (Sharding and Partitioning Architecture), using consistent hashing so a node loss reshuffles minimally.
- Cache is also a rate-limit and gateway primitive — the Redis counter behind Rate Limiting Architecture and response caching at the gateway.
- The LLM analogue is prompt/KV caching — same "reuse expensive work" logic; see Prompt-Caching Economics.
Architecture Position
| Axis | Value |
|---|---|
| Family | Distributed systems primitives |
| Boundary owned | Caching expensive work near consumers with invalidation and consistency tradeoffs. |
| Read with | Replication and Consistency Models, Load Balancing Architecture, Rate Limiting Architecture |
| Use this page when | reducing latency or cost with shared caches |
Timeline
-
2026-07-01 | Architecture category refresh added this page to the Distributed systems primitives family, linked it to Architecture System Map, and kept it standalone because it owns this boundary: Caching expensive work near consumers with invalidation and consistency tradeoffs. Source: User request, 2026-07-01
-
2026-06-02 | Page created. Captured cache placement, the cache-aside/read-through/write-through/write-back strategies, TTL/explicit invalidation + eviction, the stampede/penetration/stale failure modes with fixes, and the consistency + sharding implications. Source: compiled from caching-strategy literature