Caching is the cheapest performance win in system design and one of the easiest ways to serve wrong data. These notes cover it from the ground up, ordered so each page builds on the one before.
How to read this
Start with the overview, go deep on mechanics, then eviction, and finish with HTTP caching to see how the same ideas show up in browsers and CDNs.
Learning path
| # | Note | What you’ll get out of it |
|---|---|---|
| 1 | Caching concepts — overview | The whole map: strategies, eviction, distributed caching, failure modes, layers, technologies |
| 2 | Caching Basics & Mechanics | TTL, hit and miss, invalidation, consistency windows, warming, read and write patterns |
| 3 | Eviction Policies | LRU, LFU, FIFO and random, W-TinyLFU, Redis’s approximations, and how to choose |
| 4 | HTTP Caching (RFC 9111) | Cache-Control, freshness, ETag and 304, Vary, CDN rules |
Where caches live
flowchart TB U[User] --> B[Browser cache] B --> C[CDN / edge] C --> G[API gateway / reverse proxy] G --> A[App server] A --> L1[(L1: in-process<br/>Caffeine, Guava)] A --> L2[(L2: distributed<br/>Redis, Memcached)] L2 --> DB[(Database)]
Each step down is slower and more expensive. Each cache above the database is one more copy that can go stale.
The one-page cheat sheet
Reading
- Cache-aside: the app checks the cache, falls back to the database on a miss, then fills the cache. This is the default choice.
- Read-through: the cache loads from the database itself, so the app only talks to the cache.
Writing
- Write-through: write the cache and the database together. The cache stays consistent but writes are slower.
- Write-behind: write the cache now and the database later. Writes are fast, but data can be lost if the cache crashes.
- Write-around: skip the cache on writes. Good when data is written once and rarely read.
Failure modes to name in an interview
| Problem | What happens | Fix |
|---|---|---|
| Stampede (thundering herd) | A popular key expires and every request rebuilds it at once | Lock or single-flight, probabilistic early refresh, background refresh |
| Penetration | Requests for keys that don’t exist always reach the database | Cache the null result, use a Bloom filter |
| Avalanche | Many keys expire at the same moment | Add random jitter to TTLs, use more than one cache level |
| Pollution | Rarely used data pushes out hot data | Use LFU or W-TinyLFU, add admission control |
Interview prompt
“Your product page is backed by Redis with a 60-second TTL. A flash sale starts and the database falls over every minute, on the minute.” What’s going on, and what three changes would you make?