As a lead developer, you must understand not just “how” to cache, but the systematic trade-offs involved in maintaining consistency under scale.
1. Caching Basics & Mechanics
- Time To Live (TTL): The lifespan of a cached item before it is considered stale.
- Cache Hit vs. Cache Miss: Understanding hit ratio and the performance cost of misses.
- Cache Invalidation: The core problem of keeping the cache consistent with the source of truth (DB) when data changes.
- Cold vs. Warm vs. Hot Cache: Managing cache states during deployments or node restarts.
2. Caching Strategies (Patterns)
- Cache-Aside (Lazy Loading): Application code manually manages the cache and DB (read from cache → if miss → read from DB → write to cache).
- Read-Through: The cache itself sits between the application and the DB. The application only talks to the cache, which handles fetching data on a miss.
- Write-Through: Synchronously writing data to both the cache and the database in the same transaction. Reduces risk of data loss but adds latency to writes.
- Write-Behind (Write-Back): Writing data to the cache immediately, but asynchronously writing to the database later. Very fast writes, but risks data loss if the cache crashes before the DB sync.
- Write-Around: Writing directly to the database, bypassing the cache. Good for data that is written once and rarely read.
3. Eviction Policies
When the cache gets full, you need rules to decide what memory to free up:
- LRU (Least Recently Used): Discards the least recently accessed items first (most common).
- LFU (Least Frequently Used): Discards the items accessed the least number of times.
- FIFO (First In, First Out): Discards the oldest items first, regardless of usage.
- Random Replacement (RR): Randomly evicts items (surprisingly effective in some massive datasets).
- W-TinyLFU: A modern admission policy and eviction algorithm (used by Caffeine) that combines LRU and LFU characteristics.
4. Distributed Caching Concepts
- Local Caching (In-Memory): Caching within the app process. Very fast but duplicates memory across instances and leads to inconsistencies.
- Remote/Distributed Caching: Separate cache cluster (e.g., Redis, Memcached) shared across all application instances.
- Consistent Hashing: How distributed caches map keys to specific server nodes to minimize reshreshuffling when nodes are added or removed.
- Replication vs. Partitioning (Sharding): Distributing the workload vs. duplicating data for high availability.
- Eventual Consistency: Managing the brief moments where a distributed cache may not exactly match the database.
5. Advanced Problems & Pitfalls
- Cache Stampede (Thundering Herd): When a highly requested key expires, and hundreds of threads simultaneously hit the database to recompute it.
- Solutions: Mutex locks, Probabilistic Early Expiration (PER), Background refreshing.
- Cache Penetration: Attackers intentionally requesting missing keys repeatedly, bypassing the cache and overwhelming the DB.
- Solutions: Caching null/empty results, using Bloom Filters.
- Cache Avalanche: When a massive number of keys have the exact same TTL and expire at the exact same moment, causing a sudden spike in DB load.
- Solutions: Adding jitter (randomness) to TTL values.
- Cache Pollution: When the cache is filled with data that is not frequently accessed, leading to a low hit ratio.
6. Caching Topologies & Layers
- Multi-level Caching (L1/L2): Combining in-process local caching (fastest, L1) with a distributed cache (L2) as a fallback layer.
- Client-Side Caching (Browser Cache): Cache-Control headers (
max-age,no-store), ETag, Last-Modified, and Service Workers. - Edge Caching (CDN): Serving static or semi-static content from edge nodes geographically close to users.
- API Gateway / Reverse Proxy Caching: Caching at the NGINX or Varnish layer before it even hits your application server.
7. Technologies
- Redis: Data structures cache (Strings, Hashes, Sorted Sets), persistence options, Pub/Sub.
- Memcached: Pure, multi-threaded, volatile key-value store.
- Caffeine / Ehcache / Guava: JVM-based local application caches.