This note covers the structural mechanics of caching from the perspective of a lead developer, focusing heavily on system trade-offs, edge cases, and distributed architecture behaviors.
1. Time To Live (TTL)
TTL is your primary architectural dial for controlling data freshness and database load, not just a simple countdown timer.
- Business vs. Technical TTL:
- Business: Driven by product rules (e.g., “Exchange rates can be up to 5 minutes stale”).
- Technical: Acting as a fail-safe backstop. Even if you have perfect manual invalidation logic, you set a 24-hour TTL just in case a network failure leaves orphaned data in the cache.
- Static vs. Sliding Expiration:
- Static: Expires completely after minutes.
- Sliding (Idle): TTL resets every time the item is read. Excellent for user session tokens.
- The “Jitter” Strategy: Never give a massive batch of cached items the exact same TTL. They will all expire simultaneously, causing a DB spike (Cache Avalanche). Instead, add randomness (Jitter) to the TTL. If you want 60 minutes, assign a random TTL between 55 and 65 minutes.
2. Cache Hit vs. Cache Miss
You must design around the fact that a Cache Miss is significantly slower than not having a cache at all (due to the extra network hops: App Cache DB Cache App).
- Hit Ratio: Calculated as
Hits / (Hits + Misses). This is the most vital metric for caching. If your hit ratio is sub-70%, your cache is likely just adding latency overhead and infrastructure cost. You are aiming for > 90%.
The Cache Stampede
If a highly popular key (like a homepage config) expires under a load of 1,000 req/sec, all 1,000 threads will miss the cache simultaneously and slam the database. This is a Thundering Herd.
* **The Solution (Double-Checked Locking):**
1. All requests miss.
2. All requests try to acquire a lock (e.g., Mutex, Redis lock).
3. *One* request wins the lock. The rest wait.
4. The winner double-checks the cache, goes to the DB, saves to cache, and releases the lock.
5. The waiting requests wake up, check the cache again (**the double check**), get a Hit, and return without touching the DB.
3. Cache Invalidation Patterns
“There are only two hard things in Computer Science: cache invalidation and naming things.”
Invalidation relies on three main approaches:
- TTL (Passive): The lazy, safest approach. Let the data expire.
- Application-Driven (Write-Through / Delete-First): The application explicitly sends a
DELETEcommand to the cache immediately before or after a database write.
Dual-Write Failures
If the DB write succeeds, but the network drops before the cache is deleted, the cache is permanently stale. Never rely purely on this without a TTL safety net.
- Change Data Capture (CDC): The enterprise “Gold Standard”. The application only writes to the DB. A background worker (like Debezium) listens to the database transaction logs (e.g., Postgres WAL) and automatically issues the
DELETEto the cache. This guarantees Eventual Consistency, even if the cache temporarily drops offline.
4. Understanding Consistency Windows
When caching data, you are fundamentally accepting a system behavior known as Eventual Consistency. There will always be a micro-second window where the DB is updated, but the cache has not yet been invalidated.
- Read-Your-Own-Writes: To prevent user-experience issues (a user saving their profile but seeing old data because of that eventual consistency window), you flag their session. For the next 5 seconds, any reads from that specific user completely bypass the cache. Other users still hit the cache.
When to Avoid Caching (Strong Consistency)
If the transaction requires absolute strong consistency (e.g., checking an account balance before a financial transfer, or deducting inventory), do not use a look-aside cache! Query the primary datastore directly using ACID transactions and row-level locks. Exception: In hyper-scale systems (high-frequency trading), the cache itself (e.g., Redis, Apache Ignite) becomes the Primary Database (In-Memory Data Grid) using optimistic locking, deferring disk writes asynchronously.
5. Cache Warming and States
- Cold Cache: An empty cache. Highly dangerous to expose to full production traffic, as a 0% hit ratio will crush the primary database.
- Warm Cache: A cache deliberately pre-loaded with the most vital/frequent data before being exposed to live traffic (via a Cache Warming script).
- Hot Cache: A cache actively running under load with a high hit ratio protecting the database.
Common warming techniques???
6. Caching Strategies (Patterns)
The strategy dictates how data actually flows between your Application, Cache, and Database. As a lead, your primary decision relies on understanding Read vs. Write Heaviness and Tolerance for Data Loss.
It is critical to distinguish between Architecture (who does the work) and Write Strategy (how data is written).
The Overarching Architecture
Application vs. Infrastructure
It is crucial to understand who implements these strategies:
Cache-Aside is 100% up to the Application logic. A bare Redis instance does not know what “Cache-Aside” is. It is simply the pattern you write in your own code (e.g.,
if not in cache -> get from DB -> put in cache).Read-Through / Write-Through relies on the system automatically managing the DB connection on a cache miss. This is achieved via two main approaches:
- Using a Smart Cache Provider: Enterprise caches built specifically to connect to databases (e.g., Hazelcast and Apache Ignite natively support connecting to SQL databases for Read/Write-Through. DynamoDB DAX is natively integrated for DynamoDB). Using regular Redis requires Enterprise plugins (like Redis Data Integration) to query the DB directly.
- Using a Smart Framework (The modern approach): Application frameworks act as the “Cache Provider” abstraction, keeping business logic cleanly decoupled. For example, using Java with Spring Boot’s
@Cacheableor Hibernate L2 Cache. The framework secretly orchestrates the database interactions behind the scenes, effectively making a standard Redis instance behave like a Read-Through cache from the perspective of your business logic.
-
Cache-Aside (Lazy Loading): The most common pattern (80% of use cases). The Cache sits “to the side” and has no knowledge of the database. The Application orchestrates everything.
- Read Path: App asks Cache Miss App asks DB App updates Cache.
- Pros: Works with any standard dump cache. Complete control. Fails-over to the DB safely.
- Cons: Heavy boilerplate code traversing the application layer.
- Use Case: General purpose web APIs.
-
Read-Through:
- How it works: The App treats the Cache as the Database. On a miss, the Cache infrastructure itself (or the Framework) connects to the DB, queries it, saves the data, and returns it.
- Pros: Dramatically cleans up business logic code.
- Cons: Requires tightly coupled cache technology or heavy ORM framework reliance.
- Use Case: Microservices where you want caching logic completely decoupled from the app layer.
Write Strategies (Used inside the Architectures)
When using Cache-Aside, you must choose how your Application handles Database writes:
- Write-Through (Write-Update): Synchronous DB and Cache updating.
- How it works: App writes to DB, and then the App immediately writes that same data into the Cache.
The UX Compromise of Lazy Loading
In a mathematically pure “Lazy Loading” architecture, you should never Write-Through. You should just delete the cache key. However, we intentionally break lazy-loading rules here for UX. If a user updates their profile and the page reloads instantly, Write-Through ensures they get a lightning-fast Cache Hit, saving them the latency penalty of an immediate Cache Miss. * Pros: Perfect consistency. Zero stale reads. The next read is instantly fast. * Cons: High write latency. Causes “Cache Pollution” (filling expensive RAM with newly written data). * Use Case: High-visibility endpoints where you know the user will immediately request the changed data in the next millisecond.
-
Write-Around (Write-Invalidate): Bypassing the cache on write.
- How it works: App writes to the DB, and intentionally bypasses updating the cache. Instead, it sends a
DELETEAPI call to the cache to wipe the old data (this is standard Cache Invalidation). - Pros: Prevents cache pollution. If you bulk upload 10,000 historic invoices, they go to the DB but don’t eat up 10GB of your Redis RAM.
- Cons: The very first read of a newly written item incurs a Cache Miss latency penalty.
- Use Case: Bulk imports, audit logs, or data that is written but rarely read immediately.
- How it works: App writes to the DB, and intentionally bypasses updating the cache. Instead, it sends a
-
Write-Behind (Write-Back): The “Ferrari” write path—optimized purely for raw I/O speed.
- How it works: App writes to Cache. Cache instantly returns “Success” to the App. A background thread asynchronously flushes batches of these writes to the DB every few seconds.
- Pros: Blistering write performance. App never waits for the DB. Deduplicates writes (if a counter hits 10 in a second, it only commits “10” to the DB).
- Cons: Massive Danger of Data Loss. If the Cache server crashes before flushing, the data is permanently erased.
- Use Case: Huge telemetry streams, IoT sensor ingress, view counters (where losing 5 seconds of data is perfectly fine).