Caching Strategies

Caching stores copies of frequently accessed data in a faster storage layer. A well-designed caching strategy can reduce latency by orders of magnitude, lower database load, and dramatically improve throughput. This lesson covers caching patterns, technologies, and pitfalls.

---

Why Caching Matters

graph TD
  subgraph WithoutCache["Without Cache"]
    C1["Client"] --> S1["Server"]
    S1 --> DB1["Database (50ms)"]
  end
  subgraph WithCache["With Cache"]
    C2["Client"] --> S2["Server"]
    S2 -->|"hit (1ms)"| Ca2["Cache"]
    S2 -->|"miss (50ms)"| DB2["Database"]
  end
  note["Typical hit rate 80-95%. Effective latency = 0.9 x 1ms + 0.1 x 50ms = 5.9ms"]

---

Caching Patterns

1. Cache-Aside (Lazy Loading)

The application is responsible for reading from and writing to the cache. Data is loaded into the cache only when requested.

graph LR
  App["App"] -->|"1. Check cache"| Cache["Cache (Redis)"]
  Cache -->|"2a. Cache hit"| App
  App -->|"2b. Cache miss"| DB["Database"]
  DB -->|"3. Return data"| App
  App -->|"4. Write to cache"| Cache

def get_user(user_id: str) -> dict:
    # 1. Check cache
    cached = redis.get(f"user:{user_id}")
    if cached:
        return json.loads(cached)

    # 2. Cache miss — read from database
    user = db.query("SELECT * FROM users WHERE id = %s", user_id)

    # 3. Store in cache with TTL
    redis.setex(f"user:{user_id}", 3600, json.dumps(user))

    return user

Pros: Only caches data that is actually requested; resilient to cache failures. Cons: Cache miss results in three round trips; data can become stale.

2. Write-Through

Every write goes to the cache AND the database simultaneously. The cache is always up to date.

graph TD
  App["App"] -->|"1. Write"| Cache["Cache"]
  Cache -->|"2. Write to DB"| DB["Database"]

Pros: Cache is always consistent with the database. Cons: Higher write latency (two writes per operation); caches data that may never be read.

3. Write-Behind (Write-Back)

Writes go to the cache immediately and are asynchronously flushed to the database in batches.

graph TD
  App["App"] -->|"1. Write (fast)"| Cache["Cache"]
  Cache -->|"2. Async batch write"| DB["Database"]

Pros: Very fast writes; reduces database write load. Cons: Risk of data loss if the cache fails before flushing; added complexity.

4. Read-Through

Similar to cache-aside, but the cache itself is responsible for loading data from the database on a miss. The application only talks to the cache.

graph LR
  App["App"] -->|"1. Read"| Cache["Cache"]
  Cache -->|"2. Load on miss"| DB["Database"]
  DB -->|"4. Return"| Cache
  Cache -->|"3. Return"| App

---

Comparing Caching Patterns

Pattern	Reads	Writes	Consistency	Complexity	Best For
Cache-aside	App checks	App writes	Eventual	Low	General purpose
Write-through	Auto	Sync both	Strong	Medium	Read-heavy, needs freshness
Write-behind	Auto	Async flush	Eventual	High	Write-heavy workloads
Read-through	Auto load	Varies	Eventual	Medium	Simplified app logic

---

Cache Invalidation

Cache invalidation is famously one of the two hard problems in computer science (along with naming things). There are several strategies:

Time-Based (TTL)

Set a time-to-live on each cache entry. After the TTL expires, the entry is evicted.

# Set a 1-hour TTL
redis.setex("product:123", 3600, json.dumps(product_data))

Event-Based

Invalidate the cache when the underlying data changes.

def update_product(product_id: str, data: dict):
    db.update("UPDATE products SET ... WHERE id = %s", product_id)
    redis.delete(f"product:{product_id}")  # Invalidate cache

Version-Based