Caching Strategies

Caching stores copies of frequently accessed data in a faster storage layer. A well-designed caching strategy can reduce latency by orders of magnitude, lower database load, and dramatically improve throughput. This lesson covers caching patterns, technologies, and pitfalls.

---

Why Caching Matters

Without Cache:                      With Cache:

Client ──▶ Server ──▶ Database     Client ──▶ Server ──▶ Cache (hit!)
                      (50ms)                              (1ms)
                                                   │
                                                   ▼ (miss)
                                              Database
                                              (50ms)

Typical cache hit rate: 80-95%
Effective latency = 0.9 × 1ms + 0.1 × 50ms = 5.9ms

---

Caching Patterns

1. Cache-Aside (Lazy Loading)

The application is responsible for reading from and writing to the cache. Data is loaded into the cache only when requested.

┌────────┐    1. Check cache     ┌─────────┐
│  App   │──────────────────────▶│  Cache   │
│        │◀── 2a. Cache hit ─────│ (Redis)  │
│        │                       └─────────┘
│        │    2b. Cache miss
│        │──────────────────────▶┌─────────┐
│        │◀── 3. Return data ────│Database │
│        │                       └─────────┘
│        │──── 4. Write to cache─▶ Cache
└────────┘

def get_user(user_id: str) -> dict:
    # 1. Check cache
    cached = redis.get(f"user:{user_id}")
    if cached:
        return json.loads(cached)

    # 2. Cache miss — read from database
    user = db.query("SELECT * FROM users WHERE id = %s", user_id)

    # 3. Store in cache with TTL
    redis.setex(f"user:{user_id}", 3600, json.dumps(user))

    return user

Pros: Only caches data that is actually requested; resilient to cache failures. Cons: Cache miss results in three round trips; data can become stale.

2. Write-Through

Every write goes to the cache AND the database simultaneously. The cache is always up to date.

┌────────┐    1. Write       ┌─────────┐
│  App   │──────────────────▶│  Cache   │
└────────┘                   └────┬────┘
                                  │ 2. Write to DB
                                  ▼
                             ┌─────────┐
                             │Database │
                             └─────────┘

Pros: Cache is always consistent with the database. Cons: Higher write latency (two writes per operation); caches data that may never be read.

3. Write-Behind (Write-Back)

Writes go to the cache immediately and are asynchronously flushed to the database in batches.

┌────────┐    1. Write (fast) ┌─────────┐
│  App   │───────────────────▶│  Cache   │
└────────┘                    └────┬────┘
                                   │ 2. Async batch write
                                   ▼
                              ┌─────────┐
                              │Database │
                              └─────────┘

Pros: Very fast writes; reduces database write load. Cons: Risk of data loss if the cache fails before flushing; added complexity.

4. Read-Through

Similar to cache-aside, but the cache itself is responsible for loading data from the database on a miss. The application only talks to the cache.

┌────────┐    1. Read         ┌─────────┐    2. Load on miss    ┌─────────┐
│  App   │───────────────────▶│  Cache   │─────────────────────▶│Database │
│        │◀── 3. Return ──────│         │◀── 4. Return ─────────│         │
└────────┘                    └─────────┘                       └─────────┘

---

Comparing Caching Patterns

Pattern	Reads	Writes	Consistency	Complexity	Best For
Cache-aside	App checks	App writes	Eventual	Low	General purpose
Write-through	Auto	Sync both	Strong	Medium	Read-heavy, needs freshness
Write-behind	Auto	Async flush	Eventual	High	Write-heavy workloads
Read-through	Auto load	Varies	Eventual	Medium	Simplified app logic

---

Cache Invalidation

Cache invalidation is famously one of the two hard problems in computer science (along with naming things). There are several strategies:

Time-Based (TTL)

Set a time-to-live on each cache entry. After the TTL expires, the entry is evicted.

# Set a 1-hour TTL
redis.setex("product:123", 3600, json.dumps(product_data))

Event-Based

Invalidate the cache when the underlying data changes.

def update_product(product_id: str, data: dict):
    db.update("UPDATE products SET ... WHERE id = %s", product_id)
    redis.delete(f"product:{product_id}")  # Invalidate cache

Version-Based