INTRODUCTION: THE CACHE TRILEMMA

In Lessons 7.1 through 7.5, we built a high-performance, secure, and resilient API Gateway. We tuned the latency to 2.38ms, implemented rate limiting, circuit breakers, and DDoS protection. However, the API Gateway is still forwarding every request to the backend microservices. The Consent Service, Account Service, and Payment Service are stateless and can handle the load, but they are not free. Each request to the Account Service triggers a database query (SELECT * FROM accounts WHERE id = ?). A database query takes 5-10ms. For 10,000 RPS, this translates to 50,000 ms (50 seconds) of database time per second—which is not feasible.

The solution is caching. By caching frequently accessed data (OpenAPI documentation, account balances, transaction lists, and consent statuses), we can serve requests from the cache (Redis or CDN) without hitting the database. This reduces the load on the backend microservices, lowers latency (from 10ms to 1ms for Redis), and improves overall system throughput.

However, caching introduces the Cache Trilemma: Freshness, Consistency, and Performance. You can only have two of the three at any time. A cache with a short TTL (fresh) is consistent but has low performance (many cache misses). A cache with a long TTL (performant) has stale data. The stale-while-revalidate strategy (RFC 5861) solves this by serving stale data while asynchronously refreshing the cache, providing both performance and eventual consistency.

This lesson teaches you how to implement a multi-layer caching strategy for Open Banking. We use a CDN (Content Delivery Network) to cache static assets (OpenAPI documentation, CSS, JavaScript). We use a distributed Redis cluster to cache dynamic data (account balances, transaction lists, consent statuses). We formalise the cache invalidation algebra—defining TTL-based invalidation (time-to-live) and event-based invalidation (purging the cache when data changes). We derive the optimal cache TTL using the stale-while-revalidate strategy, balancing freshness against performance. We also quantify the cache hit ratio (expected to be > 95% for account balances) and the cache miss penalty (which is the cost of a database query, 10ms). Finally, we prove that the caching layer reduces the total p95 latency of a read request from 12ms to 2ms, a 6x improvement.

LEARNING OBJECTIVES

1. Define the Multi-Layer Caching Strategy—designing a 3-layer cache: Layer 1 (CDN: static assets), Layer 2 (Redis: dynamic data), and Layer 3 (In-Memory: hot data). We will map each Open Banking resource to the appropriate cache layer (static assets → CDN, account balances → Redis, consent statuses → Redis, transaction lists → Redis with shorter TTL).

2. Implement the Stale-While-Revalidate Strategy (RFC 5861)—defining the cache control headers: Cache-Control: max-age=60, stale-while-revalidate=300, and deriving the mathematical formula that balances the freshness window (max-age) against the revalidation window (stale-while-revalidate).

3. Formalize the Cache Invalidation Algebra—defining TTL-based invalidation (expiry = creation_time + TTL) and event-based invalidation (purging the cache on data update), and proving that event-based invalidation guarantees 100% consistency for write-through caches.

4. Quantify the Cache Hit Ratio—using the Zipf distribution (or the Pareto principle, 80/20 rule) to model the request distribution, and deriving the expected cache hit ratio (≥ 95% for account balances, ≥ 80% for transaction lists).

5. Calculate the Optimal Cache TTL—using the stale-while-revalidate parameters: max-age = 60 seconds (freshness window) and stale-while-revalidate = 300 seconds (revalidation window), and proving that this configuration balances freshness (1 minute) against performance (reduced database load).

6. Design the Cache Warming Strategy—pre-warming the Redis cache with the most frequently accessed accounts (using the top-K algorithm) during system startup, to avoid cold-start latency spikes.

7. Quantify the Latency Reduction—measuring the end-to-end latency of a cached request (2ms: Redis GET + JSON deserialization) versus an uncached request (12ms: database query + serialization), and proving that the caching layer reduces the total p95 latency by 5x.

8. Analyze the Cache Consistency Trade-off—deriving the probability of serving stale data using the exponential distribution of data update events, and proving that with max-age = 60s, the probability of serving data older than 60 seconds is < 1%.

PART 1: THE MULTI-LAYER CACHING STRATEGY — CDN, Redis, and In-Memory

We use a 3-layer cache architecture:

+-----------------------------------------------------------------------+
|                MULTI-LAYER CACHING STRATEGY                             |
+-----------------------------------------------------------------------+
|                                                                        |
|  Layer 1: CDN (CloudFlare / AWS CloudFront)                            |
|  +------------------------------------------------------------------+  |
|  |  • Content: Static assets (OpenAPI JSON, CSS, JavaScript).        |  |
|  |  • TTL: 1 hour (3600 seconds).                                   |  |
|  |  • Invalidation: Manual purge (when OpenAPI spec changes).       |  |
|  |  • Latency: 5ms (edge to user).                                   |  |
|  +------------------------------------------------------------------+  |
|          |                                                            |
|          v                                                            |
|  Layer 2: Redis (Distributed Cache)                                   |
|  +------------------------------------------------------------------+  |
|  |  • Content: Dynamic data (account balances, transaction lists,    |  |
|  |    consent statuses).                                             |  |
|  |  • TTL: 60 seconds (max-age).                                     |  |
|  |  • Invalidation: TTL-based + event-based (write-through).         |  |
|  |  • Latency: 1.2ms (Redis GET).                                   |  |
|  +------------------------------------------------------------------+  |
|          |                                                            |
|          v                                                            |
|  Layer 3: In-Memory (Local Cache)                                     |
|  +------------------------------------------------------------------+  |
|  |  • Content: Hot data (most frequently accessed accounts).         |  |
|  |  • TTL: 30 seconds.                                               |  |
|  |  • Invalidation: TTL-based.                                       |  |
|  |  • Latency: 0.05ms (in-memory lookup).                           |  |
|  +------------------------------------------------------------------+  |
|                                                                        |
+-----------------------------------------------------------------------+

Mapping Resources to Cache Layers:

PART 2: THE STALE-WHILE-REVALIDATE STRATEGY (RFC 5861)

The stale-while-revalidate strategy allows the cache to serve stale data while asynchronously refreshing it in the background.

The Cache-Control Header:

Cache-Control: max-age=60, stale-while-revalidate=300

• max-age=60: The data is considered fresh for 60 seconds after creation.

• stale-while-revalidate=300: For up to 300 seconds after the data becomes stale, the cache can serve the stale data while triggering a background revalidation.

The Flow:

+-----------------------------------------------------------------------+
|           STALE-WHILE-REVALIDATE FLOW                                  |
+-----------------------------------------------------------------------+
|                                                                        |
|  Request arrives at t=0.                                              |
|  Cache has data with creation_time = t0.                             |
|                                                                        |
|  if (t - t0) < 60 (max-age):                                          |
|      → Serve fresh data from cache.                                  |
|                                                                        |
|  if (60 ≤ (t - t0) < 360 (max-age + stale-while-revalidate)):        |
|      → Serve stale data from cache.                                 |
|      → Trigger background revalidation (async fetch from DB).        |
|                                                                        |
|  if (t - t0) ≥ 360:                                                   |
|      → Cache miss. Block on database fetch.                          |
|                                                                        |
+-----------------------------------------------------------------------+

Mathematical Derivation:

Let λ be the rate of data updates per second. The probability that the data is updated within a freshness window T_fresh is:

P(Update ≤ T_fresh) = 1 - e^(-λ × T_fresh)

For λ = 1 / 300 (one update every 5 minutes), and T_fresh = 60 seconds:
P(Update ≤ 60) = 1 - e^(-60/300) = 1 - e^(-0.2) = 1 - 0.819 = 0.181 = 18.1%.

Conclusion: With a 60-second freshness window, only 18.1% of data is updated within the freshness window. The remaining 81.9% of requests are served stale (but within the revalidation window). This is acceptable for read-heavy workloads like account balances.

PART 3: THE CACHE INVALIDATION ALGEBRA — TTL vs. Event-Based

TTL-Based Invalidation:

The cache entry expires after a fixed time TTL. The expiration time is:

Expiry = Creation_Time + TTL

Event-Based Invalidation:

When the data is updated (e.g., a new transaction is posted), the system sends an event to the cache to purge the entry. This is a write-through cache.

Example:

1. PSU makes a payment. The Payment Service updates the database.
2. Payment Service emits a "txn.created" event to Kafka.
3. The Cache Invalidation Service consumes the event and deletes the cache key: "balance:account:123".
4. The next request for the balance triggers a cache miss and fetches the fresh data from the database.

Consistency Guarantee:

Event-based invalidation guarantees 100% consistency for write-through caches. The cache is always up-to-date with the database.

PART 4: THE CACHE HIT RATIO — The Zipf Distribution

We model the request distribution using the Zipf distribution (or the Pareto principle, 80/20 rule). 80% of requests are for 20% of accounts (the “hot” accounts).

Cache Hit Ratio:

For a Redis cache with a TTL of 60 seconds:

• Hot accounts (top 20%) : Hit ratio = 95% (most requests are served from cache).

• Cold accounts (remaining 80%) : Hit ratio = 60% (fewer requests, more cache misses).

Overall Hit Ratio:
Hit_Ratio = 0.8 × 0.95 + 0.2 × 0.60 = 0.76 + 0.12 = 0.88 = 88%.

With event-based invalidation (write-through), the hit ratio increases to 95% for hot accounts.

PART 5: THE OPTIMAL CACHE TTL — Balancing Freshness and Performance

We use the following cache TTLs:

Rationale:

• Account Balances: Change frequently (updates per minute). 60-second TTL balances freshness with performance.

• Transaction Lists: New transactions are rare (a few per hour). 60-second TTL is acceptable.

• Consent Statuses: Change rarely (once per 90 days). 300-second TTL is safe.

PART 6: LATENCY REDUCTION — The Performance Impact

Average Latency (88% hit ratio) :
Avg = 0.88 × 1.7ms + 0.12 × 9.7ms = 1.496ms + 1.164ms = 2.66ms.

Conclusion: The caching layer reduces the average latency from 8.5ms (database) to 2.66ms, a 69% improvement.

CLOSING — THE PERFORMANCE ACCELERATOR

The multi-layer caching strategy is the performance accelerator of the Open Banking API. The CDN serves static assets with sub-5ms latency. The Redis cache serves dynamic data with 1.2ms latency. The stale-while-revalidate strategy ensures that stale data is served while the cache is refreshed in the background. The cache hit ratio (88%) reduces the average latency from 8.5ms to 2.66ms.

Key Takeaways:

• CDN: TTL 1 hour for static assets.

• Redis: TTL 60 seconds, stale-while-revalidate 300 seconds.

• Hit Ratio: 88% overall, 95% for hot accounts.

• Latency: Average 2.66ms (cached) vs. 8.5ms (uncached).

Transition to Lesson 7.7: With the caching layer in place, we now turn to Request Compression, Payload Optimisation, and Binary Protocols (Protobuf vs. JSON) —how to reduce the payload size of the API responses by enabling gzip compression, using Protocol Buffers (Protobuf) for serialization, and minimising the JSON footprint. We will quantify the bandwidth savings (70% reduction) and the latency impact (5ms for compression/decompression).