Cache key versioning makes the cached representation’s contract explicit. A key should identify the tenant or security scope, resource identity, representation or query shape, schema version, and any policy dimension that changes the answer. Without that structure, two callers can collide, a deployment can deserialize an old value incorrectly, and invalidation cannot target the facts that changed.
Versioning and invalidation solve different problems. A new namespace prevents new code from interpreting incompatible old entries. An invalidation event removes or marks stale entries when authoritative data changes. Expiration bounds how long mistakes survive. A reliable design uses all three according to the cost of staleness, then controls cache misses so correctness measures do not overload the origin.
Define the cache contract before the key
For every cached item, record the authoritative source, owner, readers, value schema, freshness requirement, allowed staleness, security scope, population method, invalidation trigger, TTL, size bound, and behavior on cache failure. Separate entity caches from query-result and rendered-response caches. An entity update can map to one entity key; a search result may depend on many records and require tags, generation markers, or short expiration.
Microsoft’s Cache-Aside pattern describes reading from cache, loading misses from the data store, and invalidating after a source update. It also warns that expiration must fit access patterns: too short causes repeated source loads, while too long permits stale data. Treat those parameters as part of the data contract rather than client-specific tuning.
Design a versioned key schema
| Component | Example | Why it belongs | Failure if omitted |
|---|---|---|---|
| Environment or application | prod:catalog | Prevents deployment and service collision | Tests or another service overwrite production keys |
| Tenant or security scope | tenant-42 | Separates authorized views | Cross-tenant disclosure |
| Representation | product-summary | Distinguishes projections | Different schemas share bytes |
| Schema generation | v3 | Supports incompatible rollout | New reader parses old value |
| Identity or query digest | product-991 or sha256(...) | Addresses object or canonical query | Collisions and unbounded raw keys |
| Locale or policy dimension | en-IN:retail | Captures answer-changing context | Semantically wrong cache hits |
Keep key construction in an owned library or service boundary. Canonicalize query parameters, sorting, case, default values, and locale before hashing. Do not place secrets or personal data in keys because keys appear in logs and management tools. If a digest hides the query, retain safe diagnostic metadata separately. Include a tenant dimension whenever content or authorization differs, even if the current cache cluster is tenant-specific.
Choose generation, content, or schema versioning
Use a schema generation such as v3 when application code and cached value shape change together. Use immutable content-addressed names when the content itself determines identity, which works well for built assets. Use a generation pointer when invalidating a broad query family: increment a small namespace token and let old entries expire. Avoid a global generation bump for a local change because it creates an origin-wide miss storm.
HTTP caching has its own model. RFC 9111 defines cache storage, freshness, validation, invalidation, and controls such as Cache-Control. Validators such as ETag can revalidate a stored response, while application caches often store domain projections outside HTTP semantics. Do not assume that changing an internal Redis key invalidates browser, proxy, and CDN copies; coordinate every cache layer that can serve the representation.
Publish invalidation events from authoritative commits
Emit invalidation only after the source transaction commits, ideally through a transactional outbox. Include event ID, entity or aggregate identity, source version, changed dimensions, occurred time, schema version, and trace context. Consumers should delete a specific key, invalidate a dependency tag, or advance a generation. An event containing only clear cache tells consumers neither what changed nor whether a late event is obsolete.
Assume duplicate and out-of-order delivery. Deletion is naturally repeatable, but a late delete can evict a newly populated value. Store source version with the value and compare event version, or include the version in the key so old invalidation cannot touch new content. For set-on-event designs, use conditional writes that reject older versions. For high-risk data, bypass cache briefly after a command until the read model confirms the new version.
Roll out a new namespace without a stampede
| Phase | Read behavior | Write behavior | Gate |
|---|---|---|---|
| Observe | Read old namespace | Optionally shadow-write new format | Serialization and key cardinality verified |
| Warm | Read old, asynchronously prefill new | Write old and new through one owner | Origin load stays within reserve |
| Canary | Selected cohort reads new, falls back to old only if semantically safe | Populate new on miss | Hit rate, latency, and value comparison pass |
| Expand | Most traffic reads new | Stop new writes to old | No error or origin saturation regression |
| Retire | Read new only | Write new only | Old-read metric is zero for full TTL window |
| Clean | Delete or let old generation expire | Remove dual-path code | Rollback window closed and capacity reclaimed |
Warm only valuable keys, based on observed access and acceptable data handling. Use request coalescing or single-flight so one miss populates a key while peers wait. Add TTL jitter to prevent simultaneous expiry, cap concurrent origin loads, and serve stale data only where a documented stale-while-revalidate policy permits it. If the origin reserve is exhausted, controlled degradation is safer than unlimited cache refill.
At a CDN, versioned names are often preferable to broad invalidation. CloudFront’s versioned file guidance recommends identifiers in names or directories for update control. Its invalidation documentation notes that versioning supports rollback, distinct analysis, and control beyond edge caches, while invalidation forces a subsequent origin fetch. Choose with client-cache behavior and emergency removal requirements in mind.
Observe correctness and retire safely
Measure hit and miss rate by namespace and operation, origin amplification, load duration, value size, eviction, key cardinality, invalidation lag and failures, stale reads detected by sampled comparison, event duplicates, out-of-order rejection, and old-namespace access. A high hit rate can conceal incorrect values. Sample source-versus-cache comparisons using privacy-safe identifiers and business invariants.
Retire only after all application versions, workers, batch jobs, edge functions, and rollback artifacts stop reading the old generation for at least its maximum lifetime plus operational margin. Remove subscribers and dual writes, then delete old keys in a rate-limited job if capacity recovery matters. Record the highest source version processed and retain enough event history to rebuild after cache loss.
Negative caching requires its own policy. Caching not found can protect an origin from repeated misses, but a long negative TTL hides newly created records. Include the same tenant and representation scope, choose a short lifetime, and invalidate the negative key when creation commits. Distinguish authoritative absence from a dependency timeout; never cache a transient error as if the object does not exist.
Prevent cache penetration and unbounded key growth by validating identifiers before lookup, limiting query shapes, and applying admission policy to low-reuse items. Record value size and creation rate by namespace. A key schema that embeds unconstrained user input can turn the cache into expensive storage and make invalidation enumeration impossible. Hash only after canonical validation, and retain a bounded mapping for diagnostics where necessary.
Design cache outage behavior per operation. Some reads can fall back to the source under a concurrency limiter; others should fail fast to protect the database; a few can serve a previously validated stale value. Writes should never depend on cache availability for durability. Exercise cold start and total cache loss, measure origin recovery, and confirm that prewarming does not replay unauthorized or deleted data.
For multi-region caches, state whether invalidation is region-local, replicated, or independently consumed. Compare event checkpoints and lag by region, and prevent a failed region from returning with an old warm namespace. Global correctness may require routing a user to the authoritative region during transition rather than promising instantaneous invalidation across every edge.
Key takeaways
- Define source, freshness, security scope, value schema, invalidation, TTL, and failure behavior before naming keys.
- Include tenant, representation, generation, identity, and answer-changing dimensions in a canonical key.
- Publish invalidation after authoritative commit with stable identity and source version.
- Handle duplicate and late events so old invalidation cannot remove newer content.
- Warm and canary new namespaces with origin protection; retire only after every reader and rollback path is gone.
Frequently asked questions
Is a TTL enough for cache invalidation?
Only when bounded staleness up to the TTL is acceptable and origin load at expiry is controlled. Security, entitlement, pricing, or rapidly changing operational data often needs explicit invalidation or validation in addition to expiration.
Should the version be in the key or value?
Use the key for incompatible namespace or representation changes, and keep source version in the value for ordering and sampled correctness checks. Many robust designs use both because they answer different questions.
Should invalidation delete or update the cached value?
Deletion keeps the source authoritative and is simpler. Updating can reduce misses but requires the event to contain a complete authorized representation and conditional ordering. Choose per cache type, and never let a partial event create a value that appears complete.
Conclusion
Predictable cache change comes from explicit identity and time. Version keys for incompatible representations, invalidate from committed source changes, bound residual staleness with expiration, and protect the origin during rollout. With version-aware consumers and retirement evidence, cache migrations become controlled releases instead of global delete-and-hope events.