HNT-1890 (2/4): RedisCorpusCache + corpus_cache config + unit tests#1439
Open
mmiermans wants to merge 1 commit into
Open
HNT-1890 (2/4): RedisCorpusCache + corpus_cache config + unit tests#1439mmiermans wants to merge 1 commit into
mmiermans wants to merge 1 commit into
Conversation
This was referenced Apr 27, 2026
mmiermans
force-pushed
the
hnt-1890-cache-1-infra
branch
from
24aa7f7 to
94aec75
Compare
Adds the shared L2 cache class that wraps the Pocket Corpus GraphQL backends, plus its config defaults and Dynaconf validators. Not yet wired into the providers — consumer wiring lands in PR 4/4. With the default cache="none" this code is unreachable, so this PR is behavior- preserving on production today. Files: - merino/curated_recommendations/corpus_backends/redis_cache.py: RedisCorpusCache implementation (distributed SWR + SET NX EX lock). Accepts an injected CorpusCacheConfig dataclass — no settings import. - merino/configs/default.toml: [default.curated_recommendations.corpus_cache] with cache/soft_ttl_sec/hard_ttl_sec/lock_ttl_sec/key_prefix. - merino/configs/__init__.py: Dynaconf validators for the new section (allowed values, type/range checks). - tests/unit/.../test_redis_cache.py: 616-line unit suite covering SWR semantics, lock contention between concurrent callers, the cold-miss/503 path, stale deserialization, retry-after-lock-held, and error handling. Architecture and design tradeoffs are documented in docs/operations/curated-recommendations/corpus-cache.md (PR 1/4). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
mmiermans
force-pushed
the
hnt-1890-cache-2-impl
branch
from
e65da9d to
80f8f34
Compare
mmiermans
changed the title
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR 2/4 to implement shared caching of curated recommendations between pods.
Stack: 1/4 infra + architecture doc → 2/4 (this) → 3/4 integration tests → 4/4 wire-up.
References
JIRA: HNT-1890
Description
Implements
RedisCorpusCache— the distributed stale-while-revalidate (SWR) cache that wraps the Pocket Corpus GraphQL backends. Not yet wired into the providers in this PR; consumer wiring lands in PR 4/4. Withcache = "none"(the default added in this PR), the class is unreachable from production code, so this PR is behavior-preserving.The architecture and full rationale live in
docs/operations/curated-recommendations/corpus-cache.md(added in PR 1/4).What's in this PR
merino/curated_recommendations/corpus_backends/redis_cache.py(~322 LOC) —RedisCorpusCacheand theRedisCachedScheduledSurface/RedisCachedSectionsadapters. Accepts an injectedCorpusCacheConfigdataclass, so the class is independently testable without Dynaconf or app startup.merino/configs/default.toml— new[default.curated_recommendations.corpus_cache]section (cache="none",soft_ttl_sec=60,hard_ttl_sec=86400,lock_ttl_sec=30,key_prefix="curated:v1").merino/configs/__init__.py— Dynaconf validators for the new section (allowed values, type/range checks).tests/unit/curated_recommendations/corpus_backends/test_redis_cache.py(~616 LOC) — unit suite covering SWR semantics, lock contention between concurrent callers, cold-miss/503 path, stale deserialization, retry-after-lock-held, and error handling.Implementation decisions specific to this PR
CorpusCacheConfiginjectionredis_cache.pydecoupled frommerino.configs.settingsand trivially unit-testableIntegration tests (real Redis via testcontainers) ship in PR 3/4.
PR Review Checklist
[load test: (abort|skip|warn)]keywords applied (n/a — code unreachable until PR 4/4)