Skip to content

fix: revert buggy recommendation cache causing memory bloat and latency spike#8

Merged
AlexanderWert merged 1 commit intoai-demofrom
fix/recommendation-cache-memory-bloat
Mar 11, 2026
Merged

fix: revert buggy recommendation cache causing memory bloat and latency spike#8
AlexanderWert merged 1 commit intoai-demofrom
fix/recommendation-cache-memory-bloat

Conversation

@AlexanderWert
Copy link
Copy Markdown
Owner

@AlexanderWert AlexanderWert commented Mar 11, 2026

Problem

An active latency alert fired for frontend-proxy on transaction GET /api/recommendations:

  • Observed latency: 616 ms avg (last 5 min)
  • Threshold: 200 ms

Root Cause

Commit 375a36eb40eb812fa6b1110585cd863e3daf8535 introduced a get_recommendations_ids() caching function in src/recommendation/recommendation_server.py that contains a critical bug:

for x in cat_response.products:
    ids_to_add.extend(cached_ids)  # BUG: appends entire cached_ids for EACH product
    ids_to_add.append(x.id)
    if len(ids_to_add) + len(cached_ids) < MAX_CACHED_IDS:
        cached_ids = cached_ids + ids_to_add

On every cache miss, ids_to_add is extended with the entire cached_ids list for each product in the catalog. This causes O(n²) memory growth, rapidly bloating cached_ids up to MAX_CACHED_IDS (2,000,000 entries). The resulting memory pressure and list operations cause severe latency degradation on the recommendation service, which propagates up through frontendfrontend-proxy.

Fix

Reverts recommendation_server.py to the original direct product catalog lookup, removing the faulty caching logic entirely.

Impact

  • Affected service: recommendationfrontendfrontend-proxy
  • Alert: Latency threshold rule on GET /api/recommendations (616 ms vs 200 ms threshold)
  • Correlated via git.sha attribute in latency distribution analysis

…cy spike

The get_recommendations_ids function introduced in commit 375a36e contains
a critical bug: on every cache miss, it extends ids_to_add with the entire
cached_ids list for each product, causing O(n²) memory growth. This bloats
cached_ids up to MAX_CACHED_IDS (2,000,000) entries very quickly, leading
to severe latency degradation on GET /api/recommendations.

Reverts to the original direct product catalog lookup.
@AlexanderWert AlexanderWert merged commit fca6c27 into ai-demo Mar 11, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant