Skip to content

AIC model caching#3684

Merged
markbackman merged 3 commits intopipecat-ai:mainfrom
ai-coustics:goedev/aic-model-caching
Feb 16, 2026
Merged

AIC model caching#3684
markbackman merged 3 commits intopipecat-ai:mainfrom
ai-coustics:goedev/aic-model-caching

Conversation

@gkmngrgn
Copy link
Copy Markdown
Contributor

@gkmngrgn gkmngrgn commented Feb 9, 2026

Summary

  • AIC model caching: Read-only AIC models are now shared across AICFilter instances via a singleton AICModelManager in aic_filter.py. Multiple filters using the same model path or (model_id, model_download_dir) share one loaded Model and avoid duplicate loads.
  • Reference counting: The manager acquires a reference when a filter calls acquire() in start() and releases it in stop(). When the last reference is dropped, the model is evicted from the cache.
  • Concurrent load deduplication: If several filters request the same uncached model at once, only one load runs; the rest await the same in-flight task and then use the cached result.
  • Internal helpers: Cache key is built by _get_cache_key(); reference updates use _increment_reference() and _store_new_reference() so logic is centralized and model is never optional where it is used.

Testing

  • uv run pytest tests/test_aic_filter.py tests/test_aic_vad.py -v — all 28 tests pass, including:
    • test_model_manager_reference_count: ref count increments on get_model, decrements on release, model evicted at 0.
    • test_model_manager_concurrent_load_deduplication: concurrent get_model calls for the same key trigger a single Model.from_file and all callers receive the same model.

No breaking changes: AICFilter’s public API (constructor and start/stop) is unchanged.

@gkmngrgn gkmngrgn marked this pull request as draft February 9, 2026 11:01
@codecov
Copy link
Copy Markdown

codecov Bot commented Feb 9, 2026

Codecov Report

❌ Patch coverage is 0% with 77 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/pipecat/audio/filters/aic_filter.py 0.00% 77 Missing ⚠️
Files with missing lines Coverage Δ
src/pipecat/audio/filters/aic_filter.py 0.00% <0.00%> (ø)

... and 86 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread src/pipecat/audio/filters/aic_filter.py Outdated
@gkmngrgn gkmngrgn marked this pull request as ready for review February 11, 2026 14:34
Copy link
Copy Markdown
Contributor

@markbackman markbackman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@markbackman markbackman merged commit dba4de7 into pipecat-ai:main Feb 16, 2026
6 checks passed
@gkmngrgn gkmngrgn deleted the goedev/aic-model-caching branch February 16, 2026 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants