Manual context summarization by filipi87 · Pull Request #3863 · pipecat-ai/pipecat

filipi87 · 2026-02-27T14:11:31Z

Summary

Added LLMSummarizeContextFrame: push this frame anywhere in the pipeline to trigger on-demand context summarization (e.g. from a function call tool).
Refactored LLMContextSummarizationConfig into two focused classes:
- LLMContextSummaryConfig — summary generation params (target_context_tokens, min_messages_after_summary, summarization_prompt); shared by both auto and manual modes.
- LLMAutoContextSummarizationConfig — auto-trigger thresholds (max_context_tokens, max_unsummarized_messages) plus a nested summary_config: LLMContextSummaryConfig.
Renamed LLMAssistantAggregatorParams fields for clarity:
- enable_context_summarization → enable_auto_context_summarization
- context_summarization_config → auto_context_summarization_config
LLMContextSummarizationConfig is now deprecated (emits DeprecationWarning); old LLMAssistantAggregatorParams field names are kept with deprecation warnings and auto-migration for one release cycle.
Added example 54b-context-summarization-manual-openai.py demonstrating on-demand summarization triggered via a function call tool.

Breaking Changes

LLMAssistantAggregatorParams.enable_context_summarization renamed to enable_auto_context_summarization.
LLMAssistantAggregatorParams.context_summarization_config renamed to auto_context_summarization_config and now accepts LLMAutoContextSummarizationConfig instead of LLMContextSummarizationConfig.

Migration

# Before
LLMAssistantAggregatorParams(
    enable_context_summarization=True,
    context_summarization_config=LLMContextSummarizationConfig(
        max_context_tokens=8000,
        target_context_tokens=6000,
        max_unsummarized_messages=20,
        min_messages_after_summary=4,
    ),
)

# After
LLMAssistantAggregatorParams(
    enable_auto_context_summarization=True,
    auto_context_summarization_config=LLMAutoContextSummarizationConfig(
        max_context_tokens=8000,
        max_unsummarized_messages=20,
        summary_config=LLMContextSummaryConfig(
            target_context_tokens=6000,
            min_messages_after_summary=4,
        ),
    ),
)

Testing

Run the new example to see manual summarization in action:

uv run examples/foundational/54b-context-summarization-manual-openai.py

Ask the bot to "summarize the conversation" — it will call the summarize_conversation function, which pushes an LLMContextSummaryRequestFrame into the pipeline. The LLM service generates the summary in a background task and the assistant aggregator compresses the context when the result arrives.

codecov · 2026-02-27T14:16:58Z

Codecov Report

❌ Patch coverage is 82.43243% with 13 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...t/processors/aggregators/llm_response_universal.py	52.94%	8 Missing ⚠️
...pipecat/utils/context/llm_context_summarization.py	88.23%	4 Missing ⚠️
src/pipecat/frames/frames.py	75.00%	1 Missing ⚠️

Files with missing lines	Coverage Δ
...t/processors/aggregators/llm_context_summarizer.py	`95.42% <100.00%> (+0.35%)`	⬆️
src/pipecat/frames/frames.py	`89.47% <75.00%> (-0.09%)`	⬇️
...pipecat/utils/context/llm_context_summarization.py	`84.12% <88.23%> (-0.59%)`	⬇️
...t/processors/aggregators/llm_response_universal.py	`79.65% <52.94%> (-0.15%)`	⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

kompfner · 2026-02-27T15:21:39Z

+    """Trigger manual context summarization via a pipeline frame."""
+    logger.info("Tool called: summarize_conversation")
+
+    summarization_config = LLMContextSummarizationConfig()


Having the user instantiate an LLMContextSummarizationConfig with no arguments just to then grab default values off of it feels a little awkward.

I wonder if LLMContextSummaryRequestFrame could internally create a LLMContextSummarizationConfig as a defaults provider in the case the user hasn't provided all fields?

Agreed. 👍

kompfner · 2026-02-27T15:40:11Z

+    summarization_config = LLMContextSummarizationConfig()
+    request_frame = LLMContextSummaryRequestFrame(
+        request_id=str(uuid.uuid4()),
+        context=params.context,


Our public frame-based APIs that affect the context (like LLMMessagesAppendFrame, etc) don't generally require the user to have to pass the context. For one, it's already clear which context they intend to update. Secondly, since it's a frame-based API, the effects occur at the time the frame is processed; an API that takes a context might imply that it will act on a "snapshot" of the context given to it at that time, which isn't how we want users thinking about it.

I understand why you chose to make LLMContextSummaryRequestFrame the public interface for this new manual summarization functionality: it already exists, and is used internally. But the use-case we want to enable is subtly different.

LLMContextSummaryRequestFrame is used ask an LLM to generate a summary of a given context

What we need is a new way to trigger the pipeline (the aggregator? the summarizer?) to then trigger the same code path that an auto summarization would trigger. What if we had a new LLMSummarizeContext frame that LLMContextSummarizer.process_frame handled, causing it to trigger a request summarization (with frame-provided options)? This approach would have a few advantages:

it would make greater re-use of more existing code paths

it would not require users to generate a request_id either

it would update the LLMContextSummarizer's internal bookkeeping (like _summarization_in_progress), which I assume is valuable

+1 to this feeback

See a related idea in another comment about refactoring LLMContextSummarizationConfig, below.

Ah...the more I think about it, we really do need to make sure LLMContextSummarizer's bookkeeping is updated on a manual summarization, since you can have both automatic and manual summarization going on in an app.

Yep. I believe I have implemented all the suggestions that you did here.

kompfner · 2026-02-27T15:50:52Z

@@ -56,17 +63,23 @@ def __init__(
        *,
        context: LLMContext,
        config: Optional[LLMContextSummarizationConfig] = None,


I think LLMContextSummarizationConfig could use a refactor to separate out params related to triggering auto-summarization (e.g. max_context_tokens) and those relevant to the summary generation itself and therefore relevant to both auto and manual summarization (e.g. summarization_prompt, min_messages_after_summary). Otherwise this will be messy to maintain.

Maybe we have something like:

LLMAutoContextSummarizationConfig, which contains the auto trigger related fields, as well as a nested field that is of type:

LLMContextSummaryConfig, which contains the summary generation related fields

I would then rename this field auto_summary_config or something to distinguish it from the new summary config, which will be able to differ between each manual summarization request

Actually, another benefit of the above suggested refactor: we could then use LLMContextSummaryConfig directly in the new frame that users use to request a summary 💡 .

I believe I have implemented all the suggestions that you did here. Let me know if this is what you had in mind.

kompfner · 2026-02-27T15:54:41Z

+        # Context summarization — always create the summarizer so that manually
+        # pushed LLMContextSummaryRequestFrame results are always handled.
+        # Auto-triggering based on thresholds is only enabled when
+        # enable_context_summarization is True.


I think we should rename to auto_context_summarization, and rename context_summarization_config to auto_context_summarization_config, to disambiguate between the per-summarization-request configs the user can provide.

I have renamed it to enable_auto_context_summarization and auto_context_summarization_config.

We usually use the enable_ prefix for features that we are turning on, so I thought it would be better to do the same here.

filipi87 · 2026-02-27T21:02:39Z

@markbackman, @kompfner, I think it’s ready for another round of review.

I have also already merged in the latest changes from main, which include Mark’s updates to context summarization.

kompfner · 2026-02-27T21:18:17Z

+    """Trigger manual context summarization via a pipeline frame."""
+    logger.info("Tool called: summarize_conversation")
+    await params.result_callback({"status": "summarization_requested"})
+    await params.llm.queue_frame(LLMSummarizeContextFrame())


kompfner · 2026-02-27T21:22:30Z

        Args:
            context: The LLM context to monitor and summarize.
-            config: Configuration for summarization behavior. If None, uses default config.
+            config: Auto-summarization configuration controlling both trigger


nit: do we want to call this auto_config or something other than config, which might imply LLMContextSummaryConfig? (I don't feel super strong about this, as it's internal)

kompfner · 2026-02-27T21:22:58Z

+                a per-request :class:`~pipecat.utils.context.llm_context_summarization.LLMContextSummaryConfig`.
        """
+        if self._summarization_in_progress:
+            logger.debug(f"{self}: Summarization already in progress, ignoring manual request")


markbackman

LGTM! Nice clean up 👏

kompfner

Excellent! Left a couple of tiny nits. This looks great! No need to loop back.

…ine to trigger on-demand context summarization (e.g. from a function call tool).

…MContextSummaryConfig and LLMAutoContextSummarizationConfig.

…g on-demand summarization triggered via a function call tool.

filipi87 · 2026-02-27T21:46:15Z

Thank you @kompfner , @markbackman for the great review. 🙌🚀

filipi87 marked this pull request as ready for review February 27, 2026 14:13

filipi87 requested review from aconchillo, kompfner and markbackman February 27, 2026 14:17