more full prompt reprocessing in recent update?

[fantasyz]

What happened?

I noticed after recent update, it is more frequent that my local LLM is doing more full prompt reprocessing when just continuing a conversation.

Here is what I saw from llama.cpp console:

155.00.988.881 W slot update_slots: id  0 | task 37846 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)

I also notice that an auto-complete for next prompt feature is introduced recently. I wonder if this is related. Anyway, I want to know how to disable that feature even if it doesn't help. I don't need it and want to see if issue can be fixed by disabling it.

What did you expect to happen?

No full prompt reprocessing when just continuing a conversation.

Client information

Client Information

Run qwen to enter the interactive CLI, then run the /about command.

$ qwen /about
Qwen Code v0.18.5
Model: qwen3.6-27b
Fast Model: not set
Auth: openai
Platform: win32 x64 (10.0.26200)
Node.js: v22.22.2
Session: 738e8cbf-0d10-42a5-9b54-91310e77bbdf
Git commit: 2937b09cf
LSP: disabled

Login information

n/a

Anything else we need to know?

No response

[qwen-code-ci-bot]

Triage

• Type: bug — prompt KV cache invalidation causing full reprocessing with local LLM backends
• Labels: type/bug, category/performance, scope/caching
• Reproduced: no (requires local llama.cpp server — cannot reproduce in CI)
• Confidence: confirmed by code analysis — multiple sources of prefix instability exist

Root Cause Analysis

The llama.cpp warning forcing full prompt re-processing due to lack of cache data fires when the new request's prompt prefix doesn't match the cached KV state. Code analysis confirms several mechanisms in the CLI that can change the prompt content between turns:

High-probability causes for this user's scenario

1. Git status in system instruction — getMainSessionSystemInstruction() (client.ts:754) appends getCachedGitStatus() (branch + recent commits) to the system prompt. The git status is cached per-session, but refreshSystemInstruction() (client.ts:845) invalidates it. Any code path that calls refreshSystemInstruction() — working directory change, memory refresh, MCP tool addition — rebuilds the entire system prompt with potentially different git status content. This directly breaks prefix stability for the system message.

2. MCP tools connecting after session start — When an MCP server connects mid-session, drainPendingAddedMcpToolsReminder() (client.ts:934) injects a user-role history entry announcing the new tools. This inserts a new message into the history prefix, shifting all subsequent messages. PR feat(mcp): support MCP resources and reliably surface prompts #5544 in v0.18.5 ("support MCP resources and reliably surface prompts") may have increased the frequency of MCP-related injections.

3. Memory prefetch injection — tryConsumeMemoryPrefetch() (client.ts:2089) injects recalled memory as a system reminder prepended to the user message. If the prefetch returns different results between turns (or settles on one turn but not the next), the user message content changes.

4. Microcompaction — microcompactHistoryBeforeSend() (client.ts:1593) runs on ToolResult turns and can mutate history to reduce token usage. Any history mutation breaks prefix stability.

Lower-probability causes

5. Deferred tool revelation — When ToolSearch loads a deferred tool (tool-registry.ts:703), the tools parameter changes for the next API call. This doesn't change the message prefix, but llama.cpp's KV cache may also be sensitive to the full request shape.

6. System instruction refresh cascading — refreshSystemInstruction() is called from addWorkingDirectoryChangedContext() (client.ts:730) and other paths. Each refresh rebuilds the system prompt from scratch, potentially with different git status, user memory, or model-specific content.

About the "auto-complete for next prompt" feature

The follow-up suggestion feature (PR #5145, v0.18.4) runs as a separate forked LLM call via generatePromptSuggestion() — it does not modify the main conversation prompt. Disabling it will not fix the reprocessing issue.

Fix Direction

The core problem is that the CLI's prompt assembly pipeline was optimized for cloud API caching (Anthropic's cache_control, DashScope's X-DashScope-CacheControl) but does not account for the stricter prefix-matching requirements of local KV cache reuse in llama.cpp.

Potential fixes:

• Stabilize system instruction: avoid appending mutable git status to the system prompt, or move it to a later, non-prefix-critical position (e.g., a user-role reminder at the end of history).
• Batch MCP tool announcements: defer MCP tool reminders to session boundaries rather than injecting mid-conversation.
• Make memory prefetch deterministic: if the prefetch hasn't settled, skip injection rather than injecting nothing on one turn and something on the next.
• Add a "prefix stability" mode: a setting that disables all dynamic injections (git status, memory, date reminders) for users running local LLM backends where KV cache reuse matters more than context freshness.

中文说明

根因分析

llama.cpp 的警告 forcing full prompt re-processing due to lack of cache data 在新请求的 prompt 前缀与缓存的 KV 状态不匹配时触发。代码分析确认了 CLI 中有多种机制会在多轮对话之间改变 prompt 内容：

高概率原因

1. 系统指令中的 Git 状态 — getMainSessionSystemInstruction() 将 git 状态（分支 + 最近提交）追加到系统提示词末尾。refreshSystemInstruction() 会使缓存失效并重建整个系统提示词，可能包含不同的 git 状态内容，直接破坏系统消息的前缀稳定性。

2. MCP 工具在会话启动后连接 — 当 MCP 服务器在会话中途连接时，drainPendingAddedMcpToolsReminder() 会注入一条 user 角色的历史记录来通知新工具。这会在历史前缀中插入新消息，导致后续所有消息位移。v0.18.5 的 PR feat(mcp): support MCP resources and reliably surface prompts #5544（"support MCP resources and reliably surface prompts"）可能增加了 MCP 相关注入的频率。

3. Memory 预取注入 — tryConsumeMemoryPrefetch() 将召回的记忆作为系统提醒注入到用户消息前。如果预取在不同轮次之间返回不同结果，用户消息内容就会改变。

4. 微压缩 — microcompactHistoryBeforeSend() 在 ToolResult 轮次运行，可能改变历史记录，破坏前缀稳定性。

低概率原因

5. 延迟工具揭示 — ToolSearch 加载延迟工具时，tools 参数变化。不改变消息前缀，但可能影响 llama.cpp 的缓存匹配。

6. 系统指令级联刷新 — 多个代码路径调用 refreshSystemInstruction()，每次都会从头重建系统提示词。

关于"下一提示词自动补全"功能

后续建议功能（v0.18.4 的 PR #5145）通过单独的 forked LLM 调用运行，不会修改主对话 prompt。禁用它不会修复重处理问题。

修复方向

核心问题是 CLI 的 prompt 组装流水线针对云 API 缓存（Anthropic 的 cache_control、DashScope 的缓存控制）做了优化，但没有考虑到 llama.cpp 本地 KV 缓存复用的更严格的前缀匹配要求。

建议修复方向：

• 稳定化系统指令：避免将可变的 git 状态追加到系统提示词，或将其移到非前缀关键的位置（如历史记录末尾的 user 角色提醒）。
• 批量 MCP 工具通知：将 MCP 工具提醒延迟到会话边界，而非在对话中途注入。
• 使 Memory 预取确定性化：如果预取未完成，跳过注入而非在不同轮次注入不同内容。
• 添加"前缀稳定性"模式：为使用本地 LLM 后端的用户提供禁用所有动态注入（git 状态、memory、日期提醒）的设置。

— Qwen Code · qwen3.7-max

[qwen-code-ci-bot]

Thanks for the report. A few details that would help narrow this down:

1. Which version did you upgrade from? You mention this started after a recent update — was the previous Qwen Code version you were on, and did you also update llama.cpp around the same time?
2. llama.cpp version/build? The warning you shared references an upstream change in llama.cpp#13194 related to SWA/hybrid memory cache invalidation. Could you share your llama.cpp version or commit hash?
3. Does this happen in all conversations (including short ones), or mainly in longer sessions with many turns?

This will help determine whether the behavior change is on the Qwen Code side or the llama.cpp inference side.

[fantasyz]

1. I don't remember but it just start happening recently so probably just from 1 or 2 version older.
2. I am using a fork of llama.cpp called Beellama
   0.32 branch
   version: 10265 (c8da45a37)
   built with MSVC 19.44.35227.0 for Windows AMD64
3. It do. I just saw it happen at just 30k context.

[doudouOUC]

Thanks for the extra details. This is plausible as a performance regression with local OpenAI-compatible llama.cpp-style backends.

The strongest current lead is the follow-up suggestion side request path (AppContainer -> generateFollowUpSuggestions), not direct mutation of the main conversation history. Recent versions enable follow-up suggestions by default, and after a response completes the CLI can send an additional background LLM request using recent conversation history to generate the next prompt suggestion. On a local backend with a single/shared prompt-cache slot, that extra request can plausibly disturb KV cache reuse or appear as full prompt reprocessing in llama.cpp logs.

Safe workaround: disable it in user ~/.qwen/settings.json or workspace .qwen/settings.json:

{
  "ui": {
    "enableFollowupSuggestions": false
  }
}

If full reprocessing still happens with that disabled, the next things to isolate are dynamic prompt/history changes such as MCP tool reminders, memory/context reminders, or compaction.

中文说明

补充信息很有帮助。这个问题在本地 OpenAI-compatible / llama.cpp 类后端上是一个可信的性能回退。

目前最强的线索是 follow-up suggestion 的旁路请求路径（AppContainer -> generateFollowUpSuggestions），而不是主对话历史被直接修改。近期版本默认启用 follow-up suggestions；模型回复结束后，CLI 可能会额外发起一次后台 LLM 请求，用最近的对话历史生成下一条输入建议。在单 slot 或共享 prompt cache 的本地后端上，这个额外请求可能影响 KV cache 复用，或在 llama.cpp 日志里表现为 full prompt reprocessing。

安全的临时规避方式是在用户级 ~/.qwen/settings.json 或工作区 .qwen/settings.json 中关闭它：

{
  "ui": {
    "enableFollowupSuggestions": false
  }
}

如果关闭后仍然复现，下一步应继续隔离 MCP tool reminders、memory/context reminders、compaction 等会改变 prompt/history 的动态路径。

[tt-a1i]

I can take a small pass at this. I’m going to keep it narrow: skip the default follow-up suggestion side request for loopback OpenAI-compatible backends unless the user explicitly opts in, so local single-slot servers don’t get an extra background request after every turn.