fix(agent): prevent history.jsonl bloat from raw_archive and stuck consolidation

[chengyongru]

Summary

Four changes to prevent history.jsonl from poisoning the system prompt:

• Remove _cap_consolidation_boundary: the 60-message cap caused consolidation to get stuck on sessions with long tool chains (200+ iterations), because it would search backward for a user message that didn't exist within the cap range, return None, and abort the entire consolidation loop
• Truncate archive() input: use tiktoken to precisely cap formatted text to the model's input token budget before sending to the consolidation LLM
• Truncate raw_archive() output: cap history.jsonl entries at 16K chars as a safety net
• Cap recent history in build_system_prompt(): truncate the "Recent History" section to 32K chars so accumulated entries can't bloat the system prompt

Root Cause

Cascading failure chain: consolidation stuck (60-msg cap) → auto-compact mass-archive → consolidation LLM also fails → raw_archive() writes ~1MB untruncated dump to history.jsonl → build_system_prompt() injects it into every system prompt → all LLM calls permanently exceed 200K context window (error 1261)

Test plan

• All 2146 tests pass, 0 failures
• New TestRawArchiveTruncation (3 tests): verifies raw_archive truncates large content, preserves small content, respects custom max_chars
• New TestArchiveTruncation (2 tests): verifies archive truncates formatted text to token budget
• Updated existing cap-related tests to verify behavior without _cap_consolidation_boundary

[Re-bin]

This is the right kind of fix — it addresses a real cascading failure chain and puts the right guardrails in the right places.

What this PR does

Breaks a cascading failure where stuck consolidation (60-msg cap couldn't find a user turn) → mass raw_archive dumps → 1MB entries in history.jsonl → system prompt permanently exceeds context window. Four targeted changes:

1. Remove _cap_consolidation_boundary — the 60-message cap was fundamentally flawed: with long tool chains (200+ assistant turns), it searched backward for a user turn that didn't exist within the cap range, returned None, and aborted the entire consolidation loop. pick_consolidation_boundary alone is the correct gatekeeper.
2. Truncate archive() input — tiktoken-based truncation to the model's input token budget before sending to consolidation LLM.
3. Truncate raw_archive() output — 16K char cap as a safety net on history.jsonl entries.
4. Cap Recent History in build_system_prompt() — 32K char hard cap, defense-in-depth so accumulated entries can't bloat the system prompt.

Diff scope

4 files: nanobot/agent/memory.py, nanobot/agent/context.py, tests/agent/test_consolidator.py, tests/agent/test_context_prompt_cache.py. +145/−45.

Testing

• All 2363 tests pass (0 failures) after merging latest main.
• I added 2 focused regression tests that were missing:
  • test_archive_truncates_via_tiktoken_with_positive_budget — exercises the tiktoken path with a realistic positive budget (the original 2 archive truncation tests both hit negative budget → char-based fallback, leaving the production-primary tiktoken path uncovered).
  • test_recent_history_truncated_at_max_chars — locks the new _MAX_HISTORY_CHARS boundary in build_system_prompt().

Verdict

Direction is right, boundaries are clean, implementation solves the actual cascading problem without over-engineering. Ready to merge.