Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ Versions below 1.0 are pre-production — API and file formats may change.

### Added

- **Interactive force-directed knowledge graph viewer** (#118) — upgraded `llmwiki/graph.py`'s HTML template into a full interactive viewer per Karpathy's spec. New capabilities on top of the existing vis-network force layout: **live search** input in the header filters nodes by label/id (dims non-matches); **click-to-navigate** opens the wiki page in a new tab, rewriting `wiki/entities/Foo.md` → `entities/Foo.html`; **stats overlay** (bottom-right panel) shows page/edge/orphan counts, average connections, and top-5 hubs; **orphan highlighting** draws a red border (3 px) around nodes with zero inbound links; **cluster toggle** groups nodes by type (sources / entities / concepts / syntheses) and un-clusters on re-click; **dark/light theme toggle** that mirrors the main site's `localStorage.theme` key — both palettes drive the same CSS custom properties so the viewer follows the site without a rebuild; **offline fallback notice** if vis-network CDN fails to load. New `copy_to_site()` helper wires the viewer into the static site build so `python3 -m llmwiki build` now writes `site/graph.html` and the main site nav exposes a "Graph" link between "Compare" and "Changelog". Template is XSS-defensive: stats panel uses `escapeHtml()` on user-supplied labels and `write_html()` escapes literal `</script>` in the embedded JSON payload. 25 tests cover: graph builder edge cases (orphans, broken edges, alias-pipe wikilinks, README exclusion), every interactive feature (search input, click handler, stats overlay ids, cluster toggle, theme-toggle + localStorage, CSS-var theming, orphan highlight, offline notice, legend), `write_html()` JSON injection, `write_html()` `</script>` escaping, `copy_to_site()` (writes, returns None on empty wiki, rebuilds graph when omitted), site-nav integration, and a 25 kB template-size budget guardrail.

- **Prompt caching + batch API scaffold** (#50) — new `llmwiki/cache.py` module lands the plumbing for Anthropic `cache_control: {type: "ephemeral"}` usage on the stable ingest prefix (CLAUDE.md schema + `wiki/index.md` + `wiki/overview.md`). Public surface: `make_cached_block()`, `make_plain_block()`, `CachedPrompt` (frozen dataclass with `stable_prefix` / `dynamic_suffix`), `build_messages()` that emits the Anthropic-shaped message array with the header on the prefix block only. Cost preview: `estimate_tokens()` (char/4 heuristic, stdlib-only — no tokenizer dep), `estimate_cost()` returning a `CostEstimate` with per-bucket (prefix / fresh / output) breakdown, `format_estimate()` for the `--estimate` CLI output, `warn_prefix_too_small()` that flags prefixes below the 1024-token cache floor, `MODEL_PRICING` rate card for Sonnet 4.6 / Haiku 4 / Opus 4 (input, cached_input, cache_write, output USD/MTok). Batch state persistence: `BatchJob`, `BatchState`, `load_batch_state()`, `save_batch_state()`, `add_pending()` (dedup by batch_id), `mark_completed()` — all round-tripped through `.llmwiki-batch-state.json` (gitignored). New `llmwiki synthesize --estimate` CLI flag walks the discovered raw sessions, prices the batch assuming the first call is a cache write and the rest are hits, prints a line-item breakdown plus total. Docs: `docs/reference/prompt-caching.md`. 49 tests cover: cache-block shape, CachedPrompt empty-edge cases, build_messages structure, token/cost math (invariant: cached_input < input for every model, breakdown sums to total, rejects unknown models + negative tokens), batch-state round-trip, `add_pending` dedup, CLI wiring.

- **Ollama backend scaffold for local LLM synthesis** (#35) — new `llmwiki/synth/ollama.py` delivers the `OllamaSynthesizer` backend against the existing `BaseSynthesizer` contract. Stdlib-only HTTP via `urllib` (no new dependency). Configurable through `sessions_config.json` → `synthesis.backend = "ollama"` with `model` / `base_url` / `timeout` / `max_retries` fields (defaults: `llama3.1:8b` at `http://127.0.0.1:11434`, 60s timeout, 3 retries with exponential backoff). Privacy-by-default: loopback host only; a warning logs once if the user points the backend at a non-local host. `is_available()` probes `/api/tags` so callers can branch before long synthesis runs. Graceful error handling: `OllamaUnavailableError` (connection refused / DNS failure — no retries, caller skips), `OllamaHTTPError` (non-2xx after retries), `OllamaError` (non-JSON body, non-string response field). New `resolve_backend()` in `pipeline.py` selects backend from config (`dummy` | `ollama`); unknown names fall back to dummy with a warning. New `llmwiki synthesize [--check | --dry-run | --force]` CLI subcommand surfaces backend status without running synthesis. 43 tests (mocked HTTP — no network in CI): config parsing, URL construction, availability probing, retry + backoff on 5xx and socket timeout, no-retry on 4xx / connection refused, non-JSON response handling, unicode round-trip, curly-brace-safe prompt rendering, CLI registration, resolver fallback.
Expand Down
11 changes: 11 additions & 0 deletions llmwiki/build.py
Original file line number Diff line number Diff line change
Expand Up @@ -462,6 +462,7 @@ def link(href: str, label: str, key: str) -> str:
{link("sessions/index.html", "Sessions", "sessions")}
{link("models/index.html", "Models", "models")}
{link("vs/index.html", "Compare", "vs")}
{link("graph.html", "Graph", "graph")}
{link("changelog.html", "Changelog", "changelog")}
<button class="nav-search-btn" id="open-palette" aria-label="Open command palette">
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="11" cy="11" r="8"/><line x1="21" y1="21" x2="16.65" y2="16.65"/></svg>
Expand Down Expand Up @@ -1653,6 +1654,16 @@ def build_site(
except Exception as e:
print(f" warning: AI exports failed: {e}", file=sys.stderr)

# v1.1 (#118): copy the interactive knowledge graph into the site
# so the "Graph" nav link works without a separate `llmwiki graph` step.
try:
from llmwiki.graph import copy_to_site as copy_graph_to_site
graph_path = copy_graph_to_site(out_dir)
if graph_path:
print(f" wrote {graph_path.relative_to(out_dir.parent)} (interactive graph viewer)")
except Exception as e:
print(f" warning: graph viewer copy failed: {e}", file=sys.stderr)

# v0.4: Per-page sibling .txt and .json
try:
from llmwiki.exporters import write_page_txt, write_page_json
Expand Down
Loading
Loading