feat(curator): background skill maintenance (issue #7816)#16049
Closed
teknium1 wants to merge 6 commits into
Closed
feat(curator): background skill maintenance (issue #7816)#16049teknium1 wants to merge 6 commits into
teknium1 wants to merge 6 commits into
Conversation
Adds the Curator — an auxiliary-model background task that periodically
reviews AGENT-CREATED skills and keeps the collection tidy: tracks usage,
transitions unused skills through active → stale → archived, and spawns
a forked AIAgent to consolidate overlaps and patch drift.
Default: enabled, inactivity-triggered (no cron daemon). Runs on CLI
startup and gateway boot when the last run is older than interval_hours
(default 24) AND the agent has been idle for min_idle_hours (default 2).
Invariants (all load-bearing):
- Never touches bundled or hub-installed skills (.bundled_manifest +
.hub/lock.json double-filter)
- Never auto-deletes — archive only. Archives are recoverable
via `hermes curator restore <skill>`
- Pinned skills bypass all auto-transitions
- Uses the aux client; never touches the main session's prompt cache
New files:
- tools/skill_usage.py — sidecar .usage.json telemetry, atomic writes,
provenance filter
- agent/curator.py — orchestrator: config, idle gating, state-machine
transitions (pure, no LLM), forked-agent review prompt
- hermes_cli/curator.py — `hermes curator {status,run,pause,resume,
pin,unpin,restore}` subcommand
- tests/tools/test_skill_usage.py — 29 tests
- tests/agent/test_curator.py — 25 tests
Modified files (surgical patches):
- tools/skills_tool.py — bump view_count on successful skill_view
- tools/skill_manager_tool.py — bump patch_count on skill_manage
patch/edit/write_file/remove_file; forget record on delete
- hermes_cli/config.py — add curator: section to DEFAULT_CONFIG
- hermes_cli/commands.py — add /curator CommandDef with subcommands
- hermes_cli/main.py — register `hermes curator` subparser via
register_cli() from hermes_cli.curator
- cli.py — /curator slash-command dispatch + startup hook
- gateway/run.py — gateway-boot hook (mirrors CLI)
Validation:
- 54 new tests across skill_usage + curator, all passing in 3s
- 346 tests across all touched files' neighbors green
- 2783 tests across hermes_cli/ + gateway/test_run_progress_topics.py green
- CLI smoke: `hermes curator status/pause/resume` work end-to-end
Companion to PR #16026 (class-first skill review prompt) — together
they form a loop: the review prompt stops near-duplicate skill creation
at the source, and the curator prunes/consolidates what still accumulates.
Refs #7816.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| print("curator: running review pass...") | ||
|
|
||
| def _on_summary(msg: str) -> None: | ||
| print(msg) |
| data[skill_name] = rec | ||
| save_usage(data) | ||
| except Exception as e: | ||
| logger.debug("skill_usage._mutate(%s) failed: %s", skill_name, e, exc_info=True) |
The LLM review prompt mentioned bespoke `archive_skill` and `pin_skill` tools that are not registered as model tools. Swap the prompt to rely on the real surface: - skill_manage action=patch — for patching and consolidation - terminal — to `mv` skill dirs into .archive/ Also drop `pin` from the model's decision list — pinning is a user opt-out for `hermes curator pin <skill>`, not something the model should do autonomously. Decision list is now: keep / patch / consolidate / archive. Tests updated: prompt-invariant test now asserts the existing tools are referenced and that bespoke tool names do NOT appear. New test prevents `pin` from being re-added as a model decision.
Previous invariants only gated the primary entry points
(apply_automatic_transitions, archive_skill, CLI pin). Several paths
were unprotected:
- bump_view / bump_use / bump_patch / set_state / set_pinned wrote
usage records unconditionally, which is confusing noise in
.usage.json even though the review list filtered them out
- restore_skill did not check whether a bundled skill now shadows
the archived name
- CLI unpin was asymmetric with CLI pin — it had no gate
Fixes:
- _mutate() (the shared counter / state writer) now drops silently
when the skill is not agent-created. .usage.json never gains a
record for a bundled or hub-installed skill.
- restore_skill() refuses to restore under a name that is now
bundled or hub-installed (would shadow upstream).
- CLI unpin gate matches CLI pin.
New tests:
- 5 provenance-guard tests on skill_usage (one per mutator)
- 1 end-to-end test that hammers every mutator at a bundled skill
and a hub skill, asserts both are untouched on disk, and asserts
the sidecar stays clean
- 2 CLI tests proving pin/unpin refuse bundled skills symmetrically
64/64 tests passing (29 skill_usage + 27 curator + 8 new guards).
alt-glitch
added
type/feature
Weekly is closer to how skill churn actually works — most agent-created skills don't change multiple times per day, so a daily review is pure cost without benefit. Bumping the default to 7 days reduces aux-model spend while still catching drift and staleness on the timescales that matter (30d stale, 90d archive). Changes: - DEFAULT_INTERVAL_HOURS: 24 -> 168 (7 days) - config.yaml default: interval_hours: 24 -> 24 * 7 - CLI status line renders as '7d' when interval is a whole-day multiple - Test `test_old_run_eligible` decoupled from the exact default: it now uses 2 * get_interval_hours() so future tweaks don't break it
Long-running gateways need the curator to fire on cadence without restarts. Piggy-back on the existing cron ticker thread (which already runs image/document cache cleanup every hour on the same pattern) instead of spawning a dedicated timer thread. - New CURATOR_EVERY = 60 ticks (poll hourly at default 60s interval). The inner config.interval_hours gate controls the real cadence, so 60 of these 60 hourly pokes are cheap no-ops and one runs the review. - Removed the boot-time call added in the prior commit — the ticker covers boot + every hour thereafter. Avoids double-running. Handles the weekly-default-on-24/7-gateway gap flagged in review.
| from agent.curator import maybe_run_curator | ||
| maybe_run_curator( | ||
| idle_for_seconds=float("inf"), | ||
| on_summary=lambda msg: logger.info("curator: %s", msg), |
Contributor
Author
|
Opened a higher-level RFC for design feedback: #16077. Code-level concerns welcome here on the PR; broader design questions (defaults, scope, enabled-by-default trade-off, etc.) please leave on the issue. |
This was referenced Apr 29, 2026
Contributor
Author
|
Superseded by #17277 which salvaged all 5 commits from this branch onto current main (your per-commit authorship preserved via rebase-merge) plus a follow-up commit with the umbrella-first prompt, parent-config inheritance fix, and unbounded iteration ceiling validated via 3 live runs on real skill collections. Closing this branch; the feature is now on main. |
teknium1
added a commit
that referenced
this pull request
Apr 29, 2026
…17563) Skill catalog pages (bundled/optional) were drowning out real user-guide and reference docs in search results. There are ~3100 of them and they match on almost every generic term. - Add `ignoreFiles` regexes to docusaurus-search-local for `user-guide/skills/bundled/` and `user-guide/skills/optional/`. The two human-written catalog indexes (`reference/skills-catalog`, `reference/optional-skills-catalog`) remain indexed. - Add a new feature page `user-guide/features/curator.md` covering the curator subsystem merged in #16049 and refined in #17307 (per-run reports): how it runs, config, CLI (`hermes curator status/run/pin/ restore/...`), `.usage.json` telemetry, archival semantics, and recovery. Slotted into the Core features sidebar next to Skills. Search index size dropped from 5822 docs to 2704 in the main section; `user-guide/features/curator` is indexed.
teknium1
added a commit
that referenced
this pull request
Apr 30, 2026
… fire import (#17927) Three fixes bundled for curator reliability on existing installs and broken/partial installs: 1. run_agent.py: defer `import fire` into the __main__ block. `fire` is only used by `fire.Fire(main)` when running run_agent.py directly as a CLI — it is NOT needed for library usage. Importing it at module top made `from run_agent import AIAgent` from a daemon thread (e.g. the curator's forked review agent) crash with ModuleNotFoundError on broken/partial installs where `fire` isn't present. 2. hermes_cli/config.py: add version 22 → 23 migration that writes the `curator` + `auxiliary.curator` sections to config.yaml with their defaults, only filling keys the user hasn't overridden. Existing configs from before PR #16049 / the April 2026 `auxiliary.curator` unification had neither section on disk, so users couldn't see or edit the settings in their config.yaml (runtime deep-merge papered over it at read time, but the file never reflected reality). 3. hermes_cli/config.py: `ensure_hermes_home()` now pre-creates `~/.hermes/logs/curator/` alongside cron/sessions/logs/memories on every CLI launch. Managed-mode (NixOS) variant mkdir's it defensively after the activation-script existence checks, since the activation script may not know about this subpath. 4. agent/curator.py: `_reports_root()` mkdir's the dir at call time as belt-and-suspenders for entry paths that bypass both ensure_hermes_home() and the v23 migration (gateway-only installs, bare library use). E2E validated in isolated HERMES_HOME: fresh install gets full defaults seeded; partial-override config keeps user's `enabled: false` and custom `interval_hours` while filling the missing keys; re-running the migration is a no-op.
donald131
pushed a commit
to donald131/hermes-agent
that referenced
this pull request
May 2, 2026
…ousResearch#17563) Skill catalog pages (bundled/optional) were drowning out real user-guide and reference docs in search results. There are ~3100 of them and they match on almost every generic term. - Add `ignoreFiles` regexes to docusaurus-search-local for `user-guide/skills/bundled/` and `user-guide/skills/optional/`. The two human-written catalog indexes (`reference/skills-catalog`, `reference/optional-skills-catalog`) remain indexed. - Add a new feature page `user-guide/features/curator.md` covering the curator subsystem merged in NousResearch#16049 and refined in NousResearch#17307 (per-run reports): how it runs, config, CLI (`hermes curator status/run/pin/ restore/...`), `.usage.json` telemetry, archival semantics, and recovery. Slotted into the Core features sidebar next to Skills. Search index size dropped from 5822 docs to 2704 in the main section; `user-guide/features/curator` is indexed.
donald131
pushed a commit
to donald131/hermes-agent
that referenced
this pull request
May 2, 2026
… fire import (NousResearch#17927) Three fixes bundled for curator reliability on existing installs and broken/partial installs: 1. run_agent.py: defer `import fire` into the __main__ block. `fire` is only used by `fire.Fire(main)` when running run_agent.py directly as a CLI — it is NOT needed for library usage. Importing it at module top made `from run_agent import AIAgent` from a daemon thread (e.g. the curator's forked review agent) crash with ModuleNotFoundError on broken/partial installs where `fire` isn't present. 2. hermes_cli/config.py: add version 22 → 23 migration that writes the `curator` + `auxiliary.curator` sections to config.yaml with their defaults, only filling keys the user hasn't overridden. Existing configs from before PR NousResearch#16049 / the April 2026 `auxiliary.curator` unification had neither section on disk, so users couldn't see or edit the settings in their config.yaml (runtime deep-merge papered over it at read time, but the file never reflected reality). 3. hermes_cli/config.py: `ensure_hermes_home()` now pre-creates `~/.hermes/logs/curator/` alongside cron/sessions/logs/memories on every CLI launch. Managed-mode (NixOS) variant mkdir's it defensively after the activation-script existence checks, since the activation script may not know about this subpath. 4. agent/curator.py: `_reports_root()` mkdir's the dir at call time as belt-and-suspenders for entry paths that bypass both ensure_hermes_home() and the v23 migration (gateway-only installs, bare library use). E2E validated in isolated HERMES_HOME: fresh install gets full defaults seeded; partial-override config keeps user's `enabled: false` and custom `interval_hours` while filling the missing keys; re-running the migration is a no-op.
nickdlkk
pushed a commit
to nickdlkk/hermes-agent
that referenced
this pull request
May 11, 2026
…ousResearch#17563) Skill catalog pages (bundled/optional) were drowning out real user-guide and reference docs in search results. There are ~3100 of them and they match on almost every generic term. - Add `ignoreFiles` regexes to docusaurus-search-local for `user-guide/skills/bundled/` and `user-guide/skills/optional/`. The two human-written catalog indexes (`reference/skills-catalog`, `reference/optional-skills-catalog`) remain indexed. - Add a new feature page `user-guide/features/curator.md` covering the curator subsystem merged in NousResearch#16049 and refined in NousResearch#17307 (per-run reports): how it runs, config, CLI (`hermes curator status/run/pin/ restore/...`), `.usage.json` telemetry, archival semantics, and recovery. Slotted into the Core features sidebar next to Skills. Search index size dropped from 5822 docs to 2704 in the main section; `user-guide/features/curator` is indexed.
nickdlkk
pushed a commit
to nickdlkk/hermes-agent
that referenced
this pull request
May 11, 2026
… fire import (NousResearch#17927) Three fixes bundled for curator reliability on existing installs and broken/partial installs: 1. run_agent.py: defer `import fire` into the __main__ block. `fire` is only used by `fire.Fire(main)` when running run_agent.py directly as a CLI — it is NOT needed for library usage. Importing it at module top made `from run_agent import AIAgent` from a daemon thread (e.g. the curator's forked review agent) crash with ModuleNotFoundError on broken/partial installs where `fire` isn't present. 2. hermes_cli/config.py: add version 22 → 23 migration that writes the `curator` + `auxiliary.curator` sections to config.yaml with their defaults, only filling keys the user hasn't overridden. Existing configs from before PR NousResearch#16049 / the April 2026 `auxiliary.curator` unification had neither section on disk, so users couldn't see or edit the settings in their config.yaml (runtime deep-merge papered over it at read time, but the file never reflected reality). 3. hermes_cli/config.py: `ensure_hermes_home()` now pre-creates `~/.hermes/logs/curator/` alongside cron/sessions/logs/memories on every CLI launch. Managed-mode (NixOS) variant mkdir's it defensively after the activation-script existence checks, since the activation script may not know about this subpath. 4. agent/curator.py: `_reports_root()` mkdir's the dir at call time as belt-and-suspenders for entry paths that bypass both ensure_hermes_home() and the v23 migration (gateway-only installs, bare library use). E2E validated in isolated HERMES_HOME: fresh install gets full defaults seeded; partial-override config keeps user's `enabled: false` and custom `interval_hours` while filling the missing keys; re-running the migration is a no-op.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the Curator — an auxiliary-model background task that periodically reviews agent-created skills and keeps the collection tidy. Tracks usage, transitions unused skills through active → stale → archived, and spawns a forked AIAgent that uses
skill_manage+terminalto consolidate overlaps and patch drift.Paired with the class-first review prompt from #16026, this closes the loop on issue #7816: creation-side class-first thinking + retirement-side pruning.
Invariants (all load-bearing)
.bundled_manifestAND.hub/lock.json.~/.hermes/skills/.archive/and are recoverable viahermes curator restore <skill>.How it works
Default: enabled, inactivity-triggered (no cron daemon). On CLI startup and gateway boot:
Check config +
.curator_state— run iflast_run_atis older thaninterval_hours(default 24) AND agent has been idle ≥min_idle_hours(default 2).Automatic state transitions (pure, no LLM): mark stale (>30d unused), archive (>90d unused), reactivate stale skills that were used again.
LLM review pass — spawns forked AIAgent (max_iterations=8, quiet) with the agent-created candidate list + usage stats. The model uses existing tools only:
skills_list/skill_view— survey the landscapeskill_manage action=patch— fix drift, absorb a weaker overlapping skillterminal—mva skill directory into~/.hermes/skills/.archive/to archive itDecision list:
keep/patch/consolidate/archive. Nopin(that's a user-only opt-out).Persist
.curator_statewith the summary.Changes
tools/skill_usage.py(new).usage.jsonI/O, atomic writes, provenance filter, archive/restore helpers (used by the CLI, not exposed to the model)agent/curator.py(new)hermes_cli/curator.py(new)hermes curator {status,run,pause,resume,pin,unpin,restore}subcommandtests/tools/test_skill_usage.py(new)tests/agent/test_curator.py(new)tools/skills_tool.pyskill_viewregistry handler to bumpview_counton successtools/skill_manager_tool.pypatch_counton patch/edit/write_file/remove_file;forget()record on deletehermes_cli/config.pycurator:section toDEFAULT_CONFIG(additive — no version bump needed)hermes_cli/commands.py/curatorCommandDefwith subcommand hintshermes_cli/main.pyhermes curatorargparse subparser viaregister_cli()cli.py/curatorslash-command dispatch + CLI-startup hookgateway/run.pyLifecycle
Configuration (new)
CLI surface
Validation
test_skill_usage.py(new, 29 tests)test_curator.py(new, 27 tests)tests/hermes_cli/+ gateway sweephermes curator status/pause/resume/--helpOut of scope (follow-ups)
interval_hoursgate keeps it from over-running). A simplethreading.Timeror APScheduler hook is a clean follow-up.CommandDefis in the registry so gateway/helpwill pick it up automatically, but platform-specific menus may need a refresh pass.Refs #7816.