Skip to content

fix(curator): prevent false-positive consolidation from substring matching#19573

Merged
teknium1 merged 2 commits into
mainfrom
hermes/hermes-9728230f
May 4, 2026
Merged

fix(curator): prevent false-positive consolidation from substring matching#19573
teknium1 merged 2 commits into
mainfrom
hermes/hermes-9728230f

Conversation

@teknium1
Copy link
Copy Markdown
Contributor

@teknium1 teknium1 commented May 4, 2026

Summary

Curator's _classify_removed_skills heuristic no longer matches short skill names (`api`, `git`, `test`) as substrings of longer words, so the fallback classification path doesn't invent consolidations where none exist.

Root cause

`agent/curator.py` used `needle in hay` with no word or path-component awareness. A removed skill `api` would match `references/api-design.md` or content `running latest tests`, flipping what should be a pruning into a "consolidated into ". This path is only consulted when the authoritative `absorbed_into` declaration and the model's structured YAML block are both missing/malformed, but when it fires it pollutes REPORT.md and can misdirect cron-job skill-reference rewrites.

Changes

  • `agent/curator.py`: new `needle_in_path_component()` (filename-stem / directory-name equality, normalizes `-`/``); `file_path` fields use it, content fields use `re.search(rf'\b{re.escape(needle)}\b', hay)`.
  • `tests/agent/test_curator_classification.py`: 3 regression tests — false positive in `file_path`, false positive in content, real standalone-word match still works.

Validation

`scripts/run_tests.sh tests/agent/test_curator_classification.py` → 38/38 pass.

Salvaged from PR #19222 by @SimbaKingjoe — cherry-picked with authorship preserved. Closes #19233.

SimbaKingjoe and others added 2 commits May 3, 2026 17:07
…ching

_classify_removed_skills used naive 'in' substring matching to detect
whether a removed skill's name appeared in skill_manage arguments.
Short/common skill names (api, git, test, foo, etc.) matched
incorrectly when they appeared as substrings of longer words in file
paths (references/api-design.md) or content (latest, testing).

Replace with field-aware matching:
- file_path: needle must match a complete filename stem or directory
  name, with -/_ normalised for variant tolerance
- content fields: word-boundary regex (\b) prevents embedding in
  longer words

Also add 3 regression tests covering the false-positive scenarios.
@teknium1 teknium1 merged commit c90f25d into main May 4, 2026
8 of 10 checks passed
@teknium1 teknium1 deleted the hermes/hermes-9728230f branch May 4, 2026 08:21
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 4, 2026

🚨 CRITICAL Supply Chain Risk Detected

This PR contains a pattern that has been used in real supply chain attacks. A maintainer must review the flagged code carefully before merging.

🚨 CRITICAL: Install-hook file added or modified

These files can execute code during package installation or interpreter startup.

Files:

hermes_cli/setup.py

Scanner only fires on high-signal indicators: .pth files, base64+exec/eval combos, subprocess with encoded commands, or install-hook files. Low-signal warnings were removed intentionally — if you're seeing this comment, the finding is worth inspecting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: _classify_removed_skills false-positive consolidation from naive substring matching

2 participants