Skip to content

docs: publish llms.txt and llms-full.txt for agent-friendly ingestion#18276

Merged
teknium1 merged 1 commit into
mainfrom
hermes/llms-txt
May 1, 2026
Merged

docs: publish llms.txt and llms-full.txt for agent-friendly ingestion#18276
teknium1 merged 1 commit into
mainfrom
hermes/llms-txt

Conversation

@teknium1
Copy link
Copy Markdown
Contributor

@teknium1 teknium1 commented May 1, 2026

Summary

Publish two agent-friendly entry points to the docs:

  • /llms.txt — curated index, ~17 KB. Short description for each page, grouped by section mirroring the left sidebar. Safe to load directly into an LLM context window.
  • /llms-full.txt — all 131 curated doc pages concatenated as markdown, ~1.8 MB. For one-shot ingestion by coding agents and RAG pipelines. Auto-generated skill pages are excluded (covered by the two catalog reference pages) to keep the file focused on the product.

Both files resolve at two URLs so either probing convention works:

  • https://hermes-agent.nousresearch.com/llms.txt (classic root path)
  • https://hermes-agent.nousresearch.com/docs/llms.txt (Docusaurus baseUrl)

What changed

  • website/scripts/generate-llms-txt.py — new generator. ~80 curated pages organized into 10 sections (Getting Started, Using Hermes, Core Features, Automation, Media & Web, Messaging, Integrations, Guides, Developer Guide, Reference). Descriptions pulled from each page's frontmatter so the index stays current when docs are edited. All slugs validated against actual files on disk at generation time.
  • website/scripts/prebuild.mjs — refactored into a small helper so skills.json and llms.txt generation share the python-spawn + graceful-fallback pattern. Non-fatal failure for llms.txt (site still builds).
  • website/.gitignore — adds static/llms.txt and static/llms-full.txt (generated artifacts, same precedent as src/data/skills.json).
  • .github/workflows/deploy-site.ymlStage deployment step also copies both files to _site/ root so the classic /llms.txt URL works, not just /docs/llms.txt.
  • website/docs/index.md — adds a "For LLMs and coding agents" section linking to both files.

Motivation

Matt Palmer flagged the docs as "agent unfriendly — no llms.txt / llms-full.txt." Fair critique. Conforms to the emerging llmstxt.org spec.

Validation

cd website && npm run build
Wrote website/static/llms.txt (17,092 bytes)
Wrote website/static/llms-full.txt (1,874,374 bytes)
[SUCCESS] Generated static files in "build".

Only warning is the pre-existing broken anchor in adding-platform-adapters (unrelated, same as every recent docs PR).

Maintenance

To add a new section or page to the curated index, edit the SECTIONS list in generate-llms-txt.py. Missing slugs will fail the prebuild loudly.

Two machine-readable entry points to the Hermes Agent docs:

  /llms.txt         curated index of every doc page, one link per page
                    with short descriptions. ~17 KB, safe to load into
                    an LLM context window.
  /llms-full.txt    every page under website/docs/ concatenated as markdown.
                    ~1.8 MB. For one-shot ingestion by coding agents and
                    RAG pipelines.

Both files are also served from /docs/llms.txt and /docs/llms-full.txt
(Docusaurus serves website/static/ under baseUrl=/docs/). Some agents and
IDE plugins probe the classic site-root path; the deploy workflow now copies
both files to _site root so either URL works.

Conforms to the emerging llmstxt.org spec: H1 project name, blockquote
summary, short install command, GitHub link, then curated sections
mirroring the docs-site navigation (Getting Started, Using Hermes,
Features, Messaging, Integrations, Guides, Developer Guide, Reference).

Generated by website/scripts/generate-llms-txt.py. Wired into prebuild.mjs
so every 'npm run build' and 'npm run start' refreshes the files alongside
the existing skills.json extraction. Both outputs are gitignored (same
precedent as src/data/skills.json).

Descriptions in llms.txt are pulled from each page's frontmatter, so they
stay current automatically. All ~80 section slugs are validated against
the filesystem at generation time; an invalid slug would fail the prebuild.
@teknium1 teknium1 merged commit c6eebfc into main May 1, 2026
9 of 11 checks passed
@teknium1 teknium1 deleted the hermes/llms-txt branch May 1, 2026 06:17
@alt-glitch alt-glitch added type/docs Documentation improvements P3 Low — cosmetic, nice to have labels May 1, 2026
jooray added a commit to jooray/hermes-agent that referenced this pull request May 1, 2026
* upstream/main: (208 commits)
  fix: kanban button
  fix(telegram): send seed message after creating DM topics (NousResearch#18334)
  fix(yuanbao): enforce owner identity check on group slash commands
  docs(website): add User Stories and Use Cases collage page (NousResearch#18282)
  fix: prevent tui rebuilding assets
  fix(paths): route achievements plugin + profile-tui through HERMES_HOME
  docs: publish llms.txt and llms-full.txt for agent-friendly ingestion (NousResearch#18276)
  docs: add Persistent Goals (/goal) feature page (NousResearch#18275)
  fix(moonshot): also strip nullable/enum after anyOf collapse
  chore(release): map hendrixfreire for moonshot salvage
  fix(moonshot): fill missing type before enum cleanup to handle anyOf branches without explicit type
  chore(release): add mikeyobrien to AUTHOR_MAP
  fix(gateway): honor MATRIX_HOME_ROOM in onboarding
  feat: /goal — persistent cross-turn goals (Ralph loop) (NousResearch#18262)
  docs(sidebar): collapse exploding skills tree to a single Skills node (NousResearch#18259)
  feat(update): add --yes/-y flag to skip interactive prompts (NousResearch#18261)
  feat(gateway): auto-delete slash-command system notices after TTL (NousResearch#18266)
  fix(curator): rewrite cron job skill refs after consolidation (NousResearch#18253)
  fix(deepseek): use non-empty reasoning_content placeholder for V4 Pro thinking mode
  fix(compression): include system prompt + tool schemas in token estimates (NousResearch#18265)
  ...

# Conflicts:
#	tools/tts_tool.py
donald131 pushed a commit to donald131/hermes-agent that referenced this pull request May 2, 2026
…NousResearch#18276)

Two machine-readable entry points to the Hermes Agent docs:

  /llms.txt         curated index of every doc page, one link per page
                    with short descriptions. ~17 KB, safe to load into
                    an LLM context window.
  /llms-full.txt    every page under website/docs/ concatenated as markdown.
                    ~1.8 MB. For one-shot ingestion by coding agents and
                    RAG pipelines.

Both files are also served from /docs/llms.txt and /docs/llms-full.txt
(Docusaurus serves website/static/ under baseUrl=/docs/). Some agents and
IDE plugins probe the classic site-root path; the deploy workflow now copies
both files to _site root so either URL works.

Conforms to the emerging llmstxt.org spec: H1 project name, blockquote
summary, short install command, GitHub link, then curated sections
mirroring the docs-site navigation (Getting Started, Using Hermes,
Features, Messaging, Integrations, Guides, Developer Guide, Reference).

Generated by website/scripts/generate-llms-txt.py. Wired into prebuild.mjs
so every 'npm run build' and 'npm run start' refreshes the files alongside
the existing skills.json extraction. Both outputs are gitignored (same
precedent as src/data/skills.json).

Descriptions in llms.txt are pulled from each page's frontmatter, so they
stay current automatically. All ~80 section slugs are validated against
the filesystem at generation time; an invalid slug would fail the prebuild.
nickdlkk pushed a commit to nickdlkk/hermes-agent that referenced this pull request May 11, 2026
…NousResearch#18276)

Two machine-readable entry points to the Hermes Agent docs:

  /llms.txt         curated index of every doc page, one link per page
                    with short descriptions. ~17 KB, safe to load into
                    an LLM context window.
  /llms-full.txt    every page under website/docs/ concatenated as markdown.
                    ~1.8 MB. For one-shot ingestion by coding agents and
                    RAG pipelines.

Both files are also served from /docs/llms.txt and /docs/llms-full.txt
(Docusaurus serves website/static/ under baseUrl=/docs/). Some agents and
IDE plugins probe the classic site-root path; the deploy workflow now copies
both files to _site root so either URL works.

Conforms to the emerging llmstxt.org spec: H1 project name, blockquote
summary, short install command, GitHub link, then curated sections
mirroring the docs-site navigation (Getting Started, Using Hermes,
Features, Messaging, Integrations, Guides, Developer Guide, Reference).

Generated by website/scripts/generate-llms-txt.py. Wired into prebuild.mjs
so every 'npm run build' and 'npm run start' refreshes the files alongside
the existing skills.json extraction. Both outputs are gitignored (same
precedent as src/data/skills.json).

Descriptions in llms.txt are pulled from each page's frontmatter, so they
stay current automatically. All ~80 section slugs are validated against
the filesystem at generation time; an invalid slug would fail the prebuild.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P3 Low — cosmetic, nice to have type/docs Documentation improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants