Conversation
Based on Justin Poehnelt's blog post about rewriting CLIs for AI agents. Covers input hardening, response trimming, dry-run mode, CLI introspection, and integration tests across 6 phases. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add input hardening, context window discipline, dry-run mode, CLI introspection, and integration test infrastructure to the SDK. Phase 1 - Input sanitization: rejectControlCharacters, rejectPathTraversal, rejectEmbeddedQueryParams, rejectPreEncodedStrings, validateUUID, and withInputSanitization decorator wired into withStandardDecorators chain. Phase 2 - Response trimming: trimArrayResponse, summarizeDeepResponse, estimateTokenSize, pickFields/omitFields. Added fields/excludeFields options to ApiCallOptions for field masking. Phase 3 - Dry-run mode: configureDryRunMode toggle, withDryRun decorator that intercepts mutation tools while letting read-only tools execute. Added UMBRACO_DRY_RUN config field. Phase 4 - CLI introspection: --list-tools, --describe-tool, --generate-context flags with toolToJsonSchema, formatToolTable, generateContextFile utilities. Phase 5 - CLI integration tests: StdioClientTransport-based test client, tests for input sanitization, dry-run, introspection, and tool execution. Phase 6 - Exports: All new modules exported from SDK main and helpers entry points. https://claude.ai/code/session_01S2qDhdqdt2hXAxtxCMT6HM
…ssertions - Replace import.meta.dirname with fileURLToPath+dirname for Jest ESM compat - Fix --describe-tool test to use actual tool name (get-example) - Fix dry-run test to handle MCP SDK output schema validation behavior https://claude.ai/code/session_01S2qDhdqdt2hXAxtxCMT6HM
User-facing guide covering introspection commands, tool filtering, safety features (dry-run, readonly, input sanitization), and configuration. https://claude.ai/code/session_01S2qDhdqdt2hXAxtxCMT6HM
Extracts the --list-tools, --describe-tool, and --generate-context boilerplate into a single reusable function in the SDK so consumers don't have to copy the template pattern manually. - New: handleCliCommands() in @umbraco-cms/mcp-server-sdk - New: unit tests for all CLI introspection paths - Updated template to use handleCliCommands() instead of inline logic - Updated CLAUDE.md docs with new CLI Helpers section https://claude.ai/code/session_01S2qDhdqdt2hXAxtxCMT6HM
New plugin skill that lets agents discover and inspect tools in any SDK-based Umbraco MCP server via CLI introspection. Works across all implementations (CMS, Forms, custom) since they share the same CLI interface via handleCliCommands(). - /discover-mcp-server skill with SKILL.md and discover-tools.ts script - Supports list-tools, describe-tool, and generate-context commands - Auto-detects server entry point from package.json - Tested against template project (all 3 commands verified) https://claude.ai/code/session_01S2qDhdqdt2hXAxtxCMT6HM
This reverts commit 105744d.
Resolved conflicts: - config.ts: kept dev's lazy-loaded async yargs, added CLI introspection flags (--list-tools, --describe-tool, --generate-context) to the lazy-loaded parser - server-config.ts: combined await (dev) with cliFlags destructure (ours) - template/package.json: took dev's scripts and dependencies - package-lock.json: regenerated from dev's base https://claude.ai/code/session_01S2qDhdqdt2hXAxtxCMT6HM
- server-config.test.ts: add await + mockResolvedValue for async getServerConfig - get-server-info.ts: fix createToolResult call (now takes 1 arg) - tsconfig.json: add @types/jest to types array for test file support https://claude.ai/code/session_01S2qDhdqdt2hXAxtxCMT6HM
- Fix ZodUUID/ZodLiteral/ZodNullable support in schema converter - Skip required field validation for introspection flags (--list-tools etc.) - Fix readonly mode bug: ServerConfigForCollections.readOnly casing mismatch - Add test:cli script to template, test:evals script to root - Add mcp-cli skill for CLI configuration and usage guidance - Add eval suite: runtime safety, runtime tool calling, skill knowledge, and skill-driven CLI tool calling (38 tests total) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…mprovements-pxwfN # Conflicts: # template/src/config/__tests__/server-config.test.ts
…s plugin - CLI introspection (--list-tools, --describe-tool, --generate-context) now respects filtering env vars (UMBRACO_READONLY, UMBRACO_INCLUDE_SLICES, etc.) - Add --debug-config flag to print resolved config with sources as JSON - Move mcp-cli skill to new plugins-server/ plugin (server operations vs SDK development) - Move CLI integration tests from template to tests/cli/integration/ - Move CLI eval tests from tests/evals/ to tests/cli/evals/ - Remove broken runtime eval tests (never worked, covered by integration tests) - Fix template unit tests: always start MSW, initialize UmbracoFetch in jest setup - Add docs/cli.md CLI reference documentation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Credentials now show "(set)" or "(not set)" instead of masked values. Added explicit test that secret values never appear in output. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ServerConfigForCollections uses `readonly` (lowercase) but the hosted MCP code was setting `readOnly` (camelCase). The collection config loader reads `config.readonly`, so the value was never picked up — readOnly filtering silently did nothing in the hosted worker. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Template doesn't need CLI docs — they're internal reference, not shipped to new projects. Added output format examples and dry-run preview JSON. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Lists every test suite with its command, what it covers, test count, and what infrastructure it needs (builds, Umbraco, SQL Server, API keys). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
MSW now runs unconditionally for unit tests — the USE_MOCK_API guard was removed because unit tests need MSW interception to work. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
MSW intercepts all API calls — no infrastructure needed beyond build. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
USE_MOCK_API=true enables MSW interception for unit tests in this repo. USE_INLINE_MOCKS=true enables the in-memory mock store in customInstance for eval tests running in a subprocess where MSW isn't available. Previously both used USE_MOCK_API which caused customInstance to short-circuit before MSW could intercept, breaking unit tests. Scaffolded sites default to both off (tests hit real Umbraco). Added test:template script to monorepo root that sets USE_MOCK_API. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
USE_MOCK_API=true means different things in different contexts: - In jest (unit tests): MSW intercepts HTTP requests - In eval subprocess: inline mock store handles requests They never conflict because customInstance now checks JEST_WORKER_ID to skip the inline store when MSW is available. No second env var needed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The template index.ts gained handleCliCommands + CLI introspection section but the snapshot file wasn't regenerated before push. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds comprehensive safety, debugging, and operational features to the MCP server SDK:
Key Changes
CLI Introspection (
packages/mcp-server-sdk/src/cli/)introspection.ts– Tool metadata conversion and table formattingtoolToJsonSchema()– Convert Zod schemas to JSON Schema for displaytoolToSummary()– Extract tool metadata for table rowsformatToolTable()– Pretty-print tools with columns (Name, Collection, Slices, RO, Destr, Description)context-generator.ts– Generate structured CONTEXT.md files for AI assistantsgenerateContextFile()– Markdown documentation of all collections and toolshandle-cli-commands.ts– One-call handler for--list-tools,--describe-tool,--generate-contextInput Sanitization (
packages/mcp-server-sdk/src/helpers/input-sanitizer.ts)Validation functions that protect against agent hallucinations:
rejectControlCharacters()– Block null bytes and control chars (except tab/newline/CR)rejectPathTraversal()– Prevent../, absolute paths, Windows UNC pathsrejectEmbeddedQueryParams()– Block?and&in identifiersrejectPreEncodedStrings()– Reject percent-encoded sequences (%20,%2F, etc.)validateUUID()– Validate UUID v4 formatsanitizeStringInput()– Composite validation with opt-out flagswithInputSanitization()– Decorator to wrap tools with input hardeningAll functions throw
ToolValidationErrorwith clear messages for agent self-correction.Dry-Run Mode (
packages/mcp-server-sdk/src/helpers/dry-run.ts)configureDryRunMode()– Enable/disable globallyisDryRunEnabled()– Check current statewithDryRun()– Decorator that intercepts mutation toolsResponse Trimming (
packages/mcp-server-sdk/src/helpers/response-trimmer.ts)Context window discipline utilities:
trimArrayResponse()– Limit array items with_truncatedand_totalAvailablemetadatasummarizeDeepResponse()– Collapse nested structures beyond max depthestimateTokenSize()– Rough token count estimator (chars / 4)pickFields()– Extract specific fields from objectsomitFields()– Remove specific fields from objectsConfiguration Updates
config.ts– AddeddryRunconfig option andGetServerConfigResulttype exporttool-decorators.ts– IntegratedwithInputSanitization()into tool wrapping pipelineapi-call-helpers.ts– Integrated response trimming helpersTemplate & Testing
template/src/index.ts– CallhandleCliCommands()after loading configtemplate/tests/cli/– Integration test suitecli-client.ts– Helper to spawn CLI binary and connect via MCP stdio transportintrospection.test.ts– Test--list-tools,--describe-tool,--generate-contexttool-execution.test.tshttps://claude.ai/code/session_01S2qDhdqdt2hXAxtxCMT6HM