Skip to content

Release 17.0.0-beta.6 (#25) - Add CLI E2E tests, container mode, temp…#26

Merged
hifi-phil merged 1 commit intodevfrom
main
Mar 30, 2026
Merged

Release 17.0.0-beta.6 (#25) - Add CLI E2E tests, container mode, temp…#26
hifi-phil merged 1 commit intodevfrom
main

Conversation

@hifi-phil
Copy link
Copy Markdown
Contributor

…late fixes, and skill improvements

Comprehensive E2E testing for the create-mcp-server CLI, bug fixes found during manual testing, a new container mode for wrapping MCP servers, and template/skill improvements from real-world usage feedback.

CLI Bug Fixes

  • McpOAuthComposer.cs: Try-catch resilience so Umbraco starts before DB is ready (unattended install)
  • setup-instance.ts: Write connection string to appsettings.local.json, unattended install config to appsettings.Development.json, patch Program.cs to load appsettings.local.json
  • Template TS errors: Fix createToolResult signature, jest.fn typing in server-config.test.ts
  • collections.ts cleanup: removeExamples() now cleans up collections.ts imports

E2E Tests

CLI E2E (19 deterministic steps, cli-e2e.test.ts):

  • Scaffold → init (Forms + SQL Server) → start Umbraco → API user creation → token verification → swagger discovery → .discover.json → generate client → TypeScript compile → unit tests → integration test against real Umbraco → real API calls → hosted worker

Container Mode E2E (5 steps):

  • Scaffold → remove API tools → compile → build → worker starts

Skill E2E (3 steps, skill-e2e.test.ts, opt-in via RUN_SKILL_TESTS=true):

  • /build-tools creates Language collection that compiles
  • /build-tools-tests creates integration tests
  • Integration tests pass against real Umbraco

Infrastructure: Random SQL Server database per run, KEEP_E2E_ASSETS=true for preserving project between runs, test:e2e:revert for resetting skill output, test:e2e:cleanup for teardown.

Container Mode

New init prompt choice: "Container mode — wrap other MCP servers via chaining, no direct API tools"

  • removeApiTools() strips orval, generated client, example tools while keeping chaining infrastructure
  • Tool mode question comes after Umbraco instance setup (instance always needed for OAuth)

Template Improvements

  • Self-signed certs in Jest: Override globalThis.fetch with undici's fetch (Node 22's built-in fetch ignores NODE_TLS_REJECT_UNAUTHORIZED in Jest VM sandbox)
  • MSW conditional: Gate setupMswServer() behind USE_MOCK_API=true (was blocking real API calls in integration tests)
  • .gitignore: Added to template (demo-site/, .env, appsettings.local.json, .discover.json)

Skill Improvements

  • build-tools: Compile after EACH tool file, not batch
  • build-tools-tests: Complete working builder example with delete/cleanup, anti-pattern warning for result.content[0].text (must use createSnapshotResult), initializeUmbracoFetch in setup.ts template, read-only collection guidance
  • count-tools: Replace glob npm package with Node 22 built-in fs.globSync
  • Plugin install: Correct commands in discover output (/plugin marketplace add + /plugin install)

…late fixes, and skill improvements

 Comprehensive E2E testing for the create-mcp-server CLI, bug fixes found during manual testing, a new container mode for wrapping MCP servers, and template/skill improvements from real-world usage feedback.

CLI Bug Fixes

  - McpOAuthComposer.cs: Try-catch resilience so Umbraco starts before DB is ready (unattended install)
  - setup-instance.ts: Write connection string to appsettings.local.json, unattended install config to appsettings.Development.json, patch Program.cs to load appsettings.local.json
  - Template TS errors: Fix createToolResult signature, jest.fn typing in server-config.test.ts
  - collections.ts cleanup: removeExamples() now cleans up collections.ts imports

E2E Tests

CLI E2E (19 deterministic steps, cli-e2e.test.ts):
  - Scaffold → init (Forms + SQL Server) → start Umbraco → API user creation → token verification → swagger discovery → .discover.json → generate client → TypeScript compile → unit tests → integration test against real Umbraco → real API calls → hosted worker

Container Mode E2E (5 steps):
  - Scaffold → remove API tools → compile → build → worker starts

Skill E2E (3 steps, skill-e2e.test.ts, opt-in via RUN_SKILL_TESTS=true):
  - /build-tools creates Language collection that compiles
  - /build-tools-tests creates integration tests
  - Integration tests pass against real Umbraco

Infrastructure: Random SQL Server database per run, KEEP_E2E_ASSETS=true for preserving project between runs, test:e2e:revert for resetting skill output, test:e2e:cleanup for teardown.

  Container Mode

New init prompt choice: "Container mode — wrap other MCP servers via chaining, no direct API tools"
  - removeApiTools() strips orval, generated client, example tools while keeping chaining infrastructure
  - Tool mode question comes after Umbraco instance setup (instance always needed for OAuth)

  Template Improvements

  - Self-signed certs in Jest: Override globalThis.fetch with undici's fetch (Node 22's built-in fetch ignores NODE_TLS_REJECT_UNAUTHORIZED in Jest VM sandbox)
  - MSW conditional: Gate setupMswServer() behind USE_MOCK_API=true (was blocking real API calls in integration tests)
  - .gitignore: Added to template (demo-site/, .env, appsettings.local.json, .discover.json)

Skill Improvements

  - build-tools: Compile after EACH tool file, not batch
  - build-tools-tests: Complete working builder example with delete/cleanup, anti-pattern warning for result.content[0].text (must use createSnapshotResult), initializeUmbracoFetch in setup.ts template, read-only collection guidance
  - count-tools: Replace glob npm package with Node 22 built-in fs.globSync
  - Plugin install: Correct commands in discover output (/plugin marketplace add + /plugin install)
@hifi-phil hifi-phil merged commit a1a6ea9 into dev Mar 30, 2026
7 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant