Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions docs/herdr-workspaces-mcp.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Herdr workspace reads from MCP

`herdr_workspaces` reads the selected local broker's
`GET /v1/terminal-hosts/herdr/workspaces` projection. The broker owns the
TTL-cached Herdr probes. The MCP client does not run Herdr, enumerate local
sessions, start services or use the private web server.

The existing `session`, `query` and `limit` tool inputs and ranked digest output
remain. A session is an exact host selector, not a Scout routing address. Pane
names and reported harness sessions remain observations, not verified bindings.
Stopped or saved projections never offer a live read handoff.

## Observation and error metadata

Structured results add `runtimeRead` when a valid broker observation was read.
It preserves the schema, node, scan time, requested session, availability,
reason, count, truncation and coverage limits. Its `sessions` list carries each
returned session's availability; the existing digest or candidate fields carry
the presentation. No duplicate full digest is added to the metadata.

- `available` observations can establish an observed empty bounded inventory.
- `partial` observations remain usable, but the tool marks them truncated and
does not describe an empty result as proof that no sessions exist.
- `not_scanned` and `unavailable` produce `available: false`. They are not
available-empty results. Valid unavailable observations can still carry
per-session failure provenance in `runtimeRead`.
- Transport, HTTP and validation failures produce `readError` with a safe code
and HTTP status when received. Raw server bodies are not reflected. A 403
remains a refusal; a 404 is not converted into an available empty session.

This is an additive MCP result change and a read-backend switch. It adds no
runtime route, protocol DTO, package export, host control or answer authority.
The tool remains machine-local and read-only.

## Transport and validation bounds

The client selects `OPENSCOUT_BROKER_URL` or the runtime's control URL. Only
literal `localhost`, `127.0.0.1` and `[::1]` HTTP(S) authorities are accepted.
If `OPENSCOUT_BROKER_SOCKET_PATH` is set, that socket is used; failure never
falls back to HTTP. Socket-only mode without a socket fails before a request.

One eight-second signal covers headers and body. Redirects and ambient
credentials are disabled. The client never sends web bearer tokens, cookies,
browser origins or forwarding headers, and never requests `/api/bootstrap.js`.
It makes no retry or alternate-source read after failure.

The response has a two-MiB byte ceiling, strict UTF-8/JSON validation and the
versioned digest shape. It validates the requested session, count, duplicate
sessions, saved/live consistency, nested arrays and the broker's declared
limits. Unrecognized or oversized shapes are unavailable, not silently
truncated into a valid observation. The broker's 40 emitted panes can occupy
up to 256 directory groups, including groups whose panes were all omitted.

The browser-cookie adapter and its temporary private inventory exception are
removed. The named inventory records its source as forbidden to return. Other
private app commands and packaged app artifacts remain separate cutover work;
this reader does not make the whole CLI package app-independent.
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,8 @@
"test": "bun run test:unit",
"test:unit": "bun run test:cli && bun run test:unit:source",
"test:cli": "bun run --cwd packages/cli test:happy",
"test:unit:source": "bun run --cwd packages/protocol test && bun run --cwd packages/agent-sessions test && bun run --cwd packages/runtime test && bun run --cwd packages/cli test:source && bun run --cwd apps/desktop test && bun run --cwd packages/web test && bun run --cwd packages/session-trace test && bun run --cwd packages/session-trace-react test && node --test scripts/ship-release.test.mjs scripts/bundle-scout-web.test.mjs scripts/check-packed-dependencies.test.mjs && bun run --cwd packages/mesh-extension test",
"test:unit:source": "bun run test:remote-target-programs && bun run --cwd packages/protocol test && bun run --cwd packages/agent-sessions test && bun run --cwd packages/runtime test && bun run --cwd packages/cli test:source && bun run --cwd apps/desktop test && bun run --cwd packages/web test && bun run --cwd packages/session-trace test && bun run --cwd packages/session-trace-react test && node --test scripts/ship-release.test.mjs scripts/bundle-scout-web.test.mjs scripts/check-packed-dependencies.test.mjs && bun run --cwd packages/mesh-extension test",
"test:remote-target-programs": "bun test scripts/remote-target-programs.test.ts",
"test:happy": "cd apps/desktop && bun run test:happy && cd ../../packages/cli && bun run test:happy",
"test:scenarios": "bun run --cwd packages/runtime test:scenarios",
"test:e2e": "bun run check && bun run test:scenarios && bun run test:e2e:agents",
Expand Down
44 changes: 44 additions & 0 deletions packages/cli/src/cli/commands/fixtures/web-stdio.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
// Exercise the real install, then the next command's lazy stdout initialization.
// The GC schedule follows https://github.com/oven-sh/bun/pull/44581: an older
// Bun finalizes stdio sinks against the wrong epoll while a synchronous child is active.
import { createHash } from "node:crypto";
import { readFileSync, writeSync } from "node:fs";
import { createScoutCommandContext } from "../../context.ts";
import { mock } from "bun:test";
// --no-restart never calls server control. Keep unrelated service discovery out
// of the deliberately allocation-heavy GC child (and away from real services).
mock.module("../server.ts", () => ({ requestScoutWebControl() { throw new Error("Unexpected server control"); } }));
const { runWebCommand } = await import("../web.ts");

const tarball = new Uint8Array(readFileSync(process.argv[2]!));
const held = globalThis as typeof globalThis & { stdioSinks: unknown };
held.stdioSinks = Array.from({ length: 4 }, () => {
const sink = Bun.file(1).writer();
sink.write("");
return sink;
});
await Bun.sleep(0);
const original = Bun.spawnSync;
Bun.spawnSync = ((options: { cmd: string[]; maxBuffer?: number }) => original({
...options,
get maxBuffer() {
Bun.gc(true);
held.stdioSinks = null;
return options.maxBuffer;
},
})) as typeof Bun.spawnSync;
try {
await runWebCommand(createScoutCommandContext({ env: { SCOUT_DIST_KEY: "osdist_stdio_regression" }, isTty: false, stdout() {}, stderr() {} }), ["install", "--no-restart"], {
supportDirectory: process.argv[3]!, version: "9.8.7", keyStores: [],
fetchImpl: async () => new Response(tarball, { headers: { "x-openscout-sha256": createHash("sha256").update(tarball).digest("hex") } }),
});
} finally {
Bun.spawnSync = original;
}
held.stdioSinks = null;
Bun.gc(true);
await Bun.sleep(50);
// Do not preinitialize process.stdout: this is where the following test failed.
createScoutCommandContext({ env: {}, stdout() {}, stderr() {} });
process.stdout.write("stdio survived install\n");
writeSync(2, "stdio regression finished\n");
114 changes: 94 additions & 20 deletions packages/cli/src/cli/commands/web.test.ts
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
import { afterEach, beforeEach, describe, expect, spyOn, test } from "bun:test";
import * as childProcess from "node:child_process";
import { spawnSync } from "node:child_process";
import { execFile } from "node:child_process";
import { createHash } from "node:crypto";
import { existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, statSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { promisify } from "node:util";

import { resolveInstalledWebFullClient } from "@openscout/runtime/web-full-client";

Expand Down Expand Up @@ -33,16 +34,14 @@ afterEach(() => {
rmSync(root, { recursive: true, force: true });
});

function packClient(version: string, profileVersion = version): Uint8Array<ArrayBuffer> {
async function packClient(version: string, profileVersion = version): Promise<Uint8Array<ArrayBuffer>> {
const client = join(root, `client-${version}-${profileVersion}`);
mkdirSync(join(client, "assets"), { recursive: true });
writeFileSync(join(client, "index.html"), "<title>Scout</title>");
writeFileSync(join(client, "assets", "app.js"), "Mission Control");
writeFileSync(join(client, "scout-web-profile.json"), JSON.stringify({ profile: "full", version: profileVersion }));
const tarball = join(root, `${version}-${profileVersion}.tar.gz`);
const packed = spawnSync("tar", ["-czf", tarball, "-C", client, "."], { timeout: 5_000, killSignal: "SIGKILL" });
expect(packed.error).toBeUndefined();
expect(packed.status, packed.stderr?.toString()).toBe(0);
await promisify(execFile)("tar", ["-czf", tarball, "-C", client, "."], { timeout: 5_000, killSignal: "SIGKILL", maxBuffer: 128 * 1024 });
return new Uint8Array(readFileSync(tarball));
}

Expand Down Expand Up @@ -95,6 +94,27 @@ function harness(options: {
}

describe("scout web", () => {
test.skipIf(process.platform !== "linux")("install leaves stdout usable for the next command even under GC", async () => {
const tarball = await packClient(VERSION);
const archive = join(root, "stdio.tar.gz");
writeFileSync(archive, tarball);
// Own the risky old-Bun state in a child, with real pollable stdio pipes.
const child = Bun.spawn({
cmd: [process.execPath, new URL("./fixtures/web-stdio.ts", import.meta.url).pathname, archive, support],
env: { ...process.env, BUN_JSC_slowPathAllocsBetweenGCs: "3" },
stdout: "pipe", stderr: "pipe", stdin: "ignore",
});
const timeout = setTimeout(() => child.kill("SIGKILL"), 30_000);
try {
const [stdout, stderr, exit] = await Promise.all([new Response(child.stdout).text(), new Response(child.stderr).text(), child.exited]);
expect({ exit, stdout, stderr }).toEqual({ exit: 0, stdout: "stdio survived install\n", stderr: "stdio regression finished\n" });
} finally {
clearTimeout(timeout);
child.kill("SIGKILL");
await child.exited;
}
}, 35_000);

test("login checks the key with the host and saves it privately", async () => {
const { context, deps, lines, requests } = harness();
await runWebCommand(context, ["login", "--key", KEY], deps);
Expand All @@ -114,7 +134,7 @@ describe("scout web", () => {

test("install downloads, verifies and installs the full client for this version, then restarts web", async () => {
const restarted: string[] = [];
const tarball = packClient(VERSION);
const tarball = await packClient(VERSION);
const { context, deps, lines, requests } = harness({ tarball, env: { SCOUT_DIST_KEY: KEY }, restarted });
await runWebCommand(context, ["install"], deps);

Expand All @@ -139,32 +159,53 @@ describe("scout web", () => {
});

test("a checksum mismatch or a client for another version is never installed", async () => {
const tarball = packClient(VERSION);
const tarball = await packClient(VERSION);
const tampered = harness({ tarball, digest: "0".repeat(64), env: { SCOUT_DIST_KEY: KEY } });
await expect(runWebCommand(tampered.context, ["install"], tampered.deps)).rejects.toThrow("Checksum mismatch");

const wrong = harness({ tarball: packClient(VERSION, "1.0.0"), env: { SCOUT_DIST_KEY: KEY } });
const wrong = harness({ tarball: await packClient(VERSION, "1.0.0"), env: { SCOUT_DIST_KEY: KEY } });
await expect(runWebCommand(wrong.context, ["install"], wrong.deps)).rejects.toThrow("isn't the full web app");

expect(resolveInstalledWebFullClient(VERSION, { env: {}, supportDirectory: support })).toBeNull();
expect(readdirSync(join(support, "web", "full"))).toEqual([]);
});

test("a stuck unpack child has a hard deadline, reports its error and leaves no partial install", async () => {
const tarball = packClient(VERSION);
const tarball = await packClient(VERSION);
const { context, deps } = harness({ tarball, env: { SCOUT_DIST_KEY: KEY } });
const child = spyOn(childProcess, "spawnSync").mockReturnValue({
status: null, signal: "SIGKILL", error: Object.assign(new Error("spawnSync tar ETIMEDOUT"), { code: "ETIMEDOUT" }),
pid: 0, output: [], stdout: "", stderr: "partial tar diagnostic",
});
const child = spyOn(childProcess, "execFile").mockImplementation(((_file: string, ...args: unknown[]) => {
const callback = args.at(-1) as (error: Error, stdout: string, stderr: string) => void;
callback(Object.assign(new Error("tar ETIMEDOUT"), { code: "ETIMEDOUT" }), "", "partial tar diagnostic");
return {};
}) as typeof childProcess.execFile);
try {
await expect(runWebCommand(context, ["install", "--no-restart"], deps)).rejects.toThrow("Couldn't unpack the download: spawnSync tar ETIMEDOUT");
expect(child).toHaveBeenCalledWith("tar", expect.any(Array), expect.objectContaining({ timeout: 30_000, killSignal: "SIGKILL" }));
await expect(runWebCommand(context, ["install", "--no-restart"], deps)).rejects.toThrow("Couldn't unpack the download: tar ETIMEDOUT");
expect(child).toHaveBeenCalledWith("tar", expect.any(Array), expect.objectContaining({ timeout: 30_000, killSignal: "SIGKILL", maxBuffer: 128 * 1024 }), expect.any(Function));
expect(resolveInstalledWebFullClient(VERSION, { env: {}, supportDirectory: support })).toBeNull();
expect(readdirSync(join(support, "web", "full"))).toEqual([]);
} finally { child.mockRestore(); }
});

test("a stalled body is cancelled at the download deadline and leaves no partial install", async () => {
let cancelled = false;
let controller!: ReadableStreamDefaultController<Uint8Array>;
const body = new ReadableStream<Uint8Array>({
start(value) { controller = value; controller.enqueue(new Uint8Array([1, 2, 3])); },
cancel() { cancelled = true; },
});
const run = harness({ env: { SCOUT_DIST_KEY: KEY } });
// Cleanup-only EOF: without the consumer deadline this yields a normal test failure, not a hung runner.
const watchdog = setTimeout(() => { if (!cancelled) controller.close(); }, 750);
try { await expect(runWebCommand(run.context, ["install", "--no-restart"], {
...run.deps, downloadTimeoutMs: 25,
// Deliberately not connected to fetch's signal: the consumer must own a deadline.
fetchImpl: async () => new Response(body, { headers: { "x-openscout-sha256": "0".repeat(64) } }),
})).rejects.toThrow(/abort/i); } finally { clearTimeout(watchdog); }
expect(cancelled).toBe(true);
expect(resolveInstalledWebFullClient(VERSION, { env: {}, supportDirectory: support })).toBeNull();
expect(readdirSync(join(support, "web", "full"))).toEqual([]);
});

test("install explains a missing key and an unpublished version", async () => {
const none = harness();
await expect(runWebCommand(none.context, ["install"], none.deps)).rejects.toThrow("No download key");
Expand All @@ -174,7 +215,7 @@ describe("scout web", () => {
});

test("status reports what is served; uninstall goes back to basic", async () => {
const tarball = packClient(VERSION);
const tarball = await packClient(VERSION);
const run = harness({ tarball, env: { SCOUT_DIST_KEY: KEY } });
await runWebCommand(run.context, ["status"], run.deps);
expect(run.lines.join("\n")).toContain("serves the basic web app");
Expand Down Expand Up @@ -206,11 +247,11 @@ describe("scout web", () => {

test("an upgraded CLI diagnoses the old Full client and recovers with the saved key", async () => {
const previousVersion = "9.8.6";
const previous = harness({ tarball: packClient(previousVersion), env: { SCOUT_DIST_KEY: KEY } });
const previous = harness({ tarball: await packClient(previousVersion), env: { SCOUT_DIST_KEY: KEY } });
await fileDistKeyStore(support).write(KEY);
await runWebCommand(previous.context, ["install", "--no-restart"], { ...previous.deps, version: previousVersion });

const current = harness({ tarball: packClient(VERSION), json: true });
const current = harness({ tarball: await packClient(VERSION), json: true });
await runWebCommand(current.context, ["status"], current.deps);
expect(JSON.parse(current.lines[0]!)).toMatchObject({
version: VERSION, serves: "basic", reason: "version-mismatch", installed: null,
Expand All @@ -232,8 +273,41 @@ describe("scout web", () => {
});
});

test("a stalled upgrade preserves the old Full client and key after cancelling its download", async () => {
const previousVersion = "9.8.6";
const previous = harness({ tarball: await packClient(previousVersion), env: { SCOUT_DIST_KEY: KEY } });
await fileDistKeyStore(support).write(KEY);
await runWebCommand(previous.context, ["install", "--no-restart"], { ...previous.deps, version: previousVersion });
const tarball = await packClient(VERSION);
let cancelled = false;
let controller!: ReadableStreamDefaultController<Uint8Array>;
const body = new ReadableStream<Uint8Array>({
start(value) { controller = value; controller.enqueue(tarball); }, // All bytes, but no EOF.
cancel() { cancelled = true; },
});
const restarted: string[] = [];
const current = harness({ json: true, restarted });
// Cleanup-only EOF: without the consumer deadline this yields a normal test failure, not a hung runner.
const watchdog = setTimeout(() => { if (!cancelled) controller.close(); }, 750);
try { await expect(runWebCommand(current.context, ["install"], {
...current.deps, downloadTimeoutMs: 25,
fetchImpl: async (_url, init) => {
expect(init?.signal).toBeInstanceOf(AbortSignal);
return new Response(body, { headers: { "x-openscout-sha256": sha256(tarball) } });
},
})).rejects.toThrow(/abort/i); } finally { clearTimeout(watchdog); }
expect(cancelled).toBe(true);
expect(restarted).toEqual([]);
expect(await fileDistKeyStore(support).read()).toBe(KEY);
expect(resolveInstalledWebFullClient(previousVersion, { env: {}, supportDirectory: support })).not.toBeNull();
expect(resolveInstalledWebFullClient(VERSION, { env: {}, supportDirectory: support })).toBeNull();
expect(readdirSync(join(support, "web", "full"))).toEqual([previousVersion]);
await runWebCommand(current.context, ["status"], current.deps);
expect(JSON.parse(current.lines[0]!)).toMatchObject({ reason: "version-mismatch", installedVersions: [previousVersion] });
});

test("forcing Basic preserves the installed Full path and explains the override", async () => {
const run = harness({ tarball: packClient(VERSION), env: { SCOUT_DIST_KEY: KEY, OPENSCOUT_WEB_CLIENT_PROFILE: " basic " }, json: true });
const run = harness({ tarball: await packClient(VERSION), env: { SCOUT_DIST_KEY: KEY, OPENSCOUT_WEB_CLIENT_PROFILE: " basic " }, json: true });
await runWebCommand(run.context, ["install", "--no-restart"], run.deps);
run.lines.length = 0;
await runWebCommand(run.context, ["status"], run.deps);
Expand Down Expand Up @@ -277,7 +351,7 @@ describe("scout web", () => {

test("an unpublished upgrade leaves the old Full install and saved key intact", async () => {
const previousVersion = "9.8.6";
const previous = harness({ tarball: packClient(previousVersion), env: { SCOUT_DIST_KEY: KEY } });
const previous = harness({ tarball: await packClient(previousVersion), env: { SCOUT_DIST_KEY: KEY } });
await fileDistKeyStore(support).write(KEY);
await runWebCommand(previous.context, ["install", "--no-restart"], { ...previous.deps, version: previousVersion });
const restarted: string[] = [];
Expand Down
Loading