Use shared Reborn runtime HTTP egress for WASM, Script, and MCP

[serrrfirat]

Parent

#2987

Purpose

Build the shared Reborn runtime HTTP egress path used by capability runtime lanes instead of keeping HTTP transport, DNS/SSRF checks, limits, redaction, and resource accounting inside WASM-only or tool-specific code.

This issue is the egress mechanics dependency for #3088. #3088 owns production secret brokerage, credential account metadata, and brokered credential-session wiring. This issue owns the runtime-agnostic HTTP service, its enforcement semantics, and thin adapters for WASM, Script host HTTP, and host-mediated MCP HTTP.

Locked decisions

1. Implement this as one PR.

   • Do not split Use shared Reborn runtime HTTP egress for WASM, Script, and MCP #3085 into multiple PRs unless a reviewer explicitly asks during implementation.
   • Keep the PR focused on shared HTTP egress mechanics and adapters.

2. Put concrete transport/enforcement in a new host-runtime/runtime-egress layer over ironclaw_network.

   • ironclaw_network remains low-level policy/types/helpers.
   • Do not move reqwest clients, DNS resolution, redirect execution, response streaming, credential injection, events, or resource reconciliation into ironclaw_network.
   • Preferred placement is a small runtime-egress crate/layer, e.g. crates/ironclaw_runtime_egress, composed by ironclaw_host_runtime.

3. Reuse/port V1 HTTP behavior instead of inventing new limits.

   • V1 feature parity is the minimum bar.
   • Tightening is allowed only where it is cheap and does not regress known V1 behavior.
   • Do not ask broad limit-design questions during implementation; inspect the current code and carry the behavior forward.

4. Bypass prevention is required.

   • Host-mediated HTTP calls must go through RuntimeHttpEgress.
   • External process egress remains fail-closed/unsupported unless the PR adds a concrete sandbox/proxy/network-namespace enforcement path.

Existing V1 behavior to reuse or port

Use these paths as implementation references:

• src/tools/builtin/http.rs

  • MAX_RESPONSE_SIZE = 5 * 1024 * 1024 for text/model-visible responses.
  • MAX_SAVE_TO_SIZE = 50 * 1024 * 1024 for disk downloads; only carry this forward where the Reborn caller explicitly uses an artifact/file-output path.
  • DEFAULT_TIMEOUT_SECS = 30.
  • MAX_TIMEOUT_SECS = 300.
  • MAX_REDIRECTS = 3.
  • validate_url(...), validate_and_resolve_url(...), and build_pinned_client(...) implement parse/authorize, DNS validation, and reqwest DNS pinning.
  • Rejects private/local/internal literal and resolved IPs, including IPv4-mapped IPv6 private addresses in the newer network boundary.
  • Follows redirects only for simple GET requests, with per-hop URL validation, DNS re-resolution, pinned clients, and redirect cap.
  • Blocks redirects for non-simple requests.
  • Pre-checks Content-Length and also enforces a streaming hard cap.
  • Scans outbound caller-provided request URL/headers/body for leaks before host credential injection.
  • Strips sensitive response headers before returning data.
  • Scans response bodies for credential-like leaks before returning model-visible data.
  • Keeps recording/replay snapshots at the pre-injection boundary so injected credentials do not enter fixtures.

• src/tools/wasm/wrapper.rs

  • WASM http_request host import checks capability allowlists before egress.
  • Records per-execution request counts; current cap is 50 HTTP requests per execution.
  • Uses capability-configured max_response_bytes, default 10 MiB.
  • Uses caller timeout default 30 seconds, capped at 5 minutes.
  • Uses a 10-second connect timeout.
  • Resolves and pins DNS through validate_and_resolve_http_target(...) + ssrf_safe_client_builder_for_target(...).
  • Disables automatic redirects.
  • Scans request values before host credential injection and response bodies before returning to the runtime.

• src/tools/wasm/capabilities.rs, src/tools/wasm/capabilities_schema.rs, src/tools/wasm/storage.rs, migrations/V2__wasm_secure_api.sql

  • HttpCapability defaults: max_request_bytes = 1 MiB, max_response_bytes = 10 MiB, timeout = 30s.
  • Stored capability defaults: requests_per_minute = 60, requests_per_hour = 1000, max_request_body_bytes = 1 MiB, max_response_body_bytes = 10 MiB, http_timeout_secs = 30.

• src/tools/rate_limiter.rs

  • Reuse the existing sliding-window rate-limit semantics where host-mediated runtime HTTP needs per-user/per-capability rate limiting.

• crates/ironclaw_network from feat(reborn): add secrets and network boundaries #3077

  • Keep it metadata-only.
  • Use NetworkPolicyEnforcer / NetworkRequest / NetworkPermit for allow/deny decisions.
  • Preserve fail-closed max_egress_bytes: when configured, missing outbound-byte estimate is denied.
  • Preserve literal private-range denial, including IPv4-mapped IPv6 handling.

• crates/ironclaw_resources

  • Preserve the invariant that ResourceUsage.network_egress_bytes means outbound request bytes only.
  • Response bytes and response body limits must be measured separately; if a response becomes model-visible/process output, account that through the appropriate output-byte path.

What to build

Add a shared service with a narrow API similar to:

#[async_trait]
pub trait RuntimeHttpEgress: Send + Sync {
    async fn execute(
        &self,
        request: RuntimeHttpRequest,
    ) -> Result<RuntimeHttpResponse, RuntimeHttpEgressError>;
}

pub struct RuntimeHttpRequest {
    pub scope: ResourceScope,
    pub runtime: RuntimeKind,
    pub provider: Option<ExtensionId>,
    pub capability_id: Option<CapabilityId>,
    pub method: NetworkMethod,
    pub url: String,
    pub headers: Vec<(String, String)>,
    pub body: Option<Vec<u8>>,
    pub limits: RuntimeHttpLimits,
    pub credential_context: Option<RuntimeCredentialContext>,
}

pub struct RuntimeHttpLimits {
    pub max_request_body_bytes: u64,
    pub max_response_body_bytes: u64,
    pub timeout: Duration,
    pub connect_timeout: Duration,
    pub max_redirects: usize,
}

Exact names may change, but the boundary must provide:

• one host-composed HTTP service used by multiple runtime adapters;
• typed request/response structs with no raw secret material in runtime-facing values;
• stable sanitized error variants;
• computed outbound request-byte estimate passed to ironclaw_network before I/O;
• response byte measurement and hard response limits;
• optional redacted event hooks that are best-effort only;
• a credential injection seam for [Reborn] Wire production secrets/network boundary #3088, without implementing [Reborn] Wire production secrets/network boundary #3088's durable broker/account/session model here.

Required execution semantics

For every outbound HTTP attempt:

1. Parse the URL and reject unsupported schemes and URL userinfo (user:pass@host).
2. Map URL metadata into NetworkTarget and call NetworkPolicyEnforcer::authorize(...) before I/O.
3. Compute a conservative outbound request-byte estimate from method, URL, headers, and body, and pass it as NetworkRequest.estimated_bytes.
4. Enforce request-body limits before creating the transport request.
5. Scan caller-provided URL, headers, and body before any host credential injection.
6. Apply host credential injection only through the configured injection hook/seam.
   • The default/no-production-broker implementation may be no-op or test-only.
   • Runtime adapters must never receive raw SecretMaterial.
7. Resolve DNS once for the target, reject private/internal resolved IPs when policy denies private ranges, and pin the resolved addresses into the HTTP client.
8. Disable automatic redirects in the HTTP client.
9. If redirects are supported, preserve V1 behavior:
   • only simple GET requests may follow redirects;
   • max redirects: 3;
   • every hop re-parses the URL, re-checks leak policy, re-authorizes network policy, re-resolves DNS, and uses a new pinned client;
   • non-simple redirects fail closed;
   • credentials are not forwarded to a redirected target unless that target independently matches the credential policy in [Reborn] Wire production secrets/network boundary #3088.
10. Enforce timeout defaults and caps:
    • default request timeout: 30s;
    • max caller timeout: 300s;
    • connect timeout: 10s unless a stricter host config exists.
11. Pre-check Content-Length and stream/read response with a hard max_response_body_bytes cap.
12. Strip sensitive response headers before returning data:
    • authorization
    • www-authenticate
    • set-cookie
    • x-api-key
    • x-auth-token
    • proxy-authenticate
    • proxy-authorization
13. Scan response body before returning model-visible/runtime-visible bytes.
14. Return only sanitized errors/events/log fields. Do not expose raw backend errors, raw credentials, injected URLs, injected headers, or secret handles beyond redacted metadata.
15. Reconcile/return measured resource usage so callers can preserve:
    • outbound request bytes -> network_egress_bytes;
    • response bytes -> response/output accounting, not network_egress_bytes.

Runtime adapter requirements

WASM

• Reborn WASM WIT http-request imports must route through RuntimeHttpEgress via a thin adapter.
• The adapter may translate WIT-native shapes into RuntimeHttpRequest, but it must not create HTTP clients, resolve DNS, perform ad-hoc SSRF checks, inject credentials, or stream response bodies itself.
• If [Reborn] Re-carve WASM runtime lane #3086 has not yet landed the final WASM lane, this issue must at least add the shared egress service plus WASM adapter contract tests, and [Reborn] Re-carve WASM runtime lane #3086 must be updated/rebased to use this service rather than reintroducing a WASM-only transport.

Script

• Keep existing container/script process egress fail-closed unless a concrete enforcement layer is added.
• Existing Docker script backend behavior (--network none) must not be weakened.
• Add/define the host-mediated script HTTP surface so scripts that need HTTP call a host API/adapter backed by RuntimeHttpEgress rather than opening their own sockets.
• If the current Script runtime cannot expose host-mediated HTTP to external processes in this PR, document that as unsupported/fail-closed and add a contract test proving no ambient process network is granted.

MCP

• Host-mediated MCP HTTP/SSE transports must use RuntimeHttpEgress for protocol HTTP.
• MCP stdio/external server processes must not be treated as covered by host-mediated egress.
• If a stdio/external MCP server would need arbitrary outbound network access, this PR must either:
  • route it through a concrete sandbox/proxy/network-namespace enforcement path; or
  • mark it unsupported/fail-closed until process-level egress enforcement exists.

Out of scope

Do not implement these in #3085:

• durable encrypted secret storage;
• production SecretBroker;
• CredentialAccount metadata store;
• credential session/permit lifecycle;
• OAuth refresh/account selection policy;
• arbitrary Script/MCP process-level proxying unless it is the chosen bypass-prevention mechanism for this PR;
• non-HTTP credentials;
• request-body credential injection.

Those belong to #3088 or later process/runtime enforcement issues.

Sanitized error taxonomy

Use stable variants rather than raw strings. Suggested variants:

InvalidUrl
UnsupportedScheme
UrlUserinfoDenied
NetworkTargetDenied
PrivateTargetDenied
DnsResolutionFailed
RequestBodyTooLarge
ResponseBodyTooLarge
Timeout
TooManyRedirects
RedirectDenied
CredentialInjectionDenied
SecretLeakBlocked
RateLimited
ResourceLimitExceeded
BackendUnavailable

The exact enum can differ, but tests must assert that caller-visible errors/events do not include raw secret values, injected headers/query params, raw backend error chains, or sensitive response headers.

Acceptance criteria

• Add a shared RuntimeHttpEgress / HostHttpEgressService abstraction in the host-runtime/runtime-egress layer, not inside WASM-only code and not inside low-level ironclaw_network.
• Port/reuse V1 DNS, private-IP, redirect, timeout, request-size, response-size, response-header stripping, and leak-scan behavior listed above.
• Compute outbound request-byte estimates and pass them to ironclaw_network before I/O.
• Preserve ResourceUsage.network_egress_bytes == outbound request bytes only.
• Route Reborn WASM HTTP imports through the shared service or add a contract that forces the WASM lane to use it when [Reborn] Re-carve WASM runtime lane #3086 lands.
• Define/wire the Script host-mediated HTTP surface through the shared service, while keeping arbitrary process egress fail-closed/unsupported.
• Route host-mediated MCP HTTP/SSE through the shared service, while keeping stdio/external MCP process egress fail-closed/unsupported unless sandbox/proxy enforcement is implemented.
• Add architecture/ratchet tests that fail if Reborn runtime crates create direct outbound HTTP clients, use ad-hoc DNS/SSRF checks, or bypass RuntimeHttpEgress for runtime HTTP.
• Add caller-level tests for at least one WASM path and one non-WASM path proving both use the shared service.
• Add tests for DNS pinning/private-IP denial, IPv4-mapped IPv6 denial, redirect revalidation/cap, non-simple redirect denial, request-body cap, response Content-Length cap, streaming response cap, timeout cap, and sanitized errors.
• Add no-exposure tests with sentinel forbidden strings proving secrets/injected credentials do not appear in logs, events, errors, response headers, or recording/replay fixtures.
• Update [EPIC] Track Reborn architecture landing strategy and grouped PR plan #2987 and any affected Reborn runtime/WASM compatibility tracker text after the PR lands.

Dependencies / sequencing

• Depends on feat(reborn): add secrets and network boundaries #3077 for ironclaw_network and ironclaw_secrets substrate.
• Coordinates with [Reborn] Re-carve WASM runtime lane #3086 for the final WASM runtime lane re-carve.
• Unblocks [Reborn] Wire production secrets/network boundary #3088 PR C, which plugs production credential brokerage into this egress service.

Notes

This replaces older language that described a "WASM host HTTP gateway." WASM still needs a thin WIT adapter, but policy enforcement and HTTP transport must be runtime-agnostic and shared across Reborn runtime lanes.