[Pedro Henrique Penna (ppenna)]

📖 Description

Every MicroVM create-and-run request cold boots its guest kernel. This PR starts those sandboxes as clones of the signed agent's platform snapshot templates instead.

This PR is rebased onto main after #1691 merged and #1672 replaced MXC's native DLL bindings with the aci_edge_sandboxes crate. The pinned crate, 0.1.1, already stages the sandbox ABI 3 bundle from release v0.1.0-build188642429, and #1691 already registers image references before provisioning. This PR therefore no longer changes the bundle or the ABI bindings. It enables platform snapshots through the crate's SetupConfig::with_platform_snapshots.

• Platform snapshots. Create-and-run requests, with captured output (run_json and the executor) or live output (spawn_json), open the agent, finish capability and native validation, and then reopen it with platformSnapshots (maxVcpus: 1, maxMemoryMib: 256). These capacities fit sandboxes with the agent's default resources. The reopen reuses the bundle that the first open verified. The first request in a state directory captures two templates, one with a NIC and one without. Later requests in any process clone them, and concurrent first requests capture them once.
• Where they stay off. A request that native validation rejects never enables platform snapshots, so it never boots VMs to capture templates. --dry-run validation never enables them either. Lifecycle operations open the agent without them, so the containers they provision cold boot. Otherwise, whichever lifecycle call first opened the lifecycle state directory, such as stop, would boot VMs to capture templates. A create-and-run request that sets microvm.provision.memoryMib to anything other than 256 cold boots, and its start records the coldReason.
• Fallback. If the agent cannot capture the templates, MXC keeps the agent that it opened for validation, and the request cold boots. The diagnostic log records why.
• Cancellation. Ctrl-C during a template capture is observed when the capture returns, and the request ends before it provisions a sandbox.
• Diagnostics. The diagnostic log records each start's metadata: bootMode, template or coldReason, bootMilliseconds, and phases.

These measurements were taken before the rebase, with the same agent bundle driven through MXC's former DLL bindings. They come from a Windows 11 x64 WHP host (i9-12900H), using release builds, a warm image cache, and echo hello on public.ecr.aws/docker/library/alpine:3, with the variants interleaved:

	Sequential median (p90), n=30	4 concurrent requests, median wall time (5 trials)
Cold boot (template capture blocked)	354 ms (762 ms)	695 ms
Clone	346 ms (639 ms)	719 ms

The agent reported a median boot of 243 ms for clones and 283 ms for cold boots (n=15 each). The edge guest already cold boots quickly on this host, so clones save about 40 ms of boot, which is within the end-to-end noise.

Costs and limitations:

• The templates use about 512 MiB of disk in the state directory, as two 256 MiB memory snapshots. The agent's 64 GiB placeholder disk is sparse and uses no space.
• When the agent captures a template, it writes aci_edge_agent: template store …: captured generation N of the … template in … ms to the process's standard error. In the executor, that stream also carries the workload's standard error, and MXC cannot redirect it.
• A clone runs with the guest kernel's default CPU vulnerability mitigations, which a cold boot turns off. Its boot-console log has no guest boot output.
• Lifecycle containers, and create-and-run sandboxes with a memory override other than 256 MiB, cold boot as before.
• The agent documents that it captures an expired template again, while sandboxes that need it cold boot. A probe that ran a request every minute for an hour still cloned the first templates 69 minutes after capture, without capturing them again or writing to standard error. I have not tested the behavior across a host restart.

This changes the default boot behavior, but no request schema, policy field, public API, or pinned runtime, so I don't believe it needs a spec. I'm happy to write one if maintainers prefer. Given the measurements above, maintainers may also prefer to leave platform snapshots off until they pay off on more hosts.

🔗 References

Closes #1692
Builds on #1691 and #1672

🔍 Validation

On a Windows 11 x64 WHP host (build 26300), from src, at b2f3f7a (rebased onto main at 03efb61, plus the review follow-ups):

• cargo fmt --all -- --check: passed.

• cargo clippy --locked -p mxc-sdk --features microvm --target x86_64-pc-windows-msvc --all-targets -- -D warnings: passed.

• cargo test --locked -p mxc-sdk --features microvm --target x86_64-pc-windows-msvc --lib: 2668 passed, 0 failed, 1 ignored.

• cargo test --locked -p mxc-sdk --features microvm --target x86_64-pc-windows-msvc --no-fail-fast --test nvx_e2e --test nvx_bundle_integrity -- --include-ignored --nocapture --test-threads=1, with LOCALAPPDATA set to an empty temporary directory:

  • nvx_bundle_integrity: 9 passed, 0 failed.
  • nvx_e2e: 24 passed, 0 failed, 0 ignored, and all twelve nightly VM_EXECUTED scenario markers printed.

  With this host's own LOCALAPPDATA, two nvx_bundle_integrity tests fail before they reach this change. An earlier build left %LOCALAPPDATA%\Microsoft\MXC\nvx with an inherited DACL, which MXC's state-root check rejects.

• node --test scripts\tests\microvm-validation.test.js (from the repository root): 53 passed, 0 failed.

• node scripts\ci\resolve-validation-test-matrix.mjs --plan nightly (from the repository root): passed.

• The intermediate commit 3b11170 builds, and its 58 NVX unit tests pass.

New and changed coverage:

• signed_bundle_executor_starts_sandboxes_from_platform_snapshots runs nightly. It is listed in the staging and validation scripts and their script tests.

  • In an empty state directory, the first executor request captures the templates and starts as a noNic clone on the host CPU profile.
  • A request with a network starts as a nic clone with eth0.
  • A spawn_json request in another empty state directory also captures the templates. Temporarily making spawn_streaming open the agent without platform snapshots fails this check.
  • A dry run captures no templates.
  • A state directory whose template store cannot be opened cold boots, and the log gives the reason.

• native_validation_rejects_requests_without_booting_a_vm, previously streaming_and_lifecycle_validation_reach_the_crate_without_a_vm, sends a proxy that native validation rejects through run_json and spawn_json. It then checks that no agent state directory has a template store. It fails if platform snapshots are enabled before validation.

• signed_bundle_executor_console_interrupt interrupts three phases:

  • capture: the first request's template capture, in an empty state directory.
  • startup and execution: these phases make a warm-up request first, so their interruptions reach the sandbox rather than a template capture.

  Each phase checks that every OpenVMM process the executor started has terminated, and that a later request succeeds with the same state directory.

• Unit tests cover:

  • the setup JSON with and without platform snapshots, including that the pinned crate accepts the capacities for its default resources;
  • the order of validation, enabling platform snapshots, and provisioning, including cancellation during the snapshot step and proxy-credential withholding for its errors;
  • the start log line.

✅ Checklist

• Signed the Contributor License Agreement
• Linked to an issue
• For features or changes requiring a spec: linked the issue with maintainer-applied feature-accepted
• For implementation: linked the required approved and merged design/API spec, or explained why no spec is required
• Feature issue remains open unless this PR completes all agreed implementation, validation, and documentation
• Updated documentation (if applicable)
• Updated Copilot instructions (if build, architecture, or conventions changed)
• If this PR changes Cargo.lock, the dependency-feed-check check passes (see pull request builds)

📋 Issue Type

• Bug fix
• Feature
• Task

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

🟡 Changes recommended

Template VMs and snapshots can be created before invalid requests are rejected, and first-time capture cancellation and cleanup remain untested.

2 open issues

• Validate requests before creating template VMs · New
• Test cancellation and cleanup during template capture · New

What changed in this PR

Enables NVX create-and-run requests to reuse platform snapshot templates, building on the persistent state directory introduced in #1691. The reviewed changes use the existing agent crate without changing the pinned bundle or public APIs.

Changes:

• Enable snapshot cloning for captured and streaming execution, with cold-boot fallback; keep snapshots disabled for dry runs and lifecycle operations.
• Log boot metadata and document template storage, resource limits, and diagnostic behavior.
• Add snapshot scenarios to nightly validation and warm up templates before existing sandbox-interruption tests.

File	Description
src/​mxc-sdk/​tests/​nvx_e2e.rs	Adds clone, streaming, dry-run, and fallback coverage; warms interruption tests.
src/​mxc-sdk/​src/​backends/​nvx/​runner.rs	Enables snapshots for execution and logs sandbox boot metadata.
src/​mxc-sdk/​src/​backends/​nvx/​crate_runtime.rs	Configures templates, implements cold-boot fallback, and preserves start metadata.
scripts/​tests/​microvm-validation.test.js	Checks discovery and execution requirements for the snapshot scenario.
scripts/​ci/​stage-nvx-validation-tests.ps1	Requires snapshot-test discovery in staged artifacts.
scripts/​ci/​run_backend_validation_tests.ps1	Requires the snapshot scenario’s execution marker.
docs/​development/​build-and-test/​ci-validation-infrastructure.md	Documents nightly snapshot coverage.
docs/​backends/​nvx/​nvx.md	Documents snapshot behavior, storage costs, exclusions, and diagnostics.

_{Review effort: Balanced}

[Copilot]

🟢 Approval recommended

No active review issues are recorded for the snapshot-template integration.

No open issues 🎉

2 resolved since last review

• Test cancellation and cleanup during template capture
• Validate requests before creating template VMs

_{Review effort: Balanced}