Repository navigation
tests: delete the runtime adapter, its runner and its CI (SL5.2) - #1499
Pedro Henrique Penna (ppenna) wants to merge 2 commits into
Conversation
SL5.2 deletes the runtime adapter, tests/runtime-adapter, now that the case matrix gives each of its tests a fate and the acceptance that replaces it runs on real VMs (#1118). CI, its only consumer, goes first. ci.yml no longer builds the adapter's guest agent in distro-artifacts or runs the WHP job of its scenarios, nvx-microvm-adapter-whp, and the KVM and MSHV guest and sandbox jobs no longer select those scenarios. Required status check, Publish development release and Record performance results stop waiting for that job, and check-required-ci stops requiring it. run-nvx-microvm-tests.yml loses its runtime-adapter selector, the build, download and scenario steps of the adapter, and the download of the Ubuntu distro layer for the adapter alone; the default suite and the upload of failure logs and duration reports take back the conditions that they had before that selector. check-nvx-rs no longer checks the adapter's protocol-only, host-only and guest-only configurations, validate-nvx no longer checks test-runtime-adapter's help, and the change classifier no longer counts the adapter among the inputs of the microVM tests. The tests of CI's wiring follow the case matrix: the three that only checked the adapter's wiring retire, with RuntimeAdapterSuiteTests' test of the suite's CI steps, and the six that stay lose their adapter parts. Two of them take back their form from before WHP's adapter job. doc/ci.md, doc/contribute.md, the repository instructions, the setup workflow's comment and the OpenVMM promotion skill stop describing the adapter's jobs and checks, and the measurement that moved WHP's adapter scenarios to a job of their own goes with that job. Part of #1119 (SL5.2). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
With CI no longer building or running it, the runtime adapter, tests/runtime-adapter, leaves NVX: its manifest, lockfile, README, scenarios.txt and 64 Rust sources, 49,689 lines in 68 files, with its 157 scenarios and 169 other tests, each of which doc/acceptance-case-matrix.md gives a fate. Its runner, scripts/nvx_tools/runtime_adapter_tests.py, goes with it, and so do nvx.py's test-runtime-adapter and RuntimeAdapterSuiteTests, which tested the runner. check_test_changes.py no longer names the adapter's scenarios.txt as test data, and its classification test drops the adapter's crate and paths; .gitignore no longer ignores the adapter's target directory; and the change classifier no longer lists the runner's module among those that no benchmark job runs. The docs stop describing the adapter: the usage of test-runtime-adapter, the tests/ directory of the project structure, the adapter's lockfile in the distribution docs, and the mentions of its configurations, excerpts and capture interfaces in the nvx and vectors READMEs. The case matrix keeps every fate, and its prose and the README's index say in the past tense what the deletion made past. Nothing in nvx-rs used the adapter, a crate with a workspace and a lockfile of its own, and the test layout's allowlist held no entry of it. Part of #1119 (SL5.2). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This pull request changes existing testsIt modifies or removes 1445 existing test asset(s), listed below. Review each with care: a change to a test can hide a regression that the test caught. See doc/contribute.md#changes-to-existing-tests. Test code
... and 1266 more, which the job's summary lists. Test data
End-to-end tests
Test policy
CI test wiring
... and 58 more, which the job's summary lists. Relocated unchanged: 0; added: 0. Compared the merge base |
There was a problem hiding this comment.
🔵 Needs a closer look
The adapter deletion depends on SL5.1b/SL5.1c replacements absent from this snapshot, so prerequisite rebases and successful KVM/MSHV/WHP cold-boot and snapshot acceptance still need confirmation.
No open issues 🎉
What changed in this PR
Removes NVX’s obsolete runtime adapter and its dedicated tooling as SL5.2 of the library-slimming workstream (#1119). This deletion depends on the replacement real-VM acceptance work in #1118; retained runtime behavior is outside its scope.
Changes:
- Deletes the adapter crate, its tests and scenario inventory, and its CLI runner.
- Removes adapter-specific CI builds, jobs, selectors and required-status dependencies.
- Cleans up documentation and tooling references while preserving test dispositions in the acceptance matrix.
| File | Description |
|---|---|
tests/runtime-adapter/tests/vm_steps.rs |
Deletes VM launch-step scenarios. |
tests/runtime-adapter/tests/vm_restart.rs |
Deletes restart and runtime-identity scenarios. |
tests/runtime-adapter/tests/step_hook.rs |
Deletes step-hook tests. |
tests/runtime-adapter/tests/sandbox_names.rs |
Deletes sandbox-name regression test. |
tests/runtime-adapter/tests/runtime.rs |
Deletes artifact identity tests. |
tests/runtime-adapter/tests/listener_client.rs |
Deletes listener-client regressions. |
tests/runtime-adapter/tests/initramfs.rs |
Deletes adapter initramfs tests. |
tests/runtime-adapter/src/worker/tests.rs |
Deletes worker child-management tests. |
tests/runtime-adapter/src/worker/probes/tests.rs |
Deletes worker probe tests. |
tests/runtime-adapter/src/worker/order.rs |
Deletes mapping and startup sequencing. |
tests/runtime-adapter/src/worker/mapping/tests.rs |
Deletes mapping probe tests. |
tests/runtime-adapter/src/worker/executions/tests.rs |
Deletes execution cancellation regression. |
tests/runtime-adapter/src/worker/entries/tests.rs |
Deletes entry probe tests. |
tests/runtime-adapter/src/worker/capture/tests.rs |
Deletes capture and handshake tests. |
tests/runtime-adapter/src/runtime.rs |
Deletes adapter runtime discovery. |
tests/runtime-adapter/src/probe/tests.rs |
Deletes probe encoding tests. |
tests/runtime-adapter/src/probe.rs |
Deletes probe and fact codecs. |
tests/runtime-adapter/src/lib.rs |
Deletes adapter module exports and constants. |
tests/runtime-adapter/src/initramfs.rs |
Deletes guest-agent initramfs assembly. |
tests/runtime-adapter/src/host/order_error.rs |
Deletes host sequencing errors. |
tests/runtime-adapter/src/host/map_reply.rs |
Deletes mapping reply type. |
tests/runtime-adapter/src/hook.rs |
Deletes launch pause and exit hooks. |
tests/runtime-adapter/src/generations/tests.rs |
Deletes generation eviction tests. |
tests/runtime-adapter/src/generations.rs |
Deletes adapter generation management. |
tests/runtime-adapter/scenarios.txt |
Deletes adapter scenario inventory. |
tests/runtime-adapter/Cargo.toml |
Deletes adapter crate manifest. |
scripts/test_test_changes.py |
Removes adapter classification examples. |
scripts/nvx.py |
Removes test-runtime-adapter registration and import. |
scripts/nvx_tools/ci.py |
Removes adapter gates and path classification. |
scripts/check_test_changes.py |
Removes adapter scenario-data pattern. |
README.md |
Describes the adapter deletion in past tense. |
nvx-rs/tests/vectors/README.md |
Removes adapter vector-consumer documentation. |
nvx-rs/README.md |
Removes obsolete adapter references and links. |
doc/usage.md |
Removes adapter command documentation. |
doc/project-structure.md |
Removes adapter directory and build-output entries. |
doc/distribution.md |
Removes adapter lockfile guidance. |
doc/contribute.md |
Removes adapter validation and test-placement guidance. |
doc/acceptance-case-matrix.md |
Updates historical wording while retaining dispositions. |
.gitignore |
Removes adapter build-output exclusion. |
.github/workflows/run-nvx-microvm-tests.yml |
Removes adapter selector and steps; simplifies conditions. |
.github/workflows/copilot-setup-steps.yml |
Updates only the Rust-version check comment. |
.github/workflows/ci.yml |
Removes adapter artifacts, WHP job and dependencies. |
.github/skills/nvx-openvmm-promote/SKILL.md |
Removes adapter-specific validation guidance. |
.github/copilot-instructions.md |
Removes adapter validation instructions. |
.github/actions/validate-nvx/action.yml |
Removes adapter command-help checks. |
.github/actions/check-nvx-rs/action.yml |
Removes adapter configuration checks. |
Review effort: Balanced
|
No change needed here, on Copilot's review of |
Leaving code only. This PR changes only code that this effort removes from NVX: the test adapter (
tests/runtime-adapter), its runner, and its CI jobs and gates, with the edits that remove references to it. Nothing that stays changes, and no later PR uses what it changes, so the size limits for kept code don't apply.Part of #1119 (SL5.2, "The 44K-line test adapter is gone"). Parent #1086, umbrella #1076.
Needs SL5.1c (#1118, "Snapshots work end to end on real VMs"), which hasn't merged: this PR merges only after it, and rebases over it and over SL5.1b (#1496), which edit the same workflow, microVM harness and case matrix.
#859 (the runtime adapter's consumer-contract scenarios C2, C3, C6 and C7 on real VMs) is closed as not planned, now that #816 is rescoped; the adapter's deletion is tracked in #1119, which this PR completes.
What and why
The case matrix,
doc/acceptance-case-matrix.md, gives each test of the runtime adapter a fate, and the acceptance that replaces the adapter runs on real VMs (SL5.1a to SL5.1c, #1118). So the adapter goes, consumers first:ci.yml:distro-artifactsno longer builds and uploads the adapter's guest agent; the WHP job of its scenarios,nvx-microvm-adapter-whp, is gone, and the KVM and MSHV guest and sandbox jobs no longer setruntime-adapter;Required status check,Publish development releaseandRecord performance resultsstop waiting for that job.run-nvx-microvm-tests.yml: theruntime-adapterselector goes, with the adapter's build, download and scenario steps, and the download of the Ubuntu distro layer for the adapter alone. The default suite and the uploads of failure logs and duration reports take back the conditions that they had before that selector (73b168a).check-nvx-rsno longer checks the adapter's protocol-only, host-only and guest-only configurations;validate-nvxno longer checkstest-runtime-adapter --help.scripts/nvx_tools/ci.py:check-required-cino longer requires the adapter's job, and the change classifier no longer countstests/runtime-adapter/among the inputs of the microVM tests.RuntimeAdapterSuiteTests' test of the suite's CI steps; the six that stay lose their adapter parts (see Changes to existing tests).doc/ci.md,doc/contribute.md, the repository instructions, the setup workflow's comment and the OpenVMM promotion skill stop describing the adapter's jobs and checks. The measurement that moved WHP's adapter scenarios to a job of their own (ci: shorten the runs that restore the VMM tests or skip both OpenVMM suites #982) goes with that job, and the paragraphs that it changed take back their wording from before it.tests/runtime-adapter(its manifest, lockfile, README,scenarios.txtand 64 Rust sources: 68 files and 49,689 lines atf834bcb9, with 157 scenarios and 169 other tests),scripts/nvx_tools/runtime_adapter_tests.py,nvx.py'stest-runtime-adapter, andRuntimeAdapterSuiteTests.check_test_changes.pyno longer names the adapter'sscenarios.txtas test data,.gitignoreno longer ignores itstarget/, and the change classifier's list of modules that no benchmark job runs drops the runner. The docs stop describing the adapter:test-runtime-adapterindoc/usage.md,tests/indoc/project-structure.md, the adapter's lockfile indoc/distribution.md, and the nvx README's and the vectors README's mentions of its configurations, excerpts and capture interfaces.scripts/test-layout-allowlist.jsonheld no entry of the adapter, so it doesn't change.nvx-rsor in the scripts imports it.Every removed test, and its fate
check_test_changes.pyreports 304 removed tests. Each has its row in the case matrix, with its fate:tests/runtime-adapter/tests/:template_capture3,template_operations4,clone_start11,host_session26,staged_images4,layered_launch16,vm_restart3,vm_steps3,vm_template8,vm_clone36vm_layered's 43 scenarios, which itsscenarios!macro generates, so the report shows them as the removal ofvm_layered.rs's module and helpersvm_layered(43): 38 rewritten, 5 retiredsrc/**/tests.rs78,tests/guest_session.rs66,tests/independent_build.rs9,tests/initramfs.rs4,tests/listener_client.rs3,tests/runtime.rs3,tests/sandbox_names.rs1,tests/step_hook.rs5scripts/test_nvx_tools.py::RuntimeAdapterSuiteTests, all 16scripts/test_nvx_tools.py::CiConfigurationTests:test_nvx_rs_checks_the_runtime_adapter_in_each_configuration,test_ci_change_classification_runs_microvm_tests_for_the_runtime_adapter,test_runtime_adapter_scenarios_take_a_guest_agent_that_linux_buildsci.yml: the jobnvx-microvm-adapter-whp, andrun-nvx-microvm-tests.yml, which ci.yml now names 9 times instead of 10nvx-microvm-adapter-whpretired; the adapter's steps ofrun-nvx-microvm-tests.ymland its three configurations ofcheck-nvx-rsretired or rewritten as their rows sayTogether, the 114 and
vm_layered's 43 are the matrix's 157 rows of the adapter's scenarios, of which 114 are rewritten and 43 retired. The matrix keeps none of the adapter's tests, so no test that it keeps leaves. Each name that the report gives a removed or modified test has its row, and no row names any of them as coverage (git grepof each name in the matrix); the rows of the seven modified tests already say what this PR removes from them.What the deletion leaves without a user
Not deleted here (SL6.x deletes providers after their consumers); listed for those slices. Each has no user outside its own module once the adapter goes, as
git grepover the non-test code ofnvx-rs/srcshows:nvx::exec(Runner,Peer, the frames,Limits,Timers,Unit), used only byguest::exec::binding, andguest::exec::binding::Binding, which nothing uses.host::template'sCloneControl,ClonePlan,CloneRequest, the clone and failure records (CloneRecords,CloneFailure,FailureRecord), store-based reattach (reattach_clone,ReattachedClone) andRestoreDeadlines.host::template::Store(create_templateamong its methods), its leases (Acquired), its capture transactions (Created,Progress,Teardown,Retirement) andTemplatePlan.AdmissionFence,CapturePreparation(prepare),CaptureBarrier(cancel),Restored(into_parts) andCaptureError(into_cause).nvx::guest::clone, the compatibility alias ofguest::activationthat eb1c37e kept for the adapter, has no user either.bundledare gone #1122): the planning of an adapter's own guest,SandboxPlan::adapter,SandboxPlan::check_adapter_launchandPreparedLaunch::adapter_arguments, now used only insidehost::sandboxand by the old clone path.Follow-ups, not in this PR
scripts/nvx_tools/kernel_features.pystill names the adapter's tests ("the runtime adapter's clone tests", "GPT tests", "vm_layered test", and "the runtime adapter") among the validation of five kernel features, and stays unchanged here. One later PR updates every entry that names the adapter, after SL6.11b deletesguest::sys::entry's per-exec parts, which thediagnosticsfeature names as its consumer: that PR decides that feature's validator, today only the retiredvm_layered::root_profiles_enter_the_root, and its setting, with one rebuild of the kernels, whose cache key the inventory is an input of.nvx-rs/src/host/template/clone/restore.rs's rustdoc gives the test adapter as the origin of two deadlines of the old clone path; SL6.2b deletes the file.scripts/local_gates.py, which exists ondevonly, keeps its rules for the adapter's threecheck-nvx-rssteps andtests/runtime-adapter/among its nvx-rs inputs; they go whenfeature/nvx-refactormerges intodev.Size
Measured against
feature/nvx-refactorf834bcb9:if:conditions ofrun-nvx-microvm-tests.ymlthat dropinputs.runtime-adapter,check-nvx-rs's description, the two lines ofci.py's path patterns that droptests/runtime-adapter/andruntime_adapter_tests, andcheck_test_changes.py's test-data pattern. The counted files lose 178 lines of code.ci.yml,run-nvx-microvm-tests.yml,check-nvx-rs,validate-nvx,ci.py,nvx.py), pluscopilot-setup-steps.yml, whose comment said thatcheck-nvx-rschecks the adapter, andcheck_test_changes.py, whose pattern namedscenarios.txt. The 69 files that it only deletes don't count.Commits
ci: stop building, running and gating on the runtime adaptertests: delete the runtime adapter and its runnerNot in this PR
Rewriting tests that stay (SL5.1a to SL5.1c); deleting the providers above (SL6.x); the follow-ups above; changing retained VM, snapshot, CLI or library behavior; any other CI cleanup; and the later docs pass (SL6.10f), beyond the references that the deletion leaves dangling.
Validation
Locally on Windows, on this content (the rebase onto
f834bcb9changed no line of the two commits;git range-diffshows them equal):python -m ruff check scripts .github/specula,python -m ruff format --check scripts .github/speculapython -m pyright --pythonplatform Linux,--pythonplatform Windowspython scripts/check_markdown_links.py,python scripts/check_test_layout.py,python scripts/nvx.py check-kernel-configvalidate-nvx's Windows step:python -m compileall -q scripts,python -m unittestof its 22 modulespython scripts\test_adversarial.py,python .github\skills\nvx-host-connect\scripts\test_hosts.py, and thenvx.py --helpchecksnvx.py test-runtime-adapter --helpnow exits 2python -m unittest test_nvx_tools.CiConfigurationTests test_nvx_tools.RuntimeAdapterSuiteTests test_test_changes.ClassifyTestsNVX_BUNDLE=skip:cargo clippy --locked -p nvx --lib --no-default-features -- -D warnings,cargo build --locked,cargo clippy --locked --all-targets --all-features -- -D warnings,cargo fmt --all --check,RUSTDOCFLAGS="-D warnings" cargo doc --locked --no-deps --all-featurescargo +1.89 check --locked --all-targets --all-featurescargo test --locked --no-default-features --test openvmm_vectors -- --ignored,--test openvmm_cpu_profiles -- --ignoredcargo test --locked --all-features --lib error::tests, which readsnvx-rs/README.mdThe case matrix's reference check. SL5.1a's check, a script outside the repository, verifies that each Rust test, microVM scenario, Python test class and path that a row names exists, and that the matrix has exactly one row for each of the adapter's scenarios and other tests. It read the adapter's inventory,
scenarios.txtand the adapter's sources, from the checkout, which no longer has them. So the deletion forces one change on it, and only that one: the adapter's inventory, the Python tests that retire with it, and the runner that a retired row names as what leaves are read from the base revision, where they still exist. Everything else, every pointer to a test, scenario, class or path that covers a rewritten or kept test, is checked in this branch. It passes: 157 scenarios and 169 other tests of the adapter, 52 microVM scenarios and 26 CI Python tests, each with its one fate. A pointer to a missing test, a renamed scenario, and a retired test placed in a kept row each make it fail.Dry selection.
local_gates.py,dev's copy, against this branch's own actions and workflows: with its three rules for the adapter'scheck-nvx-rssteps, it stops at "steps no longer present" for exactly those three; without them, every other scripted step ofcheck-quality,check-nvx-rs,validate-nvxandci.yml's CLI jobs is mapped, and it selects test-changes, markdown-links, test-layout, quality-code, openvmm-vectors, openvmm-cpu-profiles, the nvx-rs gates, nvx-cli and openvmm-ci on both systems.nvx.py classify-ci-changeson this change selects every suite forpull_request,merge_groupandpush; on the adapter's paths alone, it now selects none.nvx.py check-required-ci, fed the expected result of each job, passes forpull_request,merge_groupandpush;Required status check'sneedsequalci.py's 31 jobs.ci.yml,run-nvx-microvm-tests.yml,copilot-setup-steps.ymland both actions parse, every caller ofrun-nvx-microvm-tests.ymlpasses only inputs that it declares, and no job condition is unbalanced.Left to CI: the Linux nvx-rs gates (public API, macOS, the guest binaries), which only the edited READMEs and action select, since no Rust code changes; the full nvx-rs test suites; and the hypervisor jobs (microVM, debug kernel, platform and packages). No microVM scenario ran: no runtime path changes.
Changes to existing tests
python3 scripts/check_test_changes.py --base origin/feature/nvx-refactorwithdev's script, which the Test changes workflow runs, lists 1445 changes to existing tests, none relocated or added; this branch's own script lists 1444, all butscenarios.txt, which it no longer names as test data.tests/runtime-adapter/scenarios.txt, as test data.RuntimeAdapterSuiteTestsgoes with its helpers_scenariosand_run.scripts/check_test_changes.py. Its test-data pattern dropstests/runtime-adapter/scenarios.txt.CiConfigurationTests.test_ci_jobs_download_exactly_the_guest_artifacts_that_they_read: its map of thenvx.pycommands to the guest files that they read dropstest-runtime-adapter.CiConfigurationTests.test_ci_change_classification_selects_suites_by_input: drops the adapter's README from the documentation case, and the case of an adapter source as an input of the microVM tests alone.CiConfigurationTests.test_ci_change_classification_covers_each_suite_input: drops its check that the microVM workflow builds intests/runtime-adapter, and the adapter among the microVM tests' inputs.CiConfigurationTests.test_ci_shares_openvmm_inputs_and_binary_artifacts: drops the adapter's job among the consumers of the OpenVMM artifacts; the microVM workflow downloads 10 artifacts, not 12 (the distro layer for the adapter alone, and the adapter's guest agent).CiConfigurationTests.test_ci_runs_the_steps_after_the_default_microvm_suite_beside_itandCiConfigurationTests.test_every_microvm_job_uploads_its_scenario_durations: back to their form from before 73b168a, which split WHP's adapter job off, less the adapter's scenario step among the guest and sandbox steps: one selector, two jobs per backend, and every job uploads its failure logs and duration reports, withfailure()andalways().ClassifyTests.test_paths_and_their_categories: drops the adapter's crate and its two paths,scenarios.txtandtests/runtime.rs.test_test_layout.pystill checkscrate_ofacross nested crates.scripts/test_nvx_tools.pydropruntime_adapter_tests.run-nvx-microvm-tests.yml, its three configurations incheck-nvx-rs,validate-nvx's two help checks, andci.yml's job, selectors,needsand environment. 14 modified lines:check-nvx-rs's description, and the conditions that dropinputs.runtime-adapter, which the report pairs with the removed lines of the distro layer's download for the adapter alone. 5 lines "that may skip": the same fiveif:conditions, which lose a term that skipped steps, so they skip less, not more: the download of the distro layer for the whole default run, the two default-suite steps, and the two uploads. 2 removed tests: the jobnvx-microvm-adapter-whp, andci.yml's ninth instead of tenth use ofrun-nvx-microvm-tests.yml.