Repository navigation
build(repo): latest stable dependencies, coverage floors and supply-chain checks - #465
Merged
Merged
Conversation
…ate coverage Python and web dependencies, pip and ClickHouse move to their newest stable releases (TypeScript stays on 6.0.3: svelte-check and typescript-eslint do not accept 7 yet). The whole suite measures the critical sources' branch coverage against governed floors; installs are checked for tampered hashes, reproducibility on both Pythons, registry signatures and known vulnerabilities. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MFyANPkiUZYq6rvGD2sFLf
The coverage job needs only coverage (the part count moves to a standard-library module); an unmeasured push fails; the CLI exit codes, npm exceptions and a missing reason are tested; the tamper check accepts only a hash refusal; the store contract and the corpus workflow name ClickHouse 26.10. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MFyANPkiUZYq6rvGD2sFLf
…igest Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MFyANPkiUZYq6rvGD2sFLf
The descriptor test replaced os.stat and os.lstat for the whole process, so coverage and pytest's own reporting tripped it once the suite ran under coverage. The trap now fires only while the reader is on the stack, through pathlib included. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MFyANPkiUZYq6rvGD2sFLf
Coverage's monitoring callbacks run inside the reader and stat its source the first time a branch fires; reproduced by running the test alone under the sysmon core. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MFyANPkiUZYq6rvGD2sFLf
While measuring, Hypothesis is derandomized and keeps no example database, so a commit measures the same coverage on every run; the check prints each module's value for the record. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MFyANPkiUZYq6rvGD2sFLf
The explain phase registers its own sys.monitoring tool, which drops coverage's events: a package's data came out empty locally and CI runs of the same code differed by up to 3%. Without it, two random test orders measure identical values. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MFyANPkiUZYq6rvGD2sFLf
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MFyANPkiUZYq6rvGD2sFLf
On 3.15 another sys.monitoring tool still dropped sysmon's events, so the same code measured differently from run to run; the settrace core is immune to it (identical values across random orders, measured here). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MFyANPkiUZYq6rvGD2sFLf
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MFyANPkiUZYq6rvGD2sFLf
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #429. Risk tier A (lock files, CI workflows, store engine versions). Label
owner-merge: it changes.github/workflows/,docs/process/git-workflow.mdand two new governed files.What changes, by scope item
package-lock.jsonwas regenerated.@types/nodestays on 24.19.2, the newest of the runtime's line (build(web): move to Node 26 LTS on its promotion date #23).critical_sourcesintools/ci/checks.toml, lists the code the critical tests protect: access, policies, compiler, correctness, adapters, exact values, isolation and the SQL guard. That is 39 files.tools.ci.coverage_run, measures branch coverage only when a part runs its share of the whole suite (PYTEST_COVERAGE=1fromselect.py).not-measuredmarker.python-coverage-<python>, combines the parts per Python and runstools/ci/coverage_floor.py. Each Python must reach every floor, so the lower of the two decides.python-tests-<python>, which is already required, now needs this job.pylock.toml.python-locknow shows that a tampered hash fails the install, with the newtools/ci/tamper.py.pip freeze --all.pip-auditruns on both Pythons. The newweb-supply-chainjob runsnpm audit signaturesandnpm audit.tools/ci/audit.pyjudges both reports againsttools/ci/audit-exceptions.toml. An exception needs its ecosystem, package, advisory ID, reason and review date. An expired, unused or incomplete exception fails too.Justified exception (rule of #429, tracked by #466)
typescriptsvelte-check4.7.8 accepts only^5 || ^6, andtypescript-eslint8.72.2 only<6.1.0Coverage floors: measured and reproducible
42884746434on08fff6a, then the run onc7b9393.sysmoncore sharessys.monitoringwith other tools in the run, which silently dropped its events.semanticsuite's data came out empty.ctracecore (sys.settrace), which is immune to this. It is about twice as slow, and it runs only on whole-suite runs.main.packages/store/src/orvex/store/source_exactness.pyis at 24%. It is a critical module whose branches the suite barely reaches. Its floor records today's value so it cannot fall; raising it is follow-up work.packages/server/tests/test_regular_file.pyreplacedos.statfor the whole process, so coverage's own callbacks inside the reader tripped it. The trap now fires only on calls the reader makes. The failure was reproduced by running the test alone under coverage.The ClickHouse image is pinned by digest (
sha256:b2bfa4d8…). The digest is read from the CI pull log, because this sandbox cannot reach the Docker registry.Files and their purpose
pyproject.toml,packages/*/pyproject.toml,pylock.toml,pylock.py314.tomlcoverageandpip-audit; the coverage settings; the locksweb/package.json,web/package-lock.jsontools/ci/coverage_run.py(new)tools/ci/coverage_floor.py(new)tools/ci/coverage-floor.toml(new, governed)tools/ci/audit.py(new)tools/ci/audit-exceptions.toml(new, governed)tools/ci/tamper.py(new)tools/ci/select.py,tools/ci/checks.tomlPYTEST_COVERAGE; the profile, new jobs and digests.github/workflows/*.ymlpackages/store/src/orvex/store/clickhouse.py,tests/conformance/test_clickhouse_engine.pydocs/process/git-workflow.md,CONTRIBUTING.mdtools/ci/parts.py(new)coveragecontracts/store/v1/README.mdENGINE_VERSION_UNTESTED(ADR-0022: the current LTS and the latest stable)New dependencies, one line each (
engineering.md§5):coverage, Apache-2.0: the reference branch-coverage tool. The alternative,pytest-cov, is a wrapper that adds a second dependency.pip-audit, Apache-2.0, maintained by PyPA: the standard Python vulnerability audit. The alternative,osv-scanner, is a Go binary outside the Python toolchain.Case matrix
test_combined_parts_give_each_imported_file_its_branch_coveragetest_coverage_floor.py(3)test_unreadable_or_empty_data_fails,test_data_that_measured_nothing_is_empty_and_failstest_each_floor_case_fails_and_a_rise_is_only_reportedtest_coverage_floor.py(3)python-teststest_the_workflow_measures_the_whole_suite_and_its_result_gates_the_teststest_select.py::test_coverage_is_measured_only_on_the_whole_suitetest_coverage_run.py(2)test_regular_file.py::test_a_swap_after_the_check_does_not_change_what_is_readcoverageinstalledtest_coverage_floor.py(3 new)test_audit.py(2 new)test_audit.py(5)test_tamper.py(2)tests/workspace/test_lock_files.py(already there)Laws
Verification
Run locally on Python 3.14:
tests/tools,tests/workspaceandtests/contracts: 981 passed, 1 not run (root);check,lint, 13 tests andbuildall pass, run on Node 22 (CI uses.nvmrc24);purpose,structureandtools.ci.checks: clean.CI runs the whole suite here, because the change touches the locks.
Size
About 900 lines, lock files aside. Half of them are tests.
https://claude.ai/code/session_01MFyANPkiUZYq6rvGD2sFLf
Generated by Claude Code