Repository navigation
Vault refusal tests: 18 unasserted codes, extension bounds, closed lane, create_pda's guard halves - #100
Merged
Conversation
shuv-amp
added a commit
that referenced
this pull request
Oct 17, 2026
Finding 106's row in docs/09 said its 33 mutants were re-run on 16 Oct at bf8ba41. That commit is not an ancestor of this branch or of the stack above it and is on no remote branch, so the citation would point at nothing once the stack lands. The owner's review of #99-#109 found it. bf8ba41 and 86b777b (the head of PR #100, vault-error-tests-d2, the D2 replay of vault-error-tests-r4) have the same tree, 36187b4 (`git rev-parse bf8ba41^{tree} 86b777b^{tree}` prints it twice). So the run was a run on 86b777b's tree, and the row now says "on tree 36187b4, the tree of 86b777b (PR #100's head)". Why not just "at 86b777b": it was committed on 17 Oct, after the run; and main squash-merges (1045cd9 has one parent), so after PR #100 lands 86b777b is reachable only as GitHub's refs/pull/100/head. The tree id stays checkable from main: squashed bottom first, main's commit for PR #100 has that tree. "PR" keeps "#100" from reading as finding 100, the row above. These points came from two code-review passes. Sibling search: every 7-12 hex token in tracked .md/.py/.sh/.mjs/.rs/ .toml files at autopilot/integration 3e9f58d that names a local commit (51 tokens) was checked with git merge-base --is-ancestor against integration. bf8ba41 was the only one that is not an ancestor. Docs only. tools/claims.py: claims OK (26 files). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 17, 2026
Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set with fewer than threshold + 2 keys. Every test fixture and most tests' own sets were 2-of-3. This moves them to 2-of-4, which today's rule also accepts, so this commit changes no program: Floor, Vault and Harbor hash 508e5a99 / b07655d9 / 543be60b, as before. - Fixtures: Protected (Floor) and Market (Vault, through Harbor) now hold 4 guardians. Each test that built its own 3-key set gets a 4-key set with the same threshold and the same assertion. - Two tests changed meaning with a 4th key and were fixed, not loosened: a_quorum_ejects_a_rogue_guardian_who_vetoes_everything (Floor) and a_quorum_ejects_a_guardian_who_freezes_forever (Vault). With 4 keys, the last eject was refused only because the target's own vote is masked. Both now eject a second guardian (a set an ejection shrank to threshold + 1 still ejects) and then show the 2-of-2 refusal. - Account sweeps: an eject's signers may now be any two of the three others, so each of those two slots is free for the third, never for the target or the other signer again. Vault 5,365 and Floor 1,710 substitutions, refused (docs/09, dated 16 Oct). Listing all three others at each slot would have skipped the duplicate: with a mutant that counts a guardian listed twice as two (governance.rs:270), the sweeps listing all three pass, and these fail at "eject: account 2 replaced by key ..." (Vault and Floor). No other program test kills that mutant (tasks/aud2-guardian-margin-impl/dup-mutant-*.log). - floor_differential: the guardian pool is 6 keys, 4 registered; the proposed sets are 2 to 6 keys. New tests (pass on both rules; they show what the margin buys): - a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_set (Floor): g0's key lost, g1 rogue stops the program; the lost key does not help a quorum; the honest pair ejects g1 and restores; neither can veto the owner's refill, which executes after 72h. - two_rogues_in_turn_are_both_ejected_from_a_two_of_four_set (Floor): the second rogue vetoes the refill and is ejected from the 2-of-3. - a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_vault: the same in the Vault (freeze, eject, unfreeze, refill). Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest and quote sweep): the counts and the sweep figures here are from runs there (tasks/aud2-guardian-margin-impl/r4/). The E2E run, the mutants and the new-rule run below were made before, on e8640a7; this commit's programs and core are the same as there but for one comment line in views.rs (line-neutral, same hashes). Evidence: - E2E=1 scripts/check-programs.sh: PROGRAM CHECKS PASSED (Floor 49, Vault 98; S1 and E2E passed; the resumed setup found all 14 steps; vault check, Floor demo and deploy dry run passed), with the next commit's deploy rehearsal in the tree. - tools/claims.py: OK (floor 49, vault 104, programs 153). - Under the new rule (core from two commits on, programs rebuilt), only the two tests that state the old rule fail: an_n_of_n_guardian_set_is_ refused (accepts 3-of-4) and the Floor differential's model. - Mutants (tasks/coverage-map/mutants.sh), each red, then restored: AV1 Floor live_guardian ignores liveness: KILLED by a_lost_key_and_a_ rogue_do_not_lock_a_two_of_four_set (+2 others). AV2 the Vault's: KILLED by ..._two_of_four_vault (+2 others). AQ1 Floor restore quorum ignores liveness: KILLED by ..._two_of_four_set and the differential only. AQ2 the Vault's unfreeze quorum: KILLED by ..._two_of_four_vault and a_silent_guardian_does_not_count_toward_a_quorum. AE1 can_eject refuses at count == threshold + 1 (AUD-2 option (a)): KILLED by two_rogues_in_turn_..., and the two eject tests above. Hashes after: 508e5a99 / 543be60b / b07655d9. Split on 16 Oct from ec0cacc at the owner's review (D2): the deploy rehearsal's move to 4 guardians is now the next commit. programs/ and programs/tests/ here are byte-identical to ec0cacc, so its runs below hold for this commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set with fewer than threshold + 2 keys, so the deploy rehearsal must register 2-of-4 sets. Split from ec0cacc at the owner's review (D2); the text is that commit's, unchanged: Deploy rehearsal (cloud/deploy market.mjs, vault-check.mjs, floor-demo.mjs): 4 guardians (keys/guardian3), so the dry run keeps working once the rule lands; one more funding step (14, was 13). The devnet market has 3 guardian files. A first version of this change loaded guardian3 everywhere: vault-check.mjs failed there with ENOENT, and heartbeat silently created a key nobody holds as a guardian. Now market.mjs guardianKeys(dir, create) reads guardian0, 1, ... and throws on a gap (never fills it with a new key). Setup creates keys up to 4 only while the protocol is not registered, so resuming an old market adds no key the chain lacks; a new deploy-dryrun step resumes the dry-run market with guardian3 moved aside and fails if setup makes it again (with setup always creating: "DRY RUN FAILED: resuming a registered market made keys/guardian3.json", r3-dryrun-J7.log). renew-guardians tops up to 4 and logs each new key (unfunded); a 2-of-3 renewal would be refused once the rule lands. keysIn reads the guardians only when a command uses them, so caps and set-recovery never need them; heartbeat says a key is not in the set rather than dead. One THRESHOLD (2) for the register and renewal payloads and MIN_GUARDIANS. vault-check.mjs compares the chain's set with the first files, and liveness with the chain's count. An extra file (a renewal not executed yet) and a set without the margin are info lines, not failures, since such a set keeps working. GUARDIAN_MARGIN and guardianMarginProblem live in cloud/lib/solana.mjs, used by these and by the guardian bot's existing warning (cloud/guardian/main.mjs), and vault-check uses THRESHOLD. The e2e scripts fund guardian3 too. Three tests in cloud/test/deploy.test.mjs (client tests 176 -> 179). JS mutants (js-mutants-r3.log), each red then restored: always create, never create, no gap check, margin 1, `>`, GUARDIAN_MARGIN = 1. Setup always creating survives these tests and is killed by the dry run step above. cloud/, scripts/deploy-dryrun.sh and docs/09's client-test row are byte-identical to ec0cacc (git diff empty), so its runs hold for them: the E2E check-programs run with the dry run (r4), and the JS mutants. tools/claims.py: OK (client 179). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AUD-2, decided by the owner on 16 Oct (option (e) of 16 §4.4). A set stays live while its honest live guardians reach the threshold: they eject every rogue in turn, and dead guardians cannot veto the owner's new set. Finding 96 asked for one key beyond the threshold, so a 2-of-3 or 3-of-4 set survived one fault but not a lost key plus a rogue: the honest guardian left is no quorum, nobody can be ejected, and the rogue keeps Floor stopped (or the Vault frozen) and vetoes every new set. Fix, in the core so both programs inherit it through GuardianSet::parse (registration, and every SetGuardians when proposed and when executed): GUARDIAN_MARGIN = 2 guardian_set_ok := count <= 8 && t >= 2 && count >= t + 2 (saturating_add: t is a payload byte). A stored set is not checked again, so existing 2-of-3 sets keep working, and ejection is unchanged: a shrunk set still ejects. A 2- or 3-key set is BadThreshold, not BadCount; the count rule is only the bitmap size, and the error order of finding 103 is kept. This changes both programs, by intent: Floor 508e5a99 -> 8ca8a608, Vault b07655d9 -> 7086148b, Harbor 543be60b (unchanged). Darwin-arm64 builds. Compute (scratch LiteSVM test, tasks/aud2-guardian-margin-impl/cu.md): -3 CU on every path that parses a set (Floor propose 2035 -> 2032, execute 1682 -> 1679; Vault register 9592 -> 9589, SetGuardians 2979 -> 2976; Floor register varies with its PDA bump, residue also -3). Proof and tests: - Kani: accepted_sets_can_eject_anyone restated (ejection reachable exactly when count > t, so option (a) cannot slip in; accepted sets have >= 4 keys and a margin of at least 2, and 2 is accepted). New: an_honest_live_quorum_ejects_any_rogue (any slots, anyone else alive) and accepted_sets_survive_any_two_faults (any two faulty keys, lost or rogue, leave a live honest quorum, which ejects each rogue, and the rogues alone eject no honest guardian). That holds only short of a rogue quorum: two rogues in a 2-of-n set are one (out of scope, 04 T7). With the assumption rogue < t dropped, the harness fails at `!can_eject(rogue, ..)` (kani-no-rogue-assume.log), so the docs say "short of a rogue quorum", not "any two faults". 19/19 verified, 240 s (cargo kani -j 2, as CI); again after the restatement, 235 s summed (r2-prove.log). - Kani mutants (kani-mutants.sh): margin 1, margin 3, `>` for `>=`, t >= 1, count <= 9, option (a), and an eject one vote short (only the two-fault harness kills that one): each KILLED by a harness (kani-mutants-r2.log). The eject ignoring liveness is the eject harness's case, not these. - Program mutants (coverage-map/mutants.sh, rebuilt each time): margin 1: KILLED by a_guardian_set_without_two_spare_keys_is_refused (Floor and Vault), registration_rejects_weak_settings and the Floor differential. Margin 3 and `>`: KILLED by most of the suite (the fixtures are 2-of-4). t >= 1: first only the Floor differential; the 1-of-2 registration case was now refused by the margin too, so it is a 1-of-4 here and a Vault 1-of-4 case was added: then KILLED by both. count <= 9: SURVIVES the program tests, equivalent there (parse refuses more than 8 keys first), KILLED by Kani. Option (a): KILLED by the eject tests two commits back (the tests on 2-of-4 sets). - Tests: the Floor and Vault refusal tests above cover proposal and execution (a stored 2-of-4 proposal whose threshold turns 3 is refused when executed, as the devnet renewals proposed before this rule will be); registration_rejects_weak_settings gains 2-of-3 and 3-of-4. an_n_of_n_guardian_set_is_refused (Vault) is renamed and accepted a 3-of-4: it now refuses it and accepts 3-of-5. - Model: the Python rule, the Floor differential's model, and the guardian_set fuzz target state it again; the GP generator now also picks count - 2. With the Python rule put back to the old one, seed 11 (20,000 cases) reports 5 GP mismatches; restored, seeds 11-13 identical. Fuzz seeds two_of_three_refused, two_of_four, six_of_eight, and seven_of_eight renamed seven_of_eight_refused (byte 323, the threshold, is 7: now refused). With margin 1 in the core the target crashes within 60 s; restored, 222,615,288 runs in 601 s, no crash. - scripts/check.sh: ALL CHECKS PASSED, and E2E=1 check-programs.sh: PROGRAM CHECKS PASSED, both at the branch tip, whose code is this commit's (the next commit is docs only; tasks/aud2-guardian-margin- impl/r4/check.log, r4/check-programs-e2e.log). claims: kani 19, floor 50, vault 104, programs 154, client 179. Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest and quote sweep). The core and the program sources are the same as before the rebase but for one comment line in views.rs (line-neutral), so the Kani, fuzz, differential and mutant runs above were made on the pre-rebase commit; the hashes, check.sh, check-programs and the counts are from r4 runs (tasks/aud2-guardian-margin-impl/r4/). The finding 96 row said the restated harness proves "exactly"; it now says what it proved then and proves now, and gives the Vault test's new name. The finding 107 row waits for rows 105 and 106 to land (09 §4 is numbered 1..n); its text is ready. The rule's prose is the next commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The previous commit made both programs refuse a new guardian set without two keys beyond its threshold (finding 107). The docs that state the rule now say so: - README: guardians 4-8 keys; the G5 row and the "who the guardians are" paragraph give the reason (any two faults leave a quorum, unless the rogues are one themselves: two in a 2-of-n set, 04 T7). - 05 register_protocol: 2 <= threshold <= count - 2, also for every SetGuardians, proposed and executed. 05 abdicate: abdicating SetGuardians fixes the set, so ejections shrink it for good. - 06 state table, 00 D22, FAQ: the same rule. - 07 §2: 4-8 guardians, and don't abdicate SetGuardians. 07 §6: the programs enforce the margin; sets registered before keep their exposure until replaced; an ejection uses up a spare key until the owner's new set executes; a lost key counts as live until its interval passes. - 16: the AUD-2 row is decided; §4.4 gives the decision, why the margin is the number of faults a set survives (short of a rogue quorum), the options rejected and why (the honest-owner and stolen-owner cases look the same on-chain), what it costs, what it leaves (a rogue quorum; a stolen owner key with every guardian dead; a rogue's veto of the recovery key's rotation, checked: Floor's veto has no recovery exception, as a_vetoed_recovery_proposal_does_not_hold_the_owner_s_slot shows), and Lido-style lapsing vetoes as a future option only. §5: "independent guardians, 3-of-5 or more". - 10: the guardian set's minimum size is no longer an open decision. SECURITY.md and 08 describe what is on devnet, which still runs the old rule, and are left as they are. claims: OK. Docs only; no program file changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ci.yml's header and scripts/ci-changes.sh said the weekly run does every job, and 09 §8 gave `mutants` as running on any change to the core. The `mutants` job's `if` starts with `github.event_name == 'pull_request'`, so the Monday schedule (and a manual dispatch) never runs it: it mutates a pull request's diff and has none to mutate. Its trigger list in 09 §8 also left out scripts/mutants.sh and .cargo/mutants.toml, which ci-changes.sh (line 22) selects it for. Comments and docs only; nothing runs differently. Sibling search: `git grep` for "weekly run does", "every job ... weekly" and "mutants ... weekly" across md/yml/sh/mjs/py. The other hits are 09 §8's "at full depth every Monday" (its table now says which job is not weekly) and the dispatch input's "every job at weekly depth", which is the same set as the weekly run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…105) .cargo/mutants.toml excludes the core's 22 equivalent mutants, and the docs say so. Since 9 Oct it excluded 19. The murmur3 tail's pattern is pinned by line, `floor.rs:(235|238|241)`, because the body's `h ^= k` has the same column and function and is not equivalent. 1dc7861 (#52) moved those lines (now 327/330/333), and nothing checked the patterns. Failing closed, the three would have failed CI's `mutants` job on a pull request that changed those lines. Failing open, a pattern that starts matching a non-equivalent mutant hides it from every run. Root cause: nothing checked that a pattern still names its mutants. Fix: - each pattern ends with the number of mutants it excludes (`# 3`), and the murmur3 pattern names 327/330/333; - tools/mutant_exclusions.py fails unless each pattern matches exactly its number in `cargo mutants --list --no-config`, and cargo-mutants (the list with the config) excludes exactly the mutants the patterns match, so a pattern this parser can't see, or reads differently, fails too. No two patterns may match the same mutant, since the docs' count is the counts' sum, and the kept list must be part of the full one. It reads only the exclude_re array's body, needs no tomllib (Python 3.9), and names colour codes in the list as the problem; - scripts/mutants.sh makes both lists (--colors never: CI sets CARGO_TERM_COLOR=always) and runs it first, except when the caller passes --config or --no-config (as whole arguments). CI's `mutants` job runs that script; ci-changes.sh now also selects the job for the new tool; - scripts/check.sh runs its --self-test on planted configs; - tools/claims.py ties the docs' "22 equivalents" (README, 09 §6 and the summary row) to the sum of the counts, with the same parser. It finds that parser from its own directory, so `python3 -I` runs it too, and check.sh runs its self-test that way. 00's "18 equivalents" (26 Sep log entry) and run 9's "exactly the 22 equivalents below" (6 Oct) are kept as dated records. Docs: 09 §4 finding 105, §6 run 10 and the "In CI" note, §8's mutants row; README and 09's summary row 669 → 797 mutants (run 10); README and 12's post-submission range 96–105. Evidence (first on 1f2ad48 + the parent, logs in the autopilot's tasks/mutants-exclusions-r2/; re-run on 58697c6 + the parent, tasks/mutants-exclusions-r3/): - `cargo mutants --list`: 797 with --no-config, 778 with the old config (19 excluded), 775 with this one. The 22 excluded are §6's list, the murmur3 ones at 327/330/333. - scripts/mutants.sh --list: exit 0, "22 of 797 mutants excluded"; with CARGO_TERM_COLOR=always also exit 0. Old config: exit 1, every pattern "no '# N' count". Planted on the real lists: the old lines with a count (matches 0, says 3) and the body XOR added (matches 4, says 3) both exit 1; in both the set comparison alone passes. /usr/bin/python3 (3.9.6): self-test OK and the real check passes. - The self-test pins each problem by its message, not a count. Mutants of the tool (tool-mutants.log), each red: M1 no per-pattern check, M2 no set comparison, M3 no colour check, M4 no empty-array check, M5 a missing count accepted, M6 a missing count read as 0, M7 the parser reading past the array. M3 and M5 survived a first version that checked only the number of problems. After A's review (16 Oct): the set comparison was pinned only by its size, so `len(excluded) != len(matched)` passed the self-test. A case with as many excluded as matched but not the same ones now fails it (15 cases; red with that mutant, green without), and the message says how many mutants are in one set only. From the code review after the rebase: colour codes only in the kept list were reported as a kept list made with other options; the colour check now reads both lists (16 cases; dropping the kept half fails the new case). On 58697c6, with the overlap and kept-list checks: M8 no overlap check and M9 no kept-list check, red too, as M1–M7 still are. A duplicate of the can_eject pattern planted in the real config: scripts/mutants.sh --list exits 1, "the counts add up to 23, the patterns match 22 mutants" (planted-overlap.log). Without the sys.path line, `python3 -I tools/claims.py` fails with ModuleNotFoundError; with it, both claims runs pass. - claims.py (claims-mutants.log): one count 1 → 2 fails four statements (README 188; 09 33, 333, 335; kept-run9.log), not run 9's 331, which failed too before it was kept; a dropped count fails with the tool named; deleting any of the three rules fails a self-test case; deleting the KEPT entry fails 00:103. - The 22 excluded mutants run alone (--no-config, --re per pattern, 2 min): 22 missed, exactly the excluded list; --re also picked ema.rs:70 (a field deletion, no pattern matches it), caught. - Run 10 (f1f05ba, old config, -j4, 46 min; tasks/mutants-run10/): 778 tested, 712 caught, 5 timeouts, 58 unviable, 3 missed, exactly the murmur3 tail. The core's source is unchanged since (`git diff --stat f1f05ba HEAD -- crates/bulkhead-core/src` is empty; only tests/units.rs gained finding 102's tests). - No program or core source changed. Rebased on 16 Oct onto cede705 (findings 102 and 100, the attest sweep and AUD-2). AUD-2 changed the core's source (governance.rs), so the line above saying it is unchanged since run 10 held on 58697c6, not here. On cede705: scripts/mutants.sh's list check passes, "22 of 795 mutants excluded, each pattern matching its count"; the murmur3 lines are still 327/330/333. The 797 in README and 09 is run 10's figure: README now dates it ("in the last full run, 15 Oct") and finding 105's row says the core has changed since (finding 107). The setup line in scripts/mutants.sh now pins cargo-mutants 27.1.0, the version ci.yml installs: each count is the number of mutants that version lists, so another release can break the check (code review, 16 Oct; `git grep 'install.*cargo-mutants'` finds no other unpinned install). The check's report goes to stderr in scripts/mutants.sh, so `scripts/mutants.sh --list` prints only cargo-mutants' names: before, its first stdout line was the report; after, stdout is 773 lines (795 - 22) and stderr the one report line. ci.yml's `mutants` comment says the job checks the exclusions first. Row 105 and 09 §6 gave the pin's reason as the body's XORs. The finalizer's are at column 7 (`grep -n '\^=' floor.rs`, 337-342); the non-equivalent XOR at column 11 is `h ^= k`, in the body (320) and after the tail (335). The toml comment, the row and §6 now say that. 09 §8's `check` row names the two tool self-tests check.sh runs. README no longer calls run 10's 797 "injected bugs": cargo-mutants listed 797 but tested 778, 19 being excluded (09 §6, run 10's row). Sibling search: the other 14 patterns use `\d+` for the line, and each matches the count now written after it. The murmur3 one is the only line-pinned pattern. Not done, by choice: - Catching an edit that keeps a pattern's number but changes its mutants (an excluded line itself edited, or a murmur3 body XOR moved onto a pinned line). The toml and 09 §6 say so: such an edit needs the mutant judged again by hand. Failing on any diff that touches an excluded line would catch the first; it is a separate change. - Moving murmur3's tail into its own function, so its pattern needn't name lines. It shifts core lines, so it changes the program hashes. Until then a line move above it fails the check, with a message saying what to fix. - A weekly full mutants run in CI. It took 46 min here at -j4, likely past the job's 90 min on two CI cores: a minutes call for the owner. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two Vault checks that keep a ticket to its own protocol and vault had no
test. Deleting either one left every program test green
(tasks/error-codes/mutants.log, base cede705):
- brakes.rs:79, extend_ticket: tk.protocol != protocol. Without it, a live
guardian of any other protocol extends this protocol's tickets. Anyone
can register a protocol and name its guardians, so that guardian could
hold back every queued payout up to the total extension limit. The
sweep did not catch it: it has no second protocol's guardian to swap in.
- custody.rs:501, settle_ticket: tk.vault != vault. A USDC ticket settled
through the SOL vault (its tokens, its mint) got past the Vault's checks.
New test another_protocols_guardian_cannot_extend_a_ticket: a second
Market's live guardian extends this market's ticket -> WRONG_VAULT; the
same instruction from this market's guardian succeeds. The settle test
now settles the USDC ticket through the SOL vault and expects
WRONG_VAULT.
Evidence (tasks/error-codes/mutants.log, STATE=c1; full program suite
per mutant):
W02 brakes.rs:79 deleted -> KILLED by
another_protocols_guardian_cannot_extend_a_ticket ("expected
failure, got success": the stranger's extension went through)
W06 custody.rs:501 deleted -> KILLED by
a_ticket_settles_after_its_eta_to_its_stored_recipient_only (8049
InsufficientBalance from the empty SOL vault, not 8010)
Both were SURVIVED at base. No program source change: hashes
8ca8a608/543be60b/7086148b, as at cede705.
Sibling search: the other 12 WrongVault sites were deleted one at a time
in the same run. Eight are KILLED by the sweep (brakes.rs:39, lib.rs:320,
custody.rs:160, genesis.rs:491, views.rs:27, lifeboat.rs:151,
governance.rs:169/259). lib.rs:324 (vault PDA address), genesis.rs:523,
lifeboat.rs:71 and :247 survive; later commits on this branch deal with
them.
Counts: Vault 104 -> 105, programs 154 -> 155 (claims.py OK).
The test also reads the ticket's ETA: unchanged after the
refused extension, 3,600 s later after its own guardian's (code-review
fix: success alone did not show the extension happened).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two seal checks had no test; deleting either left every program test
green (tasks/error-codes/mutants.log, base cede705):
- genesis.rs:486, execute_seal: the pairs must number 3 * vault_count.
Without it, the seal checks only the vaults it is given. A vault left
out skips its claims-vs-balance check, so an uncovered import seals.
docs/05's seal row says every vault is passed exactly once.
the_seal_checks_each_vault_once covered a vault named twice in place of
another, not one left out.
- genesis.rs:431, propose_seal: refused once sealed (AlreadySealed).
Without it, a sealed protocol's admin re-proposes, which rewrites
seal_executable_s/expires_s and clears seal_vetoed on a sealed protocol.
the_seal_checks_each_vault_once now also leaves the uncovered USDC vault
out (expects VAULT_COUNT_MISMATCH) and, after the seal, proposes it again
(expects ALREADY_SEALED). No new test, so no count change.
Evidence (tasks/error-codes/mutants.log, STATE=c2, full program suite):
E05 genesis.rs:486 deleted -> KILLED by the_seal_checks_each_vault_once
("expected failure, got success": the seal went through without
the USDC vault and its 1,000 USDC uncovered import)
E04 genesis.rs:431 deleted -> KILLED by the_seal_checks_each_vault_once
("expected failure, got success")
Both were SURVIVED at base. Vault hash 7086148b unchanged (tests only).
Sibling search: the seal's other checks. :491 vault_tokens (KILLED by the
sweep), :519 lifeboat owner and :523 lifeboat mint/owner (SURVIVED; the
lifeboat is a PDA checked by expect_pda just above, which only this
program can create, as its own token account: see this branch's note in
docs/09).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three bound checks had no test; deleting any one left every program test
green (tasks/error-codes/mutants.log, base cede705):
- genesis.rs:171, create_vault: config_is_valid (bps <= 100%, windows
> 0). Without it, a vault is created with a 100.01%-an-hour burst.
- governance.rs:40, parse of a SetLimits change (at proposal and again
at execution, :254 and :478): the same rule for a proposed change.
- governance.rs:47, parse of an UpsertAsset change: valid_asset (haircut
<= 100%, liability >= 100%, the oracle bounds of finding 88). Only the
genesis upsert's copy (genesis.rs:296) was tested, by
oracle_settings_of_a_priced_asset_are_bounded.
New test invalid_limits_and_assets_are_refused_wherever_they_enter:
before the seal, create_vault with short_bps 10,001 -> INVALID_LIMITS;
after it, a SetLimits proposal with long_window_s 0 -> INVALID_LIMITS,
and an UpsertAsset proposal with a 10,001 bps haircut -> INVALID_ASSET.
Evidence (tasks/error-codes/mutants.log, STATE=c3, full program suite):
E06 genesis.rs:171 deleted -> KILLED ("expected failure, got success")
E07 governance.rs:40 deleted -> KILLED ("expected failure, got success")
E08 governance.rs:47 deleted -> KILLED ("expected failure, got success")
each by the new test only. All three were SURVIVED at base. Vault hash
7086148b unchanged (tests only).
Sibling search: every config_is_valid / valid_asset call in programs/
vault/src: genesis.rs:170 (create_vault), :295 (upsert_asset, KILLED by
the oracle test), governance.rs:39 and :46. All now killed.
Counts: Vault 105 -> 106, programs 155 -> 156 (claims.py OK).
The execution copy (:478) is not reached by any transaction: it
parses the bytes the proposal already checked, and the bounds read
nothing else. The test comment says so (code-review fix: it had
claimed execution was covered). oracle_settings_of_a_priced_asset_are_
bounded now asserts INVALID_ASSET for each of its four cases, not any
error (all four return it; E09 stays KILLED).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
genesis.rs:410, import_backing: a backing ledger holds MAX_SLOTS (8)
mints, and an import into a ninth is refused with NoSlot. Deleting the
check left every program test green (tasks/error-codes/mutants.log, base
cede705). From the code, without it the import goes through: the vault's
imported total grows (:414), and the seal must cover that, but the
owner's ledger is saved unchanged, so the owner has no backed claim for
it.
New test an_import_into_a_ninth_mint_is_refused: nine vaults before the
seal; one owner gets an import into eight (ledger_of shows 8 positions),
then the ninth -> NO_SLOT.
Evidence (tasks/error-codes/mutants.log, STATE=c4, full program suite):
E14 genesis.rs:410 deleted -> KILLED by an_import_into_a_ninth_mint_is_
refused ("expected failure, got success"); SURVIVED at base.
Vault hash 7086148b unchanged (tests only).
Sibling search: the only other ledger credit is deposit's
(custody.rs:128). There a full ledger is by design: the deposit is taken
unbacked (Ledger::credit's doc, ledger.rs:187-189), and credit's result is
the backed flag, not an error.
Counts: Vault 106 -> 107, programs 156 -> 157 (claims.py OK).
import_ix now calls import_into_ix, which takes the vault, so the
new test shares the one import layout (code-review fix).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Six reachable checks had no test that failed when the check was deleted
(tasks/error-codes/mutants.log, base cede705). Each now has an assertion
on its code, and each was seen to fail with the check deleted:
E02 genesis.rs:185 create_vault of an existing vault after the seal:
ALREADY_INITIALIZED (with the check gone: 8043 GenesisOnly, so the
"says so first" order in the comment above it was untested)
E12 governance.rs:426 veto of an executed change: CHANGE_EXECUTED
(gone: the veto succeeds and marks an executed change vetoed)
E13 governance.rs:517 cancel of an executed change: CHANGE_EXECUTED
(gone: the cancel succeeds)
E16 lifeboat.rs:256 execute_exit with no exit proposed:
NOTHING_TO_EXIT (gone: 8033 ChangeNotExecutable)
E17 custody.rs:84 a zero deposit: ZERO_AMOUNT (gone: it succeeds)
E18 custody.rs:272 a zero release: ZERO_AMOUNT (gone: it succeeds; a
zero release has no unbacked part, and the limiter, whose own zero
check custody.rs:242 maps, is called only when unbacked > 0, :361)
Where: E02 in after_the_seal_a_new_vault_starts_with_its_unbacked_lane_
closed, E12/E13 in an_unvetoed_loosening_executes_after_the_delay, E16 in
the_lifeboat_stays_shut_until_the_seal (after the move in, before an
exit is proposed), E17/E18 in the new a_zero_deposit_or_release_is_
refused.
Evidence (tasks/error-codes/mutants.log, STATE=c5, full program suite):
all six KILLED by the tests named above; all six SURVIVED at base. The
vault suite is green (108 passed). Vault hash 7086148b unchanged.
Counts: Vault 107 -> 108, programs 157 -> 158 (claims.py OK).
Helpers (code-review fixes): veto_change_ix beside cancel_change_ix;
create_vault_for_ix, create_vault_ix with a given mint and token
program, so E02's case names USDC's own accounts and creates no mint.
E02's comment now repeats the program's ("whatever the seal's state"):
create_vault checks the limits first (genesis.rs:170).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Four checks survived deletion (tasks/error-codes/mutants.log, base
cede705) because something later still refused the case, with its own
error, so no test could tell. Each now has a test that sees the Vault's
own code, and each was seen to fail with the check deleted:
E01 lib.rs:230, create_pda's guard: a second registration of an
existing protocol (register_protocol has no check of its own) ->
ALREADY_INITIALIZED. Gone: the system program refuses it, with no
custom code (custom_code 0).
W10 lifeboat.rs:71, move_in naming another mint or token program ->
WRONG_VAULT. Gone: the token program refuses (code 3, MintMismatch).
W12 lifeboat.rs:247, execute_exit naming another mint or token
program -> WRONG_VAULT. Gone: the same token-program error.
T01 token.rs:37, account_info on an account no token program owns: a
copy of a real USDC account's bytes under the system program. Gone:
the quote SUCCEEDS (it promises a payout), and the release fails
only in the runtime at the transfer ("instruction modified data of
an account it does not own"). The quote is asserted first so the
mutant is seen at the quote (STATE=c6b).
New tests: a_protocol_registers_once, a_lifeboat_move_names_the_vaults_
own_mint (also completes a move in and an exit, as a control: the
destination receives 1,000 USDC), a_payout_goes_only_to_an_account_a_
token_program_owns.
Evidence (tasks/error-codes/mutants.log, STATE=c6 and c6b, full program
suite): E01, W10, W12, T01 KILLED by the tests named above; all four
SURVIVED at base. Full program suite green: Floor 50, Vault 111. Vault
hash 7086148b unchanged (tests only).
Counts: Vault 108 -> 111, programs 158 -> 161 (claims.py OK).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
B's review of vault-error-tests-r3 (R1) found two Vault codes no test asserted by value that the earlier commits missed: ExtensionTooLong and LaneClosed. Deleting the 7-day total extension cap (brakes.rs:86, B24) left every program test green, so a live guardian, or a stolen guardian key, could hold a queued payout back without end, 24h at a time, against docs/05 row 11. The wide differential models the total (differential_wide.rs:406), but its default run never extends one ticket that far. Mapping LaneClosed to QueueFull (custody.rs:240, B23) also survived. The code review of the fix found a worse gap at the same check: no test sends a negative extension (the differential draws secs from 0, 1, 3600, 86400, 86401). With `secs <= 0` weakened to `secs == 0` (A3, brakes.rs:84) every program test passed, and one guardian key could pull a queued payout's ETA into the past, skipping the queue delay, and refund its own 7-day budget. - a_ticket_is_extended_later_by_at_most_a_day_a_call_and_seven_in_total: -1h, 0 and 24h + 1 s are each refused with EXTENSION_TOO_LONG, ETA unchanged; six 24h extensions and one of 23h pass; 2h more is refused, ETA unchanged; the last hour is granted; then 1 s more is refused. The split is unaligned so that a total that ignores this call's `secs` fails too. - a_closed_lane_refuses_an_unbacked_payout: a vault with short_bps 0, and one with long_bps 0, each refuse an unbacked admin payout with LANE_CLOSED; the same payout from a vault with both buckets open is taken (the control, so a capacity base of 0 cannot pass it). Evidence (tasks/vault-error-tests-r4/, B's runner copied: each mutant applied alone, programs rebuilt, whole program suite run): - without these tests (551e519; run-without.out, run-base2.out, run-base3.out): B23, B24, A1 (`tk.extended_s >= MAX`, this call's secs ignored), A2 (`>=` for `>`) and A3 SURVIVED. A4 (the 24h per call deleted), A5 and A6 (either half of limiter.rs:172's closed-lane test deleted) were KILLED by the differential only, not by value; - with them (run-final.out, at the branch tip): B23 KILLED by a_closed_lane_refuses_an_unbacked_payout; B24 A1 A2 A3 KILLED by the extension test only; A4 by it and the differential; A5 A6 by the closed-lane test and the differential. No program change: the runner's closing build gives 8ca8a608 / 543be60b / 7086148b, as at cede705. Counts: Vault 111 -> 113, programs 161 -> 163 (claims.py OK). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
B's review of vault-error-tests-r3 (R2): each half of lib.rs:229 (`!is_data_empty() || !owned_by(SYSTEM)`) survived deleted on its own (B20, B21), since an existing protocol account fails both, so a_protocol_registers_once pinned them only jointly. The test now also plants, at a fresh protocol address, an account that fails just one half: one with data owned by the system program, one with no data owned by another program (a fresh address). Each registration is refused with ALREADY_INITIALIZED. The comment says no transaction is known to make either account: data or a new owner at a PDA needs its signature, which only the Vault gives, and create_pda gives both in one instruction. Evidence (tasks/vault-error-tests-r4/): at 551e519 B20 and B21 SURVIVED (run-base2.out); at the branch tip both are KILLED by a_protocol_registers_once only (run-final.out). No count change (claims.py OK: vault_programs 113, programs 163). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/09 §3 gains a row for the deletion run this branch answers (tasks/error-codes/mutants.log): 38 refusals, across 14 of the 18 Vault codes no test asserted by value at cede705, each deleted on its own with the whole program suite run against it, on the AUD-2 build. The 18 (each VaultError variant with no `verr::` use in programs/tests at cede705): AlreadyInitialized NotRegistrar WrongVault WrongTokenProgram AlreadySealed VaultCountMismatch InvalidLimits InvalidAsset LaneClosed TicketNotPending ExtensionTooLong ChangeExecuted LedgerUnsound LedgerCorrupt NoSlot NothingToExit ZeroAmount Overflow. The run missed LaneClosed and ExtensionTooLong; B's review deleted one check of each (B23 custody.rs:240, B24 brakes.rs:86), and both survived until 1797ae5. That commit also pins a negative extension (A3, which survived every test before it), the 24h per call, and a lane closed by either bucket. At base: 13 KILLED, 25 SURVIVED. The commits before this one add tests that kill 18 of the 25 and both of B's, each asserting the Vault's own code (W02 W06, E04 E05, E06 E07 E08, E14, E02 E12 E13 E16 E17 E18, E01 W10 W12 T01; B23 B24). The row names the 7 that still survive and why: - six guards no transaction is known to reach: lib.rs:324 (vault PDA address; create_vault creates a vault only at its PDA), genesis.rs:519 and :523 (the lifeboat, checked by expect_pda first, created only by lifeboat_move, move_in requires the seal, lifeboat.rs:53, and the seal happens once, so no lifeboat holds data at it), brakes.rs:50 and custody.rs:541 (a pending-ticket count below zero), token.rs:235 (GetAccountDataSize's return data from an already-checked program); - custody.rs:399, stated as not shown unreachable: properties.rs:588-592 debits backed_amount and one more on random ledgers, and only crates/bulkhead-core/tests/units.rs:375 debits less, on a one-mint ledger with no liability. properties.rs:606 asserts backed(a) == min(a, backed(MAX)) for requests below 2^52 (:613), so any smaller amount is backed too, and debit_backed(b) refuses only when backed_amount(b) != b (ledger.rs:289). The known exception is a collateral value that overflows u128 (ledger.rs:262): the valid amounts need not form an interval there, so the consent or freeze cap (custody.rs:337, :347) can lower the amount to an invalid one and :399 refuses the release (fail closed). The search's own result is always valid. Not verified by a tool beyond that property test. (A full ledger cannot reach :399 through set's NoSlot: backed_amount's valid() refuses a new liability slot on a full ledger, ledger.rs:237.) Not deleted, as the row says: the two core-error mappings to ChangeExecuted and ZeroAmount, token.rs:233, and the codes Overflow and LedgerCorrupt (checked arithmetic, decoding). After this branch five codes are still asserted by no test (same grep at this commit): NotRegistrar (E03, genesis.rs:108, was KILLED at base by another test), TicketNotPending, LedgerUnsound, LedgerCorrupt, Overflow. Docs only: no count change (claims.py OK: vault_programs 113, programs 163). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
shuv-amp
force-pushed
the
auto/B/mutants-exclusions-d2
branch
from
October 17, 2026 22:51
ea09733 to
ee4e02c
Compare
shuv-amp
force-pushed
the
auto/A/vault-error-tests-d2
branch
from
October 17, 2026 22:51
86b777b to
9f552fa
Compare
shuv-amp
added a commit
that referenced
this pull request
Oct 17, 2026
Finding 106's row in docs/09 said its 33 mutants were re-run on 16 Oct at bf8ba41. That commit is not an ancestor of this branch or of the stack above it and is on no remote branch, so the citation would point at nothing once the stack lands. The owner's review of #99-#109 found it. bf8ba41 and 86b777b (the head of PR #100, vault-error-tests-d2, the D2 replay of vault-error-tests-r4) have the same tree, 36187b4 (`git rev-parse bf8ba41^{tree} 86b777b^{tree}` prints it twice). So the run was a run on 86b777b's tree, and the row now says "on tree 36187b4, the tree of 86b777b (PR #100's head)". Why not just "at 86b777b": it was committed on 17 Oct, after the run; and main squash-merges (1045cd9 has one parent), so after PR #100 lands 86b777b is reachable only as GitHub's refs/pull/100/head. The tree id stays checkable from main: squashed bottom first, main's commit for PR #100 has that tree. "PR" keeps "#100" from reading as finding 100, the row above. These points came from two code-review passes. Sibling search: every 7-12 hex token in tracked .md/.py/.sh/.mjs/.rs/ .toml files at autopilot/integration 3e9f58d that names a local commit (51 tokens) was checked with git merge-base --is-ancestor against integration. bf8ba41 was the only one that is not an ancestor. Docs only. tools/claims.py: claims OK (26 files). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts resolved: this branch's docs/09, and main's README with 163/163. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
shuv-amp
enabled auto-merge (squash)
October 18, 2026 13:38
shuv-amp
added a commit
that referenced
this pull request
Oct 18, 2026
Conflicts resolved: this branch's tools/mutant_exclusions.py and main's README. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
shuv-amp
added a commit
that referenced
this pull request
Oct 18, 2026
) * Run the tests on 2-of-4 guardian sets; test recovery from two faults Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set with fewer than threshold + 2 keys. Every test fixture and most tests' own sets were 2-of-3. This moves them to 2-of-4, which today's rule also accepts, so this commit changes no program: Floor, Vault and Harbor hash 508e5a99 / b07655d9 / 543be60b, as before. - Fixtures: Protected (Floor) and Market (Vault, through Harbor) now hold 4 guardians. Each test that built its own 3-key set gets a 4-key set with the same threshold and the same assertion. - Two tests changed meaning with a 4th key and were fixed, not loosened: a_quorum_ejects_a_rogue_guardian_who_vetoes_everything (Floor) and a_quorum_ejects_a_guardian_who_freezes_forever (Vault). With 4 keys, the last eject was refused only because the target's own vote is masked. Both now eject a second guardian (a set an ejection shrank to threshold + 1 still ejects) and then show the 2-of-2 refusal. - Account sweeps: an eject's signers may now be any two of the three others, so each of those two slots is free for the third, never for the target or the other signer again. Vault 5,365 and Floor 1,710 substitutions, refused (docs/09, dated 16 Oct). Listing all three others at each slot would have skipped the duplicate: with a mutant that counts a guardian listed twice as two (governance.rs:270), the sweeps listing all three pass, and these fail at "eject: account 2 replaced by key ..." (Vault and Floor). No other program test kills that mutant (tasks/aud2-guardian-margin-impl/dup-mutant-*.log). - floor_differential: the guardian pool is 6 keys, 4 registered; the proposed sets are 2 to 6 keys. New tests (pass on both rules; they show what the margin buys): - a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_set (Floor): g0's key lost, g1 rogue stops the program; the lost key does not help a quorum; the honest pair ejects g1 and restores; neither can veto the owner's refill, which executes after 72h. - two_rogues_in_turn_are_both_ejected_from_a_two_of_four_set (Floor): the second rogue vetoes the refill and is ejected from the 2-of-3. - a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_vault: the same in the Vault (freeze, eject, unfreeze, refill). Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest and quote sweep): the counts and the sweep figures here are from runs there (tasks/aud2-guardian-margin-impl/r4/). The E2E run, the mutants and the new-rule run below were made before, on e8640a7; this commit's programs and core are the same as there but for one comment line in views.rs (line-neutral, same hashes). Evidence: - E2E=1 scripts/check-programs.sh: PROGRAM CHECKS PASSED (Floor 49, Vault 98; S1 and E2E passed; the resumed setup found all 14 steps; vault check, Floor demo and deploy dry run passed), with the next commit's deploy rehearsal in the tree. - tools/claims.py: OK (floor 49, vault 104, programs 153). - Under the new rule (core from two commits on, programs rebuilt), only the two tests that state the old rule fail: an_n_of_n_guardian_set_is_ refused (accepts 3-of-4) and the Floor differential's model. - Mutants (tasks/coverage-map/mutants.sh), each red, then restored: AV1 Floor live_guardian ignores liveness: KILLED by a_lost_key_and_a_ rogue_do_not_lock_a_two_of_four_set (+2 others). AV2 the Vault's: KILLED by ..._two_of_four_vault (+2 others). AQ1 Floor restore quorum ignores liveness: KILLED by ..._two_of_four_set and the differential only. AQ2 the Vault's unfreeze quorum: KILLED by ..._two_of_four_vault and a_silent_guardian_does_not_count_toward_a_quorum. AE1 can_eject refuses at count == threshold + 1 (AUD-2 option (a)): KILLED by two_rogues_in_turn_..., and the two eject tests above. Hashes after: 508e5a99 / 543be60b / b07655d9. Split on 16 Oct from ec0cacc at the owner's review (D2): the deploy rehearsal's move to 4 guardians is now the next commit. programs/ and programs/tests/ here are byte-identical to ec0cacc, so its runs below hold for this commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Rehearse deploys with four guardians, and read only the keys there are Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set with fewer than threshold + 2 keys, so the deploy rehearsal must register 2-of-4 sets. Split from ec0cacc at the owner's review (D2); the text is that commit's, unchanged: Deploy rehearsal (cloud/deploy market.mjs, vault-check.mjs, floor-demo.mjs): 4 guardians (keys/guardian3), so the dry run keeps working once the rule lands; one more funding step (14, was 13). The devnet market has 3 guardian files. A first version of this change loaded guardian3 everywhere: vault-check.mjs failed there with ENOENT, and heartbeat silently created a key nobody holds as a guardian. Now market.mjs guardianKeys(dir, create) reads guardian0, 1, ... and throws on a gap (never fills it with a new key). Setup creates keys up to 4 only while the protocol is not registered, so resuming an old market adds no key the chain lacks; a new deploy-dryrun step resumes the dry-run market with guardian3 moved aside and fails if setup makes it again (with setup always creating: "DRY RUN FAILED: resuming a registered market made keys/guardian3.json", r3-dryrun-J7.log). renew-guardians tops up to 4 and logs each new key (unfunded); a 2-of-3 renewal would be refused once the rule lands. keysIn reads the guardians only when a command uses them, so caps and set-recovery never need them; heartbeat says a key is not in the set rather than dead. One THRESHOLD (2) for the register and renewal payloads and MIN_GUARDIANS. vault-check.mjs compares the chain's set with the first files, and liveness with the chain's count. An extra file (a renewal not executed yet) and a set without the margin are info lines, not failures, since such a set keeps working. GUARDIAN_MARGIN and guardianMarginProblem live in cloud/lib/solana.mjs, used by these and by the guardian bot's existing warning (cloud/guardian/main.mjs), and vault-check uses THRESHOLD. The e2e scripts fund guardian3 too. Three tests in cloud/test/deploy.test.mjs (client tests 176 -> 179). JS mutants (js-mutants-r3.log), each red then restored: always create, never create, no gap check, margin 1, `>`, GUARDIAN_MARGIN = 1. Setup always creating survives these tests and is killed by the dry run step above. cloud/, scripts/deploy-dryrun.sh and docs/09's client-test row are byte-identical to ec0cacc (git diff empty), so its runs hold for them: the E2E check-programs run with the dry run (r4), and the JS mutants. tools/claims.py: OK (client 179). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Refuse a guardian set without two keys beyond its threshold (107) AUD-2, decided by the owner on 16 Oct (option (e) of 16 §4.4). A set stays live while its honest live guardians reach the threshold: they eject every rogue in turn, and dead guardians cannot veto the owner's new set. Finding 96 asked for one key beyond the threshold, so a 2-of-3 or 3-of-4 set survived one fault but not a lost key plus a rogue: the honest guardian left is no quorum, nobody can be ejected, and the rogue keeps Floor stopped (or the Vault frozen) and vetoes every new set. Fix, in the core so both programs inherit it through GuardianSet::parse (registration, and every SetGuardians when proposed and when executed): GUARDIAN_MARGIN = 2 guardian_set_ok := count <= 8 && t >= 2 && count >= t + 2 (saturating_add: t is a payload byte). A stored set is not checked again, so existing 2-of-3 sets keep working, and ejection is unchanged: a shrunk set still ejects. A 2- or 3-key set is BadThreshold, not BadCount; the count rule is only the bitmap size, and the error order of finding 103 is kept. This changes both programs, by intent: Floor 508e5a99 -> 8ca8a608, Vault b07655d9 -> 7086148b, Harbor 543be60b (unchanged). Darwin-arm64 builds. Compute (scratch LiteSVM test, tasks/aud2-guardian-margin-impl/cu.md): -3 CU on every path that parses a set (Floor propose 2035 -> 2032, execute 1682 -> 1679; Vault register 9592 -> 9589, SetGuardians 2979 -> 2976; Floor register varies with its PDA bump, residue also -3). Proof and tests: - Kani: accepted_sets_can_eject_anyone restated (ejection reachable exactly when count > t, so option (a) cannot slip in; accepted sets have >= 4 keys and a margin of at least 2, and 2 is accepted). New: an_honest_live_quorum_ejects_any_rogue (any slots, anyone else alive) and accepted_sets_survive_any_two_faults (any two faulty keys, lost or rogue, leave a live honest quorum, which ejects each rogue, and the rogues alone eject no honest guardian). That holds only short of a rogue quorum: two rogues in a 2-of-n set are one (out of scope, 04 T7). With the assumption rogue < t dropped, the harness fails at `!can_eject(rogue, ..)` (kani-no-rogue-assume.log), so the docs say "short of a rogue quorum", not "any two faults". 19/19 verified, 240 s (cargo kani -j 2, as CI); again after the restatement, 235 s summed (r2-prove.log). - Kani mutants (kani-mutants.sh): margin 1, margin 3, `>` for `>=`, t >= 1, count <= 9, option (a), and an eject one vote short (only the two-fault harness kills that one): each KILLED by a harness (kani-mutants-r2.log). The eject ignoring liveness is the eject harness's case, not these. - Program mutants (coverage-map/mutants.sh, rebuilt each time): margin 1: KILLED by a_guardian_set_without_two_spare_keys_is_refused (Floor and Vault), registration_rejects_weak_settings and the Floor differential. Margin 3 and `>`: KILLED by most of the suite (the fixtures are 2-of-4). t >= 1: first only the Floor differential; the 1-of-2 registration case was now refused by the margin too, so it is a 1-of-4 here and a Vault 1-of-4 case was added: then KILLED by both. count <= 9: SURVIVES the program tests, equivalent there (parse refuses more than 8 keys first), KILLED by Kani. Option (a): KILLED by the eject tests two commits back (the tests on 2-of-4 sets). - Tests: the Floor and Vault refusal tests above cover proposal and execution (a stored 2-of-4 proposal whose threshold turns 3 is refused when executed, as the devnet renewals proposed before this rule will be); registration_rejects_weak_settings gains 2-of-3 and 3-of-4. an_n_of_n_guardian_set_is_refused (Vault) is renamed and accepted a 3-of-4: it now refuses it and accepts 3-of-5. - Model: the Python rule, the Floor differential's model, and the guardian_set fuzz target state it again; the GP generator now also picks count - 2. With the Python rule put back to the old one, seed 11 (20,000 cases) reports 5 GP mismatches; restored, seeds 11-13 identical. Fuzz seeds two_of_three_refused, two_of_four, six_of_eight, and seven_of_eight renamed seven_of_eight_refused (byte 323, the threshold, is 7: now refused). With margin 1 in the core the target crashes within 60 s; restored, 222,615,288 runs in 601 s, no crash. - scripts/check.sh: ALL CHECKS PASSED, and E2E=1 check-programs.sh: PROGRAM CHECKS PASSED, both at the branch tip, whose code is this commit's (the next commit is docs only; tasks/aud2-guardian-margin- impl/r4/check.log, r4/check-programs-e2e.log). claims: kani 19, floor 50, vault 104, programs 154, client 179. Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest and quote sweep). The core and the program sources are the same as before the rebase but for one comment line in views.rs (line-neutral), so the Kani, fuzz, differential and mutant runs above were made on the pre-rebase commit; the hashes, check.sh, check-programs and the counts are from r4 runs (tasks/aud2-guardian-margin-impl/r4/). The finding 96 row said the restated harness proves "exactly"; it now says what it proved then and proves now, and gives the Vault test's new name. The finding 107 row waits for rows 105 and 106 to land (09 §4 is numbered 1..n); its text is ready. The rule's prose is the next commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * State the guardian margin in the docs, with its costs (AUD-2) The previous commit made both programs refuse a new guardian set without two keys beyond its threshold (finding 107). The docs that state the rule now say so: - README: guardians 4-8 keys; the G5 row and the "who the guardians are" paragraph give the reason (any two faults leave a quorum, unless the rogues are one themselves: two in a 2-of-n set, 04 T7). - 05 register_protocol: 2 <= threshold <= count - 2, also for every SetGuardians, proposed and executed. 05 abdicate: abdicating SetGuardians fixes the set, so ejections shrink it for good. - 06 state table, 00 D22, FAQ: the same rule. - 07 §2: 4-8 guardians, and don't abdicate SetGuardians. 07 §6: the programs enforce the margin; sets registered before keep their exposure until replaced; an ejection uses up a spare key until the owner's new set executes; a lost key counts as live until its interval passes. - 16: the AUD-2 row is decided; §4.4 gives the decision, why the margin is the number of faults a set survives (short of a rogue quorum), the options rejected and why (the honest-owner and stolen-owner cases look the same on-chain), what it costs, what it leaves (a rogue quorum; a stolen owner key with every guardian dead; a rogue's veto of the recovery key's rotation, checked: Floor's veto has no recovery exception, as a_vetoed_recovery_proposal_does_not_hold_the_owner_s_slot shows), and Lido-style lapsing vetoes as a future option only. §5: "independent guardians, 3-of-5 or more". - 10: the guardian set's minimum size is no longer an open decision. SECURITY.md and 08 describe what is on devnet, which still runs the old rule, and are left as they are. claims: OK. Docs only; no program file changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Say the weekly CI run skips mutants, as its condition does ci.yml's header and scripts/ci-changes.sh said the weekly run does every job, and 09 §8 gave `mutants` as running on any change to the core. The `mutants` job's `if` starts with `github.event_name == 'pull_request'`, so the Monday schedule (and a manual dispatch) never runs it: it mutates a pull request's diff and has none to mutate. Its trigger list in 09 §8 also left out scripts/mutants.sh and .cargo/mutants.toml, which ci-changes.sh (line 22) selects it for. Comments and docs only; nothing runs differently. Sibling search: `git grep` for "weekly run does", "every job ... weekly" and "mutants ... weekly" across md/yml/sh/mjs/py. The other hits are 09 §8's "at full depth every Monday" (its table now says which job is not weekly) and the dispatch input's "every job at weekly depth", which is the same set as the weekly run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Check that the mutant exclusions still match; repin murmur3 (finding 105) .cargo/mutants.toml excludes the core's 22 equivalent mutants, and the docs say so. Since 9 Oct it excluded 19. The murmur3 tail's pattern is pinned by line, `floor.rs:(235|238|241)`, because the body's `h ^= k` has the same column and function and is not equivalent. 1dc7861 (#52) moved those lines (now 327/330/333), and nothing checked the patterns. Failing closed, the three would have failed CI's `mutants` job on a pull request that changed those lines. Failing open, a pattern that starts matching a non-equivalent mutant hides it from every run. Root cause: nothing checked that a pattern still names its mutants. Fix: - each pattern ends with the number of mutants it excludes (`# 3`), and the murmur3 pattern names 327/330/333; - tools/mutant_exclusions.py fails unless each pattern matches exactly its number in `cargo mutants --list --no-config`, and cargo-mutants (the list with the config) excludes exactly the mutants the patterns match, so a pattern this parser can't see, or reads differently, fails too. No two patterns may match the same mutant, since the docs' count is the counts' sum, and the kept list must be part of the full one. It reads only the exclude_re array's body, needs no tomllib (Python 3.9), and names colour codes in the list as the problem; - scripts/mutants.sh makes both lists (--colors never: CI sets CARGO_TERM_COLOR=always) and runs it first, except when the caller passes --config or --no-config (as whole arguments). CI's `mutants` job runs that script; ci-changes.sh now also selects the job for the new tool; - scripts/check.sh runs its --self-test on planted configs; - tools/claims.py ties the docs' "22 equivalents" (README, 09 §6 and the summary row) to the sum of the counts, with the same parser. It finds that parser from its own directory, so `python3 -I` runs it too, and check.sh runs its self-test that way. 00's "18 equivalents" (26 Sep log entry) and run 9's "exactly the 22 equivalents below" (6 Oct) are kept as dated records. Docs: 09 §4 finding 105, §6 run 10 and the "In CI" note, §8's mutants row; README and 09's summary row 669 → 797 mutants (run 10); README and 12's post-submission range 96–105. Evidence (first on 1f2ad48 + the parent, logs in the autopilot's tasks/mutants-exclusions-r2/; re-run on 58697c6 + the parent, tasks/mutants-exclusions-r3/): - `cargo mutants --list`: 797 with --no-config, 778 with the old config (19 excluded), 775 with this one. The 22 excluded are §6's list, the murmur3 ones at 327/330/333. - scripts/mutants.sh --list: exit 0, "22 of 797 mutants excluded"; with CARGO_TERM_COLOR=always also exit 0. Old config: exit 1, every pattern "no '# N' count". Planted on the real lists: the old lines with a count (matches 0, says 3) and the body XOR added (matches 4, says 3) both exit 1; in both the set comparison alone passes. /usr/bin/python3 (3.9.6): self-test OK and the real check passes. - The self-test pins each problem by its message, not a count. Mutants of the tool (tool-mutants.log), each red: M1 no per-pattern check, M2 no set comparison, M3 no colour check, M4 no empty-array check, M5 a missing count accepted, M6 a missing count read as 0, M7 the parser reading past the array. M3 and M5 survived a first version that checked only the number of problems. After A's review (16 Oct): the set comparison was pinned only by its size, so `len(excluded) != len(matched)` passed the self-test. A case with as many excluded as matched but not the same ones now fails it (15 cases; red with that mutant, green without), and the message says how many mutants are in one set only. From the code review after the rebase: colour codes only in the kept list were reported as a kept list made with other options; the colour check now reads both lists (16 cases; dropping the kept half fails the new case). On 58697c6, with the overlap and kept-list checks: M8 no overlap check and M9 no kept-list check, red too, as M1–M7 still are. A duplicate of the can_eject pattern planted in the real config: scripts/mutants.sh --list exits 1, "the counts add up to 23, the patterns match 22 mutants" (planted-overlap.log). Without the sys.path line, `python3 -I tools/claims.py` fails with ModuleNotFoundError; with it, both claims runs pass. - claims.py (claims-mutants.log): one count 1 → 2 fails four statements (README 188; 09 33, 333, 335; kept-run9.log), not run 9's 331, which failed too before it was kept; a dropped count fails with the tool named; deleting any of the three rules fails a self-test case; deleting the KEPT entry fails 00:103. - The 22 excluded mutants run alone (--no-config, --re per pattern, 2 min): 22 missed, exactly the excluded list; --re also picked ema.rs:70 (a field deletion, no pattern matches it), caught. - Run 10 (f1f05ba, old config, -j4, 46 min; tasks/mutants-run10/): 778 tested, 712 caught, 5 timeouts, 58 unviable, 3 missed, exactly the murmur3 tail. The core's source is unchanged since (`git diff --stat f1f05ba HEAD -- crates/bulkhead-core/src` is empty; only tests/units.rs gained finding 102's tests). - No program or core source changed. Rebased on 16 Oct onto cede705 (findings 102 and 100, the attest sweep and AUD-2). AUD-2 changed the core's source (governance.rs), so the line above saying it is unchanged since run 10 held on 58697c6, not here. On cede705: scripts/mutants.sh's list check passes, "22 of 795 mutants excluded, each pattern matching its count"; the murmur3 lines are still 327/330/333. The 797 in README and 09 is run 10's figure: README now dates it ("in the last full run, 15 Oct") and finding 105's row says the core has changed since (finding 107). The setup line in scripts/mutants.sh now pins cargo-mutants 27.1.0, the version ci.yml installs: each count is the number of mutants that version lists, so another release can break the check (code review, 16 Oct; `git grep 'install.*cargo-mutants'` finds no other unpinned install). The check's report goes to stderr in scripts/mutants.sh, so `scripts/mutants.sh --list` prints only cargo-mutants' names: before, its first stdout line was the report; after, stdout is 773 lines (795 - 22) and stderr the one report line. ci.yml's `mutants` comment says the job checks the exclusions first. Row 105 and 09 §6 gave the pin's reason as the body's XORs. The finalizer's are at column 7 (`grep -n '\^=' floor.rs`, 337-342); the non-equivalent XOR at column 11 is `h ^= k`, in the body (320) and after the tail (335). The toml comment, the row and §6 now say that. 09 §8's `check` row names the two tool self-tests check.sh runs. README no longer calls run 10's 797 "injected bugs": cargo-mutants listed 797 but tested 778, 19 being excluded (09 §6, run 10's row). Sibling search: the other 14 patterns use `\d+` for the line, and each matches the count now written after it. The murmur3 one is the only line-pinned pattern. Not done, by choice: - Catching an edit that keeps a pattern's number but changes its mutants (an excluded line itself edited, or a murmur3 body XOR moved onto a pinned line). The toml and 09 §6 say so: such an edit needs the mutant judged again by hand. Failing on any diff that touches an excluded line would catch the first; it is a separate change. - Moving murmur3's tail into its own function, so its pattern needn't name lines. It shifts core lines, so it changes the program hashes. Until then a line move above it fails the check, with a message saying what to fix. - A weekly full mutants run in CI. It took 46 min here at -j4, likely past the job's 90 min on two CI cores: a minutes call for the owner. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test that a ticket stays in its own protocol and vault Two Vault checks that keep a ticket to its own protocol and vault had no test. Deleting either one left every program test green (tasks/error-codes/mutants.log, base cede705): - brakes.rs:79, extend_ticket: tk.protocol != protocol. Without it, a live guardian of any other protocol extends this protocol's tickets. Anyone can register a protocol and name its guardians, so that guardian could hold back every queued payout up to the total extension limit. The sweep did not catch it: it has no second protocol's guardian to swap in. - custody.rs:501, settle_ticket: tk.vault != vault. A USDC ticket settled through the SOL vault (its tokens, its mint) got past the Vault's checks. New test another_protocols_guardian_cannot_extend_a_ticket: a second Market's live guardian extends this market's ticket -> WRONG_VAULT; the same instruction from this market's guardian succeeds. The settle test now settles the USDC ticket through the SOL vault and expects WRONG_VAULT. Evidence (tasks/error-codes/mutants.log, STATE=c1; full program suite per mutant): W02 brakes.rs:79 deleted -> KILLED by another_protocols_guardian_cannot_extend_a_ticket ("expected failure, got success": the stranger's extension went through) W06 custody.rs:501 deleted -> KILLED by a_ticket_settles_after_its_eta_to_its_stored_recipient_only (8049 InsufficientBalance from the empty SOL vault, not 8010) Both were SURVIVED at base. No program source change: hashes 8ca8a608/543be60b/7086148b, as at cede705. Sibling search: the other 12 WrongVault sites were deleted one at a time in the same run. Eight are KILLED by the sweep (brakes.rs:39, lib.rs:320, custody.rs:160, genesis.rs:491, views.rs:27, lifeboat.rs:151, governance.rs:169/259). lib.rs:324 (vault PDA address), genesis.rs:523, lifeboat.rs:71 and :247 survive; later commits on this branch deal with them. Counts: Vault 104 -> 105, programs 154 -> 155 (claims.py OK). The test also reads the ticket's ETA: unchanged after the refused extension, 3,600 s later after its own guardian's (code-review fix: success alone did not show the extension happened). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test that the seal names every vault and happens once Two seal checks had no test; deleting either left every program test green (tasks/error-codes/mutants.log, base cede705): - genesis.rs:486, execute_seal: the pairs must number 3 * vault_count. Without it, the seal checks only the vaults it is given. A vault left out skips its claims-vs-balance check, so an uncovered import seals. docs/05's seal row says every vault is passed exactly once. the_seal_checks_each_vault_once covered a vault named twice in place of another, not one left out. - genesis.rs:431, propose_seal: refused once sealed (AlreadySealed). Without it, a sealed protocol's admin re-proposes, which rewrites seal_executable_s/expires_s and clears seal_vetoed on a sealed protocol. the_seal_checks_each_vault_once now also leaves the uncovered USDC vault out (expects VAULT_COUNT_MISMATCH) and, after the seal, proposes it again (expects ALREADY_SEALED). No new test, so no count change. Evidence (tasks/error-codes/mutants.log, STATE=c2, full program suite): E05 genesis.rs:486 deleted -> KILLED by the_seal_checks_each_vault_once ("expected failure, got success": the seal went through without the USDC vault and its 1,000 USDC uncovered import) E04 genesis.rs:431 deleted -> KILLED by the_seal_checks_each_vault_once ("expected failure, got success") Both were SURVIVED at base. Vault hash 7086148b unchanged (tests only). Sibling search: the seal's other checks. :491 vault_tokens (KILLED by the sweep), :519 lifeboat owner and :523 lifeboat mint/owner (SURVIVED; the lifeboat is a PDA checked by expect_pda just above, which only this program can create, as its own token account: see this branch's note in docs/09). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test that invalid limits and asset settings are refused at every entry Three bound checks had no test; deleting any one left every program test green (tasks/error-codes/mutants.log, base cede705): - genesis.rs:171, create_vault: config_is_valid (bps <= 100%, windows > 0). Without it, a vault is created with a 100.01%-an-hour burst. - governance.rs:40, parse of a SetLimits change (at proposal and again at execution, :254 and :478): the same rule for a proposed change. - governance.rs:47, parse of an UpsertAsset change: valid_asset (haircut <= 100%, liability >= 100%, the oracle bounds of finding 88). Only the genesis upsert's copy (genesis.rs:296) was tested, by oracle_settings_of_a_priced_asset_are_bounded. New test invalid_limits_and_assets_are_refused_wherever_they_enter: before the seal, create_vault with short_bps 10,001 -> INVALID_LIMITS; after it, a SetLimits proposal with long_window_s 0 -> INVALID_LIMITS, and an UpsertAsset proposal with a 10,001 bps haircut -> INVALID_ASSET. Evidence (tasks/error-codes/mutants.log, STATE=c3, full program suite): E06 genesis.rs:171 deleted -> KILLED ("expected failure, got success") E07 governance.rs:40 deleted -> KILLED ("expected failure, got success") E08 governance.rs:47 deleted -> KILLED ("expected failure, got success") each by the new test only. All three were SURVIVED at base. Vault hash 7086148b unchanged (tests only). Sibling search: every config_is_valid / valid_asset call in programs/ vault/src: genesis.rs:170 (create_vault), :295 (upsert_asset, KILLED by the oracle test), governance.rs:39 and :46. All now killed. Counts: Vault 105 -> 106, programs 155 -> 156 (claims.py OK). The execution copy (:478) is not reached by any transaction: it parses the bytes the proposal already checked, and the bounds read nothing else. The test comment says so (code-review fix: it had claimed execution was covered). oracle_settings_of_a_priced_asset_are_ bounded now asserts INVALID_ASSET for each of its four cases, not any error (all four return it; E09 stays KILLED). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test that an import into a ninth mint is refused genesis.rs:410, import_backing: a backing ledger holds MAX_SLOTS (8) mints, and an import into a ninth is refused with NoSlot. Deleting the check left every program test green (tasks/error-codes/mutants.log, base cede705). From the code, without it the import goes through: the vault's imported total grows (:414), and the seal must cover that, but the owner's ledger is saved unchanged, so the owner has no backed claim for it. New test an_import_into_a_ninth_mint_is_refused: nine vaults before the seal; one owner gets an import into eight (ledger_of shows 8 positions), then the ninth -> NO_SLOT. Evidence (tasks/error-codes/mutants.log, STATE=c4, full program suite): E14 genesis.rs:410 deleted -> KILLED by an_import_into_a_ninth_mint_is_ refused ("expected failure, got success"); SURVIVED at base. Vault hash 7086148b unchanged (tests only). Sibling search: the only other ledger credit is deposit's (custody.rs:128). There a full ledger is by design: the deposit is taken unbacked (Ledger::credit's doc, ledger.rs:187-189), and credit's result is the backed flag, not an error. Counts: Vault 106 -> 107, programs 156 -> 157 (claims.py OK). import_ix now calls import_into_ix, which takes the vault, so the new test shares the one import layout (code-review fix). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pin six Vault refusals no test reached Six reachable checks had no test that failed when the check was deleted (tasks/error-codes/mutants.log, base cede705). Each now has an assertion on its code, and each was seen to fail with the check deleted: E02 genesis.rs:185 create_vault of an existing vault after the seal: ALREADY_INITIALIZED (with the check gone: 8043 GenesisOnly, so the "says so first" order in the comment above it was untested) E12 governance.rs:426 veto of an executed change: CHANGE_EXECUTED (gone: the veto succeeds and marks an executed change vetoed) E13 governance.rs:517 cancel of an executed change: CHANGE_EXECUTED (gone: the cancel succeeds) E16 lifeboat.rs:256 execute_exit with no exit proposed: NOTHING_TO_EXIT (gone: 8033 ChangeNotExecutable) E17 custody.rs:84 a zero deposit: ZERO_AMOUNT (gone: it succeeds) E18 custody.rs:272 a zero release: ZERO_AMOUNT (gone: it succeeds; a zero release has no unbacked part, and the limiter, whose own zero check custody.rs:242 maps, is called only when unbacked > 0, :361) Where: E02 in after_the_seal_a_new_vault_starts_with_its_unbacked_lane_ closed, E12/E13 in an_unvetoed_loosening_executes_after_the_delay, E16 in the_lifeboat_stays_shut_until_the_seal (after the move in, before an exit is proposed), E17/E18 in the new a_zero_deposit_or_release_is_ refused. Evidence (tasks/error-codes/mutants.log, STATE=c5, full program suite): all six KILLED by the tests named above; all six SURVIVED at base. The vault suite is green (108 passed). Vault hash 7086148b unchanged. Counts: Vault 107 -> 108, programs 157 -> 158 (claims.py OK). Helpers (code-review fixes): veto_change_ix beside cancel_change_ix; create_vault_for_ix, create_vault_ix with a given mint and token program, so E02's case names USDC's own accounts and creates no mint. E02's comment now repeats the program's ("whatever the seal's state"): create_vault checks the limits first (genesis.rs:170). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pin four Vault checks another layer backstops Four checks survived deletion (tasks/error-codes/mutants.log, base cede705) because something later still refused the case, with its own error, so no test could tell. Each now has a test that sees the Vault's own code, and each was seen to fail with the check deleted: E01 lib.rs:230, create_pda's guard: a second registration of an existing protocol (register_protocol has no check of its own) -> ALREADY_INITIALIZED. Gone: the system program refuses it, with no custom code (custom_code 0). W10 lifeboat.rs:71, move_in naming another mint or token program -> WRONG_VAULT. Gone: the token program refuses (code 3, MintMismatch). W12 lifeboat.rs:247, execute_exit naming another mint or token program -> WRONG_VAULT. Gone: the same token-program error. T01 token.rs:37, account_info on an account no token program owns: a copy of a real USDC account's bytes under the system program. Gone: the quote SUCCEEDS (it promises a payout), and the release fails only in the runtime at the transfer ("instruction modified data of an account it does not own"). The quote is asserted first so the mutant is seen at the quote (STATE=c6b). New tests: a_protocol_registers_once, a_lifeboat_move_names_the_vaults_ own_mint (also completes a move in and an exit, as a control: the destination receives 1,000 USDC), a_payout_goes_only_to_an_account_a_ token_program_owns. Evidence (tasks/error-codes/mutants.log, STATE=c6 and c6b, full program suite): E01, W10, W12, T01 KILLED by the tests named above; all four SURVIVED at base. Full program suite green: Floor 50, Vault 111. Vault hash 7086148b unchanged (tests only). Counts: Vault 108 -> 111, programs 158 -> 161 (claims.py OK). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test the bounds on ticket extensions and a closed lane's code B's review of vault-error-tests-r3 (R1) found two Vault codes no test asserted by value that the earlier commits missed: ExtensionTooLong and LaneClosed. Deleting the 7-day total extension cap (brakes.rs:86, B24) left every program test green, so a live guardian, or a stolen guardian key, could hold a queued payout back without end, 24h at a time, against docs/05 row 11. The wide differential models the total (differential_wide.rs:406), but its default run never extends one ticket that far. Mapping LaneClosed to QueueFull (custody.rs:240, B23) also survived. The code review of the fix found a worse gap at the same check: no test sends a negative extension (the differential draws secs from 0, 1, 3600, 86400, 86401). With `secs <= 0` weakened to `secs == 0` (A3, brakes.rs:84) every program test passed, and one guardian key could pull a queued payout's ETA into the past, skipping the queue delay, and refund its own 7-day budget. - a_ticket_is_extended_later_by_at_most_a_day_a_call_and_seven_in_total: -1h, 0 and 24h + 1 s are each refused with EXTENSION_TOO_LONG, ETA unchanged; six 24h extensions and one of 23h pass; 2h more is refused, ETA unchanged; the last hour is granted; then 1 s more is refused. The split is unaligned so that a total that ignores this call's `secs` fails too. - a_closed_lane_refuses_an_unbacked_payout: a vault with short_bps 0, and one with long_bps 0, each refuse an unbacked admin payout with LANE_CLOSED; the same payout from a vault with both buckets open is taken (the control, so a capacity base of 0 cannot pass it). Evidence (tasks/vault-error-tests-r4/, B's runner copied: each mutant applied alone, programs rebuilt, whole program suite run): - without these tests (551e519; run-without.out, run-base2.out, run-base3.out): B23, B24, A1 (`tk.extended_s >= MAX`, this call's secs ignored), A2 (`>=` for `>`) and A3 SURVIVED. A4 (the 24h per call deleted), A5 and A6 (either half of limiter.rs:172's closed-lane test deleted) were KILLED by the differential only, not by value; - with them (run-final.out, at the branch tip): B23 KILLED by a_closed_lane_refuses_an_unbacked_payout; B24 A1 A2 A3 KILLED by the extension test only; A4 by it and the differential; A5 A6 by the closed-lane test and the differential. No program change: the runner's closing build gives 8ca8a608 / 543be60b / 7086148b, as at cede705. Counts: Vault 111 -> 113, programs 161 -> 163 (claims.py OK). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pin each half of create_pda's existing-account guard on its own B's review of vault-error-tests-r3 (R2): each half of lib.rs:229 (`!is_data_empty() || !owned_by(SYSTEM)`) survived deleted on its own (B20, B21), since an existing protocol account fails both, so a_protocol_registers_once pinned them only jointly. The test now also plants, at a fresh protocol address, an account that fails just one half: one with data owned by the system program, one with no data owned by another program (a fresh address). Each registration is refused with ALREADY_INITIALIZED. The comment says no transaction is known to make either account: data or a new owner at a PDA needs its signature, which only the Vault gives, and create_pda gives both in one instruction. Evidence (tasks/vault-error-tests-r4/): at 551e519 B20 and B21 SURVIVED (run-base2.out); at the branch tip both are KILLED by a_protocol_registers_once only (run-final.out). No count change (claims.py OK: vault_programs 113, programs 163). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Record the Vault refusal deletions and what each became docs/09 §3 gains a row for the deletion run this branch answers (tasks/error-codes/mutants.log): 38 refusals, across 14 of the 18 Vault codes no test asserted by value at cede705, each deleted on its own with the whole program suite run against it, on the AUD-2 build. The 18 (each VaultError variant with no `verr::` use in programs/tests at cede705): AlreadyInitialized NotRegistrar WrongVault WrongTokenProgram AlreadySealed VaultCountMismatch InvalidLimits InvalidAsset LaneClosed TicketNotPending ExtensionTooLong ChangeExecuted LedgerUnsound LedgerCorrupt NoSlot NothingToExit ZeroAmount Overflow. The run missed LaneClosed and ExtensionTooLong; B's review deleted one check of each (B23 custody.rs:240, B24 brakes.rs:86), and both survived until 1797ae5. That commit also pins a negative extension (A3, which survived every test before it), the 24h per call, and a lane closed by either bucket. At base: 13 KILLED, 25 SURVIVED. The commits before this one add tests that kill 18 of the 25 and both of B's, each asserting the Vault's own code (W02 W06, E04 E05, E06 E07 E08, E14, E02 E12 E13 E16 E17 E18, E01 W10 W12 T01; B23 B24). The row names the 7 that still survive and why: - six guards no transaction is known to reach: lib.rs:324 (vault PDA address; create_vault creates a vault only at its PDA), genesis.rs:519 and :523 (the lifeboat, checked by expect_pda first, created only by lifeboat_move, move_in requires the seal, lifeboat.rs:53, and the seal happens once, so no lifeboat holds data at it), brakes.rs:50 and custody.rs:541 (a pending-ticket count below zero), token.rs:235 (GetAccountDataSize's return data from an already-checked program); - custody.rs:399, stated as not shown unreachable: properties.rs:588-592 debits backed_amount and one more on random ledgers, and only crates/bulkhead-core/tests/units.rs:375 debits less, on a one-mint ledger with no liability. properties.rs:606 asserts backed(a) == min(a, backed(MAX)) for requests below 2^52 (:613), so any smaller amount is backed too, and debit_backed(b) refuses only when backed_amount(b) != b (ledger.rs:289). The known exception is a collateral value that overflows u128 (ledger.rs:262): the valid amounts need not form an interval there, so the consent or freeze cap (custody.rs:337, :347) can lower the amount to an invalid one and :399 refuses the release (fail closed). The search's own result is always valid. Not verified by a tool beyond that property test. (A full ledger cannot reach :399 through set's NoSlot: backed_amount's valid() refuses a new liability slot on a full ledger, ledger.rs:237.) Not deleted, as the row says: the two core-error mappings to ChangeExecuted and ZeroAmount, token.rs:233, and the codes Overflow and LedgerCorrupt (checked arithmetic, decoding). After this branch five codes are still asserted by no test (same grep at this commit): NotRegistrar (E03, genesis.rs:108, was KILLED at base by another test), TicketNotPending, LedgerUnsound, LedgerCorrupt, Overflow. Docs only: no count change (claims.py OK: vault_programs 113, programs 163). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Run the exclusion check's main in its self-test A's review of finding 105 (REVIEWS/auto-B-mutants-exclusions-r7.md, finding 1, Low): CI's `mutants` job acts on main's exit status, but the self-test called problems() only, and check.sh never runs the tool on real lists. With main returning 0 on a mismatch (mutant A10), every in-tree check passed, and CI would have printed the problems and gone green. This was not a defect at fb771c6, since the job fails today. The gap was that a later regression would pass. Fix: main takes its arguments and, for the self-test, a config path (default: the repo's .cargo/mutants.toml, as before), and its lines name the config it read. The self-test runs main in-process on a planted config and checks the exit status and every line on each stream: - the right lists: exit 0, "1 of 3 mutants excluded" on stdout, nothing on stderr; - the lists the other way round: exit 1, nothing on stdout, and exactly the count and "full list lacks" problems, so KEPT is read as KEPT; - coloured lists: exit 1 with exactly one problem, so a main that fails only on several problems is caught; - one argument and three: exit 2 and the two usage lines. One subprocess (python3 -I, 60 s timeout, in the temporary directory so the config must be found from the tool's own path) runs the command line with two lists and the repo's config, as scripts/mutants.sh does: on a list no pattern can match, it exits 1 with an empty stdout and problem lines that each name .cargo/mutants.toml. Which problems those are depends on the config, so only their form is checked. Self-test: 16 cases -> 22. No doc states the number. Evidence (tasks/exclusions-main-exit/): - old-A10.log: the fb771c6 self-test with A10 applied prints "self-test OK: 16 cases" and exits 0. - mutants-r6.log (mutants.py applies each mutant to a scratch copy of the tool, which reads the real config through a symlink): unmutated, "self-test OK: 22 cases", exit 0. All 18 are KILLED (exit 1): A10 (return 0), A10b (fail only on more than one problem), A0 (always 1), U1 (usage returns 0), U2/U3 (argument check `< 2`, `< 1`), K1 (KEPT ignored), K2 (ALL read twice), C1 (planted config ignored), C2 (no default config), R1 (the default config read from the cwd), E1/E2 (sys.argv sliced as [:] or [1:2]), P1 (problems not printed), S1 (problems on stdout), L1 (the planted config not named), N1 (only the first problem printed), G1 (the success line printed on failure too). Code-review passes at high shaped this. The first version ran three subprocesses against the real config, and A0, K1 and U3 survived it. The second left E2, C2 and G1 alive. Declined from the third: catching exceptions in the self-test (a traceback already exits 1) and moving the config hook out of main. The fourth showed that requiring one "matches 0 mutants" line per live pattern tied the self-test to the config's contents. The subprocess now checks the form of the lines only, and the 17 stay killed. The fifth showed that R1 survived while the child ran in the repo root (it does in check.sh); now it runs in the temporary directory. Its other points repeat declined ones. - real-lists.log: on the real lists, made with scripts/mutants.sh's commands: ".cargo/mutants.toml: 22 of 795 mutants excluded, each pattern matching its count", exit 0. - python3 -I tools/mutant_exclusions.py --self-test: 22 cases OK. - scripts/check.sh: ALL CHECKS PASSED (check-final.log); bin/check-each-commit.sh green (each-commit.log). Sibling search: the other tools check.sh runs (tools/claims.py, tools/floor_check.py) also gate CI through main's exit status. claims.py has the same gap: check.sh runs it on the real tree, where it finds no problems, and its self-test calls self_test(), not the path in main that returns 1. With that return changed to 0, `--self-test` and the real run both still exit 0 (claims-C1.log). Not changed here: it is its own commit (BACKLOG-B). floor_check.py runs in CI only as --self-test, whose result main returns as is. Its RPC mode is an operator's tool that no CI job runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Record finding 106: Vault liveness and exit rules had no failing test The tests landed in 944f5e8 (lifeboat-exit-tests-r3, 15 Oct). Their 09 §4 row waited for 104 and 105 so the list stays numbered without gaps; 105 is now on integration (fb771c6), so the row lands here. Docs only: 09 §4 row 106; README and 12's count and post-submission range (96–106, 106 fixed). Two code-review passes changed the row as written on 15 Oct: a sentence for today's re-run, the nine single-guardian sites named, G5 quoted verbatim, and the exit veto attributed to T18 (G6 has no veto), as is the veto test's doc comment (vault.rs:3996, comment only). Evidence, re-run today on bf8ba41 (the row's claims are from 15 Oct, and AUD-2 changed the Vault since): B's re-review list of 33 line- neutral mutants (the ten rules, the nine single-guardian sites L1-L9, the boundaries, the six quorums, R1-R6), each applied, programs rebuilt, all program tests run: 33 KILLED, 0 survived, 0 not applied, 0 build failures. Vault hashes after the run match the tip (Vault 7086148b, Floor 8ca8a608, Harbor 543be60b). Log: autopilot tasks/finding-106-row/mutants.log. - python3 -I tools/claims.py: OK, findings 106. - git grep for "96–105", "105 findings": no other stale range. - scripts/mutants.sh still mutates the core only (SCOPE, line 11), so the row's last sentence holds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Cite PR #100's tree, not the unpublished bf8ba41, in finding 106 Finding 106's row in docs/09 said its 33 mutants were re-run on 16 Oct at bf8ba41. That commit is not an ancestor of this branch or of the stack above it and is on no remote branch, so the citation would point at nothing once the stack lands. The owner's review of #99-#109 found it. bf8ba41 and 86b777b (the head of PR #100, vault-error-tests-d2, the D2 replay of vault-error-tests-r4) have the same tree, 36187b4 (`git rev-parse bf8ba41^{tree} 86b777b^{tree}` prints it twice). So the run was a run on 86b777b's tree, and the row now says "on tree 36187b4, the tree of 86b777b (PR #100's head)". Why not just "at 86b777b": it was committed on 17 Oct, after the run; and main squash-merges (1045cd9 has one parent), so after PR #100 lands 86b777b is reachable only as GitHub's refs/pull/100/head. The tree id stays checkable from main: squashed bottom first, main's commit for PR #100 has that tree. "PR" keeps "#100" from reading as finding 100, the row above. These points came from two code-review passes. Sibling search: every 7-12 hex token in tracked .md/.py/.sh/.mjs/.rs/ .toml files at autopilot/integration 3e9f58d that names a local commit (51 tokens) was checked with git merge-base --is-ancestor against integration. bf8ba41 was the only one that is not an ancestor. Docs only. tools/claims.py: claims OK (26 files). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
shuv-amp
added a commit
that referenced
this pull request
Oct 18, 2026
* Run the tests on 2-of-4 guardian sets; test recovery from two faults
Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set
with fewer than threshold + 2 keys. Every test fixture and most tests'
own sets were 2-of-3. This moves them to 2-of-4, which today's rule
also accepts, so this commit changes no program: Floor, Vault and Harbor
hash 508e5a99 / b07655d9 / 543be60b, as before.
- Fixtures: Protected (Floor) and Market (Vault, through Harbor) now
hold 4 guardians. Each test that built its own 3-key set gets a 4-key
set with the same threshold and the same assertion.
- Two tests changed meaning with a 4th key and were fixed, not loosened:
a_quorum_ejects_a_rogue_guardian_who_vetoes_everything (Floor) and
a_quorum_ejects_a_guardian_who_freezes_forever (Vault). With 4 keys,
the last eject was refused only because the target's own vote is
masked. Both now eject a second guardian (a set an ejection shrank to
threshold + 1 still ejects) and then show the 2-of-2 refusal.
- Account sweeps: an eject's signers may now be any two of the three
others, so each of those two slots is free for the third, never for
the target or the other signer again. Vault 5,365 and Floor 1,710
substitutions, refused (docs/09, dated 16 Oct). Listing all three
others at each slot would have skipped the duplicate: with a mutant
that counts a guardian listed twice as two (governance.rs:270), the
sweeps listing all three pass, and these fail at "eject: account 2
replaced by key ..." (Vault and Floor). No other program test kills
that mutant (tasks/aud2-guardian-margin-impl/dup-mutant-*.log).
- floor_differential: the guardian pool is 6 keys, 4 registered; the
proposed sets are 2 to 6 keys.
New tests (pass on both rules; they show what the margin buys):
- a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_set (Floor): g0's
key lost, g1 rogue stops the program; the lost key does not help a
quorum; the honest pair ejects g1 and restores; neither can veto the
owner's refill, which executes after 72h.
- two_rogues_in_turn_are_both_ejected_from_a_two_of_four_set (Floor):
the second rogue vetoes the refill and is ejected from the 2-of-3.
- a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_vault: the same in
the Vault (freeze, eject, unfreeze, refill).
Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest
and quote sweep): the counts and the sweep figures here are from runs
there (tasks/aud2-guardian-margin-impl/r4/). The E2E run, the mutants
and the new-rule run below were made before, on e8640a7; this commit's
programs and core are the same as there but for one comment line in
views.rs (line-neutral, same hashes).
Evidence:
- E2E=1 scripts/check-programs.sh: PROGRAM CHECKS PASSED (Floor 49,
Vault 98; S1 and E2E passed; the resumed setup found all 14 steps;
vault check, Floor demo and deploy dry run passed), with the next
commit's deploy rehearsal in the tree.
- tools/claims.py: OK (floor 49, vault 104, programs 153).
- Under the new rule (core from two commits on, programs rebuilt), only
the two tests that state the old rule fail: an_n_of_n_guardian_set_is_
refused (accepts 3-of-4) and the Floor differential's model.
- Mutants (tasks/coverage-map/mutants.sh), each red, then restored:
AV1 Floor live_guardian ignores liveness: KILLED by a_lost_key_and_a_
rogue_do_not_lock_a_two_of_four_set (+2 others).
AV2 the Vault's: KILLED by ..._two_of_four_vault (+2 others).
AQ1 Floor restore quorum ignores liveness: KILLED by ..._two_of_four_set
and the differential only.
AQ2 the Vault's unfreeze quorum: KILLED by ..._two_of_four_vault and
a_silent_guardian_does_not_count_toward_a_quorum.
AE1 can_eject refuses at count == threshold + 1 (AUD-2 option (a)):
KILLED by two_rogues_in_turn_..., and the two eject tests above.
Hashes after: 508e5a99 / 543be60b / b07655d9.
Split on 16 Oct from ec0cacc at the owner's review (D2): the deploy
rehearsal's move to 4 guardians is now the next commit. programs/ and
programs/tests/ here are byte-identical to ec0cacc, so its runs below
hold for this commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Rehearse deploys with four guardians, and read only the keys there are
Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set
with fewer than threshold + 2 keys, so the deploy rehearsal must
register 2-of-4 sets. Split from ec0cacc at the owner's review (D2);
the text is that commit's, unchanged:
Deploy rehearsal (cloud/deploy market.mjs, vault-check.mjs,
floor-demo.mjs): 4 guardians (keys/guardian3), so the dry run keeps
working once the rule lands; one more funding step (14, was 13).
The devnet market has 3 guardian files. A first version of this change
loaded guardian3 everywhere: vault-check.mjs failed there with ENOENT,
and heartbeat silently created a key nobody holds as a guardian. Now
market.mjs guardianKeys(dir, create) reads guardian0, 1, ... and
throws on a gap (never fills it with a new key). Setup creates keys up
to 4 only while the protocol is not registered, so resuming an old
market adds no key the chain lacks; a new deploy-dryrun step resumes
the dry-run market with guardian3 moved aside and fails if setup makes
it again (with setup always creating: "DRY RUN FAILED: resuming a
registered market made keys/guardian3.json", r3-dryrun-J7.log).
renew-guardians tops up to 4 and logs each new key (unfunded); a
2-of-3 renewal would be refused once the rule lands. keysIn reads the
guardians only when a command uses them, so caps and set-recovery
never need them; heartbeat says a key is not in the set rather than
dead. One THRESHOLD (2) for the register and renewal payloads and
MIN_GUARDIANS. vault-check.mjs
compares the chain's set with the first files, and liveness with the
chain's count. An extra file (a renewal not executed yet) and a set
without the margin are info lines, not failures, since such a set
keeps working. GUARDIAN_MARGIN and guardianMarginProblem live in
cloud/lib/solana.mjs, used by these and by the guardian bot's existing
warning (cloud/guardian/main.mjs), and vault-check uses THRESHOLD. The e2e scripts fund guardian3 too. Three
tests in cloud/test/deploy.test.mjs (client tests 176 -> 179). JS
mutants (js-mutants-r3.log), each red then restored: always create,
never create, no gap check, margin 1, `>`, GUARDIAN_MARGIN = 1.
Setup always creating survives these tests and is killed by the dry
run step above.
cloud/, scripts/deploy-dryrun.sh and docs/09's client-test row are
byte-identical to ec0cacc (git diff empty), so its runs hold for them:
the E2E check-programs run with the dry run (r4), and the JS mutants.
tools/claims.py: OK (client 179).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Refuse a guardian set without two keys beyond its threshold (107)
AUD-2, decided by the owner on 16 Oct (option (e) of 16 §4.4). A set
stays live while its honest live guardians reach the threshold: they
eject every rogue in turn, and dead guardians cannot veto the owner's
new set. Finding 96 asked for one key beyond the threshold, so a 2-of-3
or 3-of-4 set survived one fault but not a lost key plus a rogue: the
honest guardian left is no quorum, nobody can be ejected, and the rogue
keeps Floor stopped (or the Vault frozen) and vetoes every new set.
Fix, in the core so both programs inherit it through GuardianSet::parse
(registration, and every SetGuardians when proposed and when executed):
GUARDIAN_MARGIN = 2
guardian_set_ok := count <= 8 && t >= 2 && count >= t + 2
(saturating_add: t is a payload byte). A stored set is not checked
again, so existing 2-of-3 sets keep working, and ejection is unchanged:
a shrunk set still ejects. A 2- or 3-key set is BadThreshold, not
BadCount; the count rule is only the bitmap size, and the error order
of finding 103 is kept.
This changes both programs, by intent:
Floor 508e5a99 -> 8ca8a608, Vault b07655d9 -> 7086148b,
Harbor 543be60b (unchanged). Darwin-arm64 builds.
Compute (scratch LiteSVM test, tasks/aud2-guardian-margin-impl/cu.md):
-3 CU on every path that parses a set (Floor propose 2035 -> 2032,
execute 1682 -> 1679; Vault register 9592 -> 9589, SetGuardians 2979 ->
2976; Floor register varies with its PDA bump, residue also -3).
Proof and tests:
- Kani: accepted_sets_can_eject_anyone restated (ejection reachable
exactly when count > t, so option (a) cannot slip in; accepted sets
have >= 4 keys and a margin of at least 2, and 2 is accepted). New:
an_honest_live_quorum_ejects_any_rogue (any slots, anyone else alive)
and accepted_sets_survive_any_two_faults (any two faulty keys, lost
or rogue, leave a live honest quorum, which ejects each rogue, and the
rogues alone eject no honest guardian). That holds only short of a
rogue quorum: two rogues in a 2-of-n set are one (out of scope, 04
T7). With the assumption rogue < t dropped, the harness fails at
`!can_eject(rogue, ..)` (kani-no-rogue-assume.log), so the docs say
"short of a rogue quorum", not "any two faults".
19/19 verified, 240 s (cargo kani -j 2, as CI); again after the
restatement, 235 s summed (r2-prove.log).
- Kani mutants (kani-mutants.sh): margin 1, margin 3, `>` for `>=`,
t >= 1, count <= 9, option (a), and an eject one vote short (only
the two-fault harness kills that one): each KILLED by a harness
(kani-mutants-r2.log). The eject
ignoring liveness is the eject harness's case, not these.
- Program mutants (coverage-map/mutants.sh, rebuilt each time):
margin 1: KILLED by a_guardian_set_without_two_spare_keys_is_refused
(Floor and Vault), registration_rejects_weak_settings and the Floor
differential. Margin 3 and `>`: KILLED by most of the suite (the
fixtures are 2-of-4). t >= 1: first only the Floor differential; the
1-of-2 registration case was now refused by the margin too, so it is a
1-of-4 here and a Vault 1-of-4 case was added: then KILLED by both.
count <= 9: SURVIVES the program tests, equivalent there (parse refuses
more than 8 keys first), KILLED by Kani. Option (a): KILLED by the
eject tests two commits back (the tests on 2-of-4 sets).
- Tests: the Floor and Vault refusal tests above cover proposal and
execution (a stored 2-of-4 proposal whose threshold turns 3 is refused
when executed, as the devnet renewals proposed before this rule will
be); registration_rejects_weak_settings gains 2-of-3 and 3-of-4.
an_n_of_n_guardian_set_is_refused (Vault) is renamed and accepted a
3-of-4: it now refuses it and accepts 3-of-5.
- Model: the Python rule, the Floor differential's model, and the
guardian_set fuzz target state it again; the GP generator now also
picks count - 2. With the Python rule put back to the old one, seed 11
(20,000 cases) reports 5 GP mismatches; restored, seeds 11-13 identical.
Fuzz seeds two_of_three_refused, two_of_four, six_of_eight, and
seven_of_eight renamed seven_of_eight_refused (byte 323, the
threshold, is 7: now refused). With margin 1 in the core the target
crashes within 60 s; restored, 222,615,288 runs in 601 s, no crash.
- scripts/check.sh: ALL CHECKS PASSED, and E2E=1 check-programs.sh:
PROGRAM CHECKS PASSED, both at the branch tip, whose code is this
commit's (the next commit is docs only; tasks/aud2-guardian-margin-
impl/r4/check.log, r4/check-programs-e2e.log). claims: kani 19,
floor 50, vault 104, programs 154, client 179.
Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest
and quote sweep). The core and the program sources are the same as
before the rebase but for one comment line in views.rs (line-neutral),
so the Kani, fuzz, differential and mutant runs above were made on the
pre-rebase commit; the hashes, check.sh, check-programs and the counts
are from r4 runs (tasks/aud2-guardian-margin-impl/r4/).
The finding 96 row said the restated harness proves "exactly"; it now
says what it proved then and proves now, and gives the Vault test's new
name. The finding 107 row waits for rows 105 and 106 to land (09 §4 is
numbered 1..n); its text is ready. The rule's prose is the next commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* State the guardian margin in the docs, with its costs (AUD-2)
The previous commit made both programs refuse a new guardian set
without two keys beyond its threshold (finding 107). The docs that
state the rule now say so:
- README: guardians 4-8 keys; the G5 row and the "who the guardians
are" paragraph give the reason (any two faults leave a quorum, unless
the rogues are one themselves: two in a 2-of-n set, 04 T7).
- 05 register_protocol: 2 <= threshold <= count - 2, also for every
SetGuardians, proposed and executed. 05 abdicate: abdicating
SetGuardians fixes the set, so ejections shrink it for good.
- 06 state table, 00 D22, FAQ: the same rule.
- 07 §2: 4-8 guardians, and don't abdicate SetGuardians. 07 §6: the
programs enforce the margin; sets registered before keep their
exposure until replaced; an ejection uses up a spare key until the
owner's new set executes; a lost key counts as live until its
interval passes.
- 16: the AUD-2 row is decided; §4.4 gives the decision, why the
margin is the number of faults a set survives (short of a rogue
quorum), the options rejected
and why (the honest-owner and stolen-owner cases look the same
on-chain), what it costs, what it leaves (a rogue quorum; a stolen
owner key with every guardian dead; a rogue's veto of the recovery
key's rotation, checked: Floor's veto has no recovery exception, as
a_vetoed_recovery_proposal_does_not_hold_the_owner_s_slot shows), and
Lido-style lapsing vetoes as a future option only. §5: "independent
guardians, 3-of-5 or more".
- 10: the guardian set's minimum size is no longer an open decision.
SECURITY.md and 08 describe what is on devnet, which still runs the
old rule, and are left as they are.
claims: OK. Docs only; no program file changed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Say the weekly CI run skips mutants, as its condition does
ci.yml's header and scripts/ci-changes.sh said the weekly run does every
job, and 09 §8 gave `mutants` as running on any change to the core. The
`mutants` job's `if` starts with `github.event_name == 'pull_request'`,
so the Monday schedule (and a manual dispatch) never runs it: it mutates
a pull request's diff and has none to mutate. Its trigger list in 09 §8
also left out scripts/mutants.sh and .cargo/mutants.toml, which
ci-changes.sh (line 22) selects it for.
Comments and docs only; nothing runs differently.
Sibling search: `git grep` for "weekly run does", "every job ... weekly"
and "mutants ... weekly" across md/yml/sh/mjs/py. The other hits are
09 §8's "at full depth every Monday" (its table now says which job is
not weekly) and the dispatch input's "every job at weekly depth", which
is the same set as the weekly run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check that the mutant exclusions still match; repin murmur3 (finding 105)
.cargo/mutants.toml excludes the core's 22 equivalent mutants, and the
docs say so. Since 9 Oct it excluded 19. The murmur3 tail's pattern is
pinned by line, `floor.rs:(235|238|241)`, because the body's `h ^= k`
has the same column and function and is not equivalent. 1dc7861 (#52)
moved those lines (now 327/330/333), and nothing checked the patterns.
Failing closed, the three would have failed CI's `mutants` job on a
pull request that changed those lines. Failing open, a pattern that
starts matching a non-equivalent mutant hides it from every run.
Root cause: nothing checked that a pattern still names its mutants.
Fix:
- each pattern ends with the number of mutants it excludes (`# 3`), and
the murmur3 pattern names 327/330/333;
- tools/mutant_exclusions.py fails unless each pattern matches exactly
its number in `cargo mutants --list --no-config`, and cargo-mutants
(the list with the config) excludes exactly the mutants the patterns
match, so a pattern this parser can't see, or reads differently, fails
too. No two patterns may match the same mutant, since the docs' count
is the counts' sum, and the kept list must be part of the full one.
It reads only the exclude_re array's body, needs no tomllib
(Python 3.9), and names colour codes in the list as the problem;
- scripts/mutants.sh makes both lists (--colors never: CI sets
CARGO_TERM_COLOR=always) and runs it first, except when the caller
passes --config or --no-config (as whole arguments). CI's `mutants` job runs that script;
ci-changes.sh now also selects the job for the new tool;
- scripts/check.sh runs its --self-test on planted configs;
- tools/claims.py ties the docs' "22 equivalents" (README, 09 §6 and the
summary row) to the sum of the counts, with the same parser. It finds
that parser from its own directory, so `python3 -I` runs it too, and
check.sh runs its self-test that way. 00's
"18 equivalents" (26 Sep log entry) and run 9's "exactly the 22
equivalents below" (6 Oct) are kept as dated records.
Docs: 09 §4 finding 105, §6 run 10 and the "In CI" note, §8's mutants
row; README and 09's summary row 669 → 797 mutants (run 10); README and
12's post-submission range 96–105.
Evidence (first on 1f2ad48 + the parent, logs in the autopilot's
tasks/mutants-exclusions-r2/; re-run on 58697c6 + the parent,
tasks/mutants-exclusions-r3/):
- `cargo mutants --list`: 797 with --no-config, 778 with the old config
(19 excluded), 775 with this one. The 22 excluded are §6's list, the
murmur3 ones at 327/330/333.
- scripts/mutants.sh --list: exit 0, "22 of 797 mutants excluded"; with
CARGO_TERM_COLOR=always also exit 0. Old config: exit 1, every pattern
"no '# N' count". Planted on the real lists: the old lines with a
count (matches 0, says 3) and the body XOR added (matches 4, says 3)
both exit 1; in both the set comparison alone passes. /usr/bin/python3
(3.9.6): self-test OK and the real check passes.
- The self-test pins each problem by its message, not a count. Mutants
of the tool (tool-mutants.log), each red: M1 no per-pattern check,
M2 no set comparison, M3 no colour check, M4 no empty-array check,
M5 a missing count accepted, M6 a missing count read as 0, M7 the
parser reading past the array. M3 and M5 survived a first version
that checked only the number of problems.
After A's review (16 Oct): the set comparison was pinned only by its
size, so `len(excluded) != len(matched)` passed the self-test. A
case with as many excluded as matched but not the same ones now
fails it (15 cases; red with that mutant, green without), and the
message says how many mutants are in one set only.
From the code review after the rebase: colour codes only in the kept
list were reported as a kept list made with other options; the
colour check now reads both lists (16 cases; dropping the kept half
fails the new case).
On 58697c6, with the overlap and kept-list checks: M8 no overlap
check and M9 no kept-list check, red too, as M1–M7 still are. A
duplicate of the can_eject pattern planted in the real config:
scripts/mutants.sh --list exits 1, "the counts add up to 23, the
patterns match 22 mutants" (planted-overlap.log). Without the
sys.path line, `python3 -I tools/claims.py` fails with
ModuleNotFoundError; with it, both claims runs pass.
- claims.py (claims-mutants.log): one count 1 → 2 fails four statements
(README 188; 09 33, 333, 335; kept-run9.log), not run 9's 331, which
failed too before it was kept; a dropped count fails with the
tool named; deleting any of the three rules fails a self-test case;
deleting the KEPT entry fails 00:103.
- The 22 excluded mutants run alone (--no-config, --re per pattern,
2 min): 22 missed, exactly the excluded list; --re also picked
ema.rs:70 (a field deletion, no pattern matches it), caught.
- Run 10 (f1f05ba, old config, -j4, 46 min; tasks/mutants-run10/):
778 tested, 712 caught, 5 timeouts, 58 unviable, 3 missed, exactly
the murmur3 tail. The core's source is unchanged since (`git diff
--stat f1f05ba HEAD -- crates/bulkhead-core/src` is empty; only
tests/units.rs gained finding 102's tests).
- No program or core source changed.
Rebased on 16 Oct onto cede705 (findings 102 and 100, the attest
sweep and AUD-2). AUD-2 changed the core's source (governance.rs), so
the line above saying it is unchanged since run 10 held on 58697c6, not
here. On cede705: scripts/mutants.sh's list check passes, "22 of 795
mutants excluded, each pattern matching its count"; the murmur3 lines
are still 327/330/333. The 797 in README and 09 is run 10's figure:
README now dates it ("in the last full run, 15 Oct") and finding
105's row says the core has changed since (finding 107).
The setup line in scripts/mutants.sh now pins cargo-mutants 27.1.0,
the version ci.yml installs: each count is the number of mutants that
version lists, so another release can break the check (code review,
16 Oct; `git grep 'install.*cargo-mutants'` finds no other unpinned
install).
The check's report goes to stderr in scripts/mutants.sh, so
`scripts/mutants.sh --list` prints only cargo-mutants' names: before,
its first stdout line was the report; after, stdout is 773 lines
(795 - 22) and stderr the one report line. ci.yml's `mutants` comment
says the job checks the exclusions first.
Row 105 and 09 §6 gave the pin's reason as the body's XORs. The
finalizer's are at column 7 (`grep -n '\^=' floor.rs`, 337-342); the
non-equivalent XOR at column 11 is `h ^= k`, in the body (320) and after
the tail (335). The toml comment, the row and §6 now say that. 09 §8's
`check` row names the two tool self-tests check.sh runs.
README no longer calls run 10's 797 "injected bugs": cargo-mutants
listed 797 but tested 778, 19 being excluded (09 §6, run 10's row).
Sibling search: the other 14 patterns use `\d+` for the line, and each
matches the count now written after it. The murmur3 one is the only
line-pinned pattern.
Not done, by choice:
- Catching an edit that keeps a pattern's number but changes its
mutants (an excluded line itself edited, or a murmur3 body XOR moved
onto a pinned line). The toml and 09 §6 say so: such an edit needs the
mutant judged again by hand. Failing on any diff that touches an
excluded line would catch the first; it is a separate change.
- Moving murmur3's tail into its own function, so its pattern needn't
name lines. It shifts core lines, so it changes the program hashes.
Until then a line move above it fails the check, with a message
saying what to fix.
- A weekly full mutants run in CI. It took 46 min here at -j4, likely
past the job's 90 min on two CI cores: a minutes call for the owner.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that a ticket stays in its own protocol and vault
Two Vault checks that keep a ticket to its own protocol and vault had no
test. Deleting either one left every program test green
(tasks/error-codes/mutants.log, base cede705):
- brakes.rs:79, extend_ticket: tk.protocol != protocol. Without it, a live
guardian of any other protocol extends this protocol's tickets. Anyone
can register a protocol and name its guardians, so that guardian could
hold back every queued payout up to the total extension limit. The
sweep did not catch it: it has no second protocol's guardian to swap in.
- custody.rs:501, settle_ticket: tk.vault != vault. A USDC ticket settled
through the SOL vault (its tokens, its mint) got past the Vault's checks.
New test another_protocols_guardian_cannot_extend_a_ticket: a second
Market's live guardian extends this market's ticket -> WRONG_VAULT; the
same instruction from this market's guardian succeeds. The settle test
now settles the USDC ticket through the SOL vault and expects
WRONG_VAULT.
Evidence (tasks/error-codes/mutants.log, STATE=c1; full program suite
per mutant):
W02 brakes.rs:79 deleted -> KILLED by
another_protocols_guardian_cannot_extend_a_ticket ("expected
failure, got success": the stranger's extension went through)
W06 custody.rs:501 deleted -> KILLED by
a_ticket_settles_after_its_eta_to_its_stored_recipient_only (8049
InsufficientBalance from the empty SOL vault, not 8010)
Both were SURVIVED at base. No program source change: hashes
8ca8a608/543be60b/7086148b, as at cede705.
Sibling search: the other 12 WrongVault sites were deleted one at a time
in the same run. Eight are KILLED by the sweep (brakes.rs:39, lib.rs:320,
custody.rs:160, genesis.rs:491, views.rs:27, lifeboat.rs:151,
governance.rs:169/259). lib.rs:324 (vault PDA address), genesis.rs:523,
lifeboat.rs:71 and :247 survive; later commits on this branch deal with
them.
Counts: Vault 104 -> 105, programs 154 -> 155 (claims.py OK).
The test also reads the ticket's ETA: unchanged after the
refused extension, 3,600 s later after its own guardian's (code-review
fix: success alone did not show the extension happened).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that the seal names every vault and happens once
Two seal checks had no test; deleting either left every program test
green (tasks/error-codes/mutants.log, base cede705):
- genesis.rs:486, execute_seal: the pairs must number 3 * vault_count.
Without it, the seal checks only the vaults it is given. A vault left
out skips its claims-vs-balance check, so an uncovered import seals.
docs/05's seal row says every vault is passed exactly once.
the_seal_checks_each_vault_once covered a vault named twice in place of
another, not one left out.
- genesis.rs:431, propose_seal: refused once sealed (AlreadySealed).
Without it, a sealed protocol's admin re-proposes, which rewrites
seal_executable_s/expires_s and clears seal_vetoed on a sealed protocol.
the_seal_checks_each_vault_once now also leaves the uncovered USDC vault
out (expects VAULT_COUNT_MISMATCH) and, after the seal, proposes it again
(expects ALREADY_SEALED). No new test, so no count change.
Evidence (tasks/error-codes/mutants.log, STATE=c2, full program suite):
E05 genesis.rs:486 deleted -> KILLED by the_seal_checks_each_vault_once
("expected failure, got success": the seal went through without
the USDC vault and its 1,000 USDC uncovered import)
E04 genesis.rs:431 deleted -> KILLED by the_seal_checks_each_vault_once
("expected failure, got success")
Both were SURVIVED at base. Vault hash 7086148b unchanged (tests only).
Sibling search: the seal's other checks. :491 vault_tokens (KILLED by the
sweep), :519 lifeboat owner and :523 lifeboat mint/owner (SURVIVED; the
lifeboat is a PDA checked by expect_pda just above, which only this
program can create, as its own token account: see this branch's note in
docs/09).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that invalid limits and asset settings are refused at every entry
Three bound checks had no test; deleting any one left every program test
green (tasks/error-codes/mutants.log, base cede705):
- genesis.rs:171, create_vault: config_is_valid (bps <= 100%, windows
> 0). Without it, a vault is created with a 100.01%-an-hour burst.
- governance.rs:40, parse of a SetLimits change (at proposal and again
at execution, :254 and :478): the same rule for a proposed change.
- governance.rs:47, parse of an UpsertAsset change: valid_asset (haircut
<= 100%, liability >= 100%, the oracle bounds of finding 88). Only the
genesis upsert's copy (genesis.rs:296) was tested, by
oracle_settings_of_a_priced_asset_are_bounded.
New test invalid_limits_and_assets_are_refused_wherever_they_enter:
before the seal, create_vault with short_bps 10,001 -> INVALID_LIMITS;
after it, a SetLimits proposal with long_window_s 0 -> INVALID_LIMITS,
and an UpsertAsset proposal with a 10,001 bps haircut -> INVALID_ASSET.
Evidence (tasks/error-codes/mutants.log, STATE=c3, full program suite):
E06 genesis.rs:171 deleted -> KILLED ("expected failure, got success")
E07 governance.rs:40 deleted -> KILLED ("expected failure, got success")
E08 governance.rs:47 deleted -> KILLED ("expected failure, got success")
each by the new test only. All three were SURVIVED at base. Vault hash
7086148b unchanged (tests only).
Sibling search: every config_is_valid / valid_asset call in programs/
vault/src: genesis.rs:170 (create_vault), :295 (upsert_asset, KILLED by
the oracle test), governance.rs:39 and :46. All now killed.
Counts: Vault 105 -> 106, programs 155 -> 156 (claims.py OK).
The execution copy (:478) is not reached by any transaction: it
parses the bytes the proposal already checked, and the bounds read
nothing else. The test comment says so (code-review fix: it had
claimed execution was covered). oracle_settings_of_a_priced_asset_are_
bounded now asserts INVALID_ASSET for each of its four cases, not any
error (all four return it; E09 stays KILLED).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that an import into a ninth mint is refused
genesis.rs:410, import_backing: a backing ledger holds MAX_SLOTS (8)
mints, and an import into a ninth is refused with NoSlot. Deleting the
check left every program test green (tasks/error-codes/mutants.log, base
cede705). From the code, without it the import goes through: the vault's
imported total grows (:414), and the seal must cover that, but the
owner's ledger is saved unchanged, so the owner has no backed claim for
it.
New test an_import_into_a_ninth_mint_is_refused: nine vaults before the
seal; one owner gets an import into eight (ledger_of shows 8 positions),
then the ninth -> NO_SLOT.
Evidence (tasks/error-codes/mutants.log, STATE=c4, full program suite):
E14 genesis.rs:410 deleted -> KILLED by an_import_into_a_ninth_mint_is_
refused ("expected failure, got success"); SURVIVED at base.
Vault hash 7086148b unchanged (tests only).
Sibling search: the only other ledger credit is deposit's
(custody.rs:128). There a full ledger is by design: the deposit is taken
unbacked (Ledger::credit's doc, ledger.rs:187-189), and credit's result is
the backed flag, not an error.
Counts: Vault 106 -> 107, programs 156 -> 157 (claims.py OK).
import_ix now calls import_into_ix, which takes the vault, so the
new test shares the one import layout (code-review fix).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin six Vault refusals no test reached
Six reachable checks had no test that failed when the check was deleted
(tasks/error-codes/mutants.log, base cede705). Each now has an assertion
on its code, and each was seen to fail with the check deleted:
E02 genesis.rs:185 create_vault of an existing vault after the seal:
ALREADY_INITIALIZED (with the check gone: 8043 GenesisOnly, so the
"says so first" order in the comment above it was untested)
E12 governance.rs:426 veto of an executed change: CHANGE_EXECUTED
(gone: the veto succeeds and marks an executed change vetoed)
E13 governance.rs:517 cancel of an executed change: CHANGE_EXECUTED
(gone: the cancel succeeds)
E16 lifeboat.rs:256 execute_exit with no exit proposed:
NOTHING_TO_EXIT (gone: 8033 ChangeNotExecutable)
E17 custody.rs:84 a zero deposit: ZERO_AMOUNT (gone: it succeeds)
E18 custody.rs:272 a zero release: ZERO_AMOUNT (gone: it succeeds; a
zero release has no unbacked part, and the limiter, whose own zero
check custody.rs:242 maps, is called only when unbacked > 0, :361)
Where: E02 in after_the_seal_a_new_vault_starts_with_its_unbacked_lane_
closed, E12/E13 in an_unvetoed_loosening_executes_after_the_delay, E16 in
the_lifeboat_stays_shut_until_the_seal (after the move in, before an
exit is proposed), E17/E18 in the new a_zero_deposit_or_release_is_
refused.
Evidence (tasks/error-codes/mutants.log, STATE=c5, full program suite):
all six KILLED by the tests named above; all six SURVIVED at base. The
vault suite is green (108 passed). Vault hash 7086148b unchanged.
Counts: Vault 107 -> 108, programs 157 -> 158 (claims.py OK).
Helpers (code-review fixes): veto_change_ix beside cancel_change_ix;
create_vault_for_ix, create_vault_ix with a given mint and token
program, so E02's case names USDC's own accounts and creates no mint.
E02's comment now repeats the program's ("whatever the seal's state"):
create_vault checks the limits first (genesis.rs:170).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin four Vault checks another layer backstops
Four checks survived deletion (tasks/error-codes/mutants.log, base
cede705) because something later still refused the case, with its own
error, so no test could tell. Each now has a test that sees the Vault's
own code, and each was seen to fail with the check deleted:
E01 lib.rs:230, create_pda's guard: a second registration of an
existing protocol (register_protocol has no check of its own) ->
ALREADY_INITIALIZED. Gone: the system program refuses it, with no
custom code (custom_code 0).
W10 lifeboat.rs:71, move_in naming another mint or token program ->
WRONG_VAULT. Gone: the token program refuses (code 3, MintMismatch).
W12 lifeboat.rs:247, execute_exit naming another mint or token
program -> WRONG_VAULT. Gone: the same token-program error.
T01 token.rs:37, account_info on an account no token program owns: a
copy of a real USDC account's bytes under the system program. Gone:
the quote SUCCEEDS (it promises a payout), and the release fails
only in the runtime at the transfer ("instruction modified data of
an account it does not own"). The quote is asserted first so the
mutant is seen at the quote (STATE=c6b).
New tests: a_protocol_registers_once, a_lifeboat_move_names_the_vaults_
own_mint (also completes a move in and an exit, as a control: the
destination receives 1,000 USDC), a_payout_goes_only_to_an_account_a_
token_program_owns.
Evidence (tasks/error-codes/mutants.log, STATE=c6 and c6b, full program
suite): E01, W10, W12, T01 KILLED by the tests named above; all four
SURVIVED at base. Full program suite green: Floor 50, Vault 111. Vault
hash 7086148b unchanged (tests only).
Counts: Vault 108 -> 111, programs 158 -> 161 (claims.py OK).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test the bounds on ticket extensions and a closed lane's code
B's review of vault-error-tests-r3 (R1) found two Vault codes no test
asserted by value that the earlier commits missed: ExtensionTooLong
and LaneClosed. Deleting the 7-day total extension cap (brakes.rs:86,
B24) left every program test green, so a live guardian, or a stolen
guardian key, could hold a queued payout back without end, 24h at a
time, against docs/05 row 11. The wide differential models the total
(differential_wide.rs:406), but its default run never extends one
ticket that far. Mapping LaneClosed to QueueFull (custody.rs:240, B23)
also survived.
The code review of the fix found a worse gap at the same check: no
test sends a negative extension (the differential draws secs from 0,
1, 3600, 86400, 86401). With `secs <= 0` weakened to `secs == 0` (A3,
brakes.rs:84) every program test passed, and one guardian key could
pull a queued payout's ETA into the past, skipping the queue delay,
and refund its own 7-day budget.
- a_ticket_is_extended_later_by_at_most_a_day_a_call_and_seven_in_total:
-1h, 0 and 24h + 1 s are each refused with EXTENSION_TOO_LONG, ETA
unchanged; six 24h extensions and one of 23h pass; 2h more is
refused, ETA unchanged; the last hour is granted; then 1 s more is
refused. The split is unaligned so that a total that ignores this
call's `secs` fails too.
- a_closed_lane_refuses_an_unbacked_payout: a vault with short_bps 0,
and one with long_bps 0, each refuse an unbacked admin payout with
LANE_CLOSED; the same payout from a vault with both buckets open is
taken (the control, so a capacity base of 0 cannot pass it).
Evidence (tasks/vault-error-tests-r4/, B's runner copied: each mutant
applied alone, programs rebuilt, whole program suite run):
- without these tests (551e519; run-without.out, run-base2.out,
run-base3.out): B23, B24, A1 (`tk.extended_s >= MAX`, this call's
secs ignored), A2 (`>=` for `>`) and A3 SURVIVED. A4 (the 24h per
call deleted), A5 and A6 (either half of limiter.rs:172's closed-lane
test deleted) were KILLED by the differential only, not by value;
- with them (run-final.out, at the branch tip): B23 KILLED by
a_closed_lane_refuses_an_unbacked_payout; B24 A1 A2 A3 KILLED by the
extension test only; A4 by it and the differential; A5 A6 by the
closed-lane test and the differential.
No program change: the runner's closing build gives 8ca8a608 /
543be60b / 7086148b, as at cede705.
Counts: Vault 111 -> 113, programs 161 -> 163 (claims.py OK).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin each half of create_pda's existing-account guard on its own
B's review of vault-error-tests-r3 (R2): each half of lib.rs:229
(`!is_data_empty() || !owned_by(SYSTEM)`) survived deleted on its own
(B20, B21), since an existing protocol account fails both, so
a_protocol_registers_once pinned them only jointly.
The test now also plants, at a fresh protocol address, an account that
fails just one half: one with data owned by the system program, one
with no data owned by another program (a fresh address). Each
registration is refused with ALREADY_INITIALIZED. The comment says no transaction is
known to make either account: data or a new owner at a PDA needs its
signature, which only the Vault gives, and create_pda gives both in one
instruction.
Evidence (tasks/vault-error-tests-r4/): at 551e519 B20 and B21
SURVIVED (run-base2.out); at the branch tip both are KILLED by
a_protocol_registers_once only (run-final.out).
No count change (claims.py OK: vault_programs 113, programs 163).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Record the Vault refusal deletions and what each became
docs/09 §3 gains a row for the deletion run this branch answers
(tasks/error-codes/mutants.log): 38 refusals, across 14 of the 18 Vault
codes no test asserted by value at cede705, each deleted on its own
with the whole program suite run against it, on the AUD-2 build.
The 18 (each VaultError variant with no `verr::` use in programs/tests
at cede705): AlreadyInitialized NotRegistrar WrongVault
WrongTokenProgram AlreadySealed VaultCountMismatch InvalidLimits
InvalidAsset LaneClosed TicketNotPending ExtensionTooLong
ChangeExecuted LedgerUnsound LedgerCorrupt NoSlot NothingToExit
ZeroAmount Overflow. The run missed LaneClosed and ExtensionTooLong;
B's review deleted one check of each (B23 custody.rs:240, B24
brakes.rs:86), and both survived until 1797ae5. That commit
also pins a negative extension (A3, which survived every test before
it), the 24h per call, and a lane closed by either bucket.
At base: 13 KILLED, 25 SURVIVED. The commits before this one add tests
that kill 18 of the 25 and both of B's, each asserting the Vault's own
code (W02 W06, E04 E05, E06 E07 E08, E14, E02 E12 E13 E16 E17 E18, E01
W10 W12 T01; B23 B24). The row names the 7 that still survive and why:
- six guards no transaction is known to reach: lib.rs:324 (vault PDA
address; create_vault creates a vault only at its PDA), genesis.rs:519
and :523 (the lifeboat, checked by expect_pda first, created only by
lifeboat_move, move_in requires the seal, lifeboat.rs:53, and the
seal happens once, so no lifeboat holds data at it), brakes.rs:50 and
custody.rs:541 (a pending-ticket count below zero), token.rs:235
(GetAccountDataSize's return data from an already-checked program);
- custody.rs:399, stated as not shown unreachable: properties.rs:588-592
debits backed_amount and one more on random ledgers, and only
crates/bulkhead-core/tests/units.rs:375 debits less, on a one-mint
ledger with no liability. properties.rs:606 asserts backed(a) ==
min(a, backed(MAX)) for requests below 2^52 (:613), so any smaller
amount is backed too, and debit_backed(b) refuses only when
backed_amount(b) != b (ledger.rs:289). The known exception is a
collateral value that overflows u128 (ledger.rs:262): the valid
amounts need not form an interval there, so the consent or freeze
cap (custody.rs:337, :347) can lower the amount to an invalid one and
:399 refuses the release (fail closed). The search's own result is
always valid. Not verified by a tool beyond that property test. (A full ledger cannot reach :399 through
set's NoSlot: backed_amount's valid() refuses a new liability slot
on a full ledger, ledger.rs:237.)
Not deleted, as the row says: the two core-error mappings to
ChangeExecuted and ZeroAmount, token.rs:233, and the codes Overflow and
LedgerCorrupt (checked arithmetic, decoding).
After this branch five codes are still asserted by no test (same grep
at this commit): NotRegistrar (E03, genesis.rs:108, was KILLED at base
by another test), TicketNotPending, LedgerUnsound, LedgerCorrupt,
Overflow.
Docs only: no count change (claims.py OK: vault_programs 113, programs
163).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Run the exclusion check's main in its self-test
A's review of finding 105 (REVIEWS/auto-B-mutants-exclusions-r7.md,
finding 1, Low): CI's `mutants` job acts on main's exit status, but the
self-test called problems() only, and check.sh never runs the tool on
real lists. With main returning 0 on a mismatch (mutant A10), every
in-tree check passed, and CI would have printed the problems and gone
green. This was not a defect at fb771c6, since the job fails today. The
gap was that a later regression would pass.
Fix: main takes its arguments and, for the self-test, a config path
(default: the repo's .cargo/mutants.toml, as before), and its lines name
the config it read. The self-test runs main in-process on a planted
config and checks the exit status and every line on each stream:
- the right lists: exit 0, "1 of 3 mutants excluded" on stdout, nothing
on stderr;
- the lists the other way round: exit 1, nothing on stdout, and exactly
the count and "full list lacks" problems, so KEPT is read as KEPT;
- coloured lists: exit 1 with exactly one problem, so a main that fails
only on several problems is caught;
- one argument and three: exit 2 and the two usage lines.
One subprocess (python3 -I, 60 s timeout, in the temporary directory so
the config must be found from the tool's own path) runs the command line
with two lists and the repo's config, as scripts/mutants.sh does: on a
list no pattern can match, it exits 1 with an empty stdout and problem
lines that each name .cargo/mutants.toml. Which problems those are
depends on the config, so only their form is checked. Self-test: 16
cases -> 22. No doc states the number.
Evidence (tasks/exclusions-main-exit/):
- old-A10.log: the fb771c6 self-test with A10 applied prints "self-test
OK: 16 cases" and exits 0.
- mutants-r6.log (mutants.py applies each mutant to a scratch copy of
the tool, which reads the real config through a symlink): unmutated,
"self-test OK: 22 cases", exit 0. All 18 are KILLED (exit 1): A10
(return 0), A10b (fail only on more than one problem), A0 (always 1),
U1 (usage returns 0), U2/U3 (argument check `< 2`, `< 1`), K1 (KEPT
ignored), K2 (ALL read twice), C1 (planted config ignored), C2 (no
default config), R1 (the default config read from the cwd), E1/E2
(sys.argv sliced as [:] or [1:2]), P1 (problems not printed), S1
(problems on stdout), L1 (the planted config not named), N1 (only the
first problem printed), G1 (the success line printed on failure too).
Code-review passes at high shaped this. The first version ran three
subprocesses against the real config, and A0, K1 and U3 survived it.
The second left E2, C2 and G1 alive. Declined from the third: catching
exceptions in the self-test (a traceback already exits 1) and moving
the config hook out of main. The fourth showed that requiring one
"matches 0 mutants" line per live pattern tied the self-test to the
config's contents. The subprocess now checks the form of the lines
only, and the 17 stay killed. The fifth showed that R1 survived while
the child ran in the repo root (it does in check.sh); now it runs in
the temporary directory. Its other points repeat declined ones.
- real-lists.log: on the real lists, made with scripts/mutants.sh's
commands: ".cargo/mutants.toml: 22 of 795 mutants excluded, each
pattern matching its count", exit 0.
- python3 -I tools/mutant_exclusions.py --self-test: 22 cases OK.
- scripts/check.sh: ALL CHECKS PASSED (check-final.log);
bin/check-each-commit.sh green (each-commit.log).
Sibling search: the other tools check.sh runs (tools/claims.py,
tools/floor_check.py) also gate CI through main's exit status. claims.py
has the same gap: check.sh runs it on the real tree, where it finds no
problems, and its self-test calls self_test(), not the path in main that
returns 1. With that return changed to 0, `--self-test` and the real run
both still exit 0 (claims-C1.log). Not changed here: it is its own
commit (BACKLOG-B). floor_check.py runs in CI only as --self-test, whose
result main returns as is. Its RPC mode is an operator's tool that no CI
job runs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Record finding 106: Vault liveness and exit rules had no failing test
The tests landed in 944f5e8 (lifeboat-exit-tests-r3, 15 Oct). Their
09 §4 row waited for 104 and 105 so the list stays numbered without
gaps; 105 is now on integration (fb771c6), so the row lands here.
Docs only: 09 §4 row 106; README and 12's count and post-submission
range (96–106, 106 fixed). Two code-review passes changed the row as
written on 15 Oct: a sentence for today's re-run, the nine
single-guardian sites named, G5 quoted verbatim, and the exit veto
attributed to T18 (G6 has no veto), as is the veto test's doc comment
(vault.rs:3996, comment only).
Evidence, re-run today on bf8ba41 (the row's claims are from 15 Oct,
and AUD-2 changed the Vault since): B's re-review list of 33 line-
neutral mutants (the ten rules, the nine single-guardian sites L1-L9,
the boundaries, the six quorums, R1-R6), each applied, programs
rebuilt, all program tests run:
33 KILLED, 0 survived, 0 not applied, 0 build failures. Vault
hashes after the run match the tip (Vault 7086148b, Floor 8ca8a608,
Harbor 543be60b).
Log: autopilot tasks/finding-106-row/mutants.log.
- python3 -I tools/claims.py: OK, findings 106.
- git grep for "96–105", "105 findings": no other stale range.
- scripts/mutants.sh still mutates the core only (SCOPE, line 11), so
the row's last sentence holds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Cite PR #100's tree, not the unpublished bf8ba41, in finding 106
Finding 106's row in docs/09 said its 33 mutants were re-run on 16 Oct
at bf8ba41. That commit is not an ancestor of this branch or of the
stack above it and is on no remote branch, so the citation would point
at nothing once the stack lands. The owner's review of #99-#109 found it.
bf8ba41 and 86b777b (the head of PR #100, vault-error-tests-d2, the D2
replay of vault-error-tests-r4) have the same tree, 36187b49
(`git rev-parse bf8ba41^{tree} 86b777b^{tree}` prints it twice). So
the run was a run on 86b777b's tree, and the row now says "on tree
36187b49, the tree of 86b777b (PR #100's head)". Why not just "at
86b777b": it was committed on 17 Oct, after the run; and main
squash-merges (1045cd9 has one parent), so after PR #100 lands 86b777b
is reachable only as GitHub's refs/pull/100/head. The tree id stays
checkable from main: squashed bottom first, main's commit for PR #100
has that tree. "PR" keeps "#100" from reading as finding 100, the
row above. These points came from two code-review passes.
Sibling search: every 7-12 hex token in tracked .md/.py/.sh/.mjs/.rs/
.toml files at autopilot/integration 3e9f58d that names a local commit
(51 tokens) was checked with git merge-base --is-ancestor against
integration. bf8ba41 was the only one that is not an ancestor.
Docs only. tools/claims.py: claims OK (26 files).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Warn about a Floor guardian set short of the margin (finding 107)
The guardian checks the Vault protocol's guardian set against finding
107's margin (main.mjs checkGuardianSet), but never a Floor config's.
A Floor set registered before the rule, say 2-of-3, keeps working and
keeps AUD-2's exposure: one lost key and one rogue guardian lock its
governance for good. The guardian said nothing, and docs/07 §6 said so
("not yet a Floor set").
The rule moves to policy.mjs as guardianSetWarning, a pure function:
the same text as before, a warn line that alerts on mainnet only. In
main.mjs, checkGuardianSet now remembers the set shape per set
("protocol", or the protected program), and checkFloor calls it for
each watched Floor config it reads. floorAlerts is unchanged. The
margin is not a readiness code there: "floor not ready" alerts on
every cluster, and only the owner's new set clears a margin problem, so
"floor ready" would never be logged again.
Evidence (tasks/floor-guardian-margin/r2/):
- node --test cloud/test/*.test.mjs: 180/180 pass (one new test).
- Mutants of guardianSetWarning, each run on policy.test.mjs and
reverted; each fails the new test: N1 never warn, N2 threshold fixed
at 2, N3 notify always, N4 notify never, N5 GUARDIAN_MARGIN = 1.
- tools/claims.py: OK, client 180 (docs/09:40 updated).
- Not tested: the main.mjs wiring (the per-set Map and the checkFloor
call). No test runs checkFloor; deleting the call leaves every test
green.
Sibling search: guardianMarginProblem's callers are vault-check.mjs (an
info line) and guardianSetWarning. The status snapshot
(snapshot.mjs:33) prints live guardians and the quorum, not the margin;
left as it is.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Judge each guardian set's margin on its own, and say when it is fixed
The parent's per-set memory (which set shapes were already reported)
lived in main.mjs, so no test could fail on it: dropping the
once-per-shape check, or keying every set the same, left every test
green. The warning did not say which set it was about (only a program
tag on Floor lines), and it gave the same text for a set ejected down
to its threshold, or registered n-of-n before finding 96. There one
rogue guardian alone locks governance; a lost key alone does not, since
dead guardians cannot veto the owner's new set (docs/16 §4.4), but it
leaves no quorum until that set executes. Nothing was said when the owner's
new set fixed the problem. And a delivered alert did not say which Floor
program it was about: the program went only into the line's `program`
field, which alertFor uses in its repeat key but does not show.
Moving the program name into the text is a trap: alertFor keys repeats
on the first 60 characters of the text, and "the Floor guardians of
<44-character id>" pushes the shape past them. A 2-of-3 set ejected to
2-of-2 within ten minutes would then give the same key, and the worse
warning would not alert. So the program goes after the shape: "the
Floor guardians are 2-of-3 for <program>: ...". The `program` field
stays for the repeat key.
guardianSetWarning becomes guardianSetCheck(shapes, which, set,
mainnet) in policy.mjs. It keeps each set's last shape and whether it
was short in the Map, keyed by "protocol" or the protected program, and
returns the line to log or null. The warning names the set and has its
own text for n-of-n. When a set that was short gets the margin, it
returns one line saying so, which alerts on mainnet like the warning,
so whoever was paged hears that it is fixed. main.mjs only logs what it
returns. docs/07 §6 says so. The advice also had a hole: a set
registered as 7-of-8 was told to "run at least 7-of-9", and no set can
have 9 keys (core MAX_GUARDIANS is 8). When threshold + 2 is over 8,
the advice is now to lower the threshold to at most 6-of-8. That needs
MAX_GUARDIANS in JS, so lib/solana.mjs gets it next to
GUARDIAN_MARGIN; the next commit ties both to the core.
Evidence (tasks/floor-guardian-margin/):
- node --test cloud/test/*.test.mjs: 180/180 pass (the test is
extended, not added).
- 19 mutants, each run on policy.test.mjs and reverted (r6/mutants.py,
r6/mutants.log, r6/N5-margin1.log). Each fails the extended test, and
only it: never warn, threshold fixed at 2, notify always, notify
never, GUARDIAN_MARGIN = 1, no once-per-shape, one Map key for every
set, the n-of-n text for threshold + 1, no fixed line, a fixed line
for a set that was never short, the fixed line never alerting, no
program tag, every set named as the protocol's, the short flag not
stored, the program before the shape, the program left out of the
text, `need < 8` for `need <= 8` (6-of-7), the 8-key limit ignored
(7-of-8), 7-of-8 as the lowered advice.
- The mainnet alerts are checked on their own in a scratch copy of the
test without the earlier text assertions (r5/C1-C3-alone.log): with
the program left out of the text, the alert's detail check fails;
with a long name pushing the shape past character 60, "2-of-2 alerts"
fails.
- A's review of the previous version (branch
auto/B/floor-guardian-margin-r7, kept) found two survivors there: X3,
the shape keyed on the count only, and X5, the advice
threshold-of-(n+1). It also found that the n-of-n text said a lost key
alone locks governance. The test now has a threshold-only fix (7-of-8
-> 6-of-8, the advice's own step) and pins the n-of-n text and advice.
r9/mutants.log: X3, X5, the r7 n-of-n text and the text without its
lost-key clause are all KILLED. r9/mutants-r7.log: X3 and X5 SURVIVED
on the r7 code and test. The 18 mutants of r6/mutants.py, re-run on
this code: all KILLED (r9/r6-mutants-rerun.log). Two code-review
passes on the fix (kept as -r8 and -r9) added the lost-key clause: a
lost key alone leaves no quorum (it never approves, and core `quorum`
needs threshold approvals) until the owner's new set executes. A third
pass (-r10) fixed the comments' wording only.
- tools/claims.py: OK, client 180.
- Still not tested: the checkFloor call in main.mjs (no test runs
checkFloor).
Not changed (code review): the "fixed" line needs the short shape to
have been seen since the guardian started (the Map is in memory, as the
old per-process shape was; docs/07 says so). A protocol that abdicated
SetGuardians can't take a new set, so the advice to run 2-of-4 can't be
followed there; REPORT lists that case for the owner. The fault text
assumes a margin of 2, the owner's decision (16 §4.4).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Tie the JS guardian margin and set size limit to the core constants
cloud/lib/solana.mjs keeps its own GUARDIAN_MARGIN = 2, which the
guardian and the deploy tooling use to judge a guardian set, and now
MAX_GUARDIANS = 8, which the guardian's advice uses. Their comments say
they are core governance.rs's constants, but nothing checked that. Had
the programs' margin moved to 3, every JS test would have stayed green
while the guardian passed sets the programs refuse.
deploy.test.mjs now reads governance.rs and requires exactly one
`pub const GUARDIAN_MARGIN: u8 = N;` and one `pub const MAX_GUARDIANS:
usize = N;`, each N equal to the JS constant. It reads the source text,
like the build-script test above it. If a declaration changes form, the
test fails rather than passing.
Evidence (tasks/floor-guardian-margin/r6/tie-mutants.log):
- node --test cloud/test/*.test.mjs: 181/181 pass (one new test).
- Mutants of governance.rs, each reverted with its mtime kept (else the
limiter differential asks for a worker rebuild): GUARDIAN_MARGIN = 3,
MAX_GUARDIANS = 9, and a trailing comment on MAX_GUARDIANS. Each fails
the new test, naming the constant. deploy.test.mjs is green again.
- tools/claims.py: OK, client 181 (docs/09:40 updated).
Sibling search: the other JS copies of core rules (the limiter port,
the account layouts) are checked by the differential and by decoding
real accounts. MIN_GUARDIAN_THRESHOLD has no JS copy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
shuv-amp
added a commit
that referenced
this pull request
Oct 18, 2026
…ls by code (#104) * Run the tests on 2-of-4 guardian sets; test recovery from two faults Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set with fewer than threshold + 2 keys. Every test fixture and most tests' own sets were 2-of-3. This moves them to 2-of-4, which today's rule also accepts, so this commit changes no program: Floor, Vault and Harbor hash 508e5a99 / b07655d9 / 543be60b, as before. - Fixtures: Protected (Floor) and Market (Vault, through Harbor) now hold 4 guardians. Each test that built its own 3-key set gets a 4-key set with the same threshold and the same assertion. - Two tests changed meaning with a 4th key and were fixed, not loosened: a_quorum_ejects_a_rogue_guardian_who_vetoes_everything (Floor) and a_quorum_ejects_a_guardian_who_freezes_forever (Vault). With 4 keys, the last eject was refused only because the target's own vote is masked. Both now eject a second guardian (a set an ejection shrank to threshold + 1 still ejects) and then show the 2-of-2 refusal. - Account sweeps: an eject's signers may now be any two of the three others, so each of those two slots is free for the third, never for the target or the other signer again. Vault 5,365 and Floor 1,710 substitutions, refused (docs/09, dated 16 Oct). Listing all three others at each slot would have skipped the duplicate: with a mutant that counts a guardian listed twice as two (governance.rs:270), the sweeps listing all three pass, and these fail at "eject: account 2 replaced by key ..." (Vault and Floor). No other program test kills that mutant (tasks/aud2-guardian-margin-impl/dup-mutant-*.log). - floor_differential: the guardian pool is 6 keys, 4 registered; the proposed sets are 2 to 6 keys. New tests (pass on both rules; they show what the margin buys): - a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_set (Floor): g0's key lost, g1 rogue stops the program; the lost key does not help a quorum; the honest pair ejects g1 and restores; neither can veto the owner's refill, which executes after 72h. - two_rogues_in_turn_are_both_ejected_from_a_two_of_four_set (Floor): the second rogue vetoes the refill and is ejected from the 2-of-3. - a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_vault: the same in the Vault (freeze, eject, unfreeze, refill). Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest and quote sweep): the counts and the sweep figures here are from runs there (tasks/aud2-guardian-margin-impl/r4/). The E2E run, the mutants and the new-rule run below were made before, on e8640a7; this commit's programs and core are the same as there but for one comment line in views.rs (line-neutral, same hashes). Evidence: - E2E=1 scripts/check-programs.sh: PROGRAM CHECKS PASSED (Floor 49, Vault 98; S1 and E2E passed; the resumed setup found all 14 steps; vault check, Floor demo and deploy dry run passed), with the next commit's deploy rehearsal in the tree. - tools/claims.py: OK (floor 49, vault 104, programs 153). - Under the new rule (core from two commits on, programs rebuilt), only the two tests that state the old rule fail: an_n_of_n_guardian_set_is_ refused (accepts 3-of-4) and the Floor differential's model. - Mutants (tasks/coverage-map/mutants.sh), each red, then restored: AV1 Floor live_guardian ignores liveness: KILLED by a_lost_key_and_a_ rogue_do_not_lock_a_two_of_four_set (+2 others). AV2 the Vault's: KILLED by ..._two_of_four_vault (+2 others). AQ1 Floor restore quorum ignores liveness: KILLED by ..._two_of_four_set and the differential only. AQ2 the Vault's unfreeze quorum: KILLED by ..._two_of_four_vault and a_silent_guardian_does_not_count_toward_a_quorum. AE1 can_eject refuses at count == threshold + 1 (AUD-2 option (a)): KILLED by two_rogues_in_turn_..., and the two eject tests above. Hashes after: 508e5a99 / 543be60b / b07655d9. Split on 16 Oct from ec0cacc at the owner's review (D2): the deploy rehearsal's move to 4 guardians is now the next commit. programs/ and programs/tests/ here are byte-identical to ec0cacc, so its runs below hold for this commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Rehearse deploys with four guardians, and read only the keys there are Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set with fewer than threshold + 2 keys, so the deploy rehearsal must register 2-of-4 sets. Split from ec0cacc at the owner's review (D2); the text is that commit's, unchanged: Deploy rehearsal (cloud/deploy market.mjs, vault-check.mjs, floor-demo.mjs): 4 guardians (keys/guardian3), so the dry run keeps working once the rule lands; one more funding step (14, was 13). The devnet market has 3 guardian files. A first version of this change loaded guardian3 everywhere: vault-check.mjs failed there with ENOENT, and heartbeat silently created a key nobody holds as a guardian. Now market.mjs guardianKeys(dir, create) reads guardian0, 1, ... and throws on a gap (never fills it with a new key). Setup creates keys up to 4 only while the protocol is not registered, so resuming an old market adds no key the chain lacks; a new deploy-dryrun step resumes the dry-run market with guardian3 moved aside and fails if setup makes it again (with setup always creating: "DRY RUN FAILED: resuming a registered market made keys/guardian3.json", r3-dryrun-J7.log). renew-guardians tops up to 4 and logs each new key (unfunded); a 2-of-3 renewal would be refused once the rule lands. keysIn reads the guardians only when a command uses them, so caps and set-recovery never need them; heartbeat says a key is not in the set rather than dead. One THRESHOLD (2) for the register and renewal payloads and MIN_GUARDIANS. vault-check.mjs compares the chain's set with the first files, and liveness with the chain's count. An extra file (a renewal not executed yet) and a set without the margin are info lines, not failures, since such a set keeps working. GUARDIAN_MARGIN and guardianMarginProblem live in cloud/lib/solana.mjs, used by these and by the guardian bot's existing warning (cloud/guardian/main.mjs), and vault-check uses THRESHOLD. The e2e scripts fund guardian3 too. Three tests in cloud/test/deploy.test.mjs (client tests 176 -> 179). JS mutants (js-mutants-r3.log), each red then restored: always create, never create, no gap check, margin 1, `>`, GUARDIAN_MARGIN = 1. Setup always creating survives these tests and is killed by the dry run step above. cloud/, scripts/deploy-dryrun.sh and docs/09's client-test row are byte-identical to ec0cacc (git diff empty), so its runs hold for them: the E2E check-programs run with the dry run (r4), and the JS mutants. tools/claims.py: OK (client 179). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Refuse a guardian set without two keys beyond its threshold (107) AUD-2, decided by the owner on 16 Oct (option (e) of 16 §4.4). A set stays live while its honest live guardians reach the threshold: they eject every rogue in turn, and dead guardians cannot veto the owner's new set. Finding 96 asked for one key beyond the threshold, so a 2-of-3 or 3-of-4 set survived one fault but not a lost key plus a rogue: the honest guardian left is no quorum, nobody can be ejected, and the rogue keeps Floor stopped (or the Vault frozen) and vetoes every new set. Fix, in the core so both programs inherit it through GuardianSet::parse (registration, and every SetGuardians when proposed and when executed): GUARDIAN_MARGIN = 2 guardian_set_ok := count <= 8 && t >= 2 && count >= t + 2 (saturating_add: t is a payload byte). A stored set is not checked again, so existing 2-of-3 sets keep working, and ejection is unchanged: a shrunk set still ejects. A 2- or 3-key set is BadThreshold, not BadCount; the count rule is only the bitmap size, and the error order of finding 103 is kept. This changes both programs, by intent: Floor 508e5a99 -> 8ca8a608, Vault b07655d9 -> 7086148b, Harbor 543be60b (unchanged). Darwin-arm64 builds. Compute (scratch LiteSVM test, tasks/aud2-guardian-margin-impl/cu.md): -3 CU on every path that parses a set (Floor propose 2035 -> 2032, execute 1682 -> 1679; Vault register 9592 -> 9589, SetGuardians 2979 -> 2976; Floor register varies with its PDA bump, residue also -3). Proof and tests: - Kani: accepted_sets_can_eject_anyone restated (ejection reachable exactly when count > t, so option (a) cannot slip in; accepted sets have >= 4 keys and a margin of at least 2, and 2 is accepted). New: an_honest_live_quorum_ejects_any_rogue (any slots, anyone else alive) and accepted_sets_survive_any_two_faults (any two faulty keys, lost or rogue, leave a live honest quorum, which ejects each rogue, and the rogues alone eject no honest guardian). That holds only short of a rogue quorum: two rogues in a 2-of-n set are one (out of scope, 04 T7). With the assumption rogue < t dropped, the harness fails at `!can_eject(rogue, ..)` (kani-no-rogue-assume.log), so the docs say "short of a rogue quorum", not "any two faults". 19/19 verified, 240 s (cargo kani -j 2, as CI); again after the restatement, 235 s summed (r2-prove.log). - Kani mutants (kani-mutants.sh): margin 1, margin 3, `>` for `>=`, t >= 1, count <= 9, option (a), and an eject one vote short (only the two-fault harness kills that one): each KILLED by a harness (kani-mutants-r2.log). The eject ignoring liveness is the eject harness's case, not these. - Program mutants (coverage-map/mutants.sh, rebuilt each time): margin 1: KILLED by a_guardian_set_without_two_spare_keys_is_refused (Floor and Vault), registration_rejects_weak_settings and the Floor differential. Margin 3 and `>`: KILLED by most of the suite (the fixtures are 2-of-4). t >= 1: first only the Floor differential; the 1-of-2 registration case was now refused by the margin too, so it is a 1-of-4 here and a Vault 1-of-4 case was added: then KILLED by both. count <= 9: SURVIVES the program tests, equivalent there (parse refuses more than 8 keys first), KILLED by Kani. Option (a): KILLED by the eject tests two commits back (the tests on 2-of-4 sets). - Tests: the Floor and Vault refusal tests above cover proposal and execution (a stored 2-of-4 proposal whose threshold turns 3 is refused when executed, as the devnet renewals proposed before this rule will be); registration_rejects_weak_settings gains 2-of-3 and 3-of-4. an_n_of_n_guardian_set_is_refused (Vault) is renamed and accepted a 3-of-4: it now refuses it and accepts 3-of-5. - Model: the Python rule, the Floor differential's model, and the guardian_set fuzz target state it again; the GP generator now also picks count - 2. With the Python rule put back to the old one, seed 11 (20,000 cases) reports 5 GP mismatches; restored, seeds 11-13 identical. Fuzz seeds two_of_three_refused, two_of_four, six_of_eight, and seven_of_eight renamed seven_of_eight_refused (byte 323, the threshold, is 7: now refused). With margin 1 in the core the target crashes within 60 s; restored, 222,615,288 runs in 601 s, no crash. - scripts/check.sh: ALL CHECKS PASSED, and E2E=1 check-programs.sh: PROGRAM CHECKS PASSED, both at the branch tip, whose code is this commit's (the next commit is docs only; tasks/aud2-guardian-margin- impl/r4/check.log, r4/check-programs-e2e.log). claims: kani 19, floor 50, vault 104, programs 154, client 179. Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest and quote sweep). The core and the program sources are the same as before the rebase but for one comment line in views.rs (line-neutral), so the Kani, fuzz, differential and mutant runs above were made on the pre-rebase commit; the hashes, check.sh, check-programs and the counts are from r4 runs (tasks/aud2-guardian-margin-impl/r4/). The finding 96 row said the restated harness proves "exactly"; it now says what it proved then and proves now, and gives the Vault test's new name. The finding 107 row waits for rows 105 and 106 to land (09 §4 is numbered 1..n); its text is ready. The rule's prose is the next commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * State the guardian margin in the docs, with its costs (AUD-2) The previous commit made both programs refuse a new guardian set without two keys beyond its threshold (finding 107). The docs that state the rule now say so: - README: guardians 4-8 keys; the G5 row and the "who the guardians are" paragraph give the reason (any two faults leave a quorum, unless the rogues are one themselves: two in a 2-of-n set, 04 T7). - 05 register_protocol: 2 <= threshold <= count - 2, also for every SetGuardians, proposed and executed. 05 abdicate: abdicating SetGuardians fixes the set, so ejections shrink it for good. - 06 state table, 00 D22, FAQ: the same rule. - 07 §2: 4-8 guardians, and don't abdicate SetGuardians. 07 §6: the programs enforce the margin; sets registered before keep their exposure until replaced; an ejection uses up a spare key until the owner's new set executes; a lost key counts as live until its interval passes. - 16: the AUD-2 row is decided; §4.4 gives the decision, why the margin is the number of faults a set survives (short of a rogue quorum), the options rejected and why (the honest-owner and stolen-owner cases look the same on-chain), what it costs, what it leaves (a rogue quorum; a stolen owner key with every guardian dead; a rogue's veto of the recovery key's rotation, checked: Floor's veto has no recovery exception, as a_vetoed_recovery_proposal_does_not_hold_the_owner_s_slot shows), and Lido-style lapsing vetoes as a future option only. §5: "independent guardians, 3-of-5 or more". - 10: the guardian set's minimum size is no longer an open decision. SECURITY.md and 08 describe what is on devnet, which still runs the old rule, and are left as they are. claims: OK. Docs only; no program file changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Say the weekly CI run skips mutants, as its condition does ci.yml's header and scripts/ci-changes.sh said the weekly run does every job, and 09 §8 gave `mutants` as running on any change to the core. The `mutants` job's `if` starts with `github.event_name == 'pull_request'`, so the Monday schedule (and a manual dispatch) never runs it: it mutates a pull request's diff and has none to mutate. Its trigger list in 09 §8 also left out scripts/mutants.sh and .cargo/mutants.toml, which ci-changes.sh (line 22) selects it for. Comments and docs only; nothing runs differently. Sibling search: `git grep` for "weekly run does", "every job ... weekly" and "mutants ... weekly" across md/yml/sh/mjs/py. The other hits are 09 §8's "at full depth every Monday" (its table now says which job is not weekly) and the dispatch input's "every job at weekly depth", which is the same set as the weekly run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Check that the mutant exclusions still match; repin murmur3 (finding 105) .cargo/mutants.toml excludes the core's 22 equivalent mutants, and the docs say so. Since 9 Oct it excluded 19. The murmur3 tail's pattern is pinned by line, `floor.rs:(235|238|241)`, because the body's `h ^= k` has the same column and function and is not equivalent. 1dc7861 (#52) moved those lines (now 327/330/333), and nothing checked the patterns. Failing closed, the three would have failed CI's `mutants` job on a pull request that changed those lines. Failing open, a pattern that starts matching a non-equivalent mutant hides it from every run. Root cause: nothing checked that a pattern still names its mutants. Fix: - each pattern ends with the number of mutants it excludes (`# 3`), and the murmur3 pattern names 327/330/333; - tools/mutant_exclusions.py fails unless each pattern matches exactly its number in `cargo mutants --list --no-config`, and cargo-mutants (the list with the config) excludes exactly the mutants the patterns match, so a pattern this parser can't see, or reads differently, fails too. No two patterns may match the same mutant, since the docs' count is the counts' sum, and the kept list must be part of the full one. It reads only the exclude_re array's body, needs no tomllib (Python 3.9), and names colour codes in the list as the problem; - scripts/mutants.sh makes both lists (--colors never: CI sets CARGO_TERM_COLOR=always) and runs it first, except when the caller passes --config or --no-config (as whole arguments). CI's `mutants` job runs that script; ci-changes.sh now also selects the job for the new tool; - scripts/check.sh runs its --self-test on planted configs; - tools/claims.py ties the docs' "22 equivalents" (README, 09 §6 and the summary row) to the sum of the counts, with the same parser. It finds that parser from its own directory, so `python3 -I` runs it too, and check.sh runs its self-test that way. 00's "18 equivalents" (26 Sep log entry) and run 9's "exactly the 22 equivalents below" (6 Oct) are kept as dated records. Docs: 09 §4 finding 105, §6 run 10 and the "In CI" note, §8's mutants row; README and 09's summary row 669 → 797 mutants (run 10); README and 12's post-submission range 96–105. Evidence (first on 1f2ad48 + the parent, logs in the autopilot's tasks/mutants-exclusions-r2/; re-run on 58697c6 + the parent, tasks/mutants-exclusions-r3/): - `cargo mutants --list`: 797 with --no-config, 778 with the old config (19 excluded), 775 with this one. The 22 excluded are §6's list, the murmur3 ones at 327/330/333. - scripts/mutants.sh --list: exit 0, "22 of 797 mutants excluded"; with CARGO_TERM_COLOR=always also exit 0. Old config: exit 1, every pattern "no '# N' count". Planted on the real lists: the old lines with a count (matches 0, says 3) and the body XOR added (matches 4, says 3) both exit 1; in both the set comparison alone passes. /usr/bin/python3 (3.9.6): self-test OK and the real check passes. - The self-test pins each problem by its message, not a count. Mutants of the tool (tool-mutants.log), each red: M1 no per-pattern check, M2 no set comparison, M3 no colour check, M4 no empty-array check, M5 a missing count accepted, M6 a missing count read as 0, M7 the parser reading past the array. M3 and M5 survived a first version that checked only the number of problems. After A's review (16 Oct): the set comparison was pinned only by its size, so `len(excluded) != len(matched)` passed the self-test. A case with as many excluded as matched but not the same ones now fails it (15 cases; red with that mutant, green without), and the message says how many mutants are in one set only. From the code review after the rebase: colour codes only in the kept list were reported as a kept list made with other options; the colour check now reads both lists (16 cases; dropping the kept half fails the new case). On 58697c6, with the overlap and kept-list checks: M8 no overlap check and M9 no kept-list check, red too, as M1–M7 still are. A duplicate of the can_eject pattern planted in the real config: scripts/mutants.sh --list exits 1, "the counts add up to 23, the patterns match 22 mutants" (planted-overlap.log). Without the sys.path line, `python3 -I tools/claims.py` fails with ModuleNotFoundError; with it, both claims runs pass. - claims.py (claims-mutants.log): one count 1 → 2 fails four statements (README 188; 09 33, 333, 335; kept-run9.log), not run 9's 331, which failed too before it was kept; a dropped count fails with the tool named; deleting any of the three rules fails a self-test case; deleting the KEPT entry fails 00:103. - The 22 excluded mutants run alone (--no-config, --re per pattern, 2 min): 22 missed, exactly the excluded list; --re also picked ema.rs:70 (a field deletion, no pattern matches it), caught. - Run 10 (f1f05ba, old config, -j4, 46 min; tasks/mutants-run10/): 778 tested, 712 caught, 5 timeouts, 58 unviable, 3 missed, exactly the murmur3 tail. The core's source is unchanged since (`git diff --stat f1f05ba HEAD -- crates/bulkhead-core/src` is empty; only tests/units.rs gained finding 102's tests). - No program or core source changed. Rebased on 16 Oct onto cede705 (findings 102 and 100, the attest sweep and AUD-2). AUD-2 changed the core's source (governance.rs), so the line above saying it is unchanged since run 10 held on 58697c6, not here. On cede705: scripts/mutants.sh's list check passes, "22 of 795 mutants excluded, each pattern matching its count"; the murmur3 lines are still 327/330/333. The 797 in README and 09 is run 10's figure: README now dates it ("in the last full run, 15 Oct") and finding 105's row says the core has changed since (finding 107). The setup line in scripts/mutants.sh now pins cargo-mutants 27.1.0, the version ci.yml installs: each count is the number of mutants that version lists, so another release can break the check (code review, 16 Oct; `git grep 'install.*cargo-mutants'` finds no other unpinned install). The check's report goes to stderr in scripts/mutants.sh, so `scripts/mutants.sh --list` prints only cargo-mutants' names: before, its first stdout line was the report; after, stdout is 773 lines (795 - 22) and stderr the one report line. ci.yml's `mutants` comment says the job checks the exclusions first. Row 105 and 09 §6 gave the pin's reason as the body's XORs. The finalizer's are at column 7 (`grep -n '\^=' floor.rs`, 337-342); the non-equivalent XOR at column 11 is `h ^= k`, in the body (320) and after the tail (335). The toml comment, the row and §6 now say that. 09 §8's `check` row names the two tool self-tests check.sh runs. README no longer calls run 10's 797 "injected bugs": cargo-mutants listed 797 but tested 778, 19 being excluded (09 §6, run 10's row). Sibling search: the other 14 patterns use `\d+` for the line, and each matches the count now written after it. The murmur3 one is the only line-pinned pattern. Not done, by choice: - Catching an edit that keeps a pattern's number but changes its mutants (an excluded line itself edited, or a murmur3 body XOR moved onto a pinned line). The toml and 09 §6 say so: such an edit needs the mutant judged again by hand. Failing on any diff that touches an excluded line would catch the first; it is a separate change. - Moving murmur3's tail into its own function, so its pattern needn't name lines. It shifts core lines, so it changes the program hashes. Until then a line move above it fails the check, with a message saying what to fix. - A weekly full mutants run in CI. It took 46 min here at -j4, likely past the job's 90 min on two CI cores: a minutes call for the owner. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test that a ticket stays in its own protocol and vault Two Vault checks that keep a ticket to its own protocol and vault had no test. Deleting either one left every program test green (tasks/error-codes/mutants.log, base cede705): - brakes.rs:79, extend_ticket: tk.protocol != protocol. Without it, a live guardian of any other protocol extends this protocol's tickets. Anyone can register a protocol and name its guardians, so that guardian could hold back every queued payout up to the total extension limit. The sweep did not catch it: it has no second protocol's guardian to swap in. - custody.rs:501, settle_ticket: tk.vault != vault. A USDC ticket settled through the SOL vault (its tokens, its mint) got past the Vault's checks. New test another_protocols_guardian_cannot_extend_a_ticket: a second Market's live guardian extends this market's ticket -> WRONG_VAULT; the same instruction from this market's guardian succeeds. The settle test now settles the USDC ticket through the SOL vault and expects WRONG_VAULT. Evidence (tasks/error-codes/mutants.log, STATE=c1; full program suite per mutant): W02 brakes.rs:79 deleted -> KILLED by another_protocols_guardian_cannot_extend_a_ticket ("expected failure, got success": the stranger's extension went through) W06 custody.rs:501 deleted -> KILLED by a_ticket_settles_after_its_eta_to_its_stored_recipient_only (8049 InsufficientBalance from the empty SOL vault, not 8010) Both were SURVIVED at base. No program source change: hashes 8ca8a608/543be60b/7086148b, as at cede705. Sibling search: the other 12 WrongVault sites were deleted one at a time in the same run. Eight are KILLED by the sweep (brakes.rs:39, lib.rs:320, custody.rs:160, genesis.rs:491, views.rs:27, lifeboat.rs:151, governance.rs:169/259). lib.rs:324 (vault PDA address), genesis.rs:523, lifeboat.rs:71 and :247 survive; later commits on this branch deal with them. Counts: Vault 104 -> 105, programs 154 -> 155 (claims.py OK). The test also reads the ticket's ETA: unchanged after the refused extension, 3,600 s later after its own guardian's (code-review fix: success alone did not show the extension happened). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test that the seal names every vault and happens once Two seal checks had no test; deleting either left every program test green (tasks/error-codes/mutants.log, base cede705): - genesis.rs:486, execute_seal: the pairs must number 3 * vault_count. Without it, the seal checks only the vaults it is given. A vault left out skips its claims-vs-balance check, so an uncovered import seals. docs/05's seal row says every vault is passed exactly once. the_seal_checks_each_vault_once covered a vault named twice in place of another, not one left out. - genesis.rs:431, propose_seal: refused once sealed (AlreadySealed). Without it, a sealed protocol's admin re-proposes, which rewrites seal_executable_s/expires_s and clears seal_vetoed on a sealed protocol. the_seal_checks_each_vault_once now also leaves the uncovered USDC vault out (expects VAULT_COUNT_MISMATCH) and, after the seal, proposes it again (expects ALREADY_SEALED). No new test, so no count change. Evidence (tasks/error-codes/mutants.log, STATE=c2, full program suite): E05 genesis.rs:486 deleted -> KILLED by the_seal_checks_each_vault_once ("expected failure, got success": the seal went through without the USDC vault and its 1,000 USDC uncovered import) E04 genesis.rs:431 deleted -> KILLED by the_seal_checks_each_vault_once ("expected failure, got success") Both were SURVIVED at base. Vault hash 7086148b unchanged (tests only). Sibling search: the seal's other checks. :491 vault_tokens (KILLED by the sweep), :519 lifeboat owner and :523 lifeboat mint/owner (SURVIVED; the lifeboat is a PDA checked by expect_pda just above, which only this program can create, as its own token account: see this branch's note in docs/09). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test that invalid limits and asset settings are refused at every entry Three bound checks had no test; deleting any one left every program test green (tasks/error-codes/mutants.log, base cede705): - genesis.rs:171, create_vault: config_is_valid (bps <= 100%, windows > 0). Without it, a vault is created with a 100.01%-an-hour burst. - governance.rs:40, parse of a SetLimits change (at proposal and again at execution, :254 and :478): the same rule for a proposed change. - governance.rs:47, parse of an UpsertAsset change: valid_asset (haircut <= 100%, liability >= 100%, the oracle bounds of finding 88). Only the genesis upsert's copy (genesis.rs:296) was tested, by oracle_settings_of_a_priced_asset_are_bounded. New test invalid_limits_and_assets_are_refused_wherever_they_enter: before the seal, create_vault with short_bps 10,001 -> INVALID_LIMITS; after it, a SetLimits proposal with long_window_s 0 -> INVALID_LIMITS, and an UpsertAsset proposal with a 10,001 bps haircut -> INVALID_ASSET. Evidence (tasks/error-codes/mutants.log, STATE=c3, full program suite): E06 genesis.rs:171 deleted -> KILLED ("expected failure, got success") E07 governance.rs:40 deleted -> KILLED ("expected failure, got success") E08 governance.rs:47 deleted -> KILLED ("expected failure, got success") each by the new test only. All three were SURVIVED at base. Vault hash 7086148b unchanged (tests only). Sibling search: every config_is_valid / valid_asset call in programs/ vault/src: genesis.rs:170 (create_vault), :295 (upsert_asset, KILLED by the oracle test), governance.rs:39 and :46. All now killed. Counts: Vault 105 -> 106, programs 155 -> 156 (claims.py OK). The execution copy (:478) is not reached by any transaction: it parses the bytes the proposal already checked, and the bounds read nothing else. The test comment says so (code-review fix: it had claimed execution was covered). oracle_settings_of_a_priced_asset_are_ bounded now asserts INVALID_ASSET for each of its four cases, not any error (all four return it; E09 stays KILLED). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test that an import into a ninth mint is refused genesis.rs:410, import_backing: a backing ledger holds MAX_SLOTS (8) mints, and an import into a ninth is refused with NoSlot. Deleting the check left every program test green (tasks/error-codes/mutants.log, base cede705). From the code, without it the import goes through: the vault's imported total grows (:414), and the seal must cover that, but the owner's ledger is saved unchanged, so the owner has no backed claim for it. New test an_import_into_a_ninth_mint_is_refused: nine vaults before the seal; one owner gets an import into eight (ledger_of shows 8 positions), then the ninth -> NO_SLOT. Evidence (tasks/error-codes/mutants.log, STATE=c4, full program suite): E14 genesis.rs:410 deleted -> KILLED by an_import_into_a_ninth_mint_is_ refused ("expected failure, got success"); SURVIVED at base. Vault hash 7086148b unchanged (tests only). Sibling search: the only other ledger credit is deposit's (custody.rs:128). There a full ledger is by design: the deposit is taken unbacked (Ledger::credit's doc, ledger.rs:187-189), and credit's result is the backed flag, not an error. Counts: Vault 106 -> 107, programs 156 -> 157 (claims.py OK). import_ix now calls import_into_ix, which takes the vault, so the new test shares the one import layout (code-review fix). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pin six Vault refusals no test reached Six reachable checks had no test that failed when the check was deleted (tasks/error-codes/mutants.log, base cede705). Each now has an assertion on its code, and each was seen to fail with the check deleted: E02 genesis.rs:185 create_vault of an existing vault after the seal: ALREADY_INITIALIZED (with the check gone: 8043 GenesisOnly, so the "says so first" order in the comment above it was untested) E12 governance.rs:426 veto of an executed change: CHANGE_EXECUTED (gone: the veto succeeds and marks an executed change vetoed) E13 governance.rs:517 cancel of an executed change: CHANGE_EXECUTED (gone: the cancel succeeds) E16 lifeboat.rs:256 execute_exit with no exit proposed: NOTHING_TO_EXIT (gone: 8033 ChangeNotExecutable) E17 custody.rs:84 a zero deposit: ZERO_AMOUNT (gone: it succeeds) E18 custody.rs:272 a zero release: ZERO_AMOUNT (gone: it succeeds; a zero release has no unbacked part, and the limiter, whose own zero check custody.rs:242 maps, is called only when unbacked > 0, :361) Where: E02 in after_the_seal_a_new_vault_starts_with_its_unbacked_lane_ closed, E12/E13 in an_unvetoed_loosening_executes_after_the_delay, E16 in the_lifeboat_stays_shut_until_the_seal (after the move in, before an exit is proposed), E17/E18 in the new a_zero_deposit_or_release_is_ refused. Evidence (tasks/error-codes/mutants.log, STATE=c5, full program suite): all six KILLED by the tests named above; all six SURVIVED at base. The vault suite is green (108 passed). Vault hash 7086148b unchanged. Counts: Vault 107 -> 108, programs 157 -> 158 (claims.py OK). Helpers (code-review fixes): veto_change_ix beside cancel_change_ix; create_vault_for_ix, create_vault_ix with a given mint and token program, so E02's case names USDC's own accounts and creates no mint. E02's comment now repeats the program's ("whatever the seal's state"): create_vault checks the limits first (genesis.rs:170). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pin four Vault checks another layer backstops Four checks survived deletion (tasks/error-codes/mutants.log, base cede705) because something later still refused the case, with its own error, so no test could tell. Each now has a test that sees the Vault's own code, and each was seen to fail with the check deleted: E01 lib.rs:230, create_pda's guard: a second registration of an existing protocol (register_protocol has no check of its own) -> ALREADY_INITIALIZED. Gone: the system program refuses it, with no custom code (custom_code 0). W10 lifeboat.rs:71, move_in naming another mint or token program -> WRONG_VAULT. Gone: the token program refuses (code 3, MintMismatch). W12 lifeboat.rs:247, execute_exit naming another mint or token program -> WRONG_VAULT. Gone: the same token-program error. T01 token.rs:37, account_info on an account no token program owns: a copy of a real USDC account's bytes under the system program. Gone: the quote SUCCEEDS (it promises a payout), and the release fails only in the runtime at the transfer ("instruction modified data of an account it does not own"). The quote is asserted first so the mutant is seen at the quote (STATE=c6b). New tests: a_protocol_registers_once, a_lifeboat_move_names_the_vaults_ own_mint (also completes a move in and an exit, as a control: the destination receives 1,000 USDC), a_payout_goes_only_to_an_account_a_ token_program_owns. Evidence (tasks/error-codes/mutants.log, STATE=c6 and c6b, full program suite): E01, W10, W12, T01 KILLED by the tests named above; all four SURVIVED at base. Full program suite green: Floor 50, Vault 111. Vault hash 7086148b unchanged (tests only). Counts: Vault 108 -> 111, programs 158 -> 161 (claims.py OK). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test the bounds on ticket extensions and a closed lane's code B's review of vault-error-tests-r3 (R1) found two Vault codes no test asserted by value that the earlier commits missed: ExtensionTooLong and LaneClosed. Deleting the 7-day total extension cap (brakes.rs:86, B24) left every program test green, so a live guardian, or a stolen guardian key, could hold a queued payout back without end, 24h at a time, against docs/05 row 11. The wide differential models the total (differential_wide.rs:406), but its default run never extends one ticket that far. Mapping LaneClosed to QueueFull (custody.rs:240, B23) also survived. The code review of the fix found a worse gap at the same check: no test sends a negative extension (the differential draws secs from 0, 1, 3600, 86400, 86401). With `secs <= 0` weakened to `secs == 0` (A3, brakes.rs:84) every program test passed, and one guardian key could pull a queued payout's ETA into the past, skipping the queue delay, and refund its own 7-day budget. - a_ticket_is_extended_later_by_at_most_a_day_a_call_and_seven_in_total: -1h, 0 and 24h + 1 s are each refused with EXTENSION_TOO_LONG, ETA unchanged; six 24h extensions and one of 23h pass; 2h more is refused, ETA unchanged; the last hour is granted; then 1 s more is refused. The split is unaligned so that a total that ignores this call's `secs` fails too. - a_closed_lane_refuses_an_unbacked_payout: a vault with short_bps 0, and one with long_bps 0, each refuse an unbacked admin payout with LANE_CLOSED; the same payout from a vault with both buckets open is taken (the control, so a capacity base of 0 cannot pass it). Evidence (tasks/vault-error-tests-r4/, B's runner copied: each mutant applied alone, programs rebuilt, whole program suite run): - without these tests (551e519; run-without.out, run-base2.out, run-base3.out): B23, B24, A1 (`tk.extended_s >= MAX`, this call's secs ignored), A2 (`>=` for `>`) and A3 SURVIVED. A4 (the 24h per call deleted), A5 and A6 (either half of limiter.rs:172's closed-lane test deleted) were KILLED by the differential only, not by value; - with them (run-final.out, at the branch tip): B23 KILLED by a_closed_lane_refuses_an_unbacked_payout; B24 A1 A2 A3 KILLED by the extension test only; A4 by it and the differential; A5 A6 by the closed-lane test and the differential. No program change: the runner's closing build gives 8ca8a608 / 543be60b / 7086148b, as at cede705. Counts: Vault 111 -> 113, programs 161 -> 163 (claims.py OK). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pin each half of create_pda's existing-account guard on its own B's review of vault-error-tests-r3 (R2): each half of lib.rs:229 (`!is_data_empty() || !owned_by(SYSTEM)`) survived deleted on its own (B20, B21), since an existing protocol account fails both, so a_protocol_registers_once pinned them only jointly. The test now also plants, at a fresh protocol address, an account that fails just one half: one with data owned by the system program, one with no data owned by another program (a fresh address). Each registration is refused with ALREADY_INITIALIZED. The comment says no transaction is known to make either account: data or a new owner at a PDA needs its signature, which only the Vault gives, and create_pda gives both in one instruction. Evidence (tasks/vault-error-tests-r4/): at 551e519 B20 and B21 SURVIVED (run-base2.out); at the branch tip both are KILLED by a_protocol_registers_once only (run-final.out). No count change (claims.py OK: vault_programs 113, programs 163). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Record the Vault refusal deletions and what each became docs/09 §3 gains a row for the deletion run this branch answers (tasks/error-codes/mutants.log): 38 refusals, across 14 of the 18 Vault codes no test asserted by value at cede705, each deleted on its own with the whole program suite run against it, on the AUD-2 build. The 18 (each VaultError variant with no `verr::` use in programs/tests at cede705): AlreadyInitialized NotRegistrar WrongVault WrongTokenProgram AlreadySealed VaultCountMismatch InvalidLimits InvalidAsset LaneClosed TicketNotPending ExtensionTooLong ChangeExecuted LedgerUnsound LedgerCorrupt NoSlot NothingToExit ZeroAmount Overflow. The run missed LaneClosed and ExtensionTooLong; B's review deleted one check of each (B23 custody.rs:240, B24 brakes.rs:86), and both survived until 1797ae5. That commit also pins a negative extension (A3, which survived every test before it), the 24h per call, and a lane closed by either bucket. At base: 13 KILLED, 25 SURVIVED. The commits before this one add tests that kill 18 of the 25 and both of B's, each asserting the Vault's own code (W02 W06, E04 E05, E06 E07 E08, E14, E02 E12 E13 E16 E17 E18, E01 W10 W12 T01; B23 B24). The row names the 7 that still survive and why: - six guards no transaction is known to reach: lib.rs:324 (vault PDA address; create_vault creates a vault only at its PDA), genesis.rs:519 and :523 (the lifeboat, checked by expect_pda first, created only by lifeboat_move, move_in requires the seal, lifeboat.rs:53, and the seal happens once, so no lifeboat holds data at it), brakes.rs:50 and custody.rs:541 (a pending-ticket count below zero), token.rs:235 (GetAccountDataSize's return data from an already-checked program); - custody.rs:399, stated as not shown unreachable: properties.rs:588-592 debits backed_amount and one more on random ledgers, and only crates/bulkhead-core/tests/units.rs:375 debits less, on a one-mint ledger with no liability. properties.rs:606 asserts backed(a) == min(a, backed(MAX)) for requests below 2^52 (:613), so any smaller amount is backed too, and debit_backed(b) refuses only when backed_amount(b) != b (ledger.rs:289). The known exception is a collateral value that overflows u128 (ledger.rs:262): the valid amounts need not form an interval there, so the consent or freeze cap (custody.rs:337, :347) can lower the amount to an invalid one and :399 refuses the release (fail closed). The search's own result is always valid. Not verified by a tool beyond that property test. (A full ledger cannot reach :399 through set's NoSlot: backed_amount's valid() refuses a new liability slot on a full ledger, ledger.rs:237.) Not deleted, as the row says: the two core-error mappings to ChangeExecuted and ZeroAmount, token.rs:233, and the codes Overflow and LedgerCorrupt (checked arithmetic, decoding). After this branch five codes are still asserted by no test (same grep at this commit): NotRegistrar (E03, genesis.rs:108, was KILLED at base by another test), TicketNotPending, LedgerUnsound, LedgerCorrupt, Overflow. Docs only: no count change (claims.py OK: vault_programs 113, programs 163). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Run the exclusion check's main in its self-test A's review of finding 105 (REVIEWS/auto-B-mutants-exclusions-r7.md, finding 1, Low): CI's `mutants` job acts on main's exit status, but the self-test called problems() only, and check.sh never runs the tool on real lists. With main returning 0 on a mismatch (mutant A10), every in-tree check passed, and CI would have printed the problems and gone green. This was not a defect at fb771c6, since the job fails today. The gap was that a later regression would pass. Fix: main takes its arguments and, for the self-test, a config path (default: the repo's .cargo/mutants.toml, as before), and its lines name the config it read. The self-test runs main in-process on a planted config and checks the exit status and every line on each stream: - the right lists: exit 0, "1 of 3 mutants excluded" on stdout, nothing on stderr; - the lists the other way round: exit 1, nothing on stdout, and exactly the count and "full list lacks" problems, so KEPT is read as KEPT; - coloured lists: exit 1 with exactly one problem, so a main that fails only on several problems is caught; - one argument and three: exit 2 and the two usage lines. One subprocess (python3 -I, 60 s timeout, in the temporary directory so the config must be found from the tool's own path) runs the command line with two lists and the repo's config, as scripts/mutants.sh does: on a list no pattern can match, it exits 1 with an empty stdout and problem lines that each name .cargo/mutants.toml. Which problems those are depends on the config, so only their form is checked. Self-test: 16 cases -> 22. No doc states the number. Evidence (tasks/exclusions-main-exit/): - old-A10.log: the fb771c6 self-test with A10 applied prints "self-test OK: 16 cases" and exits 0. - mutants-r6.log (mutants.py applies each mutant to a scratch copy of the tool, which reads the real config through a symlink): unmutated, "self-test OK: 22 cases", exit 0. All 18 are KILLED (exit 1): A10 (return 0), A10b (fail only on more than one problem), A0 (always 1), U1 (usage returns 0), U2/U3 (argument check `< 2`, `< 1`), K1 (KEPT ignored), K2 (ALL read twice), C1 (planted config ignored), C2 (no default config), R1 (the default config read from the cwd), E1/E2 (sys.argv sliced as [:] or [1:2]), P1 (problems not printed), S1 (problems on stdout), L1 (the planted config not named), N1 (only the first problem printed), G1 (the success line printed on failure too). Code-review passes at high shaped this. The first version ran three subprocesses against the real config, and A0, K1 and U3 survived it. The second left E2, C2 and G1 alive. Declined from the third: catching exceptions in the self-test (a traceback already exits 1) and moving the config hook out of main. The fourth showed that requiring one "matches 0 mutants" line per live pattern tied the self-test to the config's contents. The subprocess now checks the form of the lines only, and the 17 stay killed. The fifth showed that R1 survived while the child ran in the repo root (it does in check.sh); now it runs in the temporary directory. Its other points repeat declined ones. - real-lists.log: on the real lists, made with scripts/mutants.sh's commands: ".cargo/mutants.toml: 22 of 795 mutants excluded, each pattern matching its count", exit 0. - python3 -I tools/mutant_exclusions.py --self-test: 22 cases OK. - scripts/check.sh: ALL CHECKS PASSED (check-final.log); bin/check-each-commit.sh green (each-commit.log). Sibling search: the other tools check.sh runs (tools/claims.py, tools/floor_check.py) also gate CI through main's exit status. claims.py has the same gap: check.sh runs it on the real tree, where it finds no problems, and its self-test calls self_test(), not the path in main that returns 1. With that return changed to 0, `--self-test` and the real run both still exit 0 (claims-C1.log). Not changed here: it is its own commit (BACKLOG-B). floor_check.py runs in CI only as --self-test, whose result main returns as is. Its RPC mode is an operator's tool that no CI job runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Record finding 106: Vault liveness and exit rules had no failing test The tests landed in 944f5e8 (lifeboat-exit-tests-r3, 15 Oct). Their 09 §4 row waited for 104 and 105 so the list stays numbered without gaps; 105 is now on integration (fb771c6), so the row lands here. Docs only: 09 §4 row 106; README and 12's count and post-submission range (96–106, 106 fixed). Two code-review passes changed the row as written on 15 Oct: a sentence for today's re-run, the nine single-guardian sites named, G5 quoted verbatim, and the exit veto attributed to T18 (G6 has no veto), as is the veto test's doc comment (vault.rs:3996, comment only). Evidence, re-run today on bf8ba41 (the row's claims are from 15 Oct, and AUD-2 changed the Vault since): B's re-review list of 33 line- neutral mutants (the ten rules, the nine single-guardian sites L1-L9, the boundaries, the six quorums, R1-R6), each applied, programs rebuilt, all program tests run: 33 KILLED, 0 survived, 0 not applied, 0 build failures. Vault hashes after the run match the tip (Vault 7086148b, Floor 8ca8a608, Harbor 543be60b). Log: autopilot tasks/finding-106-row/mutants.log. - python3 -I tools/claims.py: OK, findings 106. - git grep for "96–105", "105 findings": no other stale range. - scripts/mutants.sh still mutates the core only (SCOPE, line 11), so the row's last sentence holds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Cite PR #100's tree, not the unpublished bf8ba41, in finding 106 Finding 106's row in docs/09 said its 33 mutants were re-run on 16 Oct at bf8ba41. That commit is not an ancestor of this branch or of the stack above it and is on no remote branch, so the citation would point at nothing once the stack lands. The owner's review of #99-#109 found it. bf8ba41 and 86b777b (the head of PR #100, vault-error-tests-d2, the D2 replay of vault-error-tests-r4) have the same tree, 36187b49 (`git rev-parse bf8ba41^{tree} 86b777b^{tree}` prints it twice). So the run was a run on 86b777b's tree, and the row now says "on tree 36187b49, the tree of 86b777b (PR #100's head)". Why not just "at 86b777b": it was committed on 17 Oct, after the run; and main squash-merges (1045cd9 has one parent), so after PR #100 lands 86b777b is reachable only as GitHub's refs/pull/100/head. The tree id stays checkable from main: squashed bottom first, main's commit for PR #100 has that tree. "PR" keeps "#100" from reading as finding 100, the row above. These points came from two code-review passes. Sibling search: every 7-12 hex token in tracked .md/.py/.sh/.mjs/.rs/ .toml files at autopilot/integration 3e9f58d that names a local commit (51 tokens) was checked with git merge-base --is-ancestor against integration. bf8ba41 was the only one that is not an ancestor. Docs only. tools/claims.py: claims OK (26 files). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Warn about a Floor guardian set short of the margin (finding 107) The guardian checks the Vault protocol's guardian set against finding 107's margin (main.mjs checkGuardianSet), but never a Floor config's. A Floor set registered before the rule, say 2-of-3, keeps working and keeps AUD-2's exposure: one lost key and one rogue guardian lock its governance for good. The guardian said nothing, and docs/07 §6 said so ("not yet a Floor set"). The rule moves to policy.mjs as guardianSetWarning, a pure function: the same text as before, a warn line that alerts on mainnet only. In main.mjs, checkGuardianSet now remembers the set shape per set ("protocol", or the protected program), and checkFloor calls it for each watched Floor config it reads. floorAlerts is unchanged. The margin is not a readiness code there: "floor not ready" alerts on every cluster, and only the owner's new set clears a margin problem, so "floor ready" would never be logged again. Evidence (tasks/floor-guardian-margin/r2/): - node --test cloud/test/*.test.mjs: 180/180 pass (one new test). - Mutants of guardianSetWarning, each run on policy.test.mjs and reverted; each fails the new test: N1 never warn, N2 threshold fixed at 2, N3 notify always, N4 notify never, N5 GUARDIAN_MARGIN = 1. - tools/claims.py: OK, client 180 (docs/09:40 updated). - Not tested: the main.mjs wiring (the per-set Map and the checkFloor call). No test runs checkFloor; deleting the call leaves every test green. Sibling search: guardianMarginProblem's callers are vault-check.mjs (an info line) and guardianSetWarning. The status snapshot (snapshot.mjs:33) prints live guardians and the quorum, not the margin; left as it is. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Judge each guardian set's margin on its own, and say when it is fixed The parent's per-set memory (which set shapes were already reported) lived in main.mjs, so no test could fail on it: dropping the once-per-shape check, or keying every set the same, left every test green. The warning did not say which set it was about (only a program tag on Floor lines), and it gave the same text for a set ejected down to its threshold, or registered n-of-n before finding 96. There one rogue guardian alone locks governance; a lost key alone does not, since dead guardians cannot veto the owner's new set (docs/16 §4.4), but it leaves no quorum until that set executes. Nothing was said when the owner's new set fixed the problem. And a delivered alert did not say which Floor program it was about: the program went only into the line's `program` field, which alertFor uses in its repeat key but does not show. Moving the program name into the text is a trap: alertFor keys repeats on the first 60 characters of the text, and "the Floor guardians of <44-character id>" pushes the shape past them. A 2-of-3 set ejected to 2-of-2 within ten minutes would then give the same key, and the worse warning would not alert. So the program goes after the shape: "the Floor guardians are 2-of-3 for <program>: ...". The `program` field stays for the repeat key. guardianSetWarning becomes guardianSetCheck(shapes, which, set, mainnet) in policy.mjs. It keeps each set's last shape and whether it was short in the Map, keyed by "protocol" or the protected program, and returns the line to log or null. The warning names the set and has its own text for n-of-n. When a set that was short gets the margin, it returns one line saying so, which alerts on mainnet like the warning, so whoever was paged hears that it is fixed. main.mjs only logs what it returns. docs/07 §6 says so. The advice also had a hole: a set registered as 7-of-8 was told to "run at least 7-of-9", and no set can have 9 keys (core MAX_GUARDIANS is 8). When threshold + 2 is over 8, the advice is now to lower the threshold to at most 6-of-8. That needs MAX_GUARDIANS in JS, so lib/solana.mjs gets it next to GUARDIAN_MARGIN; the next commit ties both to the core. Evidence (tasks/floor-guardian-margin/): - node --test cloud/test/*.test.mjs: 180/180 pass (the test is extended, not added). - 19 mutants, each run on policy.test.mjs and reverted (r6/mutants.py, r6/mutants.log, r6/N5-margin1.log). Each fails the extended test, and only it: never warn, threshold fixed at 2, notify always, notify never, GUARDIAN_MARGIN = 1, no once-per-shape, one Map key for every set, the n-of-n text for threshold + 1, no fixed line, a fixed line for a set that was never short, the fixed line never alerting, no program tag, every set named as the protocol's, the short flag not stored, the program before the shape, the program left out of the text, `need < 8` for `need <= 8` (6-of-7), the 8-key limit ignored (7-of-8), 7-of-8 as the lowered advice. - The mainnet alerts are checked on their own in a scratch copy of the test without the earlier text assertions (r5/C1-C3-alone.log): with the program left out of the text, the alert's detail check fails; with a long name pushing the shape past character 60, "2-of-2 alerts" fails. - A's review of the previous version (branch auto/B/floor-guardian-margin-r7, kept) found two survivors there: X3, the shape keyed on the count only, and X5, the advice threshold-of-(n+1). It also found that the n-of-n text said a lost key alone locks governance. The test now has a threshold-only fix (7-of-8 -> 6-of-8, the advice's own step) and pins the n-of-n text and advice. r9/mutants.log: X3, X5, the r7 n-of-n text and the text without its lost-key clause are all KILLED. r9/mutants-r7.log: X3 and X5 SURVIVED on the r7 code and test. The 18 mutants of r6/mutants.py, re-run on this code: all KILLED (r9/r6-mutants-rerun.log). Two code-review passes on the fix (kept as -r8 and -r9) added the lost-key clause: a lost key alone leaves no quorum (it never approves, and core `quorum` needs threshold approvals) until the owner's new set executes. A third pass (-r10) fixed the comments' wording only. - tools/claims.py: OK, client 180. - Still not tested: the checkFloor call in main.mjs (no test runs checkFloor). Not changed (code review): the "fixed" line needs the short shape to have been seen since the guardian started (the Map is in memory, as the old per-process shape was; docs/07 says so). A protocol that abdicated SetGuardians can't take a new set, so the advice to run 2-of-4 can't be followed there; REPORT lists that case for the owner. The fault text assumes a margin of 2, the owner's decision (16 §4.4). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Tie the JS guardian margin and set size limit to the core constants cloud/lib/solana.mjs keeps its own GUARDIAN_MARGIN = 2, which the guardian and the deploy tooling use to judge a guardian set, and now MAX_GUARDIANS = 8, which the guardian's advice uses. Their comments say they are core governance.rs's constants, but nothing checked that. Had the programs' margin moved to 3, every JS test would have stayed green while the guardian passed sets the programs refuse. deploy.test.mjs now reads governance.rs and requires exactly one `pub const GUARDIAN_MARGIN: u8 = N;` and one `pub const MAX_GUARDIANS: usize = N;`, each N equal to the JS constant. It reads the source text, like the build-script test above it. If a declaration changes form, the test fails rather than passing. Evidence (tasks/floor-guardian-margin/r6/tie-mutants.log): - node --test cloud/test/*.test.mjs: 181/181 pass (one new test). - Mutants of governance.rs, each reverted with its mtime kept (else the limiter differential asks for a worker rebuild): GUARDIAN_MARGIN = 3, MAX_GUARDIANS = 9, and a trailing comment on MAX_GUARDIANS. Each fails the new test, naming the constant. deploy.test.mjs is green again. - tools/claims.py: OK, client 181 (docs/09:40 updated). Sibling search: the other JS copies of core rules (the limiter port, the account layouts) are checked by the differential and by decoding real accounts. MIN_GUARDIAN_THRESHOLD has no JS copy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Reach the 7-day extension total in the wide differential differential_wide.rs drew extension seconds only from {0, 1, 3600, 86400, 86401} on a random ticket and lumped every refusal into one counter. A ticket never got near the 7-day total, no negative value was sent, and a refusal was not checked by its code. So the default run could not tell the total bound, a negative extension or the error code from a bug: only the unit tests at vault.rs:3382-3401 pinned them. Now one extension draw in three pushes the chosen ticket by up to 24h a call to exactly the 604,800 s total, then asks for 1 s more, so the edge itself is hit on every push, not only when earlier draws line up. The other draws are half legal sizes (1, 3600, 86400) and half out of range for one call (i64::MIN, -86400, -1, 0, 86401, i64::MAX). Each refusal must fail with EXTENSION_TOO_LONG. The counters are split into extend (a random legal size), extend_push, extend_refused_call and extend_refused_total, all required >= 10, so pushes cannot stand in for the random sizes. The model reads both bounds from bulkhead-core (consts.rs:37-38), as the module says it is built, rather than restating 86,400 and 604,800. docs/09 §7's row says so, and the 16 Oct "Vault refusals deleted" row, which says the default run never extended a ticket that far, now says it does since this change (its differential_wide.rs:406 cite had drifted). Default run (24 seeds x 250 steps): extend 16, extend_push 126, extend_refused_call 16, extend_refused_total 30 (was extend 40, extend_refused 27); every other required counter still >= 10 (loosen_executed 12, as before). Evidence: mutants of brakes.rs extend_ticket, each built and run against the old and the new differential_wide only: T1 drop the total check old SURVIVED new KILLED T2 total > to >= old SURVIVED new KILLED T3 drop secs <= 0 old KILLED new KILLED T4 secs <= 0 to < 0 old KILLED new KILLED T5 total ignores this call's secs old SURVIVED new KILLED T6 extended_s += secs / 2 old SURVIVED new KILLED T7 refusal returns WrongTicket old SURVIVED new KILLED T8 secs <= 0 to == 0 (negatives) old SURVIVED new KILLED T9 total > to > total + 1 old SURVIVED new KILLED (T9: a draft that pushed by 24h until refused killed it only through chance 1 s and 3600 s draws on the same ticket; the push to exactly 604,800 then 1 s more reaches the edge every time.) sBPF sha256 after the run unchanged (Floor 8ca8a608, Harbor 543be60b, Vault 7086148b): tests only. Sibling search: the other refusal counters in this file (reset_refused, lifeboat_refused_*, loosen_*) are each split by cause already. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Check settle and lifeboat refusals by code in the wide differential The same gap as the extension refusal, found in the sibling search for the previous commit: a refused settle and the three lifeboat-move refusals asserted only r.is_err(). Any refusal passed, so the order of the checks and the program's own codes went unchecked by the differential. Settle's comment (custody.rs:520-521) says it returns its own error, not the token program's, when the vault is short; nothing in this file held it to that. Now a settle refused while frozen must fail with FROZEN, and one that is only short with INSUFFICIENT_BALANCE (frozen is checked first). A lifeboat move inside the delay after the seal must fail with NOT_SEALED, one while frozen with FROZEN, and one beyond the balance with the token program's InsufficientFunds (1). The two settle refusals get their own counters: settle_refused_frozen is required >= 10 like the rest; settle_refused_short needs lifeboat moves to drain a due ticket's tokens first and occurs 7 times in the default run, so it is required >= 1. Evidence: mutants of programs/vault/src, each built and run against the previous head's differential_wide and this one, nothing else: S1 settle: drop the InsufficientBalance check old SURVIVED new KILLED S2 settle: balance check before the freeze old SURVIVED new KILLED L1 require_sealed returns Frozen old SURVIVED new KILLED L2 move_in's freeze returns NotSealed old SURVIVED new KILLED The token-program code (1) on a move beyond the balance is pinned but not shown red by a mutant. sBPF sha256 unchanged (Floor 8ca8a608, Harbor 543be60b, Vault 7086148b): tests only. Deep run of both commits together (BULKHEAD_PROGRAM_SEEDS=600 BULKHEAD_PROGRAM_STEPS=500, 85 s): 300,000 steps, all identical; extend 876, extend_push 6,462, extend_refused_call 1,175, extend_refused_total 1,453, settle 2,024, settle_refused_frozen 921, settle_refused_short 566, lifeboat refusals 3,854 + 5,768 + 1,171. docs/09 §7's row gets that run as a 16 Oct result next to the 5 Oct one, which predates these checks. Sibling search: in differential_wide.rs, reset (TICKETS_PENDING) and execute change already asserted codes, and a release or quote refused for want of tokens asserts INSUFFICIENT_BALANCE. A release or quote the limiter refuses (frozen, lane closed, queue full) is still compared only as a refusal: mapping the core's refusal to the program's code is left for its own change. Lifeboat returns only send legal amounts. docs/09's row now says which refusals are checked by code. differential.rs:327 has the same is_err settle check; left as it is, since the wide run now pins both settle codes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
shuv-amp
added a commit
that referenced
this pull request
Oct 18, 2026
* Run the tests on 2-of-4 guardian sets; test recovery from two faults
Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set
with fewer than threshold + 2 keys. Every test fixture and most tests'
own sets were 2-of-3. This moves them to 2-of-4, which today's rule
also accepts, so this commit changes no program: Floor, Vault and Harbor
hash 508e5a99 / b07655d9 / 543be60b, as before.
- Fixtures: Protected (Floor) and Market (Vault, through Harbor) now
hold 4 guardians. Each test that built its own 3-key set gets a 4-key
set with the same threshold and the same assertion.
- Two tests changed meaning with a 4th key and were fixed, not loosened:
a_quorum_ejects_a_rogue_guardian_who_vetoes_everything (Floor) and
a_quorum_ejects_a_guardian_who_freezes_forever (Vault). With 4 keys,
the last eject was refused only because the target's own vote is
masked. Both now eject a second guardian (a set an ejection shrank to
threshold + 1 still ejects) and then show the 2-of-2 refusal.
- Account sweeps: an eject's signers may now be any two of the three
others, so each of those two slots is free for the third, never for
the target or the other signer again. Vault 5,365 and Floor 1,710
substitutions, refused (docs/09, dated 16 Oct). Listing all three
others at each slot would have skipped the duplicate: with a mutant
that counts a guardian listed twice as two (governance.rs:270), the
sweeps listing all three pass, and these fail at "eject: account 2
replaced by key ..." (Vault and Floor). No other program test kills
that mutant (tasks/aud2-guardian-margin-impl/dup-mutant-*.log).
- floor_differential: the guardian pool is 6 keys, 4 registered; the
proposed sets are 2 to 6 keys.
New tests (pass on both rules; they show what the margin buys):
- a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_set (Floor): g0's
key lost, g1 rogue stops the program; the lost key does not help a
quorum; the honest pair ejects g1 and restores; neither can veto the
owner's refill, which executes after 72h.
- two_rogues_in_turn_are_both_ejected_from_a_two_of_four_set (Floor):
the second rogue vetoes the refill and is ejected from the 2-of-3.
- a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_vault: the same in
the Vault (freeze, eject, unfreeze, refill).
Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest
and quote sweep): the counts and the sweep figures here are from runs
there (tasks/aud2-guardian-margin-impl/r4/). The E2E run, the mutants
and the new-rule run below were made before, on e8640a7; this commit's
programs and core are the same as there but for one comment line in
views.rs (line-neutral, same hashes).
Evidence:
- E2E=1 scripts/check-programs.sh: PROGRAM CHECKS PASSED (Floor 49,
Vault 98; S1 and E2E passed; the resumed setup found all 14 steps;
vault check, Floor demo and deploy dry run passed), with the next
commit's deploy rehearsal in the tree.
- tools/claims.py: OK (floor 49, vault 104, programs 153).
- Under the new rule (core from two commits on, programs rebuilt), only
the two tests that state the old rule fail: an_n_of_n_guardian_set_is_
refused (accepts 3-of-4) and the Floor differential's model.
- Mutants (tasks/coverage-map/mutants.sh), each red, then restored:
AV1 Floor live_guardian ignores liveness: KILLED by a_lost_key_and_a_
rogue_do_not_lock_a_two_of_four_set (+2 others).
AV2 the Vault's: KILLED by ..._two_of_four_vault (+2 others).
AQ1 Floor restore quorum ignores liveness: KILLED by ..._two_of_four_set
and the differential only.
AQ2 the Vault's unfreeze quorum: KILLED by ..._two_of_four_vault and
a_silent_guardian_does_not_count_toward_a_quorum.
AE1 can_eject refuses at count == threshold + 1 (AUD-2 option (a)):
KILLED by two_rogues_in_turn_..., and the two eject tests above.
Hashes after: 508e5a99 / 543be60b / b07655d9.
Split on 16 Oct from ec0cacc at the owner's review (D2): the deploy
rehearsal's move to 4 guardians is now the next commit. programs/ and
programs/tests/ here are byte-identical to ec0cacc, so its runs below
hold for this commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Rehearse deploys with four guardians, and read only the keys there are
Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set
with fewer than threshold + 2 keys, so the deploy rehearsal must
register 2-of-4 sets. Split from ec0cacc at the owner's review (D2);
the text is that commit's, unchanged:
Deploy rehearsal (cloud/deploy market.mjs, vault-check.mjs,
floor-demo.mjs): 4 guardians (keys/guardian3), so the dry run keeps
working once the rule lands; one more funding step (14, was 13).
The devnet market has 3 guardian files. A first version of this change
loaded guardian3 everywhere: vault-check.mjs failed there with ENOENT,
and heartbeat silently created a key nobody holds as a guardian. Now
market.mjs guardianKeys(dir, create) reads guardian0, 1, ... and
throws on a gap (never fills it with a new key). Setup creates keys up
to 4 only while the protocol is not registered, so resuming an old
market adds no key the chain lacks; a new deploy-dryrun step resumes
the dry-run market with guardian3 moved aside and fails if setup makes
it again (with setup always creating: "DRY RUN FAILED: resuming a
registered market made keys/guardian3.json", r3-dryrun-J7.log).
renew-guardians tops up to 4 and logs each new key (unfunded); a
2-of-3 renewal would be refused once the rule lands. keysIn reads the
guardians only when a command uses them, so caps and set-recovery
never need them; heartbeat says a key is not in the set rather than
dead. One THRESHOLD (2) for the register and renewal payloads and
MIN_GUARDIANS. vault-check.mjs
compares the chain's set with the first files, and liveness with the
chain's count. An extra file (a renewal not executed yet) and a set
without the margin are info lines, not failures, since such a set
keeps working. GUARDIAN_MARGIN and guardianMarginProblem live in
cloud/lib/solana.mjs, used by these and by the guardian bot's existing
warning (cloud/guardian/main.mjs), and vault-check uses THRESHOLD. The e2e scripts fund guardian3 too. Three
tests in cloud/test/deploy.test.mjs (client tests 176 -> 179). JS
mutants (js-mutants-r3.log), each red then restored: always create,
never create, no gap check, margin 1, `>`, GUARDIAN_MARGIN = 1.
Setup always creating survives these tests and is killed by the dry
run step above.
cloud/, scripts/deploy-dryrun.sh and docs/09's client-test row are
byte-identical to ec0cacc (git diff empty), so its runs hold for them:
the E2E check-programs run with the dry run (r4), and the JS mutants.
tools/claims.py: OK (client 179).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Refuse a guardian set without two keys beyond its threshold (107)
AUD-2, decided by the owner on 16 Oct (option (e) of 16 §4.4). A set
stays live while its honest live guardians reach the threshold: they
eject every rogue in turn, and dead guardians cannot veto the owner's
new set. Finding 96 asked for one key beyond the threshold, so a 2-of-3
or 3-of-4 set survived one fault but not a lost key plus a rogue: the
honest guardian left is no quorum, nobody can be ejected, and the rogue
keeps Floor stopped (or the Vault frozen) and vetoes every new set.
Fix, in the core so both programs inherit it through GuardianSet::parse
(registration, and every SetGuardians when proposed and when executed):
GUARDIAN_MARGIN = 2
guardian_set_ok := count <= 8 && t >= 2 && count >= t + 2
(saturating_add: t is a payload byte). A stored set is not checked
again, so existing 2-of-3 sets keep working, and ejection is unchanged:
a shrunk set still ejects. A 2- or 3-key set is BadThreshold, not
BadCount; the count rule is only the bitmap size, and the error order
of finding 103 is kept.
This changes both programs, by intent:
Floor 508e5a99 -> 8ca8a608, Vault b07655d9 -> 7086148b,
Harbor 543be60b (unchanged). Darwin-arm64 builds.
Compute (scratch LiteSVM test, tasks/aud2-guardian-margin-impl/cu.md):
-3 CU on every path that parses a set (Floor propose 2035 -> 2032,
execute 1682 -> 1679; Vault register 9592 -> 9589, SetGuardians 2979 ->
2976; Floor register varies with its PDA bump, residue also -3).
Proof and tests:
- Kani: accepted_sets_can_eject_anyone restated (ejection reachable
exactly when count > t, so option (a) cannot slip in; accepted sets
have >= 4 keys and a margin of at least 2, and 2 is accepted). New:
an_honest_live_quorum_ejects_any_rogue (any slots, anyone else alive)
and accepted_sets_survive_any_two_faults (any two faulty keys, lost
or rogue, leave a live honest quorum, which ejects each rogue, and the
rogues alone eject no honest guardian). That holds only short of a
rogue quorum: two rogues in a 2-of-n set are one (out of scope, 04
T7). With the assumption rogue < t dropped, the harness fails at
`!can_eject(rogue, ..)` (kani-no-rogue-assume.log), so the docs say
"short of a rogue quorum", not "any two faults".
19/19 verified, 240 s (cargo kani -j 2, as CI); again after the
restatement, 235 s summed (r2-prove.log).
- Kani mutants (kani-mutants.sh): margin 1, margin 3, `>` for `>=`,
t >= 1, count <= 9, option (a), and an eject one vote short (only
the two-fault harness kills that one): each KILLED by a harness
(kani-mutants-r2.log). The eject
ignoring liveness is the eject harness's case, not these.
- Program mutants (coverage-map/mutants.sh, rebuilt each time):
margin 1: KILLED by a_guardian_set_without_two_spare_keys_is_refused
(Floor and Vault), registration_rejects_weak_settings and the Floor
differential. Margin 3 and `>`: KILLED by most of the suite (the
fixtures are 2-of-4). t >= 1: first only the Floor differential; the
1-of-2 registration case was now refused by the margin too, so it is a
1-of-4 here and a Vault 1-of-4 case was added: then KILLED by both.
count <= 9: SURVIVES the program tests, equivalent there (parse refuses
more than 8 keys first), KILLED by Kani. Option (a): KILLED by the
eject tests two commits back (the tests on 2-of-4 sets).
- Tests: the Floor and Vault refusal tests above cover proposal and
execution (a stored 2-of-4 proposal whose threshold turns 3 is refused
when executed, as the devnet renewals proposed before this rule will
be); registration_rejects_weak_settings gains 2-of-3 and 3-of-4.
an_n_of_n_guardian_set_is_refused (Vault) is renamed and accepted a
3-of-4: it now refuses it and accepts 3-of-5.
- Model: the Python rule, the Floor differential's model, and the
guardian_set fuzz target state it again; the GP generator now also
picks count - 2. With the Python rule put back to the old one, seed 11
(20,000 cases) reports 5 GP mismatches; restored, seeds 11-13 identical.
Fuzz seeds two_of_three_refused, two_of_four, six_of_eight, and
seven_of_eight renamed seven_of_eight_refused (byte 323, the
threshold, is 7: now refused). With margin 1 in the core the target
crashes within 60 s; restored, 222,615,288 runs in 601 s, no crash.
- scripts/check.sh: ALL CHECKS PASSED, and E2E=1 check-programs.sh:
PROGRAM CHECKS PASSED, both at the branch tip, whose code is this
commit's (the next commit is docs only; tasks/aud2-guardian-margin-
impl/r4/check.log, r4/check-programs-e2e.log). claims: kani 19,
floor 50, vault 104, programs 154, client 179.
Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest
and quote sweep). The core and the program sources are the same as
before the rebase but for one comment line in views.rs (line-neutral),
so the Kani, fuzz, differential and mutant runs above were made on the
pre-rebase commit; the hashes, check.sh, check-programs and the counts
are from r4 runs (tasks/aud2-guardian-margin-impl/r4/).
The finding 96 row said the restated harness proves "exactly"; it now
says what it proved then and proves now, and gives the Vault test's new
name. The finding 107 row waits for rows 105 and 106 to land (09 §4 is
numbered 1..n); its text is ready. The rule's prose is the next commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* State the guardian margin in the docs, with its costs (AUD-2)
The previous commit made both programs refuse a new guardian set
without two keys beyond its threshold (finding 107). The docs that
state the rule now say so:
- README: guardians 4-8 keys; the G5 row and the "who the guardians
are" paragraph give the reason (any two faults leave a quorum, unless
the rogues are one themselves: two in a 2-of-n set, 04 T7).
- 05 register_protocol: 2 <= threshold <= count - 2, also for every
SetGuardians, proposed and executed. 05 abdicate: abdicating
SetGuardians fixes the set, so ejections shrink it for good.
- 06 state table, 00 D22, FAQ: the same rule.
- 07 §2: 4-8 guardians, and don't abdicate SetGuardians. 07 §6: the
programs enforce the margin; sets registered before keep their
exposure until replaced; an ejection uses up a spare key until the
owner's new set executes; a lost key counts as live until its
interval passes.
- 16: the AUD-2 row is decided; §4.4 gives the decision, why the
margin is the number of faults a set survives (short of a rogue
quorum), the options rejected
and why (the honest-owner and stolen-owner cases look the same
on-chain), what it costs, what it leaves (a rogue quorum; a stolen
owner key with every guardian dead; a rogue's veto of the recovery
key's rotation, checked: Floor's veto has no recovery exception, as
a_vetoed_recovery_proposal_does_not_hold_the_owner_s_slot shows), and
Lido-style lapsing vetoes as a future option only. §5: "independent
guardians, 3-of-5 or more".
- 10: the guardian set's minimum size is no longer an open decision.
SECURITY.md and 08 describe what is on devnet, which still runs the
old rule, and are left as they are.
claims: OK. Docs only; no program file changed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Say the weekly CI run skips mutants, as its condition does
ci.yml's header and scripts/ci-changes.sh said the weekly run does every
job, and 09 §8 gave `mutants` as running on any change to the core. The
`mutants` job's `if` starts with `github.event_name == 'pull_request'`,
so the Monday schedule (and a manual dispatch) never runs it: it mutates
a pull request's diff and has none to mutate. Its trigger list in 09 §8
also left out scripts/mutants.sh and .cargo/mutants.toml, which
ci-changes.sh (line 22) selects it for.
Comments and docs only; nothing runs differently.
Sibling search: `git grep` for "weekly run does", "every job ... weekly"
and "mutants ... weekly" across md/yml/sh/mjs/py. The other hits are
09 §8's "at full depth every Monday" (its table now says which job is
not weekly) and the dispatch input's "every job at weekly depth", which
is the same set as the weekly run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check that the mutant exclusions still match; repin murmur3 (finding 105)
.cargo/mutants.toml excludes the core's 22 equivalent mutants, and the
docs say so. Since 9 Oct it excluded 19. The murmur3 tail's pattern is
pinned by line, `floor.rs:(235|238|241)`, because the body's `h ^= k`
has the same column and function and is not equivalent. 1dc7861 (#52)
moved those lines (now 327/330/333), and nothing checked the patterns.
Failing closed, the three would have failed CI's `mutants` job on a
pull request that changed those lines. Failing open, a pattern that
starts matching a non-equivalent mutant hides it from every run.
Root cause: nothing checked that a pattern still names its mutants.
Fix:
- each pattern ends with the number of mutants it excludes (`# 3`), and
the murmur3 pattern names 327/330/333;
- tools/mutant_exclusions.py fails unless each pattern matches exactly
its number in `cargo mutants --list --no-config`, and cargo-mutants
(the list with the config) excludes exactly the mutants the patterns
match, so a pattern this parser can't see, or reads differently, fails
too. No two patterns may match the same mutant, since the docs' count
is the counts' sum, and the kept list must be part of the full one.
It reads only the exclude_re array's body, needs no tomllib
(Python 3.9), and names colour codes in the list as the problem;
- scripts/mutants.sh makes both lists (--colors never: CI sets
CARGO_TERM_COLOR=always) and runs it first, except when the caller
passes --config or --no-config (as whole arguments). CI's `mutants` job runs that script;
ci-changes.sh now also selects the job for the new tool;
- scripts/check.sh runs its --self-test on planted configs;
- tools/claims.py ties the docs' "22 equivalents" (README, 09 §6 and the
summary row) to the sum of the counts, with the same parser. It finds
that parser from its own directory, so `python3 -I` runs it too, and
check.sh runs its self-test that way. 00's
"18 equivalents" (26 Sep log entry) and run 9's "exactly the 22
equivalents below" (6 Oct) are kept as dated records.
Docs: 09 §4 finding 105, §6 run 10 and the "In CI" note, §8's mutants
row; README and 09's summary row 669 → 797 mutants (run 10); README and
12's post-submission range 96–105.
Evidence (first on 1f2ad48 + the parent, logs in the autopilot's
tasks/mutants-exclusions-r2/; re-run on 58697c6 + the parent,
tasks/mutants-exclusions-r3/):
- `cargo mutants --list`: 797 with --no-config, 778 with the old config
(19 excluded), 775 with this one. The 22 excluded are §6's list, the
murmur3 ones at 327/330/333.
- scripts/mutants.sh --list: exit 0, "22 of 797 mutants excluded"; with
CARGO_TERM_COLOR=always also exit 0. Old config: exit 1, every pattern
"no '# N' count". Planted on the real lists: the old lines with a
count (matches 0, says 3) and the body XOR added (matches 4, says 3)
both exit 1; in both the set comparison alone passes. /usr/bin/python3
(3.9.6): self-test OK and the real check passes.
- The self-test pins each problem by its message, not a count. Mutants
of the tool (tool-mutants.log), each red: M1 no per-pattern check,
M2 no set comparison, M3 no colour check, M4 no empty-array check,
M5 a missing count accepted, M6 a missing count read as 0, M7 the
parser reading past the array. M3 and M5 survived a first version
that checked only the number of problems.
After A's review (16 Oct): the set comparison was pinned only by its
size, so `len(excluded) != len(matched)` passed the self-test. A
case with as many excluded as matched but not the same ones now
fails it (15 cases; red with that mutant, green without), and the
message says how many mutants are in one set only.
From the code review after the rebase: colour codes only in the kept
list were reported as a kept list made with other options; the
colour check now reads both lists (16 cases; dropping the kept half
fails the new case).
On 58697c6, with the overlap and kept-list checks: M8 no overlap
check and M9 no kept-list check, red too, as M1–M7 still are. A
duplicate of the can_eject pattern planted in the real config:
scripts/mutants.sh --list exits 1, "the counts add up to 23, the
patterns match 22 mutants" (planted-overlap.log). Without the
sys.path line, `python3 -I tools/claims.py` fails with
ModuleNotFoundError; with it, both claims runs pass.
- claims.py (claims-mutants.log): one count 1 → 2 fails four statements
(README 188; 09 33, 333, 335; kept-run9.log), not run 9's 331, which
failed too before it was kept; a dropped count fails with the
tool named; deleting any of the three rules fails a self-test case;
deleting the KEPT entry fails 00:103.
- The 22 excluded mutants run alone (--no-config, --re per pattern,
2 min): 22 missed, exactly the excluded list; --re also picked
ema.rs:70 (a field deletion, no pattern matches it), caught.
- Run 10 (f1f05ba, old config, -j4, 46 min; tasks/mutants-run10/):
778 tested, 712 caught, 5 timeouts, 58 unviable, 3 missed, exactly
the murmur3 tail. The core's source is unchanged since (`git diff
--stat f1f05ba HEAD -- crates/bulkhead-core/src` is empty; only
tests/units.rs gained finding 102's tests).
- No program or core source changed.
Rebased on 16 Oct onto cede705 (findings 102 and 100, the attest
sweep and AUD-2). AUD-2 changed the core's source (governance.rs), so
the line above saying it is unchanged since run 10 held on 58697c6, not
here. On cede705: scripts/mutants.sh's list check passes, "22 of 795
mutants excluded, each pattern matching its count"; the murmur3 lines
are still 327/330/333. The 797 in README and 09 is run 10's figure:
README now dates it ("in the last full run, 15 Oct") and finding
105's row says the core has changed since (finding 107).
The setup line in scripts/mutants.sh now pins cargo-mutants 27.1.0,
the version ci.yml installs: each count is the number of mutants that
version lists, so another release can break the check (code review,
16 Oct; `git grep 'install.*cargo-mutants'` finds no other unpinned
install).
The check's report goes to stderr in scripts/mutants.sh, so
`scripts/mutants.sh --list` prints only cargo-mutants' names: before,
its first stdout line was the report; after, stdout is 773 lines
(795 - 22) and stderr the one report line. ci.yml's `mutants` comment
says the job checks the exclusions first.
Row 105 and 09 §6 gave the pin's reason as the body's XORs. The
finalizer's are at column 7 (`grep -n '\^=' floor.rs`, 337-342); the
non-equivalent XOR at column 11 is `h ^= k`, in the body (320) and after
the tail (335). The toml comment, the row and §6 now say that. 09 §8's
`check` row names the two tool self-tests check.sh runs.
README no longer calls run 10's 797 "injected bugs": cargo-mutants
listed 797 but tested 778, 19 being excluded (09 §6, run 10's row).
Sibling search: the other 14 patterns use `\d+` for the line, and each
matches the count now written after it. The murmur3 one is the only
line-pinned pattern.
Not done, by choice:
- Catching an edit that keeps a pattern's number but changes its
mutants (an excluded line itself edited, or a murmur3 body XOR moved
onto a pinned line). The toml and 09 §6 say so: such an edit needs the
mutant judged again by hand. Failing on any diff that touches an
excluded line would catch the first; it is a separate change.
- Moving murmur3's tail into its own function, so its pattern needn't
name lines. It shifts core lines, so it changes the program hashes.
Until then a line move above it fails the check, with a message
saying what to fix.
- A weekly full mutants run in CI. It took 46 min here at -j4, likely
past the job's 90 min on two CI cores: a minutes call for the owner.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that a ticket stays in its own protocol and vault
Two Vault checks that keep a ticket to its own protocol and vault had no
test. Deleting either one left every program test green
(tasks/error-codes/mutants.log, base cede705):
- brakes.rs:79, extend_ticket: tk.protocol != protocol. Without it, a live
guardian of any other protocol extends this protocol's tickets. Anyone
can register a protocol and name its guardians, so that guardian could
hold back every queued payout up to the total extension limit. The
sweep did not catch it: it has no second protocol's guardian to swap in.
- custody.rs:501, settle_ticket: tk.vault != vault. A USDC ticket settled
through the SOL vault (its tokens, its mint) got past the Vault's checks.
New test another_protocols_guardian_cannot_extend_a_ticket: a second
Market's live guardian extends this market's ticket -> WRONG_VAULT; the
same instruction from this market's guardian succeeds. The settle test
now settles the USDC ticket through the SOL vault and expects
WRONG_VAULT.
Evidence (tasks/error-codes/mutants.log, STATE=c1; full program suite
per mutant):
W02 brakes.rs:79 deleted -> KILLED by
another_protocols_guardian_cannot_extend_a_ticket ("expected
failure, got success": the stranger's extension went through)
W06 custody.rs:501 deleted -> KILLED by
a_ticket_settles_after_its_eta_to_its_stored_recipient_only (8049
InsufficientBalance from the empty SOL vault, not 8010)
Both were SURVIVED at base. No program source change: hashes
8ca8a608/543be60b/7086148b, as at cede705.
Sibling search: the other 12 WrongVault sites were deleted one at a time
in the same run. Eight are KILLED by the sweep (brakes.rs:39, lib.rs:320,
custody.rs:160, genesis.rs:491, views.rs:27, lifeboat.rs:151,
governance.rs:169/259). lib.rs:324 (vault PDA address), genesis.rs:523,
lifeboat.rs:71 and :247 survive; later commits on this branch deal with
them.
Counts: Vault 104 -> 105, programs 154 -> 155 (claims.py OK).
The test also reads the ticket's ETA: unchanged after the
refused extension, 3,600 s later after its own guardian's (code-review
fix: success alone did not show the extension happened).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that the seal names every vault and happens once
Two seal checks had no test; deleting either left every program test
green (tasks/error-codes/mutants.log, base cede705):
- genesis.rs:486, execute_seal: the pairs must number 3 * vault_count.
Without it, the seal checks only the vaults it is given. A vault left
out skips its claims-vs-balance check, so an uncovered import seals.
docs/05's seal row says every vault is passed exactly once.
the_seal_checks_each_vault_once covered a vault named twice in place of
another, not one left out.
- genesis.rs:431, propose_seal: refused once sealed (AlreadySealed).
Without it, a sealed protocol's admin re-proposes, which rewrites
seal_executable_s/expires_s and clears seal_vetoed on a sealed protocol.
the_seal_checks_each_vault_once now also leaves the uncovered USDC vault
out (expects VAULT_COUNT_MISMATCH) and, after the seal, proposes it again
(expects ALREADY_SEALED). No new test, so no count change.
Evidence (tasks/error-codes/mutants.log, STATE=c2, full program suite):
E05 genesis.rs:486 deleted -> KILLED by the_seal_checks_each_vault_once
("expected failure, got success": the seal went through without
the USDC vault and its 1,000 USDC uncovered import)
E04 genesis.rs:431 deleted -> KILLED by the_seal_checks_each_vault_once
("expected failure, got success")
Both were SURVIVED at base. Vault hash 7086148b unchanged (tests only).
Sibling search: the seal's other checks. :491 vault_tokens (KILLED by the
sweep), :519 lifeboat owner and :523 lifeboat mint/owner (SURVIVED; the
lifeboat is a PDA checked by expect_pda just above, which only this
program can create, as its own token account: see this branch's note in
docs/09).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that invalid limits and asset settings are refused at every entry
Three bound checks had no test; deleting any one left every program test
green (tasks/error-codes/mutants.log, base cede705):
- genesis.rs:171, create_vault: config_is_valid (bps <= 100%, windows
> 0). Without it, a vault is created with a 100.01%-an-hour burst.
- governance.rs:40, parse of a SetLimits change (at proposal and again
at execution, :254 and :478): the same rule for a proposed change.
- governance.rs:47, parse of an UpsertAsset change: valid_asset (haircut
<= 100%, liability >= 100%, the oracle bounds of finding 88). Only the
genesis upsert's copy (genesis.rs:296) was tested, by
oracle_settings_of_a_priced_asset_are_bounded.
New test invalid_limits_and_assets_are_refused_wherever_they_enter:
before the seal, create_vault with short_bps 10,001 -> INVALID_LIMITS;
after it, a SetLimits proposal with long_window_s 0 -> INVALID_LIMITS,
and an UpsertAsset proposal with a 10,001 bps haircut -> INVALID_ASSET.
Evidence (tasks/error-codes/mutants.log, STATE=c3, full program suite):
E06 genesis.rs:171 deleted -> KILLED ("expected failure, got success")
E07 governance.rs:40 deleted -> KILLED ("expected failure, got success")
E08 governance.rs:47 deleted -> KILLED ("expected failure, got success")
each by the new test only. All three were SURVIVED at base. Vault hash
7086148b unchanged (tests only).
Sibling search: every config_is_valid / valid_asset call in programs/
vault/src: genesis.rs:170 (create_vault), :295 (upsert_asset, KILLED by
the oracle test), governance.rs:39 and :46. All now killed.
Counts: Vault 105 -> 106, programs 155 -> 156 (claims.py OK).
The execution copy (:478) is not reached by any transaction: it
parses the bytes the proposal already checked, and the bounds read
nothing else. The test comment says so (code-review fix: it had
claimed execution was covered). oracle_settings_of_a_priced_asset_are_
bounded now asserts INVALID_ASSET for each of its four cases, not any
error (all four return it; E09 stays KILLED).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that an import into a ninth mint is refused
genesis.rs:410, import_backing: a backing ledger holds MAX_SLOTS (8)
mints, and an import into a ninth is refused with NoSlot. Deleting the
check left every program test green (tasks/error-codes/mutants.log, base
cede705). From the code, without it the import goes through: the vault's
imported total grows (:414), and the seal must cover that, but the
owner's ledger is saved unchanged, so the owner has no backed claim for
it.
New test an_import_into_a_ninth_mint_is_refused: nine vaults before the
seal; one owner gets an import into eight (ledger_of shows 8 positions),
then the ninth -> NO_SLOT.
Evidence (tasks/error-codes/mutants.log, STATE=c4, full program suite):
E14 genesis.rs:410 deleted -> KILLED by an_import_into_a_ninth_mint_is_
refused ("expected failure, got success"); SURVIVED at base.
Vault hash 7086148b unchanged (tests only).
Sibling search: the only other ledger credit is deposit's
(custody.rs:128). There a full ledger is by design: the deposit is taken
unbacked (Ledger::credit's doc, ledger.rs:187-189), and credit's result is
the backed flag, not an error.
Counts: Vault 106 -> 107, programs 156 -> 157 (claims.py OK).
import_ix now calls import_into_ix, which takes the vault, so the
new test shares the one import layout (code-review fix).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin six Vault refusals no test reached
Six reachable checks had no test that failed when the check was deleted
(tasks/error-codes/mutants.log, base cede705). Each now has an assertion
on its code, and each was seen to fail with the check deleted:
E02 genesis.rs:185 create_vault of an existing vault after the seal:
ALREADY_INITIALIZED (with the check gone: 8043 GenesisOnly, so the
"says so first" order in the comment above it was untested)
E12 governance.rs:426 veto of an executed change: CHANGE_EXECUTED
(gone: the veto succeeds and marks an executed change vetoed)
E13 governance.rs:517 cancel of an executed change: CHANGE_EXECUTED
(gone: the cancel succeeds)
E16 lifeboat.rs:256 execute_exit with no exit proposed:
NOTHING_TO_EXIT (gone: 8033 ChangeNotExecutable)
E17 custody.rs:84 a zero deposit: ZERO_AMOUNT (gone: it succeeds)
E18 custody.rs:272 a zero release: ZERO_AMOUNT (gone: it succeeds; a
zero release has no unbacked part, and the limiter, whose own zero
check custody.rs:242 maps, is called only when unbacked > 0, :361)
Where: E02 in after_the_seal_a_new_vault_starts_with_its_unbacked_lane_
closed, E12/E13 in an_unvetoed_loosening_executes_after_the_delay, E16 in
the_lifeboat_stays_shut_until_the_seal (after the move in, before an
exit is proposed), E17/E18 in the new a_zero_deposit_or_release_is_
refused.
Evidence (tasks/error-codes/mutants.log, STATE=c5, full program suite):
all six KILLED by the tests named above; all six SURVIVED at base. The
vault suite is green (108 passed). Vault hash 7086148b unchanged.
Counts: Vault 107 -> 108, programs 157 -> 158 (claims.py OK).
Helpers (code-review fixes): veto_change_ix beside cancel_change_ix;
create_vault_for_ix, create_vault_ix with a given mint and token
program, so E02's case names USDC's own accounts and creates no mint.
E02's comment now repeats the program's ("whatever the seal's state"):
create_vault checks the limits first (genesis.rs:170).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin four Vault checks another layer backstops
Four checks survived deletion (tasks/error-codes/mutants.log, base
cede705) because something later still refused the case, with its own
error, so no test could tell. Each now has a test that sees the Vault's
own code, and each was seen to fail with the check deleted:
E01 lib.rs:230, create_pda's guard: a second registration of an
existing protocol (register_protocol has no check of its own) ->
ALREADY_INITIALIZED. Gone: the system program refuses it, with no
custom code (custom_code 0).
W10 lifeboat.rs:71, move_in naming another mint or token program ->
WRONG_VAULT. Gone: the token program refuses (code 3, MintMismatch).
W12 lifeboat.rs:247, execute_exit naming another mint or token
program -> WRONG_VAULT. Gone: the same token-program error.
T01 token.rs:37, account_info on an account no token program owns: a
copy of a real USDC account's bytes under the system program. Gone:
the quote SUCCEEDS (it promises a payout), and the release fails
only in the runtime at the transfer ("instruction modified data of
an account it does not own"). The quote is asserted first so the
mutant is seen at the quote (STATE=c6b).
New tests: a_protocol_registers_once, a_lifeboat_move_names_the_vaults_
own_mint (also completes a move in and an exit, as a control: the
destination receives 1,000 USDC), a_payout_goes_only_to_an_account_a_
token_program_owns.
Evidence (tasks/error-codes/mutants.log, STATE=c6 and c6b, full program
suite): E01, W10, W12, T01 KILLED by the tests named above; all four
SURVIVED at base. Full program suite green: Floor 50, Vault 111. Vault
hash 7086148b unchanged (tests only).
Counts: Vault 108 -> 111, programs 158 -> 161 (claims.py OK).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test the bounds on ticket extensions and a closed lane's code
B's review of vault-error-tests-r3 (R1) found two Vault codes no test
asserted by value that the earlier commits missed: ExtensionTooLong
and LaneClosed. Deleting the 7-day total extension cap (brakes.rs:86,
B24) left every program test green, so a live guardian, or a stolen
guardian key, could hold a queued payout back without end, 24h at a
time, against docs/05 row 11. The wide differential models the total
(differential_wide.rs:406), but its default run never extends one
ticket that far. Mapping LaneClosed to QueueFull (custody.rs:240, B23)
also survived.
The code review of the fix found a worse gap at the same check: no
test sends a negative extension (the differential draws secs from 0,
1, 3600, 86400, 86401). With `secs <= 0` weakened to `secs == 0` (A3,
brakes.rs:84) every program test passed, and one guardian key could
pull a queued payout's ETA into the past, skipping the queue delay,
and refund its own 7-day budget.
- a_ticket_is_extended_later_by_at_most_a_day_a_call_and_seven_in_total:
-1h, 0 and 24h + 1 s are each refused with EXTENSION_TOO_LONG, ETA
unchanged; six 24h extensions and one of 23h pass; 2h more is
refused, ETA unchanged; the last hour is granted; then 1 s more is
refused. The split is unaligned so that a total that ignores this
call's `secs` fails too.
- a_closed_lane_refuses_an_unbacked_payout: a vault with short_bps 0,
and one with long_bps 0, each refuse an unbacked admin payout with
LANE_CLOSED; the same payout from a vault with both buckets open is
taken (the control, so a capacity base of 0 cannot pass it).
Evidence (tasks/vault-error-tests-r4/, B's runner copied: each mutant
applied alone, programs rebuilt, whole program suite run):
- without these tests (551e519; run-without.out, run-base2.out,
run-base3.out): B23, B24, A1 (`tk.extended_s >= MAX`, this call's
secs ignored), A2 (`>=` for `>`) and A3 SURVIVED. A4 (the 24h per
call deleted), A5 and A6 (either half of limiter.rs:172's closed-lane
test deleted) were KILLED by the differential only, not by value;
- with them (run-final.out, at the branch tip): B23 KILLED by
a_closed_lane_refuses_an_unbacked_payout; B24 A1 A2 A3 KILLED by the
extension test only; A4 by it and the differential; A5 A6 by the
closed-lane test and the differential.
No program change: the runner's closing build gives 8ca8a608 /
543be60b / 7086148b, as at cede705.
Counts: Vault 111 -> 113, programs 161 -> 163 (claims.py OK).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin each half of create_pda's existing-account guard on its own
B's review of vault-error-tests-r3 (R2): each half of lib.rs:229
(`!is_data_empty() || !owned_by(SYSTEM)`) survived deleted on its own
(B20, B21), since an existing protocol account fails both, so
a_protocol_registers_once pinned them only jointly.
The test now also plants, at a fresh protocol address, an account that
fails just one half: one with data owned by the system program, one
with no data owned by another program (a fresh address). Each
registration is refused with ALREADY_INITIALIZED. The comment says no transaction is
known to make either account: data or a new owner at a PDA needs its
signature, which only the Vault gives, and create_pda gives both in one
instruction.
Evidence (tasks/vault-error-tests-r4/): at 551e519 B20 and B21
SURVIVED (run-base2.out); at the branch tip both are KILLED by
a_protocol_registers_once only (run-final.out).
No count change (claims.py OK: vault_programs 113, programs 163).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Record the Vault refusal deletions and what each became
docs/09 §3 gains a row for the deletion run this branch answers
(tasks/error-codes/mutants.log): 38 refusals, across 14 of the 18 Vault
codes no test asserted by value at cede705, each deleted on its own
with the whole program suite run against it, on the AUD-2 build.
The 18 (each VaultError variant with no `verr::` use in programs/tests
at cede705): AlreadyInitialized NotRegistrar WrongVault
WrongTokenProgram AlreadySealed VaultCountMismatch InvalidLimits
InvalidAsset LaneClosed TicketNotPending ExtensionTooLong
ChangeExecuted LedgerUnsound LedgerCorrupt NoSlot NothingToExit
ZeroAmount Overflow. The run missed LaneClosed and ExtensionTooLong;
B's review deleted one check of each (B23 custody.rs:240, B24
brakes.rs:86), and both survived until 1797ae5. That commit
also pins a negative extension (A3, which survived every test before
it), the 24h per call, and a lane closed by either bucket.
At base: 13 KILLED, 25 SURVIVED. The commits before this one add tests
that kill 18 of the 25 and both of B's, each asserting the Vault's own
code (W02 W06, E04 E05, E06 E07 E08, E14, E02 E12 E13 E16 E17 E18, E01
W10 W12 T01; B23 B24). The row names the 7 that still survive and why:
- six guards no transaction is known to reach: lib.rs:324 (vault PDA
address; create_vault creates a vault only at its PDA), genesis.rs:519
and :523 (the lifeboat, checked by expect_pda first, created only by
lifeboat_move, move_in requires the seal, lifeboat.rs:53, and the
seal happens once, so no lifeboat holds data at it), brakes.rs:50 and
custody.rs:541 (a pending-ticket count below zero), token.rs:235
(GetAccountDataSize's return data from an already-checked program);
- custody.rs:399, stated as not shown unreachable: properties.rs:588-592
debits backed_amount and one more on random ledgers, and only
crates/bulkhead-core/tests/units.rs:375 debits less, on a one-mint
ledger with no liability. properties.rs:606 asserts backed(a) ==
min(a, backed(MAX)) for requests below 2^52 (:613), so any smaller
amount is backed too, and debit_backed(b) refuses only when
backed_amount(b) != b (ledger.rs:289). The known exception is a
collateral value that overflows u128 (ledger.rs:262): the valid
amounts need not form an interval there, so the consent or freeze
cap (custody.rs:337, :347) can lower the amount to an invalid one and
:399 refuses the release (fail closed). The search's own result is
always valid. Not verified by a tool beyond that property test. (A full ledger cannot reach :399 through
set's NoSlot: backed_amount's valid() refuses a new liability slot
on a full ledger, ledger.rs:237.)
Not deleted, as the row says: the two core-error mappings to
ChangeExecuted and ZeroAmount, token.rs:233, and the codes Overflow and
LedgerCorrupt (checked arithmetic, decoding).
After this branch five codes are still asserted by no test (same grep
at this commit): NotRegistrar (E03, genesis.rs:108, was KILLED at base
by another test), TicketNotPending, LedgerUnsound, LedgerCorrupt,
Overflow.
Docs only: no count change (claims.py OK: vault_programs 113, programs
163).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Run the exclusion check's main in its self-test
A's review of finding 105 (REVIEWS/auto-B-mutants-exclusions-r7.md,
finding 1, Low): CI's `mutants` job acts on main's exit status, but the
self-test called problems() only, and check.sh never runs the tool on
real lists. With main returning 0 on a mismatch (mutant A10), every
in-tree check passed, and CI would have printed the problems and gone
green. This was not a defect at fb771c6, since the job fails today. The
gap was that a later regression would pass.
Fix: main takes its arguments and, for the self-test, a config path
(default: the repo's .cargo/mutants.toml, as before), and its lines name
the config it read. The self-test runs main in-process on a planted
config and checks the exit status and every line on each stream:
- the right lists: exit 0, "1 of 3 mutants excluded" on stdout, nothing
on stderr;
- the lists the other way round: exit 1, nothing on stdout, and exactly
the count and "full list lacks" problems, so KEPT is read as KEPT;
- coloured lists: exit 1 with exactly one problem, so a main that fails
only on several problems is caught;
- one argument and three: exit 2 and the two usage lines.
One subprocess (python3 -I, 60 s timeout, in the temporary directory so
the config must be found from the tool's own path) runs the command line
with two lists and the repo's config, as scripts/mutants.sh does: on a
list no pattern can match, it exits 1 with an empty stdout and problem
lines that each name .cargo/mutants.toml. Which problems those are
depends on the config, so only their form is checked. Self-test: 16
cases -> 22. No doc states the number.
Evidence (tasks/exclusions-main-exit/):
- old-A10.log: the fb771c6 self-test with A10 applied prints "self-test
OK: 16 cases" and exits 0.
- mutants-r6.log (mutants.py applies each mutant to a scratch copy of
the tool, which reads the real config through a symlink): unmutated,
"self-test OK: 22 cases", exit 0. All 18 are KILLED (exit 1): A10
(return 0), A10b (fail only on more than one problem), A0 (always 1),
U1 (usage returns 0), U2/U3 (argument check `< 2`, `< 1`), K1 (KEPT
ignored), K2 (ALL read twice), C1 (planted config ignored), C2 (no
default config), R1 (the default config read from the cwd), E1/E2
(sys.argv sliced as [:] or [1:2]), P1 (problems not printed), S1
(problems on stdout), L1 (the planted config not named), N1 (only the
first problem printed), G1 (the success line printed on failure too).
Code-review passes at high shaped this. The first version ran three
subprocesses against the real config, and A0, K1 and U3 survived it.
The second left E2, C2 and G1 alive. Declined from the third: catching
exceptions in the self-test (a traceback already exits 1) and moving
the config hook out of main. The fourth showed that requiring one
"matches 0 mutants" line per live pattern tied the self-test to the
config's contents. The subprocess now checks the form of the lines
only, and the 17 stay killed. The fifth showed that R1 survived while
the child ran in the repo root (it does in check.sh); now it runs in
the temporary directory. Its other points repeat declined ones.
- real-lists.log: on the real lists, made with scripts/mutants.sh's
commands: ".cargo/mutants.toml: 22 of 795 mutants excluded, each
pattern matching its count", exit 0.
- python3 -I tools/mutant_exclusions.py --self-test: 22 cases OK.
- scripts/check.sh: ALL CHECKS PASSED (check-final.log);
bin/check-each-commit.sh green (each-commit.log).
Sibling search: the other tools check.sh runs (tools/claims.py,
tools/floor_check.py) also gate CI through main's exit status. claims.py
has the same gap: check.sh runs it on the real tree, where it finds no
problems, and its self-test calls self_test(), not the path in main that
returns 1. With that return changed to 0, `--self-test` and the real run
both still exit 0 (claims-C1.log). Not changed here: it is its own
commit (BACKLOG-B). floor_check.py runs in CI only as --self-test, whose
result main returns as is. Its RPC mode is an operator's tool that no CI
job runs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Record finding 106: Vault liveness and exit rules had no failing test
The tests landed in 944f5e8 (lifeboat-exit-tests-r3, 15 Oct). Their
09 §4 row waited for 104 and 105 so the list stays numbered without
gaps; 105 is now on integration (fb771c6), so the row lands here.
Docs only: 09 §4 row 106; README and 12's count and post-submission
range (96–106, 106 fixed). Two code-review passes changed the row as
written on 15 Oct: a sentence for today's re-run, the nine
single-guardian sites named, G5 quoted verbatim, and the exit veto
attributed to T18 (G6 has no veto), as is the veto test's doc comment
(vault.rs:3996, comment only).
Evidence, re-run today on bf8ba41 (the row's claims are from 15 Oct,
and AUD-2 changed the Vault since): B's re-review list of 33 line-
neutral mutants (the ten rules, the nine single-guardian sites L1-L9,
the boundaries, the six quorums, R1-R6), each applied, programs
rebuilt, all program tests run:
33 KILLED, 0 survived, 0 not applied, 0 build failures. Vault
hashes after the run match the tip (Vault 7086148b, Floor 8ca8a608,
Harbor 543be60b).
Log: autopilot tasks/finding-106-row/mutants.log.
- python3 -I tools/claims.py: OK, findings 106.
- git grep for "96–105", "105 findings": no other stale range.
- scripts/mutants.sh still mutates the core only (SCOPE, line 11), so
the row's last sentence holds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Cite PR #100's tree, not the unpublished bf8ba41, in finding 106
Finding 106's row in docs/09 said its 33 mutants were re-run on 16 Oct
at bf8ba41. That commit is not an ancestor of this branch or of the
stack above it and is on no remote branch, so the citation would point
at nothing once the stack lands. The owner's review of #99-#109 found it.
bf8ba41 and 86b777b (the head of PR #100, vault-error-tests-d2, the D2
replay of vault-error-tests-r4) have the same tree, 36187b49
(`git rev-parse bf8ba41^{tree} 86b777b^{tree}` prints it twice). So
the run was a run on 86b777b's tree, and the row now says "on tree
36187b49, the tree of 86b777b (PR #100's head)". Why not just "at
86b777b": it was committed on 17 Oct, after the run; and main
squash-merges (1045cd9 has one parent), so after PR #100 lands 86b777b
is reachable only as GitHub's refs/pull/100/head. The tree id stays
checkable from main: squashed bottom first, main's commit for PR #100
has that tree. "PR" keeps "#100" from reading as finding 100, the
row above. These points came from two code-review passes.
Sibling search: every 7-12 hex token in tracked .md/.py/.sh/.mjs/.rs/
.toml files at autopilot/integration 3e9f58d that names a local commit
(51 tokens) was checked with git merge-base --is-ancestor against
integration. bf8ba41 was the only one that is not an ancestor.
Docs only. tools/claims.py: claims OK (26 files).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Warn about a Floor guardian set short of the margin (finding 107)
The guardian checks the Vault protocol's guardian set against finding
107's margin (main.mjs checkGuardianSet), but never a Floor config's.
A Floor set registered before the rule, say 2-of-3, keeps working and
keeps AUD-2's exposure: one lost key and one rogue guardian lock its
governance for good. The guardian said nothing, and docs/07 §6 said so
("not yet a Floor set").
The rule moves to policy.mjs as guardianSetWarning, a pure function:
the same text as before, a warn line that alerts on mainnet only. In
main.mjs, checkGuardianSet now remembers the set shape per set
("protocol", or the protected program), and checkFloor calls it for
each watched Floor config it reads. floorAlerts is unchanged. The
margin is not a readiness code there: "floor not ready" alerts on
every cluster, and only the owner's new set clears a margin problem, so
"floor ready" would never be logged again.
Evidence (tasks/floor-guardian-margin/r2/):
- node --test cloud/test/*.test.mjs: 180/180 pass (one new test).
- Mutants of guardianSetWarning, each run on policy.test.mjs and
reverted; each fails the new test: N1 never warn, N2 threshold fixed
at 2, N3 notify always, N4 notify never, N5 GUARDIAN_MARGIN = 1.
- tools/claims.py: OK, client 180 (docs/09:40 updated).
- Not tested: the main.mjs wiring (the per-set Map and the checkFloor
call). No test runs checkFloor; deleting the call leaves every test
green.
Sibling search: guardianMarginProblem's callers are vault-check.mjs (an
info line) and guardianSetWarning. The status snapshot
(snapshot.mjs:33) prints live guardians and the quorum, not the margin;
left as it is.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Judge each guardian set's margin on its own, and say when it is fixed
The parent's per-set memory (which set shapes were already reported)
lived in main.mjs, so no test could fail on it: dropping the
once-per-shape check, or keying every set the same, left every test
green. The warning did not say which set it was about (only a program
tag on Floor lines), and it gave the same text for a set ejected down
to its threshold, or registered n-of-n before finding 96. There one
rogue guardian alone locks governance; a lost key alone does not, since
dead guardians cannot veto the owner's new set (docs/16 §4.4), but it
leaves no quorum until that set executes. Nothing was said when the owner's
new set fixed the problem. And a delivered alert did not say which Floor
program it was about: the program went only into the line's `program`
field, which alertFor uses in its repeat key but does not show.
Moving the program name into the text is a trap: alertFor keys repeats
on the first 60 characters of the text, and "the Floor guardians of
<44-character id>" pushes the shape past them. A 2-of-3 set ejected to
2-of-2 within ten minutes would then give the same key, and the worse
warning would not alert. So the program goes after the shape: "the
Floor guardians are 2-of-3 for <program>: ...". The `program` field
stays for the repeat key.
guardianSetWarning becomes guardianSetCheck(shapes, which, set,
mainnet) in policy.mjs. It keeps each set's last shape and whether it
was short in the Map, keyed by "protocol" or the protected program, and
returns the line to log or null. The warning names the set and has its
own text for n-of-n. When a set that was short gets the margin, it
returns one line saying so, which alerts on mainnet like the warning,
so whoever was paged hears that it is fixed. main.mjs only logs what it
returns. docs/07 §6 says so. The advice also had a hole: a set
registered as 7-of-8 was told to "run at least 7-of-9", and no set can
have 9 keys (core MAX_GUARDIANS is 8). When threshold + 2 is over 8,
the advice is now to lower the threshold to at most 6-of-8. That needs
MAX_GUARDIANS in JS, so lib/solana.mjs gets it next to
GUARDIAN_MARGIN; the next commit ties both to the core.
Evidence (tasks/floor-guardian-margin/):
- node --test cloud/test/*.test.mjs: 180/180 pass (the test is
extended, not added).
- 19 mutants, each run on policy.test.mjs and reverted (r6/mutants.py,
r6/mutants.log, r6/N5-margin1.log). Each fails the extended test, and
only it: never warn, threshold fixed at 2, notify always, notify
never, GUARDIAN_MARGIN = 1, no once-per-shape, one Map key for every
set, the n-of-n text for threshold + 1, no fixed line, a fixed line
for a set that was never short, the fixed line never alerting, no
program tag, every set named as the protocol's, the short flag not
stored, the program before the shape, the program left out of the
text, `need < 8` for `need <= 8` (6-of-7), the 8-key limit ignored
(7-of-8), 7-of-8 as the lowered advice.
- The mainnet alerts are checked on their own in a scratch copy of the
test without the earlier text assertions (r5/C1-C3-alone.log): with
the program left out of the text, the alert's detail check fails;
with a long name pushing the shape past character 60, "2-of-2 alerts"
fails.
- A's review of the previous version (branch
auto/B/floor-guardian-margin-r7, kept) found two survivors there: X3,
the shape keyed on the count only, and X5, the advice
threshold-of-(n+1). It also found that the n-of-n text said a lost key
alone locks governance. The test now has a threshold-only fix (7-of-8
-> 6-of-8, the advice's own step) and pins the n-of-n text and advice.
r9/mutants.log: X3, X5, the r7 n-of-n text and the text without its
lost-key clause are all KILLED. r9/mutants-r7.log: X3 and X5 SURVIVED
on the r7 code and test. The 18 mutants of r6/mutants.py, re-run on
this code: all KILLED (r9/r6-mutants-rerun.log). Two code-review
passes on the fix (kept as -r8 and -r9) added the lost-key clause: a
lost key alone leaves no quorum (it never approves, and core `quorum`
needs threshold approvals) until the owner's new set executes. A third
pass (-r10) fixed the comments' wording only.
- tools/claims.py: OK, client 180.
- Still not tested: the checkFloor call in main.mjs (no test runs
checkFloor).
Not changed (code review): the "fixed" line needs the short shape to
have been seen since the guardian started (the Map is in memory, as the
old per-process shape was; docs/07 says so). A protocol that abdicated
SetGuardians can't take a new set, so the advice to run 2-of-4 can't be
followed there; REPORT lists that case for the owner. The fault text
assumes a margin of 2, the owner's decision (16 §4.4).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Tie the JS guardian margin and set size limit to the core constants
cloud/lib/solana.mjs keeps its own GUARDIAN_MARGIN = 2, which the
guardian and the deploy tooling use to judge a guardian set, and now
MAX_GUARDIANS = 8, which the guardian's advice uses. Their comments say
they are core governance.rs's constants, but nothing checked that. Had
the programs' margin moved to 3, every JS test would have stayed green
while the guardian passed sets the programs refuse.
deploy.test.mjs now reads governance.rs and requires exactly one
`pub const GUARDIAN_MARGIN: u8 = N;` and one `pub const MAX_GUARDIANS:
usize = N;`, each N equal to the JS constant. It reads the source text,
like the build-script test above it. If a declaration changes form, the
test fails rather than passing.
Evidence (tasks/floor-guardian-margin/r6/tie-mutants.log):
- node --test cloud/test/*.test.mjs: 181/181 pass (one new test).
- Mutants of governance.rs, each reverted with its mtime kept (else the
limiter differential asks for a worker rebuild): GUARDIAN_MARGIN = 3,
MAX_GUARDIANS = 9, and a trailing comment on MAX_GUARDIANS. Each fails
the new test, naming the constant. deploy.test.mjs is green again.
- tools/claims.py: OK, client 181 (docs/09:40 updated).
Sibling search: the other JS copies of core rules (the limiter port,
the account layouts) are checked by the differential and by decoding
real accounts. MIN_GUARDIAN_THRESHOLD has no JS copy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Reach the 7-day extension total in the wide differential
differential_wide.rs drew extension seconds only from {0, 1, 3600,
86400, 86401} on a random ticket and lumped every refusal into one
counter. A ticket never got near the 7-day total, no negative value was
sent, and a refusal was not checked by its code. So the default run
could not tell the total bound, a negative extension or the error code
from a bug: only the unit tests at vault.rs:3382-3401 pinned them.
Now one extension draw in three pushes the chosen ticket by up to 24h
a call to exactly the 604,800 s total, then asks for 1 s more, so the
edge itself is hit on every push, not only when earlier draws line up.
The other draws are half legal sizes (1, 3600, 86400) and half out of
range for one call (i64::MIN, -86400, -1, 0, 86401, i64::MAX). Each
refusal must fail with EXTENSION_TOO_LONG. The counters are split into
extend (a random legal size), extend_push, extend_refused_call and
extend_refused_total, all required >= 10, so pushes cannot stand in for
the random sizes.
The model reads both bounds from bulkhead-core (consts.rs:37-38), as
the module says it is built, rather than restating 86,400 and 604,800.
docs/09 §7's row says so, and the 16 Oct "Vault refusals deleted" row,
which says the default run never extended a ticket that far, now says
it does since this change (its differential_wide.rs:406 cite had drifted).
Default run (24 seeds x 250 steps): extend 16, extend_push 126,
extend_refused_call 16, extend_refused_total 30 (was extend 40,
extend_refused 27); every other required counter still >= 10
(loosen_executed 12, as before).
Evidence: mutants of brakes.rs extend_ticket, each built and run against
the old and the new differential_wide only:
T1 drop the total check old SURVIVED new KILLED
T2 total > to >= old SURVIVED new KILLED
T3 drop secs <= 0 old KILLED new KILLED
T4 secs <= 0 to < 0 old KILLED new KILLED
T5 total ignores this call's secs old SURVIVED new KILLED
T6 extended_s += secs / 2 old SURVIVED new KILLED
T7 refusal returns WrongTicket old SURVIVED new KILLED
T8 secs <= 0 to == 0 (negatives) old SURVIVED new KILLED
T9 total > to > total + 1 old SURVIVED new KILLED
(T9: a draft that pushed by 24h until refused killed it only through
chance 1 s and 3600 s draws on the same ticket; the push to exactly
604,800 then 1 s more reaches the edge every time.)
sBPF sha256 after the run unchanged (Floor 8ca8a608, Harbor 543be60b,
Vault 7086148b): tests only.
Sibling search: the other refusal counters in this file (reset_refused,
lifeboat_refused_*, loosen_*) are each split by cause already.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check settle and lifeboat refusals by code in the wide differential
The same gap as the extension refusal, found in the sibling search for
the previous commit: a refused settle and the three lifeboat-move
refusals asserted only r.is_err(). Any refusal passed, so the order of
the checks and the program's own codes went unchecked by the
differential. Settle's comment (custody.rs:520-521) says it returns its
own error, not the token program's, when the vault is short; nothing in
this file held it to that.
Now a settle refused while frozen must fail with FROZEN, and one that is
only short with INSUFFICIENT_BALANCE (frozen is checked first). A
lifeboat move inside the delay after the seal must fail with NOT_SEALED,
one while frozen with FROZEN, and one beyond the balance with the token
program's InsufficientFunds (1). The two settle refusals get their own
counters: settle_refused_frozen is required >= 10 like the rest;
settle_refused_short needs lifeboat moves to drain a due ticket's
tokens first and occurs 7 times in the default run, so it is required
>= 1.
Evidence: mutants of programs/vault/src, each built and run against the
previous head's differential_wide and this one, nothing else:
S1 settle: drop the InsufficientBalance check old SURVIVED new KILLED
S2 settle: balance check before the freeze old SURVIVED new KILLED
L1 require_sealed returns Frozen old SURVIVED new KILLED
L2 move_in's freeze returns NotSealed old SURVIVED new KILLED
The token-program code (1) on a move beyond the balance is pinned but
not shown red by a mutant. sBPF sha256 unchanged (Floor 8ca8a608,
Harbor 543be60b, Vault 7086148b): tests only.
Deep run of both commits together (BULKHEAD_PROGRAM_SEEDS=600
BULKHEAD_PROGRAM_STEPS=500, 85 s): 300,000 steps, all identical;
extend 876, extend_push 6,462, extend_refused_call 1,175,
extend_refused_total 1,453, settle 2,024, settle_refused_frozen 921,
settle_refused_short 566, lifeboat refusals 3,854 + 5,768 + 1,171. docs/09 §7's
row gets that run as a 16 Oct result next to the 5 Oct one, which
predates these checks.
Sibling search: in differential_wide.rs, reset (TICKETS_PENDING) and
execute change already asserted codes, and a release or quote refused
for want of tokens asserts INSUFFICIENT_BALANCE. A release or quote the
limiter refuses (frozen, lane closed, queue full) is still compared only
as a refusal: mapping the core's refusal to the program's code is left
for its own change. Lifeboat returns only send legal amounts. docs/09's
row now says which refusals are checked by code. differential.rs:327
has the same is_err settle check; left as it is, since the wide run now pins both
settle codes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check limiter refusals by code in the wide differential
A release or quote that the limiter refused (lane closed, queue full)
was compared with the model only as Err. Any refusal code passed: a
program that swapped LaneClosed and QueueFull in custody.rs's
limiter_error, or checked the backed part against the balance before
the limiter and so answered InsufficientBalance for a full queue,
still matched the model. About 670 such refusals happen in a default
run.
The model now maps core's LimiterError to the program's code with
verr::limiter in the test harness (all four variants, as custody.rs's
limiter_error does), and every refused quote and release must fail
with the model's code. Lane-closed and queue-full refusals have their
own counters, each required at least 10 times; a refusal code with no
counter panics rather than being counted as want of tokens. docs/09
row 362 says so and records the deep run.
Evidence (autopilot tasks/differential-limiter-codes/):
- mutants.log: C1 (LaneClosed and QueueFull swapped in
limiter_error), C2 (QueueFull -> InsufficientBalance), C3
(LaneClosed -> ZeroAmount) and C4 (backed > balance checked before
the limiter, same line) each SURVIVED the old test and are KILLED
by the new one. Hashes back to 8ca8a608/543be60b/7086148b after.
- c4-full-suite.log: under C4 the whole program suite (Vault 113,
Floor 50, the fixed differential) passes; only this test fails.
C1-C3 are also caught by vault.rs's code tests (:434, :3428).
- Default run: refused_lane_closed 254, refused_queue_full 414,
insufficient 118.
- deep-600x500.log: 300,000 steps, 15,561 lane closed, 18,853 queue
full, 6,345 want of tokens, all identical.
Sibling search: every other Err arm in differential_wide.rs already
asserts a code (extend, settle, reset, lifeboat in, execute change).
Not here: lifeboat returns are never refused (backlog), and lifeboat
move_in beyond the balance fails with the token program's code
(backlog, a program change).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check release refusals by code in the fixed differential
differential.rs reduced every refused release to Err(()) on both
sides, so a program that refused for the wrong reason still matched
the model. The same gap the previous commit closed in the wide
differential (found by its code review).
The model's refusal now carries the program's code: verr::limiter for
a limiter refusal, INSUFFICIENT_BALANCE when the vault holds less
than it would pay. A refused release must fail with that code, and
the assert prints the program's error. Refusals for a full queue get
their own counter, required at least 10 times like the others. Want of
tokens and a closed lane are counted and printed but not required:
the wide differential requires both. A refusal with any other code
panics, as in the wide differential, so the counts always add up.
docs/09 row 361 says so; its undated figure (4,843 releases) no longer
matched a run, so it is replaced by a dated one.
Evidence (autopilot tasks/differential-limiter-codes/):
- mutants-narrow.log (mutants.sh <wt> differential): C1 (LaneClosed
and QueueFull swapped), C2 (QueueFull -> InsufficientBalance) and C4
(backed > balance checked before the limiter) SURVIVED the old test
and are KILLED by the new one. C3 (LaneClosed -> ZeroAmount)
survives both: no release in this test's runs is refused for a
closed lane. The wide differential kills it, and docs/09 says so.
Hashes back to 8ca8a608/543be60b/7086148b after.
- r2/default-run.log: 1,000 steps, 82 releases refused, all for a
full queue (want of tokens 0, lane closed 0).
-…
shuv-amp
added a commit
that referenced
this pull request
Oct 18, 2026
* Run the tests on 2-of-4 guardian sets; test recovery from two faults
Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set
with fewer than threshold + 2 keys. Every test fixture and most tests'
own sets were 2-of-3. This moves them to 2-of-4, which today's rule
also accepts, so this commit changes no program: Floor, Vault and Harbor
hash 508e5a99 / b07655d9 / 543be60b, as before.
- Fixtures: Protected (Floor) and Market (Vault, through Harbor) now
hold 4 guardians. Each test that built its own 3-key set gets a 4-key
set with the same threshold and the same assertion.
- Two tests changed meaning with a 4th key and were fixed, not loosened:
a_quorum_ejects_a_rogue_guardian_who_vetoes_everything (Floor) and
a_quorum_ejects_a_guardian_who_freezes_forever (Vault). With 4 keys,
the last eject was refused only because the target's own vote is
masked. Both now eject a second guardian (a set an ejection shrank to
threshold + 1 still ejects) and then show the 2-of-2 refusal.
- Account sweeps: an eject's signers may now be any two of the three
others, so each of those two slots is free for the third, never for
the target or the other signer again. Vault 5,365 and Floor 1,710
substitutions, refused (docs/09, dated 16 Oct). Listing all three
others at each slot would have skipped the duplicate: with a mutant
that counts a guardian listed twice as two (governance.rs:270), the
sweeps listing all three pass, and these fail at "eject: account 2
replaced by key ..." (Vault and Floor). No other program test kills
that mutant (tasks/aud2-guardian-margin-impl/dup-mutant-*.log).
- floor_differential: the guardian pool is 6 keys, 4 registered; the
proposed sets are 2 to 6 keys.
New tests (pass on both rules; they show what the margin buys):
- a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_set (Floor): g0's
key lost, g1 rogue stops the program; the lost key does not help a
quorum; the honest pair ejects g1 and restores; neither can veto the
owner's refill, which executes after 72h.
- two_rogues_in_turn_are_both_ejected_from_a_two_of_four_set (Floor):
the second rogue vetoes the refill and is ejected from the 2-of-3.
- a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_vault: the same in
the Vault (freeze, eject, unfreeze, refill).
Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest
and quote sweep): the counts and the sweep figures here are from runs
there (tasks/aud2-guardian-margin-impl/r4/). The E2E run, the mutants
and the new-rule run below were made before, on e8640a7; this commit's
programs and core are the same as there but for one comment line in
views.rs (line-neutral, same hashes).
Evidence:
- E2E=1 scripts/check-programs.sh: PROGRAM CHECKS PASSED (Floor 49,
Vault 98; S1 and E2E passed; the resumed setup found all 14 steps;
vault check, Floor demo and deploy dry run passed), with the next
commit's deploy rehearsal in the tree.
- tools/claims.py: OK (floor 49, vault 104, programs 153).
- Under the new rule (core from two commits on, programs rebuilt), only
the two tests that state the old rule fail: an_n_of_n_guardian_set_is_
refused (accepts 3-of-4) and the Floor differential's model.
- Mutants (tasks/coverage-map/mutants.sh), each red, then restored:
AV1 Floor live_guardian ignores liveness: KILLED by a_lost_key_and_a_
rogue_do_not_lock_a_two_of_four_set (+2 others).
AV2 the Vault's: KILLED by ..._two_of_four_vault (+2 others).
AQ1 Floor restore quorum ignores liveness: KILLED by ..._two_of_four_set
and the differential only.
AQ2 the Vault's unfreeze quorum: KILLED by ..._two_of_four_vault and
a_silent_guardian_does_not_count_toward_a_quorum.
AE1 can_eject refuses at count == threshold + 1 (AUD-2 option (a)):
KILLED by two_rogues_in_turn_..., and the two eject tests above.
Hashes after: 508e5a99 / 543be60b / b07655d9.
Split on 16 Oct from ec0cacc at the owner's review (D2): the deploy
rehearsal's move to 4 guardians is now the next commit. programs/ and
programs/tests/ here are byte-identical to ec0cacc, so its runs below
hold for this commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Rehearse deploys with four guardians, and read only the keys there are
Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set
with fewer than threshold + 2 keys, so the deploy rehearsal must
register 2-of-4 sets. Split from ec0cacc at the owner's review (D2);
the text is that commit's, unchanged:
Deploy rehearsal (cloud/deploy market.mjs, vault-check.mjs,
floor-demo.mjs): 4 guardians (keys/guardian3), so the dry run keeps
working once the rule lands; one more funding step (14, was 13).
The devnet market has 3 guardian files. A first version of this change
loaded guardian3 everywhere: vault-check.mjs failed there with ENOENT,
and heartbeat silently created a key nobody holds as a guardian. Now
market.mjs guardianKeys(dir, create) reads guardian0, 1, ... and
throws on a gap (never fills it with a new key). Setup creates keys up
to 4 only while the protocol is not registered, so resuming an old
market adds no key the chain lacks; a new deploy-dryrun step resumes
the dry-run market with guardian3 moved aside and fails if setup makes
it again (with setup always creating: "DRY RUN FAILED: resuming a
registered market made keys/guardian3.json", r3-dryrun-J7.log).
renew-guardians tops up to 4 and logs each new key (unfunded); a
2-of-3 renewal would be refused once the rule lands. keysIn reads the
guardians only when a command uses them, so caps and set-recovery
never need them; heartbeat says a key is not in the set rather than
dead. One THRESHOLD (2) for the register and renewal payloads and
MIN_GUARDIANS. vault-check.mjs
compares the chain's set with the first files, and liveness with the
chain's count. An extra file (a renewal not executed yet) and a set
without the margin are info lines, not failures, since such a set
keeps working. GUARDIAN_MARGIN and guardianMarginProblem live in
cloud/lib/solana.mjs, used by these and by the guardian bot's existing
warning (cloud/guardian/main.mjs), and vault-check uses THRESHOLD. The e2e scripts fund guardian3 too. Three
tests in cloud/test/deploy.test.mjs (client tests 176 -> 179). JS
mutants (js-mutants-r3.log), each red then restored: always create,
never create, no gap check, margin 1, `>`, GUARDIAN_MARGIN = 1.
Setup always creating survives these tests and is killed by the dry
run step above.
cloud/, scripts/deploy-dryrun.sh and docs/09's client-test row are
byte-identical to ec0cacc (git diff empty), so its runs hold for them:
the E2E check-programs run with the dry run (r4), and the JS mutants.
tools/claims.py: OK (client 179).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Refuse a guardian set without two keys beyond its threshold (107)
AUD-2, decided by the owner on 16 Oct (option (e) of 16 §4.4). A set
stays live while its honest live guardians reach the threshold: they
eject every rogue in turn, and dead guardians cannot veto the owner's
new set. Finding 96 asked for one key beyond the threshold, so a 2-of-3
or 3-of-4 set survived one fault but not a lost key plus a rogue: the
honest guardian left is no quorum, nobody can be ejected, and the rogue
keeps Floor stopped (or the Vault frozen) and vetoes every new set.
Fix, in the core so both programs inherit it through GuardianSet::parse
(registration, and every SetGuardians when proposed and when executed):
GUARDIAN_MARGIN = 2
guardian_set_ok := count <= 8 && t >= 2 && count >= t + 2
(saturating_add: t is a payload byte). A stored set is not checked
again, so existing 2-of-3 sets keep working, and ejection is unchanged:
a shrunk set still ejects. A 2- or 3-key set is BadThreshold, not
BadCount; the count rule is only the bitmap size, and the error order
of finding 103 is kept.
This changes both programs, by intent:
Floor 508e5a99 -> 8ca8a608, Vault b07655d9 -> 7086148b,
Harbor 543be60b (unchanged). Darwin-arm64 builds.
Compute (scratch LiteSVM test, tasks/aud2-guardian-margin-impl/cu.md):
-3 CU on every path that parses a set (Floor propose 2035 -> 2032,
execute 1682 -> 1679; Vault register 9592 -> 9589, SetGuardians 2979 ->
2976; Floor register varies with its PDA bump, residue also -3).
Proof and tests:
- Kani: accepted_sets_can_eject_anyone restated (ejection reachable
exactly when count > t, so option (a) cannot slip in; accepted sets
have >= 4 keys and a margin of at least 2, and 2 is accepted). New:
an_honest_live_quorum_ejects_any_rogue (any slots, anyone else alive)
and accepted_sets_survive_any_two_faults (any two faulty keys, lost
or rogue, leave a live honest quorum, which ejects each rogue, and the
rogues alone eject no honest guardian). That holds only short of a
rogue quorum: two rogues in a 2-of-n set are one (out of scope, 04
T7). With the assumption rogue < t dropped, the harness fails at
`!can_eject(rogue, ..)` (kani-no-rogue-assume.log), so the docs say
"short of a rogue quorum", not "any two faults".
19/19 verified, 240 s (cargo kani -j 2, as CI); again after the
restatement, 235 s summed (r2-prove.log).
- Kani mutants (kani-mutants.sh): margin 1, margin 3, `>` for `>=`,
t >= 1, count <= 9, option (a), and an eject one vote short (only
the two-fault harness kills that one): each KILLED by a harness
(kani-mutants-r2.log). The eject
ignoring liveness is the eject harness's case, not these.
- Program mutants (coverage-map/mutants.sh, rebuilt each time):
margin 1: KILLED by a_guardian_set_without_two_spare_keys_is_refused
(Floor and Vault), registration_rejects_weak_settings and the Floor
differential. Margin 3 and `>`: KILLED by most of the suite (the
fixtures are 2-of-4). t >= 1: first only the Floor differential; the
1-of-2 registration case was now refused by the margin too, so it is a
1-of-4 here and a Vault 1-of-4 case was added: then KILLED by both.
count <= 9: SURVIVES the program tests, equivalent there (parse refuses
more than 8 keys first), KILLED by Kani. Option (a): KILLED by the
eject tests two commits back (the tests on 2-of-4 sets).
- Tests: the Floor and Vault refusal tests above cover proposal and
execution (a stored 2-of-4 proposal whose threshold turns 3 is refused
when executed, as the devnet renewals proposed before this rule will
be); registration_rejects_weak_settings gains 2-of-3 and 3-of-4.
an_n_of_n_guardian_set_is_refused (Vault) is renamed and accepted a
3-of-4: it now refuses it and accepts 3-of-5.
- Model: the Python rule, the Floor differential's model, and the
guardian_set fuzz target state it again; the GP generator now also
picks count - 2. With the Python rule put back to the old one, seed 11
(20,000 cases) reports 5 GP mismatches; restored, seeds 11-13 identical.
Fuzz seeds two_of_three_refused, two_of_four, six_of_eight, and
seven_of_eight renamed seven_of_eight_refused (byte 323, the
threshold, is 7: now refused). With margin 1 in the core the target
crashes within 60 s; restored, 222,615,288 runs in 601 s, no crash.
- scripts/check.sh: ALL CHECKS PASSED, and E2E=1 check-programs.sh:
PROGRAM CHECKS PASSED, both at the branch tip, whose code is this
commit's (the next commit is docs only; tasks/aud2-guardian-margin-
impl/r4/check.log, r4/check-programs-e2e.log). claims: kani 19,
floor 50, vault 104, programs 154, client 179.
Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest
and quote sweep). The core and the program sources are the same as
before the rebase but for one comment line in views.rs (line-neutral),
so the Kani, fuzz, differential and mutant runs above were made on the
pre-rebase commit; the hashes, check.sh, check-programs and the counts
are from r4 runs (tasks/aud2-guardian-margin-impl/r4/).
The finding 96 row said the restated harness proves "exactly"; it now
says what it proved then and proves now, and gives the Vault test's new
name. The finding 107 row waits for rows 105 and 106 to land (09 §4 is
numbered 1..n); its text is ready. The rule's prose is the next commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* State the guardian margin in the docs, with its costs (AUD-2)
The previous commit made both programs refuse a new guardian set
without two keys beyond its threshold (finding 107). The docs that
state the rule now say so:
- README: guardians 4-8 keys; the G5 row and the "who the guardians
are" paragraph give the reason (any two faults leave a quorum, unless
the rogues are one themselves: two in a 2-of-n set, 04 T7).
- 05 register_protocol: 2 <= threshold <= count - 2, also for every
SetGuardians, proposed and executed. 05 abdicate: abdicating
SetGuardians fixes the set, so ejections shrink it for good.
- 06 state table, 00 D22, FAQ: the same rule.
- 07 §2: 4-8 guardians, and don't abdicate SetGuardians. 07 §6: the
programs enforce the margin; sets registered before keep their
exposure until replaced; an ejection uses up a spare key until the
owner's new set executes; a lost key counts as live until its
interval passes.
- 16: the AUD-2 row is decided; §4.4 gives the decision, why the
margin is the number of faults a set survives (short of a rogue
quorum), the options rejected
and why (the honest-owner and stolen-owner cases look the same
on-chain), what it costs, what it leaves (a rogue quorum; a stolen
owner key with every guardian dead; a rogue's veto of the recovery
key's rotation, checked: Floor's veto has no recovery exception, as
a_vetoed_recovery_proposal_does_not_hold_the_owner_s_slot shows), and
Lido-style lapsing vetoes as a future option only. §5: "independent
guardians, 3-of-5 or more".
- 10: the guardian set's minimum size is no longer an open decision.
SECURITY.md and 08 describe what is on devnet, which still runs the
old rule, and are left as they are.
claims: OK. Docs only; no program file changed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Say the weekly CI run skips mutants, as its condition does
ci.yml's header and scripts/ci-changes.sh said the weekly run does every
job, and 09 §8 gave `mutants` as running on any change to the core. The
`mutants` job's `if` starts with `github.event_name == 'pull_request'`,
so the Monday schedule (and a manual dispatch) never runs it: it mutates
a pull request's diff and has none to mutate. Its trigger list in 09 §8
also left out scripts/mutants.sh and .cargo/mutants.toml, which
ci-changes.sh (line 22) selects it for.
Comments and docs only; nothing runs differently.
Sibling search: `git grep` for "weekly run does", "every job ... weekly"
and "mutants ... weekly" across md/yml/sh/mjs/py. The other hits are
09 §8's "at full depth every Monday" (its table now says which job is
not weekly) and the dispatch input's "every job at weekly depth", which
is the same set as the weekly run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check that the mutant exclusions still match; repin murmur3 (finding 105)
.cargo/mutants.toml excludes the core's 22 equivalent mutants, and the
docs say so. Since 9 Oct it excluded 19. The murmur3 tail's pattern is
pinned by line, `floor.rs:(235|238|241)`, because the body's `h ^= k`
has the same column and function and is not equivalent. 1dc7861 (#52)
moved those lines (now 327/330/333), and nothing checked the patterns.
Failing closed, the three would have failed CI's `mutants` job on a
pull request that changed those lines. Failing open, a pattern that
starts matching a non-equivalent mutant hides it from every run.
Root cause: nothing checked that a pattern still names its mutants.
Fix:
- each pattern ends with the number of mutants it excludes (`# 3`), and
the murmur3 pattern names 327/330/333;
- tools/mutant_exclusions.py fails unless each pattern matches exactly
its number in `cargo mutants --list --no-config`, and cargo-mutants
(the list with the config) excludes exactly the mutants the patterns
match, so a pattern this parser can't see, or reads differently, fails
too. No two patterns may match the same mutant, since the docs' count
is the counts' sum, and the kept list must be part of the full one.
It reads only the exclude_re array's body, needs no tomllib
(Python 3.9), and names colour codes in the list as the problem;
- scripts/mutants.sh makes both lists (--colors never: CI sets
CARGO_TERM_COLOR=always) and runs it first, except when the caller
passes --config or --no-config (as whole arguments). CI's `mutants` job runs that script;
ci-changes.sh now also selects the job for the new tool;
- scripts/check.sh runs its --self-test on planted configs;
- tools/claims.py ties the docs' "22 equivalents" (README, 09 §6 and the
summary row) to the sum of the counts, with the same parser. It finds
that parser from its own directory, so `python3 -I` runs it too, and
check.sh runs its self-test that way. 00's
"18 equivalents" (26 Sep log entry) and run 9's "exactly the 22
equivalents below" (6 Oct) are kept as dated records.
Docs: 09 §4 finding 105, §6 run 10 and the "In CI" note, §8's mutants
row; README and 09's summary row 669 → 797 mutants (run 10); README and
12's post-submission range 96–105.
Evidence (first on 1f2ad48 + the parent, logs in the autopilot's
tasks/mutants-exclusions-r2/; re-run on 58697c6 + the parent,
tasks/mutants-exclusions-r3/):
- `cargo mutants --list`: 797 with --no-config, 778 with the old config
(19 excluded), 775 with this one. The 22 excluded are §6's list, the
murmur3 ones at 327/330/333.
- scripts/mutants.sh --list: exit 0, "22 of 797 mutants excluded"; with
CARGO_TERM_COLOR=always also exit 0. Old config: exit 1, every pattern
"no '# N' count". Planted on the real lists: the old lines with a
count (matches 0, says 3) and the body XOR added (matches 4, says 3)
both exit 1; in both the set comparison alone passes. /usr/bin/python3
(3.9.6): self-test OK and the real check passes.
- The self-test pins each problem by its message, not a count. Mutants
of the tool (tool-mutants.log), each red: M1 no per-pattern check,
M2 no set comparison, M3 no colour check, M4 no empty-array check,
M5 a missing count accepted, M6 a missing count read as 0, M7 the
parser reading past the array. M3 and M5 survived a first version
that checked only the number of problems.
After A's review (16 Oct): the set comparison was pinned only by its
size, so `len(excluded) != len(matched)` passed the self-test. A
case with as many excluded as matched but not the same ones now
fails it (15 cases; red with that mutant, green without), and the
message says how many mutants are in one set only.
From the code review after the rebase: colour codes only in the kept
list were reported as a kept list made with other options; the
colour check now reads both lists (16 cases; dropping the kept half
fails the new case).
On 58697c6, with the overlap and kept-list checks: M8 no overlap
check and M9 no kept-list check, red too, as M1–M7 still are. A
duplicate of the can_eject pattern planted in the real config:
scripts/mutants.sh --list exits 1, "the counts add up to 23, the
patterns match 22 mutants" (planted-overlap.log). Without the
sys.path line, `python3 -I tools/claims.py` fails with
ModuleNotFoundError; with it, both claims runs pass.
- claims.py (claims-mutants.log): one count 1 → 2 fails four statements
(README 188; 09 33, 333, 335; kept-run9.log), not run 9's 331, which
failed too before it was kept; a dropped count fails with the
tool named; deleting any of the three rules fails a self-test case;
deleting the KEPT entry fails 00:103.
- The 22 excluded mutants run alone (--no-config, --re per pattern,
2 min): 22 missed, exactly the excluded list; --re also picked
ema.rs:70 (a field deletion, no pattern matches it), caught.
- Run 10 (f1f05ba, old config, -j4, 46 min; tasks/mutants-run10/):
778 tested, 712 caught, 5 timeouts, 58 unviable, 3 missed, exactly
the murmur3 tail. The core's source is unchanged since (`git diff
--stat f1f05ba HEAD -- crates/bulkhead-core/src` is empty; only
tests/units.rs gained finding 102's tests).
- No program or core source changed.
Rebased on 16 Oct onto cede705 (findings 102 and 100, the attest
sweep and AUD-2). AUD-2 changed the core's source (governance.rs), so
the line above saying it is unchanged since run 10 held on 58697c6, not
here. On cede705: scripts/mutants.sh's list check passes, "22 of 795
mutants excluded, each pattern matching its count"; the murmur3 lines
are still 327/330/333. The 797 in README and 09 is run 10's figure:
README now dates it ("in the last full run, 15 Oct") and finding
105's row says the core has changed since (finding 107).
The setup line in scripts/mutants.sh now pins cargo-mutants 27.1.0,
the version ci.yml installs: each count is the number of mutants that
version lists, so another release can break the check (code review,
16 Oct; `git grep 'install.*cargo-mutants'` finds no other unpinned
install).
The check's report goes to stderr in scripts/mutants.sh, so
`scripts/mutants.sh --list` prints only cargo-mutants' names: before,
its first stdout line was the report; after, stdout is 773 lines
(795 - 22) and stderr the one report line. ci.yml's `mutants` comment
says the job checks the exclusions first.
Row 105 and 09 §6 gave the pin's reason as the body's XORs. The
finalizer's are at column 7 (`grep -n '\^=' floor.rs`, 337-342); the
non-equivalent XOR at column 11 is `h ^= k`, in the body (320) and after
the tail (335). The toml comment, the row and §6 now say that. 09 §8's
`check` row names the two tool self-tests check.sh runs.
README no longer calls run 10's 797 "injected bugs": cargo-mutants
listed 797 but tested 778, 19 being excluded (09 §6, run 10's row).
Sibling search: the other 14 patterns use `\d+` for the line, and each
matches the count now written after it. The murmur3 one is the only
line-pinned pattern.
Not done, by choice:
- Catching an edit that keeps a pattern's number but changes its
mutants (an excluded line itself edited, or a murmur3 body XOR moved
onto a pinned line). The toml and 09 §6 say so: such an edit needs the
mutant judged again by hand. Failing on any diff that touches an
excluded line would catch the first; it is a separate change.
- Moving murmur3's tail into its own function, so its pattern needn't
name lines. It shifts core lines, so it changes the program hashes.
Until then a line move above it fails the check, with a message
saying what to fix.
- A weekly full mutants run in CI. It took 46 min here at -j4, likely
past the job's 90 min on two CI cores: a minutes call for the owner.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that a ticket stays in its own protocol and vault
Two Vault checks that keep a ticket to its own protocol and vault had no
test. Deleting either one left every program test green
(tasks/error-codes/mutants.log, base cede705):
- brakes.rs:79, extend_ticket: tk.protocol != protocol. Without it, a live
guardian of any other protocol extends this protocol's tickets. Anyone
can register a protocol and name its guardians, so that guardian could
hold back every queued payout up to the total extension limit. The
sweep did not catch it: it has no second protocol's guardian to swap in.
- custody.rs:501, settle_ticket: tk.vault != vault. A USDC ticket settled
through the SOL vault (its tokens, its mint) got past the Vault's checks.
New test another_protocols_guardian_cannot_extend_a_ticket: a second
Market's live guardian extends this market's ticket -> WRONG_VAULT; the
same instruction from this market's guardian succeeds. The settle test
now settles the USDC ticket through the SOL vault and expects
WRONG_VAULT.
Evidence (tasks/error-codes/mutants.log, STATE=c1; full program suite
per mutant):
W02 brakes.rs:79 deleted -> KILLED by
another_protocols_guardian_cannot_extend_a_ticket ("expected
failure, got success": the stranger's extension went through)
W06 custody.rs:501 deleted -> KILLED by
a_ticket_settles_after_its_eta_to_its_stored_recipient_only (8049
InsufficientBalance from the empty SOL vault, not 8010)
Both were SURVIVED at base. No program source change: hashes
8ca8a608/543be60b/7086148b, as at cede705.
Sibling search: the other 12 WrongVault sites were deleted one at a time
in the same run. Eight are KILLED by the sweep (brakes.rs:39, lib.rs:320,
custody.rs:160, genesis.rs:491, views.rs:27, lifeboat.rs:151,
governance.rs:169/259). lib.rs:324 (vault PDA address), genesis.rs:523,
lifeboat.rs:71 and :247 survive; later commits on this branch deal with
them.
Counts: Vault 104 -> 105, programs 154 -> 155 (claims.py OK).
The test also reads the ticket's ETA: unchanged after the
refused extension, 3,600 s later after its own guardian's (code-review
fix: success alone did not show the extension happened).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that the seal names every vault and happens once
Two seal checks had no test; deleting either left every program test
green (tasks/error-codes/mutants.log, base cede705):
- genesis.rs:486, execute_seal: the pairs must number 3 * vault_count.
Without it, the seal checks only the vaults it is given. A vault left
out skips its claims-vs-balance check, so an uncovered import seals.
docs/05's seal row says every vault is passed exactly once.
the_seal_checks_each_vault_once covered a vault named twice in place of
another, not one left out.
- genesis.rs:431, propose_seal: refused once sealed (AlreadySealed).
Without it, a sealed protocol's admin re-proposes, which rewrites
seal_executable_s/expires_s and clears seal_vetoed on a sealed protocol.
the_seal_checks_each_vault_once now also leaves the uncovered USDC vault
out (expects VAULT_COUNT_MISMATCH) and, after the seal, proposes it again
(expects ALREADY_SEALED). No new test, so no count change.
Evidence (tasks/error-codes/mutants.log, STATE=c2, full program suite):
E05 genesis.rs:486 deleted -> KILLED by the_seal_checks_each_vault_once
("expected failure, got success": the seal went through without
the USDC vault and its 1,000 USDC uncovered import)
E04 genesis.rs:431 deleted -> KILLED by the_seal_checks_each_vault_once
("expected failure, got success")
Both were SURVIVED at base. Vault hash 7086148b unchanged (tests only).
Sibling search: the seal's other checks. :491 vault_tokens (KILLED by the
sweep), :519 lifeboat owner and :523 lifeboat mint/owner (SURVIVED; the
lifeboat is a PDA checked by expect_pda just above, which only this
program can create, as its own token account: see this branch's note in
docs/09).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that invalid limits and asset settings are refused at every entry
Three bound checks had no test; deleting any one left every program test
green (tasks/error-codes/mutants.log, base cede705):
- genesis.rs:171, create_vault: config_is_valid (bps <= 100%, windows
> 0). Without it, a vault is created with a 100.01%-an-hour burst.
- governance.rs:40, parse of a SetLimits change (at proposal and again
at execution, :254 and :478): the same rule for a proposed change.
- governance.rs:47, parse of an UpsertAsset change: valid_asset (haircut
<= 100%, liability >= 100%, the oracle bounds of finding 88). Only the
genesis upsert's copy (genesis.rs:296) was tested, by
oracle_settings_of_a_priced_asset_are_bounded.
New test invalid_limits_and_assets_are_refused_wherever_they_enter:
before the seal, create_vault with short_bps 10,001 -> INVALID_LIMITS;
after it, a SetLimits proposal with long_window_s 0 -> INVALID_LIMITS,
and an UpsertAsset proposal with a 10,001 bps haircut -> INVALID_ASSET.
Evidence (tasks/error-codes/mutants.log, STATE=c3, full program suite):
E06 genesis.rs:171 deleted -> KILLED ("expected failure, got success")
E07 governance.rs:40 deleted -> KILLED ("expected failure, got success")
E08 governance.rs:47 deleted -> KILLED ("expected failure, got success")
each by the new test only. All three were SURVIVED at base. Vault hash
7086148b unchanged (tests only).
Sibling search: every config_is_valid / valid_asset call in programs/
vault/src: genesis.rs:170 (create_vault), :295 (upsert_asset, KILLED by
the oracle test), governance.rs:39 and :46. All now killed.
Counts: Vault 105 -> 106, programs 155 -> 156 (claims.py OK).
The execution copy (:478) is not reached by any transaction: it
parses the bytes the proposal already checked, and the bounds read
nothing else. The test comment says so (code-review fix: it had
claimed execution was covered). oracle_settings_of_a_priced_asset_are_
bounded now asserts INVALID_ASSET for each of its four cases, not any
error (all four return it; E09 stays KILLED).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that an import into a ninth mint is refused
genesis.rs:410, import_backing: a backing ledger holds MAX_SLOTS (8)
mints, and an import into a ninth is refused with NoSlot. Deleting the
check left every program test green (tasks/error-codes/mutants.log, base
cede705). From the code, without it the import goes through: the vault's
imported total grows (:414), and the seal must cover that, but the
owner's ledger is saved unchanged, so the owner has no backed claim for
it.
New test an_import_into_a_ninth_mint_is_refused: nine vaults before the
seal; one owner gets an import into eight (ledger_of shows 8 positions),
then the ninth -> NO_SLOT.
Evidence (tasks/error-codes/mutants.log, STATE=c4, full program suite):
E14 genesis.rs:410 deleted -> KILLED by an_import_into_a_ninth_mint_is_
refused ("expected failure, got success"); SURVIVED at base.
Vault hash 7086148b unchanged (tests only).
Sibling search: the only other ledger credit is deposit's
(custody.rs:128). There a full ledger is by design: the deposit is taken
unbacked (Ledger::credit's doc, ledger.rs:187-189), and credit's result is
the backed flag, not an error.
Counts: Vault 106 -> 107, programs 156 -> 157 (claims.py OK).
import_ix now calls import_into_ix, which takes the vault, so the
new test shares the one import layout (code-review fix).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin six Vault refusals no test reached
Six reachable checks had no test that failed when the check was deleted
(tasks/error-codes/mutants.log, base cede705). Each now has an assertion
on its code, and each was seen to fail with the check deleted:
E02 genesis.rs:185 create_vault of an existing vault after the seal:
ALREADY_INITIALIZED (with the check gone: 8043 GenesisOnly, so the
"says so first" order in the comment above it was untested)
E12 governance.rs:426 veto of an executed change: CHANGE_EXECUTED
(gone: the veto succeeds and marks an executed change vetoed)
E13 governance.rs:517 cancel of an executed change: CHANGE_EXECUTED
(gone: the cancel succeeds)
E16 lifeboat.rs:256 execute_exit with no exit proposed:
NOTHING_TO_EXIT (gone: 8033 ChangeNotExecutable)
E17 custody.rs:84 a zero deposit: ZERO_AMOUNT (gone: it succeeds)
E18 custody.rs:272 a zero release: ZERO_AMOUNT (gone: it succeeds; a
zero release has no unbacked part, and the limiter, whose own zero
check custody.rs:242 maps, is called only when unbacked > 0, :361)
Where: E02 in after_the_seal_a_new_vault_starts_with_its_unbacked_lane_
closed, E12/E13 in an_unvetoed_loosening_executes_after_the_delay, E16 in
the_lifeboat_stays_shut_until_the_seal (after the move in, before an
exit is proposed), E17/E18 in the new a_zero_deposit_or_release_is_
refused.
Evidence (tasks/error-codes/mutants.log, STATE=c5, full program suite):
all six KILLED by the tests named above; all six SURVIVED at base. The
vault suite is green (108 passed). Vault hash 7086148b unchanged.
Counts: Vault 107 -> 108, programs 157 -> 158 (claims.py OK).
Helpers (code-review fixes): veto_change_ix beside cancel_change_ix;
create_vault_for_ix, create_vault_ix with a given mint and token
program, so E02's case names USDC's own accounts and creates no mint.
E02's comment now repeats the program's ("whatever the seal's state"):
create_vault checks the limits first (genesis.rs:170).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin four Vault checks another layer backstops
Four checks survived deletion (tasks/error-codes/mutants.log, base
cede705) because something later still refused the case, with its own
error, so no test could tell. Each now has a test that sees the Vault's
own code, and each was seen to fail with the check deleted:
E01 lib.rs:230, create_pda's guard: a second registration of an
existing protocol (register_protocol has no check of its own) ->
ALREADY_INITIALIZED. Gone: the system program refuses it, with no
custom code (custom_code 0).
W10 lifeboat.rs:71, move_in naming another mint or token program ->
WRONG_VAULT. Gone: the token program refuses (code 3, MintMismatch).
W12 lifeboat.rs:247, execute_exit naming another mint or token
program -> WRONG_VAULT. Gone: the same token-program error.
T01 token.rs:37, account_info on an account no token program owns: a
copy of a real USDC account's bytes under the system program. Gone:
the quote SUCCEEDS (it promises a payout), and the release fails
only in the runtime at the transfer ("instruction modified data of
an account it does not own"). The quote is asserted first so the
mutant is seen at the quote (STATE=c6b).
New tests: a_protocol_registers_once, a_lifeboat_move_names_the_vaults_
own_mint (also completes a move in and an exit, as a control: the
destination receives 1,000 USDC), a_payout_goes_only_to_an_account_a_
token_program_owns.
Evidence (tasks/error-codes/mutants.log, STATE=c6 and c6b, full program
suite): E01, W10, W12, T01 KILLED by the tests named above; all four
SURVIVED at base. Full program suite green: Floor 50, Vault 111. Vault
hash 7086148b unchanged (tests only).
Counts: Vault 108 -> 111, programs 158 -> 161 (claims.py OK).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test the bounds on ticket extensions and a closed lane's code
B's review of vault-error-tests-r3 (R1) found two Vault codes no test
asserted by value that the earlier commits missed: ExtensionTooLong
and LaneClosed. Deleting the 7-day total extension cap (brakes.rs:86,
B24) left every program test green, so a live guardian, or a stolen
guardian key, could hold a queued payout back without end, 24h at a
time, against docs/05 row 11. The wide differential models the total
(differential_wide.rs:406), but its default run never extends one
ticket that far. Mapping LaneClosed to QueueFull (custody.rs:240, B23)
also survived.
The code review of the fix found a worse gap at the same check: no
test sends a negative extension (the differential draws secs from 0,
1, 3600, 86400, 86401). With `secs <= 0` weakened to `secs == 0` (A3,
brakes.rs:84) every program test passed, and one guardian key could
pull a queued payout's ETA into the past, skipping the queue delay,
and refund its own 7-day budget.
- a_ticket_is_extended_later_by_at_most_a_day_a_call_and_seven_in_total:
-1h, 0 and 24h + 1 s are each refused with EXTENSION_TOO_LONG, ETA
unchanged; six 24h extensions and one of 23h pass; 2h more is
refused, ETA unchanged; the last hour is granted; then 1 s more is
refused. The split is unaligned so that a total that ignores this
call's `secs` fails too.
- a_closed_lane_refuses_an_unbacked_payout: a vault with short_bps 0,
and one with long_bps 0, each refuse an unbacked admin payout with
LANE_CLOSED; the same payout from a vault with both buckets open is
taken (the control, so a capacity base of 0 cannot pass it).
Evidence (tasks/vault-error-tests-r4/, B's runner copied: each mutant
applied alone, programs rebuilt, whole program suite run):
- without these tests (551e519; run-without.out, run-base2.out,
run-base3.out): B23, B24, A1 (`tk.extended_s >= MAX`, this call's
secs ignored), A2 (`>=` for `>`) and A3 SURVIVED. A4 (the 24h per
call deleted), A5 and A6 (either half of limiter.rs:172's closed-lane
test deleted) were KILLED by the differential only, not by value;
- with them (run-final.out, at the branch tip): B23 KILLED by
a_closed_lane_refuses_an_unbacked_payout; B24 A1 A2 A3 KILLED by the
extension test only; A4 by it and the differential; A5 A6 by the
closed-lane test and the differential.
No program change: the runner's closing build gives 8ca8a608 /
543be60b / 7086148b, as at cede705.
Counts: Vault 111 -> 113, programs 161 -> 163 (claims.py OK).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin each half of create_pda's existing-account guard on its own
B's review of vault-error-tests-r3 (R2): each half of lib.rs:229
(`!is_data_empty() || !owned_by(SYSTEM)`) survived deleted on its own
(B20, B21), since an existing protocol account fails both, so
a_protocol_registers_once pinned them only jointly.
The test now also plants, at a fresh protocol address, an account that
fails just one half: one with data owned by the system program, one
with no data owned by another program (a fresh address). Each
registration is refused with ALREADY_INITIALIZED. The comment says no transaction is
known to make either account: data or a new owner at a PDA needs its
signature, which only the Vault gives, and create_pda gives both in one
instruction.
Evidence (tasks/vault-error-tests-r4/): at 551e519 B20 and B21
SURVIVED (run-base2.out); at the branch tip both are KILLED by
a_protocol_registers_once only (run-final.out).
No count change (claims.py OK: vault_programs 113, programs 163).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Record the Vault refusal deletions and what each became
docs/09 §3 gains a row for the deletion run this branch answers
(tasks/error-codes/mutants.log): 38 refusals, across 14 of the 18 Vault
codes no test asserted by value at cede705, each deleted on its own
with the whole program suite run against it, on the AUD-2 build.
The 18 (each VaultError variant with no `verr::` use in programs/tests
at cede705): AlreadyInitialized NotRegistrar WrongVault
WrongTokenProgram AlreadySealed VaultCountMismatch InvalidLimits
InvalidAsset LaneClosed TicketNotPending ExtensionTooLong
ChangeExecuted LedgerUnsound LedgerCorrupt NoSlot NothingToExit
ZeroAmount Overflow. The run missed LaneClosed and ExtensionTooLong;
B's review deleted one check of each (B23 custody.rs:240, B24
brakes.rs:86), and both survived until 1797ae5. That commit
also pins a negative extension (A3, which survived every test before
it), the 24h per call, and a lane closed by either bucket.
At base: 13 KILLED, 25 SURVIVED. The commits before this one add tests
that kill 18 of the 25 and both of B's, each asserting the Vault's own
code (W02 W06, E04 E05, E06 E07 E08, E14, E02 E12 E13 E16 E17 E18, E01
W10 W12 T01; B23 B24). The row names the 7 that still survive and why:
- six guards no transaction is known to reach: lib.rs:324 (vault PDA
address; create_vault creates a vault only at its PDA), genesis.rs:519
and :523 (the lifeboat, checked by expect_pda first, created only by
lifeboat_move, move_in requires the seal, lifeboat.rs:53, and the
seal happens once, so no lifeboat holds data at it), brakes.rs:50 and
custody.rs:541 (a pending-ticket count below zero), token.rs:235
(GetAccountDataSize's return data from an already-checked program);
- custody.rs:399, stated as not shown unreachable: properties.rs:588-592
debits backed_amount and one more on random ledgers, and only
crates/bulkhead-core/tests/units.rs:375 debits less, on a one-mint
ledger with no liability. properties.rs:606 asserts backed(a) ==
min(a, backed(MAX)) for requests below 2^52 (:613), so any smaller
amount is backed too, and debit_backed(b) refuses only when
backed_amount(b) != b (ledger.rs:289). The known exception is a
collateral value that overflows u128 (ledger.rs:262): the valid
amounts need not form an interval there, so the consent or freeze
cap (custody.rs:337, :347) can lower the amount to an invalid one and
:399 refuses the release (fail closed). The search's own result is
always valid. Not verified by a tool beyond that property test. (A full ledger cannot reach :399 through
set's NoSlot: backed_amount's valid() refuses a new liability slot
on a full ledger, ledger.rs:237.)
Not deleted, as the row says: the two core-error mappings to
ChangeExecuted and ZeroAmount, token.rs:233, and the codes Overflow and
LedgerCorrupt (checked arithmetic, decoding).
After this branch five codes are still asserted by no test (same grep
at this commit): NotRegistrar (E03, genesis.rs:108, was KILLED at base
by another test), TicketNotPending, LedgerUnsound, LedgerCorrupt,
Overflow.
Docs only: no count change (claims.py OK: vault_programs 113, programs
163).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Run the exclusion check's main in its self-test
A's review of finding 105 (REVIEWS/auto-B-mutants-exclusions-r7.md,
finding 1, Low): CI's `mutants` job acts on main's exit status, but the
self-test called problems() only, and check.sh never runs the tool on
real lists. With main returning 0 on a mismatch (mutant A10), every
in-tree check passed, and CI would have printed the problems and gone
green. This was not a defect at fb771c6, since the job fails today. The
gap was that a later regression would pass.
Fix: main takes its arguments and, for the self-test, a config path
(default: the repo's .cargo/mutants.toml, as before), and its lines name
the config it read. The self-test runs main in-process on a planted
config and checks the exit status and every line on each stream:
- the right lists: exit 0, "1 of 3 mutants excluded" on stdout, nothing
on stderr;
- the lists the other way round: exit 1, nothing on stdout, and exactly
the count and "full list lacks" problems, so KEPT is read as KEPT;
- coloured lists: exit 1 with exactly one problem, so a main that fails
only on several problems is caught;
- one argument and three: exit 2 and the two usage lines.
One subprocess (python3 -I, 60 s timeout, in the temporary directory so
the config must be found from the tool's own path) runs the command line
with two lists and the repo's config, as scripts/mutants.sh does: on a
list no pattern can match, it exits 1 with an empty stdout and problem
lines that each name .cargo/mutants.toml. Which problems those are
depends on the config, so only their form is checked. Self-test: 16
cases -> 22. No doc states the number.
Evidence (tasks/exclusions-main-exit/):
- old-A10.log: the fb771c6 self-test with A10 applied prints "self-test
OK: 16 cases" and exits 0.
- mutants-r6.log (mutants.py applies each mutant to a scratch copy of
the tool, which reads the real config through a symlink): unmutated,
"self-test OK: 22 cases", exit 0. All 18 are KILLED (exit 1): A10
(return 0), A10b (fail only on more than one problem), A0 (always 1),
U1 (usage returns 0), U2/U3 (argument check `< 2`, `< 1`), K1 (KEPT
ignored), K2 (ALL read twice), C1 (planted config ignored), C2 (no
default config), R1 (the default config read from the cwd), E1/E2
(sys.argv sliced as [:] or [1:2]), P1 (problems not printed), S1
(problems on stdout), L1 (the planted config not named), N1 (only the
first problem printed), G1 (the success line printed on failure too).
Code-review passes at high shaped this. The first version ran three
subprocesses against the real config, and A0, K1 and U3 survived it.
The second left E2, C2 and G1 alive. Declined from the third: catching
exceptions in the self-test (a traceback already exits 1) and moving
the config hook out of main. The fourth showed that requiring one
"matches 0 mutants" line per live pattern tied the self-test to the
config's contents. The subprocess now checks the form of the lines
only, and the 17 stay killed. The fifth showed that R1 survived while
the child ran in the repo root (it does in check.sh); now it runs in
the temporary directory. Its other points repeat declined ones.
- real-lists.log: on the real lists, made with scripts/mutants.sh's
commands: ".cargo/mutants.toml: 22 of 795 mutants excluded, each
pattern matching its count", exit 0.
- python3 -I tools/mutant_exclusions.py --self-test: 22 cases OK.
- scripts/check.sh: ALL CHECKS PASSED (check-final.log);
bin/check-each-commit.sh green (each-commit.log).
Sibling search: the other tools check.sh runs (tools/claims.py,
tools/floor_check.py) also gate CI through main's exit status. claims.py
has the same gap: check.sh runs it on the real tree, where it finds no
problems, and its self-test calls self_test(), not the path in main that
returns 1. With that return changed to 0, `--self-test` and the real run
both still exit 0 (claims-C1.log). Not changed here: it is its own
commit (BACKLOG-B). floor_check.py runs in CI only as --self-test, whose
result main returns as is. Its RPC mode is an operator's tool that no CI
job runs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Record finding 106: Vault liveness and exit rules had no failing test
The tests landed in 944f5e8 (lifeboat-exit-tests-r3, 15 Oct). Their
09 §4 row waited for 104 and 105 so the list stays numbered without
gaps; 105 is now on integration (fb771c6), so the row lands here.
Docs only: 09 §4 row 106; README and 12's count and post-submission
range (96–106, 106 fixed). Two code-review passes changed the row as
written on 15 Oct: a sentence for today's re-run, the nine
single-guardian sites named, G5 quoted verbatim, and the exit veto
attributed to T18 (G6 has no veto), as is the veto test's doc comment
(vault.rs:3996, comment only).
Evidence, re-run today on bf8ba41 (the row's claims are from 15 Oct,
and AUD-2 changed the Vault since): B's re-review list of 33 line-
neutral mutants (the ten rules, the nine single-guardian sites L1-L9,
the boundaries, the six quorums, R1-R6), each applied, programs
rebuilt, all program tests run:
33 KILLED, 0 survived, 0 not applied, 0 build failures. Vault
hashes after the run match the tip (Vault 7086148b, Floor 8ca8a608,
Harbor 543be60b).
Log: autopilot tasks/finding-106-row/mutants.log.
- python3 -I tools/claims.py: OK, findings 106.
- git grep for "96–105", "105 findings": no other stale range.
- scripts/mutants.sh still mutates the core only (SCOPE, line 11), so
the row's last sentence holds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Cite PR #100's tree, not the unpublished bf8ba41, in finding 106
Finding 106's row in docs/09 said its 33 mutants were re-run on 16 Oct
at bf8ba41. That commit is not an ancestor of this branch or of the
stack above it and is on no remote branch, so the citation would point
at nothing once the stack lands. The owner's review of #99-#109 found it.
bf8ba41 and 86b777b (the head of PR #100, vault-error-tests-d2, the D2
replay of vault-error-tests-r4) have the same tree, 36187b49
(`git rev-parse bf8ba41^{tree} 86b777b^{tree}` prints it twice). So
the run was a run on 86b777b's tree, and the row now says "on tree
36187b49, the tree of 86b777b (PR #100's head)". Why not just "at
86b777b": it was committed on 17 Oct, after the run; and main
squash-merges (1045cd9 has one parent), so after PR #100 lands 86b777b
is reachable only as GitHub's refs/pull/100/head. The tree id stays
checkable from main: squashed bottom first, main's commit for PR #100
has that tree. "PR" keeps "#100" from reading as finding 100, the
row above. These points came from two code-review passes.
Sibling search: every 7-12 hex token in tracked .md/.py/.sh/.mjs/.rs/
.toml files at autopilot/integration 3e9f58d that names a local commit
(51 tokens) was checked with git merge-base --is-ancestor against
integration. bf8ba41 was the only one that is not an ancestor.
Docs only. tools/claims.py: claims OK (26 files).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Warn about a Floor guardian set short of the margin (finding 107)
The guardian checks the Vault protocol's guardian set against finding
107's margin (main.mjs checkGuardianSet), but never a Floor config's.
A Floor set registered before the rule, say 2-of-3, keeps working and
keeps AUD-2's exposure: one lost key and one rogue guardian lock its
governance for good. The guardian said nothing, and docs/07 §6 said so
("not yet a Floor set").
The rule moves to policy.mjs as guardianSetWarning, a pure function:
the same text as before, a warn line that alerts on mainnet only. In
main.mjs, checkGuardianSet now remembers the set shape per set
("protocol", or the protected program), and checkFloor calls it for
each watched Floor config it reads. floorAlerts is unchanged. The
margin is not a readiness code there: "floor not ready" alerts on
every cluster, and only the owner's new set clears a margin problem, so
"floor ready" would never be logged again.
Evidence (tasks/floor-guardian-margin/r2/):
- node --test cloud/test/*.test.mjs: 180/180 pass (one new test).
- Mutants of guardianSetWarning, each run on policy.test.mjs and
reverted; each fails the new test: N1 never warn, N2 threshold fixed
at 2, N3 notify always, N4 notify never, N5 GUARDIAN_MARGIN = 1.
- tools/claims.py: OK, client 180 (docs/09:40 updated).
- Not tested: the main.mjs wiring (the per-set Map and the checkFloor
call). No test runs checkFloor; deleting the call leaves every test
green.
Sibling search: guardianMarginProblem's callers are vault-check.mjs (an
info line) and guardianSetWarning. The status snapshot
(snapshot.mjs:33) prints live guardians and the quorum, not the margin;
left as it is.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Judge each guardian set's margin on its own, and say when it is fixed
The parent's per-set memory (which set shapes were already reported)
lived in main.mjs, so no test could fail on it: dropping the
once-per-shape check, or keying every set the same, left every test
green. The warning did not say which set it was about (only a program
tag on Floor lines), and it gave the same text for a set ejected down
to its threshold, or registered n-of-n before finding 96. There one
rogue guardian alone locks governance; a lost key alone does not, since
dead guardians cannot veto the owner's new set (docs/16 §4.4), but it
leaves no quorum until that set executes. Nothing was said when the owner's
new set fixed the problem. And a delivered alert did not say which Floor
program it was about: the program went only into the line's `program`
field, which alertFor uses in its repeat key but does not show.
Moving the program name into the text is a trap: alertFor keys repeats
on the first 60 characters of the text, and "the Floor guardians of
<44-character id>" pushes the shape past them. A 2-of-3 set ejected to
2-of-2 within ten minutes would then give the same key, and the worse
warning would not alert. So the program goes after the shape: "the
Floor guardians are 2-of-3 for <program>: ...". The `program` field
stays for the repeat key.
guardianSetWarning becomes guardianSetCheck(shapes, which, set,
mainnet) in policy.mjs. It keeps each set's last shape and whether it
was short in the Map, keyed by "protocol" or the protected program, and
returns the line to log or null. The warning names the set and has its
own text for n-of-n. When a set that was short gets the margin, it
returns one line saying so, which alerts on mainnet like the warning,
so whoever was paged hears that it is fixed. main.mjs only logs what it
returns. docs/07 §6 says so. The advice also had a hole: a set
registered as 7-of-8 was told to "run at least 7-of-9", and no set can
have 9 keys (core MAX_GUARDIANS is 8). When threshold + 2 is over 8,
the advice is now to lower the threshold to at most 6-of-8. That needs
MAX_GUARDIANS in JS, so lib/solana.mjs gets it next to
GUARDIAN_MARGIN; the next commit ties both to the core.
Evidence (tasks/floor-guardian-margin/):
- node --test cloud/test/*.test.mjs: 180/180 pass (the test is
extended, not added).
- 19 mutants, each run on policy.test.mjs and reverted (r6/mutants.py,
r6/mutants.log, r6/N5-margin1.log). Each fails the extended test, and
only it: never warn, threshold fixed at 2, notify always, notify
never, GUARDIAN_MARGIN = 1, no once-per-shape, one Map key for every
set, the n-of-n text for threshold + 1, no fixed line, a fixed line
for a set that was never short, the fixed line never alerting, no
program tag, every set named as the protocol's, the short flag not
stored, the program before the shape, the program left out of the
text, `need < 8` for `need <= 8` (6-of-7), the 8-key limit ignored
(7-of-8), 7-of-8 as the lowered advice.
- The mainnet alerts are checked on their own in a scratch copy of the
test without the earlier text assertions (r5/C1-C3-alone.log): with
the program left out of the text, the alert's detail check fails;
with a long name pushing the shape past character 60, "2-of-2 alerts"
fails.
- A's review of the previous version (branch
auto/B/floor-guardian-margin-r7, kept) found two survivors there: X3,
the shape keyed on the count only, and X5, the advice
threshold-of-(n+1). It also found that the n-of-n text said a lost key
alone locks governance. The test now has a threshold-only fix (7-of-8
-> 6-of-8, the advice's own step) and pins the n-of-n text and advice.
r9/mutants.log: X3, X5, the r7 n-of-n text and the text without its
lost-key clause are all KILLED. r9/mutants-r7.log: X3 and X5 SURVIVED
on the r7 code and test. The 18 mutants of r6/mutants.py, re-run on
this code: all KILLED (r9/r6-mutants-rerun.log). Two code-review
passes on the fix (kept as -r8 and -r9) added the lost-key clause: a
lost key alone leaves no quorum (it never approves, and core `quorum`
needs threshold approvals) until the owner's new set executes. A third
pass (-r10) fixed the comments' wording only.
- tools/claims.py: OK, client 180.
- Still not tested: the checkFloor call in main.mjs (no test runs
checkFloor).
Not changed (code review): the "fixed" line needs the short shape to
have been seen since the guardian started (the Map is in memory, as the
old per-process shape was; docs/07 says so). A protocol that abdicated
SetGuardians can't take a new set, so the advice to run 2-of-4 can't be
followed there; REPORT lists that case for the owner. The fault text
assumes a margin of 2, the owner's decision (16 §4.4).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Tie the JS guardian margin and set size limit to the core constants
cloud/lib/solana.mjs keeps its own GUARDIAN_MARGIN = 2, which the
guardian and the deploy tooling use to judge a guardian set, and now
MAX_GUARDIANS = 8, which the guardian's advice uses. Their comments say
they are core governance.rs's constants, but nothing checked that. Had
the programs' margin moved to 3, every JS test would have stayed green
while the guardian passed sets the programs refuse.
deploy.test.mjs now reads governance.rs and requires exactly one
`pub const GUARDIAN_MARGIN: u8 = N;` and one `pub const MAX_GUARDIANS:
usize = N;`, each N equal to the JS constant. It reads the source text,
like the build-script test above it. If a declaration changes form, the
test fails rather than passing.
Evidence (tasks/floor-guardian-margin/r6/tie-mutants.log):
- node --test cloud/test/*.test.mjs: 181/181 pass (one new test).
- Mutants of governance.rs, each reverted with its mtime kept (else the
limiter differential asks for a worker rebuild): GUARDIAN_MARGIN = 3,
MAX_GUARDIANS = 9, and a trailing comment on MAX_GUARDIANS. Each fails
the new test, naming the constant. deploy.test.mjs is green again.
- tools/claims.py: OK, client 181 (docs/09:40 updated).
Sibling search: the other JS copies of core rules (the limiter port,
the account layouts) are checked by the differential and by decoding
real accounts. MIN_GUARDIAN_THRESHOLD has no JS copy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Reach the 7-day extension total in the wide differential
differential_wide.rs drew extension seconds only from {0, 1, 3600,
86400, 86401} on a random ticket and lumped every refusal into one
counter. A ticket never got near the 7-day total, no negative value was
sent, and a refusal was not checked by its code. So the default run
could not tell the total bound, a negative extension or the error code
from a bug: only the unit tests at vault.rs:3382-3401 pinned them.
Now one extension draw in three pushes the chosen ticket by up to 24h
a call to exactly the 604,800 s total, then asks for 1 s more, so the
edge itself is hit on every push, not only when earlier draws line up.
The other draws are half legal sizes (1, 3600, 86400) and half out of
range for one call (i64::MIN, -86400, -1, 0, 86401, i64::MAX). Each
refusal must fail with EXTENSION_TOO_LONG. The counters are split into
extend (a random legal size), extend_push, extend_refused_call and
extend_refused_total, all required >= 10, so pushes cannot stand in for
the random sizes.
The model reads both bounds from bulkhead-core (consts.rs:37-38), as
the module says it is built, rather than restating 86,400 and 604,800.
docs/09 §7's row says so, and the 16 Oct "Vault refusals deleted" row,
which says the default run never extended a ticket that far, now says
it does since this change (its differential_wide.rs:406 cite had drifted).
Default run (24 seeds x 250 steps): extend 16, extend_push 126,
extend_refused_call 16, extend_refused_total 30 (was extend 40,
extend_refused 27); every other required counter still >= 10
(loosen_executed 12, as before).
Evidence: mutants of brakes.rs extend_ticket, each built and run against
the old and the new differential_wide only:
T1 drop the total check old SURVIVED new KILLED
T2 total > to >= old SURVIVED new KILLED
T3 drop secs <= 0 old KILLED new KILLED
T4 secs <= 0 to < 0 old KILLED new KILLED
T5 total ignores this call's secs old SURVIVED new KILLED
T6 extended_s += secs / 2 old SURVIVED new KILLED
T7 refusal returns WrongTicket old SURVIVED new KILLED
T8 secs <= 0 to == 0 (negatives) old SURVIVED new KILLED
T9 total > to > total + 1 old SURVIVED new KILLED
(T9: a draft that pushed by 24h until refused killed it only through
chance 1 s and 3600 s draws on the same ticket; the push to exactly
604,800 then 1 s more reaches the edge every time.)
sBPF sha256 after the run unchanged (Floor 8ca8a608, Harbor 543be60b,
Vault 7086148b): tests only.
Sibling search: the other refusal counters in this file (reset_refused,
lifeboat_refused_*, loosen_*) are each split by cause already.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check settle and lifeboat refusals by code in the wide differential
The same gap as the extension refusal, found in the sibling search for
the previous commit: a refused settle and the three lifeboat-move
refusals asserted only r.is_err(). Any refusal passed, so the order of
the checks and the program's own codes went unchecked by the
differential. Settle's comment (custody.rs:520-521) says it returns its
own error, not the token program's, when the vault is short; nothing in
this file held it to that.
Now a settle refused while frozen must fail with FROZEN, and one that is
only short with INSUFFICIENT_BALANCE (frozen is checked first). A
lifeboat move inside the delay after the seal must fail with NOT_SEALED,
one while frozen with FROZEN, and one beyond the balance with the token
program's InsufficientFunds (1). The two settle refusals get their own
counters: settle_refused_frozen is required >= 10 like the rest;
settle_refused_short needs lifeboat moves to drain a due ticket's
tokens first and occurs 7 times in the default run, so it is required
>= 1.
Evidence: mutants of programs/vault/src, each built and run against the
previous head's differential_wide and this one, nothing else:
S1 settle: drop the InsufficientBalance check old SURVIVED new KILLED
S2 settle: balance check before the freeze old SURVIVED new KILLED
L1 require_sealed returns Frozen old SURVIVED new KILLED
L2 move_in's freeze returns NotSealed old SURVIVED new KILLED
The token-program code (1) on a move beyond the balance is pinned but
not shown red by a mutant. sBPF sha256 unchanged (Floor 8ca8a608,
Harbor 543be60b, Vault 7086148b): tests only.
Deep run of both commits together (BULKHEAD_PROGRAM_SEEDS=600
BULKHEAD_PROGRAM_STEPS=500, 85 s): 300,000 steps, all identical;
extend 876, extend_push 6,462, extend_refused_call 1,175,
extend_refused_total 1,453, settle 2,024, settle_refused_frozen 921,
settle_refused_short 566, lifeboat refusals 3,854 + 5,768 + 1,171. docs/09 §7's
row gets that run as a 16 Oct result next to the 5 Oct one, which
predates these checks.
Sibling search: in differential_wide.rs, reset (TICKETS_PENDING) and
execute change already asserted codes, and a release or quote refused
for want of tokens asserts INSUFFICIENT_BALANCE. A release or quote the
limiter refuses (frozen, lane closed, queue full) is still compared only
as a refusal: mapping the core's refusal to the program's code is left
for its own change. Lifeboat returns only send legal amounts. docs/09's
row now says which refusals are checked by code. differential.rs:327
has the same is_err settle check; left as it is, since the wide run now pins both
settle codes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check limiter refusals by code in the wide differential
A release or quote that the limiter refused (lane closed, queue full)
was compared with the model only as Err. Any refusal code passed: a
program that swapped LaneClosed and QueueFull in custody.rs's
limiter_error, or checked the backed part against the balance before
the limiter and so answered InsufficientBalance for a full queue,
still matched the model. About 670 such refusals happen in a default
run.
The model now maps core's LimiterError to the program's code with
verr::limiter in the test harness (all four variants, as custody.rs's
limiter_error does), and every refused quote and release must fail
with the model's code. Lane-closed and queue-full refusals have their
own counters, each required at least 10 times; a refusal code with no
counter panics rather than being counted as want of tokens. docs/09
row 362 says so and records the deep run.
Evidence (autopilot tasks/differential-limiter-codes/):
- mutants.log: C1 (LaneClosed and QueueFull swapped in
limiter_error), C2 (QueueFull -> InsufficientBalance), C3
(LaneClosed -> ZeroAmount) and C4 (backed > balance checked before
the limiter, same line) each SURVIVED the old test and are KILLED
by the new one. Hashes back to 8ca8a608/543be60b/7086148b after.
- c4-full-suite.log: under C4 the whole program suite (Vault 113,
Floor 50, the fixed differential) passes; only this test fails.
C1-C3 are also caught by vault.rs's code tests (:434, :3428).
- Default run: refused_lane_closed 254, refused_queue_full 414,
insufficient 118.
- deep-600x500.log: 300,000 steps, 15,561 lane closed, 18,853 queue
full, 6,345 want of tokens, all identical.
Sibling search: every other Err arm in differential_wide.rs already
asserts a code (extend, settle, reset, lifeboat in, execute change).
Not here: lifeboat returns are never refused (backlog), and lifeboat
move_in beyond the balance fails with the token program's code
(backlog, a program change).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check release refusals by code in the fixed differential
differential.rs reduced every refused release to Err(()) on both
sides, so a program that refused for the wrong reason still matched
the model. The same gap the previous commit closed in the wide
differential (found by its code review).
The model's refusal now carries the program's code: verr::limiter for
a limiter refusal, INSUFFICIENT_BALANCE when the vault holds less
than it would pay. A refused release must fail with that code, and
the assert prints the program's error. Refusals for a full queue get
their own counter, required at least 10 times like the others. Want of
tokens and a closed lane are counted and printed but not required:
the wide differential requires both. A refusal with any other code
panics, as in the wide differential, so the counts always add up.
docs/09 row 361 says so; its undated figure (4,843 releases) no longer
matched a run, so it is replaced by a dated one.
Evidence (autopilot tasks/differential-limiter-codes/):
- mutants-narrow.log (mutants.sh <wt> differential): C1 (LaneClosed
and QueueFull swapped), C2 (QueueFull -> InsufficientBalance) and C4
(backed > balance checked before the limiter) SURVIVED the old test
and are KILLED by the new one. C3 (LaneClosed -> ZeroAmount)
survives both: no release in this test's runs is refused for a
closed lane. The wide differential kills it, and docs/09 says so.
Hashes back to 8ca8a608/543be60b/7086148b after.
- r2/default-run.log: 1,000 steps, 82 releases refused, all for a
full queue (want of tokens 0, lane closed 0).
- r2/na…
shuv-amp
added a commit
that referenced
this pull request
Oct 18, 2026
#107) * Run the tests on 2-of-4 guardian sets; test recovery from two faults Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set with fewer than threshold + 2 keys. Every test fixture and most tests' own sets were 2-of-3. This moves them to 2-of-4, which today's rule also accepts, so this commit changes no program: Floor, Vault and Harbor hash 508e5a99 / b07655d9 / 543be60b, as before. - Fixtures: Protected (Floor) and Market (Vault, through Harbor) now hold 4 guardians. Each test that built its own 3-key set gets a 4-key set with the same threshold and the same assertion. - Two tests changed meaning with a 4th key and were fixed, not loosened: a_quorum_ejects_a_rogue_guardian_who_vetoes_everything (Floor) and a_quorum_ejects_a_guardian_who_freezes_forever (Vault). With 4 keys, the last eject was refused only because the target's own vote is masked. Both now eject a second guardian (a set an ejection shrank to threshold + 1 still ejects) and then show the 2-of-2 refusal. - Account sweeps: an eject's signers may now be any two of the three others, so each of those two slots is free for the third, never for the target or the other signer again. Vault 5,365 and Floor 1,710 substitutions, refused (docs/09, dated 16 Oct). Listing all three others at each slot would have skipped the duplicate: with a mutant that counts a guardian listed twice as two (governance.rs:270), the sweeps listing all three pass, and these fail at "eject: account 2 replaced by key ..." (Vault and Floor). No other program test kills that mutant (tasks/aud2-guardian-margin-impl/dup-mutant-*.log). - floor_differential: the guardian pool is 6 keys, 4 registered; the proposed sets are 2 to 6 keys. New tests (pass on both rules; they show what the margin buys): - a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_set (Floor): g0's key lost, g1 rogue stops the program; the lost key does not help a quorum; the honest pair ejects g1 and restores; neither can veto the owner's refill, which executes after 72h. - two_rogues_in_turn_are_both_ejected_from_a_two_of_four_set (Floor): the second rogue vetoes the refill and is ejected from the 2-of-3. - a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_vault: the same in the Vault (freeze, eject, unfreeze, refill). Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest and quote sweep): the counts and the sweep figures here are from runs there (tasks/aud2-guardian-margin-impl/r4/). The E2E run, the mutants and the new-rule run below were made before, on e8640a7; this commit's programs and core are the same as there but for one comment line in views.rs (line-neutral, same hashes). Evidence: - E2E=1 scripts/check-programs.sh: PROGRAM CHECKS PASSED (Floor 49, Vault 98; S1 and E2E passed; the resumed setup found all 14 steps; vault check, Floor demo and deploy dry run passed), with the next commit's deploy rehearsal in the tree. - tools/claims.py: OK (floor 49, vault 104, programs 153). - Under the new rule (core from two commits on, programs rebuilt), only the two tests that state the old rule fail: an_n_of_n_guardian_set_is_ refused (accepts 3-of-4) and the Floor differential's model. - Mutants (tasks/coverage-map/mutants.sh), each red, then restored: AV1 Floor live_guardian ignores liveness: KILLED by a_lost_key_and_a_ rogue_do_not_lock_a_two_of_four_set (+2 others). AV2 the Vault's: KILLED by ..._two_of_four_vault (+2 others). AQ1 Floor restore quorum ignores liveness: KILLED by ..._two_of_four_set and the differential only. AQ2 the Vault's unfreeze quorum: KILLED by ..._two_of_four_vault and a_silent_guardian_does_not_count_toward_a_quorum. AE1 can_eject refuses at count == threshold + 1 (AUD-2 option (a)): KILLED by two_rogues_in_turn_..., and the two eject tests above. Hashes after: 508e5a99 / 543be60b / b07655d9. Split on 16 Oct from ec0cacc at the owner's review (D2): the deploy rehearsal's move to 4 guardians is now the next commit. programs/ and programs/tests/ here are byte-identical to ec0cacc, so its runs below hold for this commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Rehearse deploys with four guardians, and read only the keys there are Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set with fewer than threshold + 2 keys, so the deploy rehearsal must register 2-of-4 sets. Split from ec0cacc at the owner's review (D2); the text is that commit's, unchanged: Deploy rehearsal (cloud/deploy market.mjs, vault-check.mjs, floor-demo.mjs): 4 guardians (keys/guardian3), so the dry run keeps working once the rule lands; one more funding step (14, was 13). The devnet market has 3 guardian files. A first version of this change loaded guardian3 everywhere: vault-check.mjs failed there with ENOENT, and heartbeat silently created a key nobody holds as a guardian. Now market.mjs guardianKeys(dir, create) reads guardian0, 1, ... and throws on a gap (never fills it with a new key). Setup creates keys up to 4 only while the protocol is not registered, so resuming an old market adds no key the chain lacks; a new deploy-dryrun step resumes the dry-run market with guardian3 moved aside and fails if setup makes it again (with setup always creating: "DRY RUN FAILED: resuming a registered market made keys/guardian3.json", r3-dryrun-J7.log). renew-guardians tops up to 4 and logs each new key (unfunded); a 2-of-3 renewal would be refused once the rule lands. keysIn reads the guardians only when a command uses them, so caps and set-recovery never need them; heartbeat says a key is not in the set rather than dead. One THRESHOLD (2) for the register and renewal payloads and MIN_GUARDIANS. vault-check.mjs compares the chain's set with the first files, and liveness with the chain's count. An extra file (a renewal not executed yet) and a set without the margin are info lines, not failures, since such a set keeps working. GUARDIAN_MARGIN and guardianMarginProblem live in cloud/lib/solana.mjs, used by these and by the guardian bot's existing warning (cloud/guardian/main.mjs), and vault-check uses THRESHOLD. The e2e scripts fund guardian3 too. Three tests in cloud/test/deploy.test.mjs (client tests 176 -> 179). JS mutants (js-mutants-r3.log), each red then restored: always create, never create, no gap check, margin 1, `>`, GUARDIAN_MARGIN = 1. Setup always creating survives these tests and is killed by the dry run step above. cloud/, scripts/deploy-dryrun.sh and docs/09's client-test row are byte-identical to ec0cacc (git diff empty), so its runs hold for them: the E2E check-programs run with the dry run (r4), and the JS mutants. tools/claims.py: OK (client 179). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Refuse a guardian set without two keys beyond its threshold (107) AUD-2, decided by the owner on 16 Oct (option (e) of 16 §4.4). A set stays live while its honest live guardians reach the threshold: they eject every rogue in turn, and dead guardians cannot veto the owner's new set. Finding 96 asked for one key beyond the threshold, so a 2-of-3 or 3-of-4 set survived one fault but not a lost key plus a rogue: the honest guardian left is no quorum, nobody can be ejected, and the rogue keeps Floor stopped (or the Vault frozen) and vetoes every new set. Fix, in the core so both programs inherit it through GuardianSet::parse (registration, and every SetGuardians when proposed and when executed): GUARDIAN_MARGIN = 2 guardian_set_ok := count <= 8 && t >= 2 && count >= t + 2 (saturating_add: t is a payload byte). A stored set is not checked again, so existing 2-of-3 sets keep working, and ejection is unchanged: a shrunk set still ejects. A 2- or 3-key set is BadThreshold, not BadCount; the count rule is only the bitmap size, and the error order of finding 103 is kept. This changes both programs, by intent: Floor 508e5a99 -> 8ca8a608, Vault b07655d9 -> 7086148b, Harbor 543be60b (unchanged). Darwin-arm64 builds. Compute (scratch LiteSVM test, tasks/aud2-guardian-margin-impl/cu.md): -3 CU on every path that parses a set (Floor propose 2035 -> 2032, execute 1682 -> 1679; Vault register 9592 -> 9589, SetGuardians 2979 -> 2976; Floor register varies with its PDA bump, residue also -3). Proof and tests: - Kani: accepted_sets_can_eject_anyone restated (ejection reachable exactly when count > t, so option (a) cannot slip in; accepted sets have >= 4 keys and a margin of at least 2, and 2 is accepted). New: an_honest_live_quorum_ejects_any_rogue (any slots, anyone else alive) and accepted_sets_survive_any_two_faults (any two faulty keys, lost or rogue, leave a live honest quorum, which ejects each rogue, and the rogues alone eject no honest guardian). That holds only short of a rogue quorum: two rogues in a 2-of-n set are one (out of scope, 04 T7). With the assumption rogue < t dropped, the harness fails at `!can_eject(rogue, ..)` (kani-no-rogue-assume.log), so the docs say "short of a rogue quorum", not "any two faults". 19/19 verified, 240 s (cargo kani -j 2, as CI); again after the restatement, 235 s summed (r2-prove.log). - Kani mutants (kani-mutants.sh): margin 1, margin 3, `>` for `>=`, t >= 1, count <= 9, option (a), and an eject one vote short (only the two-fault harness kills that one): each KILLED by a harness (kani-mutants-r2.log). The eject ignoring liveness is the eject harness's case, not these. - Program mutants (coverage-map/mutants.sh, rebuilt each time): margin 1: KILLED by a_guardian_set_without_two_spare_keys_is_refused (Floor and Vault), registration_rejects_weak_settings and the Floor differential. Margin 3 and `>`: KILLED by most of the suite (the fixtures are 2-of-4). t >= 1: first only the Floor differential; the 1-of-2 registration case was now refused by the margin too, so it is a 1-of-4 here and a Vault 1-of-4 case was added: then KILLED by both. count <= 9: SURVIVES the program tests, equivalent there (parse refuses more than 8 keys first), KILLED by Kani. Option (a): KILLED by the eject tests two commits back (the tests on 2-of-4 sets). - Tests: the Floor and Vault refusal tests above cover proposal and execution (a stored 2-of-4 proposal whose threshold turns 3 is refused when executed, as the devnet renewals proposed before this rule will be); registration_rejects_weak_settings gains 2-of-3 and 3-of-4. an_n_of_n_guardian_set_is_refused (Vault) is renamed and accepted a 3-of-4: it now refuses it and accepts 3-of-5. - Model: the Python rule, the Floor differential's model, and the guardian_set fuzz target state it again; the GP generator now also picks count - 2. With the Python rule put back to the old one, seed 11 (20,000 cases) reports 5 GP mismatches; restored, seeds 11-13 identical. Fuzz seeds two_of_three_refused, two_of_four, six_of_eight, and seven_of_eight renamed seven_of_eight_refused (byte 323, the threshold, is 7: now refused). With margin 1 in the core the target crashes within 60 s; restored, 222,615,288 runs in 601 s, no crash. - scripts/check.sh: ALL CHECKS PASSED, and E2E=1 check-programs.sh: PROGRAM CHECKS PASSED, both at the branch tip, whose code is this commit's (the next commit is docs only; tasks/aud2-guardian-margin- impl/r4/check.log, r4/check-programs-e2e.log). claims: kani 19, floor 50, vault 104, programs 154, client 179. Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest and quote sweep). The core and the program sources are the same as before the rebase but for one comment line in views.rs (line-neutral), so the Kani, fuzz, differential and mutant runs above were made on the pre-rebase commit; the hashes, check.sh, check-programs and the counts are from r4 runs (tasks/aud2-guardian-margin-impl/r4/). The finding 96 row said the restated harness proves "exactly"; it now says what it proved then and proves now, and gives the Vault test's new name. The finding 107 row waits for rows 105 and 106 to land (09 §4 is numbered 1..n); its text is ready. The rule's prose is the next commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * State the guardian margin in the docs, with its costs (AUD-2) The previous commit made both programs refuse a new guardian set without two keys beyond its threshold (finding 107). The docs that state the rule now say so: - README: guardians 4-8 keys; the G5 row and the "who the guardians are" paragraph give the reason (any two faults leave a quorum, unless the rogues are one themselves: two in a 2-of-n set, 04 T7). - 05 register_protocol: 2 <= threshold <= count - 2, also for every SetGuardians, proposed and executed. 05 abdicate: abdicating SetGuardians fixes the set, so ejections shrink it for good. - 06 state table, 00 D22, FAQ: the same rule. - 07 §2: 4-8 guardians, and don't abdicate SetGuardians. 07 §6: the programs enforce the margin; sets registered before keep their exposure until replaced; an ejection uses up a spare key until the owner's new set executes; a lost key counts as live until its interval passes. - 16: the AUD-2 row is decided; §4.4 gives the decision, why the margin is the number of faults a set survives (short of a rogue quorum), the options rejected and why (the honest-owner and stolen-owner cases look the same on-chain), what it costs, what it leaves (a rogue quorum; a stolen owner key with every guardian dead; a rogue's veto of the recovery key's rotation, checked: Floor's veto has no recovery exception, as a_vetoed_recovery_proposal_does_not_hold_the_owner_s_slot shows), and Lido-style lapsing vetoes as a future option only. §5: "independent guardians, 3-of-5 or more". - 10: the guardian set's minimum size is no longer an open decision. SECURITY.md and 08 describe what is on devnet, which still runs the old rule, and are left as they are. claims: OK. Docs only; no program file changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Say the weekly CI run skips mutants, as its condition does ci.yml's header and scripts/ci-changes.sh said the weekly run does every job, and 09 §8 gave `mutants` as running on any change to the core. The `mutants` job's `if` starts with `github.event_name == 'pull_request'`, so the Monday schedule (and a manual dispatch) never runs it: it mutates a pull request's diff and has none to mutate. Its trigger list in 09 §8 also left out scripts/mutants.sh and .cargo/mutants.toml, which ci-changes.sh (line 22) selects it for. Comments and docs only; nothing runs differently. Sibling search: `git grep` for "weekly run does", "every job ... weekly" and "mutants ... weekly" across md/yml/sh/mjs/py. The other hits are 09 §8's "at full depth every Monday" (its table now says which job is not weekly) and the dispatch input's "every job at weekly depth", which is the same set as the weekly run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Check that the mutant exclusions still match; repin murmur3 (finding 105) .cargo/mutants.toml excludes the core's 22 equivalent mutants, and the docs say so. Since 9 Oct it excluded 19. The murmur3 tail's pattern is pinned by line, `floor.rs:(235|238|241)`, because the body's `h ^= k` has the same column and function and is not equivalent. 1dc7861 (#52) moved those lines (now 327/330/333), and nothing checked the patterns. Failing closed, the three would have failed CI's `mutants` job on a pull request that changed those lines. Failing open, a pattern that starts matching a non-equivalent mutant hides it from every run. Root cause: nothing checked that a pattern still names its mutants. Fix: - each pattern ends with the number of mutants it excludes (`# 3`), and the murmur3 pattern names 327/330/333; - tools/mutant_exclusions.py fails unless each pattern matches exactly its number in `cargo mutants --list --no-config`, and cargo-mutants (the list with the config) excludes exactly the mutants the patterns match, so a pattern this parser can't see, or reads differently, fails too. No two patterns may match the same mutant, since the docs' count is the counts' sum, and the kept list must be part of the full one. It reads only the exclude_re array's body, needs no tomllib (Python 3.9), and names colour codes in the list as the problem; - scripts/mutants.sh makes both lists (--colors never: CI sets CARGO_TERM_COLOR=always) and runs it first, except when the caller passes --config or --no-config (as whole arguments). CI's `mutants` job runs that script; ci-changes.sh now also selects the job for the new tool; - scripts/check.sh runs its --self-test on planted configs; - tools/claims.py ties the docs' "22 equivalents" (README, 09 §6 and the summary row) to the sum of the counts, with the same parser. It finds that parser from its own directory, so `python3 -I` runs it too, and check.sh runs its self-test that way. 00's "18 equivalents" (26 Sep log entry) and run 9's "exactly the 22 equivalents below" (6 Oct) are kept as dated records. Docs: 09 §4 finding 105, §6 run 10 and the "In CI" note, §8's mutants row; README and 09's summary row 669 → 797 mutants (run 10); README and 12's post-submission range 96–105. Evidence (first on 1f2ad48 + the parent, logs in the autopilot's tasks/mutants-exclusions-r2/; re-run on 58697c6 + the parent, tasks/mutants-exclusions-r3/): - `cargo mutants --list`: 797 with --no-config, 778 with the old config (19 excluded), 775 with this one. The 22 excluded are §6's list, the murmur3 ones at 327/330/333. - scripts/mutants.sh --list: exit 0, "22 of 797 mutants excluded"; with CARGO_TERM_COLOR=always also exit 0. Old config: exit 1, every pattern "no '# N' count". Planted on the real lists: the old lines with a count (matches 0, says 3) and the body XOR added (matches 4, says 3) both exit 1; in both the set comparison alone passes. /usr/bin/python3 (3.9.6): self-test OK and the real check passes. - The self-test pins each problem by its message, not a count. Mutants of the tool (tool-mutants.log), each red: M1 no per-pattern check, M2 no set comparison, M3 no colour check, M4 no empty-array check, M5 a missing count accepted, M6 a missing count read as 0, M7 the parser reading past the array. M3 and M5 survived a first version that checked only the number of problems. After A's review (16 Oct): the set comparison was pinned only by its size, so `len(excluded) != len(matched)` passed the self-test. A case with as many excluded as matched but not the same ones now fails it (15 cases; red with that mutant, green without), and the message says how many mutants are in one set only. From the code review after the rebase: colour codes only in the kept list were reported as a kept list made with other options; the colour check now reads both lists (16 cases; dropping the kept half fails the new case). On 58697c6, with the overlap and kept-list checks: M8 no overlap check and M9 no kept-list check, red too, as M1–M7 still are. A duplicate of the can_eject pattern planted in the real config: scripts/mutants.sh --list exits 1, "the counts add up to 23, the patterns match 22 mutants" (planted-overlap.log). Without the sys.path line, `python3 -I tools/claims.py` fails with ModuleNotFoundError; with it, both claims runs pass. - claims.py (claims-mutants.log): one count 1 → 2 fails four statements (README 188; 09 33, 333, 335; kept-run9.log), not run 9's 331, which failed too before it was kept; a dropped count fails with the tool named; deleting any of the three rules fails a self-test case; deleting the KEPT entry fails 00:103. - The 22 excluded mutants run alone (--no-config, --re per pattern, 2 min): 22 missed, exactly the excluded list; --re also picked ema.rs:70 (a field deletion, no pattern matches it), caught. - Run 10 (f1f05ba, old config, -j4, 46 min; tasks/mutants-run10/): 778 tested, 712 caught, 5 timeouts, 58 unviable, 3 missed, exactly the murmur3 tail. The core's source is unchanged since (`git diff --stat f1f05ba HEAD -- crates/bulkhead-core/src` is empty; only tests/units.rs gained finding 102's tests). - No program or core source changed. Rebased on 16 Oct onto cede705 (findings 102 and 100, the attest sweep and AUD-2). AUD-2 changed the core's source (governance.rs), so the line above saying it is unchanged since run 10 held on 58697c6, not here. On cede705: scripts/mutants.sh's list check passes, "22 of 795 mutants excluded, each pattern matching its count"; the murmur3 lines are still 327/330/333. The 797 in README and 09 is run 10's figure: README now dates it ("in the last full run, 15 Oct") and finding 105's row says the core has changed since (finding 107). The setup line in scripts/mutants.sh now pins cargo-mutants 27.1.0, the version ci.yml installs: each count is the number of mutants that version lists, so another release can break the check (code review, 16 Oct; `git grep 'install.*cargo-mutants'` finds no other unpinned install). The check's report goes to stderr in scripts/mutants.sh, so `scripts/mutants.sh --list` prints only cargo-mutants' names: before, its first stdout line was the report; after, stdout is 773 lines (795 - 22) and stderr the one report line. ci.yml's `mutants` comment says the job checks the exclusions first. Row 105 and 09 §6 gave the pin's reason as the body's XORs. The finalizer's are at column 7 (`grep -n '\^=' floor.rs`, 337-342); the non-equivalent XOR at column 11 is `h ^= k`, in the body (320) and after the tail (335). The toml comment, the row and §6 now say that. 09 §8's `check` row names the two tool self-tests check.sh runs. README no longer calls run 10's 797 "injected bugs": cargo-mutants listed 797 but tested 778, 19 being excluded (09 §6, run 10's row). Sibling search: the other 14 patterns use `\d+` for the line, and each matches the count now written after it. The murmur3 one is the only line-pinned pattern. Not done, by choice: - Catching an edit that keeps a pattern's number but changes its mutants (an excluded line itself edited, or a murmur3 body XOR moved onto a pinned line). The toml and 09 §6 say so: such an edit needs the mutant judged again by hand. Failing on any diff that touches an excluded line would catch the first; it is a separate change. - Moving murmur3's tail into its own function, so its pattern needn't name lines. It shifts core lines, so it changes the program hashes. Until then a line move above it fails the check, with a message saying what to fix. - A weekly full mutants run in CI. It took 46 min here at -j4, likely past the job's 90 min on two CI cores: a minutes call for the owner. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test that a ticket stays in its own protocol and vault Two Vault checks that keep a ticket to its own protocol and vault had no test. Deleting either one left every program test green (tasks/error-codes/mutants.log, base cede705): - brakes.rs:79, extend_ticket: tk.protocol != protocol. Without it, a live guardian of any other protocol extends this protocol's tickets. Anyone can register a protocol and name its guardians, so that guardian could hold back every queued payout up to the total extension limit. The sweep did not catch it: it has no second protocol's guardian to swap in. - custody.rs:501, settle_ticket: tk.vault != vault. A USDC ticket settled through the SOL vault (its tokens, its mint) got past the Vault's checks. New test another_protocols_guardian_cannot_extend_a_ticket: a second Market's live guardian extends this market's ticket -> WRONG_VAULT; the same instruction from this market's guardian succeeds. The settle test now settles the USDC ticket through the SOL vault and expects WRONG_VAULT. Evidence (tasks/error-codes/mutants.log, STATE=c1; full program suite per mutant): W02 brakes.rs:79 deleted -> KILLED by another_protocols_guardian_cannot_extend_a_ticket ("expected failure, got success": the stranger's extension went through) W06 custody.rs:501 deleted -> KILLED by a_ticket_settles_after_its_eta_to_its_stored_recipient_only (8049 InsufficientBalance from the empty SOL vault, not 8010) Both were SURVIVED at base. No program source change: hashes 8ca8a608/543be60b/7086148b, as at cede705. Sibling search: the other 12 WrongVault sites were deleted one at a time in the same run. Eight are KILLED by the sweep (brakes.rs:39, lib.rs:320, custody.rs:160, genesis.rs:491, views.rs:27, lifeboat.rs:151, governance.rs:169/259). lib.rs:324 (vault PDA address), genesis.rs:523, lifeboat.rs:71 and :247 survive; later commits on this branch deal with them. Counts: Vault 104 -> 105, programs 154 -> 155 (claims.py OK). The test also reads the ticket's ETA: unchanged after the refused extension, 3,600 s later after its own guardian's (code-review fix: success alone did not show the extension happened). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test that the seal names every vault and happens once Two seal checks had no test; deleting either left every program test green (tasks/error-codes/mutants.log, base cede705): - genesis.rs:486, execute_seal: the pairs must number 3 * vault_count. Without it, the seal checks only the vaults it is given. A vault left out skips its claims-vs-balance check, so an uncovered import seals. docs/05's seal row says every vault is passed exactly once. the_seal_checks_each_vault_once covered a vault named twice in place of another, not one left out. - genesis.rs:431, propose_seal: refused once sealed (AlreadySealed). Without it, a sealed protocol's admin re-proposes, which rewrites seal_executable_s/expires_s and clears seal_vetoed on a sealed protocol. the_seal_checks_each_vault_once now also leaves the uncovered USDC vault out (expects VAULT_COUNT_MISMATCH) and, after the seal, proposes it again (expects ALREADY_SEALED). No new test, so no count change. Evidence (tasks/error-codes/mutants.log, STATE=c2, full program suite): E05 genesis.rs:486 deleted -> KILLED by the_seal_checks_each_vault_once ("expected failure, got success": the seal went through without the USDC vault and its 1,000 USDC uncovered import) E04 genesis.rs:431 deleted -> KILLED by the_seal_checks_each_vault_once ("expected failure, got success") Both were SURVIVED at base. Vault hash 7086148b unchanged (tests only). Sibling search: the seal's other checks. :491 vault_tokens (KILLED by the sweep), :519 lifeboat owner and :523 lifeboat mint/owner (SURVIVED; the lifeboat is a PDA checked by expect_pda just above, which only this program can create, as its own token account: see this branch's note in docs/09). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test that invalid limits and asset settings are refused at every entry Three bound checks had no test; deleting any one left every program test green (tasks/error-codes/mutants.log, base cede705): - genesis.rs:171, create_vault: config_is_valid (bps <= 100%, windows > 0). Without it, a vault is created with a 100.01%-an-hour burst. - governance.rs:40, parse of a SetLimits change (at proposal and again at execution, :254 and :478): the same rule for a proposed change. - governance.rs:47, parse of an UpsertAsset change: valid_asset (haircut <= 100%, liability >= 100%, the oracle bounds of finding 88). Only the genesis upsert's copy (genesis.rs:296) was tested, by oracle_settings_of_a_priced_asset_are_bounded. New test invalid_limits_and_assets_are_refused_wherever_they_enter: before the seal, create_vault with short_bps 10,001 -> INVALID_LIMITS; after it, a SetLimits proposal with long_window_s 0 -> INVALID_LIMITS, and an UpsertAsset proposal with a 10,001 bps haircut -> INVALID_ASSET. Evidence (tasks/error-codes/mutants.log, STATE=c3, full program suite): E06 genesis.rs:171 deleted -> KILLED ("expected failure, got success") E07 governance.rs:40 deleted -> KILLED ("expected failure, got success") E08 governance.rs:47 deleted -> KILLED ("expected failure, got success") each by the new test only. All three were SURVIVED at base. Vault hash 7086148b unchanged (tests only). Sibling search: every config_is_valid / valid_asset call in programs/ vault/src: genesis.rs:170 (create_vault), :295 (upsert_asset, KILLED by the oracle test), governance.rs:39 and :46. All now killed. Counts: Vault 105 -> 106, programs 155 -> 156 (claims.py OK). The execution copy (:478) is not reached by any transaction: it parses the bytes the proposal already checked, and the bounds read nothing else. The test comment says so (code-review fix: it had claimed execution was covered). oracle_settings_of_a_priced_asset_are_ bounded now asserts INVALID_ASSET for each of its four cases, not any error (all four return it; E09 stays KILLED). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test that an import into a ninth mint is refused genesis.rs:410, import_backing: a backing ledger holds MAX_SLOTS (8) mints, and an import into a ninth is refused with NoSlot. Deleting the check left every program test green (tasks/error-codes/mutants.log, base cede705). From the code, without it the import goes through: the vault's imported total grows (:414), and the seal must cover that, but the owner's ledger is saved unchanged, so the owner has no backed claim for it. New test an_import_into_a_ninth_mint_is_refused: nine vaults before the seal; one owner gets an import into eight (ledger_of shows 8 positions), then the ninth -> NO_SLOT. Evidence (tasks/error-codes/mutants.log, STATE=c4, full program suite): E14 genesis.rs:410 deleted -> KILLED by an_import_into_a_ninth_mint_is_ refused ("expected failure, got success"); SURVIVED at base. Vault hash 7086148b unchanged (tests only). Sibling search: the only other ledger credit is deposit's (custody.rs:128). There a full ledger is by design: the deposit is taken unbacked (Ledger::credit's doc, ledger.rs:187-189), and credit's result is the backed flag, not an error. Counts: Vault 106 -> 107, programs 156 -> 157 (claims.py OK). import_ix now calls import_into_ix, which takes the vault, so the new test shares the one import layout (code-review fix). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pin six Vault refusals no test reached Six reachable checks had no test that failed when the check was deleted (tasks/error-codes/mutants.log, base cede705). Each now has an assertion on its code, and each was seen to fail with the check deleted: E02 genesis.rs:185 create_vault of an existing vault after the seal: ALREADY_INITIALIZED (with the check gone: 8043 GenesisOnly, so the "says so first" order in the comment above it was untested) E12 governance.rs:426 veto of an executed change: CHANGE_EXECUTED (gone: the veto succeeds and marks an executed change vetoed) E13 governance.rs:517 cancel of an executed change: CHANGE_EXECUTED (gone: the cancel succeeds) E16 lifeboat.rs:256 execute_exit with no exit proposed: NOTHING_TO_EXIT (gone: 8033 ChangeNotExecutable) E17 custody.rs:84 a zero deposit: ZERO_AMOUNT (gone: it succeeds) E18 custody.rs:272 a zero release: ZERO_AMOUNT (gone: it succeeds; a zero release has no unbacked part, and the limiter, whose own zero check custody.rs:242 maps, is called only when unbacked > 0, :361) Where: E02 in after_the_seal_a_new_vault_starts_with_its_unbacked_lane_ closed, E12/E13 in an_unvetoed_loosening_executes_after_the_delay, E16 in the_lifeboat_stays_shut_until_the_seal (after the move in, before an exit is proposed), E17/E18 in the new a_zero_deposit_or_release_is_ refused. Evidence (tasks/error-codes/mutants.log, STATE=c5, full program suite): all six KILLED by the tests named above; all six SURVIVED at base. The vault suite is green (108 passed). Vault hash 7086148b unchanged. Counts: Vault 107 -> 108, programs 157 -> 158 (claims.py OK). Helpers (code-review fixes): veto_change_ix beside cancel_change_ix; create_vault_for_ix, create_vault_ix with a given mint and token program, so E02's case names USDC's own accounts and creates no mint. E02's comment now repeats the program's ("whatever the seal's state"): create_vault checks the limits first (genesis.rs:170). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pin four Vault checks another layer backstops Four checks survived deletion (tasks/error-codes/mutants.log, base cede705) because something later still refused the case, with its own error, so no test could tell. Each now has a test that sees the Vault's own code, and each was seen to fail with the check deleted: E01 lib.rs:230, create_pda's guard: a second registration of an existing protocol (register_protocol has no check of its own) -> ALREADY_INITIALIZED. Gone: the system program refuses it, with no custom code (custom_code 0). W10 lifeboat.rs:71, move_in naming another mint or token program -> WRONG_VAULT. Gone: the token program refuses (code 3, MintMismatch). W12 lifeboat.rs:247, execute_exit naming another mint or token program -> WRONG_VAULT. Gone: the same token-program error. T01 token.rs:37, account_info on an account no token program owns: a copy of a real USDC account's bytes under the system program. Gone: the quote SUCCEEDS (it promises a payout), and the release fails only in the runtime at the transfer ("instruction modified data of an account it does not own"). The quote is asserted first so the mutant is seen at the quote (STATE=c6b). New tests: a_protocol_registers_once, a_lifeboat_move_names_the_vaults_ own_mint (also completes a move in and an exit, as a control: the destination receives 1,000 USDC), a_payout_goes_only_to_an_account_a_ token_program_owns. Evidence (tasks/error-codes/mutants.log, STATE=c6 and c6b, full program suite): E01, W10, W12, T01 KILLED by the tests named above; all four SURVIVED at base. Full program suite green: Floor 50, Vault 111. Vault hash 7086148b unchanged (tests only). Counts: Vault 108 -> 111, programs 158 -> 161 (claims.py OK). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test the bounds on ticket extensions and a closed lane's code B's review of vault-error-tests-r3 (R1) found two Vault codes no test asserted by value that the earlier commits missed: ExtensionTooLong and LaneClosed. Deleting the 7-day total extension cap (brakes.rs:86, B24) left every program test green, so a live guardian, or a stolen guardian key, could hold a queued payout back without end, 24h at a time, against docs/05 row 11. The wide differential models the total (differential_wide.rs:406), but its default run never extends one ticket that far. Mapping LaneClosed to QueueFull (custody.rs:240, B23) also survived. The code review of the fix found a worse gap at the same check: no test sends a negative extension (the differential draws secs from 0, 1, 3600, 86400, 86401). With `secs <= 0` weakened to `secs == 0` (A3, brakes.rs:84) every program test passed, and one guardian key could pull a queued payout's ETA into the past, skipping the queue delay, and refund its own 7-day budget. - a_ticket_is_extended_later_by_at_most_a_day_a_call_and_seven_in_total: -1h, 0 and 24h + 1 s are each refused with EXTENSION_TOO_LONG, ETA unchanged; six 24h extensions and one of 23h pass; 2h more is refused, ETA unchanged; the last hour is granted; then 1 s more is refused. The split is unaligned so that a total that ignores this call's `secs` fails too. - a_closed_lane_refuses_an_unbacked_payout: a vault with short_bps 0, and one with long_bps 0, each refuse an unbacked admin payout with LANE_CLOSED; the same payout from a vault with both buckets open is taken (the control, so a capacity base of 0 cannot pass it). Evidence (tasks/vault-error-tests-r4/, B's runner copied: each mutant applied alone, programs rebuilt, whole program suite run): - without these tests (551e519; run-without.out, run-base2.out, run-base3.out): B23, B24, A1 (`tk.extended_s >= MAX`, this call's secs ignored), A2 (`>=` for `>`) and A3 SURVIVED. A4 (the 24h per call deleted), A5 and A6 (either half of limiter.rs:172's closed-lane test deleted) were KILLED by the differential only, not by value; - with them (run-final.out, at the branch tip): B23 KILLED by a_closed_lane_refuses_an_unbacked_payout; B24 A1 A2 A3 KILLED by the extension test only; A4 by it and the differential; A5 A6 by the closed-lane test and the differential. No program change: the runner's closing build gives 8ca8a608 / 543be60b / 7086148b, as at cede705. Counts: Vault 111 -> 113, programs 161 -> 163 (claims.py OK). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pin each half of create_pda's existing-account guard on its own B's review of vault-error-tests-r3 (R2): each half of lib.rs:229 (`!is_data_empty() || !owned_by(SYSTEM)`) survived deleted on its own (B20, B21), since an existing protocol account fails both, so a_protocol_registers_once pinned them only jointly. The test now also plants, at a fresh protocol address, an account that fails just one half: one with data owned by the system program, one with no data owned by another program (a fresh address). Each registration is refused with ALREADY_INITIALIZED. The comment says no transaction is known to make either account: data or a new owner at a PDA needs its signature, which only the Vault gives, and create_pda gives both in one instruction. Evidence (tasks/vault-error-tests-r4/): at 551e519 B20 and B21 SURVIVED (run-base2.out); at the branch tip both are KILLED by a_protocol_registers_once only (run-final.out). No count change (claims.py OK: vault_programs 113, programs 163). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Record the Vault refusal deletions and what each became docs/09 §3 gains a row for the deletion run this branch answers (tasks/error-codes/mutants.log): 38 refusals, across 14 of the 18 Vault codes no test asserted by value at cede705, each deleted on its own with the whole program suite run against it, on the AUD-2 build. The 18 (each VaultError variant with no `verr::` use in programs/tests at cede705): AlreadyInitialized NotRegistrar WrongVault WrongTokenProgram AlreadySealed VaultCountMismatch InvalidLimits InvalidAsset LaneClosed TicketNotPending ExtensionTooLong ChangeExecuted LedgerUnsound LedgerCorrupt NoSlot NothingToExit ZeroAmount Overflow. The run missed LaneClosed and ExtensionTooLong; B's review deleted one check of each (B23 custody.rs:240, B24 brakes.rs:86), and both survived until 1797ae5. That commit also pins a negative extension (A3, which survived every test before it), the 24h per call, and a lane closed by either bucket. At base: 13 KILLED, 25 SURVIVED. The commits before this one add tests that kill 18 of the 25 and both of B's, each asserting the Vault's own code (W02 W06, E04 E05, E06 E07 E08, E14, E02 E12 E13 E16 E17 E18, E01 W10 W12 T01; B23 B24). The row names the 7 that still survive and why: - six guards no transaction is known to reach: lib.rs:324 (vault PDA address; create_vault creates a vault only at its PDA), genesis.rs:519 and :523 (the lifeboat, checked by expect_pda first, created only by lifeboat_move, move_in requires the seal, lifeboat.rs:53, and the seal happens once, so no lifeboat holds data at it), brakes.rs:50 and custody.rs:541 (a pending-ticket count below zero), token.rs:235 (GetAccountDataSize's return data from an already-checked program); - custody.rs:399, stated as not shown unreachable: properties.rs:588-592 debits backed_amount and one more on random ledgers, and only crates/bulkhead-core/tests/units.rs:375 debits less, on a one-mint ledger with no liability. properties.rs:606 asserts backed(a) == min(a, backed(MAX)) for requests below 2^52 (:613), so any smaller amount is backed too, and debit_backed(b) refuses only when backed_amount(b) != b (ledger.rs:289). The known exception is a collateral value that overflows u128 (ledger.rs:262): the valid amounts need not form an interval there, so the consent or freeze cap (custody.rs:337, :347) can lower the amount to an invalid one and :399 refuses the release (fail closed). The search's own result is always valid. Not verified by a tool beyond that property test. (A full ledger cannot reach :399 through set's NoSlot: backed_amount's valid() refuses a new liability slot on a full ledger, ledger.rs:237.) Not deleted, as the row says: the two core-error mappings to ChangeExecuted and ZeroAmount, token.rs:233, and the codes Overflow and LedgerCorrupt (checked arithmetic, decoding). After this branch five codes are still asserted by no test (same grep at this commit): NotRegistrar (E03, genesis.rs:108, was KILLED at base by another test), TicketNotPending, LedgerUnsound, LedgerCorrupt, Overflow. Docs only: no count change (claims.py OK: vault_programs 113, programs 163). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Run the exclusion check's main in its self-test A's review of finding 105 (REVIEWS/auto-B-mutants-exclusions-r7.md, finding 1, Low): CI's `mutants` job acts on main's exit status, but the self-test called problems() only, and check.sh never runs the tool on real lists. With main returning 0 on a mismatch (mutant A10), every in-tree check passed, and CI would have printed the problems and gone green. This was not a defect at fb771c6, since the job fails today. The gap was that a later regression would pass. Fix: main takes its arguments and, for the self-test, a config path (default: the repo's .cargo/mutants.toml, as before), and its lines name the config it read. The self-test runs main in-process on a planted config and checks the exit status and every line on each stream: - the right lists: exit 0, "1 of 3 mutants excluded" on stdout, nothing on stderr; - the lists the other way round: exit 1, nothing on stdout, and exactly the count and "full list lacks" problems, so KEPT is read as KEPT; - coloured lists: exit 1 with exactly one problem, so a main that fails only on several problems is caught; - one argument and three: exit 2 and the two usage lines. One subprocess (python3 -I, 60 s timeout, in the temporary directory so the config must be found from the tool's own path) runs the command line with two lists and the repo's config, as scripts/mutants.sh does: on a list no pattern can match, it exits 1 with an empty stdout and problem lines that each name .cargo/mutants.toml. Which problems those are depends on the config, so only their form is checked. Self-test: 16 cases -> 22. No doc states the number. Evidence (tasks/exclusions-main-exit/): - old-A10.log: the fb771c6 self-test with A10 applied prints "self-test OK: 16 cases" and exits 0. - mutants-r6.log (mutants.py applies each mutant to a scratch copy of the tool, which reads the real config through a symlink): unmutated, "self-test OK: 22 cases", exit 0. All 18 are KILLED (exit 1): A10 (return 0), A10b (fail only on more than one problem), A0 (always 1), U1 (usage returns 0), U2/U3 (argument check `< 2`, `< 1`), K1 (KEPT ignored), K2 (ALL read twice), C1 (planted config ignored), C2 (no default config), R1 (the default config read from the cwd), E1/E2 (sys.argv sliced as [:] or [1:2]), P1 (problems not printed), S1 (problems on stdout), L1 (the planted config not named), N1 (only the first problem printed), G1 (the success line printed on failure too). Code-review passes at high shaped this. The first version ran three subprocesses against the real config, and A0, K1 and U3 survived it. The second left E2, C2 and G1 alive. Declined from the third: catching exceptions in the self-test (a traceback already exits 1) and moving the config hook out of main. The fourth showed that requiring one "matches 0 mutants" line per live pattern tied the self-test to the config's contents. The subprocess now checks the form of the lines only, and the 17 stay killed. The fifth showed that R1 survived while the child ran in the repo root (it does in check.sh); now it runs in the temporary directory. Its other points repeat declined ones. - real-lists.log: on the real lists, made with scripts/mutants.sh's commands: ".cargo/mutants.toml: 22 of 795 mutants excluded, each pattern matching its count", exit 0. - python3 -I tools/mutant_exclusions.py --self-test: 22 cases OK. - scripts/check.sh: ALL CHECKS PASSED (check-final.log); bin/check-each-commit.sh green (each-commit.log). Sibling search: the other tools check.sh runs (tools/claims.py, tools/floor_check.py) also gate CI through main's exit status. claims.py has the same gap: check.sh runs it on the real tree, where it finds no problems, and its self-test calls self_test(), not the path in main that returns 1. With that return changed to 0, `--self-test` and the real run both still exit 0 (claims-C1.log). Not changed here: it is its own commit (BACKLOG-B). floor_check.py runs in CI only as --self-test, whose result main returns as is. Its RPC mode is an operator's tool that no CI job runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Record finding 106: Vault liveness and exit rules had no failing test The tests landed in 944f5e8 (lifeboat-exit-tests-r3, 15 Oct). Their 09 §4 row waited for 104 and 105 so the list stays numbered without gaps; 105 is now on integration (fb771c6), so the row lands here. Docs only: 09 §4 row 106; README and 12's count and post-submission range (96–106, 106 fixed). Two code-review passes changed the row as written on 15 Oct: a sentence for today's re-run, the nine single-guardian sites named, G5 quoted verbatim, and the exit veto attributed to T18 (G6 has no veto), as is the veto test's doc comment (vault.rs:3996, comment only). Evidence, re-run today on bf8ba41 (the row's claims are from 15 Oct, and AUD-2 changed the Vault since): B's re-review list of 33 line- neutral mutants (the ten rules, the nine single-guardian sites L1-L9, the boundaries, the six quorums, R1-R6), each applied, programs rebuilt, all program tests run: 33 KILLED, 0 survived, 0 not applied, 0 build failures. Vault hashes after the run match the tip (Vault 7086148b, Floor 8ca8a608, Harbor 543be60b). Log: autopilot tasks/finding-106-row/mutants.log. - python3 -I tools/claims.py: OK, findings 106. - git grep for "96–105", "105 findings": no other stale range. - scripts/mutants.sh still mutates the core only (SCOPE, line 11), so the row's last sentence holds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Cite PR #100's tree, not the unpublished bf8ba41, in finding 106 Finding 106's row in docs/09 said its 33 mutants were re-run on 16 Oct at bf8ba41. That commit is not an ancestor of this branch or of the stack above it and is on no remote branch, so the citation would point at nothing once the stack lands. The owner's review of #99-#109 found it. bf8ba41 and 86b777b (the head of PR #100, vault-error-tests-d2, the D2 replay of vault-error-tests-r4) have the same tree, 36187b49 (`git rev-parse bf8ba41^{tree} 86b777b^{tree}` prints it twice). So the run was a run on 86b777b's tree, and the row now says "on tree 36187b49, the tree of 86b777b (PR #100's head)". Why not just "at 86b777b": it was committed on 17 Oct, after the run; and main squash-merges (1045cd9 has one parent), so after PR #100 lands 86b777b is reachable only as GitHub's refs/pull/100/head. The tree id stays checkable from main: squashed bottom first, main's commit for PR #100 has that tree. "PR" keeps "#100" from reading as finding 100, the row above. These points came from two code-review passes. Sibling search: every 7-12 hex token in tracked .md/.py/.sh/.mjs/.rs/ .toml files at autopilot/integration 3e9f58d that names a local commit (51 tokens) was checked with git merge-base --is-ancestor against integration. bf8ba41 was the only one that is not an ancestor. Docs only. tools/claims.py: claims OK (26 files). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Warn about a Floor guardian set short of the margin (finding 107) The guardian checks the Vault protocol's guardian set against finding 107's margin (main.mjs checkGuardianSet), but never a Floor config's. A Floor set registered before the rule, say 2-of-3, keeps working and keeps AUD-2's exposure: one lost key and one rogue guardian lock its governance for good. The guardian said nothing, and docs/07 §6 said so ("not yet a Floor set"). The rule moves to policy.mjs as guardianSetWarning, a pure function: the same text as before, a warn line that alerts on mainnet only. In main.mjs, checkGuardianSet now remembers the set shape per set ("protocol", or the protected program), and checkFloor calls it for each watched Floor config it reads. floorAlerts is unchanged. The margin is not a readiness code there: "floor not ready" alerts on every cluster, and only the owner's new set clears a margin problem, so "floor ready" would never be logged again. Evidence (tasks/floor-guardian-margin/r2/): - node --test cloud/test/*.test.mjs: 180/180 pass (one new test). - Mutants of guardianSetWarning, each run on policy.test.mjs and reverted; each fails the new test: N1 never warn, N2 threshold fixed at 2, N3 notify always, N4 notify never, N5 GUARDIAN_MARGIN = 1. - tools/claims.py: OK, client 180 (docs/09:40 updated). - Not tested: the main.mjs wiring (the per-set Map and the checkFloor call). No test runs checkFloor; deleting the call leaves every test green. Sibling search: guardianMarginProblem's callers are vault-check.mjs (an info line) and guardianSetWarning. The status snapshot (snapshot.mjs:33) prints live guardians and the quorum, not the margin; left as it is. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Judge each guardian set's margin on its own, and say when it is fixed The parent's per-set memory (which set shapes were already reported) lived in main.mjs, so no test could fail on it: dropping the once-per-shape check, or keying every set the same, left every test green. The warning did not say which set it was about (only a program tag on Floor lines), and it gave the same text for a set ejected down to its threshold, or registered n-of-n before finding 96. There one rogue guardian alone locks governance; a lost key alone does not, since dead guardians cannot veto the owner's new set (docs/16 §4.4), but it leaves no quorum until that set executes. Nothing was said when the owner's new set fixed the problem. And a delivered alert did not say which Floor program it was about: the program went only into the line's `program` field, which alertFor uses in its repeat key but does not show. Moving the program name into the text is a trap: alertFor keys repeats on the first 60 characters of the text, and "the Floor guardians of <44-character id>" pushes the shape past them. A 2-of-3 set ejected to 2-of-2 within ten minutes would then give the same key, and the worse warning would not alert. So the program goes after the shape: "the Floor guardians are 2-of-3 for <program>: ...". The `program` field stays for the repeat key. guardianSetWarning becomes guardianSetCheck(shapes, which, set, mainnet) in policy.mjs. It keeps each set's last shape and whether it was short in the Map, keyed by "protocol" or the protected program, and returns the line to log or null. The warning names the set and has its own text for n-of-n. When a set that was short gets the margin, it returns one line saying so, which alerts on mainnet like the warning, so whoever was paged hears that it is fixed. main.mjs only logs what it returns. docs/07 §6 says so. The advice also had a hole: a set registered as 7-of-8 was told to "run at least 7-of-9", and no set can have 9 keys (core MAX_GUARDIANS is 8). When threshold + 2 is over 8, the advice is now to lower the threshold to at most 6-of-8. That needs MAX_GUARDIANS in JS, so lib/solana.mjs gets it next to GUARDIAN_MARGIN; the next commit ties both to the core. Evidence (tasks/floor-guardian-margin/): - node --test cloud/test/*.test.mjs: 180/180 pass (the test is extended, not added). - 19 mutants, each run on policy.test.mjs and reverted (r6/mutants.py, r6/mutants.log, r6/N5-margin1.log). Each fails the extended test, and only it: never warn, threshold fixed at 2, notify always, notify never, GUARDIAN_MARGIN = 1, no once-per-shape, one Map key for every set, the n-of-n text for threshold + 1, no fixed line, a fixed line for a set that was never short, the fixed line never alerting, no program tag, every set named as the protocol's, the short flag not stored, the program before the shape, the program left out of the text, `need < 8` for `need <= 8` (6-of-7), the 8-key limit ignored (7-of-8), 7-of-8 as the lowered advice. - The mainnet alerts are checked on their own in a scratch copy of the test without the earlier text assertions (r5/C1-C3-alone.log): with the program left out of the text, the alert's detail check fails; with a long name pushing the shape past character 60, "2-of-2 alerts" fails. - A's review of the previous version (branch auto/B/floor-guardian-margin-r7, kept) found two survivors there: X3, the shape keyed on the count only, and X5, the advice threshold-of-(n+1). It also found that the n-of-n text said a lost key alone locks governance. The test now has a threshold-only fix (7-of-8 -> 6-of-8, the advice's own step) and pins the n-of-n text and advice. r9/mutants.log: X3, X5, the r7 n-of-n text and the text without its lost-key clause are all KILLED. r9/mutants-r7.log: X3 and X5 SURVIVED on the r7 code and test. The 18 mutants of r6/mutants.py, re-run on this code: all KILLED (r9/r6-mutants-rerun.log). Two code-review passes on the fix (kept as -r8 and -r9) added the lost-key clause: a lost key alone leaves no quorum (it never approves, and core `quorum` needs threshold approvals) until the owner's new set executes. A third pass (-r10) fixed the comments' wording only. - tools/claims.py: OK, client 180. - Still not tested: the checkFloor call in main.mjs (no test runs checkFloor). Not changed (code review): the "fixed" line needs the short shape to have been seen since the guardian started (the Map is in memory, as the old per-process shape was; docs/07 says so). A protocol that abdicated SetGuardians can't take a new set, so the advice to run 2-of-4 can't be followed there; REPORT lists that case for the owner. The fault text assumes a margin of 2, the owner's decision (16 §4.4). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Tie the JS guardian margin and set size limit to the core constants cloud/lib/solana.mjs keeps its own GUARDIAN_MARGIN = 2, which the guardian and the deploy tooling use to judge a guardian set, and now MAX_GUARDIANS = 8, which the guardian's advice uses. Their comments say they are core governance.rs's constants, but nothing checked that. Had the programs' margin moved to 3, every JS test would have stayed green while the guardian passed sets the programs refuse. deploy.test.mjs now reads governance.rs and requires exactly one `pub const GUARDIAN_MARGIN: u8 = N;` and one `pub const MAX_GUARDIANS: usize = N;`, each N equal to the JS constant. It reads the source text, like the build-script test above it. If a declaration changes form, the test fails rather than passing. Evidence (tasks/floor-guardian-margin/r6/tie-mutants.log): - node --test cloud/test/*.test.mjs: 181/181 pass (one new test). - Mutants of governance.rs, each reverted with its mtime kept (else the limiter differential asks for a worker rebuild): GUARDIAN_MARGIN = 3, MAX_GUARDIANS = 9, and a trailing comment on MAX_GUARDIANS. Each fails the new test, naming the constant. deploy.test.mjs is green again. - tools/claims.py: OK, client 181 (docs/09:40 updated). Sibling search: the other JS copies of core rules (the limiter port, the account layouts) are checked by the differential and by decoding real accounts. MIN_GUARDIAN_THRESHOLD has no JS copy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Reach the 7-day extension total in the wide differential differential_wide.rs drew extension seconds only from {0, 1, 3600, 86400, 86401} on a random ticket and lumped every refusal into one counter. A ticket never got near the 7-day total, no negative value was sent, and a refusal was not checked by its code. So the default run could not tell the total bound, a negative extension or the error code from a bug: only the unit tests at vault.rs:3382-3401 pinned them. Now one extension draw in three pushes the chosen ticket by up to 24h a call to exactly the 604,800 s total, then asks for 1 s more, so the edge itself is hit on every push, not only when earlier draws line up. The other draws are half legal sizes (1, 3600, 86400) and half out of range for one call (i64::MIN, -86400, -1, 0, 86401, i64::MAX). Each refusal must fail with EXTENSION_TOO_LONG. The counters are split into extend (a random legal size), extend_push, extend_refused_call and extend_refused_total, all required >= 10, so pushes cannot stand in for the random sizes. The model reads both bounds from bulkhead-core (consts.rs:37-38), as the module says it is built, rather than restating 86,400 and 604,800. docs/09 §7's row says so, and the 16 Oct "Vault refusals deleted" row, which says the default run never extended a ticket that far, now says it does since this change (its differential_wide.rs:406 cite had drifted). Default run (24 seeds x 250 steps): extend 16, extend_push 126, extend_refused_call 16, extend_refused_total 30 (was extend 40, extend_refused 27); every other required counter still >= 10 (loosen_executed 12, as before). Evidence: mutants of brakes.rs extend_ticket, each built and run against the old and the new differential_wide only: T1 drop the total check old SURVIVED new KILLED T2 total > to >= old SURVIVED new KILLED T3 drop secs <= 0 old KILLED new KILLED T4 secs <= 0 to < 0 old KILLED new KILLED T5 total ignores this call's secs old SURVIVED new KILLED T6 extended_s += secs / 2 old SURVIVED new KILLED T7 refusal returns WrongTicket old SURVIVED new KILLED T8 secs <= 0 to == 0 (negatives) old SURVIVED new KILLED T9 total > to > total + 1 old SURVIVED new KILLED (T9: a draft that pushed by 24h until refused killed it only through chance 1 s and 3600 s draws on the same ticket; the push to exactly 604,800 then 1 s more reaches the edge every time.) sBPF sha256 after the run unchanged (Floor 8ca8a608, Harbor 543be60b, Vault 7086148b): tests only. Sibling search: the other refusal counters in this file (reset_refused, lifeboat_refused_*, loosen_*) are each split by cause already. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Check settle and lifeboat refusals by code in the wide differential The same gap as the extension refusal, found in the sibling search for the previous commit: a refused settle and the three lifeboat-move refusals asserted only r.is_err(). Any refusal passed, so the order of the checks and the program's own codes went unchecked by the differential. Settle's comment (custody.rs:520-521) says it returns its own error, not the token program's, when the vault is short; nothing in this file held it to that. Now a settle refused while frozen must fail with FROZEN, and one that is only short with INSUFFICIENT_BALANCE (frozen is checked first). A lifeboat move inside the delay after the seal must fail with NOT_SEALED, one while frozen with FROZEN, and one beyond the balance with the token program's InsufficientFunds (1). The two settle refusals get their own counters: settle_refused_frozen is required >= 10 like the rest; settle_refused_short needs lifeboat moves to drain a due ticket's tokens first and occurs 7 times in the default run, so it is required >= 1. Evidence: mutants of programs/vault/src, each built and run against the previous head's differential_wide and this one, nothing else: S1 settle: drop the InsufficientBalance check old SURVIVED new KILLED S2 settle: balance check before the freeze old SURVIVED new KILLED L1 require_sealed returns Frozen old SURVIVED new KILLED L2 move_in's freeze returns NotSealed old SURVIVED new KILLED The token-program code (1) on a move beyond the balance is pinned but not shown red by a mutant. sBPF sha256 unchanged (Floor 8ca8a608, Harbor 543be60b, Vault 7086148b): tests only. Deep run of both commits together (BULKHEAD_PROGRAM_SEEDS=600 BULKHEAD_PROGRAM_STEPS=500, 85 s): 300,000 steps, all identical; extend 876, extend_push 6,462, extend_refused_call 1,175, extend_refused_total 1,453, settle 2,024, settle_refused_frozen 921, settle_refused_short 566, lifeboat refusals 3,854 + 5,768 + 1,171. docs/09 §7's row gets that run as a 16 Oct result next to the 5 Oct one, which predates these checks. Sibling search: in differential_wide.rs, reset (TICKETS_PENDING) and execute change already asserted codes, and a release or quote refused for want of tokens asserts INSUFFICIENT_BALANCE. A release or quote the limiter refuses (frozen, lane closed, queue full) is still compared only as a refusal: mapping the core's refusal to the program's code is left for its own change. Lifeboat returns only send legal amounts. docs/09's row now says which refusals are checked by code. differential.rs:327 has the same is_err settle check; left as it is, since the wide run now pins both settle codes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Check limiter refusals by code in the wide differential A release or quote that the limiter refused (lane closed, queue full) was compared with the model only as Err. Any refusal code passed: a program that swapped LaneClosed and QueueFull in custody.rs's limiter_error, or checked the backed part against the balance before the limiter and so answered InsufficientBalance for a full queue, still matched the model. About 670 such refusals happen in a default run. The model now maps core's LimiterError to the program's code with verr::limiter in the test harness (all four variants, as custody.rs's limiter_error does), and every refused quote and release must fail with the model's code. Lane-closed and queue-full refusals have their own counters, each required at least 10 times; a refusal code with no counter panics rather than being counted as want of tokens. docs/09 row 362 says so and records the deep run. Evidence (autopilot tasks/differential-limiter-codes/): - mutants.log: C1 (LaneClosed and QueueFull swapped in limiter_error), C2 (QueueFull -> InsufficientBalance), C3 (LaneClosed -> ZeroAmount) and C4 (backed > balance checked before the limiter, same line) each SURVIVED the old test and are KILLED by the new one. Hashes back to 8ca8a608/543be60b/7086148b after. - c4-full-suite.log: under C4 the whole program suite (Vault 113, Floor 50, the fixed differential) passes; only this test fails. C1-C3 are also caught by vault.rs's code tests (:434, :3428). - Default run: refused_lane_closed 254, refused_queue_full 414, insufficient 118. - deep-600x500.log: 300,000 steps, 15,561 lane closed, 18,853 queue full, 6,345 want of tokens, all identical. Sibling search: every other Err arm in differential_wide.rs already asserts a code (extend, settle, reset, lifeboat in, execute change). Not here: lifeboat returns are never refused (backlog), and lifeboat move_in beyond the balance fails with the token program's code (backlog, a program change). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Check release refusals by code in the fixed differential differential.rs reduced every refused release to Err(()) on both sides, so a program that refused for the wrong reason still matched the model. The same gap the previous commit closed in the wide differential (found by its code review). The model's refusal now carries the program's code: verr::limiter for a limiter refusal, INSUFFICIENT_BALANCE when the vault holds less than it would pay. A refused release must fail with that code, and the assert prints the program's error. Refusals for a full queue get their own counter, required at least 10 times like the others. Want of tokens and a closed lane are counted and printed but not required: the wide differential requires both. A refusal with any other code panics, as in the wide differential, so the counts always add up. docs/09 row 361 says so; its undated figure (4,843 releases) no longer matched a run, so it is replaced by a dated one. Evidence (autopilot tasks/differential-limiter-codes/): - mutants-narrow.log (mutants.sh <wt> differential): C1 (LaneClosed and QueueFull swapped), C2 (QueueFull -> InsufficientBalance) and C4 (backed > balance checked before the limiter) SURVIVED the old test and are KILLED by the new one. C3 (LaneClosed -> ZeroAmount) survives both: no release in this test's runs is refused for a closed lane. The wide differential kills it, and docs/09 says so. Hashes back to 8ca8a608/543be60b/7086148b after. - r2/default-run.log: 1,000 steps, 82 releases refused, all for a full queue (want of tokens 0, lane…
shuv-amp
added a commit
that referenced
this pull request
Oct 18, 2026
* Run the tests on 2-of-4 guardian sets; test recovery from two faults
Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set
with fewer than threshold + 2 keys. Every test fixture and most tests'
own sets were 2-of-3. This moves them to 2-of-4, which today's rule
also accepts, so this commit changes no program: Floor, Vault and Harbor
hash 508e5a99 / b07655d9 / 543be60b, as before.
- Fixtures: Protected (Floor) and Market (Vault, through Harbor) now
hold 4 guardians. Each test that built its own 3-key set gets a 4-key
set with the same threshold and the same assertion.
- Two tests changed meaning with a 4th key and were fixed, not loosened:
a_quorum_ejects_a_rogue_guardian_who_vetoes_everything (Floor) and
a_quorum_ejects_a_guardian_who_freezes_forever (Vault). With 4 keys,
the last eject was refused only because the target's own vote is
masked. Both now eject a second guardian (a set an ejection shrank to
threshold + 1 still ejects) and then show the 2-of-2 refusal.
- Account sweeps: an eject's signers may now be any two of the three
others, so each of those two slots is free for the third, never for
the target or the other signer again. Vault 5,365 and Floor 1,710
substitutions, refused (docs/09, dated 16 Oct). Listing all three
others at each slot would have skipped the duplicate: with a mutant
that counts a guardian listed twice as two (governance.rs:270), the
sweeps listing all three pass, and these fail at "eject: account 2
replaced by key ..." (Vault and Floor). No other program test kills
that mutant (tasks/aud2-guardian-margin-impl/dup-mutant-*.log).
- floor_differential: the guardian pool is 6 keys, 4 registered; the
proposed sets are 2 to 6 keys.
New tests (pass on both rules; they show what the margin buys):
- a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_set (Floor): g0's
key lost, g1 rogue stops the program; the lost key does not help a
quorum; the honest pair ejects g1 and restores; neither can veto the
owner's refill, which executes after 72h.
- two_rogues_in_turn_are_both_ejected_from_a_two_of_four_set (Floor):
the second rogue vetoes the refill and is ejected from the 2-of-3.
- a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_vault: the same in
the Vault (freeze, eject, unfreeze, refill).
Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest
and quote sweep): the counts and the sweep figures here are from runs
there (tasks/aud2-guardian-margin-impl/r4/). The E2E run, the mutants
and the new-rule run below were made before, on e8640a7; this commit's
programs and core are the same as there but for one comment line in
views.rs (line-neutral, same hashes).
Evidence:
- E2E=1 scripts/check-programs.sh: PROGRAM CHECKS PASSED (Floor 49,
Vault 98; S1 and E2E passed; the resumed setup found all 14 steps;
vault check, Floor demo and deploy dry run passed), with the next
commit's deploy rehearsal in the tree.
- tools/claims.py: OK (floor 49, vault 104, programs 153).
- Under the new rule (core from two commits on, programs rebuilt), only
the two tests that state the old rule fail: an_n_of_n_guardian_set_is_
refused (accepts 3-of-4) and the Floor differential's model.
- Mutants (tasks/coverage-map/mutants.sh), each red, then restored:
AV1 Floor live_guardian ignores liveness: KILLED by a_lost_key_and_a_
rogue_do_not_lock_a_two_of_four_set (+2 others).
AV2 the Vault's: KILLED by ..._two_of_four_vault (+2 others).
AQ1 Floor restore quorum ignores liveness: KILLED by ..._two_of_four_set
and the differential only.
AQ2 the Vault's unfreeze quorum: KILLED by ..._two_of_four_vault and
a_silent_guardian_does_not_count_toward_a_quorum.
AE1 can_eject refuses at count == threshold + 1 (AUD-2 option (a)):
KILLED by two_rogues_in_turn_..., and the two eject tests above.
Hashes after: 508e5a99 / 543be60b / b07655d9.
Split on 16 Oct from ec0cacc at the owner's review (D2): the deploy
rehearsal's move to 4 guardians is now the next commit. programs/ and
programs/tests/ here are byte-identical to ec0cacc, so its runs below
hold for this commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Rehearse deploys with four guardians, and read only the keys there are
Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set
with fewer than threshold + 2 keys, so the deploy rehearsal must
register 2-of-4 sets. Split from ec0cacc at the owner's review (D2);
the text is that commit's, unchanged:
Deploy rehearsal (cloud/deploy market.mjs, vault-check.mjs,
floor-demo.mjs): 4 guardians (keys/guardian3), so the dry run keeps
working once the rule lands; one more funding step (14, was 13).
The devnet market has 3 guardian files. A first version of this change
loaded guardian3 everywhere: vault-check.mjs failed there with ENOENT,
and heartbeat silently created a key nobody holds as a guardian. Now
market.mjs guardianKeys(dir, create) reads guardian0, 1, ... and
throws on a gap (never fills it with a new key). Setup creates keys up
to 4 only while the protocol is not registered, so resuming an old
market adds no key the chain lacks; a new deploy-dryrun step resumes
the dry-run market with guardian3 moved aside and fails if setup makes
it again (with setup always creating: "DRY RUN FAILED: resuming a
registered market made keys/guardian3.json", r3-dryrun-J7.log).
renew-guardians tops up to 4 and logs each new key (unfunded); a
2-of-3 renewal would be refused once the rule lands. keysIn reads the
guardians only when a command uses them, so caps and set-recovery
never need them; heartbeat says a key is not in the set rather than
dead. One THRESHOLD (2) for the register and renewal payloads and
MIN_GUARDIANS. vault-check.mjs
compares the chain's set with the first files, and liveness with the
chain's count. An extra file (a renewal not executed yet) and a set
without the margin are info lines, not failures, since such a set
keeps working. GUARDIAN_MARGIN and guardianMarginProblem live in
cloud/lib/solana.mjs, used by these and by the guardian bot's existing
warning (cloud/guardian/main.mjs), and vault-check uses THRESHOLD. The e2e scripts fund guardian3 too. Three
tests in cloud/test/deploy.test.mjs (client tests 176 -> 179). JS
mutants (js-mutants-r3.log), each red then restored: always create,
never create, no gap check, margin 1, `>`, GUARDIAN_MARGIN = 1.
Setup always creating survives these tests and is killed by the dry
run step above.
cloud/, scripts/deploy-dryrun.sh and docs/09's client-test row are
byte-identical to ec0cacc (git diff empty), so its runs hold for them:
the E2E check-programs run with the dry run (r4), and the JS mutants.
tools/claims.py: OK (client 179).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Refuse a guardian set without two keys beyond its threshold (107)
AUD-2, decided by the owner on 16 Oct (option (e) of 16 §4.4). A set
stays live while its honest live guardians reach the threshold: they
eject every rogue in turn, and dead guardians cannot veto the owner's
new set. Finding 96 asked for one key beyond the threshold, so a 2-of-3
or 3-of-4 set survived one fault but not a lost key plus a rogue: the
honest guardian left is no quorum, nobody can be ejected, and the rogue
keeps Floor stopped (or the Vault frozen) and vetoes every new set.
Fix, in the core so both programs inherit it through GuardianSet::parse
(registration, and every SetGuardians when proposed and when executed):
GUARDIAN_MARGIN = 2
guardian_set_ok := count <= 8 && t >= 2 && count >= t + 2
(saturating_add: t is a payload byte). A stored set is not checked
again, so existing 2-of-3 sets keep working, and ejection is unchanged:
a shrunk set still ejects. A 2- or 3-key set is BadThreshold, not
BadCount; the count rule is only the bitmap size, and the error order
of finding 103 is kept.
This changes both programs, by intent:
Floor 508e5a99 -> 8ca8a608, Vault b07655d9 -> 7086148b,
Harbor 543be60b (unchanged). Darwin-arm64 builds.
Compute (scratch LiteSVM test, tasks/aud2-guardian-margin-impl/cu.md):
-3 CU on every path that parses a set (Floor propose 2035 -> 2032,
execute 1682 -> 1679; Vault register 9592 -> 9589, SetGuardians 2979 ->
2976; Floor register varies with its PDA bump, residue also -3).
Proof and tests:
- Kani: accepted_sets_can_eject_anyone restated (ejection reachable
exactly when count > t, so option (a) cannot slip in; accepted sets
have >= 4 keys and a margin of at least 2, and 2 is accepted). New:
an_honest_live_quorum_ejects_any_rogue (any slots, anyone else alive)
and accepted_sets_survive_any_two_faults (any two faulty keys, lost
or rogue, leave a live honest quorum, which ejects each rogue, and the
rogues alone eject no honest guardian). That holds only short of a
rogue quorum: two rogues in a 2-of-n set are one (out of scope, 04
T7). With the assumption rogue < t dropped, the harness fails at
`!can_eject(rogue, ..)` (kani-no-rogue-assume.log), so the docs say
"short of a rogue quorum", not "any two faults".
19/19 verified, 240 s (cargo kani -j 2, as CI); again after the
restatement, 235 s summed (r2-prove.log).
- Kani mutants (kani-mutants.sh): margin 1, margin 3, `>` for `>=`,
t >= 1, count <= 9, option (a), and an eject one vote short (only
the two-fault harness kills that one): each KILLED by a harness
(kani-mutants-r2.log). The eject
ignoring liveness is the eject harness's case, not these.
- Program mutants (coverage-map/mutants.sh, rebuilt each time):
margin 1: KILLED by a_guardian_set_without_two_spare_keys_is_refused
(Floor and Vault), registration_rejects_weak_settings and the Floor
differential. Margin 3 and `>`: KILLED by most of the suite (the
fixtures are 2-of-4). t >= 1: first only the Floor differential; the
1-of-2 registration case was now refused by the margin too, so it is a
1-of-4 here and a Vault 1-of-4 case was added: then KILLED by both.
count <= 9: SURVIVES the program tests, equivalent there (parse refuses
more than 8 keys first), KILLED by Kani. Option (a): KILLED by the
eject tests two commits back (the tests on 2-of-4 sets).
- Tests: the Floor and Vault refusal tests above cover proposal and
execution (a stored 2-of-4 proposal whose threshold turns 3 is refused
when executed, as the devnet renewals proposed before this rule will
be); registration_rejects_weak_settings gains 2-of-3 and 3-of-4.
an_n_of_n_guardian_set_is_refused (Vault) is renamed and accepted a
3-of-4: it now refuses it and accepts 3-of-5.
- Model: the Python rule, the Floor differential's model, and the
guardian_set fuzz target state it again; the GP generator now also
picks count - 2. With the Python rule put back to the old one, seed 11
(20,000 cases) reports 5 GP mismatches; restored, seeds 11-13 identical.
Fuzz seeds two_of_three_refused, two_of_four, six_of_eight, and
seven_of_eight renamed seven_of_eight_refused (byte 323, the
threshold, is 7: now refused). With margin 1 in the core the target
crashes within 60 s; restored, 222,615,288 runs in 601 s, no crash.
- scripts/check.sh: ALL CHECKS PASSED, and E2E=1 check-programs.sh:
PROGRAM CHECKS PASSED, both at the branch tip, whose code is this
commit's (the next commit is docs only; tasks/aud2-guardian-margin-
impl/r4/check.log, r4/check-programs-e2e.log). claims: kani 19,
floor 50, vault 104, programs 154, client 179.
Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest
and quote sweep). The core and the program sources are the same as
before the rebase but for one comment line in views.rs (line-neutral),
so the Kani, fuzz, differential and mutant runs above were made on the
pre-rebase commit; the hashes, check.sh, check-programs and the counts
are from r4 runs (tasks/aud2-guardian-margin-impl/r4/).
The finding 96 row said the restated harness proves "exactly"; it now
says what it proved then and proves now, and gives the Vault test's new
name. The finding 107 row waits for rows 105 and 106 to land (09 §4 is
numbered 1..n); its text is ready. The rule's prose is the next commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* State the guardian margin in the docs, with its costs (AUD-2)
The previous commit made both programs refuse a new guardian set
without two keys beyond its threshold (finding 107). The docs that
state the rule now say so:
- README: guardians 4-8 keys; the G5 row and the "who the guardians
are" paragraph give the reason (any two faults leave a quorum, unless
the rogues are one themselves: two in a 2-of-n set, 04 T7).
- 05 register_protocol: 2 <= threshold <= count - 2, also for every
SetGuardians, proposed and executed. 05 abdicate: abdicating
SetGuardians fixes the set, so ejections shrink it for good.
- 06 state table, 00 D22, FAQ: the same rule.
- 07 §2: 4-8 guardians, and don't abdicate SetGuardians. 07 §6: the
programs enforce the margin; sets registered before keep their
exposure until replaced; an ejection uses up a spare key until the
owner's new set executes; a lost key counts as live until its
interval passes.
- 16: the AUD-2 row is decided; §4.4 gives the decision, why the
margin is the number of faults a set survives (short of a rogue
quorum), the options rejected
and why (the honest-owner and stolen-owner cases look the same
on-chain), what it costs, what it leaves (a rogue quorum; a stolen
owner key with every guardian dead; a rogue's veto of the recovery
key's rotation, checked: Floor's veto has no recovery exception, as
a_vetoed_recovery_proposal_does_not_hold_the_owner_s_slot shows), and
Lido-style lapsing vetoes as a future option only. §5: "independent
guardians, 3-of-5 or more".
- 10: the guardian set's minimum size is no longer an open decision.
SECURITY.md and 08 describe what is on devnet, which still runs the
old rule, and are left as they are.
claims: OK. Docs only; no program file changed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Say the weekly CI run skips mutants, as its condition does
ci.yml's header and scripts/ci-changes.sh said the weekly run does every
job, and 09 §8 gave `mutants` as running on any change to the core. The
`mutants` job's `if` starts with `github.event_name == 'pull_request'`,
so the Monday schedule (and a manual dispatch) never runs it: it mutates
a pull request's diff and has none to mutate. Its trigger list in 09 §8
also left out scripts/mutants.sh and .cargo/mutants.toml, which
ci-changes.sh (line 22) selects it for.
Comments and docs only; nothing runs differently.
Sibling search: `git grep` for "weekly run does", "every job ... weekly"
and "mutants ... weekly" across md/yml/sh/mjs/py. The other hits are
09 §8's "at full depth every Monday" (its table now says which job is
not weekly) and the dispatch input's "every job at weekly depth", which
is the same set as the weekly run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check that the mutant exclusions still match; repin murmur3 (finding 105)
.cargo/mutants.toml excludes the core's 22 equivalent mutants, and the
docs say so. Since 9 Oct it excluded 19. The murmur3 tail's pattern is
pinned by line, `floor.rs:(235|238|241)`, because the body's `h ^= k`
has the same column and function and is not equivalent. 1dc7861 (#52)
moved those lines (now 327/330/333), and nothing checked the patterns.
Failing closed, the three would have failed CI's `mutants` job on a
pull request that changed those lines. Failing open, a pattern that
starts matching a non-equivalent mutant hides it from every run.
Root cause: nothing checked that a pattern still names its mutants.
Fix:
- each pattern ends with the number of mutants it excludes (`# 3`), and
the murmur3 pattern names 327/330/333;
- tools/mutant_exclusions.py fails unless each pattern matches exactly
its number in `cargo mutants --list --no-config`, and cargo-mutants
(the list with the config) excludes exactly the mutants the patterns
match, so a pattern this parser can't see, or reads differently, fails
too. No two patterns may match the same mutant, since the docs' count
is the counts' sum, and the kept list must be part of the full one.
It reads only the exclude_re array's body, needs no tomllib
(Python 3.9), and names colour codes in the list as the problem;
- scripts/mutants.sh makes both lists (--colors never: CI sets
CARGO_TERM_COLOR=always) and runs it first, except when the caller
passes --config or --no-config (as whole arguments). CI's `mutants` job runs that script;
ci-changes.sh now also selects the job for the new tool;
- scripts/check.sh runs its --self-test on planted configs;
- tools/claims.py ties the docs' "22 equivalents" (README, 09 §6 and the
summary row) to the sum of the counts, with the same parser. It finds
that parser from its own directory, so `python3 -I` runs it too, and
check.sh runs its self-test that way. 00's
"18 equivalents" (26 Sep log entry) and run 9's "exactly the 22
equivalents below" (6 Oct) are kept as dated records.
Docs: 09 §4 finding 105, §6 run 10 and the "In CI" note, §8's mutants
row; README and 09's summary row 669 → 797 mutants (run 10); README and
12's post-submission range 96–105.
Evidence (first on 1f2ad48 + the parent, logs in the autopilot's
tasks/mutants-exclusions-r2/; re-run on 58697c6 + the parent,
tasks/mutants-exclusions-r3/):
- `cargo mutants --list`: 797 with --no-config, 778 with the old config
(19 excluded), 775 with this one. The 22 excluded are §6's list, the
murmur3 ones at 327/330/333.
- scripts/mutants.sh --list: exit 0, "22 of 797 mutants excluded"; with
CARGO_TERM_COLOR=always also exit 0. Old config: exit 1, every pattern
"no '# N' count". Planted on the real lists: the old lines with a
count (matches 0, says 3) and the body XOR added (matches 4, says 3)
both exit 1; in both the set comparison alone passes. /usr/bin/python3
(3.9.6): self-test OK and the real check passes.
- The self-test pins each problem by its message, not a count. Mutants
of the tool (tool-mutants.log), each red: M1 no per-pattern check,
M2 no set comparison, M3 no colour check, M4 no empty-array check,
M5 a missing count accepted, M6 a missing count read as 0, M7 the
parser reading past the array. M3 and M5 survived a first version
that checked only the number of problems.
After A's review (16 Oct): the set comparison was pinned only by its
size, so `len(excluded) != len(matched)` passed the self-test. A
case with as many excluded as matched but not the same ones now
fails it (15 cases; red with that mutant, green without), and the
message says how many mutants are in one set only.
From the code review after the rebase: colour codes only in the kept
list were reported as a kept list made with other options; the
colour check now reads both lists (16 cases; dropping the kept half
fails the new case).
On 58697c6, with the overlap and kept-list checks: M8 no overlap
check and M9 no kept-list check, red too, as M1–M7 still are. A
duplicate of the can_eject pattern planted in the real config:
scripts/mutants.sh --list exits 1, "the counts add up to 23, the
patterns match 22 mutants" (planted-overlap.log). Without the
sys.path line, `python3 -I tools/claims.py` fails with
ModuleNotFoundError; with it, both claims runs pass.
- claims.py (claims-mutants.log): one count 1 → 2 fails four statements
(README 188; 09 33, 333, 335; kept-run9.log), not run 9's 331, which
failed too before it was kept; a dropped count fails with the
tool named; deleting any of the three rules fails a self-test case;
deleting the KEPT entry fails 00:103.
- The 22 excluded mutants run alone (--no-config, --re per pattern,
2 min): 22 missed, exactly the excluded list; --re also picked
ema.rs:70 (a field deletion, no pattern matches it), caught.
- Run 10 (f1f05ba, old config, -j4, 46 min; tasks/mutants-run10/):
778 tested, 712 caught, 5 timeouts, 58 unviable, 3 missed, exactly
the murmur3 tail. The core's source is unchanged since (`git diff
--stat f1f05ba HEAD -- crates/bulkhead-core/src` is empty; only
tests/units.rs gained finding 102's tests).
- No program or core source changed.
Rebased on 16 Oct onto cede705 (findings 102 and 100, the attest
sweep and AUD-2). AUD-2 changed the core's source (governance.rs), so
the line above saying it is unchanged since run 10 held on 58697c6, not
here. On cede705: scripts/mutants.sh's list check passes, "22 of 795
mutants excluded, each pattern matching its count"; the murmur3 lines
are still 327/330/333. The 797 in README and 09 is run 10's figure:
README now dates it ("in the last full run, 15 Oct") and finding
105's row says the core has changed since (finding 107).
The setup line in scripts/mutants.sh now pins cargo-mutants 27.1.0,
the version ci.yml installs: each count is the number of mutants that
version lists, so another release can break the check (code review,
16 Oct; `git grep 'install.*cargo-mutants'` finds no other unpinned
install).
The check's report goes to stderr in scripts/mutants.sh, so
`scripts/mutants.sh --list` prints only cargo-mutants' names: before,
its first stdout line was the report; after, stdout is 773 lines
(795 - 22) and stderr the one report line. ci.yml's `mutants` comment
says the job checks the exclusions first.
Row 105 and 09 §6 gave the pin's reason as the body's XORs. The
finalizer's are at column 7 (`grep -n '\^=' floor.rs`, 337-342); the
non-equivalent XOR at column 11 is `h ^= k`, in the body (320) and after
the tail (335). The toml comment, the row and §6 now say that. 09 §8's
`check` row names the two tool self-tests check.sh runs.
README no longer calls run 10's 797 "injected bugs": cargo-mutants
listed 797 but tested 778, 19 being excluded (09 §6, run 10's row).
Sibling search: the other 14 patterns use `\d+` for the line, and each
matches the count now written after it. The murmur3 one is the only
line-pinned pattern.
Not done, by choice:
- Catching an edit that keeps a pattern's number but changes its
mutants (an excluded line itself edited, or a murmur3 body XOR moved
onto a pinned line). The toml and 09 §6 say so: such an edit needs the
mutant judged again by hand. Failing on any diff that touches an
excluded line would catch the first; it is a separate change.
- Moving murmur3's tail into its own function, so its pattern needn't
name lines. It shifts core lines, so it changes the program hashes.
Until then a line move above it fails the check, with a message
saying what to fix.
- A weekly full mutants run in CI. It took 46 min here at -j4, likely
past the job's 90 min on two CI cores: a minutes call for the owner.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that a ticket stays in its own protocol and vault
Two Vault checks that keep a ticket to its own protocol and vault had no
test. Deleting either one left every program test green
(tasks/error-codes/mutants.log, base cede705):
- brakes.rs:79, extend_ticket: tk.protocol != protocol. Without it, a live
guardian of any other protocol extends this protocol's tickets. Anyone
can register a protocol and name its guardians, so that guardian could
hold back every queued payout up to the total extension limit. The
sweep did not catch it: it has no second protocol's guardian to swap in.
- custody.rs:501, settle_ticket: tk.vault != vault. A USDC ticket settled
through the SOL vault (its tokens, its mint) got past the Vault's checks.
New test another_protocols_guardian_cannot_extend_a_ticket: a second
Market's live guardian extends this market's ticket -> WRONG_VAULT; the
same instruction from this market's guardian succeeds. The settle test
now settles the USDC ticket through the SOL vault and expects
WRONG_VAULT.
Evidence (tasks/error-codes/mutants.log, STATE=c1; full program suite
per mutant):
W02 brakes.rs:79 deleted -> KILLED by
another_protocols_guardian_cannot_extend_a_ticket ("expected
failure, got success": the stranger's extension went through)
W06 custody.rs:501 deleted -> KILLED by
a_ticket_settles_after_its_eta_to_its_stored_recipient_only (8049
InsufficientBalance from the empty SOL vault, not 8010)
Both were SURVIVED at base. No program source change: hashes
8ca8a608/543be60b/7086148b, as at cede705.
Sibling search: the other 12 WrongVault sites were deleted one at a time
in the same run. Eight are KILLED by the sweep (brakes.rs:39, lib.rs:320,
custody.rs:160, genesis.rs:491, views.rs:27, lifeboat.rs:151,
governance.rs:169/259). lib.rs:324 (vault PDA address), genesis.rs:523,
lifeboat.rs:71 and :247 survive; later commits on this branch deal with
them.
Counts: Vault 104 -> 105, programs 154 -> 155 (claims.py OK).
The test also reads the ticket's ETA: unchanged after the
refused extension, 3,600 s later after its own guardian's (code-review
fix: success alone did not show the extension happened).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that the seal names every vault and happens once
Two seal checks had no test; deleting either left every program test
green (tasks/error-codes/mutants.log, base cede705):
- genesis.rs:486, execute_seal: the pairs must number 3 * vault_count.
Without it, the seal checks only the vaults it is given. A vault left
out skips its claims-vs-balance check, so an uncovered import seals.
docs/05's seal row says every vault is passed exactly once.
the_seal_checks_each_vault_once covered a vault named twice in place of
another, not one left out.
- genesis.rs:431, propose_seal: refused once sealed (AlreadySealed).
Without it, a sealed protocol's admin re-proposes, which rewrites
seal_executable_s/expires_s and clears seal_vetoed on a sealed protocol.
the_seal_checks_each_vault_once now also leaves the uncovered USDC vault
out (expects VAULT_COUNT_MISMATCH) and, after the seal, proposes it again
(expects ALREADY_SEALED). No new test, so no count change.
Evidence (tasks/error-codes/mutants.log, STATE=c2, full program suite):
E05 genesis.rs:486 deleted -> KILLED by the_seal_checks_each_vault_once
("expected failure, got success": the seal went through without
the USDC vault and its 1,000 USDC uncovered import)
E04 genesis.rs:431 deleted -> KILLED by the_seal_checks_each_vault_once
("expected failure, got success")
Both were SURVIVED at base. Vault hash 7086148b unchanged (tests only).
Sibling search: the seal's other checks. :491 vault_tokens (KILLED by the
sweep), :519 lifeboat owner and :523 lifeboat mint/owner (SURVIVED; the
lifeboat is a PDA checked by expect_pda just above, which only this
program can create, as its own token account: see this branch's note in
docs/09).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that invalid limits and asset settings are refused at every entry
Three bound checks had no test; deleting any one left every program test
green (tasks/error-codes/mutants.log, base cede705):
- genesis.rs:171, create_vault: config_is_valid (bps <= 100%, windows
> 0). Without it, a vault is created with a 100.01%-an-hour burst.
- governance.rs:40, parse of a SetLimits change (at proposal and again
at execution, :254 and :478): the same rule for a proposed change.
- governance.rs:47, parse of an UpsertAsset change: valid_asset (haircut
<= 100%, liability >= 100%, the oracle bounds of finding 88). Only the
genesis upsert's copy (genesis.rs:296) was tested, by
oracle_settings_of_a_priced_asset_are_bounded.
New test invalid_limits_and_assets_are_refused_wherever_they_enter:
before the seal, create_vault with short_bps 10,001 -> INVALID_LIMITS;
after it, a SetLimits proposal with long_window_s 0 -> INVALID_LIMITS,
and an UpsertAsset proposal with a 10,001 bps haircut -> INVALID_ASSET.
Evidence (tasks/error-codes/mutants.log, STATE=c3, full program suite):
E06 genesis.rs:171 deleted -> KILLED ("expected failure, got success")
E07 governance.rs:40 deleted -> KILLED ("expected failure, got success")
E08 governance.rs:47 deleted -> KILLED ("expected failure, got success")
each by the new test only. All three were SURVIVED at base. Vault hash
7086148b unchanged (tests only).
Sibling search: every config_is_valid / valid_asset call in programs/
vault/src: genesis.rs:170 (create_vault), :295 (upsert_asset, KILLED by
the oracle test), governance.rs:39 and :46. All now killed.
Counts: Vault 105 -> 106, programs 155 -> 156 (claims.py OK).
The execution copy (:478) is not reached by any transaction: it
parses the bytes the proposal already checked, and the bounds read
nothing else. The test comment says so (code-review fix: it had
claimed execution was covered). oracle_settings_of_a_priced_asset_are_
bounded now asserts INVALID_ASSET for each of its four cases, not any
error (all four return it; E09 stays KILLED).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that an import into a ninth mint is refused
genesis.rs:410, import_backing: a backing ledger holds MAX_SLOTS (8)
mints, and an import into a ninth is refused with NoSlot. Deleting the
check left every program test green (tasks/error-codes/mutants.log, base
cede705). From the code, without it the import goes through: the vault's
imported total grows (:414), and the seal must cover that, but the
owner's ledger is saved unchanged, so the owner has no backed claim for
it.
New test an_import_into_a_ninth_mint_is_refused: nine vaults before the
seal; one owner gets an import into eight (ledger_of shows 8 positions),
then the ninth -> NO_SLOT.
Evidence (tasks/error-codes/mutants.log, STATE=c4, full program suite):
E14 genesis.rs:410 deleted -> KILLED by an_import_into_a_ninth_mint_is_
refused ("expected failure, got success"); SURVIVED at base.
Vault hash 7086148b unchanged (tests only).
Sibling search: the only other ledger credit is deposit's
(custody.rs:128). There a full ledger is by design: the deposit is taken
unbacked (Ledger::credit's doc, ledger.rs:187-189), and credit's result is
the backed flag, not an error.
Counts: Vault 106 -> 107, programs 156 -> 157 (claims.py OK).
import_ix now calls import_into_ix, which takes the vault, so the
new test shares the one import layout (code-review fix).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin six Vault refusals no test reached
Six reachable checks had no test that failed when the check was deleted
(tasks/error-codes/mutants.log, base cede705). Each now has an assertion
on its code, and each was seen to fail with the check deleted:
E02 genesis.rs:185 create_vault of an existing vault after the seal:
ALREADY_INITIALIZED (with the check gone: 8043 GenesisOnly, so the
"says so first" order in the comment above it was untested)
E12 governance.rs:426 veto of an executed change: CHANGE_EXECUTED
(gone: the veto succeeds and marks an executed change vetoed)
E13 governance.rs:517 cancel of an executed change: CHANGE_EXECUTED
(gone: the cancel succeeds)
E16 lifeboat.rs:256 execute_exit with no exit proposed:
NOTHING_TO_EXIT (gone: 8033 ChangeNotExecutable)
E17 custody.rs:84 a zero deposit: ZERO_AMOUNT (gone: it succeeds)
E18 custody.rs:272 a zero release: ZERO_AMOUNT (gone: it succeeds; a
zero release has no unbacked part, and the limiter, whose own zero
check custody.rs:242 maps, is called only when unbacked > 0, :361)
Where: E02 in after_the_seal_a_new_vault_starts_with_its_unbacked_lane_
closed, E12/E13 in an_unvetoed_loosening_executes_after_the_delay, E16 in
the_lifeboat_stays_shut_until_the_seal (after the move in, before an
exit is proposed), E17/E18 in the new a_zero_deposit_or_release_is_
refused.
Evidence (tasks/error-codes/mutants.log, STATE=c5, full program suite):
all six KILLED by the tests named above; all six SURVIVED at base. The
vault suite is green (108 passed). Vault hash 7086148b unchanged.
Counts: Vault 107 -> 108, programs 157 -> 158 (claims.py OK).
Helpers (code-review fixes): veto_change_ix beside cancel_change_ix;
create_vault_for_ix, create_vault_ix with a given mint and token
program, so E02's case names USDC's own accounts and creates no mint.
E02's comment now repeats the program's ("whatever the seal's state"):
create_vault checks the limits first (genesis.rs:170).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin four Vault checks another layer backstops
Four checks survived deletion (tasks/error-codes/mutants.log, base
cede705) because something later still refused the case, with its own
error, so no test could tell. Each now has a test that sees the Vault's
own code, and each was seen to fail with the check deleted:
E01 lib.rs:230, create_pda's guard: a second registration of an
existing protocol (register_protocol has no check of its own) ->
ALREADY_INITIALIZED. Gone: the system program refuses it, with no
custom code (custom_code 0).
W10 lifeboat.rs:71, move_in naming another mint or token program ->
WRONG_VAULT. Gone: the token program refuses (code 3, MintMismatch).
W12 lifeboat.rs:247, execute_exit naming another mint or token
program -> WRONG_VAULT. Gone: the same token-program error.
T01 token.rs:37, account_info on an account no token program owns: a
copy of a real USDC account's bytes under the system program. Gone:
the quote SUCCEEDS (it promises a payout), and the release fails
only in the runtime at the transfer ("instruction modified data of
an account it does not own"). The quote is asserted first so the
mutant is seen at the quote (STATE=c6b).
New tests: a_protocol_registers_once, a_lifeboat_move_names_the_vaults_
own_mint (also completes a move in and an exit, as a control: the
destination receives 1,000 USDC), a_payout_goes_only_to_an_account_a_
token_program_owns.
Evidence (tasks/error-codes/mutants.log, STATE=c6 and c6b, full program
suite): E01, W10, W12, T01 KILLED by the tests named above; all four
SURVIVED at base. Full program suite green: Floor 50, Vault 111. Vault
hash 7086148b unchanged (tests only).
Counts: Vault 108 -> 111, programs 158 -> 161 (claims.py OK).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test the bounds on ticket extensions and a closed lane's code
B's review of vault-error-tests-r3 (R1) found two Vault codes no test
asserted by value that the earlier commits missed: ExtensionTooLong
and LaneClosed. Deleting the 7-day total extension cap (brakes.rs:86,
B24) left every program test green, so a live guardian, or a stolen
guardian key, could hold a queued payout back without end, 24h at a
time, against docs/05 row 11. The wide differential models the total
(differential_wide.rs:406), but its default run never extends one
ticket that far. Mapping LaneClosed to QueueFull (custody.rs:240, B23)
also survived.
The code review of the fix found a worse gap at the same check: no
test sends a negative extension (the differential draws secs from 0,
1, 3600, 86400, 86401). With `secs <= 0` weakened to `secs == 0` (A3,
brakes.rs:84) every program test passed, and one guardian key could
pull a queued payout's ETA into the past, skipping the queue delay,
and refund its own 7-day budget.
- a_ticket_is_extended_later_by_at_most_a_day_a_call_and_seven_in_total:
-1h, 0 and 24h + 1 s are each refused with EXTENSION_TOO_LONG, ETA
unchanged; six 24h extensions and one of 23h pass; 2h more is
refused, ETA unchanged; the last hour is granted; then 1 s more is
refused. The split is unaligned so that a total that ignores this
call's `secs` fails too.
- a_closed_lane_refuses_an_unbacked_payout: a vault with short_bps 0,
and one with long_bps 0, each refuse an unbacked admin payout with
LANE_CLOSED; the same payout from a vault with both buckets open is
taken (the control, so a capacity base of 0 cannot pass it).
Evidence (tasks/vault-error-tests-r4/, B's runner copied: each mutant
applied alone, programs rebuilt, whole program suite run):
- without these tests (551e519; run-without.out, run-base2.out,
run-base3.out): B23, B24, A1 (`tk.extended_s >= MAX`, this call's
secs ignored), A2 (`>=` for `>`) and A3 SURVIVED. A4 (the 24h per
call deleted), A5 and A6 (either half of limiter.rs:172's closed-lane
test deleted) were KILLED by the differential only, not by value;
- with them (run-final.out, at the branch tip): B23 KILLED by
a_closed_lane_refuses_an_unbacked_payout; B24 A1 A2 A3 KILLED by the
extension test only; A4 by it and the differential; A5 A6 by the
closed-lane test and the differential.
No program change: the runner's closing build gives 8ca8a608 /
543be60b / 7086148b, as at cede705.
Counts: Vault 111 -> 113, programs 161 -> 163 (claims.py OK).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin each half of create_pda's existing-account guard on its own
B's review of vault-error-tests-r3 (R2): each half of lib.rs:229
(`!is_data_empty() || !owned_by(SYSTEM)`) survived deleted on its own
(B20, B21), since an existing protocol account fails both, so
a_protocol_registers_once pinned them only jointly.
The test now also plants, at a fresh protocol address, an account that
fails just one half: one with data owned by the system program, one
with no data owned by another program (a fresh address). Each
registration is refused with ALREADY_INITIALIZED. The comment says no transaction is
known to make either account: data or a new owner at a PDA needs its
signature, which only the Vault gives, and create_pda gives both in one
instruction.
Evidence (tasks/vault-error-tests-r4/): at 551e519 B20 and B21
SURVIVED (run-base2.out); at the branch tip both are KILLED by
a_protocol_registers_once only (run-final.out).
No count change (claims.py OK: vault_programs 113, programs 163).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Record the Vault refusal deletions and what each became
docs/09 §3 gains a row for the deletion run this branch answers
(tasks/error-codes/mutants.log): 38 refusals, across 14 of the 18 Vault
codes no test asserted by value at cede705, each deleted on its own
with the whole program suite run against it, on the AUD-2 build.
The 18 (each VaultError variant with no `verr::` use in programs/tests
at cede705): AlreadyInitialized NotRegistrar WrongVault
WrongTokenProgram AlreadySealed VaultCountMismatch InvalidLimits
InvalidAsset LaneClosed TicketNotPending ExtensionTooLong
ChangeExecuted LedgerUnsound LedgerCorrupt NoSlot NothingToExit
ZeroAmount Overflow. The run missed LaneClosed and ExtensionTooLong;
B's review deleted one check of each (B23 custody.rs:240, B24
brakes.rs:86), and both survived until 1797ae5. That commit
also pins a negative extension (A3, which survived every test before
it), the 24h per call, and a lane closed by either bucket.
At base: 13 KILLED, 25 SURVIVED. The commits before this one add tests
that kill 18 of the 25 and both of B's, each asserting the Vault's own
code (W02 W06, E04 E05, E06 E07 E08, E14, E02 E12 E13 E16 E17 E18, E01
W10 W12 T01; B23 B24). The row names the 7 that still survive and why:
- six guards no transaction is known to reach: lib.rs:324 (vault PDA
address; create_vault creates a vault only at its PDA), genesis.rs:519
and :523 (the lifeboat, checked by expect_pda first, created only by
lifeboat_move, move_in requires the seal, lifeboat.rs:53, and the
seal happens once, so no lifeboat holds data at it), brakes.rs:50 and
custody.rs:541 (a pending-ticket count below zero), token.rs:235
(GetAccountDataSize's return data from an already-checked program);
- custody.rs:399, stated as not shown unreachable: properties.rs:588-592
debits backed_amount and one more on random ledgers, and only
crates/bulkhead-core/tests/units.rs:375 debits less, on a one-mint
ledger with no liability. properties.rs:606 asserts backed(a) ==
min(a, backed(MAX)) for requests below 2^52 (:613), so any smaller
amount is backed too, and debit_backed(b) refuses only when
backed_amount(b) != b (ledger.rs:289). The known exception is a
collateral value that overflows u128 (ledger.rs:262): the valid
amounts need not form an interval there, so the consent or freeze
cap (custody.rs:337, :347) can lower the amount to an invalid one and
:399 refuses the release (fail closed). The search's own result is
always valid. Not verified by a tool beyond that property test. (A full ledger cannot reach :399 through
set's NoSlot: backed_amount's valid() refuses a new liability slot
on a full ledger, ledger.rs:237.)
Not deleted, as the row says: the two core-error mappings to
ChangeExecuted and ZeroAmount, token.rs:233, and the codes Overflow and
LedgerCorrupt (checked arithmetic, decoding).
After this branch five codes are still asserted by no test (same grep
at this commit): NotRegistrar (E03, genesis.rs:108, was KILLED at base
by another test), TicketNotPending, LedgerUnsound, LedgerCorrupt,
Overflow.
Docs only: no count change (claims.py OK: vault_programs 113, programs
163).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Run the exclusion check's main in its self-test
A's review of finding 105 (REVIEWS/auto-B-mutants-exclusions-r7.md,
finding 1, Low): CI's `mutants` job acts on main's exit status, but the
self-test called problems() only, and check.sh never runs the tool on
real lists. With main returning 0 on a mismatch (mutant A10), every
in-tree check passed, and CI would have printed the problems and gone
green. This was not a defect at fb771c6, since the job fails today. The
gap was that a later regression would pass.
Fix: main takes its arguments and, for the self-test, a config path
(default: the repo's .cargo/mutants.toml, as before), and its lines name
the config it read. The self-test runs main in-process on a planted
config and checks the exit status and every line on each stream:
- the right lists: exit 0, "1 of 3 mutants excluded" on stdout, nothing
on stderr;
- the lists the other way round: exit 1, nothing on stdout, and exactly
the count and "full list lacks" problems, so KEPT is read as KEPT;
- coloured lists: exit 1 with exactly one problem, so a main that fails
only on several problems is caught;
- one argument and three: exit 2 and the two usage lines.
One subprocess (python3 -I, 60 s timeout, in the temporary directory so
the config must be found from the tool's own path) runs the command line
with two lists and the repo's config, as scripts/mutants.sh does: on a
list no pattern can match, it exits 1 with an empty stdout and problem
lines that each name .cargo/mutants.toml. Which problems those are
depends on the config, so only their form is checked. Self-test: 16
cases -> 22. No doc states the number.
Evidence (tasks/exclusions-main-exit/):
- old-A10.log: the fb771c6 self-test with A10 applied prints "self-test
OK: 16 cases" and exits 0.
- mutants-r6.log (mutants.py applies each mutant to a scratch copy of
the tool, which reads the real config through a symlink): unmutated,
"self-test OK: 22 cases", exit 0. All 18 are KILLED (exit 1): A10
(return 0), A10b (fail only on more than one problem), A0 (always 1),
U1 (usage returns 0), U2/U3 (argument check `< 2`, `< 1`), K1 (KEPT
ignored), K2 (ALL read twice), C1 (planted config ignored), C2 (no
default config), R1 (the default config read from the cwd), E1/E2
(sys.argv sliced as [:] or [1:2]), P1 (problems not printed), S1
(problems on stdout), L1 (the planted config not named), N1 (only the
first problem printed), G1 (the success line printed on failure too).
Code-review passes at high shaped this. The first version ran three
subprocesses against the real config, and A0, K1 and U3 survived it.
The second left E2, C2 and G1 alive. Declined from the third: catching
exceptions in the self-test (a traceback already exits 1) and moving
the config hook out of main. The fourth showed that requiring one
"matches 0 mutants" line per live pattern tied the self-test to the
config's contents. The subprocess now checks the form of the lines
only, and the 17 stay killed. The fifth showed that R1 survived while
the child ran in the repo root (it does in check.sh); now it runs in
the temporary directory. Its other points repeat declined ones.
- real-lists.log: on the real lists, made with scripts/mutants.sh's
commands: ".cargo/mutants.toml: 22 of 795 mutants excluded, each
pattern matching its count", exit 0.
- python3 -I tools/mutant_exclusions.py --self-test: 22 cases OK.
- scripts/check.sh: ALL CHECKS PASSED (check-final.log);
bin/check-each-commit.sh green (each-commit.log).
Sibling search: the other tools check.sh runs (tools/claims.py,
tools/floor_check.py) also gate CI through main's exit status. claims.py
has the same gap: check.sh runs it on the real tree, where it finds no
problems, and its self-test calls self_test(), not the path in main that
returns 1. With that return changed to 0, `--self-test` and the real run
both still exit 0 (claims-C1.log). Not changed here: it is its own
commit (BACKLOG-B). floor_check.py runs in CI only as --self-test, whose
result main returns as is. Its RPC mode is an operator's tool that no CI
job runs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Record finding 106: Vault liveness and exit rules had no failing test
The tests landed in 944f5e8 (lifeboat-exit-tests-r3, 15 Oct). Their
09 §4 row waited for 104 and 105 so the list stays numbered without
gaps; 105 is now on integration (fb771c6), so the row lands here.
Docs only: 09 §4 row 106; README and 12's count and post-submission
range (96–106, 106 fixed). Two code-review passes changed the row as
written on 15 Oct: a sentence for today's re-run, the nine
single-guardian sites named, G5 quoted verbatim, and the exit veto
attributed to T18 (G6 has no veto), as is the veto test's doc comment
(vault.rs:3996, comment only).
Evidence, re-run today on bf8ba41 (the row's claims are from 15 Oct,
and AUD-2 changed the Vault since): B's re-review list of 33 line-
neutral mutants (the ten rules, the nine single-guardian sites L1-L9,
the boundaries, the six quorums, R1-R6), each applied, programs
rebuilt, all program tests run:
33 KILLED, 0 survived, 0 not applied, 0 build failures. Vault
hashes after the run match the tip (Vault 7086148b, Floor 8ca8a608,
Harbor 543be60b).
Log: autopilot tasks/finding-106-row/mutants.log.
- python3 -I tools/claims.py: OK, findings 106.
- git grep for "96–105", "105 findings": no other stale range.
- scripts/mutants.sh still mutates the core only (SCOPE, line 11), so
the row's last sentence holds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Cite PR #100's tree, not the unpublished bf8ba41, in finding 106
Finding 106's row in docs/09 said its 33 mutants were re-run on 16 Oct
at bf8ba41. That commit is not an ancestor of this branch or of the
stack above it and is on no remote branch, so the citation would point
at nothing once the stack lands. The owner's review of #99-#109 found it.
bf8ba41 and 86b777b (the head of PR #100, vault-error-tests-d2, the D2
replay of vault-error-tests-r4) have the same tree, 36187b49
(`git rev-parse bf8ba41^{tree} 86b777b^{tree}` prints it twice). So
the run was a run on 86b777b's tree, and the row now says "on tree
36187b49, the tree of 86b777b (PR #100's head)". Why not just "at
86b777b": it was committed on 17 Oct, after the run; and main
squash-merges (1045cd9 has one parent), so after PR #100 lands 86b777b
is reachable only as GitHub's refs/pull/100/head. The tree id stays
checkable from main: squashed bottom first, main's commit for PR #100
has that tree. "PR" keeps "#100" from reading as finding 100, the
row above. These points came from two code-review passes.
Sibling search: every 7-12 hex token in tracked .md/.py/.sh/.mjs/.rs/
.toml files at autopilot/integration 3e9f58d that names a local commit
(51 tokens) was checked with git merge-base --is-ancestor against
integration. bf8ba41 was the only one that is not an ancestor.
Docs only. tools/claims.py: claims OK (26 files).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Warn about a Floor guardian set short of the margin (finding 107)
The guardian checks the Vault protocol's guardian set against finding
107's margin (main.mjs checkGuardianSet), but never a Floor config's.
A Floor set registered before the rule, say 2-of-3, keeps working and
keeps AUD-2's exposure: one lost key and one rogue guardian lock its
governance for good. The guardian said nothing, and docs/07 §6 said so
("not yet a Floor set").
The rule moves to policy.mjs as guardianSetWarning, a pure function:
the same text as before, a warn line that alerts on mainnet only. In
main.mjs, checkGuardianSet now remembers the set shape per set
("protocol", or the protected program), and checkFloor calls it for
each watched Floor config it reads. floorAlerts is unchanged. The
margin is not a readiness code there: "floor not ready" alerts on
every cluster, and only the owner's new set clears a margin problem, so
"floor ready" would never be logged again.
Evidence (tasks/floor-guardian-margin/r2/):
- node --test cloud/test/*.test.mjs: 180/180 pass (one new test).
- Mutants of guardianSetWarning, each run on policy.test.mjs and
reverted; each fails the new test: N1 never warn, N2 threshold fixed
at 2, N3 notify always, N4 notify never, N5 GUARDIAN_MARGIN = 1.
- tools/claims.py: OK, client 180 (docs/09:40 updated).
- Not tested: the main.mjs wiring (the per-set Map and the checkFloor
call). No test runs checkFloor; deleting the call leaves every test
green.
Sibling search: guardianMarginProblem's callers are vault-check.mjs (an
info line) and guardianSetWarning. The status snapshot
(snapshot.mjs:33) prints live guardians and the quorum, not the margin;
left as it is.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Judge each guardian set's margin on its own, and say when it is fixed
The parent's per-set memory (which set shapes were already reported)
lived in main.mjs, so no test could fail on it: dropping the
once-per-shape check, or keying every set the same, left every test
green. The warning did not say which set it was about (only a program
tag on Floor lines), and it gave the same text for a set ejected down
to its threshold, or registered n-of-n before finding 96. There one
rogue guardian alone locks governance; a lost key alone does not, since
dead guardians cannot veto the owner's new set (docs/16 §4.4), but it
leaves no quorum until that set executes. Nothing was said when the owner's
new set fixed the problem. And a delivered alert did not say which Floor
program it was about: the program went only into the line's `program`
field, which alertFor uses in its repeat key but does not show.
Moving the program name into the text is a trap: alertFor keys repeats
on the first 60 characters of the text, and "the Floor guardians of
<44-character id>" pushes the shape past them. A 2-of-3 set ejected to
2-of-2 within ten minutes would then give the same key, and the worse
warning would not alert. So the program goes after the shape: "the
Floor guardians are 2-of-3 for <program>: ...". The `program` field
stays for the repeat key.
guardianSetWarning becomes guardianSetCheck(shapes, which, set,
mainnet) in policy.mjs. It keeps each set's last shape and whether it
was short in the Map, keyed by "protocol" or the protected program, and
returns the line to log or null. The warning names the set and has its
own text for n-of-n. When a set that was short gets the margin, it
returns one line saying so, which alerts on mainnet like the warning,
so whoever was paged hears that it is fixed. main.mjs only logs what it
returns. docs/07 §6 says so. The advice also had a hole: a set
registered as 7-of-8 was told to "run at least 7-of-9", and no set can
have 9 keys (core MAX_GUARDIANS is 8). When threshold + 2 is over 8,
the advice is now to lower the threshold to at most 6-of-8. That needs
MAX_GUARDIANS in JS, so lib/solana.mjs gets it next to
GUARDIAN_MARGIN; the next commit ties both to the core.
Evidence (tasks/floor-guardian-margin/):
- node --test cloud/test/*.test.mjs: 180/180 pass (the test is
extended, not added).
- 19 mutants, each run on policy.test.mjs and reverted (r6/mutants.py,
r6/mutants.log, r6/N5-margin1.log). Each fails the extended test, and
only it: never warn, threshold fixed at 2, notify always, notify
never, GUARDIAN_MARGIN = 1, no once-per-shape, one Map key for every
set, the n-of-n text for threshold + 1, no fixed line, a fixed line
for a set that was never short, the fixed line never alerting, no
program tag, every set named as the protocol's, the short flag not
stored, the program before the shape, the program left out of the
text, `need < 8` for `need <= 8` (6-of-7), the 8-key limit ignored
(7-of-8), 7-of-8 as the lowered advice.
- The mainnet alerts are checked on their own in a scratch copy of the
test without the earlier text assertions (r5/C1-C3-alone.log): with
the program left out of the text, the alert's detail check fails;
with a long name pushing the shape past character 60, "2-of-2 alerts"
fails.
- A's review of the previous version (branch
auto/B/floor-guardian-margin-r7, kept) found two survivors there: X3,
the shape keyed on the count only, and X5, the advice
threshold-of-(n+1). It also found that the n-of-n text said a lost key
alone locks governance. The test now has a threshold-only fix (7-of-8
-> 6-of-8, the advice's own step) and pins the n-of-n text and advice.
r9/mutants.log: X3, X5, the r7 n-of-n text and the text without its
lost-key clause are all KILLED. r9/mutants-r7.log: X3 and X5 SURVIVED
on the r7 code and test. The 18 mutants of r6/mutants.py, re-run on
this code: all KILLED (r9/r6-mutants-rerun.log). Two code-review
passes on the fix (kept as -r8 and -r9) added the lost-key clause: a
lost key alone leaves no quorum (it never approves, and core `quorum`
needs threshold approvals) until the owner's new set executes. A third
pass (-r10) fixed the comments' wording only.
- tools/claims.py: OK, client 180.
- Still not tested: the checkFloor call in main.mjs (no test runs
checkFloor).
Not changed (code review): the "fixed" line needs the short shape to
have been seen since the guardian started (the Map is in memory, as the
old per-process shape was; docs/07 says so). A protocol that abdicated
SetGuardians can't take a new set, so the advice to run 2-of-4 can't be
followed there; REPORT lists that case for the owner. The fault text
assumes a margin of 2, the owner's decision (16 §4.4).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Tie the JS guardian margin and set size limit to the core constants
cloud/lib/solana.mjs keeps its own GUARDIAN_MARGIN = 2, which the
guardian and the deploy tooling use to judge a guardian set, and now
MAX_GUARDIANS = 8, which the guardian's advice uses. Their comments say
they are core governance.rs's constants, but nothing checked that. Had
the programs' margin moved to 3, every JS test would have stayed green
while the guardian passed sets the programs refuse.
deploy.test.mjs now reads governance.rs and requires exactly one
`pub const GUARDIAN_MARGIN: u8 = N;` and one `pub const MAX_GUARDIANS:
usize = N;`, each N equal to the JS constant. It reads the source text,
like the build-script test above it. If a declaration changes form, the
test fails rather than passing.
Evidence (tasks/floor-guardian-margin/r6/tie-mutants.log):
- node --test cloud/test/*.test.mjs: 181/181 pass (one new test).
- Mutants of governance.rs, each reverted with its mtime kept (else the
limiter differential asks for a worker rebuild): GUARDIAN_MARGIN = 3,
MAX_GUARDIANS = 9, and a trailing comment on MAX_GUARDIANS. Each fails
the new test, naming the constant. deploy.test.mjs is green again.
- tools/claims.py: OK, client 181 (docs/09:40 updated).
Sibling search: the other JS copies of core rules (the limiter port,
the account layouts) are checked by the differential and by decoding
real accounts. MIN_GUARDIAN_THRESHOLD has no JS copy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Reach the 7-day extension total in the wide differential
differential_wide.rs drew extension seconds only from {0, 1, 3600,
86400, 86401} on a random ticket and lumped every refusal into one
counter. A ticket never got near the 7-day total, no negative value was
sent, and a refusal was not checked by its code. So the default run
could not tell the total bound, a negative extension or the error code
from a bug: only the unit tests at vault.rs:3382-3401 pinned them.
Now one extension draw in three pushes the chosen ticket by up to 24h
a call to exactly the 604,800 s total, then asks for 1 s more, so the
edge itself is hit on every push, not only when earlier draws line up.
The other draws are half legal sizes (1, 3600, 86400) and half out of
range for one call (i64::MIN, -86400, -1, 0, 86401, i64::MAX). Each
refusal must fail with EXTENSION_TOO_LONG. The counters are split into
extend (a random legal size), extend_push, extend_refused_call and
extend_refused_total, all required >= 10, so pushes cannot stand in for
the random sizes.
The model reads both bounds from bulkhead-core (consts.rs:37-38), as
the module says it is built, rather than restating 86,400 and 604,800.
docs/09 §7's row says so, and the 16 Oct "Vault refusals deleted" row,
which says the default run never extended a ticket that far, now says
it does since this change (its differential_wide.rs:406 cite had drifted).
Default run (24 seeds x 250 steps): extend 16, extend_push 126,
extend_refused_call 16, extend_refused_total 30 (was extend 40,
extend_refused 27); every other required counter still >= 10
(loosen_executed 12, as before).
Evidence: mutants of brakes.rs extend_ticket, each built and run against
the old and the new differential_wide only:
T1 drop the total check old SURVIVED new KILLED
T2 total > to >= old SURVIVED new KILLED
T3 drop secs <= 0 old KILLED new KILLED
T4 secs <= 0 to < 0 old KILLED new KILLED
T5 total ignores this call's secs old SURVIVED new KILLED
T6 extended_s += secs / 2 old SURVIVED new KILLED
T7 refusal returns WrongTicket old SURVIVED new KILLED
T8 secs <= 0 to == 0 (negatives) old SURVIVED new KILLED
T9 total > to > total + 1 old SURVIVED new KILLED
(T9: a draft that pushed by 24h until refused killed it only through
chance 1 s and 3600 s draws on the same ticket; the push to exactly
604,800 then 1 s more reaches the edge every time.)
sBPF sha256 after the run unchanged (Floor 8ca8a608, Harbor 543be60b,
Vault 7086148b): tests only.
Sibling search: the other refusal counters in this file (reset_refused,
lifeboat_refused_*, loosen_*) are each split by cause already.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check settle and lifeboat refusals by code in the wide differential
The same gap as the extension refusal, found in the sibling search for
the previous commit: a refused settle and the three lifeboat-move
refusals asserted only r.is_err(). Any refusal passed, so the order of
the checks and the program's own codes went unchecked by the
differential. Settle's comment (custody.rs:520-521) says it returns its
own error, not the token program's, when the vault is short; nothing in
this file held it to that.
Now a settle refused while frozen must fail with FROZEN, and one that is
only short with INSUFFICIENT_BALANCE (frozen is checked first). A
lifeboat move inside the delay after the seal must fail with NOT_SEALED,
one while frozen with FROZEN, and one beyond the balance with the token
program's InsufficientFunds (1). The two settle refusals get their own
counters: settle_refused_frozen is required >= 10 like the rest;
settle_refused_short needs lifeboat moves to drain a due ticket's
tokens first and occurs 7 times in the default run, so it is required
>= 1.
Evidence: mutants of programs/vault/src, each built and run against the
previous head's differential_wide and this one, nothing else:
S1 settle: drop the InsufficientBalance check old SURVIVED new KILLED
S2 settle: balance check before the freeze old SURVIVED new KILLED
L1 require_sealed returns Frozen old SURVIVED new KILLED
L2 move_in's freeze returns NotSealed old SURVIVED new KILLED
The token-program code (1) on a move beyond the balance is pinned but
not shown red by a mutant. sBPF sha256 unchanged (Floor 8ca8a608,
Harbor 543be60b, Vault 7086148b): tests only.
Deep run of both commits together (BULKHEAD_PROGRAM_SEEDS=600
BULKHEAD_PROGRAM_STEPS=500, 85 s): 300,000 steps, all identical;
extend 876, extend_push 6,462, extend_refused_call 1,175,
extend_refused_total 1,453, settle 2,024, settle_refused_frozen 921,
settle_refused_short 566, lifeboat refusals 3,854 + 5,768 + 1,171. docs/09 §7's
row gets that run as a 16 Oct result next to the 5 Oct one, which
predates these checks.
Sibling search: in differential_wide.rs, reset (TICKETS_PENDING) and
execute change already asserted codes, and a release or quote refused
for want of tokens asserts INSUFFICIENT_BALANCE. A release or quote the
limiter refuses (frozen, lane closed, queue full) is still compared only
as a refusal: mapping the core's refusal to the program's code is left
for its own change. Lifeboat returns only send legal amounts. docs/09's
row now says which refusals are checked by code. differential.rs:327
has the same is_err settle check; left as it is, since the wide run now pins both
settle codes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check limiter refusals by code in the wide differential
A release or quote that the limiter refused (lane closed, queue full)
was compared with the model only as Err. Any refusal code passed: a
program that swapped LaneClosed and QueueFull in custody.rs's
limiter_error, or checked the backed part against the balance before
the limiter and so answered InsufficientBalance for a full queue,
still matched the model. About 670 such refusals happen in a default
run.
The model now maps core's LimiterError to the program's code with
verr::limiter in the test harness (all four variants, as custody.rs's
limiter_error does), and every refused quote and release must fail
with the model's code. Lane-closed and queue-full refusals have their
own counters, each required at least 10 times; a refusal code with no
counter panics rather than being counted as want of tokens. docs/09
row 362 says so and records the deep run.
Evidence (autopilot tasks/differential-limiter-codes/):
- mutants.log: C1 (LaneClosed and QueueFull swapped in
limiter_error), C2 (QueueFull -> InsufficientBalance), C3
(LaneClosed -> ZeroAmount) and C4 (backed > balance checked before
the limiter, same line) each SURVIVED the old test and are KILLED
by the new one. Hashes back to 8ca8a608/543be60b/7086148b after.
- c4-full-suite.log: under C4 the whole program suite (Vault 113,
Floor 50, the fixed differential) passes; only this test fails.
C1-C3 are also caught by vault.rs's code tests (:434, :3428).
- Default run: refused_lane_closed 254, refused_queue_full 414,
insufficient 118.
- deep-600x500.log: 300,000 steps, 15,561 lane closed, 18,853 queue
full, 6,345 want of tokens, all identical.
Sibling search: every other Err arm in differential_wide.rs already
asserts a code (extend, settle, reset, lifeboat in, execute change).
Not here: lifeboat returns are never refused (backlog), and lifeboat
move_in beyond the balance fails with the token program's code
(backlog, a program change).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check release refusals by code in the fixed differential
differential.rs reduced every refused release to Err(()) on both
sides, so a program that refused for the wrong reason still matched
the model. The same gap the previous commit closed in the wide
differential (found by its code review).
The model's refusal now carries the program's code: verr::limiter for
a limiter refusal, INSUFFICIENT_BALANCE when the vault holds less
than it would pay. A refused release must fail with that code, and
the assert prints the program's error. Refusals for a full queue get
their own counter, required at least 10 times like the others. Want of
tokens and a closed lane are counted and printed but not required:
the wide differential requires both. A refusal with any other code
panics, as in the wide differential, so the counts always add up.
docs/09 row 361 says so; its undated figure (4,843 releases) no longer
matched a run, so it is replaced by a dated one.
Evidence (autopilot tasks/differential-limiter-codes/):
- mutants-narrow.log (mutants.sh <wt> differential): C1 (LaneClosed
and QueueFull swapped), C2 (QueueFull -> InsufficientBalance) and C4
(backed > balance checked before the limiter) SURVIVED the old test
and are KILLED by the new one. C3 (LaneClosed -> ZeroAmount)
survives both: no release in this test's runs is refused for a
closed lane. The wide differential kills it, and docs/09 says so.
Hashes back to 8ca8a608/543be60b/7086148b after.
- r2/default-run.log: 1,000 steps, 82 releases refused, all for a
full queue (want of tokens 0, lane closed 0).
…
shuv-amp
added a commit
that referenced
this pull request
Oct 18, 2026
* Run the tests on 2-of-4 guardian sets; test recovery from two faults
Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set
with fewer than threshold + 2 keys. Every test fixture and most tests'
own sets were 2-of-3. This moves them to 2-of-4, which today's rule
also accepts, so this commit changes no program: Floor, Vault and Harbor
hash 508e5a99 / b07655d9 / 543be60b, as before.
- Fixtures: Protected (Floor) and Market (Vault, through Harbor) now
hold 4 guardians. Each test that built its own 3-key set gets a 4-key
set with the same threshold and the same assertion.
- Two tests changed meaning with a 4th key and were fixed, not loosened:
a_quorum_ejects_a_rogue_guardian_who_vetoes_everything (Floor) and
a_quorum_ejects_a_guardian_who_freezes_forever (Vault). With 4 keys,
the last eject was refused only because the target's own vote is
masked. Both now eject a second guardian (a set an ejection shrank to
threshold + 1 still ejects) and then show the 2-of-2 refusal.
- Account sweeps: an eject's signers may now be any two of the three
others, so each of those two slots is free for the third, never for
the target or the other signer again. Vault 5,365 and Floor 1,710
substitutions, refused (docs/09, dated 16 Oct). Listing all three
others at each slot would have skipped the duplicate: with a mutant
that counts a guardian listed twice as two (governance.rs:270), the
sweeps listing all three pass, and these fail at "eject: account 2
replaced by key ..." (Vault and Floor). No other program test kills
that mutant (tasks/aud2-guardian-margin-impl/dup-mutant-*.log).
- floor_differential: the guardian pool is 6 keys, 4 registered; the
proposed sets are 2 to 6 keys.
New tests (pass on both rules; they show what the margin buys):
- a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_set (Floor): g0's
key lost, g1 rogue stops the program; the lost key does not help a
quorum; the honest pair ejects g1 and restores; neither can veto the
owner's refill, which executes after 72h.
- two_rogues_in_turn_are_both_ejected_from_a_two_of_four_set (Floor):
the second rogue vetoes the refill and is ejected from the 2-of-3.
- a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_vault: the same in
the Vault (freeze, eject, unfreeze, refill).
Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest
and quote sweep): the counts and the sweep figures here are from runs
there (tasks/aud2-guardian-margin-impl/r4/). The E2E run, the mutants
and the new-rule run below were made before, on e8640a7; this commit's
programs and core are the same as there but for one comment line in
views.rs (line-neutral, same hashes).
Evidence:
- E2E=1 scripts/check-programs.sh: PROGRAM CHECKS PASSED (Floor 49,
Vault 98; S1 and E2E passed; the resumed setup found all 14 steps;
vault check, Floor demo and deploy dry run passed), with the next
commit's deploy rehearsal in the tree.
- tools/claims.py: OK (floor 49, vault 104, programs 153).
- Under the new rule (core from two commits on, programs rebuilt), only
the two tests that state the old rule fail: an_n_of_n_guardian_set_is_
refused (accepts 3-of-4) and the Floor differential's model.
- Mutants (tasks/coverage-map/mutants.sh), each red, then restored:
AV1 Floor live_guardian ignores liveness: KILLED by a_lost_key_and_a_
rogue_do_not_lock_a_two_of_four_set (+2 others).
AV2 the Vault's: KILLED by ..._two_of_four_vault (+2 others).
AQ1 Floor restore quorum ignores liveness: KILLED by ..._two_of_four_set
and the differential only.
AQ2 the Vault's unfreeze quorum: KILLED by ..._two_of_four_vault and
a_silent_guardian_does_not_count_toward_a_quorum.
AE1 can_eject refuses at count == threshold + 1 (AUD-2 option (a)):
KILLED by two_rogues_in_turn_..., and the two eject tests above.
Hashes after: 508e5a99 / 543be60b / b07655d9.
Split on 16 Oct from ec0cacc at the owner's review (D2): the deploy
rehearsal's move to 4 guardians is now the next commit. programs/ and
programs/tests/ here are byte-identical to ec0cacc, so its runs below
hold for this commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Rehearse deploys with four guardians, and read only the keys there are
Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set
with fewer than threshold + 2 keys, so the deploy rehearsal must
register 2-of-4 sets. Split from ec0cacc at the owner's review (D2);
the text is that commit's, unchanged:
Deploy rehearsal (cloud/deploy market.mjs, vault-check.mjs,
floor-demo.mjs): 4 guardians (keys/guardian3), so the dry run keeps
working once the rule lands; one more funding step (14, was 13).
The devnet market has 3 guardian files. A first version of this change
loaded guardian3 everywhere: vault-check.mjs failed there with ENOENT,
and heartbeat silently created a key nobody holds as a guardian. Now
market.mjs guardianKeys(dir, create) reads guardian0, 1, ... and
throws on a gap (never fills it with a new key). Setup creates keys up
to 4 only while the protocol is not registered, so resuming an old
market adds no key the chain lacks; a new deploy-dryrun step resumes
the dry-run market with guardian3 moved aside and fails if setup makes
it again (with setup always creating: "DRY RUN FAILED: resuming a
registered market made keys/guardian3.json", r3-dryrun-J7.log).
renew-guardians tops up to 4 and logs each new key (unfunded); a
2-of-3 renewal would be refused once the rule lands. keysIn reads the
guardians only when a command uses them, so caps and set-recovery
never need them; heartbeat says a key is not in the set rather than
dead. One THRESHOLD (2) for the register and renewal payloads and
MIN_GUARDIANS. vault-check.mjs
compares the chain's set with the first files, and liveness with the
chain's count. An extra file (a renewal not executed yet) and a set
without the margin are info lines, not failures, since such a set
keeps working. GUARDIAN_MARGIN and guardianMarginProblem live in
cloud/lib/solana.mjs, used by these and by the guardian bot's existing
warning (cloud/guardian/main.mjs), and vault-check uses THRESHOLD. The e2e scripts fund guardian3 too. Three
tests in cloud/test/deploy.test.mjs (client tests 176 -> 179). JS
mutants (js-mutants-r3.log), each red then restored: always create,
never create, no gap check, margin 1, `>`, GUARDIAN_MARGIN = 1.
Setup always creating survives these tests and is killed by the dry
run step above.
cloud/, scripts/deploy-dryrun.sh and docs/09's client-test row are
byte-identical to ec0cacc (git diff empty), so its runs hold for them:
the E2E check-programs run with the dry run (r4), and the JS mutants.
tools/claims.py: OK (client 179).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Refuse a guardian set without two keys beyond its threshold (107)
AUD-2, decided by the owner on 16 Oct (option (e) of 16 §4.4). A set
stays live while its honest live guardians reach the threshold: they
eject every rogue in turn, and dead guardians cannot veto the owner's
new set. Finding 96 asked for one key beyond the threshold, so a 2-of-3
or 3-of-4 set survived one fault but not a lost key plus a rogue: the
honest guardian left is no quorum, nobody can be ejected, and the rogue
keeps Floor stopped (or the Vault frozen) and vetoes every new set.
Fix, in the core so both programs inherit it through GuardianSet::parse
(registration, and every SetGuardians when proposed and when executed):
GUARDIAN_MARGIN = 2
guardian_set_ok := count <= 8 && t >= 2 && count >= t + 2
(saturating_add: t is a payload byte). A stored set is not checked
again, so existing 2-of-3 sets keep working, and ejection is unchanged:
a shrunk set still ejects. A 2- or 3-key set is BadThreshold, not
BadCount; the count rule is only the bitmap size, and the error order
of finding 103 is kept.
This changes both programs, by intent:
Floor 508e5a99 -> 8ca8a608, Vault b07655d9 -> 7086148b,
Harbor 543be60b (unchanged). Darwin-arm64 builds.
Compute (scratch LiteSVM test, tasks/aud2-guardian-margin-impl/cu.md):
-3 CU on every path that parses a set (Floor propose 2035 -> 2032,
execute 1682 -> 1679; Vault register 9592 -> 9589, SetGuardians 2979 ->
2976; Floor register varies with its PDA bump, residue also -3).
Proof and tests:
- Kani: accepted_sets_can_eject_anyone restated (ejection reachable
exactly when count > t, so option (a) cannot slip in; accepted sets
have >= 4 keys and a margin of at least 2, and 2 is accepted). New:
an_honest_live_quorum_ejects_any_rogue (any slots, anyone else alive)
and accepted_sets_survive_any_two_faults (any two faulty keys, lost
or rogue, leave a live honest quorum, which ejects each rogue, and the
rogues alone eject no honest guardian). That holds only short of a
rogue quorum: two rogues in a 2-of-n set are one (out of scope, 04
T7). With the assumption rogue < t dropped, the harness fails at
`!can_eject(rogue, ..)` (kani-no-rogue-assume.log), so the docs say
"short of a rogue quorum", not "any two faults".
19/19 verified, 240 s (cargo kani -j 2, as CI); again after the
restatement, 235 s summed (r2-prove.log).
- Kani mutants (kani-mutants.sh): margin 1, margin 3, `>` for `>=`,
t >= 1, count <= 9, option (a), and an eject one vote short (only
the two-fault harness kills that one): each KILLED by a harness
(kani-mutants-r2.log). The eject
ignoring liveness is the eject harness's case, not these.
- Program mutants (coverage-map/mutants.sh, rebuilt each time):
margin 1: KILLED by a_guardian_set_without_two_spare_keys_is_refused
(Floor and Vault), registration_rejects_weak_settings and the Floor
differential. Margin 3 and `>`: KILLED by most of the suite (the
fixtures are 2-of-4). t >= 1: first only the Floor differential; the
1-of-2 registration case was now refused by the margin too, so it is a
1-of-4 here and a Vault 1-of-4 case was added: then KILLED by both.
count <= 9: SURVIVES the program tests, equivalent there (parse refuses
more than 8 keys first), KILLED by Kani. Option (a): KILLED by the
eject tests two commits back (the tests on 2-of-4 sets).
- Tests: the Floor and Vault refusal tests above cover proposal and
execution (a stored 2-of-4 proposal whose threshold turns 3 is refused
when executed, as the devnet renewals proposed before this rule will
be); registration_rejects_weak_settings gains 2-of-3 and 3-of-4.
an_n_of_n_guardian_set_is_refused (Vault) is renamed and accepted a
3-of-4: it now refuses it and accepts 3-of-5.
- Model: the Python rule, the Floor differential's model, and the
guardian_set fuzz target state it again; the GP generator now also
picks count - 2. With the Python rule put back to the old one, seed 11
(20,000 cases) reports 5 GP mismatches; restored, seeds 11-13 identical.
Fuzz seeds two_of_three_refused, two_of_four, six_of_eight, and
seven_of_eight renamed seven_of_eight_refused (byte 323, the
threshold, is 7: now refused). With margin 1 in the core the target
crashes within 60 s; restored, 222,615,288 runs in 601 s, no crash.
- scripts/check.sh: ALL CHECKS PASSED, and E2E=1 check-programs.sh:
PROGRAM CHECKS PASSED, both at the branch tip, whose code is this
commit's (the next commit is docs only; tasks/aud2-guardian-margin-
impl/r4/check.log, r4/check-programs-e2e.log). claims: kani 19,
floor 50, vault 104, programs 154, client 179.
Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest
and quote sweep). The core and the program sources are the same as
before the rebase but for one comment line in views.rs (line-neutral),
so the Kani, fuzz, differential and mutant runs above were made on the
pre-rebase commit; the hashes, check.sh, check-programs and the counts
are from r4 runs (tasks/aud2-guardian-margin-impl/r4/).
The finding 96 row said the restated harness proves "exactly"; it now
says what it proved then and proves now, and gives the Vault test's new
name. The finding 107 row waits for rows 105 and 106 to land (09 §4 is
numbered 1..n); its text is ready. The rule's prose is the next commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* State the guardian margin in the docs, with its costs (AUD-2)
The previous commit made both programs refuse a new guardian set
without two keys beyond its threshold (finding 107). The docs that
state the rule now say so:
- README: guardians 4-8 keys; the G5 row and the "who the guardians
are" paragraph give the reason (any two faults leave a quorum, unless
the rogues are one themselves: two in a 2-of-n set, 04 T7).
- 05 register_protocol: 2 <= threshold <= count - 2, also for every
SetGuardians, proposed and executed. 05 abdicate: abdicating
SetGuardians fixes the set, so ejections shrink it for good.
- 06 state table, 00 D22, FAQ: the same rule.
- 07 §2: 4-8 guardians, and don't abdicate SetGuardians. 07 §6: the
programs enforce the margin; sets registered before keep their
exposure until replaced; an ejection uses up a spare key until the
owner's new set executes; a lost key counts as live until its
interval passes.
- 16: the AUD-2 row is decided; §4.4 gives the decision, why the
margin is the number of faults a set survives (short of a rogue
quorum), the options rejected
and why (the honest-owner and stolen-owner cases look the same
on-chain), what it costs, what it leaves (a rogue quorum; a stolen
owner key with every guardian dead; a rogue's veto of the recovery
key's rotation, checked: Floor's veto has no recovery exception, as
a_vetoed_recovery_proposal_does_not_hold_the_owner_s_slot shows), and
Lido-style lapsing vetoes as a future option only. §5: "independent
guardians, 3-of-5 or more".
- 10: the guardian set's minimum size is no longer an open decision.
SECURITY.md and 08 describe what is on devnet, which still runs the
old rule, and are left as they are.
claims: OK. Docs only; no program file changed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Say the weekly CI run skips mutants, as its condition does
ci.yml's header and scripts/ci-changes.sh said the weekly run does every
job, and 09 §8 gave `mutants` as running on any change to the core. The
`mutants` job's `if` starts with `github.event_name == 'pull_request'`,
so the Monday schedule (and a manual dispatch) never runs it: it mutates
a pull request's diff and has none to mutate. Its trigger list in 09 §8
also left out scripts/mutants.sh and .cargo/mutants.toml, which
ci-changes.sh (line 22) selects it for.
Comments and docs only; nothing runs differently.
Sibling search: `git grep` for "weekly run does", "every job ... weekly"
and "mutants ... weekly" across md/yml/sh/mjs/py. The other hits are
09 §8's "at full depth every Monday" (its table now says which job is
not weekly) and the dispatch input's "every job at weekly depth", which
is the same set as the weekly run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check that the mutant exclusions still match; repin murmur3 (finding 105)
.cargo/mutants.toml excludes the core's 22 equivalent mutants, and the
docs say so. Since 9 Oct it excluded 19. The murmur3 tail's pattern is
pinned by line, `floor.rs:(235|238|241)`, because the body's `h ^= k`
has the same column and function and is not equivalent. 1dc7861 (#52)
moved those lines (now 327/330/333), and nothing checked the patterns.
Failing closed, the three would have failed CI's `mutants` job on a
pull request that changed those lines. Failing open, a pattern that
starts matching a non-equivalent mutant hides it from every run.
Root cause: nothing checked that a pattern still names its mutants.
Fix:
- each pattern ends with the number of mutants it excludes (`# 3`), and
the murmur3 pattern names 327/330/333;
- tools/mutant_exclusions.py fails unless each pattern matches exactly
its number in `cargo mutants --list --no-config`, and cargo-mutants
(the list with the config) excludes exactly the mutants the patterns
match, so a pattern this parser can't see, or reads differently, fails
too. No two patterns may match the same mutant, since the docs' count
is the counts' sum, and the kept list must be part of the full one.
It reads only the exclude_re array's body, needs no tomllib
(Python 3.9), and names colour codes in the list as the problem;
- scripts/mutants.sh makes both lists (--colors never: CI sets
CARGO_TERM_COLOR=always) and runs it first, except when the caller
passes --config or --no-config (as whole arguments). CI's `mutants` job runs that script;
ci-changes.sh now also selects the job for the new tool;
- scripts/check.sh runs its --self-test on planted configs;
- tools/claims.py ties the docs' "22 equivalents" (README, 09 §6 and the
summary row) to the sum of the counts, with the same parser. It finds
that parser from its own directory, so `python3 -I` runs it too, and
check.sh runs its self-test that way. 00's
"18 equivalents" (26 Sep log entry) and run 9's "exactly the 22
equivalents below" (6 Oct) are kept as dated records.
Docs: 09 §4 finding 105, §6 run 10 and the "In CI" note, §8's mutants
row; README and 09's summary row 669 → 797 mutants (run 10); README and
12's post-submission range 96–105.
Evidence (first on 1f2ad48 + the parent, logs in the autopilot's
tasks/mutants-exclusions-r2/; re-run on 58697c6 + the parent,
tasks/mutants-exclusions-r3/):
- `cargo mutants --list`: 797 with --no-config, 778 with the old config
(19 excluded), 775 with this one. The 22 excluded are §6's list, the
murmur3 ones at 327/330/333.
- scripts/mutants.sh --list: exit 0, "22 of 797 mutants excluded"; with
CARGO_TERM_COLOR=always also exit 0. Old config: exit 1, every pattern
"no '# N' count". Planted on the real lists: the old lines with a
count (matches 0, says 3) and the body XOR added (matches 4, says 3)
both exit 1; in both the set comparison alone passes. /usr/bin/python3
(3.9.6): self-test OK and the real check passes.
- The self-test pins each problem by its message, not a count. Mutants
of the tool (tool-mutants.log), each red: M1 no per-pattern check,
M2 no set comparison, M3 no colour check, M4 no empty-array check,
M5 a missing count accepted, M6 a missing count read as 0, M7 the
parser reading past the array. M3 and M5 survived a first version
that checked only the number of problems.
After A's review (16 Oct): the set comparison was pinned only by its
size, so `len(excluded) != len(matched)` passed the self-test. A
case with as many excluded as matched but not the same ones now
fails it (15 cases; red with that mutant, green without), and the
message says how many mutants are in one set only.
From the code review after the rebase: colour codes only in the kept
list were reported as a kept list made with other options; the
colour check now reads both lists (16 cases; dropping the kept half
fails the new case).
On 58697c6, with the overlap and kept-list checks: M8 no overlap
check and M9 no kept-list check, red too, as M1–M7 still are. A
duplicate of the can_eject pattern planted in the real config:
scripts/mutants.sh --list exits 1, "the counts add up to 23, the
patterns match 22 mutants" (planted-overlap.log). Without the
sys.path line, `python3 -I tools/claims.py` fails with
ModuleNotFoundError; with it, both claims runs pass.
- claims.py (claims-mutants.log): one count 1 → 2 fails four statements
(README 188; 09 33, 333, 335; kept-run9.log), not run 9's 331, which
failed too before it was kept; a dropped count fails with the
tool named; deleting any of the three rules fails a self-test case;
deleting the KEPT entry fails 00:103.
- The 22 excluded mutants run alone (--no-config, --re per pattern,
2 min): 22 missed, exactly the excluded list; --re also picked
ema.rs:70 (a field deletion, no pattern matches it), caught.
- Run 10 (f1f05ba, old config, -j4, 46 min; tasks/mutants-run10/):
778 tested, 712 caught, 5 timeouts, 58 unviable, 3 missed, exactly
the murmur3 tail. The core's source is unchanged since (`git diff
--stat f1f05ba HEAD -- crates/bulkhead-core/src` is empty; only
tests/units.rs gained finding 102's tests).
- No program or core source changed.
Rebased on 16 Oct onto cede705 (findings 102 and 100, the attest
sweep and AUD-2). AUD-2 changed the core's source (governance.rs), so
the line above saying it is unchanged since run 10 held on 58697c6, not
here. On cede705: scripts/mutants.sh's list check passes, "22 of 795
mutants excluded, each pattern matching its count"; the murmur3 lines
are still 327/330/333. The 797 in README and 09 is run 10's figure:
README now dates it ("in the last full run, 15 Oct") and finding
105's row says the core has changed since (finding 107).
The setup line in scripts/mutants.sh now pins cargo-mutants 27.1.0,
the version ci.yml installs: each count is the number of mutants that
version lists, so another release can break the check (code review,
16 Oct; `git grep 'install.*cargo-mutants'` finds no other unpinned
install).
The check's report goes to stderr in scripts/mutants.sh, so
`scripts/mutants.sh --list` prints only cargo-mutants' names: before,
its first stdout line was the report; after, stdout is 773 lines
(795 - 22) and stderr the one report line. ci.yml's `mutants` comment
says the job checks the exclusions first.
Row 105 and 09 §6 gave the pin's reason as the body's XORs. The
finalizer's are at column 7 (`grep -n '\^=' floor.rs`, 337-342); the
non-equivalent XOR at column 11 is `h ^= k`, in the body (320) and after
the tail (335). The toml comment, the row and §6 now say that. 09 §8's
`check` row names the two tool self-tests check.sh runs.
README no longer calls run 10's 797 "injected bugs": cargo-mutants
listed 797 but tested 778, 19 being excluded (09 §6, run 10's row).
Sibling search: the other 14 patterns use `\d+` for the line, and each
matches the count now written after it. The murmur3 one is the only
line-pinned pattern.
Not done, by choice:
- Catching an edit that keeps a pattern's number but changes its
mutants (an excluded line itself edited, or a murmur3 body XOR moved
onto a pinned line). The toml and 09 §6 say so: such an edit needs the
mutant judged again by hand. Failing on any diff that touches an
excluded line would catch the first; it is a separate change.
- Moving murmur3's tail into its own function, so its pattern needn't
name lines. It shifts core lines, so it changes the program hashes.
Until then a line move above it fails the check, with a message
saying what to fix.
- A weekly full mutants run in CI. It took 46 min here at -j4, likely
past the job's 90 min on two CI cores: a minutes call for the owner.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that a ticket stays in its own protocol and vault
Two Vault checks that keep a ticket to its own protocol and vault had no
test. Deleting either one left every program test green
(tasks/error-codes/mutants.log, base cede705):
- brakes.rs:79, extend_ticket: tk.protocol != protocol. Without it, a live
guardian of any other protocol extends this protocol's tickets. Anyone
can register a protocol and name its guardians, so that guardian could
hold back every queued payout up to the total extension limit. The
sweep did not catch it: it has no second protocol's guardian to swap in.
- custody.rs:501, settle_ticket: tk.vault != vault. A USDC ticket settled
through the SOL vault (its tokens, its mint) got past the Vault's checks.
New test another_protocols_guardian_cannot_extend_a_ticket: a second
Market's live guardian extends this market's ticket -> WRONG_VAULT; the
same instruction from this market's guardian succeeds. The settle test
now settles the USDC ticket through the SOL vault and expects
WRONG_VAULT.
Evidence (tasks/error-codes/mutants.log, STATE=c1; full program suite
per mutant):
W02 brakes.rs:79 deleted -> KILLED by
another_protocols_guardian_cannot_extend_a_ticket ("expected
failure, got success": the stranger's extension went through)
W06 custody.rs:501 deleted -> KILLED by
a_ticket_settles_after_its_eta_to_its_stored_recipient_only (8049
InsufficientBalance from the empty SOL vault, not 8010)
Both were SURVIVED at base. No program source change: hashes
8ca8a608/543be60b/7086148b, as at cede705.
Sibling search: the other 12 WrongVault sites were deleted one at a time
in the same run. Eight are KILLED by the sweep (brakes.rs:39, lib.rs:320,
custody.rs:160, genesis.rs:491, views.rs:27, lifeboat.rs:151,
governance.rs:169/259). lib.rs:324 (vault PDA address), genesis.rs:523,
lifeboat.rs:71 and :247 survive; later commits on this branch deal with
them.
Counts: Vault 104 -> 105, programs 154 -> 155 (claims.py OK).
The test also reads the ticket's ETA: unchanged after the
refused extension, 3,600 s later after its own guardian's (code-review
fix: success alone did not show the extension happened).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that the seal names every vault and happens once
Two seal checks had no test; deleting either left every program test
green (tasks/error-codes/mutants.log, base cede705):
- genesis.rs:486, execute_seal: the pairs must number 3 * vault_count.
Without it, the seal checks only the vaults it is given. A vault left
out skips its claims-vs-balance check, so an uncovered import seals.
docs/05's seal row says every vault is passed exactly once.
the_seal_checks_each_vault_once covered a vault named twice in place of
another, not one left out.
- genesis.rs:431, propose_seal: refused once sealed (AlreadySealed).
Without it, a sealed protocol's admin re-proposes, which rewrites
seal_executable_s/expires_s and clears seal_vetoed on a sealed protocol.
the_seal_checks_each_vault_once now also leaves the uncovered USDC vault
out (expects VAULT_COUNT_MISMATCH) and, after the seal, proposes it again
(expects ALREADY_SEALED). No new test, so no count change.
Evidence (tasks/error-codes/mutants.log, STATE=c2, full program suite):
E05 genesis.rs:486 deleted -> KILLED by the_seal_checks_each_vault_once
("expected failure, got success": the seal went through without
the USDC vault and its 1,000 USDC uncovered import)
E04 genesis.rs:431 deleted -> KILLED by the_seal_checks_each_vault_once
("expected failure, got success")
Both were SURVIVED at base. Vault hash 7086148b unchanged (tests only).
Sibling search: the seal's other checks. :491 vault_tokens (KILLED by the
sweep), :519 lifeboat owner and :523 lifeboat mint/owner (SURVIVED; the
lifeboat is a PDA checked by expect_pda just above, which only this
program can create, as its own token account: see this branch's note in
docs/09).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that invalid limits and asset settings are refused at every entry
Three bound checks had no test; deleting any one left every program test
green (tasks/error-codes/mutants.log, base cede705):
- genesis.rs:171, create_vault: config_is_valid (bps <= 100%, windows
> 0). Without it, a vault is created with a 100.01%-an-hour burst.
- governance.rs:40, parse of a SetLimits change (at proposal and again
at execution, :254 and :478): the same rule for a proposed change.
- governance.rs:47, parse of an UpsertAsset change: valid_asset (haircut
<= 100%, liability >= 100%, the oracle bounds of finding 88). Only the
genesis upsert's copy (genesis.rs:296) was tested, by
oracle_settings_of_a_priced_asset_are_bounded.
New test invalid_limits_and_assets_are_refused_wherever_they_enter:
before the seal, create_vault with short_bps 10,001 -> INVALID_LIMITS;
after it, a SetLimits proposal with long_window_s 0 -> INVALID_LIMITS,
and an UpsertAsset proposal with a 10,001 bps haircut -> INVALID_ASSET.
Evidence (tasks/error-codes/mutants.log, STATE=c3, full program suite):
E06 genesis.rs:171 deleted -> KILLED ("expected failure, got success")
E07 governance.rs:40 deleted -> KILLED ("expected failure, got success")
E08 governance.rs:47 deleted -> KILLED ("expected failure, got success")
each by the new test only. All three were SURVIVED at base. Vault hash
7086148b unchanged (tests only).
Sibling search: every config_is_valid / valid_asset call in programs/
vault/src: genesis.rs:170 (create_vault), :295 (upsert_asset, KILLED by
the oracle test), governance.rs:39 and :46. All now killed.
Counts: Vault 105 -> 106, programs 155 -> 156 (claims.py OK).
The execution copy (:478) is not reached by any transaction: it
parses the bytes the proposal already checked, and the bounds read
nothing else. The test comment says so (code-review fix: it had
claimed execution was covered). oracle_settings_of_a_priced_asset_are_
bounded now asserts INVALID_ASSET for each of its four cases, not any
error (all four return it; E09 stays KILLED).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that an import into a ninth mint is refused
genesis.rs:410, import_backing: a backing ledger holds MAX_SLOTS (8)
mints, and an import into a ninth is refused with NoSlot. Deleting the
check left every program test green (tasks/error-codes/mutants.log, base
cede705). From the code, without it the import goes through: the vault's
imported total grows (:414), and the seal must cover that, but the
owner's ledger is saved unchanged, so the owner has no backed claim for
it.
New test an_import_into_a_ninth_mint_is_refused: nine vaults before the
seal; one owner gets an import into eight (ledger_of shows 8 positions),
then the ninth -> NO_SLOT.
Evidence (tasks/error-codes/mutants.log, STATE=c4, full program suite):
E14 genesis.rs:410 deleted -> KILLED by an_import_into_a_ninth_mint_is_
refused ("expected failure, got success"); SURVIVED at base.
Vault hash 7086148b unchanged (tests only).
Sibling search: the only other ledger credit is deposit's
(custody.rs:128). There a full ledger is by design: the deposit is taken
unbacked (Ledger::credit's doc, ledger.rs:187-189), and credit's result is
the backed flag, not an error.
Counts: Vault 106 -> 107, programs 156 -> 157 (claims.py OK).
import_ix now calls import_into_ix, which takes the vault, so the
new test shares the one import layout (code-review fix).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin six Vault refusals no test reached
Six reachable checks had no test that failed when the check was deleted
(tasks/error-codes/mutants.log, base cede705). Each now has an assertion
on its code, and each was seen to fail with the check deleted:
E02 genesis.rs:185 create_vault of an existing vault after the seal:
ALREADY_INITIALIZED (with the check gone: 8043 GenesisOnly, so the
"says so first" order in the comment above it was untested)
E12 governance.rs:426 veto of an executed change: CHANGE_EXECUTED
(gone: the veto succeeds and marks an executed change vetoed)
E13 governance.rs:517 cancel of an executed change: CHANGE_EXECUTED
(gone: the cancel succeeds)
E16 lifeboat.rs:256 execute_exit with no exit proposed:
NOTHING_TO_EXIT (gone: 8033 ChangeNotExecutable)
E17 custody.rs:84 a zero deposit: ZERO_AMOUNT (gone: it succeeds)
E18 custody.rs:272 a zero release: ZERO_AMOUNT (gone: it succeeds; a
zero release has no unbacked part, and the limiter, whose own zero
check custody.rs:242 maps, is called only when unbacked > 0, :361)
Where: E02 in after_the_seal_a_new_vault_starts_with_its_unbacked_lane_
closed, E12/E13 in an_unvetoed_loosening_executes_after_the_delay, E16 in
the_lifeboat_stays_shut_until_the_seal (after the move in, before an
exit is proposed), E17/E18 in the new a_zero_deposit_or_release_is_
refused.
Evidence (tasks/error-codes/mutants.log, STATE=c5, full program suite):
all six KILLED by the tests named above; all six SURVIVED at base. The
vault suite is green (108 passed). Vault hash 7086148b unchanged.
Counts: Vault 107 -> 108, programs 157 -> 158 (claims.py OK).
Helpers (code-review fixes): veto_change_ix beside cancel_change_ix;
create_vault_for_ix, create_vault_ix with a given mint and token
program, so E02's case names USDC's own accounts and creates no mint.
E02's comment now repeats the program's ("whatever the seal's state"):
create_vault checks the limits first (genesis.rs:170).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin four Vault checks another layer backstops
Four checks survived deletion (tasks/error-codes/mutants.log, base
cede705) because something later still refused the case, with its own
error, so no test could tell. Each now has a test that sees the Vault's
own code, and each was seen to fail with the check deleted:
E01 lib.rs:230, create_pda's guard: a second registration of an
existing protocol (register_protocol has no check of its own) ->
ALREADY_INITIALIZED. Gone: the system program refuses it, with no
custom code (custom_code 0).
W10 lifeboat.rs:71, move_in naming another mint or token program ->
WRONG_VAULT. Gone: the token program refuses (code 3, MintMismatch).
W12 lifeboat.rs:247, execute_exit naming another mint or token
program -> WRONG_VAULT. Gone: the same token-program error.
T01 token.rs:37, account_info on an account no token program owns: a
copy of a real USDC account's bytes under the system program. Gone:
the quote SUCCEEDS (it promises a payout), and the release fails
only in the runtime at the transfer ("instruction modified data of
an account it does not own"). The quote is asserted first so the
mutant is seen at the quote (STATE=c6b).
New tests: a_protocol_registers_once, a_lifeboat_move_names_the_vaults_
own_mint (also completes a move in and an exit, as a control: the
destination receives 1,000 USDC), a_payout_goes_only_to_an_account_a_
token_program_owns.
Evidence (tasks/error-codes/mutants.log, STATE=c6 and c6b, full program
suite): E01, W10, W12, T01 KILLED by the tests named above; all four
SURVIVED at base. Full program suite green: Floor 50, Vault 111. Vault
hash 7086148b unchanged (tests only).
Counts: Vault 108 -> 111, programs 158 -> 161 (claims.py OK).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test the bounds on ticket extensions and a closed lane's code
B's review of vault-error-tests-r3 (R1) found two Vault codes no test
asserted by value that the earlier commits missed: ExtensionTooLong
and LaneClosed. Deleting the 7-day total extension cap (brakes.rs:86,
B24) left every program test green, so a live guardian, or a stolen
guardian key, could hold a queued payout back without end, 24h at a
time, against docs/05 row 11. The wide differential models the total
(differential_wide.rs:406), but its default run never extends one
ticket that far. Mapping LaneClosed to QueueFull (custody.rs:240, B23)
also survived.
The code review of the fix found a worse gap at the same check: no
test sends a negative extension (the differential draws secs from 0,
1, 3600, 86400, 86401). With `secs <= 0` weakened to `secs == 0` (A3,
brakes.rs:84) every program test passed, and one guardian key could
pull a queued payout's ETA into the past, skipping the queue delay,
and refund its own 7-day budget.
- a_ticket_is_extended_later_by_at_most_a_day_a_call_and_seven_in_total:
-1h, 0 and 24h + 1 s are each refused with EXTENSION_TOO_LONG, ETA
unchanged; six 24h extensions and one of 23h pass; 2h more is
refused, ETA unchanged; the last hour is granted; then 1 s more is
refused. The split is unaligned so that a total that ignores this
call's `secs` fails too.
- a_closed_lane_refuses_an_unbacked_payout: a vault with short_bps 0,
and one with long_bps 0, each refuse an unbacked admin payout with
LANE_CLOSED; the same payout from a vault with both buckets open is
taken (the control, so a capacity base of 0 cannot pass it).
Evidence (tasks/vault-error-tests-r4/, B's runner copied: each mutant
applied alone, programs rebuilt, whole program suite run):
- without these tests (551e519; run-without.out, run-base2.out,
run-base3.out): B23, B24, A1 (`tk.extended_s >= MAX`, this call's
secs ignored), A2 (`>=` for `>`) and A3 SURVIVED. A4 (the 24h per
call deleted), A5 and A6 (either half of limiter.rs:172's closed-lane
test deleted) were KILLED by the differential only, not by value;
- with them (run-final.out, at the branch tip): B23 KILLED by
a_closed_lane_refuses_an_unbacked_payout; B24 A1 A2 A3 KILLED by the
extension test only; A4 by it and the differential; A5 A6 by the
closed-lane test and the differential.
No program change: the runner's closing build gives 8ca8a608 /
543be60b / 7086148b, as at cede705.
Counts: Vault 111 -> 113, programs 161 -> 163 (claims.py OK).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin each half of create_pda's existing-account guard on its own
B's review of vault-error-tests-r3 (R2): each half of lib.rs:229
(`!is_data_empty() || !owned_by(SYSTEM)`) survived deleted on its own
(B20, B21), since an existing protocol account fails both, so
a_protocol_registers_once pinned them only jointly.
The test now also plants, at a fresh protocol address, an account that
fails just one half: one with data owned by the system program, one
with no data owned by another program (a fresh address). Each
registration is refused with ALREADY_INITIALIZED. The comment says no transaction is
known to make either account: data or a new owner at a PDA needs its
signature, which only the Vault gives, and create_pda gives both in one
instruction.
Evidence (tasks/vault-error-tests-r4/): at 551e519 B20 and B21
SURVIVED (run-base2.out); at the branch tip both are KILLED by
a_protocol_registers_once only (run-final.out).
No count change (claims.py OK: vault_programs 113, programs 163).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Record the Vault refusal deletions and what each became
docs/09 §3 gains a row for the deletion run this branch answers
(tasks/error-codes/mutants.log): 38 refusals, across 14 of the 18 Vault
codes no test asserted by value at cede705, each deleted on its own
with the whole program suite run against it, on the AUD-2 build.
The 18 (each VaultError variant with no `verr::` use in programs/tests
at cede705): AlreadyInitialized NotRegistrar WrongVault
WrongTokenProgram AlreadySealed VaultCountMismatch InvalidLimits
InvalidAsset LaneClosed TicketNotPending ExtensionTooLong
ChangeExecuted LedgerUnsound LedgerCorrupt NoSlot NothingToExit
ZeroAmount Overflow. The run missed LaneClosed and ExtensionTooLong;
B's review deleted one check of each (B23 custody.rs:240, B24
brakes.rs:86), and both survived until 1797ae5. That commit
also pins a negative extension (A3, which survived every test before
it), the 24h per call, and a lane closed by either bucket.
At base: 13 KILLED, 25 SURVIVED. The commits before this one add tests
that kill 18 of the 25 and both of B's, each asserting the Vault's own
code (W02 W06, E04 E05, E06 E07 E08, E14, E02 E12 E13 E16 E17 E18, E01
W10 W12 T01; B23 B24). The row names the 7 that still survive and why:
- six guards no transaction is known to reach: lib.rs:324 (vault PDA
address; create_vault creates a vault only at its PDA), genesis.rs:519
and :523 (the lifeboat, checked by expect_pda first, created only by
lifeboat_move, move_in requires the seal, lifeboat.rs:53, and the
seal happens once, so no lifeboat holds data at it), brakes.rs:50 and
custody.rs:541 (a pending-ticket count below zero), token.rs:235
(GetAccountDataSize's return data from an already-checked program);
- custody.rs:399, stated as not shown unreachable: properties.rs:588-592
debits backed_amount and one more on random ledgers, and only
crates/bulkhead-core/tests/units.rs:375 debits less, on a one-mint
ledger with no liability. properties.rs:606 asserts backed(a) ==
min(a, backed(MAX)) for requests below 2^52 (:613), so any smaller
amount is backed too, and debit_backed(b) refuses only when
backed_amount(b) != b (ledger.rs:289). The known exception is a
collateral value that overflows u128 (ledger.rs:262): the valid
amounts need not form an interval there, so the consent or freeze
cap (custody.rs:337, :347) can lower the amount to an invalid one and
:399 refuses the release (fail closed). The search's own result is
always valid. Not verified by a tool beyond that property test. (A full ledger cannot reach :399 through
set's NoSlot: backed_amount's valid() refuses a new liability slot
on a full ledger, ledger.rs:237.)
Not deleted, as the row says: the two core-error mappings to
ChangeExecuted and ZeroAmount, token.rs:233, and the codes Overflow and
LedgerCorrupt (checked arithmetic, decoding).
After this branch five codes are still asserted by no test (same grep
at this commit): NotRegistrar (E03, genesis.rs:108, was KILLED at base
by another test), TicketNotPending, LedgerUnsound, LedgerCorrupt,
Overflow.
Docs only: no count change (claims.py OK: vault_programs 113, programs
163).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Run the exclusion check's main in its self-test
A's review of finding 105 (REVIEWS/auto-B-mutants-exclusions-r7.md,
finding 1, Low): CI's `mutants` job acts on main's exit status, but the
self-test called problems() only, and check.sh never runs the tool on
real lists. With main returning 0 on a mismatch (mutant A10), every
in-tree check passed, and CI would have printed the problems and gone
green. This was not a defect at fb771c6, since the job fails today. The
gap was that a later regression would pass.
Fix: main takes its arguments and, for the self-test, a config path
(default: the repo's .cargo/mutants.toml, as before), and its lines name
the config it read. The self-test runs main in-process on a planted
config and checks the exit status and every line on each stream:
- the right lists: exit 0, "1 of 3 mutants excluded" on stdout, nothing
on stderr;
- the lists the other way round: exit 1, nothing on stdout, and exactly
the count and "full list lacks" problems, so KEPT is read as KEPT;
- coloured lists: exit 1 with exactly one problem, so a main that fails
only on several problems is caught;
- one argument and three: exit 2 and the two usage lines.
One subprocess (python3 -I, 60 s timeout, in the temporary directory so
the config must be found from the tool's own path) runs the command line
with two lists and the repo's config, as scripts/mutants.sh does: on a
list no pattern can match, it exits 1 with an empty stdout and problem
lines that each name .cargo/mutants.toml. Which problems those are
depends on the config, so only their form is checked. Self-test: 16
cases -> 22. No doc states the number.
Evidence (tasks/exclusions-main-exit/):
- old-A10.log: the fb771c6 self-test with A10 applied prints "self-test
OK: 16 cases" and exits 0.
- mutants-r6.log (mutants.py applies each mutant to a scratch copy of
the tool, which reads the real config through a symlink): unmutated,
"self-test OK: 22 cases", exit 0. All 18 are KILLED (exit 1): A10
(return 0), A10b (fail only on more than one problem), A0 (always 1),
U1 (usage returns 0), U2/U3 (argument check `< 2`, `< 1`), K1 (KEPT
ignored), K2 (ALL read twice), C1 (planted config ignored), C2 (no
default config), R1 (the default config read from the cwd), E1/E2
(sys.argv sliced as [:] or [1:2]), P1 (problems not printed), S1
(problems on stdout), L1 (the planted config not named), N1 (only the
first problem printed), G1 (the success line printed on failure too).
Code-review passes at high shaped this. The first version ran three
subprocesses against the real config, and A0, K1 and U3 survived it.
The second left E2, C2 and G1 alive. Declined from the third: catching
exceptions in the self-test (a traceback already exits 1) and moving
the config hook out of main. The fourth showed that requiring one
"matches 0 mutants" line per live pattern tied the self-test to the
config's contents. The subprocess now checks the form of the lines
only, and the 17 stay killed. The fifth showed that R1 survived while
the child ran in the repo root (it does in check.sh); now it runs in
the temporary directory. Its other points repeat declined ones.
- real-lists.log: on the real lists, made with scripts/mutants.sh's
commands: ".cargo/mutants.toml: 22 of 795 mutants excluded, each
pattern matching its count", exit 0.
- python3 -I tools/mutant_exclusions.py --self-test: 22 cases OK.
- scripts/check.sh: ALL CHECKS PASSED (check-final.log);
bin/check-each-commit.sh green (each-commit.log).
Sibling search: the other tools check.sh runs (tools/claims.py,
tools/floor_check.py) also gate CI through main's exit status. claims.py
has the same gap: check.sh runs it on the real tree, where it finds no
problems, and its self-test calls self_test(), not the path in main that
returns 1. With that return changed to 0, `--self-test` and the real run
both still exit 0 (claims-C1.log). Not changed here: it is its own
commit (BACKLOG-B). floor_check.py runs in CI only as --self-test, whose
result main returns as is. Its RPC mode is an operator's tool that no CI
job runs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Record finding 106: Vault liveness and exit rules had no failing test
The tests landed in 944f5e8 (lifeboat-exit-tests-r3, 15 Oct). Their
09 §4 row waited for 104 and 105 so the list stays numbered without
gaps; 105 is now on integration (fb771c6), so the row lands here.
Docs only: 09 §4 row 106; README and 12's count and post-submission
range (96–106, 106 fixed). Two code-review passes changed the row as
written on 15 Oct: a sentence for today's re-run, the nine
single-guardian sites named, G5 quoted verbatim, and the exit veto
attributed to T18 (G6 has no veto), as is the veto test's doc comment
(vault.rs:3996, comment only).
Evidence, re-run today on bf8ba41 (the row's claims are from 15 Oct,
and AUD-2 changed the Vault since): B's re-review list of 33 line-
neutral mutants (the ten rules, the nine single-guardian sites L1-L9,
the boundaries, the six quorums, R1-R6), each applied, programs
rebuilt, all program tests run:
33 KILLED, 0 survived, 0 not applied, 0 build failures. Vault
hashes after the run match the tip (Vault 7086148b, Floor 8ca8a608,
Harbor 543be60b).
Log: autopilot tasks/finding-106-row/mutants.log.
- python3 -I tools/claims.py: OK, findings 106.
- git grep for "96–105", "105 findings": no other stale range.
- scripts/mutants.sh still mutates the core only (SCOPE, line 11), so
the row's last sentence holds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Cite PR #100's tree, not the unpublished bf8ba41, in finding 106
Finding 106's row in docs/09 said its 33 mutants were re-run on 16 Oct
at bf8ba41. That commit is not an ancestor of this branch or of the
stack above it and is on no remote branch, so the citation would point
at nothing once the stack lands. The owner's review of #99-#109 found it.
bf8ba41 and 86b777b (the head of PR #100, vault-error-tests-d2, the D2
replay of vault-error-tests-r4) have the same tree, 36187b49
(`git rev-parse bf8ba41^{tree} 86b777b^{tree}` prints it twice). So
the run was a run on 86b777b's tree, and the row now says "on tree
36187b49, the tree of 86b777b (PR #100's head)". Why not just "at
86b777b": it was committed on 17 Oct, after the run; and main
squash-merges (1045cd9 has one parent), so after PR #100 lands 86b777b
is reachable only as GitHub's refs/pull/100/head. The tree id stays
checkable from main: squashed bottom first, main's commit for PR #100
has that tree. "PR" keeps "#100" from reading as finding 100, the
row above. These points came from two code-review passes.
Sibling search: every 7-12 hex token in tracked .md/.py/.sh/.mjs/.rs/
.toml files at autopilot/integration 3e9f58d that names a local commit
(51 tokens) was checked with git merge-base --is-ancestor against
integration. bf8ba41 was the only one that is not an ancestor.
Docs only. tools/claims.py: claims OK (26 files).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Warn about a Floor guardian set short of the margin (finding 107)
The guardian checks the Vault protocol's guardian set against finding
107's margin (main.mjs checkGuardianSet), but never a Floor config's.
A Floor set registered before the rule, say 2-of-3, keeps working and
keeps AUD-2's exposure: one lost key and one rogue guardian lock its
governance for good. The guardian said nothing, and docs/07 §6 said so
("not yet a Floor set").
The rule moves to policy.mjs as guardianSetWarning, a pure function:
the same text as before, a warn line that alerts on mainnet only. In
main.mjs, checkGuardianSet now remembers the set shape per set
("protocol", or the protected program), and checkFloor calls it for
each watched Floor config it reads. floorAlerts is unchanged. The
margin is not a readiness code there: "floor not ready" alerts on
every cluster, and only the owner's new set clears a margin problem, so
"floor ready" would never be logged again.
Evidence (tasks/floor-guardian-margin/r2/):
- node --test cloud/test/*.test.mjs: 180/180 pass (one new test).
- Mutants of guardianSetWarning, each run on policy.test.mjs and
reverted; each fails the new test: N1 never warn, N2 threshold fixed
at 2, N3 notify always, N4 notify never, N5 GUARDIAN_MARGIN = 1.
- tools/claims.py: OK, client 180 (docs/09:40 updated).
- Not tested: the main.mjs wiring (the per-set Map and the checkFloor
call). No test runs checkFloor; deleting the call leaves every test
green.
Sibling search: guardianMarginProblem's callers are vault-check.mjs (an
info line) and guardianSetWarning. The status snapshot
(snapshot.mjs:33) prints live guardians and the quorum, not the margin;
left as it is.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Judge each guardian set's margin on its own, and say when it is fixed
The parent's per-set memory (which set shapes were already reported)
lived in main.mjs, so no test could fail on it: dropping the
once-per-shape check, or keying every set the same, left every test
green. The warning did not say which set it was about (only a program
tag on Floor lines), and it gave the same text for a set ejected down
to its threshold, or registered n-of-n before finding 96. There one
rogue guardian alone locks governance; a lost key alone does not, since
dead guardians cannot veto the owner's new set (docs/16 §4.4), but it
leaves no quorum until that set executes. Nothing was said when the owner's
new set fixed the problem. And a delivered alert did not say which Floor
program it was about: the program went only into the line's `program`
field, which alertFor uses in its repeat key but does not show.
Moving the program name into the text is a trap: alertFor keys repeats
on the first 60 characters of the text, and "the Floor guardians of
<44-character id>" pushes the shape past them. A 2-of-3 set ejected to
2-of-2 within ten minutes would then give the same key, and the worse
warning would not alert. So the program goes after the shape: "the
Floor guardians are 2-of-3 for <program>: ...". The `program` field
stays for the repeat key.
guardianSetWarning becomes guardianSetCheck(shapes, which, set,
mainnet) in policy.mjs. It keeps each set's last shape and whether it
was short in the Map, keyed by "protocol" or the protected program, and
returns the line to log or null. The warning names the set and has its
own text for n-of-n. When a set that was short gets the margin, it
returns one line saying so, which alerts on mainnet like the warning,
so whoever was paged hears that it is fixed. main.mjs only logs what it
returns. docs/07 §6 says so. The advice also had a hole: a set
registered as 7-of-8 was told to "run at least 7-of-9", and no set can
have 9 keys (core MAX_GUARDIANS is 8). When threshold + 2 is over 8,
the advice is now to lower the threshold to at most 6-of-8. That needs
MAX_GUARDIANS in JS, so lib/solana.mjs gets it next to
GUARDIAN_MARGIN; the next commit ties both to the core.
Evidence (tasks/floor-guardian-margin/):
- node --test cloud/test/*.test.mjs: 180/180 pass (the test is
extended, not added).
- 19 mutants, each run on policy.test.mjs and reverted (r6/mutants.py,
r6/mutants.log, r6/N5-margin1.log). Each fails the extended test, and
only it: never warn, threshold fixed at 2, notify always, notify
never, GUARDIAN_MARGIN = 1, no once-per-shape, one Map key for every
set, the n-of-n text for threshold + 1, no fixed line, a fixed line
for a set that was never short, the fixed line never alerting, no
program tag, every set named as the protocol's, the short flag not
stored, the program before the shape, the program left out of the
text, `need < 8` for `need <= 8` (6-of-7), the 8-key limit ignored
(7-of-8), 7-of-8 as the lowered advice.
- The mainnet alerts are checked on their own in a scratch copy of the
test without the earlier text assertions (r5/C1-C3-alone.log): with
the program left out of the text, the alert's detail check fails;
with a long name pushing the shape past character 60, "2-of-2 alerts"
fails.
- A's review of the previous version (branch
auto/B/floor-guardian-margin-r7, kept) found two survivors there: X3,
the shape keyed on the count only, and X5, the advice
threshold-of-(n+1). It also found that the n-of-n text said a lost key
alone locks governance. The test now has a threshold-only fix (7-of-8
-> 6-of-8, the advice's own step) and pins the n-of-n text and advice.
r9/mutants.log: X3, X5, the r7 n-of-n text and the text without its
lost-key clause are all KILLED. r9/mutants-r7.log: X3 and X5 SURVIVED
on the r7 code and test. The 18 mutants of r6/mutants.py, re-run on
this code: all KILLED (r9/r6-mutants-rerun.log). Two code-review
passes on the fix (kept as -r8 and -r9) added the lost-key clause: a
lost key alone leaves no quorum (it never approves, and core `quorum`
needs threshold approvals) until the owner's new set executes. A third
pass (-r10) fixed the comments' wording only.
- tools/claims.py: OK, client 180.
- Still not tested: the checkFloor call in main.mjs (no test runs
checkFloor).
Not changed (code review): the "fixed" line needs the short shape to
have been seen since the guardian started (the Map is in memory, as the
old per-process shape was; docs/07 says so). A protocol that abdicated
SetGuardians can't take a new set, so the advice to run 2-of-4 can't be
followed there; REPORT lists that case for the owner. The fault text
assumes a margin of 2, the owner's decision (16 §4.4).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Tie the JS guardian margin and set size limit to the core constants
cloud/lib/solana.mjs keeps its own GUARDIAN_MARGIN = 2, which the
guardian and the deploy tooling use to judge a guardian set, and now
MAX_GUARDIANS = 8, which the guardian's advice uses. Their comments say
they are core governance.rs's constants, but nothing checked that. Had
the programs' margin moved to 3, every JS test would have stayed green
while the guardian passed sets the programs refuse.
deploy.test.mjs now reads governance.rs and requires exactly one
`pub const GUARDIAN_MARGIN: u8 = N;` and one `pub const MAX_GUARDIANS:
usize = N;`, each N equal to the JS constant. It reads the source text,
like the build-script test above it. If a declaration changes form, the
test fails rather than passing.
Evidence (tasks/floor-guardian-margin/r6/tie-mutants.log):
- node --test cloud/test/*.test.mjs: 181/181 pass (one new test).
- Mutants of governance.rs, each reverted with its mtime kept (else the
limiter differential asks for a worker rebuild): GUARDIAN_MARGIN = 3,
MAX_GUARDIANS = 9, and a trailing comment on MAX_GUARDIANS. Each fails
the new test, naming the constant. deploy.test.mjs is green again.
- tools/claims.py: OK, client 181 (docs/09:40 updated).
Sibling search: the other JS copies of core rules (the limiter port,
the account layouts) are checked by the differential and by decoding
real accounts. MIN_GUARDIAN_THRESHOLD has no JS copy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Reach the 7-day extension total in the wide differential
differential_wide.rs drew extension seconds only from {0, 1, 3600,
86400, 86401} on a random ticket and lumped every refusal into one
counter. A ticket never got near the 7-day total, no negative value was
sent, and a refusal was not checked by its code. So the default run
could not tell the total bound, a negative extension or the error code
from a bug: only the unit tests at vault.rs:3382-3401 pinned them.
Now one extension draw in three pushes the chosen ticket by up to 24h
a call to exactly the 604,800 s total, then asks for 1 s more, so the
edge itself is hit on every push, not only when earlier draws line up.
The other draws are half legal sizes (1, 3600, 86400) and half out of
range for one call (i64::MIN, -86400, -1, 0, 86401, i64::MAX). Each
refusal must fail with EXTENSION_TOO_LONG. The counters are split into
extend (a random legal size), extend_push, extend_refused_call and
extend_refused_total, all required >= 10, so pushes cannot stand in for
the random sizes.
The model reads both bounds from bulkhead-core (consts.rs:37-38), as
the module says it is built, rather than restating 86,400 and 604,800.
docs/09 §7's row says so, and the 16 Oct "Vault refusals deleted" row,
which says the default run never extended a ticket that far, now says
it does since this change (its differential_wide.rs:406 cite had drifted).
Default run (24 seeds x 250 steps): extend 16, extend_push 126,
extend_refused_call 16, extend_refused_total 30 (was extend 40,
extend_refused 27); every other required counter still >= 10
(loosen_executed 12, as before).
Evidence: mutants of brakes.rs extend_ticket, each built and run against
the old and the new differential_wide only:
T1 drop the total check old SURVIVED new KILLED
T2 total > to >= old SURVIVED new KILLED
T3 drop secs <= 0 old KILLED new KILLED
T4 secs <= 0 to < 0 old KILLED new KILLED
T5 total ignores this call's secs old SURVIVED new KILLED
T6 extended_s += secs / 2 old SURVIVED new KILLED
T7 refusal returns WrongTicket old SURVIVED new KILLED
T8 secs <= 0 to == 0 (negatives) old SURVIVED new KILLED
T9 total > to > total + 1 old SURVIVED new KILLED
(T9: a draft that pushed by 24h until refused killed it only through
chance 1 s and 3600 s draws on the same ticket; the push to exactly
604,800 then 1 s more reaches the edge every time.)
sBPF sha256 after the run unchanged (Floor 8ca8a608, Harbor 543be60b,
Vault 7086148b): tests only.
Sibling search: the other refusal counters in this file (reset_refused,
lifeboat_refused_*, loosen_*) are each split by cause already.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check settle and lifeboat refusals by code in the wide differential
The same gap as the extension refusal, found in the sibling search for
the previous commit: a refused settle and the three lifeboat-move
refusals asserted only r.is_err(). Any refusal passed, so the order of
the checks and the program's own codes went unchecked by the
differential. Settle's comment (custody.rs:520-521) says it returns its
own error, not the token program's, when the vault is short; nothing in
this file held it to that.
Now a settle refused while frozen must fail with FROZEN, and one that is
only short with INSUFFICIENT_BALANCE (frozen is checked first). A
lifeboat move inside the delay after the seal must fail with NOT_SEALED,
one while frozen with FROZEN, and one beyond the balance with the token
program's InsufficientFunds (1). The two settle refusals get their own
counters: settle_refused_frozen is required >= 10 like the rest;
settle_refused_short needs lifeboat moves to drain a due ticket's
tokens first and occurs 7 times in the default run, so it is required
>= 1.
Evidence: mutants of programs/vault/src, each built and run against the
previous head's differential_wide and this one, nothing else:
S1 settle: drop the InsufficientBalance check old SURVIVED new KILLED
S2 settle: balance check before the freeze old SURVIVED new KILLED
L1 require_sealed returns Frozen old SURVIVED new KILLED
L2 move_in's freeze returns NotSealed old SURVIVED new KILLED
The token-program code (1) on a move beyond the balance is pinned but
not shown red by a mutant. sBPF sha256 unchanged (Floor 8ca8a608,
Harbor 543be60b, Vault 7086148b): tests only.
Deep run of both commits together (BULKHEAD_PROGRAM_SEEDS=600
BULKHEAD_PROGRAM_STEPS=500, 85 s): 300,000 steps, all identical;
extend 876, extend_push 6,462, extend_refused_call 1,175,
extend_refused_total 1,453, settle 2,024, settle_refused_frozen 921,
settle_refused_short 566, lifeboat refusals 3,854 + 5,768 + 1,171. docs/09 §7's
row gets that run as a 16 Oct result next to the 5 Oct one, which
predates these checks.
Sibling search: in differential_wide.rs, reset (TICKETS_PENDING) and
execute change already asserted codes, and a release or quote refused
for want of tokens asserts INSUFFICIENT_BALANCE. A release or quote the
limiter refuses (frozen, lane closed, queue full) is still compared only
as a refusal: mapping the core's refusal to the program's code is left
for its own change. Lifeboat returns only send legal amounts. docs/09's
row now says which refusals are checked by code. differential.rs:327
has the same is_err settle check; left as it is, since the wide run now pins both
settle codes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check limiter refusals by code in the wide differential
A release or quote that the limiter refused (lane closed, queue full)
was compared with the model only as Err. Any refusal code passed: a
program that swapped LaneClosed and QueueFull in custody.rs's
limiter_error, or checked the backed part against the balance before
the limiter and so answered InsufficientBalance for a full queue,
still matched the model. About 670 such refusals happen in a default
run.
The model now maps core's LimiterError to the program's code with
verr::limiter in the test harness (all four variants, as custody.rs's
limiter_error does), and every refused quote and release must fail
with the model's code. Lane-closed and queue-full refusals have their
own counters, each required at least 10 times; a refusal code with no
counter panics rather than being counted as want of tokens. docs/09
row 362 says so and records the deep run.
Evidence (autopilot tasks/differential-limiter-codes/):
- mutants.log: C1 (LaneClosed and QueueFull swapped in
limiter_error), C2 (QueueFull -> InsufficientBalance), C3
(LaneClosed -> ZeroAmount) and C4 (backed > balance checked before
the limiter, same line) each SURVIVED the old test and are KILLED
by the new one. Hashes back to 8ca8a608/543be60b/7086148b after.
- c4-full-suite.log: under C4 the whole program suite (Vault 113,
Floor 50, the fixed differential) passes; only this test fails.
C1-C3 are also caught by vault.rs's code tests (:434, :3428).
- Default run: refused_lane_closed 254, refused_queue_full 414,
insufficient 118.
- deep-600x500.log: 300,000 steps, 15,561 lane closed, 18,853 queue
full, 6,345 want of tokens, all identical.
Sibling search: every other Err arm in differential_wide.rs already
asserts a code (extend, settle, reset, lifeboat in, execute change).
Not here: lifeboat returns are never refused (backlog), and lifeboat
move_in beyond the balance fails with the token program's code
(backlog, a program change).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check release refusals by code in the fixed differential
differential.rs reduced every refused release to Err(()) on both
sides, so a program that refused for the wrong reason still matched
the model. The same gap the previous commit closed in the wide
differential (found by its code review).
The model's refusal now carries the program's code: verr::limiter for
a limiter refusal, INSUFFICIENT_BALANCE when the vault holds less
than it would pay. A refused release must fail with that code, and
the assert prints the program's error. Refusals for a full queue get
their own counter, required at least 10 times like the others. Want of
tokens and a closed lane are counted and printed but not required:
the wide differential requires both. A refusal with any other code
panics, as in the wide differential, so the counts always add up.
docs/09 row 361 says so; its undated figure (4,843 releases) no longer
matched a run, so it is replaced by a dated one.
Evidence (autopilot tasks/differential-limiter-codes/):
- mutants-narrow.log (mutants.sh <wt> differential): C1 (LaneClosed
and QueueFull swapped), C2 (QueueFull -> InsufficientBalance) and C4
(backed > balance checked before the limiter) SURVIVED the old test
and are KILLED by the new one. C3 (LaneClosed -> ZeroAmount)
survives both: no release in this test's runs is refused for a
closed lane. The wide differential kills it, and docs/09 says so.
Hashes back to 8ca8a608/543be60b/7086148b after.
- r2/default-run.log: 1,000 steps, 82 releases refused, all for a
full queue (want of tokens 0, lane closed 0).
- r2/narrow-24…
shuv-amp
added a commit
that referenced
this pull request
Oct 18, 2026
* Run the tests on 2-of-4 guardian sets; test recovery from two faults
Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set
with fewer than threshold + 2 keys. Every test fixture and most tests'
own sets were 2-of-3. This moves them to 2-of-4, which today's rule
also accepts, so this commit changes no program: Floor, Vault and Harbor
hash 508e5a99 / b07655d9 / 543be60b, as before.
- Fixtures: Protected (Floor) and Market (Vault, through Harbor) now
hold 4 guardians. Each test that built its own 3-key set gets a 4-key
set with the same threshold and the same assertion.
- Two tests changed meaning with a 4th key and were fixed, not loosened:
a_quorum_ejects_a_rogue_guardian_who_vetoes_everything (Floor) and
a_quorum_ejects_a_guardian_who_freezes_forever (Vault). With 4 keys,
the last eject was refused only because the target's own vote is
masked. Both now eject a second guardian (a set an ejection shrank to
threshold + 1 still ejects) and then show the 2-of-2 refusal.
- Account sweeps: an eject's signers may now be any two of the three
others, so each of those two slots is free for the third, never for
the target or the other signer again. Vault 5,365 and Floor 1,710
substitutions, refused (docs/09, dated 16 Oct). Listing all three
others at each slot would have skipped the duplicate: with a mutant
that counts a guardian listed twice as two (governance.rs:270), the
sweeps listing all three pass, and these fail at "eject: account 2
replaced by key ..." (Vault and Floor). No other program test kills
that mutant (tasks/aud2-guardian-margin-impl/dup-mutant-*.log).
- floor_differential: the guardian pool is 6 keys, 4 registered; the
proposed sets are 2 to 6 keys.
New tests (pass on both rules; they show what the margin buys):
- a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_set (Floor): g0's
key lost, g1 rogue stops the program; the lost key does not help a
quorum; the honest pair ejects g1 and restores; neither can veto the
owner's refill, which executes after 72h.
- two_rogues_in_turn_are_both_ejected_from_a_two_of_four_set (Floor):
the second rogue vetoes the refill and is ejected from the 2-of-3.
- a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_vault: the same in
the Vault (freeze, eject, unfreeze, refill).
Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest
and quote sweep): the counts and the sweep figures here are from runs
there (tasks/aud2-guardian-margin-impl/r4/). The E2E run, the mutants
and the new-rule run below were made before, on e8640a7; this commit's
programs and core are the same as there but for one comment line in
views.rs (line-neutral, same hashes).
Evidence:
- E2E=1 scripts/check-programs.sh: PROGRAM CHECKS PASSED (Floor 49,
Vault 98; S1 and E2E passed; the resumed setup found all 14 steps;
vault check, Floor demo and deploy dry run passed), with the next
commit's deploy rehearsal in the tree.
- tools/claims.py: OK (floor 49, vault 104, programs 153).
- Under the new rule (core from two commits on, programs rebuilt), only
the two tests that state the old rule fail: an_n_of_n_guardian_set_is_
refused (accepts 3-of-4) and the Floor differential's model.
- Mutants (tasks/coverage-map/mutants.sh), each red, then restored:
AV1 Floor live_guardian ignores liveness: KILLED by a_lost_key_and_a_
rogue_do_not_lock_a_two_of_four_set (+2 others).
AV2 the Vault's: KILLED by ..._two_of_four_vault (+2 others).
AQ1 Floor restore quorum ignores liveness: KILLED by ..._two_of_four_set
and the differential only.
AQ2 the Vault's unfreeze quorum: KILLED by ..._two_of_four_vault and
a_silent_guardian_does_not_count_toward_a_quorum.
AE1 can_eject refuses at count == threshold + 1 (AUD-2 option (a)):
KILLED by two_rogues_in_turn_..., and the two eject tests above.
Hashes after: 508e5a99 / 543be60b / b07655d9.
Split on 16 Oct from ec0cacc at the owner's review (D2): the deploy
rehearsal's move to 4 guardians is now the next commit. programs/ and
programs/tests/ here are byte-identical to ec0cacc, so its runs below
hold for this commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Rehearse deploys with four guardians, and read only the keys there are
Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set
with fewer than threshold + 2 keys, so the deploy rehearsal must
register 2-of-4 sets. Split from ec0cacc at the owner's review (D2);
the text is that commit's, unchanged:
Deploy rehearsal (cloud/deploy market.mjs, vault-check.mjs,
floor-demo.mjs): 4 guardians (keys/guardian3), so the dry run keeps
working once the rule lands; one more funding step (14, was 13).
The devnet market has 3 guardian files. A first version of this change
loaded guardian3 everywhere: vault-check.mjs failed there with ENOENT,
and heartbeat silently created a key nobody holds as a guardian. Now
market.mjs guardianKeys(dir, create) reads guardian0, 1, ... and
throws on a gap (never fills it with a new key). Setup creates keys up
to 4 only while the protocol is not registered, so resuming an old
market adds no key the chain lacks; a new deploy-dryrun step resumes
the dry-run market with guardian3 moved aside and fails if setup makes
it again (with setup always creating: "DRY RUN FAILED: resuming a
registered market made keys/guardian3.json", r3-dryrun-J7.log).
renew-guardians tops up to 4 and logs each new key (unfunded); a
2-of-3 renewal would be refused once the rule lands. keysIn reads the
guardians only when a command uses them, so caps and set-recovery
never need them; heartbeat says a key is not in the set rather than
dead. One THRESHOLD (2) for the register and renewal payloads and
MIN_GUARDIANS. vault-check.mjs
compares the chain's set with the first files, and liveness with the
chain's count. An extra file (a renewal not executed yet) and a set
without the margin are info lines, not failures, since such a set
keeps working. GUARDIAN_MARGIN and guardianMarginProblem live in
cloud/lib/solana.mjs, used by these and by the guardian bot's existing
warning (cloud/guardian/main.mjs), and vault-check uses THRESHOLD. The e2e scripts fund guardian3 too. Three
tests in cloud/test/deploy.test.mjs (client tests 176 -> 179). JS
mutants (js-mutants-r3.log), each red then restored: always create,
never create, no gap check, margin 1, `>`, GUARDIAN_MARGIN = 1.
Setup always creating survives these tests and is killed by the dry
run step above.
cloud/, scripts/deploy-dryrun.sh and docs/09's client-test row are
byte-identical to ec0cacc (git diff empty), so its runs hold for them:
the E2E check-programs run with the dry run (r4), and the JS mutants.
tools/claims.py: OK (client 179).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Refuse a guardian set without two keys beyond its threshold (107)
AUD-2, decided by the owner on 16 Oct (option (e) of 16 §4.4). A set
stays live while its honest live guardians reach the threshold: they
eject every rogue in turn, and dead guardians cannot veto the owner's
new set. Finding 96 asked for one key beyond the threshold, so a 2-of-3
or 3-of-4 set survived one fault but not a lost key plus a rogue: the
honest guardian left is no quorum, nobody can be ejected, and the rogue
keeps Floor stopped (or the Vault frozen) and vetoes every new set.
Fix, in the core so both programs inherit it through GuardianSet::parse
(registration, and every SetGuardians when proposed and when executed):
GUARDIAN_MARGIN = 2
guardian_set_ok := count <= 8 && t >= 2 && count >= t + 2
(saturating_add: t is a payload byte). A stored set is not checked
again, so existing 2-of-3 sets keep working, and ejection is unchanged:
a shrunk set still ejects. A 2- or 3-key set is BadThreshold, not
BadCount; the count rule is only the bitmap size, and the error order
of finding 103 is kept.
This changes both programs, by intent:
Floor 508e5a99 -> 8ca8a608, Vault b07655d9 -> 7086148b,
Harbor 543be60b (unchanged). Darwin-arm64 builds.
Compute (scratch LiteSVM test, tasks/aud2-guardian-margin-impl/cu.md):
-3 CU on every path that parses a set (Floor propose 2035 -> 2032,
execute 1682 -> 1679; Vault register 9592 -> 9589, SetGuardians 2979 ->
2976; Floor register varies with its PDA bump, residue also -3).
Proof and tests:
- Kani: accepted_sets_can_eject_anyone restated (ejection reachable
exactly when count > t, so option (a) cannot slip in; accepted sets
have >= 4 keys and a margin of at least 2, and 2 is accepted). New:
an_honest_live_quorum_ejects_any_rogue (any slots, anyone else alive)
and accepted_sets_survive_any_two_faults (any two faulty keys, lost
or rogue, leave a live honest quorum, which ejects each rogue, and the
rogues alone eject no honest guardian). That holds only short of a
rogue quorum: two rogues in a 2-of-n set are one (out of scope, 04
T7). With the assumption rogue < t dropped, the harness fails at
`!can_eject(rogue, ..)` (kani-no-rogue-assume.log), so the docs say
"short of a rogue quorum", not "any two faults".
19/19 verified, 240 s (cargo kani -j 2, as CI); again after the
restatement, 235 s summed (r2-prove.log).
- Kani mutants (kani-mutants.sh): margin 1, margin 3, `>` for `>=`,
t >= 1, count <= 9, option (a), and an eject one vote short (only
the two-fault harness kills that one): each KILLED by a harness
(kani-mutants-r2.log). The eject
ignoring liveness is the eject harness's case, not these.
- Program mutants (coverage-map/mutants.sh, rebuilt each time):
margin 1: KILLED by a_guardian_set_without_two_spare_keys_is_refused
(Floor and Vault), registration_rejects_weak_settings and the Floor
differential. Margin 3 and `>`: KILLED by most of the suite (the
fixtures are 2-of-4). t >= 1: first only the Floor differential; the
1-of-2 registration case was now refused by the margin too, so it is a
1-of-4 here and a Vault 1-of-4 case was added: then KILLED by both.
count <= 9: SURVIVES the program tests, equivalent there (parse refuses
more than 8 keys first), KILLED by Kani. Option (a): KILLED by the
eject tests two commits back (the tests on 2-of-4 sets).
- Tests: the Floor and Vault refusal tests above cover proposal and
execution (a stored 2-of-4 proposal whose threshold turns 3 is refused
when executed, as the devnet renewals proposed before this rule will
be); registration_rejects_weak_settings gains 2-of-3 and 3-of-4.
an_n_of_n_guardian_set_is_refused (Vault) is renamed and accepted a
3-of-4: it now refuses it and accepts 3-of-5.
- Model: the Python rule, the Floor differential's model, and the
guardian_set fuzz target state it again; the GP generator now also
picks count - 2. With the Python rule put back to the old one, seed 11
(20,000 cases) reports 5 GP mismatches; restored, seeds 11-13 identical.
Fuzz seeds two_of_three_refused, two_of_four, six_of_eight, and
seven_of_eight renamed seven_of_eight_refused (byte 323, the
threshold, is 7: now refused). With margin 1 in the core the target
crashes within 60 s; restored, 222,615,288 runs in 601 s, no crash.
- scripts/check.sh: ALL CHECKS PASSED, and E2E=1 check-programs.sh:
PROGRAM CHECKS PASSED, both at the branch tip, whose code is this
commit's (the next commit is docs only; tasks/aud2-guardian-margin-
impl/r4/check.log, r4/check-programs-e2e.log). claims: kani 19,
floor 50, vault 104, programs 154, client 179.
Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest
and quote sweep). The core and the program sources are the same as
before the rebase but for one comment line in views.rs (line-neutral),
so the Kani, fuzz, differential and mutant runs above were made on the
pre-rebase commit; the hashes, check.sh, check-programs and the counts
are from r4 runs (tasks/aud2-guardian-margin-impl/r4/).
The finding 96 row said the restated harness proves "exactly"; it now
says what it proved then and proves now, and gives the Vault test's new
name. The finding 107 row waits for rows 105 and 106 to land (09 §4 is
numbered 1..n); its text is ready. The rule's prose is the next commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* State the guardian margin in the docs, with its costs (AUD-2)
The previous commit made both programs refuse a new guardian set
without two keys beyond its threshold (finding 107). The docs that
state the rule now say so:
- README: guardians 4-8 keys; the G5 row and the "who the guardians
are" paragraph give the reason (any two faults leave a quorum, unless
the rogues are one themselves: two in a 2-of-n set, 04 T7).
- 05 register_protocol: 2 <= threshold <= count - 2, also for every
SetGuardians, proposed and executed. 05 abdicate: abdicating
SetGuardians fixes the set, so ejections shrink it for good.
- 06 state table, 00 D22, FAQ: the same rule.
- 07 §2: 4-8 guardians, and don't abdicate SetGuardians. 07 §6: the
programs enforce the margin; sets registered before keep their
exposure until replaced; an ejection uses up a spare key until the
owner's new set executes; a lost key counts as live until its
interval passes.
- 16: the AUD-2 row is decided; §4.4 gives the decision, why the
margin is the number of faults a set survives (short of a rogue
quorum), the options rejected
and why (the honest-owner and stolen-owner cases look the same
on-chain), what it costs, what it leaves (a rogue quorum; a stolen
owner key with every guardian dead; a rogue's veto of the recovery
key's rotation, checked: Floor's veto has no recovery exception, as
a_vetoed_recovery_proposal_does_not_hold_the_owner_s_slot shows), and
Lido-style lapsing vetoes as a future option only. §5: "independent
guardians, 3-of-5 or more".
- 10: the guardian set's minimum size is no longer an open decision.
SECURITY.md and 08 describe what is on devnet, which still runs the
old rule, and are left as they are.
claims: OK. Docs only; no program file changed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Say the weekly CI run skips mutants, as its condition does
ci.yml's header and scripts/ci-changes.sh said the weekly run does every
job, and 09 §8 gave `mutants` as running on any change to the core. The
`mutants` job's `if` starts with `github.event_name == 'pull_request'`,
so the Monday schedule (and a manual dispatch) never runs it: it mutates
a pull request's diff and has none to mutate. Its trigger list in 09 §8
also left out scripts/mutants.sh and .cargo/mutants.toml, which
ci-changes.sh (line 22) selects it for.
Comments and docs only; nothing runs differently.
Sibling search: `git grep` for "weekly run does", "every job ... weekly"
and "mutants ... weekly" across md/yml/sh/mjs/py. The other hits are
09 §8's "at full depth every Monday" (its table now says which job is
not weekly) and the dispatch input's "every job at weekly depth", which
is the same set as the weekly run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check that the mutant exclusions still match; repin murmur3 (finding 105)
.cargo/mutants.toml excludes the core's 22 equivalent mutants, and the
docs say so. Since 9 Oct it excluded 19. The murmur3 tail's pattern is
pinned by line, `floor.rs:(235|238|241)`, because the body's `h ^= k`
has the same column and function and is not equivalent. 1dc7861 (#52)
moved those lines (now 327/330/333), and nothing checked the patterns.
Failing closed, the three would have failed CI's `mutants` job on a
pull request that changed those lines. Failing open, a pattern that
starts matching a non-equivalent mutant hides it from every run.
Root cause: nothing checked that a pattern still names its mutants.
Fix:
- each pattern ends with the number of mutants it excludes (`# 3`), and
the murmur3 pattern names 327/330/333;
- tools/mutant_exclusions.py fails unless each pattern matches exactly
its number in `cargo mutants --list --no-config`, and cargo-mutants
(the list with the config) excludes exactly the mutants the patterns
match, so a pattern this parser can't see, or reads differently, fails
too. No two patterns may match the same mutant, since the docs' count
is the counts' sum, and the kept list must be part of the full one.
It reads only the exclude_re array's body, needs no tomllib
(Python 3.9), and names colour codes in the list as the problem;
- scripts/mutants.sh makes both lists (--colors never: CI sets
CARGO_TERM_COLOR=always) and runs it first, except when the caller
passes --config or --no-config (as whole arguments). CI's `mutants` job runs that script;
ci-changes.sh now also selects the job for the new tool;
- scripts/check.sh runs its --self-test on planted configs;
- tools/claims.py ties the docs' "22 equivalents" (README, 09 §6 and the
summary row) to the sum of the counts, with the same parser. It finds
that parser from its own directory, so `python3 -I` runs it too, and
check.sh runs its self-test that way. 00's
"18 equivalents" (26 Sep log entry) and run 9's "exactly the 22
equivalents below" (6 Oct) are kept as dated records.
Docs: 09 §4 finding 105, §6 run 10 and the "In CI" note, §8's mutants
row; README and 09's summary row 669 → 797 mutants (run 10); README and
12's post-submission range 96–105.
Evidence (first on 1f2ad48 + the parent, logs in the autopilot's
tasks/mutants-exclusions-r2/; re-run on 58697c6 + the parent,
tasks/mutants-exclusions-r3/):
- `cargo mutants --list`: 797 with --no-config, 778 with the old config
(19 excluded), 775 with this one. The 22 excluded are §6's list, the
murmur3 ones at 327/330/333.
- scripts/mutants.sh --list: exit 0, "22 of 797 mutants excluded"; with
CARGO_TERM_COLOR=always also exit 0. Old config: exit 1, every pattern
"no '# N' count". Planted on the real lists: the old lines with a
count (matches 0, says 3) and the body XOR added (matches 4, says 3)
both exit 1; in both the set comparison alone passes. /usr/bin/python3
(3.9.6): self-test OK and the real check passes.
- The self-test pins each problem by its message, not a count. Mutants
of the tool (tool-mutants.log), each red: M1 no per-pattern check,
M2 no set comparison, M3 no colour check, M4 no empty-array check,
M5 a missing count accepted, M6 a missing count read as 0, M7 the
parser reading past the array. M3 and M5 survived a first version
that checked only the number of problems.
After A's review (16 Oct): the set comparison was pinned only by its
size, so `len(excluded) != len(matched)` passed the self-test. A
case with as many excluded as matched but not the same ones now
fails it (15 cases; red with that mutant, green without), and the
message says how many mutants are in one set only.
From the code review after the rebase: colour codes only in the kept
list were reported as a kept list made with other options; the
colour check now reads both lists (16 cases; dropping the kept half
fails the new case).
On 58697c6, with the overlap and kept-list checks: M8 no overlap
check and M9 no kept-list check, red too, as M1–M7 still are. A
duplicate of the can_eject pattern planted in the real config:
scripts/mutants.sh --list exits 1, "the counts add up to 23, the
patterns match 22 mutants" (planted-overlap.log). Without the
sys.path line, `python3 -I tools/claims.py` fails with
ModuleNotFoundError; with it, both claims runs pass.
- claims.py (claims-mutants.log): one count 1 → 2 fails four statements
(README 188; 09 33, 333, 335; kept-run9.log), not run 9's 331, which
failed too before it was kept; a dropped count fails with the
tool named; deleting any of the three rules fails a self-test case;
deleting the KEPT entry fails 00:103.
- The 22 excluded mutants run alone (--no-config, --re per pattern,
2 min): 22 missed, exactly the excluded list; --re also picked
ema.rs:70 (a field deletion, no pattern matches it), caught.
- Run 10 (f1f05ba, old config, -j4, 46 min; tasks/mutants-run10/):
778 tested, 712 caught, 5 timeouts, 58 unviable, 3 missed, exactly
the murmur3 tail. The core's source is unchanged since (`git diff
--stat f1f05ba HEAD -- crates/bulkhead-core/src` is empty; only
tests/units.rs gained finding 102's tests).
- No program or core source changed.
Rebased on 16 Oct onto cede705 (findings 102 and 100, the attest
sweep and AUD-2). AUD-2 changed the core's source (governance.rs), so
the line above saying it is unchanged since run 10 held on 58697c6, not
here. On cede705: scripts/mutants.sh's list check passes, "22 of 795
mutants excluded, each pattern matching its count"; the murmur3 lines
are still 327/330/333. The 797 in README and 09 is run 10's figure:
README now dates it ("in the last full run, 15 Oct") and finding
105's row says the core has changed since (finding 107).
The setup line in scripts/mutants.sh now pins cargo-mutants 27.1.0,
the version ci.yml installs: each count is the number of mutants that
version lists, so another release can break the check (code review,
16 Oct; `git grep 'install.*cargo-mutants'` finds no other unpinned
install).
The check's report goes to stderr in scripts/mutants.sh, so
`scripts/mutants.sh --list` prints only cargo-mutants' names: before,
its first stdout line was the report; after, stdout is 773 lines
(795 - 22) and stderr the one report line. ci.yml's `mutants` comment
says the job checks the exclusions first.
Row 105 and 09 §6 gave the pin's reason as the body's XORs. The
finalizer's are at column 7 (`grep -n '\^=' floor.rs`, 337-342); the
non-equivalent XOR at column 11 is `h ^= k`, in the body (320) and after
the tail (335). The toml comment, the row and §6 now say that. 09 §8's
`check` row names the two tool self-tests check.sh runs.
README no longer calls run 10's 797 "injected bugs": cargo-mutants
listed 797 but tested 778, 19 being excluded (09 §6, run 10's row).
Sibling search: the other 14 patterns use `\d+` for the line, and each
matches the count now written after it. The murmur3 one is the only
line-pinned pattern.
Not done, by choice:
- Catching an edit that keeps a pattern's number but changes its
mutants (an excluded line itself edited, or a murmur3 body XOR moved
onto a pinned line). The toml and 09 §6 say so: such an edit needs the
mutant judged again by hand. Failing on any diff that touches an
excluded line would catch the first; it is a separate change.
- Moving murmur3's tail into its own function, so its pattern needn't
name lines. It shifts core lines, so it changes the program hashes.
Until then a line move above it fails the check, with a message
saying what to fix.
- A weekly full mutants run in CI. It took 46 min here at -j4, likely
past the job's 90 min on two CI cores: a minutes call for the owner.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that a ticket stays in its own protocol and vault
Two Vault checks that keep a ticket to its own protocol and vault had no
test. Deleting either one left every program test green
(tasks/error-codes/mutants.log, base cede705):
- brakes.rs:79, extend_ticket: tk.protocol != protocol. Without it, a live
guardian of any other protocol extends this protocol's tickets. Anyone
can register a protocol and name its guardians, so that guardian could
hold back every queued payout up to the total extension limit. The
sweep did not catch it: it has no second protocol's guardian to swap in.
- custody.rs:501, settle_ticket: tk.vault != vault. A USDC ticket settled
through the SOL vault (its tokens, its mint) got past the Vault's checks.
New test another_protocols_guardian_cannot_extend_a_ticket: a second
Market's live guardian extends this market's ticket -> WRONG_VAULT; the
same instruction from this market's guardian succeeds. The settle test
now settles the USDC ticket through the SOL vault and expects
WRONG_VAULT.
Evidence (tasks/error-codes/mutants.log, STATE=c1; full program suite
per mutant):
W02 brakes.rs:79 deleted -> KILLED by
another_protocols_guardian_cannot_extend_a_ticket ("expected
failure, got success": the stranger's extension went through)
W06 custody.rs:501 deleted -> KILLED by
a_ticket_settles_after_its_eta_to_its_stored_recipient_only (8049
InsufficientBalance from the empty SOL vault, not 8010)
Both were SURVIVED at base. No program source change: hashes
8ca8a608/543be60b/7086148b, as at cede705.
Sibling search: the other 12 WrongVault sites were deleted one at a time
in the same run. Eight are KILLED by the sweep (brakes.rs:39, lib.rs:320,
custody.rs:160, genesis.rs:491, views.rs:27, lifeboat.rs:151,
governance.rs:169/259). lib.rs:324 (vault PDA address), genesis.rs:523,
lifeboat.rs:71 and :247 survive; later commits on this branch deal with
them.
Counts: Vault 104 -> 105, programs 154 -> 155 (claims.py OK).
The test also reads the ticket's ETA: unchanged after the
refused extension, 3,600 s later after its own guardian's (code-review
fix: success alone did not show the extension happened).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that the seal names every vault and happens once
Two seal checks had no test; deleting either left every program test
green (tasks/error-codes/mutants.log, base cede705):
- genesis.rs:486, execute_seal: the pairs must number 3 * vault_count.
Without it, the seal checks only the vaults it is given. A vault left
out skips its claims-vs-balance check, so an uncovered import seals.
docs/05's seal row says every vault is passed exactly once.
the_seal_checks_each_vault_once covered a vault named twice in place of
another, not one left out.
- genesis.rs:431, propose_seal: refused once sealed (AlreadySealed).
Without it, a sealed protocol's admin re-proposes, which rewrites
seal_executable_s/expires_s and clears seal_vetoed on a sealed protocol.
the_seal_checks_each_vault_once now also leaves the uncovered USDC vault
out (expects VAULT_COUNT_MISMATCH) and, after the seal, proposes it again
(expects ALREADY_SEALED). No new test, so no count change.
Evidence (tasks/error-codes/mutants.log, STATE=c2, full program suite):
E05 genesis.rs:486 deleted -> KILLED by the_seal_checks_each_vault_once
("expected failure, got success": the seal went through without
the USDC vault and its 1,000 USDC uncovered import)
E04 genesis.rs:431 deleted -> KILLED by the_seal_checks_each_vault_once
("expected failure, got success")
Both were SURVIVED at base. Vault hash 7086148b unchanged (tests only).
Sibling search: the seal's other checks. :491 vault_tokens (KILLED by the
sweep), :519 lifeboat owner and :523 lifeboat mint/owner (SURVIVED; the
lifeboat is a PDA checked by expect_pda just above, which only this
program can create, as its own token account: see this branch's note in
docs/09).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that invalid limits and asset settings are refused at every entry
Three bound checks had no test; deleting any one left every program test
green (tasks/error-codes/mutants.log, base cede705):
- genesis.rs:171, create_vault: config_is_valid (bps <= 100%, windows
> 0). Without it, a vault is created with a 100.01%-an-hour burst.
- governance.rs:40, parse of a SetLimits change (at proposal and again
at execution, :254 and :478): the same rule for a proposed change.
- governance.rs:47, parse of an UpsertAsset change: valid_asset (haircut
<= 100%, liability >= 100%, the oracle bounds of finding 88). Only the
genesis upsert's copy (genesis.rs:296) was tested, by
oracle_settings_of_a_priced_asset_are_bounded.
New test invalid_limits_and_assets_are_refused_wherever_they_enter:
before the seal, create_vault with short_bps 10,001 -> INVALID_LIMITS;
after it, a SetLimits proposal with long_window_s 0 -> INVALID_LIMITS,
and an UpsertAsset proposal with a 10,001 bps haircut -> INVALID_ASSET.
Evidence (tasks/error-codes/mutants.log, STATE=c3, full program suite):
E06 genesis.rs:171 deleted -> KILLED ("expected failure, got success")
E07 governance.rs:40 deleted -> KILLED ("expected failure, got success")
E08 governance.rs:47 deleted -> KILLED ("expected failure, got success")
each by the new test only. All three were SURVIVED at base. Vault hash
7086148b unchanged (tests only).
Sibling search: every config_is_valid / valid_asset call in programs/
vault/src: genesis.rs:170 (create_vault), :295 (upsert_asset, KILLED by
the oracle test), governance.rs:39 and :46. All now killed.
Counts: Vault 105 -> 106, programs 155 -> 156 (claims.py OK).
The execution copy (:478) is not reached by any transaction: it
parses the bytes the proposal already checked, and the bounds read
nothing else. The test comment says so (code-review fix: it had
claimed execution was covered). oracle_settings_of_a_priced_asset_are_
bounded now asserts INVALID_ASSET for each of its four cases, not any
error (all four return it; E09 stays KILLED).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that an import into a ninth mint is refused
genesis.rs:410, import_backing: a backing ledger holds MAX_SLOTS (8)
mints, and an import into a ninth is refused with NoSlot. Deleting the
check left every program test green (tasks/error-codes/mutants.log, base
cede705). From the code, without it the import goes through: the vault's
imported total grows (:414), and the seal must cover that, but the
owner's ledger is saved unchanged, so the owner has no backed claim for
it.
New test an_import_into_a_ninth_mint_is_refused: nine vaults before the
seal; one owner gets an import into eight (ledger_of shows 8 positions),
then the ninth -> NO_SLOT.
Evidence (tasks/error-codes/mutants.log, STATE=c4, full program suite):
E14 genesis.rs:410 deleted -> KILLED by an_import_into_a_ninth_mint_is_
refused ("expected failure, got success"); SURVIVED at base.
Vault hash 7086148b unchanged (tests only).
Sibling search: the only other ledger credit is deposit's
(custody.rs:128). There a full ledger is by design: the deposit is taken
unbacked (Ledger::credit's doc, ledger.rs:187-189), and credit's result is
the backed flag, not an error.
Counts: Vault 106 -> 107, programs 156 -> 157 (claims.py OK).
import_ix now calls import_into_ix, which takes the vault, so the
new test shares the one import layout (code-review fix).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin six Vault refusals no test reached
Six reachable checks had no test that failed when the check was deleted
(tasks/error-codes/mutants.log, base cede705). Each now has an assertion
on its code, and each was seen to fail with the check deleted:
E02 genesis.rs:185 create_vault of an existing vault after the seal:
ALREADY_INITIALIZED (with the check gone: 8043 GenesisOnly, so the
"says so first" order in the comment above it was untested)
E12 governance.rs:426 veto of an executed change: CHANGE_EXECUTED
(gone: the veto succeeds and marks an executed change vetoed)
E13 governance.rs:517 cancel of an executed change: CHANGE_EXECUTED
(gone: the cancel succeeds)
E16 lifeboat.rs:256 execute_exit with no exit proposed:
NOTHING_TO_EXIT (gone: 8033 ChangeNotExecutable)
E17 custody.rs:84 a zero deposit: ZERO_AMOUNT (gone: it succeeds)
E18 custody.rs:272 a zero release: ZERO_AMOUNT (gone: it succeeds; a
zero release has no unbacked part, and the limiter, whose own zero
check custody.rs:242 maps, is called only when unbacked > 0, :361)
Where: E02 in after_the_seal_a_new_vault_starts_with_its_unbacked_lane_
closed, E12/E13 in an_unvetoed_loosening_executes_after_the_delay, E16 in
the_lifeboat_stays_shut_until_the_seal (after the move in, before an
exit is proposed), E17/E18 in the new a_zero_deposit_or_release_is_
refused.
Evidence (tasks/error-codes/mutants.log, STATE=c5, full program suite):
all six KILLED by the tests named above; all six SURVIVED at base. The
vault suite is green (108 passed). Vault hash 7086148b unchanged.
Counts: Vault 107 -> 108, programs 157 -> 158 (claims.py OK).
Helpers (code-review fixes): veto_change_ix beside cancel_change_ix;
create_vault_for_ix, create_vault_ix with a given mint and token
program, so E02's case names USDC's own accounts and creates no mint.
E02's comment now repeats the program's ("whatever the seal's state"):
create_vault checks the limits first (genesis.rs:170).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin four Vault checks another layer backstops
Four checks survived deletion (tasks/error-codes/mutants.log, base
cede705) because something later still refused the case, with its own
error, so no test could tell. Each now has a test that sees the Vault's
own code, and each was seen to fail with the check deleted:
E01 lib.rs:230, create_pda's guard: a second registration of an
existing protocol (register_protocol has no check of its own) ->
ALREADY_INITIALIZED. Gone: the system program refuses it, with no
custom code (custom_code 0).
W10 lifeboat.rs:71, move_in naming another mint or token program ->
WRONG_VAULT. Gone: the token program refuses (code 3, MintMismatch).
W12 lifeboat.rs:247, execute_exit naming another mint or token
program -> WRONG_VAULT. Gone: the same token-program error.
T01 token.rs:37, account_info on an account no token program owns: a
copy of a real USDC account's bytes under the system program. Gone:
the quote SUCCEEDS (it promises a payout), and the release fails
only in the runtime at the transfer ("instruction modified data of
an account it does not own"). The quote is asserted first so the
mutant is seen at the quote (STATE=c6b).
New tests: a_protocol_registers_once, a_lifeboat_move_names_the_vaults_
own_mint (also completes a move in and an exit, as a control: the
destination receives 1,000 USDC), a_payout_goes_only_to_an_account_a_
token_program_owns.
Evidence (tasks/error-codes/mutants.log, STATE=c6 and c6b, full program
suite): E01, W10, W12, T01 KILLED by the tests named above; all four
SURVIVED at base. Full program suite green: Floor 50, Vault 111. Vault
hash 7086148b unchanged (tests only).
Counts: Vault 108 -> 111, programs 158 -> 161 (claims.py OK).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test the bounds on ticket extensions and a closed lane's code
B's review of vault-error-tests-r3 (R1) found two Vault codes no test
asserted by value that the earlier commits missed: ExtensionTooLong
and LaneClosed. Deleting the 7-day total extension cap (brakes.rs:86,
B24) left every program test green, so a live guardian, or a stolen
guardian key, could hold a queued payout back without end, 24h at a
time, against docs/05 row 11. The wide differential models the total
(differential_wide.rs:406), but its default run never extends one
ticket that far. Mapping LaneClosed to QueueFull (custody.rs:240, B23)
also survived.
The code review of the fix found a worse gap at the same check: no
test sends a negative extension (the differential draws secs from 0,
1, 3600, 86400, 86401). With `secs <= 0` weakened to `secs == 0` (A3,
brakes.rs:84) every program test passed, and one guardian key could
pull a queued payout's ETA into the past, skipping the queue delay,
and refund its own 7-day budget.
- a_ticket_is_extended_later_by_at_most_a_day_a_call_and_seven_in_total:
-1h, 0 and 24h + 1 s are each refused with EXTENSION_TOO_LONG, ETA
unchanged; six 24h extensions and one of 23h pass; 2h more is
refused, ETA unchanged; the last hour is granted; then 1 s more is
refused. The split is unaligned so that a total that ignores this
call's `secs` fails too.
- a_closed_lane_refuses_an_unbacked_payout: a vault with short_bps 0,
and one with long_bps 0, each refuse an unbacked admin payout with
LANE_CLOSED; the same payout from a vault with both buckets open is
taken (the control, so a capacity base of 0 cannot pass it).
Evidence (tasks/vault-error-tests-r4/, B's runner copied: each mutant
applied alone, programs rebuilt, whole program suite run):
- without these tests (551e519; run-without.out, run-base2.out,
run-base3.out): B23, B24, A1 (`tk.extended_s >= MAX`, this call's
secs ignored), A2 (`>=` for `>`) and A3 SURVIVED. A4 (the 24h per
call deleted), A5 and A6 (either half of limiter.rs:172's closed-lane
test deleted) were KILLED by the differential only, not by value;
- with them (run-final.out, at the branch tip): B23 KILLED by
a_closed_lane_refuses_an_unbacked_payout; B24 A1 A2 A3 KILLED by the
extension test only; A4 by it and the differential; A5 A6 by the
closed-lane test and the differential.
No program change: the runner's closing build gives 8ca8a608 /
543be60b / 7086148b, as at cede705.
Counts: Vault 111 -> 113, programs 161 -> 163 (claims.py OK).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin each half of create_pda's existing-account guard on its own
B's review of vault-error-tests-r3 (R2): each half of lib.rs:229
(`!is_data_empty() || !owned_by(SYSTEM)`) survived deleted on its own
(B20, B21), since an existing protocol account fails both, so
a_protocol_registers_once pinned them only jointly.
The test now also plants, at a fresh protocol address, an account that
fails just one half: one with data owned by the system program, one
with no data owned by another program (a fresh address). Each
registration is refused with ALREADY_INITIALIZED. The comment says no transaction is
known to make either account: data or a new owner at a PDA needs its
signature, which only the Vault gives, and create_pda gives both in one
instruction.
Evidence (tasks/vault-error-tests-r4/): at 551e519 B20 and B21
SURVIVED (run-base2.out); at the branch tip both are KILLED by
a_protocol_registers_once only (run-final.out).
No count change (claims.py OK: vault_programs 113, programs 163).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Record the Vault refusal deletions and what each became
docs/09 §3 gains a row for the deletion run this branch answers
(tasks/error-codes/mutants.log): 38 refusals, across 14 of the 18 Vault
codes no test asserted by value at cede705, each deleted on its own
with the whole program suite run against it, on the AUD-2 build.
The 18 (each VaultError variant with no `verr::` use in programs/tests
at cede705): AlreadyInitialized NotRegistrar WrongVault
WrongTokenProgram AlreadySealed VaultCountMismatch InvalidLimits
InvalidAsset LaneClosed TicketNotPending ExtensionTooLong
ChangeExecuted LedgerUnsound LedgerCorrupt NoSlot NothingToExit
ZeroAmount Overflow. The run missed LaneClosed and ExtensionTooLong;
B's review deleted one check of each (B23 custody.rs:240, B24
brakes.rs:86), and both survived until 1797ae5. That commit
also pins a negative extension (A3, which survived every test before
it), the 24h per call, and a lane closed by either bucket.
At base: 13 KILLED, 25 SURVIVED. The commits before this one add tests
that kill 18 of the 25 and both of B's, each asserting the Vault's own
code (W02 W06, E04 E05, E06 E07 E08, E14, E02 E12 E13 E16 E17 E18, E01
W10 W12 T01; B23 B24). The row names the 7 that still survive and why:
- six guards no transaction is known to reach: lib.rs:324 (vault PDA
address; create_vault creates a vault only at its PDA), genesis.rs:519
and :523 (the lifeboat, checked by expect_pda first, created only by
lifeboat_move, move_in requires the seal, lifeboat.rs:53, and the
seal happens once, so no lifeboat holds data at it), brakes.rs:50 and
custody.rs:541 (a pending-ticket count below zero), token.rs:235
(GetAccountDataSize's return data from an already-checked program);
- custody.rs:399, stated as not shown unreachable: properties.rs:588-592
debits backed_amount and one more on random ledgers, and only
crates/bulkhead-core/tests/units.rs:375 debits less, on a one-mint
ledger with no liability. properties.rs:606 asserts backed(a) ==
min(a, backed(MAX)) for requests below 2^52 (:613), so any smaller
amount is backed too, and debit_backed(b) refuses only when
backed_amount(b) != b (ledger.rs:289). The known exception is a
collateral value that overflows u128 (ledger.rs:262): the valid
amounts need not form an interval there, so the consent or freeze
cap (custody.rs:337, :347) can lower the amount to an invalid one and
:399 refuses the release (fail closed). The search's own result is
always valid. Not verified by a tool beyond that property test. (A full ledger cannot reach :399 through
set's NoSlot: backed_amount's valid() refuses a new liability slot
on a full ledger, ledger.rs:237.)
Not deleted, as the row says: the two core-error mappings to
ChangeExecuted and ZeroAmount, token.rs:233, and the codes Overflow and
LedgerCorrupt (checked arithmetic, decoding).
After this branch five codes are still asserted by no test (same grep
at this commit): NotRegistrar (E03, genesis.rs:108, was KILLED at base
by another test), TicketNotPending, LedgerUnsound, LedgerCorrupt,
Overflow.
Docs only: no count change (claims.py OK: vault_programs 113, programs
163).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Run the exclusion check's main in its self-test
A's review of finding 105 (REVIEWS/auto-B-mutants-exclusions-r7.md,
finding 1, Low): CI's `mutants` job acts on main's exit status, but the
self-test called problems() only, and check.sh never runs the tool on
real lists. With main returning 0 on a mismatch (mutant A10), every
in-tree check passed, and CI would have printed the problems and gone
green. This was not a defect at fb771c6, since the job fails today. The
gap was that a later regression would pass.
Fix: main takes its arguments and, for the self-test, a config path
(default: the repo's .cargo/mutants.toml, as before), and its lines name
the config it read. The self-test runs main in-process on a planted
config and checks the exit status and every line on each stream:
- the right lists: exit 0, "1 of 3 mutants excluded" on stdout, nothing
on stderr;
- the lists the other way round: exit 1, nothing on stdout, and exactly
the count and "full list lacks" problems, so KEPT is read as KEPT;
- coloured lists: exit 1 with exactly one problem, so a main that fails
only on several problems is caught;
- one argument and three: exit 2 and the two usage lines.
One subprocess (python3 -I, 60 s timeout, in the temporary directory so
the config must be found from the tool's own path) runs the command line
with two lists and the repo's config, as scripts/mutants.sh does: on a
list no pattern can match, it exits 1 with an empty stdout and problem
lines that each name .cargo/mutants.toml. Which problems those are
depends on the config, so only their form is checked. Self-test: 16
cases -> 22. No doc states the number.
Evidence (tasks/exclusions-main-exit/):
- old-A10.log: the fb771c6 self-test with A10 applied prints "self-test
OK: 16 cases" and exits 0.
- mutants-r6.log (mutants.py applies each mutant to a scratch copy of
the tool, which reads the real config through a symlink): unmutated,
"self-test OK: 22 cases", exit 0. All 18 are KILLED (exit 1): A10
(return 0), A10b (fail only on more than one problem), A0 (always 1),
U1 (usage returns 0), U2/U3 (argument check `< 2`, `< 1`), K1 (KEPT
ignored), K2 (ALL read twice), C1 (planted config ignored), C2 (no
default config), R1 (the default config read from the cwd), E1/E2
(sys.argv sliced as [:] or [1:2]), P1 (problems not printed), S1
(problems on stdout), L1 (the planted config not named), N1 (only the
first problem printed), G1 (the success line printed on failure too).
Code-review passes at high shaped this. The first version ran three
subprocesses against the real config, and A0, K1 and U3 survived it.
The second left E2, C2 and G1 alive. Declined from the third: catching
exceptions in the self-test (a traceback already exits 1) and moving
the config hook out of main. The fourth showed that requiring one
"matches 0 mutants" line per live pattern tied the self-test to the
config's contents. The subprocess now checks the form of the lines
only, and the 17 stay killed. The fifth showed that R1 survived while
the child ran in the repo root (it does in check.sh); now it runs in
the temporary directory. Its other points repeat declined ones.
- real-lists.log: on the real lists, made with scripts/mutants.sh's
commands: ".cargo/mutants.toml: 22 of 795 mutants excluded, each
pattern matching its count", exit 0.
- python3 -I tools/mutant_exclusions.py --self-test: 22 cases OK.
- scripts/check.sh: ALL CHECKS PASSED (check-final.log);
bin/check-each-commit.sh green (each-commit.log).
Sibling search: the other tools check.sh runs (tools/claims.py,
tools/floor_check.py) also gate CI through main's exit status. claims.py
has the same gap: check.sh runs it on the real tree, where it finds no
problems, and its self-test calls self_test(), not the path in main that
returns 1. With that return changed to 0, `--self-test` and the real run
both still exit 0 (claims-C1.log). Not changed here: it is its own
commit (BACKLOG-B). floor_check.py runs in CI only as --self-test, whose
result main returns as is. Its RPC mode is an operator's tool that no CI
job runs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Record finding 106: Vault liveness and exit rules had no failing test
The tests landed in 944f5e8 (lifeboat-exit-tests-r3, 15 Oct). Their
09 §4 row waited for 104 and 105 so the list stays numbered without
gaps; 105 is now on integration (fb771c6), so the row lands here.
Docs only: 09 §4 row 106; README and 12's count and post-submission
range (96–106, 106 fixed). Two code-review passes changed the row as
written on 15 Oct: a sentence for today's re-run, the nine
single-guardian sites named, G5 quoted verbatim, and the exit veto
attributed to T18 (G6 has no veto), as is the veto test's doc comment
(vault.rs:3996, comment only).
Evidence, re-run today on bf8ba41 (the row's claims are from 15 Oct,
and AUD-2 changed the Vault since): B's re-review list of 33 line-
neutral mutants (the ten rules, the nine single-guardian sites L1-L9,
the boundaries, the six quorums, R1-R6), each applied, programs
rebuilt, all program tests run:
33 KILLED, 0 survived, 0 not applied, 0 build failures. Vault
hashes after the run match the tip (Vault 7086148b, Floor 8ca8a608,
Harbor 543be60b).
Log: autopilot tasks/finding-106-row/mutants.log.
- python3 -I tools/claims.py: OK, findings 106.
- git grep for "96–105", "105 findings": no other stale range.
- scripts/mutants.sh still mutates the core only (SCOPE, line 11), so
the row's last sentence holds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Cite PR #100's tree, not the unpublished bf8ba41, in finding 106
Finding 106's row in docs/09 said its 33 mutants were re-run on 16 Oct
at bf8ba41. That commit is not an ancestor of this branch or of the
stack above it and is on no remote branch, so the citation would point
at nothing once the stack lands. The owner's review of #99-#109 found it.
bf8ba41 and 86b777b (the head of PR #100, vault-error-tests-d2, the D2
replay of vault-error-tests-r4) have the same tree, 36187b49
(`git rev-parse bf8ba41^{tree} 86b777b^{tree}` prints it twice). So
the run was a run on 86b777b's tree, and the row now says "on tree
36187b49, the tree of 86b777b (PR #100's head)". Why not just "at
86b777b": it was committed on 17 Oct, after the run; and main
squash-merges (1045cd9 has one parent), so after PR #100 lands 86b777b
is reachable only as GitHub's refs/pull/100/head. The tree id stays
checkable from main: squashed bottom first, main's commit for PR #100
has that tree. "PR" keeps "#100" from reading as finding 100, the
row above. These points came from two code-review passes.
Sibling search: every 7-12 hex token in tracked .md/.py/.sh/.mjs/.rs/
.toml files at autopilot/integration 3e9f58d that names a local commit
(51 tokens) was checked with git merge-base --is-ancestor against
integration. bf8ba41 was the only one that is not an ancestor.
Docs only. tools/claims.py: claims OK (26 files).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Warn about a Floor guardian set short of the margin (finding 107)
The guardian checks the Vault protocol's guardian set against finding
107's margin (main.mjs checkGuardianSet), but never a Floor config's.
A Floor set registered before the rule, say 2-of-3, keeps working and
keeps AUD-2's exposure: one lost key and one rogue guardian lock its
governance for good. The guardian said nothing, and docs/07 §6 said so
("not yet a Floor set").
The rule moves to policy.mjs as guardianSetWarning, a pure function:
the same text as before, a warn line that alerts on mainnet only. In
main.mjs, checkGuardianSet now remembers the set shape per set
("protocol", or the protected program), and checkFloor calls it for
each watched Floor config it reads. floorAlerts is unchanged. The
margin is not a readiness code there: "floor not ready" alerts on
every cluster, and only the owner's new set clears a margin problem, so
"floor ready" would never be logged again.
Evidence (tasks/floor-guardian-margin/r2/):
- node --test cloud/test/*.test.mjs: 180/180 pass (one new test).
- Mutants of guardianSetWarning, each run on policy.test.mjs and
reverted; each fails the new test: N1 never warn, N2 threshold fixed
at 2, N3 notify always, N4 notify never, N5 GUARDIAN_MARGIN = 1.
- tools/claims.py: OK, client 180 (docs/09:40 updated).
- Not tested: the main.mjs wiring (the per-set Map and the checkFloor
call). No test runs checkFloor; deleting the call leaves every test
green.
Sibling search: guardianMarginProblem's callers are vault-check.mjs (an
info line) and guardianSetWarning. The status snapshot
(snapshot.mjs:33) prints live guardians and the quorum, not the margin;
left as it is.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Judge each guardian set's margin on its own, and say when it is fixed
The parent's per-set memory (which set shapes were already reported)
lived in main.mjs, so no test could fail on it: dropping the
once-per-shape check, or keying every set the same, left every test
green. The warning did not say which set it was about (only a program
tag on Floor lines), and it gave the same text for a set ejected down
to its threshold, or registered n-of-n before finding 96. There one
rogue guardian alone locks governance; a lost key alone does not, since
dead guardians cannot veto the owner's new set (docs/16 §4.4), but it
leaves no quorum until that set executes. Nothing was said when the owner's
new set fixed the problem. And a delivered alert did not say which Floor
program it was about: the program went only into the line's `program`
field, which alertFor uses in its repeat key but does not show.
Moving the program name into the text is a trap: alertFor keys repeats
on the first 60 characters of the text, and "the Floor guardians of
<44-character id>" pushes the shape past them. A 2-of-3 set ejected to
2-of-2 within ten minutes would then give the same key, and the worse
warning would not alert. So the program goes after the shape: "the
Floor guardians are 2-of-3 for <program>: ...". The `program` field
stays for the repeat key.
guardianSetWarning becomes guardianSetCheck(shapes, which, set,
mainnet) in policy.mjs. It keeps each set's last shape and whether it
was short in the Map, keyed by "protocol" or the protected program, and
returns the line to log or null. The warning names the set and has its
own text for n-of-n. When a set that was short gets the margin, it
returns one line saying so, which alerts on mainnet like the warning,
so whoever was paged hears that it is fixed. main.mjs only logs what it
returns. docs/07 §6 says so. The advice also had a hole: a set
registered as 7-of-8 was told to "run at least 7-of-9", and no set can
have 9 keys (core MAX_GUARDIANS is 8). When threshold + 2 is over 8,
the advice is now to lower the threshold to at most 6-of-8. That needs
MAX_GUARDIANS in JS, so lib/solana.mjs gets it next to
GUARDIAN_MARGIN; the next commit ties both to the core.
Evidence (tasks/floor-guardian-margin/):
- node --test cloud/test/*.test.mjs: 180/180 pass (the test is
extended, not added).
- 19 mutants, each run on policy.test.mjs and reverted (r6/mutants.py,
r6/mutants.log, r6/N5-margin1.log). Each fails the extended test, and
only it: never warn, threshold fixed at 2, notify always, notify
never, GUARDIAN_MARGIN = 1, no once-per-shape, one Map key for every
set, the n-of-n text for threshold + 1, no fixed line, a fixed line
for a set that was never short, the fixed line never alerting, no
program tag, every set named as the protocol's, the short flag not
stored, the program before the shape, the program left out of the
text, `need < 8` for `need <= 8` (6-of-7), the 8-key limit ignored
(7-of-8), 7-of-8 as the lowered advice.
- The mainnet alerts are checked on their own in a scratch copy of the
test without the earlier text assertions (r5/C1-C3-alone.log): with
the program left out of the text, the alert's detail check fails;
with a long name pushing the shape past character 60, "2-of-2 alerts"
fails.
- A's review of the previous version (branch
auto/B/floor-guardian-margin-r7, kept) found two survivors there: X3,
the shape keyed on the count only, and X5, the advice
threshold-of-(n+1). It also found that the n-of-n text said a lost key
alone locks governance. The test now has a threshold-only fix (7-of-8
-> 6-of-8, the advice's own step) and pins the n-of-n text and advice.
r9/mutants.log: X3, X5, the r7 n-of-n text and the text without its
lost-key clause are all KILLED. r9/mutants-r7.log: X3 and X5 SURVIVED
on the r7 code and test. The 18 mutants of r6/mutants.py, re-run on
this code: all KILLED (r9/r6-mutants-rerun.log). Two code-review
passes on the fix (kept as -r8 and -r9) added the lost-key clause: a
lost key alone leaves no quorum (it never approves, and core `quorum`
needs threshold approvals) until the owner's new set executes. A third
pass (-r10) fixed the comments' wording only.
- tools/claims.py: OK, client 180.
- Still not tested: the checkFloor call in main.mjs (no test runs
checkFloor).
Not changed (code review): the "fixed" line needs the short shape to
have been seen since the guardian started (the Map is in memory, as the
old per-process shape was; docs/07 says so). A protocol that abdicated
SetGuardians can't take a new set, so the advice to run 2-of-4 can't be
followed there; REPORT lists that case for the owner. The fault text
assumes a margin of 2, the owner's decision (16 §4.4).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Tie the JS guardian margin and set size limit to the core constants
cloud/lib/solana.mjs keeps its own GUARDIAN_MARGIN = 2, which the
guardian and the deploy tooling use to judge a guardian set, and now
MAX_GUARDIANS = 8, which the guardian's advice uses. Their comments say
they are core governance.rs's constants, but nothing checked that. Had
the programs' margin moved to 3, every JS test would have stayed green
while the guardian passed sets the programs refuse.
deploy.test.mjs now reads governance.rs and requires exactly one
`pub const GUARDIAN_MARGIN: u8 = N;` and one `pub const MAX_GUARDIANS:
usize = N;`, each N equal to the JS constant. It reads the source text,
like the build-script test above it. If a declaration changes form, the
test fails rather than passing.
Evidence (tasks/floor-guardian-margin/r6/tie-mutants.log):
- node --test cloud/test/*.test.mjs: 181/181 pass (one new test).
- Mutants of governance.rs, each reverted with its mtime kept (else the
limiter differential asks for a worker rebuild): GUARDIAN_MARGIN = 3,
MAX_GUARDIANS = 9, and a trailing comment on MAX_GUARDIANS. Each fails
the new test, naming the constant. deploy.test.mjs is green again.
- tools/claims.py: OK, client 181 (docs/09:40 updated).
Sibling search: the other JS copies of core rules (the limiter port,
the account layouts) are checked by the differential and by decoding
real accounts. MIN_GUARDIAN_THRESHOLD has no JS copy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Reach the 7-day extension total in the wide differential
differential_wide.rs drew extension seconds only from {0, 1, 3600,
86400, 86401} on a random ticket and lumped every refusal into one
counter. A ticket never got near the 7-day total, no negative value was
sent, and a refusal was not checked by its code. So the default run
could not tell the total bound, a negative extension or the error code
from a bug: only the unit tests at vault.rs:3382-3401 pinned them.
Now one extension draw in three pushes the chosen ticket by up to 24h
a call to exactly the 604,800 s total, then asks for 1 s more, so the
edge itself is hit on every push, not only when earlier draws line up.
The other draws are half legal sizes (1, 3600, 86400) and half out of
range for one call (i64::MIN, -86400, -1, 0, 86401, i64::MAX). Each
refusal must fail with EXTENSION_TOO_LONG. The counters are split into
extend (a random legal size), extend_push, extend_refused_call and
extend_refused_total, all required >= 10, so pushes cannot stand in for
the random sizes.
The model reads both bounds from bulkhead-core (consts.rs:37-38), as
the module says it is built, rather than restating 86,400 and 604,800.
docs/09 §7's row says so, and the 16 Oct "Vault refusals deleted" row,
which says the default run never extended a ticket that far, now says
it does since this change (its differential_wide.rs:406 cite had drifted).
Default run (24 seeds x 250 steps): extend 16, extend_push 126,
extend_refused_call 16, extend_refused_total 30 (was extend 40,
extend_refused 27); every other required counter still >= 10
(loosen_executed 12, as before).
Evidence: mutants of brakes.rs extend_ticket, each built and run against
the old and the new differential_wide only:
T1 drop the total check old SURVIVED new KILLED
T2 total > to >= old SURVIVED new KILLED
T3 drop secs <= 0 old KILLED new KILLED
T4 secs <= 0 to < 0 old KILLED new KILLED
T5 total ignores this call's secs old SURVIVED new KILLED
T6 extended_s += secs / 2 old SURVIVED new KILLED
T7 refusal returns WrongTicket old SURVIVED new KILLED
T8 secs <= 0 to == 0 (negatives) old SURVIVED new KILLED
T9 total > to > total + 1 old SURVIVED new KILLED
(T9: a draft that pushed by 24h until refused killed it only through
chance 1 s and 3600 s draws on the same ticket; the push to exactly
604,800 then 1 s more reaches the edge every time.)
sBPF sha256 after the run unchanged (Floor 8ca8a608, Harbor 543be60b,
Vault 7086148b): tests only.
Sibling search: the other refusal counters in this file (reset_refused,
lifeboat_refused_*, loosen_*) are each split by cause already.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check settle and lifeboat refusals by code in the wide differential
The same gap as the extension refusal, found in the sibling search for
the previous commit: a refused settle and the three lifeboat-move
refusals asserted only r.is_err(). Any refusal passed, so the order of
the checks and the program's own codes went unchecked by the
differential. Settle's comment (custody.rs:520-521) says it returns its
own error, not the token program's, when the vault is short; nothing in
this file held it to that.
Now a settle refused while frozen must fail with FROZEN, and one that is
only short with INSUFFICIENT_BALANCE (frozen is checked first). A
lifeboat move inside the delay after the seal must fail with NOT_SEALED,
one while frozen with FROZEN, and one beyond the balance with the token
program's InsufficientFunds (1). The two settle refusals get their own
counters: settle_refused_frozen is required >= 10 like the rest;
settle_refused_short needs lifeboat moves to drain a due ticket's
tokens first and occurs 7 times in the default run, so it is required
>= 1.
Evidence: mutants of programs/vault/src, each built and run against the
previous head's differential_wide and this one, nothing else:
S1 settle: drop the InsufficientBalance check old SURVIVED new KILLED
S2 settle: balance check before the freeze old SURVIVED new KILLED
L1 require_sealed returns Frozen old SURVIVED new KILLED
L2 move_in's freeze returns NotSealed old SURVIVED new KILLED
The token-program code (1) on a move beyond the balance is pinned but
not shown red by a mutant. sBPF sha256 unchanged (Floor 8ca8a608,
Harbor 543be60b, Vault 7086148b): tests only.
Deep run of both commits together (BULKHEAD_PROGRAM_SEEDS=600
BULKHEAD_PROGRAM_STEPS=500, 85 s): 300,000 steps, all identical;
extend 876, extend_push 6,462, extend_refused_call 1,175,
extend_refused_total 1,453, settle 2,024, settle_refused_frozen 921,
settle_refused_short 566, lifeboat refusals 3,854 + 5,768 + 1,171. docs/09 §7's
row gets that run as a 16 Oct result next to the 5 Oct one, which
predates these checks.
Sibling search: in differential_wide.rs, reset (TICKETS_PENDING) and
execute change already asserted codes, and a release or quote refused
for want of tokens asserts INSUFFICIENT_BALANCE. A release or quote the
limiter refuses (frozen, lane closed, queue full) is still compared only
as a refusal: mapping the core's refusal to the program's code is left
for its own change. Lifeboat returns only send legal amounts. docs/09's
row now says which refusals are checked by code. differential.rs:327
has the same is_err settle check; left as it is, since the wide run now pins both
settle codes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check limiter refusals by code in the wide differential
A release or quote that the limiter refused (lane closed, queue full)
was compared with the model only as Err. Any refusal code passed: a
program that swapped LaneClosed and QueueFull in custody.rs's
limiter_error, or checked the backed part against the balance before
the limiter and so answered InsufficientBalance for a full queue,
still matched the model. About 670 such refusals happen in a default
run.
The model now maps core's LimiterError to the program's code with
verr::limiter in the test harness (all four variants, as custody.rs's
limiter_error does), and every refused quote and release must fail
with the model's code. Lane-closed and queue-full refusals have their
own counters, each required at least 10 times; a refusal code with no
counter panics rather than being counted as want of tokens. docs/09
row 362 says so and records the deep run.
Evidence (autopilot tasks/differential-limiter-codes/):
- mutants.log: C1 (LaneClosed and QueueFull swapped in
limiter_error), C2 (QueueFull -> InsufficientBalance), C3
(LaneClosed -> ZeroAmount) and C4 (backed > balance checked before
the limiter, same line) each SURVIVED the old test and are KILLED
by the new one. Hashes back to 8ca8a608/543be60b/7086148b after.
- c4-full-suite.log: under C4 the whole program suite (Vault 113,
Floor 50, the fixed differential) passes; only this test fails.
C1-C3 are also caught by vault.rs's code tests (:434, :3428).
- Default run: refused_lane_closed 254, refused_queue_full 414,
insufficient 118.
- deep-600x500.log: 300,000 steps, 15,561 lane closed, 18,853 queue
full, 6,345 want of tokens, all identical.
Sibling search: every other Err arm in differential_wide.rs already
asserts a code (extend, settle, reset, lifeboat in, execute change).
Not here: lifeboat returns are never refused (backlog), and lifeboat
move_in beyond the balance fails with the token program's code
(backlog, a program change).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check release refusals by code in the fixed differential
differential.rs reduced every refused release to Err(()) on both
sides, so a program that refused for the wrong reason still matched
the model. The same gap the previous commit closed in the wide
differential (found by its code review).
The model's refusal now carries the program's code: verr::limiter for
a limiter refusal, INSUFFICIENT_BALANCE when the vault holds less
than it would pay. A refused release must fail with that code, and
the assert prints the program's error. Refusals for a full queue get
their own counter, required at least 10 times like the others. Want of
tokens and a closed lane are counted and printed but not required:
the wide differential requires both. A refusal with any other code
panics, as in the wide differential, so the counts always add up.
docs/09 row 361 says so; its undated figure (4,843 releases) no longer
matched a run, so it is replaced by a dated one.
Evidence (autopilot tasks/differential-limiter-codes/):
- mutants-narrow.log (mutants.sh <wt> differential): C1 (LaneClosed
and QueueFull swapped), C2 (QueueFull -> InsufficientBalance) and C4
(backed > balance checked before the limiter) SURVIVED the old test
and are KILLED by the new one. C3 (LaneClosed -> ZeroAmount)
survives both: no release in this test's runs is refused for a
closed lane. The wide differential kills it, and docs/09 says so.
Hashes back to 8ca8a608/543be60b/7086148b after.
- r2/default-run.log: 1,000 steps, 82 releases refused, all for a
full queue (want of tokens 0, lane closed 0).
- r2…
shuv-amp
added a commit
that referenced
this pull request
Oct 18, 2026
* Run the tests on 2-of-4 guardian sets; test recovery from two faults
Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set
with fewer than threshold + 2 keys. Every test fixture and most tests'
own sets were 2-of-3. This moves them to 2-of-4, which today's rule
also accepts, so this commit changes no program: Floor, Vault and Harbor
hash 508e5a99 / b07655d9 / 543be60b, as before.
- Fixtures: Protected (Floor) and Market (Vault, through Harbor) now
hold 4 guardians. Each test that built its own 3-key set gets a 4-key
set with the same threshold and the same assertion.
- Two tests changed meaning with a 4th key and were fixed, not loosened:
a_quorum_ejects_a_rogue_guardian_who_vetoes_everything (Floor) and
a_quorum_ejects_a_guardian_who_freezes_forever (Vault). With 4 keys,
the last eject was refused only because the target's own vote is
masked. Both now eject a second guardian (a set an ejection shrank to
threshold + 1 still ejects) and then show the 2-of-2 refusal.
- Account sweeps: an eject's signers may now be any two of the three
others, so each of those two slots is free for the third, never for
the target or the other signer again. Vault 5,365 and Floor 1,710
substitutions, refused (docs/09, dated 16 Oct). Listing all three
others at each slot would have skipped the duplicate: with a mutant
that counts a guardian listed twice as two (governance.rs:270), the
sweeps listing all three pass, and these fail at "eject: account 2
replaced by key ..." (Vault and Floor). No other program test kills
that mutant (tasks/aud2-guardian-margin-impl/dup-mutant-*.log).
- floor_differential: the guardian pool is 6 keys, 4 registered; the
proposed sets are 2 to 6 keys.
New tests (pass on both rules; they show what the margin buys):
- a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_set (Floor): g0's
key lost, g1 rogue stops the program; the lost key does not help a
quorum; the honest pair ejects g1 and restores; neither can veto the
owner's refill, which executes after 72h.
- two_rogues_in_turn_are_both_ejected_from_a_two_of_four_set (Floor):
the second rogue vetoes the refill and is ejected from the 2-of-3.
- a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_vault: the same in
the Vault (freeze, eject, unfreeze, refill).
Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest
and quote sweep): the counts and the sweep figures here are from runs
there (tasks/aud2-guardian-margin-impl/r4/). The E2E run, the mutants
and the new-rule run below were made before, on e8640a7; this commit's
programs and core are the same as there but for one comment line in
views.rs (line-neutral, same hashes).
Evidence:
- E2E=1 scripts/check-programs.sh: PROGRAM CHECKS PASSED (Floor 49,
Vault 98; S1 and E2E passed; the resumed setup found all 14 steps;
vault check, Floor demo and deploy dry run passed), with the next
commit's deploy rehearsal in the tree.
- tools/claims.py: OK (floor 49, vault 104, programs 153).
- Under the new rule (core from two commits on, programs rebuilt), only
the two tests that state the old rule fail: an_n_of_n_guardian_set_is_
refused (accepts 3-of-4) and the Floor differential's model.
- Mutants (tasks/coverage-map/mutants.sh), each red, then restored:
AV1 Floor live_guardian ignores liveness: KILLED by a_lost_key_and_a_
rogue_do_not_lock_a_two_of_four_set (+2 others).
AV2 the Vault's: KILLED by ..._two_of_four_vault (+2 others).
AQ1 Floor restore quorum ignores liveness: KILLED by ..._two_of_four_set
and the differential only.
AQ2 the Vault's unfreeze quorum: KILLED by ..._two_of_four_vault and
a_silent_guardian_does_not_count_toward_a_quorum.
AE1 can_eject refuses at count == threshold + 1 (AUD-2 option (a)):
KILLED by two_rogues_in_turn_..., and the two eject tests above.
Hashes after: 508e5a99 / 543be60b / b07655d9.
Split on 16 Oct from ec0cacc at the owner's review (D2): the deploy
rehearsal's move to 4 guardians is now the next commit. programs/ and
programs/tests/ here are byte-identical to ec0cacc, so its runs below
hold for this commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Rehearse deploys with four guardians, and read only the keys there are
Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set
with fewer than threshold + 2 keys, so the deploy rehearsal must
register 2-of-4 sets. Split from ec0cacc at the owner's review (D2);
the text is that commit's, unchanged:
Deploy rehearsal (cloud/deploy market.mjs, vault-check.mjs,
floor-demo.mjs): 4 guardians (keys/guardian3), so the dry run keeps
working once the rule lands; one more funding step (14, was 13).
The devnet market has 3 guardian files. A first version of this change
loaded guardian3 everywhere: vault-check.mjs failed there with ENOENT,
and heartbeat silently created a key nobody holds as a guardian. Now
market.mjs guardianKeys(dir, create) reads guardian0, 1, ... and
throws on a gap (never fills it with a new key). Setup creates keys up
to 4 only while the protocol is not registered, so resuming an old
market adds no key the chain lacks; a new deploy-dryrun step resumes
the dry-run market with guardian3 moved aside and fails if setup makes
it again (with setup always creating: "DRY RUN FAILED: resuming a
registered market made keys/guardian3.json", r3-dryrun-J7.log).
renew-guardians tops up to 4 and logs each new key (unfunded); a
2-of-3 renewal would be refused once the rule lands. keysIn reads the
guardians only when a command uses them, so caps and set-recovery
never need them; heartbeat says a key is not in the set rather than
dead. One THRESHOLD (2) for the register and renewal payloads and
MIN_GUARDIANS. vault-check.mjs
compares the chain's set with the first files, and liveness with the
chain's count. An extra file (a renewal not executed yet) and a set
without the margin are info lines, not failures, since such a set
keeps working. GUARDIAN_MARGIN and guardianMarginProblem live in
cloud/lib/solana.mjs, used by these and by the guardian bot's existing
warning (cloud/guardian/main.mjs), and vault-check uses THRESHOLD. The e2e scripts fund guardian3 too. Three
tests in cloud/test/deploy.test.mjs (client tests 176 -> 179). JS
mutants (js-mutants-r3.log), each red then restored: always create,
never create, no gap check, margin 1, `>`, GUARDIAN_MARGIN = 1.
Setup always creating survives these tests and is killed by the dry
run step above.
cloud/, scripts/deploy-dryrun.sh and docs/09's client-test row are
byte-identical to ec0cacc (git diff empty), so its runs hold for them:
the E2E check-programs run with the dry run (r4), and the JS mutants.
tools/claims.py: OK (client 179).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Refuse a guardian set without two keys beyond its threshold (107)
AUD-2, decided by the owner on 16 Oct (option (e) of 16 §4.4). A set
stays live while its honest live guardians reach the threshold: they
eject every rogue in turn, and dead guardians cannot veto the owner's
new set. Finding 96 asked for one key beyond the threshold, so a 2-of-3
or 3-of-4 set survived one fault but not a lost key plus a rogue: the
honest guardian left is no quorum, nobody can be ejected, and the rogue
keeps Floor stopped (or the Vault frozen) and vetoes every new set.
Fix, in the core so both programs inherit it through GuardianSet::parse
(registration, and every SetGuardians when proposed and when executed):
GUARDIAN_MARGIN = 2
guardian_set_ok := count <= 8 && t >= 2 && count >= t + 2
(saturating_add: t is a payload byte). A stored set is not checked
again, so existing 2-of-3 sets keep working, and ejection is unchanged:
a shrunk set still ejects. A 2- or 3-key set is BadThreshold, not
BadCount; the count rule is only the bitmap size, and the error order
of finding 103 is kept.
This changes both programs, by intent:
Floor 508e5a99 -> 8ca8a608, Vault b07655d9 -> 7086148b,
Harbor 543be60b (unchanged). Darwin-arm64 builds.
Compute (scratch LiteSVM test, tasks/aud2-guardian-margin-impl/cu.md):
-3 CU on every path that parses a set (Floor propose 2035 -> 2032,
execute 1682 -> 1679; Vault register 9592 -> 9589, SetGuardians 2979 ->
2976; Floor register varies with its PDA bump, residue also -3).
Proof and tests:
- Kani: accepted_sets_can_eject_anyone restated (ejection reachable
exactly when count > t, so option (a) cannot slip in; accepted sets
have >= 4 keys and a margin of at least 2, and 2 is accepted). New:
an_honest_live_quorum_ejects_any_rogue (any slots, anyone else alive)
and accepted_sets_survive_any_two_faults (any two faulty keys, lost
or rogue, leave a live honest quorum, which ejects each rogue, and the
rogues alone eject no honest guardian). That holds only short of a
rogue quorum: two rogues in a 2-of-n set are one (out of scope, 04
T7). With the assumption rogue < t dropped, the harness fails at
`!can_eject(rogue, ..)` (kani-no-rogue-assume.log), so the docs say
"short of a rogue quorum", not "any two faults".
19/19 verified, 240 s (cargo kani -j 2, as CI); again after the
restatement, 235 s summed (r2-prove.log).
- Kani mutants (kani-mutants.sh): margin 1, margin 3, `>` for `>=`,
t >= 1, count <= 9, option (a), and an eject one vote short (only
the two-fault harness kills that one): each KILLED by a harness
(kani-mutants-r2.log). The eject
ignoring liveness is the eject harness's case, not these.
- Program mutants (coverage-map/mutants.sh, rebuilt each time):
margin 1: KILLED by a_guardian_set_without_two_spare_keys_is_refused
(Floor and Vault), registration_rejects_weak_settings and the Floor
differential. Margin 3 and `>`: KILLED by most of the suite (the
fixtures are 2-of-4). t >= 1: first only the Floor differential; the
1-of-2 registration case was now refused by the margin too, so it is a
1-of-4 here and a Vault 1-of-4 case was added: then KILLED by both.
count <= 9: SURVIVES the program tests, equivalent there (parse refuses
more than 8 keys first), KILLED by Kani. Option (a): KILLED by the
eject tests two commits back (the tests on 2-of-4 sets).
- Tests: the Floor and Vault refusal tests above cover proposal and
execution (a stored 2-of-4 proposal whose threshold turns 3 is refused
when executed, as the devnet renewals proposed before this rule will
be); registration_rejects_weak_settings gains 2-of-3 and 3-of-4.
an_n_of_n_guardian_set_is_refused (Vault) is renamed and accepted a
3-of-4: it now refuses it and accepts 3-of-5.
- Model: the Python rule, the Floor differential's model, and the
guardian_set fuzz target state it again; the GP generator now also
picks count - 2. With the Python rule put back to the old one, seed 11
(20,000 cases) reports 5 GP mismatches; restored, seeds 11-13 identical.
Fuzz seeds two_of_three_refused, two_of_four, six_of_eight, and
seven_of_eight renamed seven_of_eight_refused (byte 323, the
threshold, is 7: now refused). With margin 1 in the core the target
crashes within 60 s; restored, 222,615,288 runs in 601 s, no crash.
- scripts/check.sh: ALL CHECKS PASSED, and E2E=1 check-programs.sh:
PROGRAM CHECKS PASSED, both at the branch tip, whose code is this
commit's (the next commit is docs only; tasks/aud2-guardian-margin-
impl/r4/check.log, r4/check-programs-e2e.log). claims: kani 19,
floor 50, vault 104, programs 154, client 179.
Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest
and quote sweep). The core and the program sources are the same as
before the rebase but for one comment line in views.rs (line-neutral),
so the Kani, fuzz, differential and mutant runs above were made on the
pre-rebase commit; the hashes, check.sh, check-programs and the counts
are from r4 runs (tasks/aud2-guardian-margin-impl/r4/).
The finding 96 row said the restated harness proves "exactly"; it now
says what it proved then and proves now, and gives the Vault test's new
name. The finding 107 row waits for rows 105 and 106 to land (09 §4 is
numbered 1..n); its text is ready. The rule's prose is the next commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* State the guardian margin in the docs, with its costs (AUD-2)
The previous commit made both programs refuse a new guardian set
without two keys beyond its threshold (finding 107). The docs that
state the rule now say so:
- README: guardians 4-8 keys; the G5 row and the "who the guardians
are" paragraph give the reason (any two faults leave a quorum, unless
the rogues are one themselves: two in a 2-of-n set, 04 T7).
- 05 register_protocol: 2 <= threshold <= count - 2, also for every
SetGuardians, proposed and executed. 05 abdicate: abdicating
SetGuardians fixes the set, so ejections shrink it for good.
- 06 state table, 00 D22, FAQ: the same rule.
- 07 §2: 4-8 guardians, and don't abdicate SetGuardians. 07 §6: the
programs enforce the margin; sets registered before keep their
exposure until replaced; an ejection uses up a spare key until the
owner's new set executes; a lost key counts as live until its
interval passes.
- 16: the AUD-2 row is decided; §4.4 gives the decision, why the
margin is the number of faults a set survives (short of a rogue
quorum), the options rejected
and why (the honest-owner and stolen-owner cases look the same
on-chain), what it costs, what it leaves (a rogue quorum; a stolen
owner key with every guardian dead; a rogue's veto of the recovery
key's rotation, checked: Floor's veto has no recovery exception, as
a_vetoed_recovery_proposal_does_not_hold_the_owner_s_slot shows), and
Lido-style lapsing vetoes as a future option only. §5: "independent
guardians, 3-of-5 or more".
- 10: the guardian set's minimum size is no longer an open decision.
SECURITY.md and 08 describe what is on devnet, which still runs the
old rule, and are left as they are.
claims: OK. Docs only; no program file changed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Say the weekly CI run skips mutants, as its condition does
ci.yml's header and scripts/ci-changes.sh said the weekly run does every
job, and 09 §8 gave `mutants` as running on any change to the core. The
`mutants` job's `if` starts with `github.event_name == 'pull_request'`,
so the Monday schedule (and a manual dispatch) never runs it: it mutates
a pull request's diff and has none to mutate. Its trigger list in 09 §8
also left out scripts/mutants.sh and .cargo/mutants.toml, which
ci-changes.sh (line 22) selects it for.
Comments and docs only; nothing runs differently.
Sibling search: `git grep` for "weekly run does", "every job ... weekly"
and "mutants ... weekly" across md/yml/sh/mjs/py. The other hits are
09 §8's "at full depth every Monday" (its table now says which job is
not weekly) and the dispatch input's "every job at weekly depth", which
is the same set as the weekly run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check that the mutant exclusions still match; repin murmur3 (finding 105)
.cargo/mutants.toml excludes the core's 22 equivalent mutants, and the
docs say so. Since 9 Oct it excluded 19. The murmur3 tail's pattern is
pinned by line, `floor.rs:(235|238|241)`, because the body's `h ^= k`
has the same column and function and is not equivalent. 1dc7861 (#52)
moved those lines (now 327/330/333), and nothing checked the patterns.
Failing closed, the three would have failed CI's `mutants` job on a
pull request that changed those lines. Failing open, a pattern that
starts matching a non-equivalent mutant hides it from every run.
Root cause: nothing checked that a pattern still names its mutants.
Fix:
- each pattern ends with the number of mutants it excludes (`# 3`), and
the murmur3 pattern names 327/330/333;
- tools/mutant_exclusions.py fails unless each pattern matches exactly
its number in `cargo mutants --list --no-config`, and cargo-mutants
(the list with the config) excludes exactly the mutants the patterns
match, so a pattern this parser can't see, or reads differently, fails
too. No two patterns may match the same mutant, since the docs' count
is the counts' sum, and the kept list must be part of the full one.
It reads only the exclude_re array's body, needs no tomllib
(Python 3.9), and names colour codes in the list as the problem;
- scripts/mutants.sh makes both lists (--colors never: CI sets
CARGO_TERM_COLOR=always) and runs it first, except when the caller
passes --config or --no-config (as whole arguments). CI's `mutants` job runs that script;
ci-changes.sh now also selects the job for the new tool;
- scripts/check.sh runs its --self-test on planted configs;
- tools/claims.py ties the docs' "22 equivalents" (README, 09 §6 and the
summary row) to the sum of the counts, with the same parser. It finds
that parser from its own directory, so `python3 -I` runs it too, and
check.sh runs its self-test that way. 00's
"18 equivalents" (26 Sep log entry) and run 9's "exactly the 22
equivalents below" (6 Oct) are kept as dated records.
Docs: 09 §4 finding 105, §6 run 10 and the "In CI" note, §8's mutants
row; README and 09's summary row 669 → 797 mutants (run 10); README and
12's post-submission range 96–105.
Evidence (first on 1f2ad48 + the parent, logs in the autopilot's
tasks/mutants-exclusions-r2/; re-run on 58697c6 + the parent,
tasks/mutants-exclusions-r3/):
- `cargo mutants --list`: 797 with --no-config, 778 with the old config
(19 excluded), 775 with this one. The 22 excluded are §6's list, the
murmur3 ones at 327/330/333.
- scripts/mutants.sh --list: exit 0, "22 of 797 mutants excluded"; with
CARGO_TERM_COLOR=always also exit 0. Old config: exit 1, every pattern
"no '# N' count". Planted on the real lists: the old lines with a
count (matches 0, says 3) and the body XOR added (matches 4, says 3)
both exit 1; in both the set comparison alone passes. /usr/bin/python3
(3.9.6): self-test OK and the real check passes.
- The self-test pins each problem by its message, not a count. Mutants
of the tool (tool-mutants.log), each red: M1 no per-pattern check,
M2 no set comparison, M3 no colour check, M4 no empty-array check,
M5 a missing count accepted, M6 a missing count read as 0, M7 the
parser reading past the array. M3 and M5 survived a first version
that checked only the number of problems.
After A's review (16 Oct): the set comparison was pinned only by its
size, so `len(excluded) != len(matched)` passed the self-test. A
case with as many excluded as matched but not the same ones now
fails it (15 cases; red with that mutant, green without), and the
message says how many mutants are in one set only.
From the code review after the rebase: colour codes only in the kept
list were reported as a kept list made with other options; the
colour check now reads both lists (16 cases; dropping the kept half
fails the new case).
On 58697c6, with the overlap and kept-list checks: M8 no overlap
check and M9 no kept-list check, red too, as M1–M7 still are. A
duplicate of the can_eject pattern planted in the real config:
scripts/mutants.sh --list exits 1, "the counts add up to 23, the
patterns match 22 mutants" (planted-overlap.log). Without the
sys.path line, `python3 -I tools/claims.py` fails with
ModuleNotFoundError; with it, both claims runs pass.
- claims.py (claims-mutants.log): one count 1 → 2 fails four statements
(README 188; 09 33, 333, 335; kept-run9.log), not run 9's 331, which
failed too before it was kept; a dropped count fails with the
tool named; deleting any of the three rules fails a self-test case;
deleting the KEPT entry fails 00:103.
- The 22 excluded mutants run alone (--no-config, --re per pattern,
2 min): 22 missed, exactly the excluded list; --re also picked
ema.rs:70 (a field deletion, no pattern matches it), caught.
- Run 10 (f1f05ba, old config, -j4, 46 min; tasks/mutants-run10/):
778 tested, 712 caught, 5 timeouts, 58 unviable, 3 missed, exactly
the murmur3 tail. The core's source is unchanged since (`git diff
--stat f1f05ba HEAD -- crates/bulkhead-core/src` is empty; only
tests/units.rs gained finding 102's tests).
- No program or core source changed.
Rebased on 16 Oct onto cede705 (findings 102 and 100, the attest
sweep and AUD-2). AUD-2 changed the core's source (governance.rs), so
the line above saying it is unchanged since run 10 held on 58697c6, not
here. On cede705: scripts/mutants.sh's list check passes, "22 of 795
mutants excluded, each pattern matching its count"; the murmur3 lines
are still 327/330/333. The 797 in README and 09 is run 10's figure:
README now dates it ("in the last full run, 15 Oct") and finding
105's row says the core has changed since (finding 107).
The setup line in scripts/mutants.sh now pins cargo-mutants 27.1.0,
the version ci.yml installs: each count is the number of mutants that
version lists, so another release can break the check (code review,
16 Oct; `git grep 'install.*cargo-mutants'` finds no other unpinned
install).
The check's report goes to stderr in scripts/mutants.sh, so
`scripts/mutants.sh --list` prints only cargo-mutants' names: before,
its first stdout line was the report; after, stdout is 773 lines
(795 - 22) and stderr the one report line. ci.yml's `mutants` comment
says the job checks the exclusions first.
Row 105 and 09 §6 gave the pin's reason as the body's XORs. The
finalizer's are at column 7 (`grep -n '\^=' floor.rs`, 337-342); the
non-equivalent XOR at column 11 is `h ^= k`, in the body (320) and after
the tail (335). The toml comment, the row and §6 now say that. 09 §8's
`check` row names the two tool self-tests check.sh runs.
README no longer calls run 10's 797 "injected bugs": cargo-mutants
listed 797 but tested 778, 19 being excluded (09 §6, run 10's row).
Sibling search: the other 14 patterns use `\d+` for the line, and each
matches the count now written after it. The murmur3 one is the only
line-pinned pattern.
Not done, by choice:
- Catching an edit that keeps a pattern's number but changes its
mutants (an excluded line itself edited, or a murmur3 body XOR moved
onto a pinned line). The toml and 09 §6 say so: such an edit needs the
mutant judged again by hand. Failing on any diff that touches an
excluded line would catch the first; it is a separate change.
- Moving murmur3's tail into its own function, so its pattern needn't
name lines. It shifts core lines, so it changes the program hashes.
Until then a line move above it fails the check, with a message
saying what to fix.
- A weekly full mutants run in CI. It took 46 min here at -j4, likely
past the job's 90 min on two CI cores: a minutes call for the owner.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that a ticket stays in its own protocol and vault
Two Vault checks that keep a ticket to its own protocol and vault had no
test. Deleting either one left every program test green
(tasks/error-codes/mutants.log, base cede705):
- brakes.rs:79, extend_ticket: tk.protocol != protocol. Without it, a live
guardian of any other protocol extends this protocol's tickets. Anyone
can register a protocol and name its guardians, so that guardian could
hold back every queued payout up to the total extension limit. The
sweep did not catch it: it has no second protocol's guardian to swap in.
- custody.rs:501, settle_ticket: tk.vault != vault. A USDC ticket settled
through the SOL vault (its tokens, its mint) got past the Vault's checks.
New test another_protocols_guardian_cannot_extend_a_ticket: a second
Market's live guardian extends this market's ticket -> WRONG_VAULT; the
same instruction from this market's guardian succeeds. The settle test
now settles the USDC ticket through the SOL vault and expects
WRONG_VAULT.
Evidence (tasks/error-codes/mutants.log, STATE=c1; full program suite
per mutant):
W02 brakes.rs:79 deleted -> KILLED by
another_protocols_guardian_cannot_extend_a_ticket ("expected
failure, got success": the stranger's extension went through)
W06 custody.rs:501 deleted -> KILLED by
a_ticket_settles_after_its_eta_to_its_stored_recipient_only (8049
InsufficientBalance from the empty SOL vault, not 8010)
Both were SURVIVED at base. No program source change: hashes
8ca8a608/543be60b/7086148b, as at cede705.
Sibling search: the other 12 WrongVault sites were deleted one at a time
in the same run. Eight are KILLED by the sweep (brakes.rs:39, lib.rs:320,
custody.rs:160, genesis.rs:491, views.rs:27, lifeboat.rs:151,
governance.rs:169/259). lib.rs:324 (vault PDA address), genesis.rs:523,
lifeboat.rs:71 and :247 survive; later commits on this branch deal with
them.
Counts: Vault 104 -> 105, programs 154 -> 155 (claims.py OK).
The test also reads the ticket's ETA: unchanged after the
refused extension, 3,600 s later after its own guardian's (code-review
fix: success alone did not show the extension happened).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that the seal names every vault and happens once
Two seal checks had no test; deleting either left every program test
green (tasks/error-codes/mutants.log, base cede705):
- genesis.rs:486, execute_seal: the pairs must number 3 * vault_count.
Without it, the seal checks only the vaults it is given. A vault left
out skips its claims-vs-balance check, so an uncovered import seals.
docs/05's seal row says every vault is passed exactly once.
the_seal_checks_each_vault_once covered a vault named twice in place of
another, not one left out.
- genesis.rs:431, propose_seal: refused once sealed (AlreadySealed).
Without it, a sealed protocol's admin re-proposes, which rewrites
seal_executable_s/expires_s and clears seal_vetoed on a sealed protocol.
the_seal_checks_each_vault_once now also leaves the uncovered USDC vault
out (expects VAULT_COUNT_MISMATCH) and, after the seal, proposes it again
(expects ALREADY_SEALED). No new test, so no count change.
Evidence (tasks/error-codes/mutants.log, STATE=c2, full program suite):
E05 genesis.rs:486 deleted -> KILLED by the_seal_checks_each_vault_once
("expected failure, got success": the seal went through without
the USDC vault and its 1,000 USDC uncovered import)
E04 genesis.rs:431 deleted -> KILLED by the_seal_checks_each_vault_once
("expected failure, got success")
Both were SURVIVED at base. Vault hash 7086148b unchanged (tests only).
Sibling search: the seal's other checks. :491 vault_tokens (KILLED by the
sweep), :519 lifeboat owner and :523 lifeboat mint/owner (SURVIVED; the
lifeboat is a PDA checked by expect_pda just above, which only this
program can create, as its own token account: see this branch's note in
docs/09).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that invalid limits and asset settings are refused at every entry
Three bound checks had no test; deleting any one left every program test
green (tasks/error-codes/mutants.log, base cede705):
- genesis.rs:171, create_vault: config_is_valid (bps <= 100%, windows
> 0). Without it, a vault is created with a 100.01%-an-hour burst.
- governance.rs:40, parse of a SetLimits change (at proposal and again
at execution, :254 and :478): the same rule for a proposed change.
- governance.rs:47, parse of an UpsertAsset change: valid_asset (haircut
<= 100%, liability >= 100%, the oracle bounds of finding 88). Only the
genesis upsert's copy (genesis.rs:296) was tested, by
oracle_settings_of_a_priced_asset_are_bounded.
New test invalid_limits_and_assets_are_refused_wherever_they_enter:
before the seal, create_vault with short_bps 10,001 -> INVALID_LIMITS;
after it, a SetLimits proposal with long_window_s 0 -> INVALID_LIMITS,
and an UpsertAsset proposal with a 10,001 bps haircut -> INVALID_ASSET.
Evidence (tasks/error-codes/mutants.log, STATE=c3, full program suite):
E06 genesis.rs:171 deleted -> KILLED ("expected failure, got success")
E07 governance.rs:40 deleted -> KILLED ("expected failure, got success")
E08 governance.rs:47 deleted -> KILLED ("expected failure, got success")
each by the new test only. All three were SURVIVED at base. Vault hash
7086148b unchanged (tests only).
Sibling search: every config_is_valid / valid_asset call in programs/
vault/src: genesis.rs:170 (create_vault), :295 (upsert_asset, KILLED by
the oracle test), governance.rs:39 and :46. All now killed.
Counts: Vault 105 -> 106, programs 155 -> 156 (claims.py OK).
The execution copy (:478) is not reached by any transaction: it
parses the bytes the proposal already checked, and the bounds read
nothing else. The test comment says so (code-review fix: it had
claimed execution was covered). oracle_settings_of_a_priced_asset_are_
bounded now asserts INVALID_ASSET for each of its four cases, not any
error (all four return it; E09 stays KILLED).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that an import into a ninth mint is refused
genesis.rs:410, import_backing: a backing ledger holds MAX_SLOTS (8)
mints, and an import into a ninth is refused with NoSlot. Deleting the
check left every program test green (tasks/error-codes/mutants.log, base
cede705). From the code, without it the import goes through: the vault's
imported total grows (:414), and the seal must cover that, but the
owner's ledger is saved unchanged, so the owner has no backed claim for
it.
New test an_import_into_a_ninth_mint_is_refused: nine vaults before the
seal; one owner gets an import into eight (ledger_of shows 8 positions),
then the ninth -> NO_SLOT.
Evidence (tasks/error-codes/mutants.log, STATE=c4, full program suite):
E14 genesis.rs:410 deleted -> KILLED by an_import_into_a_ninth_mint_is_
refused ("expected failure, got success"); SURVIVED at base.
Vault hash 7086148b unchanged (tests only).
Sibling search: the only other ledger credit is deposit's
(custody.rs:128). There a full ledger is by design: the deposit is taken
unbacked (Ledger::credit's doc, ledger.rs:187-189), and credit's result is
the backed flag, not an error.
Counts: Vault 106 -> 107, programs 156 -> 157 (claims.py OK).
import_ix now calls import_into_ix, which takes the vault, so the
new test shares the one import layout (code-review fix).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin six Vault refusals no test reached
Six reachable checks had no test that failed when the check was deleted
(tasks/error-codes/mutants.log, base cede705). Each now has an assertion
on its code, and each was seen to fail with the check deleted:
E02 genesis.rs:185 create_vault of an existing vault after the seal:
ALREADY_INITIALIZED (with the check gone: 8043 GenesisOnly, so the
"says so first" order in the comment above it was untested)
E12 governance.rs:426 veto of an executed change: CHANGE_EXECUTED
(gone: the veto succeeds and marks an executed change vetoed)
E13 governance.rs:517 cancel of an executed change: CHANGE_EXECUTED
(gone: the cancel succeeds)
E16 lifeboat.rs:256 execute_exit with no exit proposed:
NOTHING_TO_EXIT (gone: 8033 ChangeNotExecutable)
E17 custody.rs:84 a zero deposit: ZERO_AMOUNT (gone: it succeeds)
E18 custody.rs:272 a zero release: ZERO_AMOUNT (gone: it succeeds; a
zero release has no unbacked part, and the limiter, whose own zero
check custody.rs:242 maps, is called only when unbacked > 0, :361)
Where: E02 in after_the_seal_a_new_vault_starts_with_its_unbacked_lane_
closed, E12/E13 in an_unvetoed_loosening_executes_after_the_delay, E16 in
the_lifeboat_stays_shut_until_the_seal (after the move in, before an
exit is proposed), E17/E18 in the new a_zero_deposit_or_release_is_
refused.
Evidence (tasks/error-codes/mutants.log, STATE=c5, full program suite):
all six KILLED by the tests named above; all six SURVIVED at base. The
vault suite is green (108 passed). Vault hash 7086148b unchanged.
Counts: Vault 107 -> 108, programs 157 -> 158 (claims.py OK).
Helpers (code-review fixes): veto_change_ix beside cancel_change_ix;
create_vault_for_ix, create_vault_ix with a given mint and token
program, so E02's case names USDC's own accounts and creates no mint.
E02's comment now repeats the program's ("whatever the seal's state"):
create_vault checks the limits first (genesis.rs:170).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin four Vault checks another layer backstops
Four checks survived deletion (tasks/error-codes/mutants.log, base
cede705) because something later still refused the case, with its own
error, so no test could tell. Each now has a test that sees the Vault's
own code, and each was seen to fail with the check deleted:
E01 lib.rs:230, create_pda's guard: a second registration of an
existing protocol (register_protocol has no check of its own) ->
ALREADY_INITIALIZED. Gone: the system program refuses it, with no
custom code (custom_code 0).
W10 lifeboat.rs:71, move_in naming another mint or token program ->
WRONG_VAULT. Gone: the token program refuses (code 3, MintMismatch).
W12 lifeboat.rs:247, execute_exit naming another mint or token
program -> WRONG_VAULT. Gone: the same token-program error.
T01 token.rs:37, account_info on an account no token program owns: a
copy of a real USDC account's bytes under the system program. Gone:
the quote SUCCEEDS (it promises a payout), and the release fails
only in the runtime at the transfer ("instruction modified data of
an account it does not own"). The quote is asserted first so the
mutant is seen at the quote (STATE=c6b).
New tests: a_protocol_registers_once, a_lifeboat_move_names_the_vaults_
own_mint (also completes a move in and an exit, as a control: the
destination receives 1,000 USDC), a_payout_goes_only_to_an_account_a_
token_program_owns.
Evidence (tasks/error-codes/mutants.log, STATE=c6 and c6b, full program
suite): E01, W10, W12, T01 KILLED by the tests named above; all four
SURVIVED at base. Full program suite green: Floor 50, Vault 111. Vault
hash 7086148b unchanged (tests only).
Counts: Vault 108 -> 111, programs 158 -> 161 (claims.py OK).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test the bounds on ticket extensions and a closed lane's code
B's review of vault-error-tests-r3 (R1) found two Vault codes no test
asserted by value that the earlier commits missed: ExtensionTooLong
and LaneClosed. Deleting the 7-day total extension cap (brakes.rs:86,
B24) left every program test green, so a live guardian, or a stolen
guardian key, could hold a queued payout back without end, 24h at a
time, against docs/05 row 11. The wide differential models the total
(differential_wide.rs:406), but its default run never extends one
ticket that far. Mapping LaneClosed to QueueFull (custody.rs:240, B23)
also survived.
The code review of the fix found a worse gap at the same check: no
test sends a negative extension (the differential draws secs from 0,
1, 3600, 86400, 86401). With `secs <= 0` weakened to `secs == 0` (A3,
brakes.rs:84) every program test passed, and one guardian key could
pull a queued payout's ETA into the past, skipping the queue delay,
and refund its own 7-day budget.
- a_ticket_is_extended_later_by_at_most_a_day_a_call_and_seven_in_total:
-1h, 0 and 24h + 1 s are each refused with EXTENSION_TOO_LONG, ETA
unchanged; six 24h extensions and one of 23h pass; 2h more is
refused, ETA unchanged; the last hour is granted; then 1 s more is
refused. The split is unaligned so that a total that ignores this
call's `secs` fails too.
- a_closed_lane_refuses_an_unbacked_payout: a vault with short_bps 0,
and one with long_bps 0, each refuse an unbacked admin payout with
LANE_CLOSED; the same payout from a vault with both buckets open is
taken (the control, so a capacity base of 0 cannot pass it).
Evidence (tasks/vault-error-tests-r4/, B's runner copied: each mutant
applied alone, programs rebuilt, whole program suite run):
- without these tests (551e519; run-without.out, run-base2.out,
run-base3.out): B23, B24, A1 (`tk.extended_s >= MAX`, this call's
secs ignored), A2 (`>=` for `>`) and A3 SURVIVED. A4 (the 24h per
call deleted), A5 and A6 (either half of limiter.rs:172's closed-lane
test deleted) were KILLED by the differential only, not by value;
- with them (run-final.out, at the branch tip): B23 KILLED by
a_closed_lane_refuses_an_unbacked_payout; B24 A1 A2 A3 KILLED by the
extension test only; A4 by it and the differential; A5 A6 by the
closed-lane test and the differential.
No program change: the runner's closing build gives 8ca8a608 /
543be60b / 7086148b, as at cede705.
Counts: Vault 111 -> 113, programs 161 -> 163 (claims.py OK).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin each half of create_pda's existing-account guard on its own
B's review of vault-error-tests-r3 (R2): each half of lib.rs:229
(`!is_data_empty() || !owned_by(SYSTEM)`) survived deleted on its own
(B20, B21), since an existing protocol account fails both, so
a_protocol_registers_once pinned them only jointly.
The test now also plants, at a fresh protocol address, an account that
fails just one half: one with data owned by the system program, one
with no data owned by another program (a fresh address). Each
registration is refused with ALREADY_INITIALIZED. The comment says no transaction is
known to make either account: data or a new owner at a PDA needs its
signature, which only the Vault gives, and create_pda gives both in one
instruction.
Evidence (tasks/vault-error-tests-r4/): at 551e519 B20 and B21
SURVIVED (run-base2.out); at the branch tip both are KILLED by
a_protocol_registers_once only (run-final.out).
No count change (claims.py OK: vault_programs 113, programs 163).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Record the Vault refusal deletions and what each became
docs/09 §3 gains a row for the deletion run this branch answers
(tasks/error-codes/mutants.log): 38 refusals, across 14 of the 18 Vault
codes no test asserted by value at cede705, each deleted on its own
with the whole program suite run against it, on the AUD-2 build.
The 18 (each VaultError variant with no `verr::` use in programs/tests
at cede705): AlreadyInitialized NotRegistrar WrongVault
WrongTokenProgram AlreadySealed VaultCountMismatch InvalidLimits
InvalidAsset LaneClosed TicketNotPending ExtensionTooLong
ChangeExecuted LedgerUnsound LedgerCorrupt NoSlot NothingToExit
ZeroAmount Overflow. The run missed LaneClosed and ExtensionTooLong;
B's review deleted one check of each (B23 custody.rs:240, B24
brakes.rs:86), and both survived until 1797ae5. That commit
also pins a negative extension (A3, which survived every test before
it), the 24h per call, and a lane closed by either bucket.
At base: 13 KILLED, 25 SURVIVED. The commits before this one add tests
that kill 18 of the 25 and both of B's, each asserting the Vault's own
code (W02 W06, E04 E05, E06 E07 E08, E14, E02 E12 E13 E16 E17 E18, E01
W10 W12 T01; B23 B24). The row names the 7 that still survive and why:
- six guards no transaction is known to reach: lib.rs:324 (vault PDA
address; create_vault creates a vault only at its PDA), genesis.rs:519
and :523 (the lifeboat, checked by expect_pda first, created only by
lifeboat_move, move_in requires the seal, lifeboat.rs:53, and the
seal happens once, so no lifeboat holds data at it), brakes.rs:50 and
custody.rs:541 (a pending-ticket count below zero), token.rs:235
(GetAccountDataSize's return data from an already-checked program);
- custody.rs:399, stated as not shown unreachable: properties.rs:588-592
debits backed_amount and one more on random ledgers, and only
crates/bulkhead-core/tests/units.rs:375 debits less, on a one-mint
ledger with no liability. properties.rs:606 asserts backed(a) ==
min(a, backed(MAX)) for requests below 2^52 (:613), so any smaller
amount is backed too, and debit_backed(b) refuses only when
backed_amount(b) != b (ledger.rs:289). The known exception is a
collateral value that overflows u128 (ledger.rs:262): the valid
amounts need not form an interval there, so the consent or freeze
cap (custody.rs:337, :347) can lower the amount to an invalid one and
:399 refuses the release (fail closed). The search's own result is
always valid. Not verified by a tool beyond that property test. (A full ledger cannot reach :399 through
set's NoSlot: backed_amount's valid() refuses a new liability slot
on a full ledger, ledger.rs:237.)
Not deleted, as the row says: the two core-error mappings to
ChangeExecuted and ZeroAmount, token.rs:233, and the codes Overflow and
LedgerCorrupt (checked arithmetic, decoding).
After this branch five codes are still asserted by no test (same grep
at this commit): NotRegistrar (E03, genesis.rs:108, was KILLED at base
by another test), TicketNotPending, LedgerUnsound, LedgerCorrupt,
Overflow.
Docs only: no count change (claims.py OK: vault_programs 113, programs
163).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Run the exclusion check's main in its self-test
A's review of finding 105 (REVIEWS/auto-B-mutants-exclusions-r7.md,
finding 1, Low): CI's `mutants` job acts on main's exit status, but the
self-test called problems() only, and check.sh never runs the tool on
real lists. With main returning 0 on a mismatch (mutant A10), every
in-tree check passed, and CI would have printed the problems and gone
green. This was not a defect at fb771c6, since the job fails today. The
gap was that a later regression would pass.
Fix: main takes its arguments and, for the self-test, a config path
(default: the repo's .cargo/mutants.toml, as before), and its lines name
the config it read. The self-test runs main in-process on a planted
config and checks the exit status and every line on each stream:
- the right lists: exit 0, "1 of 3 mutants excluded" on stdout, nothing
on stderr;
- the lists the other way round: exit 1, nothing on stdout, and exactly
the count and "full list lacks" problems, so KEPT is read as KEPT;
- coloured lists: exit 1 with exactly one problem, so a main that fails
only on several problems is caught;
- one argument and three: exit 2 and the two usage lines.
One subprocess (python3 -I, 60 s timeout, in the temporary directory so
the config must be found from the tool's own path) runs the command line
with two lists and the repo's config, as scripts/mutants.sh does: on a
list no pattern can match, it exits 1 with an empty stdout and problem
lines that each name .cargo/mutants.toml. Which problems those are
depends on the config, so only their form is checked. Self-test: 16
cases -> 22. No doc states the number.
Evidence (tasks/exclusions-main-exit/):
- old-A10.log: the fb771c6 self-test with A10 applied prints "self-test
OK: 16 cases" and exits 0.
- mutants-r6.log (mutants.py applies each mutant to a scratch copy of
the tool, which reads the real config through a symlink): unmutated,
"self-test OK: 22 cases", exit 0. All 18 are KILLED (exit 1): A10
(return 0), A10b (fail only on more than one problem), A0 (always 1),
U1 (usage returns 0), U2/U3 (argument check `< 2`, `< 1`), K1 (KEPT
ignored), K2 (ALL read twice), C1 (planted config ignored), C2 (no
default config), R1 (the default config read from the cwd), E1/E2
(sys.argv sliced as [:] or [1:2]), P1 (problems not printed), S1
(problems on stdout), L1 (the planted config not named), N1 (only the
first problem printed), G1 (the success line printed on failure too).
Code-review passes at high shaped this. The first version ran three
subprocesses against the real config, and A0, K1 and U3 survived it.
The second left E2, C2 and G1 alive. Declined from the third: catching
exceptions in the self-test (a traceback already exits 1) and moving
the config hook out of main. The fourth showed that requiring one
"matches 0 mutants" line per live pattern tied the self-test to the
config's contents. The subprocess now checks the form of the lines
only, and the 17 stay killed. The fifth showed that R1 survived while
the child ran in the repo root (it does in check.sh); now it runs in
the temporary directory. Its other points repeat declined ones.
- real-lists.log: on the real lists, made with scripts/mutants.sh's
commands: ".cargo/mutants.toml: 22 of 795 mutants excluded, each
pattern matching its count", exit 0.
- python3 -I tools/mutant_exclusions.py --self-test: 22 cases OK.
- scripts/check.sh: ALL CHECKS PASSED (check-final.log);
bin/check-each-commit.sh green (each-commit.log).
Sibling search: the other tools check.sh runs (tools/claims.py,
tools/floor_check.py) also gate CI through main's exit status. claims.py
has the same gap: check.sh runs it on the real tree, where it finds no
problems, and its self-test calls self_test(), not the path in main that
returns 1. With that return changed to 0, `--self-test` and the real run
both still exit 0 (claims-C1.log). Not changed here: it is its own
commit (BACKLOG-B). floor_check.py runs in CI only as --self-test, whose
result main returns as is. Its RPC mode is an operator's tool that no CI
job runs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Record finding 106: Vault liveness and exit rules had no failing test
The tests landed in 944f5e8 (lifeboat-exit-tests-r3, 15 Oct). Their
09 §4 row waited for 104 and 105 so the list stays numbered without
gaps; 105 is now on integration (fb771c6), so the row lands here.
Docs only: 09 §4 row 106; README and 12's count and post-submission
range (96–106, 106 fixed). Two code-review passes changed the row as
written on 15 Oct: a sentence for today's re-run, the nine
single-guardian sites named, G5 quoted verbatim, and the exit veto
attributed to T18 (G6 has no veto), as is the veto test's doc comment
(vault.rs:3996, comment only).
Evidence, re-run today on bf8ba41 (the row's claims are from 15 Oct,
and AUD-2 changed the Vault since): B's re-review list of 33 line-
neutral mutants (the ten rules, the nine single-guardian sites L1-L9,
the boundaries, the six quorums, R1-R6), each applied, programs
rebuilt, all program tests run:
33 KILLED, 0 survived, 0 not applied, 0 build failures. Vault
hashes after the run match the tip (Vault 7086148b, Floor 8ca8a608,
Harbor 543be60b).
Log: autopilot tasks/finding-106-row/mutants.log.
- python3 -I tools/claims.py: OK, findings 106.
- git grep for "96–105", "105 findings": no other stale range.
- scripts/mutants.sh still mutates the core only (SCOPE, line 11), so
the row's last sentence holds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Cite PR #100's tree, not the unpublished bf8ba41, in finding 106
Finding 106's row in docs/09 said its 33 mutants were re-run on 16 Oct
at bf8ba41. That commit is not an ancestor of this branch or of the
stack above it and is on no remote branch, so the citation would point
at nothing once the stack lands. The owner's review of #99-#109 found it.
bf8ba41 and 86b777b (the head of PR #100, vault-error-tests-d2, the D2
replay of vault-error-tests-r4) have the same tree, 36187b49
(`git rev-parse bf8ba41^{tree} 86b777b^{tree}` prints it twice). So
the run was a run on 86b777b's tree, and the row now says "on tree
36187b49, the tree of 86b777b (PR #100's head)". Why not just "at
86b777b": it was committed on 17 Oct, after the run; and main
squash-merges (1045cd9 has one parent), so after PR #100 lands 86b777b
is reachable only as GitHub's refs/pull/100/head. The tree id stays
checkable from main: squashed bottom first, main's commit for PR #100
has that tree. "PR" keeps "#100" from reading as finding 100, the
row above. These points came from two code-review passes.
Sibling search: every 7-12 hex token in tracked .md/.py/.sh/.mjs/.rs/
.toml files at autopilot/integration 3e9f58d that names a local commit
(51 tokens) was checked with git merge-base --is-ancestor against
integration. bf8ba41 was the only one that is not an ancestor.
Docs only. tools/claims.py: claims OK (26 files).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Warn about a Floor guardian set short of the margin (finding 107)
The guardian checks the Vault protocol's guardian set against finding
107's margin (main.mjs checkGuardianSet), but never a Floor config's.
A Floor set registered before the rule, say 2-of-3, keeps working and
keeps AUD-2's exposure: one lost key and one rogue guardian lock its
governance for good. The guardian said nothing, and docs/07 §6 said so
("not yet a Floor set").
The rule moves to policy.mjs as guardianSetWarning, a pure function:
the same text as before, a warn line that alerts on mainnet only. In
main.mjs, checkGuardianSet now remembers the set shape per set
("protocol", or the protected program), and checkFloor calls it for
each watched Floor config it reads. floorAlerts is unchanged. The
margin is not a readiness code there: "floor not ready" alerts on
every cluster, and only the owner's new set clears a margin problem, so
"floor ready" would never be logged again.
Evidence (tasks/floor-guardian-margin/r2/):
- node --test cloud/test/*.test.mjs: 180/180 pass (one new test).
- Mutants of guardianSetWarning, each run on policy.test.mjs and
reverted; each fails the new test: N1 never warn, N2 threshold fixed
at 2, N3 notify always, N4 notify never, N5 GUARDIAN_MARGIN = 1.
- tools/claims.py: OK, client 180 (docs/09:40 updated).
- Not tested: the main.mjs wiring (the per-set Map and the checkFloor
call). No test runs checkFloor; deleting the call leaves every test
green.
Sibling search: guardianMarginProblem's callers are vault-check.mjs (an
info line) and guardianSetWarning. The status snapshot
(snapshot.mjs:33) prints live guardians and the quorum, not the margin;
left as it is.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Judge each guardian set's margin on its own, and say when it is fixed
The parent's per-set memory (which set shapes were already reported)
lived in main.mjs, so no test could fail on it: dropping the
once-per-shape check, or keying every set the same, left every test
green. The warning did not say which set it was about (only a program
tag on Floor lines), and it gave the same text for a set ejected down
to its threshold, or registered n-of-n before finding 96. There one
rogue guardian alone locks governance; a lost key alone does not, since
dead guardians cannot veto the owner's new set (docs/16 §4.4), but it
leaves no quorum until that set executes. Nothing was said when the owner's
new set fixed the problem. And a delivered alert did not say which Floor
program it was about: the program went only into the line's `program`
field, which alertFor uses in its repeat key but does not show.
Moving the program name into the text is a trap: alertFor keys repeats
on the first 60 characters of the text, and "the Floor guardians of
<44-character id>" pushes the shape past them. A 2-of-3 set ejected to
2-of-2 within ten minutes would then give the same key, and the worse
warning would not alert. So the program goes after the shape: "the
Floor guardians are 2-of-3 for <program>: ...". The `program` field
stays for the repeat key.
guardianSetWarning becomes guardianSetCheck(shapes, which, set,
mainnet) in policy.mjs. It keeps each set's last shape and whether it
was short in the Map, keyed by "protocol" or the protected program, and
returns the line to log or null. The warning names the set and has its
own text for n-of-n. When a set that was short gets the margin, it
returns one line saying so, which alerts on mainnet like the warning,
so whoever was paged hears that it is fixed. main.mjs only logs what it
returns. docs/07 §6 says so. The advice also had a hole: a set
registered as 7-of-8 was told to "run at least 7-of-9", and no set can
have 9 keys (core MAX_GUARDIANS is 8). When threshold + 2 is over 8,
the advice is now to lower the threshold to at most 6-of-8. That needs
MAX_GUARDIANS in JS, so lib/solana.mjs gets it next to
GUARDIAN_MARGIN; the next commit ties both to the core.
Evidence (tasks/floor-guardian-margin/):
- node --test cloud/test/*.test.mjs: 180/180 pass (the test is
extended, not added).
- 19 mutants, each run on policy.test.mjs and reverted (r6/mutants.py,
r6/mutants.log, r6/N5-margin1.log). Each fails the extended test, and
only it: never warn, threshold fixed at 2, notify always, notify
never, GUARDIAN_MARGIN = 1, no once-per-shape, one Map key for every
set, the n-of-n text for threshold + 1, no fixed line, a fixed line
for a set that was never short, the fixed line never alerting, no
program tag, every set named as the protocol's, the short flag not
stored, the program before the shape, the program left out of the
text, `need < 8` for `need <= 8` (6-of-7), the 8-key limit ignored
(7-of-8), 7-of-8 as the lowered advice.
- The mainnet alerts are checked on their own in a scratch copy of the
test without the earlier text assertions (r5/C1-C3-alone.log): with
the program left out of the text, the alert's detail check fails;
with a long name pushing the shape past character 60, "2-of-2 alerts"
fails.
- A's review of the previous version (branch
auto/B/floor-guardian-margin-r7, kept) found two survivors there: X3,
the shape keyed on the count only, and X5, the advice
threshold-of-(n+1). It also found that the n-of-n text said a lost key
alone locks governance. The test now has a threshold-only fix (7-of-8
-> 6-of-8, the advice's own step) and pins the n-of-n text and advice.
r9/mutants.log: X3, X5, the r7 n-of-n text and the text without its
lost-key clause are all KILLED. r9/mutants-r7.log: X3 and X5 SURVIVED
on the r7 code and test. The 18 mutants of r6/mutants.py, re-run on
this code: all KILLED (r9/r6-mutants-rerun.log). Two code-review
passes on the fix (kept as -r8 and -r9) added the lost-key clause: a
lost key alone leaves no quorum (it never approves, and core `quorum`
needs threshold approvals) until the owner's new set executes. A third
pass (-r10) fixed the comments' wording only.
- tools/claims.py: OK, client 180.
- Still not tested: the checkFloor call in main.mjs (no test runs
checkFloor).
Not changed (code review): the "fixed" line needs the short shape to
have been seen since the guardian started (the Map is in memory, as the
old per-process shape was; docs/07 says so). A protocol that abdicated
SetGuardians can't take a new set, so the advice to run 2-of-4 can't be
followed there; REPORT lists that case for the owner. The fault text
assumes a margin of 2, the owner's decision (16 §4.4).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Tie the JS guardian margin and set size limit to the core constants
cloud/lib/solana.mjs keeps its own GUARDIAN_MARGIN = 2, which the
guardian and the deploy tooling use to judge a guardian set, and now
MAX_GUARDIANS = 8, which the guardian's advice uses. Their comments say
they are core governance.rs's constants, but nothing checked that. Had
the programs' margin moved to 3, every JS test would have stayed green
while the guardian passed sets the programs refuse.
deploy.test.mjs now reads governance.rs and requires exactly one
`pub const GUARDIAN_MARGIN: u8 = N;` and one `pub const MAX_GUARDIANS:
usize = N;`, each N equal to the JS constant. It reads the source text,
like the build-script test above it. If a declaration changes form, the
test fails rather than passing.
Evidence (tasks/floor-guardian-margin/r6/tie-mutants.log):
- node --test cloud/test/*.test.mjs: 181/181 pass (one new test).
- Mutants of governance.rs, each reverted with its mtime kept (else the
limiter differential asks for a worker rebuild): GUARDIAN_MARGIN = 3,
MAX_GUARDIANS = 9, and a trailing comment on MAX_GUARDIANS. Each fails
the new test, naming the constant. deploy.test.mjs is green again.
- tools/claims.py: OK, client 181 (docs/09:40 updated).
Sibling search: the other JS copies of core rules (the limiter port,
the account layouts) are checked by the differential and by decoding
real accounts. MIN_GUARDIAN_THRESHOLD has no JS copy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Reach the 7-day extension total in the wide differential
differential_wide.rs drew extension seconds only from {0, 1, 3600,
86400, 86401} on a random ticket and lumped every refusal into one
counter. A ticket never got near the 7-day total, no negative value was
sent, and a refusal was not checked by its code. So the default run
could not tell the total bound, a negative extension or the error code
from a bug: only the unit tests at vault.rs:3382-3401 pinned them.
Now one extension draw in three pushes the chosen ticket by up to 24h
a call to exactly the 604,800 s total, then asks for 1 s more, so the
edge itself is hit on every push, not only when earlier draws line up.
The other draws are half legal sizes (1, 3600, 86400) and half out of
range for one call (i64::MIN, -86400, -1, 0, 86401, i64::MAX). Each
refusal must fail with EXTENSION_TOO_LONG. The counters are split into
extend (a random legal size), extend_push, extend_refused_call and
extend_refused_total, all required >= 10, so pushes cannot stand in for
the random sizes.
The model reads both bounds from bulkhead-core (consts.rs:37-38), as
the module says it is built, rather than restating 86,400 and 604,800.
docs/09 §7's row says so, and the 16 Oct "Vault refusals deleted" row,
which says the default run never extended a ticket that far, now says
it does since this change (its differential_wide.rs:406 cite had drifted).
Default run (24 seeds x 250 steps): extend 16, extend_push 126,
extend_refused_call 16, extend_refused_total 30 (was extend 40,
extend_refused 27); every other required counter still >= 10
(loosen_executed 12, as before).
Evidence: mutants of brakes.rs extend_ticket, each built and run against
the old and the new differential_wide only:
T1 drop the total check old SURVIVED new KILLED
T2 total > to >= old SURVIVED new KILLED
T3 drop secs <= 0 old KILLED new KILLED
T4 secs <= 0 to < 0 old KILLED new KILLED
T5 total ignores this call's secs old SURVIVED new KILLED
T6 extended_s += secs / 2 old SURVIVED new KILLED
T7 refusal returns WrongTicket old SURVIVED new KILLED
T8 secs <= 0 to == 0 (negatives) old SURVIVED new KILLED
T9 total > to > total + 1 old SURVIVED new KILLED
(T9: a draft that pushed by 24h until refused killed it only through
chance 1 s and 3600 s draws on the same ticket; the push to exactly
604,800 then 1 s more reaches the edge every time.)
sBPF sha256 after the run unchanged (Floor 8ca8a608, Harbor 543be60b,
Vault 7086148b): tests only.
Sibling search: the other refusal counters in this file (reset_refused,
lifeboat_refused_*, loosen_*) are each split by cause already.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check settle and lifeboat refusals by code in the wide differential
The same gap as the extension refusal, found in the sibling search for
the previous commit: a refused settle and the three lifeboat-move
refusals asserted only r.is_err(). Any refusal passed, so the order of
the checks and the program's own codes went unchecked by the
differential. Settle's comment (custody.rs:520-521) says it returns its
own error, not the token program's, when the vault is short; nothing in
this file held it to that.
Now a settle refused while frozen must fail with FROZEN, and one that is
only short with INSUFFICIENT_BALANCE (frozen is checked first). A
lifeboat move inside the delay after the seal must fail with NOT_SEALED,
one while frozen with FROZEN, and one beyond the balance with the token
program's InsufficientFunds (1). The two settle refusals get their own
counters: settle_refused_frozen is required >= 10 like the rest;
settle_refused_short needs lifeboat moves to drain a due ticket's
tokens first and occurs 7 times in the default run, so it is required
>= 1.
Evidence: mutants of programs/vault/src, each built and run against the
previous head's differential_wide and this one, nothing else:
S1 settle: drop the InsufficientBalance check old SURVIVED new KILLED
S2 settle: balance check before the freeze old SURVIVED new KILLED
L1 require_sealed returns Frozen old SURVIVED new KILLED
L2 move_in's freeze returns NotSealed old SURVIVED new KILLED
The token-program code (1) on a move beyond the balance is pinned but
not shown red by a mutant. sBPF sha256 unchanged (Floor 8ca8a608,
Harbor 543be60b, Vault 7086148b): tests only.
Deep run of both commits together (BULKHEAD_PROGRAM_SEEDS=600
BULKHEAD_PROGRAM_STEPS=500, 85 s): 300,000 steps, all identical;
extend 876, extend_push 6,462, extend_refused_call 1,175,
extend_refused_total 1,453, settle 2,024, settle_refused_frozen 921,
settle_refused_short 566, lifeboat refusals 3,854 + 5,768 + 1,171. docs/09 §7's
row gets that run as a 16 Oct result next to the 5 Oct one, which
predates these checks.
Sibling search: in differential_wide.rs, reset (TICKETS_PENDING) and
execute change already asserted codes, and a release or quote refused
for want of tokens asserts INSUFFICIENT_BALANCE. A release or quote the
limiter refuses (frozen, lane closed, queue full) is still compared only
as a refusal: mapping the core's refusal to the program's code is left
for its own change. Lifeboat returns only send legal amounts. docs/09's
row now says which refusals are checked by code. differential.rs:327
has the same is_err settle check; left as it is, since the wide run now pins both
settle codes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check limiter refusals by code in the wide differential
A release or quote that the limiter refused (lane closed, queue full)
was compared with the model only as Err. Any refusal code passed: a
program that swapped LaneClosed and QueueFull in custody.rs's
limiter_error, or checked the backed part against the balance before
the limiter and so answered InsufficientBalance for a full queue,
still matched the model. About 670 such refusals happen in a default
run.
The model now maps core's LimiterError to the program's code with
verr::limiter in the test harness (all four variants, as custody.rs's
limiter_error does), and every refused quote and release must fail
with the model's code. Lane-closed and queue-full refusals have their
own counters, each required at least 10 times; a refusal code with no
counter panics rather than being counted as want of tokens. docs/09
row 362 says so and records the deep run.
Evidence (autopilot tasks/differential-limiter-codes/):
- mutants.log: C1 (LaneClosed and QueueFull swapped in
limiter_error), C2 (QueueFull -> InsufficientBalance), C3
(LaneClosed -> ZeroAmount) and C4 (backed > balance checked before
the limiter, same line) each SURVIVED the old test and are KILLED
by the new one. Hashes back to 8ca8a608/543be60b/7086148b after.
- c4-full-suite.log: under C4 the whole program suite (Vault 113,
Floor 50, the fixed differential) passes; only this test fails.
C1-C3 are also caught by vault.rs's code tests (:434, :3428).
- Default run: refused_lane_closed 254, refused_queue_full 414,
insufficient 118.
- deep-600x500.log: 300,000 steps, 15,561 lane closed, 18,853 queue
full, 6,345 want of tokens, all identical.
Sibling search: every other Err arm in differential_wide.rs already
asserts a code (extend, settle, reset, lifeboat in, execute change).
Not here: lifeboat returns are never refused (backlog), and lifeboat
move_in beyond the balance fails with the token program's code
(backlog, a program change).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check release refusals by code in the fixed differential
differential.rs reduced every refused release to Err(()) on both
sides, so a program that refused for the wrong reason still matched
the model. The same gap the previous commit closed in the wide
differential (found by its code review).
The model's refusal now carries the program's code: verr::limiter for
a limiter refusal, INSUFFICIENT_BALANCE when the vault holds less
than it would pay. A refused release must fail with that code, and
the assert prints the program's error. Refusals for a full queue get
their own counter, required at least 10 times like the others. Want of
tokens and a closed lane are counted and printed but not required:
the wide differential requires both. A refusal with any other code
panics, as in the wide differential, so the counts always add up.
docs/09 row 361 says so; its undated figure (4,843 releases) no longer
matched a run, so it is replaced by a dated one.
Evidence (autopilot tasks/differential-limiter-codes/):
- mutants-narrow.log (mutants.sh <wt> differential): C1 (LaneClosed
and QueueFull swapped), C2 (QueueFull -> InsufficientBalance) and C4
(backed > balance checked before the limiter) SURVIVED the old test
and are KILLED by the new one. C3 (LaneClosed -> ZeroAmount)
survives both: no release in this test's runs is refused for a
closed lane. The wide differential kills it, and docs/09 says so.
Hashes back to 8ca8a608/543be60b/7086148b after.
- r2/default-run.log: 1,000 steps, 82 releases refused, all for a
full queue (want of tokens 0, lane closed 0).
- r…
shuv-amp
added a commit
that referenced
this pull request
Oct 18, 2026
* Run the tests on 2-of-4 guardian sets; test recovery from two faults
Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set
with fewer than threshold + 2 keys. Every test fixture and most tests'
own sets were 2-of-3. This moves them to 2-of-4, which today's rule
also accepts, so this commit changes no program: Floor, Vault and Harbor
hash 508e5a99 / b07655d9 / 543be60b, as before.
- Fixtures: Protected (Floor) and Market (Vault, through Harbor) now
hold 4 guardians. Each test that built its own 3-key set gets a 4-key
set with the same threshold and the same assertion.
- Two tests changed meaning with a 4th key and were fixed, not loosened:
a_quorum_ejects_a_rogue_guardian_who_vetoes_everything (Floor) and
a_quorum_ejects_a_guardian_who_freezes_forever (Vault). With 4 keys,
the last eject was refused only because the target's own vote is
masked. Both now eject a second guardian (a set an ejection shrank to
threshold + 1 still ejects) and then show the 2-of-2 refusal.
- Account sweeps: an eject's signers may now be any two of the three
others, so each of those two slots is free for the third, never for
the target or the other signer again. Vault 5,365 and Floor 1,710
substitutions, refused (docs/09, dated 16 Oct). Listing all three
others at each slot would have skipped the duplicate: with a mutant
that counts a guardian listed twice as two (governance.rs:270), the
sweeps listing all three pass, and these fail at "eject: account 2
replaced by key ..." (Vault and Floor). No other program test kills
that mutant (tasks/aud2-guardian-margin-impl/dup-mutant-*.log).
- floor_differential: the guardian pool is 6 keys, 4 registered; the
proposed sets are 2 to 6 keys.
New tests (pass on both rules; they show what the margin buys):
- a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_set (Floor): g0's
key lost, g1 rogue stops the program; the lost key does not help a
quorum; the honest pair ejects g1 and restores; neither can veto the
owner's refill, which executes after 72h.
- two_rogues_in_turn_are_both_ejected_from_a_two_of_four_set (Floor):
the second rogue vetoes the refill and is ejected from the 2-of-3.
- a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_vault: the same in
the Vault (freeze, eject, unfreeze, refill).
Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest
and quote sweep): the counts and the sweep figures here are from runs
there (tasks/aud2-guardian-margin-impl/r4/). The E2E run, the mutants
and the new-rule run below were made before, on e8640a7; this commit's
programs and core are the same as there but for one comment line in
views.rs (line-neutral, same hashes).
Evidence:
- E2E=1 scripts/check-programs.sh: PROGRAM CHECKS PASSED (Floor 49,
Vault 98; S1 and E2E passed; the resumed setup found all 14 steps;
vault check, Floor demo and deploy dry run passed), with the next
commit's deploy rehearsal in the tree.
- tools/claims.py: OK (floor 49, vault 104, programs 153).
- Under the new rule (core from two commits on, programs rebuilt), only
the two tests that state the old rule fail: an_n_of_n_guardian_set_is_
refused (accepts 3-of-4) and the Floor differential's model.
- Mutants (tasks/coverage-map/mutants.sh), each red, then restored:
AV1 Floor live_guardian ignores liveness: KILLED by a_lost_key_and_a_
rogue_do_not_lock_a_two_of_four_set (+2 others).
AV2 the Vault's: KILLED by ..._two_of_four_vault (+2 others).
AQ1 Floor restore quorum ignores liveness: KILLED by ..._two_of_four_set
and the differential only.
AQ2 the Vault's unfreeze quorum: KILLED by ..._two_of_four_vault and
a_silent_guardian_does_not_count_toward_a_quorum.
AE1 can_eject refuses at count == threshold + 1 (AUD-2 option (a)):
KILLED by two_rogues_in_turn_..., and the two eject tests above.
Hashes after: 508e5a99 / 543be60b / b07655d9.
Split on 16 Oct from ec0cacc at the owner's review (D2): the deploy
rehearsal's move to 4 guardians is now the next commit. programs/ and
programs/tests/ here are byte-identical to ec0cacc, so its runs below
hold for this commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Rehearse deploys with four guardians, and read only the keys there are
Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set
with fewer than threshold + 2 keys, so the deploy rehearsal must
register 2-of-4 sets. Split from ec0cacc at the owner's review (D2);
the text is that commit's, unchanged:
Deploy rehearsal (cloud/deploy market.mjs, vault-check.mjs,
floor-demo.mjs): 4 guardians (keys/guardian3), so the dry run keeps
working once the rule lands; one more funding step (14, was 13).
The devnet market has 3 guardian files. A first version of this change
loaded guardian3 everywhere: vault-check.mjs failed there with ENOENT,
and heartbeat silently created a key nobody holds as a guardian. Now
market.mjs guardianKeys(dir, create) reads guardian0, 1, ... and
throws on a gap (never fills it with a new key). Setup creates keys up
to 4 only while the protocol is not registered, so resuming an old
market adds no key the chain lacks; a new deploy-dryrun step resumes
the dry-run market with guardian3 moved aside and fails if setup makes
it again (with setup always creating: "DRY RUN FAILED: resuming a
registered market made keys/guardian3.json", r3-dryrun-J7.log).
renew-guardians tops up to 4 and logs each new key (unfunded); a
2-of-3 renewal would be refused once the rule lands. keysIn reads the
guardians only when a command uses them, so caps and set-recovery
never need them; heartbeat says a key is not in the set rather than
dead. One THRESHOLD (2) for the register and renewal payloads and
MIN_GUARDIANS. vault-check.mjs
compares the chain's set with the first files, and liveness with the
chain's count. An extra file (a renewal not executed yet) and a set
without the margin are info lines, not failures, since such a set
keeps working. GUARDIAN_MARGIN and guardianMarginProblem live in
cloud/lib/solana.mjs, used by these and by the guardian bot's existing
warning (cloud/guardian/main.mjs), and vault-check uses THRESHOLD. The e2e scripts fund guardian3 too. Three
tests in cloud/test/deploy.test.mjs (client tests 176 -> 179). JS
mutants (js-mutants-r3.log), each red then restored: always create,
never create, no gap check, margin 1, `>`, GUARDIAN_MARGIN = 1.
Setup always creating survives these tests and is killed by the dry
run step above.
cloud/, scripts/deploy-dryrun.sh and docs/09's client-test row are
byte-identical to ec0cacc (git diff empty), so its runs hold for them:
the E2E check-programs run with the dry run (r4), and the JS mutants.
tools/claims.py: OK (client 179).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Refuse a guardian set without two keys beyond its threshold (107)
AUD-2, decided by the owner on 16 Oct (option (e) of 16 §4.4). A set
stays live while its honest live guardians reach the threshold: they
eject every rogue in turn, and dead guardians cannot veto the owner's
new set. Finding 96 asked for one key beyond the threshold, so a 2-of-3
or 3-of-4 set survived one fault but not a lost key plus a rogue: the
honest guardian left is no quorum, nobody can be ejected, and the rogue
keeps Floor stopped (or the Vault frozen) and vetoes every new set.
Fix, in the core so both programs inherit it through GuardianSet::parse
(registration, and every SetGuardians when proposed and when executed):
GUARDIAN_MARGIN = 2
guardian_set_ok := count <= 8 && t >= 2 && count >= t + 2
(saturating_add: t is a payload byte). A stored set is not checked
again, so existing 2-of-3 sets keep working, and ejection is unchanged:
a shrunk set still ejects. A 2- or 3-key set is BadThreshold, not
BadCount; the count rule is only the bitmap size, and the error order
of finding 103 is kept.
This changes both programs, by intent:
Floor 508e5a99 -> 8ca8a608, Vault b07655d9 -> 7086148b,
Harbor 543be60b (unchanged). Darwin-arm64 builds.
Compute (scratch LiteSVM test, tasks/aud2-guardian-margin-impl/cu.md):
-3 CU on every path that parses a set (Floor propose 2035 -> 2032,
execute 1682 -> 1679; Vault register 9592 -> 9589, SetGuardians 2979 ->
2976; Floor register varies with its PDA bump, residue also -3).
Proof and tests:
- Kani: accepted_sets_can_eject_anyone restated (ejection reachable
exactly when count > t, so option (a) cannot slip in; accepted sets
have >= 4 keys and a margin of at least 2, and 2 is accepted). New:
an_honest_live_quorum_ejects_any_rogue (any slots, anyone else alive)
and accepted_sets_survive_any_two_faults (any two faulty keys, lost
or rogue, leave a live honest quorum, which ejects each rogue, and the
rogues alone eject no honest guardian). That holds only short of a
rogue quorum: two rogues in a 2-of-n set are one (out of scope, 04
T7). With the assumption rogue < t dropped, the harness fails at
`!can_eject(rogue, ..)` (kani-no-rogue-assume.log), so the docs say
"short of a rogue quorum", not "any two faults".
19/19 verified, 240 s (cargo kani -j 2, as CI); again after the
restatement, 235 s summed (r2-prove.log).
- Kani mutants (kani-mutants.sh): margin 1, margin 3, `>` for `>=`,
t >= 1, count <= 9, option (a), and an eject one vote short (only
the two-fault harness kills that one): each KILLED by a harness
(kani-mutants-r2.log). The eject
ignoring liveness is the eject harness's case, not these.
- Program mutants (coverage-map/mutants.sh, rebuilt each time):
margin 1: KILLED by a_guardian_set_without_two_spare_keys_is_refused
(Floor and Vault), registration_rejects_weak_settings and the Floor
differential. Margin 3 and `>`: KILLED by most of the suite (the
fixtures are 2-of-4). t >= 1: first only the Floor differential; the
1-of-2 registration case was now refused by the margin too, so it is a
1-of-4 here and a Vault 1-of-4 case was added: then KILLED by both.
count <= 9: SURVIVES the program tests, equivalent there (parse refuses
more than 8 keys first), KILLED by Kani. Option (a): KILLED by the
eject tests two commits back (the tests on 2-of-4 sets).
- Tests: the Floor and Vault refusal tests above cover proposal and
execution (a stored 2-of-4 proposal whose threshold turns 3 is refused
when executed, as the devnet renewals proposed before this rule will
be); registration_rejects_weak_settings gains 2-of-3 and 3-of-4.
an_n_of_n_guardian_set_is_refused (Vault) is renamed and accepted a
3-of-4: it now refuses it and accepts 3-of-5.
- Model: the Python rule, the Floor differential's model, and the
guardian_set fuzz target state it again; the GP generator now also
picks count - 2. With the Python rule put back to the old one, seed 11
(20,000 cases) reports 5 GP mismatches; restored, seeds 11-13 identical.
Fuzz seeds two_of_three_refused, two_of_four, six_of_eight, and
seven_of_eight renamed seven_of_eight_refused (byte 323, the
threshold, is 7: now refused). With margin 1 in the core the target
crashes within 60 s; restored, 222,615,288 runs in 601 s, no crash.
- scripts/check.sh: ALL CHECKS PASSED, and E2E=1 check-programs.sh:
PROGRAM CHECKS PASSED, both at the branch tip, whose code is this
commit's (the next commit is docs only; tasks/aud2-guardian-margin-
impl/r4/check.log, r4/check-programs-e2e.log). claims: kani 19,
floor 50, vault 104, programs 154, client 179.
Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest
and quote sweep). The core and the program sources are the same as
before the rebase but for one comment line in views.rs (line-neutral),
so the Kani, fuzz, differential and mutant runs above were made on the
pre-rebase commit; the hashes, check.sh, check-programs and the counts
are from r4 runs (tasks/aud2-guardian-margin-impl/r4/).
The finding 96 row said the restated harness proves "exactly"; it now
says what it proved then and proves now, and gives the Vault test's new
name. The finding 107 row waits for rows 105 and 106 to land (09 §4 is
numbered 1..n); its text is ready. The rule's prose is the next commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* State the guardian margin in the docs, with its costs (AUD-2)
The previous commit made both programs refuse a new guardian set
without two keys beyond its threshold (finding 107). The docs that
state the rule now say so:
- README: guardians 4-8 keys; the G5 row and the "who the guardians
are" paragraph give the reason (any two faults leave a quorum, unless
the rogues are one themselves: two in a 2-of-n set, 04 T7).
- 05 register_protocol: 2 <= threshold <= count - 2, also for every
SetGuardians, proposed and executed. 05 abdicate: abdicating
SetGuardians fixes the set, so ejections shrink it for good.
- 06 state table, 00 D22, FAQ: the same rule.
- 07 §2: 4-8 guardians, and don't abdicate SetGuardians. 07 §6: the
programs enforce the margin; sets registered before keep their
exposure until replaced; an ejection uses up a spare key until the
owner's new set executes; a lost key counts as live until its
interval passes.
- 16: the AUD-2 row is decided; §4.4 gives the decision, why the
margin is the number of faults a set survives (short of a rogue
quorum), the options rejected
and why (the honest-owner and stolen-owner cases look the same
on-chain), what it costs, what it leaves (a rogue quorum; a stolen
owner key with every guardian dead; a rogue's veto of the recovery
key's rotation, checked: Floor's veto has no recovery exception, as
a_vetoed_recovery_proposal_does_not_hold_the_owner_s_slot shows), and
Lido-style lapsing vetoes as a future option only. §5: "independent
guardians, 3-of-5 or more".
- 10: the guardian set's minimum size is no longer an open decision.
SECURITY.md and 08 describe what is on devnet, which still runs the
old rule, and are left as they are.
claims: OK. Docs only; no program file changed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Say the weekly CI run skips mutants, as its condition does
ci.yml's header and scripts/ci-changes.sh said the weekly run does every
job, and 09 §8 gave `mutants` as running on any change to the core. The
`mutants` job's `if` starts with `github.event_name == 'pull_request'`,
so the Monday schedule (and a manual dispatch) never runs it: it mutates
a pull request's diff and has none to mutate. Its trigger list in 09 §8
also left out scripts/mutants.sh and .cargo/mutants.toml, which
ci-changes.sh (line 22) selects it for.
Comments and docs only; nothing runs differently.
Sibling search: `git grep` for "weekly run does", "every job ... weekly"
and "mutants ... weekly" across md/yml/sh/mjs/py. The other hits are
09 §8's "at full depth every Monday" (its table now says which job is
not weekly) and the dispatch input's "every job at weekly depth", which
is the same set as the weekly run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check that the mutant exclusions still match; repin murmur3 (finding 105)
.cargo/mutants.toml excludes the core's 22 equivalent mutants, and the
docs say so. Since 9 Oct it excluded 19. The murmur3 tail's pattern is
pinned by line, `floor.rs:(235|238|241)`, because the body's `h ^= k`
has the same column and function and is not equivalent. 1dc7861 (#52)
moved those lines (now 327/330/333), and nothing checked the patterns.
Failing closed, the three would have failed CI's `mutants` job on a
pull request that changed those lines. Failing open, a pattern that
starts matching a non-equivalent mutant hides it from every run.
Root cause: nothing checked that a pattern still names its mutants.
Fix:
- each pattern ends with the number of mutants it excludes (`# 3`), and
the murmur3 pattern names 327/330/333;
- tools/mutant_exclusions.py fails unless each pattern matches exactly
its number in `cargo mutants --list --no-config`, and cargo-mutants
(the list with the config) excludes exactly the mutants the patterns
match, so a pattern this parser can't see, or reads differently, fails
too. No two patterns may match the same mutant, since the docs' count
is the counts' sum, and the kept list must be part of the full one.
It reads only the exclude_re array's body, needs no tomllib
(Python 3.9), and names colour codes in the list as the problem;
- scripts/mutants.sh makes both lists (--colors never: CI sets
CARGO_TERM_COLOR=always) and runs it first, except when the caller
passes --config or --no-config (as whole arguments). CI's `mutants` job runs that script;
ci-changes.sh now also selects the job for the new tool;
- scripts/check.sh runs its --self-test on planted configs;
- tools/claims.py ties the docs' "22 equivalents" (README, 09 §6 and the
summary row) to the sum of the counts, with the same parser. It finds
that parser from its own directory, so `python3 -I` runs it too, and
check.sh runs its self-test that way. 00's
"18 equivalents" (26 Sep log entry) and run 9's "exactly the 22
equivalents below" (6 Oct) are kept as dated records.
Docs: 09 §4 finding 105, §6 run 10 and the "In CI" note, §8's mutants
row; README and 09's summary row 669 → 797 mutants (run 10); README and
12's post-submission range 96–105.
Evidence (first on 1f2ad48 + the parent, logs in the autopilot's
tasks/mutants-exclusions-r2/; re-run on 58697c6 + the parent,
tasks/mutants-exclusions-r3/):
- `cargo mutants --list`: 797 with --no-config, 778 with the old config
(19 excluded), 775 with this one. The 22 excluded are §6's list, the
murmur3 ones at 327/330/333.
- scripts/mutants.sh --list: exit 0, "22 of 797 mutants excluded"; with
CARGO_TERM_COLOR=always also exit 0. Old config: exit 1, every pattern
"no '# N' count". Planted on the real lists: the old lines with a
count (matches 0, says 3) and the body XOR added (matches 4, says 3)
both exit 1; in both the set comparison alone passes. /usr/bin/python3
(3.9.6): self-test OK and the real check passes.
- The self-test pins each problem by its message, not a count. Mutants
of the tool (tool-mutants.log), each red: M1 no per-pattern check,
M2 no set comparison, M3 no colour check, M4 no empty-array check,
M5 a missing count accepted, M6 a missing count read as 0, M7 the
parser reading past the array. M3 and M5 survived a first version
that checked only the number of problems.
After A's review (16 Oct): the set comparison was pinned only by its
size, so `len(excluded) != len(matched)` passed the self-test. A
case with as many excluded as matched but not the same ones now
fails it (15 cases; red with that mutant, green without), and the
message says how many mutants are in one set only.
From the code review after the rebase: colour codes only in the kept
list were reported as a kept list made with other options; the
colour check now reads both lists (16 cases; dropping the kept half
fails the new case).
On 58697c6, with the overlap and kept-list checks: M8 no overlap
check and M9 no kept-list check, red too, as M1–M7 still are. A
duplicate of the can_eject pattern planted in the real config:
scripts/mutants.sh --list exits 1, "the counts add up to 23, the
patterns match 22 mutants" (planted-overlap.log). Without the
sys.path line, `python3 -I tools/claims.py` fails with
ModuleNotFoundError; with it, both claims runs pass.
- claims.py (claims-mutants.log): one count 1 → 2 fails four statements
(README 188; 09 33, 333, 335; kept-run9.log), not run 9's 331, which
failed too before it was kept; a dropped count fails with the
tool named; deleting any of the three rules fails a self-test case;
deleting the KEPT entry fails 00:103.
- The 22 excluded mutants run alone (--no-config, --re per pattern,
2 min): 22 missed, exactly the excluded list; --re also picked
ema.rs:70 (a field deletion, no pattern matches it), caught.
- Run 10 (f1f05ba, old config, -j4, 46 min; tasks/mutants-run10/):
778 tested, 712 caught, 5 timeouts, 58 unviable, 3 missed, exactly
the murmur3 tail. The core's source is unchanged since (`git diff
--stat f1f05ba HEAD -- crates/bulkhead-core/src` is empty; only
tests/units.rs gained finding 102's tests).
- No program or core source changed.
Rebased on 16 Oct onto cede705 (findings 102 and 100, the attest
sweep and AUD-2). AUD-2 changed the core's source (governance.rs), so
the line above saying it is unchanged since run 10 held on 58697c6, not
here. On cede705: scripts/mutants.sh's list check passes, "22 of 795
mutants excluded, each pattern matching its count"; the murmur3 lines
are still 327/330/333. The 797 in README and 09 is run 10's figure:
README now dates it ("in the last full run, 15 Oct") and finding
105's row says the core has changed since (finding 107).
The setup line in scripts/mutants.sh now pins cargo-mutants 27.1.0,
the version ci.yml installs: each count is the number of mutants that
version lists, so another release can break the check (code review,
16 Oct; `git grep 'install.*cargo-mutants'` finds no other unpinned
install).
The check's report goes to stderr in scripts/mutants.sh, so
`scripts/mutants.sh --list` prints only cargo-mutants' names: before,
its first stdout line was the report; after, stdout is 773 lines
(795 - 22) and stderr the one report line. ci.yml's `mutants` comment
says the job checks the exclusions first.
Row 105 and 09 §6 gave the pin's reason as the body's XORs. The
finalizer's are at column 7 (`grep -n '\^=' floor.rs`, 337-342); the
non-equivalent XOR at column 11 is `h ^= k`, in the body (320) and after
the tail (335). The toml comment, the row and §6 now say that. 09 §8's
`check` row names the two tool self-tests check.sh runs.
README no longer calls run 10's 797 "injected bugs": cargo-mutants
listed 797 but tested 778, 19 being excluded (09 §6, run 10's row).
Sibling search: the other 14 patterns use `\d+` for the line, and each
matches the count now written after it. The murmur3 one is the only
line-pinned pattern.
Not done, by choice:
- Catching an edit that keeps a pattern's number but changes its
mutants (an excluded line itself edited, or a murmur3 body XOR moved
onto a pinned line). The toml and 09 §6 say so: such an edit needs the
mutant judged again by hand. Failing on any diff that touches an
excluded line would catch the first; it is a separate change.
- Moving murmur3's tail into its own function, so its pattern needn't
name lines. It shifts core lines, so it changes the program hashes.
Until then a line move above it fails the check, with a message
saying what to fix.
- A weekly full mutants run in CI. It took 46 min here at -j4, likely
past the job's 90 min on two CI cores: a minutes call for the owner.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that a ticket stays in its own protocol and vault
Two Vault checks that keep a ticket to its own protocol and vault had no
test. Deleting either one left every program test green
(tasks/error-codes/mutants.log, base cede705):
- brakes.rs:79, extend_ticket: tk.protocol != protocol. Without it, a live
guardian of any other protocol extends this protocol's tickets. Anyone
can register a protocol and name its guardians, so that guardian could
hold back every queued payout up to the total extension limit. The
sweep did not catch it: it has no second protocol's guardian to swap in.
- custody.rs:501, settle_ticket: tk.vault != vault. A USDC ticket settled
through the SOL vault (its tokens, its mint) got past the Vault's checks.
New test another_protocols_guardian_cannot_extend_a_ticket: a second
Market's live guardian extends this market's ticket -> WRONG_VAULT; the
same instruction from this market's guardian succeeds. The settle test
now settles the USDC ticket through the SOL vault and expects
WRONG_VAULT.
Evidence (tasks/error-codes/mutants.log, STATE=c1; full program suite
per mutant):
W02 brakes.rs:79 deleted -> KILLED by
another_protocols_guardian_cannot_extend_a_ticket ("expected
failure, got success": the stranger's extension went through)
W06 custody.rs:501 deleted -> KILLED by
a_ticket_settles_after_its_eta_to_its_stored_recipient_only (8049
InsufficientBalance from the empty SOL vault, not 8010)
Both were SURVIVED at base. No program source change: hashes
8ca8a608/543be60b/7086148b, as at cede705.
Sibling search: the other 12 WrongVault sites were deleted one at a time
in the same run. Eight are KILLED by the sweep (brakes.rs:39, lib.rs:320,
custody.rs:160, genesis.rs:491, views.rs:27, lifeboat.rs:151,
governance.rs:169/259). lib.rs:324 (vault PDA address), genesis.rs:523,
lifeboat.rs:71 and :247 survive; later commits on this branch deal with
them.
Counts: Vault 104 -> 105, programs 154 -> 155 (claims.py OK).
The test also reads the ticket's ETA: unchanged after the
refused extension, 3,600 s later after its own guardian's (code-review
fix: success alone did not show the extension happened).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that the seal names every vault and happens once
Two seal checks had no test; deleting either left every program test
green (tasks/error-codes/mutants.log, base cede705):
- genesis.rs:486, execute_seal: the pairs must number 3 * vault_count.
Without it, the seal checks only the vaults it is given. A vault left
out skips its claims-vs-balance check, so an uncovered import seals.
docs/05's seal row says every vault is passed exactly once.
the_seal_checks_each_vault_once covered a vault named twice in place of
another, not one left out.
- genesis.rs:431, propose_seal: refused once sealed (AlreadySealed).
Without it, a sealed protocol's admin re-proposes, which rewrites
seal_executable_s/expires_s and clears seal_vetoed on a sealed protocol.
the_seal_checks_each_vault_once now also leaves the uncovered USDC vault
out (expects VAULT_COUNT_MISMATCH) and, after the seal, proposes it again
(expects ALREADY_SEALED). No new test, so no count change.
Evidence (tasks/error-codes/mutants.log, STATE=c2, full program suite):
E05 genesis.rs:486 deleted -> KILLED by the_seal_checks_each_vault_once
("expected failure, got success": the seal went through without
the USDC vault and its 1,000 USDC uncovered import)
E04 genesis.rs:431 deleted -> KILLED by the_seal_checks_each_vault_once
("expected failure, got success")
Both were SURVIVED at base. Vault hash 7086148b unchanged (tests only).
Sibling search: the seal's other checks. :491 vault_tokens (KILLED by the
sweep), :519 lifeboat owner and :523 lifeboat mint/owner (SURVIVED; the
lifeboat is a PDA checked by expect_pda just above, which only this
program can create, as its own token account: see this branch's note in
docs/09).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that invalid limits and asset settings are refused at every entry
Three bound checks had no test; deleting any one left every program test
green (tasks/error-codes/mutants.log, base cede705):
- genesis.rs:171, create_vault: config_is_valid (bps <= 100%, windows
> 0). Without it, a vault is created with a 100.01%-an-hour burst.
- governance.rs:40, parse of a SetLimits change (at proposal and again
at execution, :254 and :478): the same rule for a proposed change.
- governance.rs:47, parse of an UpsertAsset change: valid_asset (haircut
<= 100%, liability >= 100%, the oracle bounds of finding 88). Only the
genesis upsert's copy (genesis.rs:296) was tested, by
oracle_settings_of_a_priced_asset_are_bounded.
New test invalid_limits_and_assets_are_refused_wherever_they_enter:
before the seal, create_vault with short_bps 10,001 -> INVALID_LIMITS;
after it, a SetLimits proposal with long_window_s 0 -> INVALID_LIMITS,
and an UpsertAsset proposal with a 10,001 bps haircut -> INVALID_ASSET.
Evidence (tasks/error-codes/mutants.log, STATE=c3, full program suite):
E06 genesis.rs:171 deleted -> KILLED ("expected failure, got success")
E07 governance.rs:40 deleted -> KILLED ("expected failure, got success")
E08 governance.rs:47 deleted -> KILLED ("expected failure, got success")
each by the new test only. All three were SURVIVED at base. Vault hash
7086148b unchanged (tests only).
Sibling search: every config_is_valid / valid_asset call in programs/
vault/src: genesis.rs:170 (create_vault), :295 (upsert_asset, KILLED by
the oracle test), governance.rs:39 and :46. All now killed.
Counts: Vault 105 -> 106, programs 155 -> 156 (claims.py OK).
The execution copy (:478) is not reached by any transaction: it
parses the bytes the proposal already checked, and the bounds read
nothing else. The test comment says so (code-review fix: it had
claimed execution was covered). oracle_settings_of_a_priced_asset_are_
bounded now asserts INVALID_ASSET for each of its four cases, not any
error (all four return it; E09 stays KILLED).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that an import into a ninth mint is refused
genesis.rs:410, import_backing: a backing ledger holds MAX_SLOTS (8)
mints, and an import into a ninth is refused with NoSlot. Deleting the
check left every program test green (tasks/error-codes/mutants.log, base
cede705). From the code, without it the import goes through: the vault's
imported total grows (:414), and the seal must cover that, but the
owner's ledger is saved unchanged, so the owner has no backed claim for
it.
New test an_import_into_a_ninth_mint_is_refused: nine vaults before the
seal; one owner gets an import into eight (ledger_of shows 8 positions),
then the ninth -> NO_SLOT.
Evidence (tasks/error-codes/mutants.log, STATE=c4, full program suite):
E14 genesis.rs:410 deleted -> KILLED by an_import_into_a_ninth_mint_is_
refused ("expected failure, got success"); SURVIVED at base.
Vault hash 7086148b unchanged (tests only).
Sibling search: the only other ledger credit is deposit's
(custody.rs:128). There a full ledger is by design: the deposit is taken
unbacked (Ledger::credit's doc, ledger.rs:187-189), and credit's result is
the backed flag, not an error.
Counts: Vault 106 -> 107, programs 156 -> 157 (claims.py OK).
import_ix now calls import_into_ix, which takes the vault, so the
new test shares the one import layout (code-review fix).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin six Vault refusals no test reached
Six reachable checks had no test that failed when the check was deleted
(tasks/error-codes/mutants.log, base cede705). Each now has an assertion
on its code, and each was seen to fail with the check deleted:
E02 genesis.rs:185 create_vault of an existing vault after the seal:
ALREADY_INITIALIZED (with the check gone: 8043 GenesisOnly, so the
"says so first" order in the comment above it was untested)
E12 governance.rs:426 veto of an executed change: CHANGE_EXECUTED
(gone: the veto succeeds and marks an executed change vetoed)
E13 governance.rs:517 cancel of an executed change: CHANGE_EXECUTED
(gone: the cancel succeeds)
E16 lifeboat.rs:256 execute_exit with no exit proposed:
NOTHING_TO_EXIT (gone: 8033 ChangeNotExecutable)
E17 custody.rs:84 a zero deposit: ZERO_AMOUNT (gone: it succeeds)
E18 custody.rs:272 a zero release: ZERO_AMOUNT (gone: it succeeds; a
zero release has no unbacked part, and the limiter, whose own zero
check custody.rs:242 maps, is called only when unbacked > 0, :361)
Where: E02 in after_the_seal_a_new_vault_starts_with_its_unbacked_lane_
closed, E12/E13 in an_unvetoed_loosening_executes_after_the_delay, E16 in
the_lifeboat_stays_shut_until_the_seal (after the move in, before an
exit is proposed), E17/E18 in the new a_zero_deposit_or_release_is_
refused.
Evidence (tasks/error-codes/mutants.log, STATE=c5, full program suite):
all six KILLED by the tests named above; all six SURVIVED at base. The
vault suite is green (108 passed). Vault hash 7086148b unchanged.
Counts: Vault 107 -> 108, programs 157 -> 158 (claims.py OK).
Helpers (code-review fixes): veto_change_ix beside cancel_change_ix;
create_vault_for_ix, create_vault_ix with a given mint and token
program, so E02's case names USDC's own accounts and creates no mint.
E02's comment now repeats the program's ("whatever the seal's state"):
create_vault checks the limits first (genesis.rs:170).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin four Vault checks another layer backstops
Four checks survived deletion (tasks/error-codes/mutants.log, base
cede705) because something later still refused the case, with its own
error, so no test could tell. Each now has a test that sees the Vault's
own code, and each was seen to fail with the check deleted:
E01 lib.rs:230, create_pda's guard: a second registration of an
existing protocol (register_protocol has no check of its own) ->
ALREADY_INITIALIZED. Gone: the system program refuses it, with no
custom code (custom_code 0).
W10 lifeboat.rs:71, move_in naming another mint or token program ->
WRONG_VAULT. Gone: the token program refuses (code 3, MintMismatch).
W12 lifeboat.rs:247, execute_exit naming another mint or token
program -> WRONG_VAULT. Gone: the same token-program error.
T01 token.rs:37, account_info on an account no token program owns: a
copy of a real USDC account's bytes under the system program. Gone:
the quote SUCCEEDS (it promises a payout), and the release fails
only in the runtime at the transfer ("instruction modified data of
an account it does not own"). The quote is asserted first so the
mutant is seen at the quote (STATE=c6b).
New tests: a_protocol_registers_once, a_lifeboat_move_names_the_vaults_
own_mint (also completes a move in and an exit, as a control: the
destination receives 1,000 USDC), a_payout_goes_only_to_an_account_a_
token_program_owns.
Evidence (tasks/error-codes/mutants.log, STATE=c6 and c6b, full program
suite): E01, W10, W12, T01 KILLED by the tests named above; all four
SURVIVED at base. Full program suite green: Floor 50, Vault 111. Vault
hash 7086148b unchanged (tests only).
Counts: Vault 108 -> 111, programs 158 -> 161 (claims.py OK).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test the bounds on ticket extensions and a closed lane's code
B's review of vault-error-tests-r3 (R1) found two Vault codes no test
asserted by value that the earlier commits missed: ExtensionTooLong
and LaneClosed. Deleting the 7-day total extension cap (brakes.rs:86,
B24) left every program test green, so a live guardian, or a stolen
guardian key, could hold a queued payout back without end, 24h at a
time, against docs/05 row 11. The wide differential models the total
(differential_wide.rs:406), but its default run never extends one
ticket that far. Mapping LaneClosed to QueueFull (custody.rs:240, B23)
also survived.
The code review of the fix found a worse gap at the same check: no
test sends a negative extension (the differential draws secs from 0,
1, 3600, 86400, 86401). With `secs <= 0` weakened to `secs == 0` (A3,
brakes.rs:84) every program test passed, and one guardian key could
pull a queued payout's ETA into the past, skipping the queue delay,
and refund its own 7-day budget.
- a_ticket_is_extended_later_by_at_most_a_day_a_call_and_seven_in_total:
-1h, 0 and 24h + 1 s are each refused with EXTENSION_TOO_LONG, ETA
unchanged; six 24h extensions and one of 23h pass; 2h more is
refused, ETA unchanged; the last hour is granted; then 1 s more is
refused. The split is unaligned so that a total that ignores this
call's `secs` fails too.
- a_closed_lane_refuses_an_unbacked_payout: a vault with short_bps 0,
and one with long_bps 0, each refuse an unbacked admin payout with
LANE_CLOSED; the same payout from a vault with both buckets open is
taken (the control, so a capacity base of 0 cannot pass it).
Evidence (tasks/vault-error-tests-r4/, B's runner copied: each mutant
applied alone, programs rebuilt, whole program suite run):
- without these tests (551e519; run-without.out, run-base2.out,
run-base3.out): B23, B24, A1 (`tk.extended_s >= MAX`, this call's
secs ignored), A2 (`>=` for `>`) and A3 SURVIVED. A4 (the 24h per
call deleted), A5 and A6 (either half of limiter.rs:172's closed-lane
test deleted) were KILLED by the differential only, not by value;
- with them (run-final.out, at the branch tip): B23 KILLED by
a_closed_lane_refuses_an_unbacked_payout; B24 A1 A2 A3 KILLED by the
extension test only; A4 by it and the differential; A5 A6 by the
closed-lane test and the differential.
No program change: the runner's closing build gives 8ca8a608 /
543be60b / 7086148b, as at cede705.
Counts: Vault 111 -> 113, programs 161 -> 163 (claims.py OK).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin each half of create_pda's existing-account guard on its own
B's review of vault-error-tests-r3 (R2): each half of lib.rs:229
(`!is_data_empty() || !owned_by(SYSTEM)`) survived deleted on its own
(B20, B21), since an existing protocol account fails both, so
a_protocol_registers_once pinned them only jointly.
The test now also plants, at a fresh protocol address, an account that
fails just one half: one with data owned by the system program, one
with no data owned by another program (a fresh address). Each
registration is refused with ALREADY_INITIALIZED. The comment says no transaction is
known to make either account: data or a new owner at a PDA needs its
signature, which only the Vault gives, and create_pda gives both in one
instruction.
Evidence (tasks/vault-error-tests-r4/): at 551e519 B20 and B21
SURVIVED (run-base2.out); at the branch tip both are KILLED by
a_protocol_registers_once only (run-final.out).
No count change (claims.py OK: vault_programs 113, programs 163).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Record the Vault refusal deletions and what each became
docs/09 §3 gains a row for the deletion run this branch answers
(tasks/error-codes/mutants.log): 38 refusals, across 14 of the 18 Vault
codes no test asserted by value at cede705, each deleted on its own
with the whole program suite run against it, on the AUD-2 build.
The 18 (each VaultError variant with no `verr::` use in programs/tests
at cede705): AlreadyInitialized NotRegistrar WrongVault
WrongTokenProgram AlreadySealed VaultCountMismatch InvalidLimits
InvalidAsset LaneClosed TicketNotPending ExtensionTooLong
ChangeExecuted LedgerUnsound LedgerCorrupt NoSlot NothingToExit
ZeroAmount Overflow. The run missed LaneClosed and ExtensionTooLong;
B's review deleted one check of each (B23 custody.rs:240, B24
brakes.rs:86), and both survived until 1797ae5. That commit
also pins a negative extension (A3, which survived every test before
it), the 24h per call, and a lane closed by either bucket.
At base: 13 KILLED, 25 SURVIVED. The commits before this one add tests
that kill 18 of the 25 and both of B's, each asserting the Vault's own
code (W02 W06, E04 E05, E06 E07 E08, E14, E02 E12 E13 E16 E17 E18, E01
W10 W12 T01; B23 B24). The row names the 7 that still survive and why:
- six guards no transaction is known to reach: lib.rs:324 (vault PDA
address; create_vault creates a vault only at its PDA), genesis.rs:519
and :523 (the lifeboat, checked by expect_pda first, created only by
lifeboat_move, move_in requires the seal, lifeboat.rs:53, and the
seal happens once, so no lifeboat holds data at it), brakes.rs:50 and
custody.rs:541 (a pending-ticket count below zero), token.rs:235
(GetAccountDataSize's return data from an already-checked program);
- custody.rs:399, stated as not shown unreachable: properties.rs:588-592
debits backed_amount and one more on random ledgers, and only
crates/bulkhead-core/tests/units.rs:375 debits less, on a one-mint
ledger with no liability. properties.rs:606 asserts backed(a) ==
min(a, backed(MAX)) for requests below 2^52 (:613), so any smaller
amount is backed too, and debit_backed(b) refuses only when
backed_amount(b) != b (ledger.rs:289). The known exception is a
collateral value that overflows u128 (ledger.rs:262): the valid
amounts need not form an interval there, so the consent or freeze
cap (custody.rs:337, :347) can lower the amount to an invalid one and
:399 refuses the release (fail closed). The search's own result is
always valid. Not verified by a tool beyond that property test. (A full ledger cannot reach :399 through
set's NoSlot: backed_amount's valid() refuses a new liability slot
on a full ledger, ledger.rs:237.)
Not deleted, as the row says: the two core-error mappings to
ChangeExecuted and ZeroAmount, token.rs:233, and the codes Overflow and
LedgerCorrupt (checked arithmetic, decoding).
After this branch five codes are still asserted by no test (same grep
at this commit): NotRegistrar (E03, genesis.rs:108, was KILLED at base
by another test), TicketNotPending, LedgerUnsound, LedgerCorrupt,
Overflow.
Docs only: no count change (claims.py OK: vault_programs 113, programs
163).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Run the exclusion check's main in its self-test
A's review of finding 105 (REVIEWS/auto-B-mutants-exclusions-r7.md,
finding 1, Low): CI's `mutants` job acts on main's exit status, but the
self-test called problems() only, and check.sh never runs the tool on
real lists. With main returning 0 on a mismatch (mutant A10), every
in-tree check passed, and CI would have printed the problems and gone
green. This was not a defect at fb771c6, since the job fails today. The
gap was that a later regression would pass.
Fix: main takes its arguments and, for the self-test, a config path
(default: the repo's .cargo/mutants.toml, as before), and its lines name
the config it read. The self-test runs main in-process on a planted
config and checks the exit status and every line on each stream:
- the right lists: exit 0, "1 of 3 mutants excluded" on stdout, nothing
on stderr;
- the lists the other way round: exit 1, nothing on stdout, and exactly
the count and "full list lacks" problems, so KEPT is read as KEPT;
- coloured lists: exit 1 with exactly one problem, so a main that fails
only on several problems is caught;
- one argument and three: exit 2 and the two usage lines.
One subprocess (python3 -I, 60 s timeout, in the temporary directory so
the config must be found from the tool's own path) runs the command line
with two lists and the repo's config, as scripts/mutants.sh does: on a
list no pattern can match, it exits 1 with an empty stdout and problem
lines that each name .cargo/mutants.toml. Which problems those are
depends on the config, so only their form is checked. Self-test: 16
cases -> 22. No doc states the number.
Evidence (tasks/exclusions-main-exit/):
- old-A10.log: the fb771c6 self-test with A10 applied prints "self-test
OK: 16 cases" and exits 0.
- mutants-r6.log (mutants.py applies each mutant to a scratch copy of
the tool, which reads the real config through a symlink): unmutated,
"self-test OK: 22 cases", exit 0. All 18 are KILLED (exit 1): A10
(return 0), A10b (fail only on more than one problem), A0 (always 1),
U1 (usage returns 0), U2/U3 (argument check `< 2`, `< 1`), K1 (KEPT
ignored), K2 (ALL read twice), C1 (planted config ignored), C2 (no
default config), R1 (the default config read from the cwd), E1/E2
(sys.argv sliced as [:] or [1:2]), P1 (problems not printed), S1
(problems on stdout), L1 (the planted config not named), N1 (only the
first problem printed), G1 (the success line printed on failure too).
Code-review passes at high shaped this. The first version ran three
subprocesses against the real config, and A0, K1 and U3 survived it.
The second left E2, C2 and G1 alive. Declined from the third: catching
exceptions in the self-test (a traceback already exits 1) and moving
the config hook out of main. The fourth showed that requiring one
"matches 0 mutants" line per live pattern tied the self-test to the
config's contents. The subprocess now checks the form of the lines
only, and the 17 stay killed. The fifth showed that R1 survived while
the child ran in the repo root (it does in check.sh); now it runs in
the temporary directory. Its other points repeat declined ones.
- real-lists.log: on the real lists, made with scripts/mutants.sh's
commands: ".cargo/mutants.toml: 22 of 795 mutants excluded, each
pattern matching its count", exit 0.
- python3 -I tools/mutant_exclusions.py --self-test: 22 cases OK.
- scripts/check.sh: ALL CHECKS PASSED (check-final.log);
bin/check-each-commit.sh green (each-commit.log).
Sibling search: the other tools check.sh runs (tools/claims.py,
tools/floor_check.py) also gate CI through main's exit status. claims.py
has the same gap: check.sh runs it on the real tree, where it finds no
problems, and its self-test calls self_test(), not the path in main that
returns 1. With that return changed to 0, `--self-test` and the real run
both still exit 0 (claims-C1.log). Not changed here: it is its own
commit (BACKLOG-B). floor_check.py runs in CI only as --self-test, whose
result main returns as is. Its RPC mode is an operator's tool that no CI
job runs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Record finding 106: Vault liveness and exit rules had no failing test
The tests landed in 944f5e8 (lifeboat-exit-tests-r3, 15 Oct). Their
09 §4 row waited for 104 and 105 so the list stays numbered without
gaps; 105 is now on integration (fb771c6), so the row lands here.
Docs only: 09 §4 row 106; README and 12's count and post-submission
range (96–106, 106 fixed). Two code-review passes changed the row as
written on 15 Oct: a sentence for today's re-run, the nine
single-guardian sites named, G5 quoted verbatim, and the exit veto
attributed to T18 (G6 has no veto), as is the veto test's doc comment
(vault.rs:3996, comment only).
Evidence, re-run today on bf8ba41 (the row's claims are from 15 Oct,
and AUD-2 changed the Vault since): B's re-review list of 33 line-
neutral mutants (the ten rules, the nine single-guardian sites L1-L9,
the boundaries, the six quorums, R1-R6), each applied, programs
rebuilt, all program tests run:
33 KILLED, 0 survived, 0 not applied, 0 build failures. Vault
hashes after the run match the tip (Vault 7086148b, Floor 8ca8a608,
Harbor 543be60b).
Log: autopilot tasks/finding-106-row/mutants.log.
- python3 -I tools/claims.py: OK, findings 106.
- git grep for "96–105", "105 findings": no other stale range.
- scripts/mutants.sh still mutates the core only (SCOPE, line 11), so
the row's last sentence holds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Cite PR #100's tree, not the unpublished bf8ba41, in finding 106
Finding 106's row in docs/09 said its 33 mutants were re-run on 16 Oct
at bf8ba41. That commit is not an ancestor of this branch or of the
stack above it and is on no remote branch, so the citation would point
at nothing once the stack lands. The owner's review of #99-#109 found it.
bf8ba41 and 86b777b (the head of PR #100, vault-error-tests-d2, the D2
replay of vault-error-tests-r4) have the same tree, 36187b49
(`git rev-parse bf8ba41^{tree} 86b777b^{tree}` prints it twice). So
the run was a run on 86b777b's tree, and the row now says "on tree
36187b49, the tree of 86b777b (PR #100's head)". Why not just "at
86b777b": it was committed on 17 Oct, after the run; and main
squash-merges (1045cd9 has one parent), so after PR #100 lands 86b777b
is reachable only as GitHub's refs/pull/100/head. The tree id stays
checkable from main: squashed bottom first, main's commit for PR #100
has that tree. "PR" keeps "#100" from reading as finding 100, the
row above. These points came from two code-review passes.
Sibling search: every 7-12 hex token in tracked .md/.py/.sh/.mjs/.rs/
.toml files at autopilot/integration 3e9f58d that names a local commit
(51 tokens) was checked with git merge-base --is-ancestor against
integration. bf8ba41 was the only one that is not an ancestor.
Docs only. tools/claims.py: claims OK (26 files).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Warn about a Floor guardian set short of the margin (finding 107)
The guardian checks the Vault protocol's guardian set against finding
107's margin (main.mjs checkGuardianSet), but never a Floor config's.
A Floor set registered before the rule, say 2-of-3, keeps working and
keeps AUD-2's exposure: one lost key and one rogue guardian lock its
governance for good. The guardian said nothing, and docs/07 §6 said so
("not yet a Floor set").
The rule moves to policy.mjs as guardianSetWarning, a pure function:
the same text as before, a warn line that alerts on mainnet only. In
main.mjs, checkGuardianSet now remembers the set shape per set
("protocol", or the protected program), and checkFloor calls it for
each watched Floor config it reads. floorAlerts is unchanged. The
margin is not a readiness code there: "floor not ready" alerts on
every cluster, and only the owner's new set clears a margin problem, so
"floor ready" would never be logged again.
Evidence (tasks/floor-guardian-margin/r2/):
- node --test cloud/test/*.test.mjs: 180/180 pass (one new test).
- Mutants of guardianSetWarning, each run on policy.test.mjs and
reverted; each fails the new test: N1 never warn, N2 threshold fixed
at 2, N3 notify always, N4 notify never, N5 GUARDIAN_MARGIN = 1.
- tools/claims.py: OK, client 180 (docs/09:40 updated).
- Not tested: the main.mjs wiring (the per-set Map and the checkFloor
call). No test runs checkFloor; deleting the call leaves every test
green.
Sibling search: guardianMarginProblem's callers are vault-check.mjs (an
info line) and guardianSetWarning. The status snapshot
(snapshot.mjs:33) prints live guardians and the quorum, not the margin;
left as it is.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Judge each guardian set's margin on its own, and say when it is fixed
The parent's per-set memory (which set shapes were already reported)
lived in main.mjs, so no test could fail on it: dropping the
once-per-shape check, or keying every set the same, left every test
green. The warning did not say which set it was about (only a program
tag on Floor lines), and it gave the same text for a set ejected down
to its threshold, or registered n-of-n before finding 96. There one
rogue guardian alone locks governance; a lost key alone does not, since
dead guardians cannot veto the owner's new set (docs/16 §4.4), but it
leaves no quorum until that set executes. Nothing was said when the owner's
new set fixed the problem. And a delivered alert did not say which Floor
program it was about: the program went only into the line's `program`
field, which alertFor uses in its repeat key but does not show.
Moving the program name into the text is a trap: alertFor keys repeats
on the first 60 characters of the text, and "the Floor guardians of
<44-character id>" pushes the shape past them. A 2-of-3 set ejected to
2-of-2 within ten minutes would then give the same key, and the worse
warning would not alert. So the program goes after the shape: "the
Floor guardians are 2-of-3 for <program>: ...". The `program` field
stays for the repeat key.
guardianSetWarning becomes guardianSetCheck(shapes, which, set,
mainnet) in policy.mjs. It keeps each set's last shape and whether it
was short in the Map, keyed by "protocol" or the protected program, and
returns the line to log or null. The warning names the set and has its
own text for n-of-n. When a set that was short gets the margin, it
returns one line saying so, which alerts on mainnet like the warning,
so whoever was paged hears that it is fixed. main.mjs only logs what it
returns. docs/07 §6 says so. The advice also had a hole: a set
registered as 7-of-8 was told to "run at least 7-of-9", and no set can
have 9 keys (core MAX_GUARDIANS is 8). When threshold + 2 is over 8,
the advice is now to lower the threshold to at most 6-of-8. That needs
MAX_GUARDIANS in JS, so lib/solana.mjs gets it next to
GUARDIAN_MARGIN; the next commit ties both to the core.
Evidence (tasks/floor-guardian-margin/):
- node --test cloud/test/*.test.mjs: 180/180 pass (the test is
extended, not added).
- 19 mutants, each run on policy.test.mjs and reverted (r6/mutants.py,
r6/mutants.log, r6/N5-margin1.log). Each fails the extended test, and
only it: never warn, threshold fixed at 2, notify always, notify
never, GUARDIAN_MARGIN = 1, no once-per-shape, one Map key for every
set, the n-of-n text for threshold + 1, no fixed line, a fixed line
for a set that was never short, the fixed line never alerting, no
program tag, every set named as the protocol's, the short flag not
stored, the program before the shape, the program left out of the
text, `need < 8` for `need <= 8` (6-of-7), the 8-key limit ignored
(7-of-8), 7-of-8 as the lowered advice.
- The mainnet alerts are checked on their own in a scratch copy of the
test without the earlier text assertions (r5/C1-C3-alone.log): with
the program left out of the text, the alert's detail check fails;
with a long name pushing the shape past character 60, "2-of-2 alerts"
fails.
- A's review of the previous version (branch
auto/B/floor-guardian-margin-r7, kept) found two survivors there: X3,
the shape keyed on the count only, and X5, the advice
threshold-of-(n+1). It also found that the n-of-n text said a lost key
alone locks governance. The test now has a threshold-only fix (7-of-8
-> 6-of-8, the advice's own step) and pins the n-of-n text and advice.
r9/mutants.log: X3, X5, the r7 n-of-n text and the text without its
lost-key clause are all KILLED. r9/mutants-r7.log: X3 and X5 SURVIVED
on the r7 code and test. The 18 mutants of r6/mutants.py, re-run on
this code: all KILLED (r9/r6-mutants-rerun.log). Two code-review
passes on the fix (kept as -r8 and -r9) added the lost-key clause: a
lost key alone leaves no quorum (it never approves, and core `quorum`
needs threshold approvals) until the owner's new set executes. A third
pass (-r10) fixed the comments' wording only.
- tools/claims.py: OK, client 180.
- Still not tested: the checkFloor call in main.mjs (no test runs
checkFloor).
Not changed (code review): the "fixed" line needs the short shape to
have been seen since the guardian started (the Map is in memory, as the
old per-process shape was; docs/07 says so). A protocol that abdicated
SetGuardians can't take a new set, so the advice to run 2-of-4 can't be
followed there; REPORT lists that case for the owner. The fault text
assumes a margin of 2, the owner's decision (16 §4.4).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Tie the JS guardian margin and set size limit to the core constants
cloud/lib/solana.mjs keeps its own GUARDIAN_MARGIN = 2, which the
guardian and the deploy tooling use to judge a guardian set, and now
MAX_GUARDIANS = 8, which the guardian's advice uses. Their comments say
they are core governance.rs's constants, but nothing checked that. Had
the programs' margin moved to 3, every JS test would have stayed green
while the guardian passed sets the programs refuse.
deploy.test.mjs now reads governance.rs and requires exactly one
`pub const GUARDIAN_MARGIN: u8 = N;` and one `pub const MAX_GUARDIANS:
usize = N;`, each N equal to the JS constant. It reads the source text,
like the build-script test above it. If a declaration changes form, the
test fails rather than passing.
Evidence (tasks/floor-guardian-margin/r6/tie-mutants.log):
- node --test cloud/test/*.test.mjs: 181/181 pass (one new test).
- Mutants of governance.rs, each reverted with its mtime kept (else the
limiter differential asks for a worker rebuild): GUARDIAN_MARGIN = 3,
MAX_GUARDIANS = 9, and a trailing comment on MAX_GUARDIANS. Each fails
the new test, naming the constant. deploy.test.mjs is green again.
- tools/claims.py: OK, client 181 (docs/09:40 updated).
Sibling search: the other JS copies of core rules (the limiter port,
the account layouts) are checked by the differential and by decoding
real accounts. MIN_GUARDIAN_THRESHOLD has no JS copy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Reach the 7-day extension total in the wide differential
differential_wide.rs drew extension seconds only from {0, 1, 3600,
86400, 86401} on a random ticket and lumped every refusal into one
counter. A ticket never got near the 7-day total, no negative value was
sent, and a refusal was not checked by its code. So the default run
could not tell the total bound, a negative extension or the error code
from a bug: only the unit tests at vault.rs:3382-3401 pinned them.
Now one extension draw in three pushes the chosen ticket by up to 24h
a call to exactly the 604,800 s total, then asks for 1 s more, so the
edge itself is hit on every push, not only when earlier draws line up.
The other draws are half legal sizes (1, 3600, 86400) and half out of
range for one call (i64::MIN, -86400, -1, 0, 86401, i64::MAX). Each
refusal must fail with EXTENSION_TOO_LONG. The counters are split into
extend (a random legal size), extend_push, extend_refused_call and
extend_refused_total, all required >= 10, so pushes cannot stand in for
the random sizes.
The model reads both bounds from bulkhead-core (consts.rs:37-38), as
the module says it is built, rather than restating 86,400 and 604,800.
docs/09 §7's row says so, and the 16 Oct "Vault refusals deleted" row,
which says the default run never extended a ticket that far, now says
it does since this change (its differential_wide.rs:406 cite had drifted).
Default run (24 seeds x 250 steps): extend 16, extend_push 126,
extend_refused_call 16, extend_refused_total 30 (was extend 40,
extend_refused 27); every other required counter still >= 10
(loosen_executed 12, as before).
Evidence: mutants of brakes.rs extend_ticket, each built and run against
the old and the new differential_wide only:
T1 drop the total check old SURVIVED new KILLED
T2 total > to >= old SURVIVED new KILLED
T3 drop secs <= 0 old KILLED new KILLED
T4 secs <= 0 to < 0 old KILLED new KILLED
T5 total ignores this call's secs old SURVIVED new KILLED
T6 extended_s += secs / 2 old SURVIVED new KILLED
T7 refusal returns WrongTicket old SURVIVED new KILLED
T8 secs <= 0 to == 0 (negatives) old SURVIVED new KILLED
T9 total > to > total + 1 old SURVIVED new KILLED
(T9: a draft that pushed by 24h until refused killed it only through
chance 1 s and 3600 s draws on the same ticket; the push to exactly
604,800 then 1 s more reaches the edge every time.)
sBPF sha256 after the run unchanged (Floor 8ca8a608, Harbor 543be60b,
Vault 7086148b): tests only.
Sibling search: the other refusal counters in this file (reset_refused,
lifeboat_refused_*, loosen_*) are each split by cause already.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check settle and lifeboat refusals by code in the wide differential
The same gap as the extension refusal, found in the sibling search for
the previous commit: a refused settle and the three lifeboat-move
refusals asserted only r.is_err(). Any refusal passed, so the order of
the checks and the program's own codes went unchecked by the
differential. Settle's comment (custody.rs:520-521) says it returns its
own error, not the token program's, when the vault is short; nothing in
this file held it to that.
Now a settle refused while frozen must fail with FROZEN, and one that is
only short with INSUFFICIENT_BALANCE (frozen is checked first). A
lifeboat move inside the delay after the seal must fail with NOT_SEALED,
one while frozen with FROZEN, and one beyond the balance with the token
program's InsufficientFunds (1). The two settle refusals get their own
counters: settle_refused_frozen is required >= 10 like the rest;
settle_refused_short needs lifeboat moves to drain a due ticket's
tokens first and occurs 7 times in the default run, so it is required
>= 1.
Evidence: mutants of programs/vault/src, each built and run against the
previous head's differential_wide and this one, nothing else:
S1 settle: drop the InsufficientBalance check old SURVIVED new KILLED
S2 settle: balance check before the freeze old SURVIVED new KILLED
L1 require_sealed returns Frozen old SURVIVED new KILLED
L2 move_in's freeze returns NotSealed old SURVIVED new KILLED
The token-program code (1) on a move beyond the balance is pinned but
not shown red by a mutant. sBPF sha256 unchanged (Floor 8ca8a608,
Harbor 543be60b, Vault 7086148b): tests only.
Deep run of both commits together (BULKHEAD_PROGRAM_SEEDS=600
BULKHEAD_PROGRAM_STEPS=500, 85 s): 300,000 steps, all identical;
extend 876, extend_push 6,462, extend_refused_call 1,175,
extend_refused_total 1,453, settle 2,024, settle_refused_frozen 921,
settle_refused_short 566, lifeboat refusals 3,854 + 5,768 + 1,171. docs/09 §7's
row gets that run as a 16 Oct result next to the 5 Oct one, which
predates these checks.
Sibling search: in differential_wide.rs, reset (TICKETS_PENDING) and
execute change already asserted codes, and a release or quote refused
for want of tokens asserts INSUFFICIENT_BALANCE. A release or quote the
limiter refuses (frozen, lane closed, queue full) is still compared only
as a refusal: mapping the core's refusal to the program's code is left
for its own change. Lifeboat returns only send legal amounts. docs/09's
row now says which refusals are checked by code. differential.rs:327
has the same is_err settle check; left as it is, since the wide run now pins both
settle codes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check limiter refusals by code in the wide differential
A release or quote that the limiter refused (lane closed, queue full)
was compared with the model only as Err. Any refusal code passed: a
program that swapped LaneClosed and QueueFull in custody.rs's
limiter_error, or checked the backed part against the balance before
the limiter and so answered InsufficientBalance for a full queue,
still matched the model. About 670 such refusals happen in a default
run.
The model now maps core's LimiterError to the program's code with
verr::limiter in the test harness (all four variants, as custody.rs's
limiter_error does), and every refused quote and release must fail
with the model's code. Lane-closed and queue-full refusals have their
own counters, each required at least 10 times; a refusal code with no
counter panics rather than being counted as want of tokens. docs/09
row 362 says so and records the deep run.
Evidence (autopilot tasks/differential-limiter-codes/):
- mutants.log: C1 (LaneClosed and QueueFull swapped in
limiter_error), C2 (QueueFull -> InsufficientBalance), C3
(LaneClosed -> ZeroAmount) and C4 (backed > balance checked before
the limiter, same line) each SURVIVED the old test and are KILLED
by the new one. Hashes back to 8ca8a608/543be60b/7086148b after.
- c4-full-suite.log: under C4 the whole program suite (Vault 113,
Floor 50, the fixed differential) passes; only this test fails.
C1-C3 are also caught by vault.rs's code tests (:434, :3428).
- Default run: refused_lane_closed 254, refused_queue_full 414,
insufficient 118.
- deep-600x500.log: 300,000 steps, 15,561 lane closed, 18,853 queue
full, 6,345 want of tokens, all identical.
Sibling search: every other Err arm in differential_wide.rs already
asserts a code (extend, settle, reset, lifeboat in, execute change).
Not here: lifeboat returns are never refused (backlog), and lifeboat
move_in beyond the balance fails with the token program's code
(backlog, a program change).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check release refusals by code in the fixed differential
differential.rs reduced every refused release to Err(()) on both
sides, so a program that refused for the wrong reason still matched
the model. The same gap the previous commit closed in the wide
differential (found by its code review).
The model's refusal now carries the program's code: verr::limiter for
a limiter refusal, INSUFFICIENT_BALANCE when the vault holds less
than it would pay. A refused release must fail with that code, and
the assert prints the program's error. Refusals for a full queue get
their own counter, required at least 10 times like the others. Want of
tokens and a closed lane are counted and printed but not required:
the wide differential requires both. A refusal with any other code
panics, as in the wide differential, so the counts always add up.
docs/09 row 361 says so; its undated figure (4,843 releases) no longer
matched a run, so it is replaced by a dated one.
Evidence (autopilot tasks/differential-limiter-codes/):
- mutants-narrow.log (mutants.sh <wt> differential): C1 (LaneClosed
and QueueFull swapped), C2 (QueueFull -> InsufficientBalance) and C4
(backed > balance checked before the limiter) SURVIVED the old test
and are KILLED by the new one. C3 (LaneClosed -> ZeroAmount)
survives both: no release in this test's runs is refused for a
closed lane. The wide differential kills it, and docs/09 says so.
Hashes back to 8ca8a608/543be60b/7086148b after.
- r2/default-run.log: 1,000 steps, 82 releases refused, all for a
full queue (want of tokens 0, lane closed 0).
- r2…
shuv-amp
added a commit
that referenced
this pull request
Oct 18, 2026
* Run the tests on 2-of-4 guardian sets; test recovery from two faults
Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set
with fewer than threshold + 2 keys. Every test fixture and most tests'
own sets were 2-of-3. This moves them to 2-of-4, which today's rule
also accepts, so this commit changes no program: Floor, Vault and Harbor
hash 508e5a99 / b07655d9 / 543be60b, as before.
- Fixtures: Protected (Floor) and Market (Vault, through Harbor) now
hold 4 guardians. Each test that built its own 3-key set gets a 4-key
set with the same threshold and the same assertion.
- Two tests changed meaning with a 4th key and were fixed, not loosened:
a_quorum_ejects_a_rogue_guardian_who_vetoes_everything (Floor) and
a_quorum_ejects_a_guardian_who_freezes_forever (Vault). With 4 keys,
the last eject was refused only because the target's own vote is
masked. Both now eject a second guardian (a set an ejection shrank to
threshold + 1 still ejects) and then show the 2-of-2 refusal.
- Account sweeps: an eject's signers may now be any two of the three
others, so each of those two slots is free for the third, never for
the target or the other signer again. Vault 5,365 and Floor 1,710
substitutions, refused (docs/09, dated 16 Oct). Listing all three
others at each slot would have skipped the duplicate: with a mutant
that counts a guardian listed twice as two (governance.rs:270), the
sweeps listing all three pass, and these fail at "eject: account 2
replaced by key ..." (Vault and Floor). No other program test kills
that mutant (tasks/aud2-guardian-margin-impl/dup-mutant-*.log).
- floor_differential: the guardian pool is 6 keys, 4 registered; the
proposed sets are 2 to 6 keys.
New tests (pass on both rules; they show what the margin buys):
- a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_set (Floor): g0's
key lost, g1 rogue stops the program; the lost key does not help a
quorum; the honest pair ejects g1 and restores; neither can veto the
owner's refill, which executes after 72h.
- two_rogues_in_turn_are_both_ejected_from_a_two_of_four_set (Floor):
the second rogue vetoes the refill and is ejected from the 2-of-3.
- a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_vault: the same in
the Vault (freeze, eject, unfreeze, refill).
Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest
and quote sweep): the counts and the sweep figures here are from runs
there (tasks/aud2-guardian-margin-impl/r4/). The E2E run, the mutants
and the new-rule run below were made before, on e8640a7; this commit's
programs and core are the same as there but for one comment line in
views.rs (line-neutral, same hashes).
Evidence:
- E2E=1 scripts/check-programs.sh: PROGRAM CHECKS PASSED (Floor 49,
Vault 98; S1 and E2E passed; the resumed setup found all 14 steps;
vault check, Floor demo and deploy dry run passed), with the next
commit's deploy rehearsal in the tree.
- tools/claims.py: OK (floor 49, vault 104, programs 153).
- Under the new rule (core from two commits on, programs rebuilt), only
the two tests that state the old rule fail: an_n_of_n_guardian_set_is_
refused (accepts 3-of-4) and the Floor differential's model.
- Mutants (tasks/coverage-map/mutants.sh), each red, then restored:
AV1 Floor live_guardian ignores liveness: KILLED by a_lost_key_and_a_
rogue_do_not_lock_a_two_of_four_set (+2 others).
AV2 the Vault's: KILLED by ..._two_of_four_vault (+2 others).
AQ1 Floor restore quorum ignores liveness: KILLED by ..._two_of_four_set
and the differential only.
AQ2 the Vault's unfreeze quorum: KILLED by ..._two_of_four_vault and
a_silent_guardian_does_not_count_toward_a_quorum.
AE1 can_eject refuses at count == threshold + 1 (AUD-2 option (a)):
KILLED by two_rogues_in_turn_..., and the two eject tests above.
Hashes after: 508e5a99 / 543be60b / b07655d9.
Split on 16 Oct from ec0cacc at the owner's review (D2): the deploy
rehearsal's move to 4 guardians is now the next commit. programs/ and
programs/tests/ here are byte-identical to ec0cacc, so its runs below
hold for this commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Rehearse deploys with four guardians, and read only the keys there are
Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set
with fewer than threshold + 2 keys, so the deploy rehearsal must
register 2-of-4 sets. Split from ec0cacc at the owner's review (D2);
the text is that commit's, unchanged:
Deploy rehearsal (cloud/deploy market.mjs, vault-check.mjs,
floor-demo.mjs): 4 guardians (keys/guardian3), so the dry run keeps
working once the rule lands; one more funding step (14, was 13).
The devnet market has 3 guardian files. A first version of this change
loaded guardian3 everywhere: vault-check.mjs failed there with ENOENT,
and heartbeat silently created a key nobody holds as a guardian. Now
market.mjs guardianKeys(dir, create) reads guardian0, 1, ... and
throws on a gap (never fills it with a new key). Setup creates keys up
to 4 only while the protocol is not registered, so resuming an old
market adds no key the chain lacks; a new deploy-dryrun step resumes
the dry-run market with guardian3 moved aside and fails if setup makes
it again (with setup always creating: "DRY RUN FAILED: resuming a
registered market made keys/guardian3.json", r3-dryrun-J7.log).
renew-guardians tops up to 4 and logs each new key (unfunded); a
2-of-3 renewal would be refused once the rule lands. keysIn reads the
guardians only when a command uses them, so caps and set-recovery
never need them; heartbeat says a key is not in the set rather than
dead. One THRESHOLD (2) for the register and renewal payloads and
MIN_GUARDIANS. vault-check.mjs
compares the chain's set with the first files, and liveness with the
chain's count. An extra file (a renewal not executed yet) and a set
without the margin are info lines, not failures, since such a set
keeps working. GUARDIAN_MARGIN and guardianMarginProblem live in
cloud/lib/solana.mjs, used by these and by the guardian bot's existing
warning (cloud/guardian/main.mjs), and vault-check uses THRESHOLD. The e2e scripts fund guardian3 too. Three
tests in cloud/test/deploy.test.mjs (client tests 176 -> 179). JS
mutants (js-mutants-r3.log), each red then restored: always create,
never create, no gap check, margin 1, `>`, GUARDIAN_MARGIN = 1.
Setup always creating survives these tests and is killed by the dry
run step above.
cloud/, scripts/deploy-dryrun.sh and docs/09's client-test row are
byte-identical to ec0cacc (git diff empty), so its runs hold for them:
the E2E check-programs run with the dry run (r4), and the JS mutants.
tools/claims.py: OK (client 179).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Refuse a guardian set without two keys beyond its threshold (107)
AUD-2, decided by the owner on 16 Oct (option (e) of 16 §4.4). A set
stays live while its honest live guardians reach the threshold: they
eject every rogue in turn, and dead guardians cannot veto the owner's
new set. Finding 96 asked for one key beyond the threshold, so a 2-of-3
or 3-of-4 set survived one fault but not a lost key plus a rogue: the
honest guardian left is no quorum, nobody can be ejected, and the rogue
keeps Floor stopped (or the Vault frozen) and vetoes every new set.
Fix, in the core so both programs inherit it through GuardianSet::parse
(registration, and every SetGuardians when proposed and when executed):
GUARDIAN_MARGIN = 2
guardian_set_ok := count <= 8 && t >= 2 && count >= t + 2
(saturating_add: t is a payload byte). A stored set is not checked
again, so existing 2-of-3 sets keep working, and ejection is unchanged:
a shrunk set still ejects. A 2- or 3-key set is BadThreshold, not
BadCount; the count rule is only the bitmap size, and the error order
of finding 103 is kept.
This changes both programs, by intent:
Floor 508e5a99 -> 8ca8a608, Vault b07655d9 -> 7086148b,
Harbor 543be60b (unchanged). Darwin-arm64 builds.
Compute (scratch LiteSVM test, tasks/aud2-guardian-margin-impl/cu.md):
-3 CU on every path that parses a set (Floor propose 2035 -> 2032,
execute 1682 -> 1679; Vault register 9592 -> 9589, SetGuardians 2979 ->
2976; Floor register varies with its PDA bump, residue also -3).
Proof and tests:
- Kani: accepted_sets_can_eject_anyone restated (ejection reachable
exactly when count > t, so option (a) cannot slip in; accepted sets
have >= 4 keys and a margin of at least 2, and 2 is accepted). New:
an_honest_live_quorum_ejects_any_rogue (any slots, anyone else alive)
and accepted_sets_survive_any_two_faults (any two faulty keys, lost
or rogue, leave a live honest quorum, which ejects each rogue, and the
rogues alone eject no honest guardian). That holds only short of a
rogue quorum: two rogues in a 2-of-n set are one (out of scope, 04
T7). With the assumption rogue < t dropped, the harness fails at
`!can_eject(rogue, ..)` (kani-no-rogue-assume.log), so the docs say
"short of a rogue quorum", not "any two faults".
19/19 verified, 240 s (cargo kani -j 2, as CI); again after the
restatement, 235 s summed (r2-prove.log).
- Kani mutants (kani-mutants.sh): margin 1, margin 3, `>` for `>=`,
t >= 1, count <= 9, option (a), and an eject one vote short (only
the two-fault harness kills that one): each KILLED by a harness
(kani-mutants-r2.log). The eject
ignoring liveness is the eject harness's case, not these.
- Program mutants (coverage-map/mutants.sh, rebuilt each time):
margin 1: KILLED by a_guardian_set_without_two_spare_keys_is_refused
(Floor and Vault), registration_rejects_weak_settings and the Floor
differential. Margin 3 and `>`: KILLED by most of the suite (the
fixtures are 2-of-4). t >= 1: first only the Floor differential; the
1-of-2 registration case was now refused by the margin too, so it is a
1-of-4 here and a Vault 1-of-4 case was added: then KILLED by both.
count <= 9: SURVIVES the program tests, equivalent there (parse refuses
more than 8 keys first), KILLED by Kani. Option (a): KILLED by the
eject tests two commits back (the tests on 2-of-4 sets).
- Tests: the Floor and Vault refusal tests above cover proposal and
execution (a stored 2-of-4 proposal whose threshold turns 3 is refused
when executed, as the devnet renewals proposed before this rule will
be); registration_rejects_weak_settings gains 2-of-3 and 3-of-4.
an_n_of_n_guardian_set_is_refused (Vault) is renamed and accepted a
3-of-4: it now refuses it and accepts 3-of-5.
- Model: the Python rule, the Floor differential's model, and the
guardian_set fuzz target state it again; the GP generator now also
picks count - 2. With the Python rule put back to the old one, seed 11
(20,000 cases) reports 5 GP mismatches; restored, seeds 11-13 identical.
Fuzz seeds two_of_three_refused, two_of_four, six_of_eight, and
seven_of_eight renamed seven_of_eight_refused (byte 323, the
threshold, is 7: now refused). With margin 1 in the core the target
crashes within 60 s; restored, 222,615,288 runs in 601 s, no crash.
- scripts/check.sh: ALL CHECKS PASSED, and E2E=1 check-programs.sh:
PROGRAM CHECKS PASSED, both at the branch tip, whose code is this
commit's (the next commit is docs only; tasks/aud2-guardian-margin-
impl/r4/check.log, r4/check-programs-e2e.log). claims: kani 19,
floor 50, vault 104, programs 154, client 179.
Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest
and quote sweep). The core and the program sources are the same as
before the rebase but for one comment line in views.rs (line-neutral),
so the Kani, fuzz, differential and mutant runs above were made on the
pre-rebase commit; the hashes, check.sh, check-programs and the counts
are from r4 runs (tasks/aud2-guardian-margin-impl/r4/).
The finding 96 row said the restated harness proves "exactly"; it now
says what it proved then and proves now, and gives the Vault test's new
name. The finding 107 row waits for rows 105 and 106 to land (09 §4 is
numbered 1..n); its text is ready. The rule's prose is the next commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* State the guardian margin in the docs, with its costs (AUD-2)
The previous commit made both programs refuse a new guardian set
without two keys beyond its threshold (finding 107). The docs that
state the rule now say so:
- README: guardians 4-8 keys; the G5 row and the "who the guardians
are" paragraph give the reason (any two faults leave a quorum, unless
the rogues are one themselves: two in a 2-of-n set, 04 T7).
- 05 register_protocol: 2 <= threshold <= count - 2, also for every
SetGuardians, proposed and executed. 05 abdicate: abdicating
SetGuardians fixes the set, so ejections shrink it for good.
- 06 state table, 00 D22, FAQ: the same rule.
- 07 §2: 4-8 guardians, and don't abdicate SetGuardians. 07 §6: the
programs enforce the margin; sets registered before keep their
exposure until replaced; an ejection uses up a spare key until the
owner's new set executes; a lost key counts as live until its
interval passes.
- 16: the AUD-2 row is decided; §4.4 gives the decision, why the
margin is the number of faults a set survives (short of a rogue
quorum), the options rejected
and why (the honest-owner and stolen-owner cases look the same
on-chain), what it costs, what it leaves (a rogue quorum; a stolen
owner key with every guardian dead; a rogue's veto of the recovery
key's rotation, checked: Floor's veto has no recovery exception, as
a_vetoed_recovery_proposal_does_not_hold_the_owner_s_slot shows), and
Lido-style lapsing vetoes as a future option only. §5: "independent
guardians, 3-of-5 or more".
- 10: the guardian set's minimum size is no longer an open decision.
SECURITY.md and 08 describe what is on devnet, which still runs the
old rule, and are left as they are.
claims: OK. Docs only; no program file changed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Say the weekly CI run skips mutants, as its condition does
ci.yml's header and scripts/ci-changes.sh said the weekly run does every
job, and 09 §8 gave `mutants` as running on any change to the core. The
`mutants` job's `if` starts with `github.event_name == 'pull_request'`,
so the Monday schedule (and a manual dispatch) never runs it: it mutates
a pull request's diff and has none to mutate. Its trigger list in 09 §8
also left out scripts/mutants.sh and .cargo/mutants.toml, which
ci-changes.sh (line 22) selects it for.
Comments and docs only; nothing runs differently.
Sibling search: `git grep` for "weekly run does", "every job ... weekly"
and "mutants ... weekly" across md/yml/sh/mjs/py. The other hits are
09 §8's "at full depth every Monday" (its table now says which job is
not weekly) and the dispatch input's "every job at weekly depth", which
is the same set as the weekly run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check that the mutant exclusions still match; repin murmur3 (finding 105)
.cargo/mutants.toml excludes the core's 22 equivalent mutants, and the
docs say so. Since 9 Oct it excluded 19. The murmur3 tail's pattern is
pinned by line, `floor.rs:(235|238|241)`, because the body's `h ^= k`
has the same column and function and is not equivalent. 1dc7861 (#52)
moved those lines (now 327/330/333), and nothing checked the patterns.
Failing closed, the three would have failed CI's `mutants` job on a
pull request that changed those lines. Failing open, a pattern that
starts matching a non-equivalent mutant hides it from every run.
Root cause: nothing checked that a pattern still names its mutants.
Fix:
- each pattern ends with the number of mutants it excludes (`# 3`), and
the murmur3 pattern names 327/330/333;
- tools/mutant_exclusions.py fails unless each pattern matches exactly
its number in `cargo mutants --list --no-config`, and cargo-mutants
(the list with the config) excludes exactly the mutants the patterns
match, so a pattern this parser can't see, or reads differently, fails
too. No two patterns may match the same mutant, since the docs' count
is the counts' sum, and the kept list must be part of the full one.
It reads only the exclude_re array's body, needs no tomllib
(Python 3.9), and names colour codes in the list as the problem;
- scripts/mutants.sh makes both lists (--colors never: CI sets
CARGO_TERM_COLOR=always) and runs it first, except when the caller
passes --config or --no-config (as whole arguments). CI's `mutants` job runs that script;
ci-changes.sh now also selects the job for the new tool;
- scripts/check.sh runs its --self-test on planted configs;
- tools/claims.py ties the docs' "22 equivalents" (README, 09 §6 and the
summary row) to the sum of the counts, with the same parser. It finds
that parser from its own directory, so `python3 -I` runs it too, and
check.sh runs its self-test that way. 00's
"18 equivalents" (26 Sep log entry) and run 9's "exactly the 22
equivalents below" (6 Oct) are kept as dated records.
Docs: 09 §4 finding 105, §6 run 10 and the "In CI" note, §8's mutants
row; README and 09's summary row 669 → 797 mutants (run 10); README and
12's post-submission range 96–105.
Evidence (first on 1f2ad48 + the parent, logs in the autopilot's
tasks/mutants-exclusions-r2/; re-run on 58697c6 + the parent,
tasks/mutants-exclusions-r3/):
- `cargo mutants --list`: 797 with --no-config, 778 with the old config
(19 excluded), 775 with this one. The 22 excluded are §6's list, the
murmur3 ones at 327/330/333.
- scripts/mutants.sh --list: exit 0, "22 of 797 mutants excluded"; with
CARGO_TERM_COLOR=always also exit 0. Old config: exit 1, every pattern
"no '# N' count". Planted on the real lists: the old lines with a
count (matches 0, says 3) and the body XOR added (matches 4, says 3)
both exit 1; in both the set comparison alone passes. /usr/bin/python3
(3.9.6): self-test OK and the real check passes.
- The self-test pins each problem by its message, not a count. Mutants
of the tool (tool-mutants.log), each red: M1 no per-pattern check,
M2 no set comparison, M3 no colour check, M4 no empty-array check,
M5 a missing count accepted, M6 a missing count read as 0, M7 the
parser reading past the array. M3 and M5 survived a first version
that checked only the number of problems.
After A's review (16 Oct): the set comparison was pinned only by its
size, so `len(excluded) != len(matched)` passed the self-test. A
case with as many excluded as matched but not the same ones now
fails it (15 cases; red with that mutant, green without), and the
message says how many mutants are in one set only.
From the code review after the rebase: colour codes only in the kept
list were reported as a kept list made with other options; the
colour check now reads both lists (16 cases; dropping the kept half
fails the new case).
On 58697c6, with the overlap and kept-list checks: M8 no overlap
check and M9 no kept-list check, red too, as M1–M7 still are. A
duplicate of the can_eject pattern planted in the real config:
scripts/mutants.sh --list exits 1, "the counts add up to 23, the
patterns match 22 mutants" (planted-overlap.log). Without the
sys.path line, `python3 -I tools/claims.py` fails with
ModuleNotFoundError; with it, both claims runs pass.
- claims.py (claims-mutants.log): one count 1 → 2 fails four statements
(README 188; 09 33, 333, 335; kept-run9.log), not run 9's 331, which
failed too before it was kept; a dropped count fails with the
tool named; deleting any of the three rules fails a self-test case;
deleting the KEPT entry fails 00:103.
- The 22 excluded mutants run alone (--no-config, --re per pattern,
2 min): 22 missed, exactly the excluded list; --re also picked
ema.rs:70 (a field deletion, no pattern matches it), caught.
- Run 10 (f1f05ba, old config, -j4, 46 min; tasks/mutants-run10/):
778 tested, 712 caught, 5 timeouts, 58 unviable, 3 missed, exactly
the murmur3 tail. The core's source is unchanged since (`git diff
--stat f1f05ba HEAD -- crates/bulkhead-core/src` is empty; only
tests/units.rs gained finding 102's tests).
- No program or core source changed.
Rebased on 16 Oct onto cede705 (findings 102 and 100, the attest
sweep and AUD-2). AUD-2 changed the core's source (governance.rs), so
the line above saying it is unchanged since run 10 held on 58697c6, not
here. On cede705: scripts/mutants.sh's list check passes, "22 of 795
mutants excluded, each pattern matching its count"; the murmur3 lines
are still 327/330/333. The 797 in README and 09 is run 10's figure:
README now dates it ("in the last full run, 15 Oct") and finding
105's row says the core has changed since (finding 107).
The setup line in scripts/mutants.sh now pins cargo-mutants 27.1.0,
the version ci.yml installs: each count is the number of mutants that
version lists, so another release can break the check (code review,
16 Oct; `git grep 'install.*cargo-mutants'` finds no other unpinned
install).
The check's report goes to stderr in scripts/mutants.sh, so
`scripts/mutants.sh --list` prints only cargo-mutants' names: before,
its first stdout line was the report; after, stdout is 773 lines
(795 - 22) and stderr the one report line. ci.yml's `mutants` comment
says the job checks the exclusions first.
Row 105 and 09 §6 gave the pin's reason as the body's XORs. The
finalizer's are at column 7 (`grep -n '\^=' floor.rs`, 337-342); the
non-equivalent XOR at column 11 is `h ^= k`, in the body (320) and after
the tail (335). The toml comment, the row and §6 now say that. 09 §8's
`check` row names the two tool self-tests check.sh runs.
README no longer calls run 10's 797 "injected bugs": cargo-mutants
listed 797 but tested 778, 19 being excluded (09 §6, run 10's row).
Sibling search: the other 14 patterns use `\d+` for the line, and each
matches the count now written after it. The murmur3 one is the only
line-pinned pattern.
Not done, by choice:
- Catching an edit that keeps a pattern's number but changes its
mutants (an excluded line itself edited, or a murmur3 body XOR moved
onto a pinned line). The toml and 09 §6 say so: such an edit needs the
mutant judged again by hand. Failing on any diff that touches an
excluded line would catch the first; it is a separate change.
- Moving murmur3's tail into its own function, so its pattern needn't
name lines. It shifts core lines, so it changes the program hashes.
Until then a line move above it fails the check, with a message
saying what to fix.
- A weekly full mutants run in CI. It took 46 min here at -j4, likely
past the job's 90 min on two CI cores: a minutes call for the owner.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that a ticket stays in its own protocol and vault
Two Vault checks that keep a ticket to its own protocol and vault had no
test. Deleting either one left every program test green
(tasks/error-codes/mutants.log, base cede705):
- brakes.rs:79, extend_ticket: tk.protocol != protocol. Without it, a live
guardian of any other protocol extends this protocol's tickets. Anyone
can register a protocol and name its guardians, so that guardian could
hold back every queued payout up to the total extension limit. The
sweep did not catch it: it has no second protocol's guardian to swap in.
- custody.rs:501, settle_ticket: tk.vault != vault. A USDC ticket settled
through the SOL vault (its tokens, its mint) got past the Vault's checks.
New test another_protocols_guardian_cannot_extend_a_ticket: a second
Market's live guardian extends this market's ticket -> WRONG_VAULT; the
same instruction from this market's guardian succeeds. The settle test
now settles the USDC ticket through the SOL vault and expects
WRONG_VAULT.
Evidence (tasks/error-codes/mutants.log, STATE=c1; full program suite
per mutant):
W02 brakes.rs:79 deleted -> KILLED by
another_protocols_guardian_cannot_extend_a_ticket ("expected
failure, got success": the stranger's extension went through)
W06 custody.rs:501 deleted -> KILLED by
a_ticket_settles_after_its_eta_to_its_stored_recipient_only (8049
InsufficientBalance from the empty SOL vault, not 8010)
Both were SURVIVED at base. No program source change: hashes
8ca8a608/543be60b/7086148b, as at cede705.
Sibling search: the other 12 WrongVault sites were deleted one at a time
in the same run. Eight are KILLED by the sweep (brakes.rs:39, lib.rs:320,
custody.rs:160, genesis.rs:491, views.rs:27, lifeboat.rs:151,
governance.rs:169/259). lib.rs:324 (vault PDA address), genesis.rs:523,
lifeboat.rs:71 and :247 survive; later commits on this branch deal with
them.
Counts: Vault 104 -> 105, programs 154 -> 155 (claims.py OK).
The test also reads the ticket's ETA: unchanged after the
refused extension, 3,600 s later after its own guardian's (code-review
fix: success alone did not show the extension happened).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that the seal names every vault and happens once
Two seal checks had no test; deleting either left every program test
green (tasks/error-codes/mutants.log, base cede705):
- genesis.rs:486, execute_seal: the pairs must number 3 * vault_count.
Without it, the seal checks only the vaults it is given. A vault left
out skips its claims-vs-balance check, so an uncovered import seals.
docs/05's seal row says every vault is passed exactly once.
the_seal_checks_each_vault_once covered a vault named twice in place of
another, not one left out.
- genesis.rs:431, propose_seal: refused once sealed (AlreadySealed).
Without it, a sealed protocol's admin re-proposes, which rewrites
seal_executable_s/expires_s and clears seal_vetoed on a sealed protocol.
the_seal_checks_each_vault_once now also leaves the uncovered USDC vault
out (expects VAULT_COUNT_MISMATCH) and, after the seal, proposes it again
(expects ALREADY_SEALED). No new test, so no count change.
Evidence (tasks/error-codes/mutants.log, STATE=c2, full program suite):
E05 genesis.rs:486 deleted -> KILLED by the_seal_checks_each_vault_once
("expected failure, got success": the seal went through without
the USDC vault and its 1,000 USDC uncovered import)
E04 genesis.rs:431 deleted -> KILLED by the_seal_checks_each_vault_once
("expected failure, got success")
Both were SURVIVED at base. Vault hash 7086148b unchanged (tests only).
Sibling search: the seal's other checks. :491 vault_tokens (KILLED by the
sweep), :519 lifeboat owner and :523 lifeboat mint/owner (SURVIVED; the
lifeboat is a PDA checked by expect_pda just above, which only this
program can create, as its own token account: see this branch's note in
docs/09).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that invalid limits and asset settings are refused at every entry
Three bound checks had no test; deleting any one left every program test
green (tasks/error-codes/mutants.log, base cede705):
- genesis.rs:171, create_vault: config_is_valid (bps <= 100%, windows
> 0). Without it, a vault is created with a 100.01%-an-hour burst.
- governance.rs:40, parse of a SetLimits change (at proposal and again
at execution, :254 and :478): the same rule for a proposed change.
- governance.rs:47, parse of an UpsertAsset change: valid_asset (haircut
<= 100%, liability >= 100%, the oracle bounds of finding 88). Only the
genesis upsert's copy (genesis.rs:296) was tested, by
oracle_settings_of_a_priced_asset_are_bounded.
New test invalid_limits_and_assets_are_refused_wherever_they_enter:
before the seal, create_vault with short_bps 10,001 -> INVALID_LIMITS;
after it, a SetLimits proposal with long_window_s 0 -> INVALID_LIMITS,
and an UpsertAsset proposal with a 10,001 bps haircut -> INVALID_ASSET.
Evidence (tasks/error-codes/mutants.log, STATE=c3, full program suite):
E06 genesis.rs:171 deleted -> KILLED ("expected failure, got success")
E07 governance.rs:40 deleted -> KILLED ("expected failure, got success")
E08 governance.rs:47 deleted -> KILLED ("expected failure, got success")
each by the new test only. All three were SURVIVED at base. Vault hash
7086148b unchanged (tests only).
Sibling search: every config_is_valid / valid_asset call in programs/
vault/src: genesis.rs:170 (create_vault), :295 (upsert_asset, KILLED by
the oracle test), governance.rs:39 and :46. All now killed.
Counts: Vault 105 -> 106, programs 155 -> 156 (claims.py OK).
The execution copy (:478) is not reached by any transaction: it
parses the bytes the proposal already checked, and the bounds read
nothing else. The test comment says so (code-review fix: it had
claimed execution was covered). oracle_settings_of_a_priced_asset_are_
bounded now asserts INVALID_ASSET for each of its four cases, not any
error (all four return it; E09 stays KILLED).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test that an import into a ninth mint is refused
genesis.rs:410, import_backing: a backing ledger holds MAX_SLOTS (8)
mints, and an import into a ninth is refused with NoSlot. Deleting the
check left every program test green (tasks/error-codes/mutants.log, base
cede705). From the code, without it the import goes through: the vault's
imported total grows (:414), and the seal must cover that, but the
owner's ledger is saved unchanged, so the owner has no backed claim for
it.
New test an_import_into_a_ninth_mint_is_refused: nine vaults before the
seal; one owner gets an import into eight (ledger_of shows 8 positions),
then the ninth -> NO_SLOT.
Evidence (tasks/error-codes/mutants.log, STATE=c4, full program suite):
E14 genesis.rs:410 deleted -> KILLED by an_import_into_a_ninth_mint_is_
refused ("expected failure, got success"); SURVIVED at base.
Vault hash 7086148b unchanged (tests only).
Sibling search: the only other ledger credit is deposit's
(custody.rs:128). There a full ledger is by design: the deposit is taken
unbacked (Ledger::credit's doc, ledger.rs:187-189), and credit's result is
the backed flag, not an error.
Counts: Vault 106 -> 107, programs 156 -> 157 (claims.py OK).
import_ix now calls import_into_ix, which takes the vault, so the
new test shares the one import layout (code-review fix).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin six Vault refusals no test reached
Six reachable checks had no test that failed when the check was deleted
(tasks/error-codes/mutants.log, base cede705). Each now has an assertion
on its code, and each was seen to fail with the check deleted:
E02 genesis.rs:185 create_vault of an existing vault after the seal:
ALREADY_INITIALIZED (with the check gone: 8043 GenesisOnly, so the
"says so first" order in the comment above it was untested)
E12 governance.rs:426 veto of an executed change: CHANGE_EXECUTED
(gone: the veto succeeds and marks an executed change vetoed)
E13 governance.rs:517 cancel of an executed change: CHANGE_EXECUTED
(gone: the cancel succeeds)
E16 lifeboat.rs:256 execute_exit with no exit proposed:
NOTHING_TO_EXIT (gone: 8033 ChangeNotExecutable)
E17 custody.rs:84 a zero deposit: ZERO_AMOUNT (gone: it succeeds)
E18 custody.rs:272 a zero release: ZERO_AMOUNT (gone: it succeeds; a
zero release has no unbacked part, and the limiter, whose own zero
check custody.rs:242 maps, is called only when unbacked > 0, :361)
Where: E02 in after_the_seal_a_new_vault_starts_with_its_unbacked_lane_
closed, E12/E13 in an_unvetoed_loosening_executes_after_the_delay, E16 in
the_lifeboat_stays_shut_until_the_seal (after the move in, before an
exit is proposed), E17/E18 in the new a_zero_deposit_or_release_is_
refused.
Evidence (tasks/error-codes/mutants.log, STATE=c5, full program suite):
all six KILLED by the tests named above; all six SURVIVED at base. The
vault suite is green (108 passed). Vault hash 7086148b unchanged.
Counts: Vault 107 -> 108, programs 157 -> 158 (claims.py OK).
Helpers (code-review fixes): veto_change_ix beside cancel_change_ix;
create_vault_for_ix, create_vault_ix with a given mint and token
program, so E02's case names USDC's own accounts and creates no mint.
E02's comment now repeats the program's ("whatever the seal's state"):
create_vault checks the limits first (genesis.rs:170).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin four Vault checks another layer backstops
Four checks survived deletion (tasks/error-codes/mutants.log, base
cede705) because something later still refused the case, with its own
error, so no test could tell. Each now has a test that sees the Vault's
own code, and each was seen to fail with the check deleted:
E01 lib.rs:230, create_pda's guard: a second registration of an
existing protocol (register_protocol has no check of its own) ->
ALREADY_INITIALIZED. Gone: the system program refuses it, with no
custom code (custom_code 0).
W10 lifeboat.rs:71, move_in naming another mint or token program ->
WRONG_VAULT. Gone: the token program refuses (code 3, MintMismatch).
W12 lifeboat.rs:247, execute_exit naming another mint or token
program -> WRONG_VAULT. Gone: the same token-program error.
T01 token.rs:37, account_info on an account no token program owns: a
copy of a real USDC account's bytes under the system program. Gone:
the quote SUCCEEDS (it promises a payout), and the release fails
only in the runtime at the transfer ("instruction modified data of
an account it does not own"). The quote is asserted first so the
mutant is seen at the quote (STATE=c6b).
New tests: a_protocol_registers_once, a_lifeboat_move_names_the_vaults_
own_mint (also completes a move in and an exit, as a control: the
destination receives 1,000 USDC), a_payout_goes_only_to_an_account_a_
token_program_owns.
Evidence (tasks/error-codes/mutants.log, STATE=c6 and c6b, full program
suite): E01, W10, W12, T01 KILLED by the tests named above; all four
SURVIVED at base. Full program suite green: Floor 50, Vault 111. Vault
hash 7086148b unchanged (tests only).
Counts: Vault 108 -> 111, programs 158 -> 161 (claims.py OK).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test the bounds on ticket extensions and a closed lane's code
B's review of vault-error-tests-r3 (R1) found two Vault codes no test
asserted by value that the earlier commits missed: ExtensionTooLong
and LaneClosed. Deleting the 7-day total extension cap (brakes.rs:86,
B24) left every program test green, so a live guardian, or a stolen
guardian key, could hold a queued payout back without end, 24h at a
time, against docs/05 row 11. The wide differential models the total
(differential_wide.rs:406), but its default run never extends one
ticket that far. Mapping LaneClosed to QueueFull (custody.rs:240, B23)
also survived.
The code review of the fix found a worse gap at the same check: no
test sends a negative extension (the differential draws secs from 0,
1, 3600, 86400, 86401). With `secs <= 0` weakened to `secs == 0` (A3,
brakes.rs:84) every program test passed, and one guardian key could
pull a queued payout's ETA into the past, skipping the queue delay,
and refund its own 7-day budget.
- a_ticket_is_extended_later_by_at_most_a_day_a_call_and_seven_in_total:
-1h, 0 and 24h + 1 s are each refused with EXTENSION_TOO_LONG, ETA
unchanged; six 24h extensions and one of 23h pass; 2h more is
refused, ETA unchanged; the last hour is granted; then 1 s more is
refused. The split is unaligned so that a total that ignores this
call's `secs` fails too.
- a_closed_lane_refuses_an_unbacked_payout: a vault with short_bps 0,
and one with long_bps 0, each refuse an unbacked admin payout with
LANE_CLOSED; the same payout from a vault with both buckets open is
taken (the control, so a capacity base of 0 cannot pass it).
Evidence (tasks/vault-error-tests-r4/, B's runner copied: each mutant
applied alone, programs rebuilt, whole program suite run):
- without these tests (551e519; run-without.out, run-base2.out,
run-base3.out): B23, B24, A1 (`tk.extended_s >= MAX`, this call's
secs ignored), A2 (`>=` for `>`) and A3 SURVIVED. A4 (the 24h per
call deleted), A5 and A6 (either half of limiter.rs:172's closed-lane
test deleted) were KILLED by the differential only, not by value;
- with them (run-final.out, at the branch tip): B23 KILLED by
a_closed_lane_refuses_an_unbacked_payout; B24 A1 A2 A3 KILLED by the
extension test only; A4 by it and the differential; A5 A6 by the
closed-lane test and the differential.
No program change: the runner's closing build gives 8ca8a608 /
543be60b / 7086148b, as at cede705.
Counts: Vault 111 -> 113, programs 161 -> 163 (claims.py OK).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pin each half of create_pda's existing-account guard on its own
B's review of vault-error-tests-r3 (R2): each half of lib.rs:229
(`!is_data_empty() || !owned_by(SYSTEM)`) survived deleted on its own
(B20, B21), since an existing protocol account fails both, so
a_protocol_registers_once pinned them only jointly.
The test now also plants, at a fresh protocol address, an account that
fails just one half: one with data owned by the system program, one
with no data owned by another program (a fresh address). Each
registration is refused with ALREADY_INITIALIZED. The comment says no transaction is
known to make either account: data or a new owner at a PDA needs its
signature, which only the Vault gives, and create_pda gives both in one
instruction.
Evidence (tasks/vault-error-tests-r4/): at 551e519 B20 and B21
SURVIVED (run-base2.out); at the branch tip both are KILLED by
a_protocol_registers_once only (run-final.out).
No count change (claims.py OK: vault_programs 113, programs 163).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Record the Vault refusal deletions and what each became
docs/09 §3 gains a row for the deletion run this branch answers
(tasks/error-codes/mutants.log): 38 refusals, across 14 of the 18 Vault
codes no test asserted by value at cede705, each deleted on its own
with the whole program suite run against it, on the AUD-2 build.
The 18 (each VaultError variant with no `verr::` use in programs/tests
at cede705): AlreadyInitialized NotRegistrar WrongVault
WrongTokenProgram AlreadySealed VaultCountMismatch InvalidLimits
InvalidAsset LaneClosed TicketNotPending ExtensionTooLong
ChangeExecuted LedgerUnsound LedgerCorrupt NoSlot NothingToExit
ZeroAmount Overflow. The run missed LaneClosed and ExtensionTooLong;
B's review deleted one check of each (B23 custody.rs:240, B24
brakes.rs:86), and both survived until 1797ae5. That commit
also pins a negative extension (A3, which survived every test before
it), the 24h per call, and a lane closed by either bucket.
At base: 13 KILLED, 25 SURVIVED. The commits before this one add tests
that kill 18 of the 25 and both of B's, each asserting the Vault's own
code (W02 W06, E04 E05, E06 E07 E08, E14, E02 E12 E13 E16 E17 E18, E01
W10 W12 T01; B23 B24). The row names the 7 that still survive and why:
- six guards no transaction is known to reach: lib.rs:324 (vault PDA
address; create_vault creates a vault only at its PDA), genesis.rs:519
and :523 (the lifeboat, checked by expect_pda first, created only by
lifeboat_move, move_in requires the seal, lifeboat.rs:53, and the
seal happens once, so no lifeboat holds data at it), brakes.rs:50 and
custody.rs:541 (a pending-ticket count below zero), token.rs:235
(GetAccountDataSize's return data from an already-checked program);
- custody.rs:399, stated as not shown unreachable: properties.rs:588-592
debits backed_amount and one more on random ledgers, and only
crates/bulkhead-core/tests/units.rs:375 debits less, on a one-mint
ledger with no liability. properties.rs:606 asserts backed(a) ==
min(a, backed(MAX)) for requests below 2^52 (:613), so any smaller
amount is backed too, and debit_backed(b) refuses only when
backed_amount(b) != b (ledger.rs:289). The known exception is a
collateral value that overflows u128 (ledger.rs:262): the valid
amounts need not form an interval there, so the consent or freeze
cap (custody.rs:337, :347) can lower the amount to an invalid one and
:399 refuses the release (fail closed). The search's own result is
always valid. Not verified by a tool beyond that property test. (A full ledger cannot reach :399 through
set's NoSlot: backed_amount's valid() refuses a new liability slot
on a full ledger, ledger.rs:237.)
Not deleted, as the row says: the two core-error mappings to
ChangeExecuted and ZeroAmount, token.rs:233, and the codes Overflow and
LedgerCorrupt (checked arithmetic, decoding).
After this branch five codes are still asserted by no test (same grep
at this commit): NotRegistrar (E03, genesis.rs:108, was KILLED at base
by another test), TicketNotPending, LedgerUnsound, LedgerCorrupt,
Overflow.
Docs only: no count change (claims.py OK: vault_programs 113, programs
163).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Run the exclusion check's main in its self-test
A's review of finding 105 (REVIEWS/auto-B-mutants-exclusions-r7.md,
finding 1, Low): CI's `mutants` job acts on main's exit status, but the
self-test called problems() only, and check.sh never runs the tool on
real lists. With main returning 0 on a mismatch (mutant A10), every
in-tree check passed, and CI would have printed the problems and gone
green. This was not a defect at fb771c6, since the job fails today. The
gap was that a later regression would pass.
Fix: main takes its arguments and, for the self-test, a config path
(default: the repo's .cargo/mutants.toml, as before), and its lines name
the config it read. The self-test runs main in-process on a planted
config and checks the exit status and every line on each stream:
- the right lists: exit 0, "1 of 3 mutants excluded" on stdout, nothing
on stderr;
- the lists the other way round: exit 1, nothing on stdout, and exactly
the count and "full list lacks" problems, so KEPT is read as KEPT;
- coloured lists: exit 1 with exactly one problem, so a main that fails
only on several problems is caught;
- one argument and three: exit 2 and the two usage lines.
One subprocess (python3 -I, 60 s timeout, in the temporary directory so
the config must be found from the tool's own path) runs the command line
with two lists and the repo's config, as scripts/mutants.sh does: on a
list no pattern can match, it exits 1 with an empty stdout and problem
lines that each name .cargo/mutants.toml. Which problems those are
depends on the config, so only their form is checked. Self-test: 16
cases -> 22. No doc states the number.
Evidence (tasks/exclusions-main-exit/):
- old-A10.log: the fb771c6 self-test with A10 applied prints "self-test
OK: 16 cases" and exits 0.
- mutants-r6.log (mutants.py applies each mutant to a scratch copy of
the tool, which reads the real config through a symlink): unmutated,
"self-test OK: 22 cases", exit 0. All 18 are KILLED (exit 1): A10
(return 0), A10b (fail only on more than one problem), A0 (always 1),
U1 (usage returns 0), U2/U3 (argument check `< 2`, `< 1`), K1 (KEPT
ignored), K2 (ALL read twice), C1 (planted config ignored), C2 (no
default config), R1 (the default config read from the cwd), E1/E2
(sys.argv sliced as [:] or [1:2]), P1 (problems not printed), S1
(problems on stdout), L1 (the planted config not named), N1 (only the
first problem printed), G1 (the success line printed on failure too).
Code-review passes at high shaped this. The first version ran three
subprocesses against the real config, and A0, K1 and U3 survived it.
The second left E2, C2 and G1 alive. Declined from the third: catching
exceptions in the self-test (a traceback already exits 1) and moving
the config hook out of main. The fourth showed that requiring one
"matches 0 mutants" line per live pattern tied the self-test to the
config's contents. The subprocess now checks the form of the lines
only, and the 17 stay killed. The fifth showed that R1 survived while
the child ran in the repo root (it does in check.sh); now it runs in
the temporary directory. Its other points repeat declined ones.
- real-lists.log: on the real lists, made with scripts/mutants.sh's
commands: ".cargo/mutants.toml: 22 of 795 mutants excluded, each
pattern matching its count", exit 0.
- python3 -I tools/mutant_exclusions.py --self-test: 22 cases OK.
- scripts/check.sh: ALL CHECKS PASSED (check-final.log);
bin/check-each-commit.sh green (each-commit.log).
Sibling search: the other tools check.sh runs (tools/claims.py,
tools/floor_check.py) also gate CI through main's exit status. claims.py
has the same gap: check.sh runs it on the real tree, where it finds no
problems, and its self-test calls self_test(), not the path in main that
returns 1. With that return changed to 0, `--self-test` and the real run
both still exit 0 (claims-C1.log). Not changed here: it is its own
commit (BACKLOG-B). floor_check.py runs in CI only as --self-test, whose
result main returns as is. Its RPC mode is an operator's tool that no CI
job runs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Record finding 106: Vault liveness and exit rules had no failing test
The tests landed in 944f5e8 (lifeboat-exit-tests-r3, 15 Oct). Their
09 §4 row waited for 104 and 105 so the list stays numbered without
gaps; 105 is now on integration (fb771c6), so the row lands here.
Docs only: 09 §4 row 106; README and 12's count and post-submission
range (96–106, 106 fixed). Two code-review passes changed the row as
written on 15 Oct: a sentence for today's re-run, the nine
single-guardian sites named, G5 quoted verbatim, and the exit veto
attributed to T18 (G6 has no veto), as is the veto test's doc comment
(vault.rs:3996, comment only).
Evidence, re-run today on bf8ba41 (the row's claims are from 15 Oct,
and AUD-2 changed the Vault since): B's re-review list of 33 line-
neutral mutants (the ten rules, the nine single-guardian sites L1-L9,
the boundaries, the six quorums, R1-R6), each applied, programs
rebuilt, all program tests run:
33 KILLED, 0 survived, 0 not applied, 0 build failures. Vault
hashes after the run match the tip (Vault 7086148b, Floor 8ca8a608,
Harbor 543be60b).
Log: autopilot tasks/finding-106-row/mutants.log.
- python3 -I tools/claims.py: OK, findings 106.
- git grep for "96–105", "105 findings": no other stale range.
- scripts/mutants.sh still mutates the core only (SCOPE, line 11), so
the row's last sentence holds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Cite PR #100's tree, not the unpublished bf8ba41, in finding 106
Finding 106's row in docs/09 said its 33 mutants were re-run on 16 Oct
at bf8ba41. That commit is not an ancestor of this branch or of the
stack above it and is on no remote branch, so the citation would point
at nothing once the stack lands. The owner's review of #99-#109 found it.
bf8ba41 and 86b777b (the head of PR #100, vault-error-tests-d2, the D2
replay of vault-error-tests-r4) have the same tree, 36187b49
(`git rev-parse bf8ba41^{tree} 86b777b^{tree}` prints it twice). So
the run was a run on 86b777b's tree, and the row now says "on tree
36187b49, the tree of 86b777b (PR #100's head)". Why not just "at
86b777b": it was committed on 17 Oct, after the run; and main
squash-merges (1045cd9 has one parent), so after PR #100 lands 86b777b
is reachable only as GitHub's refs/pull/100/head. The tree id stays
checkable from main: squashed bottom first, main's commit for PR #100
has that tree. "PR" keeps "#100" from reading as finding 100, the
row above. These points came from two code-review passes.
Sibling search: every 7-12 hex token in tracked .md/.py/.sh/.mjs/.rs/
.toml files at autopilot/integration 3e9f58d that names a local commit
(51 tokens) was checked with git merge-base --is-ancestor against
integration. bf8ba41 was the only one that is not an ancestor.
Docs only. tools/claims.py: claims OK (26 files).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Warn about a Floor guardian set short of the margin (finding 107)
The guardian checks the Vault protocol's guardian set against finding
107's margin (main.mjs checkGuardianSet), but never a Floor config's.
A Floor set registered before the rule, say 2-of-3, keeps working and
keeps AUD-2's exposure: one lost key and one rogue guardian lock its
governance for good. The guardian said nothing, and docs/07 §6 said so
("not yet a Floor set").
The rule moves to policy.mjs as guardianSetWarning, a pure function:
the same text as before, a warn line that alerts on mainnet only. In
main.mjs, checkGuardianSet now remembers the set shape per set
("protocol", or the protected program), and checkFloor calls it for
each watched Floor config it reads. floorAlerts is unchanged. The
margin is not a readiness code there: "floor not ready" alerts on
every cluster, and only the owner's new set clears a margin problem, so
"floor ready" would never be logged again.
Evidence (tasks/floor-guardian-margin/r2/):
- node --test cloud/test/*.test.mjs: 180/180 pass (one new test).
- Mutants of guardianSetWarning, each run on policy.test.mjs and
reverted; each fails the new test: N1 never warn, N2 threshold fixed
at 2, N3 notify always, N4 notify never, N5 GUARDIAN_MARGIN = 1.
- tools/claims.py: OK, client 180 (docs/09:40 updated).
- Not tested: the main.mjs wiring (the per-set Map and the checkFloor
call). No test runs checkFloor; deleting the call leaves every test
green.
Sibling search: guardianMarginProblem's callers are vault-check.mjs (an
info line) and guardianSetWarning. The status snapshot
(snapshot.mjs:33) prints live guardians and the quorum, not the margin;
left as it is.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Judge each guardian set's margin on its own, and say when it is fixed
The parent's per-set memory (which set shapes were already reported)
lived in main.mjs, so no test could fail on it: dropping the
once-per-shape check, or keying every set the same, left every test
green. The warning did not say which set it was about (only a program
tag on Floor lines), and it gave the same text for a set ejected down
to its threshold, or registered n-of-n before finding 96. There one
rogue guardian alone locks governance; a lost key alone does not, since
dead guardians cannot veto the owner's new set (docs/16 §4.4), but it
leaves no quorum until that set executes. Nothing was said when the owner's
new set fixed the problem. And a delivered alert did not say which Floor
program it was about: the program went only into the line's `program`
field, which alertFor uses in its repeat key but does not show.
Moving the program name into the text is a trap: alertFor keys repeats
on the first 60 characters of the text, and "the Floor guardians of
<44-character id>" pushes the shape past them. A 2-of-3 set ejected to
2-of-2 within ten minutes would then give the same key, and the worse
warning would not alert. So the program goes after the shape: "the
Floor guardians are 2-of-3 for <program>: ...". The `program` field
stays for the repeat key.
guardianSetWarning becomes guardianSetCheck(shapes, which, set,
mainnet) in policy.mjs. It keeps each set's last shape and whether it
was short in the Map, keyed by "protocol" or the protected program, and
returns the line to log or null. The warning names the set and has its
own text for n-of-n. When a set that was short gets the margin, it
returns one line saying so, which alerts on mainnet like the warning,
so whoever was paged hears that it is fixed. main.mjs only logs what it
returns. docs/07 §6 says so. The advice also had a hole: a set
registered as 7-of-8 was told to "run at least 7-of-9", and no set can
have 9 keys (core MAX_GUARDIANS is 8). When threshold + 2 is over 8,
the advice is now to lower the threshold to at most 6-of-8. That needs
MAX_GUARDIANS in JS, so lib/solana.mjs gets it next to
GUARDIAN_MARGIN; the next commit ties both to the core.
Evidence (tasks/floor-guardian-margin/):
- node --test cloud/test/*.test.mjs: 180/180 pass (the test is
extended, not added).
- 19 mutants, each run on policy.test.mjs and reverted (r6/mutants.py,
r6/mutants.log, r6/N5-margin1.log). Each fails the extended test, and
only it: never warn, threshold fixed at 2, notify always, notify
never, GUARDIAN_MARGIN = 1, no once-per-shape, one Map key for every
set, the n-of-n text for threshold + 1, no fixed line, a fixed line
for a set that was never short, the fixed line never alerting, no
program tag, every set named as the protocol's, the short flag not
stored, the program before the shape, the program left out of the
text, `need < 8` for `need <= 8` (6-of-7), the 8-key limit ignored
(7-of-8), 7-of-8 as the lowered advice.
- The mainnet alerts are checked on their own in a scratch copy of the
test without the earlier text assertions (r5/C1-C3-alone.log): with
the program left out of the text, the alert's detail check fails;
with a long name pushing the shape past character 60, "2-of-2 alerts"
fails.
- A's review of the previous version (branch
auto/B/floor-guardian-margin-r7, kept) found two survivors there: X3,
the shape keyed on the count only, and X5, the advice
threshold-of-(n+1). It also found that the n-of-n text said a lost key
alone locks governance. The test now has a threshold-only fix (7-of-8
-> 6-of-8, the advice's own step) and pins the n-of-n text and advice.
r9/mutants.log: X3, X5, the r7 n-of-n text and the text without its
lost-key clause are all KILLED. r9/mutants-r7.log: X3 and X5 SURVIVED
on the r7 code and test. The 18 mutants of r6/mutants.py, re-run on
this code: all KILLED (r9/r6-mutants-rerun.log). Two code-review
passes on the fix (kept as -r8 and -r9) added the lost-key clause: a
lost key alone leaves no quorum (it never approves, and core `quorum`
needs threshold approvals) until the owner's new set executes. A third
pass (-r10) fixed the comments' wording only.
- tools/claims.py: OK, client 180.
- Still not tested: the checkFloor call in main.mjs (no test runs
checkFloor).
Not changed (code review): the "fixed" line needs the short shape to
have been seen since the guardian started (the Map is in memory, as the
old per-process shape was; docs/07 says so). A protocol that abdicated
SetGuardians can't take a new set, so the advice to run 2-of-4 can't be
followed there; REPORT lists that case for the owner. The fault text
assumes a margin of 2, the owner's decision (16 §4.4).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Tie the JS guardian margin and set size limit to the core constants
cloud/lib/solana.mjs keeps its own GUARDIAN_MARGIN = 2, which the
guardian and the deploy tooling use to judge a guardian set, and now
MAX_GUARDIANS = 8, which the guardian's advice uses. Their comments say
they are core governance.rs's constants, but nothing checked that. Had
the programs' margin moved to 3, every JS test would have stayed green
while the guardian passed sets the programs refuse.
deploy.test.mjs now reads governance.rs and requires exactly one
`pub const GUARDIAN_MARGIN: u8 = N;` and one `pub const MAX_GUARDIANS:
usize = N;`, each N equal to the JS constant. It reads the source text,
like the build-script test above it. If a declaration changes form, the
test fails rather than passing.
Evidence (tasks/floor-guardian-margin/r6/tie-mutants.log):
- node --test cloud/test/*.test.mjs: 181/181 pass (one new test).
- Mutants of governance.rs, each reverted with its mtime kept (else the
limiter differential asks for a worker rebuild): GUARDIAN_MARGIN = 3,
MAX_GUARDIANS = 9, and a trailing comment on MAX_GUARDIANS. Each fails
the new test, naming the constant. deploy.test.mjs is green again.
- tools/claims.py: OK, client 181 (docs/09:40 updated).
Sibling search: the other JS copies of core rules (the limiter port,
the account layouts) are checked by the differential and by decoding
real accounts. MIN_GUARDIAN_THRESHOLD has no JS copy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Reach the 7-day extension total in the wide differential
differential_wide.rs drew extension seconds only from {0, 1, 3600,
86400, 86401} on a random ticket and lumped every refusal into one
counter. A ticket never got near the 7-day total, no negative value was
sent, and a refusal was not checked by its code. So the default run
could not tell the total bound, a negative extension or the error code
from a bug: only the unit tests at vault.rs:3382-3401 pinned them.
Now one extension draw in three pushes the chosen ticket by up to 24h
a call to exactly the 604,800 s total, then asks for 1 s more, so the
edge itself is hit on every push, not only when earlier draws line up.
The other draws are half legal sizes (1, 3600, 86400) and half out of
range for one call (i64::MIN, -86400, -1, 0, 86401, i64::MAX). Each
refusal must fail with EXTENSION_TOO_LONG. The counters are split into
extend (a random legal size), extend_push, extend_refused_call and
extend_refused_total, all required >= 10, so pushes cannot stand in for
the random sizes.
The model reads both bounds from bulkhead-core (consts.rs:37-38), as
the module says it is built, rather than restating 86,400 and 604,800.
docs/09 §7's row says so, and the 16 Oct "Vault refusals deleted" row,
which says the default run never extended a ticket that far, now says
it does since this change (its differential_wide.rs:406 cite had drifted).
Default run (24 seeds x 250 steps): extend 16, extend_push 126,
extend_refused_call 16, extend_refused_total 30 (was extend 40,
extend_refused 27); every other required counter still >= 10
(loosen_executed 12, as before).
Evidence: mutants of brakes.rs extend_ticket, each built and run against
the old and the new differential_wide only:
T1 drop the total check old SURVIVED new KILLED
T2 total > to >= old SURVIVED new KILLED
T3 drop secs <= 0 old KILLED new KILLED
T4 secs <= 0 to < 0 old KILLED new KILLED
T5 total ignores this call's secs old SURVIVED new KILLED
T6 extended_s += secs / 2 old SURVIVED new KILLED
T7 refusal returns WrongTicket old SURVIVED new KILLED
T8 secs <= 0 to == 0 (negatives) old SURVIVED new KILLED
T9 total > to > total + 1 old SURVIVED new KILLED
(T9: a draft that pushed by 24h until refused killed it only through
chance 1 s and 3600 s draws on the same ticket; the push to exactly
604,800 then 1 s more reaches the edge every time.)
sBPF sha256 after the run unchanged (Floor 8ca8a608, Harbor 543be60b,
Vault 7086148b): tests only.
Sibling search: the other refusal counters in this file (reset_refused,
lifeboat_refused_*, loosen_*) are each split by cause already.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check settle and lifeboat refusals by code in the wide differential
The same gap as the extension refusal, found in the sibling search for
the previous commit: a refused settle and the three lifeboat-move
refusals asserted only r.is_err(). Any refusal passed, so the order of
the checks and the program's own codes went unchecked by the
differential. Settle's comment (custody.rs:520-521) says it returns its
own error, not the token program's, when the vault is short; nothing in
this file held it to that.
Now a settle refused while frozen must fail with FROZEN, and one that is
only short with INSUFFICIENT_BALANCE (frozen is checked first). A
lifeboat move inside the delay after the seal must fail with NOT_SEALED,
one while frozen with FROZEN, and one beyond the balance with the token
program's InsufficientFunds (1). The two settle refusals get their own
counters: settle_refused_frozen is required >= 10 like the rest;
settle_refused_short needs lifeboat moves to drain a due ticket's
tokens first and occurs 7 times in the default run, so it is required
>= 1.
Evidence: mutants of programs/vault/src, each built and run against the
previous head's differential_wide and this one, nothing else:
S1 settle: drop the InsufficientBalance check old SURVIVED new KILLED
S2 settle: balance check before the freeze old SURVIVED new KILLED
L1 require_sealed returns Frozen old SURVIVED new KILLED
L2 move_in's freeze returns NotSealed old SURVIVED new KILLED
The token-program code (1) on a move beyond the balance is pinned but
not shown red by a mutant. sBPF sha256 unchanged (Floor 8ca8a608,
Harbor 543be60b, Vault 7086148b): tests only.
Deep run of both commits together (BULKHEAD_PROGRAM_SEEDS=600
BULKHEAD_PROGRAM_STEPS=500, 85 s): 300,000 steps, all identical;
extend 876, extend_push 6,462, extend_refused_call 1,175,
extend_refused_total 1,453, settle 2,024, settle_refused_frozen 921,
settle_refused_short 566, lifeboat refusals 3,854 + 5,768 + 1,171. docs/09 §7's
row gets that run as a 16 Oct result next to the 5 Oct one, which
predates these checks.
Sibling search: in differential_wide.rs, reset (TICKETS_PENDING) and
execute change already asserted codes, and a release or quote refused
for want of tokens asserts INSUFFICIENT_BALANCE. A release or quote the
limiter refuses (frozen, lane closed, queue full) is still compared only
as a refusal: mapping the core's refusal to the program's code is left
for its own change. Lifeboat returns only send legal amounts. docs/09's
row now says which refusals are checked by code. differential.rs:327
has the same is_err settle check; left as it is, since the wide run now pins both
settle codes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check limiter refusals by code in the wide differential
A release or quote that the limiter refused (lane closed, queue full)
was compared with the model only as Err. Any refusal code passed: a
program that swapped LaneClosed and QueueFull in custody.rs's
limiter_error, or checked the backed part against the balance before
the limiter and so answered InsufficientBalance for a full queue,
still matched the model. About 670 such refusals happen in a default
run.
The model now maps core's LimiterError to the program's code with
verr::limiter in the test harness (all four variants, as custody.rs's
limiter_error does), and every refused quote and release must fail
with the model's code. Lane-closed and queue-full refusals have their
own counters, each required at least 10 times; a refusal code with no
counter panics rather than being counted as want of tokens. docs/09
row 362 says so and records the deep run.
Evidence (autopilot tasks/differential-limiter-codes/):
- mutants.log: C1 (LaneClosed and QueueFull swapped in
limiter_error), C2 (QueueFull -> InsufficientBalance), C3
(LaneClosed -> ZeroAmount) and C4 (backed > balance checked before
the limiter, same line) each SURVIVED the old test and are KILLED
by the new one. Hashes back to 8ca8a608/543be60b/7086148b after.
- c4-full-suite.log: under C4 the whole program suite (Vault 113,
Floor 50, the fixed differential) passes; only this test fails.
C1-C3 are also caught by vault.rs's code tests (:434, :3428).
- Default run: refused_lane_closed 254, refused_queue_full 414,
insufficient 118.
- deep-600x500.log: 300,000 steps, 15,561 lane closed, 18,853 queue
full, 6,345 want of tokens, all identical.
Sibling search: every other Err arm in differential_wide.rs already
asserts a code (extend, settle, reset, lifeboat in, execute change).
Not here: lifeboat returns are never refused (backlog), and lifeboat
move_in beyond the balance fails with the token program's code
(backlog, a program change).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check release refusals by code in the fixed differential
differential.rs reduced every refused release to Err(()) on both
sides, so a program that refused for the wrong reason still matched
the model. The same gap the previous commit closed in the wide
differential (found by its code review).
The model's refusal now carries the program's code: verr::limiter for
a limiter refusal, INSUFFICIENT_BALANCE when the vault holds less
than it would pay. A refused release must fail with that code, and
the assert prints the program's error. Refusals for a full queue get
their own counter, required at least 10 times like the others. Want of
tokens and a closed lane are counted and printed but not required:
the wide differential requires both. A refusal with any other code
panics, as in the wide differential, so the counts always add up.
docs/09 row 361 says so; its undated figure (4,843 releases) no longer
matched a run, so it is replaced by a dated one.
Evidence (autopilot tasks/differential-limiter-codes/):
- mutants-narrow.log (mutants.sh <wt> differential): C1 (LaneClosed
and QueueFull swapped), C2 (QueueFull -> InsufficientBalance) and C4
(backed > balance checked before the limiter) SURVIVED the old test
and are KILLED by the new one. C3 (LaneClosed -> ZeroAmount)
survives both: no release in this test's runs is refused for a
closed lane. The wide differential kills it, and docs/09 says so.
Hashes back to 8ca8a608/543be60b/7086148b after.
- r2/default-run.log: 1,000 steps, 82 releases refused, all for a
full queue (want of tokens 0, lane closed 0).
- r2…
shuv-amp
added a commit
that referenced
this pull request
Oct 18, 2026
…oint's error (#114) * Run the tests on 2-of-4 guardian sets; test recovery from two faults Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set with fewer than threshold + 2 keys. Every test fixture and most tests' own sets were 2-of-3. This moves them to 2-of-4, which today's rule also accepts, so this commit changes no program: Floor, Vault and Harbor hash 508e5a99 / b07655d9 / 543be60b, as before. - Fixtures: Protected (Floor) and Market (Vault, through Harbor) now hold 4 guardians. Each test that built its own 3-key set gets a 4-key set with the same threshold and the same assertion. - Two tests changed meaning with a 4th key and were fixed, not loosened: a_quorum_ejects_a_rogue_guardian_who_vetoes_everything (Floor) and a_quorum_ejects_a_guardian_who_freezes_forever (Vault). With 4 keys, the last eject was refused only because the target's own vote is masked. Both now eject a second guardian (a set an ejection shrank to threshold + 1 still ejects) and then show the 2-of-2 refusal. - Account sweeps: an eject's signers may now be any two of the three others, so each of those two slots is free for the third, never for the target or the other signer again. Vault 5,365 and Floor 1,710 substitutions, refused (docs/09, dated 16 Oct). Listing all three others at each slot would have skipped the duplicate: with a mutant that counts a guardian listed twice as two (governance.rs:270), the sweeps listing all three pass, and these fail at "eject: account 2 replaced by key ..." (Vault and Floor). No other program test kills that mutant (tasks/aud2-guardian-margin-impl/dup-mutant-*.log). - floor_differential: the guardian pool is 6 keys, 4 registered; the proposed sets are 2 to 6 keys. New tests (pass on both rules; they show what the margin buys): - a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_set (Floor): g0's key lost, g1 rogue stops the program; the lost key does not help a quorum; the honest pair ejects g1 and restores; neither can veto the owner's refill, which executes after 72h. - two_rogues_in_turn_are_both_ejected_from_a_two_of_four_set (Floor): the second rogue vetoes the refill and is ejected from the 2-of-3. - a_lost_key_and_a_rogue_do_not_lock_a_two_of_four_vault: the same in the Vault (freeze, eject, unfreeze, refill). Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest and quote sweep): the counts and the sweep figures here are from runs there (tasks/aud2-guardian-margin-impl/r4/). The E2E run, the mutants and the new-rule run below were made before, on e8640a7; this commit's programs and core are the same as there but for one comment line in views.rs (line-neutral, same hashes). Evidence: - E2E=1 scripts/check-programs.sh: PROGRAM CHECKS PASSED (Floor 49, Vault 98; S1 and E2E passed; the resumed setup found all 14 steps; vault check, Floor demo and deploy dry run passed), with the next commit's deploy rehearsal in the tree. - tools/claims.py: OK (floor 49, vault 104, programs 153). - Under the new rule (core from two commits on, programs rebuilt), only the two tests that state the old rule fail: an_n_of_n_guardian_set_is_ refused (accepts 3-of-4) and the Floor differential's model. - Mutants (tasks/coverage-map/mutants.sh), each red, then restored: AV1 Floor live_guardian ignores liveness: KILLED by a_lost_key_and_a_ rogue_do_not_lock_a_two_of_four_set (+2 others). AV2 the Vault's: KILLED by ..._two_of_four_vault (+2 others). AQ1 Floor restore quorum ignores liveness: KILLED by ..._two_of_four_set and the differential only. AQ2 the Vault's unfreeze quorum: KILLED by ..._two_of_four_vault and a_silent_guardian_does_not_count_toward_a_quorum. AE1 can_eject refuses at count == threshold + 1 (AUD-2 option (a)): KILLED by two_rogues_in_turn_..., and the two eject tests above. Hashes after: 508e5a99 / 543be60b / b07655d9. Split on 16 Oct from ec0cacc at the owner's review (D2): the deploy rehearsal's move to 4 guardians is now the next commit. programs/ and programs/tests/ here are byte-identical to ec0cacc, so its runs below hold for this commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Rehearse deploys with four guardians, and read only the keys there are Finding 107 (AUD-2, owner decision 16 Oct) will refuse a guardian set with fewer than threshold + 2 keys, so the deploy rehearsal must register 2-of-4 sets. Split from ec0cacc at the owner's review (D2); the text is that commit's, unchanged: Deploy rehearsal (cloud/deploy market.mjs, vault-check.mjs, floor-demo.mjs): 4 guardians (keys/guardian3), so the dry run keeps working once the rule lands; one more funding step (14, was 13). The devnet market has 3 guardian files. A first version of this change loaded guardian3 everywhere: vault-check.mjs failed there with ENOENT, and heartbeat silently created a key nobody holds as a guardian. Now market.mjs guardianKeys(dir, create) reads guardian0, 1, ... and throws on a gap (never fills it with a new key). Setup creates keys up to 4 only while the protocol is not registered, so resuming an old market adds no key the chain lacks; a new deploy-dryrun step resumes the dry-run market with guardian3 moved aside and fails if setup makes it again (with setup always creating: "DRY RUN FAILED: resuming a registered market made keys/guardian3.json", r3-dryrun-J7.log). renew-guardians tops up to 4 and logs each new key (unfunded); a 2-of-3 renewal would be refused once the rule lands. keysIn reads the guardians only when a command uses them, so caps and set-recovery never need them; heartbeat says a key is not in the set rather than dead. One THRESHOLD (2) for the register and renewal payloads and MIN_GUARDIANS. vault-check.mjs compares the chain's set with the first files, and liveness with the chain's count. An extra file (a renewal not executed yet) and a set without the margin are info lines, not failures, since such a set keeps working. GUARDIAN_MARGIN and guardianMarginProblem live in cloud/lib/solana.mjs, used by these and by the guardian bot's existing warning (cloud/guardian/main.mjs), and vault-check uses THRESHOLD. The e2e scripts fund guardian3 too. Three tests in cloud/test/deploy.test.mjs (client tests 176 -> 179). JS mutants (js-mutants-r3.log), each red then restored: always create, never create, no gap check, margin 1, `>`, GUARDIAN_MARGIN = 1. Setup always creating survives these tests and is killed by the dry run step above. cloud/, scripts/deploy-dryrun.sh and docs/09's client-test row are byte-identical to ec0cacc (git diff empty), so its runs hold for them: the E2E check-programs run with the dry run (r4), and the JS mutants. tools/claims.py: OK (client 179). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Refuse a guardian set without two keys beyond its threshold (107) AUD-2, decided by the owner on 16 Oct (option (e) of 16 §4.4). A set stays live while its honest live guardians reach the threshold: they eject every rogue in turn, and dead guardians cannot veto the owner's new set. Finding 96 asked for one key beyond the threshold, so a 2-of-3 or 3-of-4 set survived one fault but not a lost key plus a rogue: the honest guardian left is no quorum, nobody can be ejected, and the rogue keeps Floor stopped (or the Vault frozen) and vetoes every new set. Fix, in the core so both programs inherit it through GuardianSet::parse (registration, and every SetGuardians when proposed and when executed): GUARDIAN_MARGIN = 2 guardian_set_ok := count <= 8 && t >= 2 && count >= t + 2 (saturating_add: t is a payload byte). A stored set is not checked again, so existing 2-of-3 sets keep working, and ejection is unchanged: a shrunk set still ejects. A 2- or 3-key set is BadThreshold, not BadCount; the count rule is only the bitmap size, and the error order of finding 103 is kept. This changes both programs, by intent: Floor 508e5a99 -> 8ca8a608, Vault b07655d9 -> 7086148b, Harbor 543be60b (unchanged). Darwin-arm64 builds. Compute (scratch LiteSVM test, tasks/aud2-guardian-margin-impl/cu.md): -3 CU on every path that parses a set (Floor propose 2035 -> 2032, execute 1682 -> 1679; Vault register 9592 -> 9589, SetGuardians 2979 -> 2976; Floor register varies with its PDA bump, residue also -3). Proof and tests: - Kani: accepted_sets_can_eject_anyone restated (ejection reachable exactly when count > t, so option (a) cannot slip in; accepted sets have >= 4 keys and a margin of at least 2, and 2 is accepted). New: an_honest_live_quorum_ejects_any_rogue (any slots, anyone else alive) and accepted_sets_survive_any_two_faults (any two faulty keys, lost or rogue, leave a live honest quorum, which ejects each rogue, and the rogues alone eject no honest guardian). That holds only short of a rogue quorum: two rogues in a 2-of-n set are one (out of scope, 04 T7). With the assumption rogue < t dropped, the harness fails at `!can_eject(rogue, ..)` (kani-no-rogue-assume.log), so the docs say "short of a rogue quorum", not "any two faults". 19/19 verified, 240 s (cargo kani -j 2, as CI); again after the restatement, 235 s summed (r2-prove.log). - Kani mutants (kani-mutants.sh): margin 1, margin 3, `>` for `>=`, t >= 1, count <= 9, option (a), and an eject one vote short (only the two-fault harness kills that one): each KILLED by a harness (kani-mutants-r2.log). The eject ignoring liveness is the eject harness's case, not these. - Program mutants (coverage-map/mutants.sh, rebuilt each time): margin 1: KILLED by a_guardian_set_without_two_spare_keys_is_refused (Floor and Vault), registration_rejects_weak_settings and the Floor differential. Margin 3 and `>`: KILLED by most of the suite (the fixtures are 2-of-4). t >= 1: first only the Floor differential; the 1-of-2 registration case was now refused by the margin too, so it is a 1-of-4 here and a Vault 1-of-4 case was added: then KILLED by both. count <= 9: SURVIVES the program tests, equivalent there (parse refuses more than 8 keys first), KILLED by Kani. Option (a): KILLED by the eject tests two commits back (the tests on 2-of-4 sets). - Tests: the Floor and Vault refusal tests above cover proposal and execution (a stored 2-of-4 proposal whose threshold turns 3 is refused when executed, as the devnet renewals proposed before this rule will be); registration_rejects_weak_settings gains 2-of-3 and 3-of-4. an_n_of_n_guardian_set_is_refused (Vault) is renamed and accepted a 3-of-4: it now refuses it and accepts 3-of-5. - Model: the Python rule, the Floor differential's model, and the guardian_set fuzz target state it again; the GP generator now also picks count - 2. With the Python rule put back to the old one, seed 11 (20,000 cases) reports 5 GP mismatches; restored, seeds 11-13 identical. Fuzz seeds two_of_three_refused, two_of_four, six_of_eight, and seven_of_eight renamed seven_of_eight_refused (byte 323, the threshold, is 7: now refused). With margin 1 in the core the target crashes within 60 s; restored, 222,615,288 runs in 601 s, no crash. - scripts/check.sh: ALL CHECKS PASSED, and E2E=1 check-programs.sh: PROGRAM CHECKS PASSED, both at the branch tip, whose code is this commit's (the next commit is docs only; tasks/aud2-guardian-margin- impl/r4/check.log, r4/check-programs-e2e.log). claims: kani 19, floor 50, vault 104, programs 154, client 179. Rebased on 16 Oct onto 12b91a8 (findings 102 and 100, and the attest and quote sweep). The core and the program sources are the same as before the rebase but for one comment line in views.rs (line-neutral), so the Kani, fuzz, differential and mutant runs above were made on the pre-rebase commit; the hashes, check.sh, check-programs and the counts are from r4 runs (tasks/aud2-guardian-margin-impl/r4/). The finding 96 row said the restated harness proves "exactly"; it now says what it proved then and proves now, and gives the Vault test's new name. The finding 107 row waits for rows 105 and 106 to land (09 §4 is numbered 1..n); its text is ready. The rule's prose is the next commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * State the guardian margin in the docs, with its costs (AUD-2) The previous commit made both programs refuse a new guardian set without two keys beyond its threshold (finding 107). The docs that state the rule now say so: - README: guardians 4-8 keys; the G5 row and the "who the guardians are" paragraph give the reason (any two faults leave a quorum, unless the rogues are one themselves: two in a 2-of-n set, 04 T7). - 05 register_protocol: 2 <= threshold <= count - 2, also for every SetGuardians, proposed and executed. 05 abdicate: abdicating SetGuardians fixes the set, so ejections shrink it for good. - 06 state table, 00 D22, FAQ: the same rule. - 07 §2: 4-8 guardians, and don't abdicate SetGuardians. 07 §6: the programs enforce the margin; sets registered before keep their exposure until replaced; an ejection uses up a spare key until the owner's new set executes; a lost key counts as live until its interval passes. - 16: the AUD-2 row is decided; §4.4 gives the decision, why the margin is the number of faults a set survives (short of a rogue quorum), the options rejected and why (the honest-owner and stolen-owner cases look the same on-chain), what it costs, what it leaves (a rogue quorum; a stolen owner key with every guardian dead; a rogue's veto of the recovery key's rotation, checked: Floor's veto has no recovery exception, as a_vetoed_recovery_proposal_does_not_hold_the_owner_s_slot shows), and Lido-style lapsing vetoes as a future option only. §5: "independent guardians, 3-of-5 or more". - 10: the guardian set's minimum size is no longer an open decision. SECURITY.md and 08 describe what is on devnet, which still runs the old rule, and are left as they are. claims: OK. Docs only; no program file changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Say the weekly CI run skips mutants, as its condition does ci.yml's header and scripts/ci-changes.sh said the weekly run does every job, and 09 §8 gave `mutants` as running on any change to the core. The `mutants` job's `if` starts with `github.event_name == 'pull_request'`, so the Monday schedule (and a manual dispatch) never runs it: it mutates a pull request's diff and has none to mutate. Its trigger list in 09 §8 also left out scripts/mutants.sh and .cargo/mutants.toml, which ci-changes.sh (line 22) selects it for. Comments and docs only; nothing runs differently. Sibling search: `git grep` for "weekly run does", "every job ... weekly" and "mutants ... weekly" across md/yml/sh/mjs/py. The other hits are 09 §8's "at full depth every Monday" (its table now says which job is not weekly) and the dispatch input's "every job at weekly depth", which is the same set as the weekly run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Check that the mutant exclusions still match; repin murmur3 (finding 105) .cargo/mutants.toml excludes the core's 22 equivalent mutants, and the docs say so. Since 9 Oct it excluded 19. The murmur3 tail's pattern is pinned by line, `floor.rs:(235|238|241)`, because the body's `h ^= k` has the same column and function and is not equivalent. 1dc7861 (#52) moved those lines (now 327/330/333), and nothing checked the patterns. Failing closed, the three would have failed CI's `mutants` job on a pull request that changed those lines. Failing open, a pattern that starts matching a non-equivalent mutant hides it from every run. Root cause: nothing checked that a pattern still names its mutants. Fix: - each pattern ends with the number of mutants it excludes (`# 3`), and the murmur3 pattern names 327/330/333; - tools/mutant_exclusions.py fails unless each pattern matches exactly its number in `cargo mutants --list --no-config`, and cargo-mutants (the list with the config) excludes exactly the mutants the patterns match, so a pattern this parser can't see, or reads differently, fails too. No two patterns may match the same mutant, since the docs' count is the counts' sum, and the kept list must be part of the full one. It reads only the exclude_re array's body, needs no tomllib (Python 3.9), and names colour codes in the list as the problem; - scripts/mutants.sh makes both lists (--colors never: CI sets CARGO_TERM_COLOR=always) and runs it first, except when the caller passes --config or --no-config (as whole arguments). CI's `mutants` job runs that script; ci-changes.sh now also selects the job for the new tool; - scripts/check.sh runs its --self-test on planted configs; - tools/claims.py ties the docs' "22 equivalents" (README, 09 §6 and the summary row) to the sum of the counts, with the same parser. It finds that parser from its own directory, so `python3 -I` runs it too, and check.sh runs its self-test that way. 00's "18 equivalents" (26 Sep log entry) and run 9's "exactly the 22 equivalents below" (6 Oct) are kept as dated records. Docs: 09 §4 finding 105, §6 run 10 and the "In CI" note, §8's mutants row; README and 09's summary row 669 → 797 mutants (run 10); README and 12's post-submission range 96–105. Evidence (first on 1f2ad48 + the parent, logs in the autopilot's tasks/mutants-exclusions-r2/; re-run on 58697c6 + the parent, tasks/mutants-exclusions-r3/): - `cargo mutants --list`: 797 with --no-config, 778 with the old config (19 excluded), 775 with this one. The 22 excluded are §6's list, the murmur3 ones at 327/330/333. - scripts/mutants.sh --list: exit 0, "22 of 797 mutants excluded"; with CARGO_TERM_COLOR=always also exit 0. Old config: exit 1, every pattern "no '# N' count". Planted on the real lists: the old lines with a count (matches 0, says 3) and the body XOR added (matches 4, says 3) both exit 1; in both the set comparison alone passes. /usr/bin/python3 (3.9.6): self-test OK and the real check passes. - The self-test pins each problem by its message, not a count. Mutants of the tool (tool-mutants.log), each red: M1 no per-pattern check, M2 no set comparison, M3 no colour check, M4 no empty-array check, M5 a missing count accepted, M6 a missing count read as 0, M7 the parser reading past the array. M3 and M5 survived a first version that checked only the number of problems. After A's review (16 Oct): the set comparison was pinned only by its size, so `len(excluded) != len(matched)` passed the self-test. A case with as many excluded as matched but not the same ones now fails it (15 cases; red with that mutant, green without), and the message says how many mutants are in one set only. From the code review after the rebase: colour codes only in the kept list were reported as a kept list made with other options; the colour check now reads both lists (16 cases; dropping the kept half fails the new case). On 58697c6, with the overlap and kept-list checks: M8 no overlap check and M9 no kept-list check, red too, as M1–M7 still are. A duplicate of the can_eject pattern planted in the real config: scripts/mutants.sh --list exits 1, "the counts add up to 23, the patterns match 22 mutants" (planted-overlap.log). Without the sys.path line, `python3 -I tools/claims.py` fails with ModuleNotFoundError; with it, both claims runs pass. - claims.py (claims-mutants.log): one count 1 → 2 fails four statements (README 188; 09 33, 333, 335; kept-run9.log), not run 9's 331, which failed too before it was kept; a dropped count fails with the tool named; deleting any of the three rules fails a self-test case; deleting the KEPT entry fails 00:103. - The 22 excluded mutants run alone (--no-config, --re per pattern, 2 min): 22 missed, exactly the excluded list; --re also picked ema.rs:70 (a field deletion, no pattern matches it), caught. - Run 10 (f1f05ba, old config, -j4, 46 min; tasks/mutants-run10/): 778 tested, 712 caught, 5 timeouts, 58 unviable, 3 missed, exactly the murmur3 tail. The core's source is unchanged since (`git diff --stat f1f05ba HEAD -- crates/bulkhead-core/src` is empty; only tests/units.rs gained finding 102's tests). - No program or core source changed. Rebased on 16 Oct onto cede705 (findings 102 and 100, the attest sweep and AUD-2). AUD-2 changed the core's source (governance.rs), so the line above saying it is unchanged since run 10 held on 58697c6, not here. On cede705: scripts/mutants.sh's list check passes, "22 of 795 mutants excluded, each pattern matching its count"; the murmur3 lines are still 327/330/333. The 797 in README and 09 is run 10's figure: README now dates it ("in the last full run, 15 Oct") and finding 105's row says the core has changed since (finding 107). The setup line in scripts/mutants.sh now pins cargo-mutants 27.1.0, the version ci.yml installs: each count is the number of mutants that version lists, so another release can break the check (code review, 16 Oct; `git grep 'install.*cargo-mutants'` finds no other unpinned install). The check's report goes to stderr in scripts/mutants.sh, so `scripts/mutants.sh --list` prints only cargo-mutants' names: before, its first stdout line was the report; after, stdout is 773 lines (795 - 22) and stderr the one report line. ci.yml's `mutants` comment says the job checks the exclusions first. Row 105 and 09 §6 gave the pin's reason as the body's XORs. The finalizer's are at column 7 (`grep -n '\^=' floor.rs`, 337-342); the non-equivalent XOR at column 11 is `h ^= k`, in the body (320) and after the tail (335). The toml comment, the row and §6 now say that. 09 §8's `check` row names the two tool self-tests check.sh runs. README no longer calls run 10's 797 "injected bugs": cargo-mutants listed 797 but tested 778, 19 being excluded (09 §6, run 10's row). Sibling search: the other 14 patterns use `\d+` for the line, and each matches the count now written after it. The murmur3 one is the only line-pinned pattern. Not done, by choice: - Catching an edit that keeps a pattern's number but changes its mutants (an excluded line itself edited, or a murmur3 body XOR moved onto a pinned line). The toml and 09 §6 say so: such an edit needs the mutant judged again by hand. Failing on any diff that touches an excluded line would catch the first; it is a separate change. - Moving murmur3's tail into its own function, so its pattern needn't name lines. It shifts core lines, so it changes the program hashes. Until then a line move above it fails the check, with a message saying what to fix. - A weekly full mutants run in CI. It took 46 min here at -j4, likely past the job's 90 min on two CI cores: a minutes call for the owner. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test that a ticket stays in its own protocol and vault Two Vault checks that keep a ticket to its own protocol and vault had no test. Deleting either one left every program test green (tasks/error-codes/mutants.log, base cede705): - brakes.rs:79, extend_ticket: tk.protocol != protocol. Without it, a live guardian of any other protocol extends this protocol's tickets. Anyone can register a protocol and name its guardians, so that guardian could hold back every queued payout up to the total extension limit. The sweep did not catch it: it has no second protocol's guardian to swap in. - custody.rs:501, settle_ticket: tk.vault != vault. A USDC ticket settled through the SOL vault (its tokens, its mint) got past the Vault's checks. New test another_protocols_guardian_cannot_extend_a_ticket: a second Market's live guardian extends this market's ticket -> WRONG_VAULT; the same instruction from this market's guardian succeeds. The settle test now settles the USDC ticket through the SOL vault and expects WRONG_VAULT. Evidence (tasks/error-codes/mutants.log, STATE=c1; full program suite per mutant): W02 brakes.rs:79 deleted -> KILLED by another_protocols_guardian_cannot_extend_a_ticket ("expected failure, got success": the stranger's extension went through) W06 custody.rs:501 deleted -> KILLED by a_ticket_settles_after_its_eta_to_its_stored_recipient_only (8049 InsufficientBalance from the empty SOL vault, not 8010) Both were SURVIVED at base. No program source change: hashes 8ca8a608/543be60b/7086148b, as at cede705. Sibling search: the other 12 WrongVault sites were deleted one at a time in the same run. Eight are KILLED by the sweep (brakes.rs:39, lib.rs:320, custody.rs:160, genesis.rs:491, views.rs:27, lifeboat.rs:151, governance.rs:169/259). lib.rs:324 (vault PDA address), genesis.rs:523, lifeboat.rs:71 and :247 survive; later commits on this branch deal with them. Counts: Vault 104 -> 105, programs 154 -> 155 (claims.py OK). The test also reads the ticket's ETA: unchanged after the refused extension, 3,600 s later after its own guardian's (code-review fix: success alone did not show the extension happened). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test that the seal names every vault and happens once Two seal checks had no test; deleting either left every program test green (tasks/error-codes/mutants.log, base cede705): - genesis.rs:486, execute_seal: the pairs must number 3 * vault_count. Without it, the seal checks only the vaults it is given. A vault left out skips its claims-vs-balance check, so an uncovered import seals. docs/05's seal row says every vault is passed exactly once. the_seal_checks_each_vault_once covered a vault named twice in place of another, not one left out. - genesis.rs:431, propose_seal: refused once sealed (AlreadySealed). Without it, a sealed protocol's admin re-proposes, which rewrites seal_executable_s/expires_s and clears seal_vetoed on a sealed protocol. the_seal_checks_each_vault_once now also leaves the uncovered USDC vault out (expects VAULT_COUNT_MISMATCH) and, after the seal, proposes it again (expects ALREADY_SEALED). No new test, so no count change. Evidence (tasks/error-codes/mutants.log, STATE=c2, full program suite): E05 genesis.rs:486 deleted -> KILLED by the_seal_checks_each_vault_once ("expected failure, got success": the seal went through without the USDC vault and its 1,000 USDC uncovered import) E04 genesis.rs:431 deleted -> KILLED by the_seal_checks_each_vault_once ("expected failure, got success") Both were SURVIVED at base. Vault hash 7086148b unchanged (tests only). Sibling search: the seal's other checks. :491 vault_tokens (KILLED by the sweep), :519 lifeboat owner and :523 lifeboat mint/owner (SURVIVED; the lifeboat is a PDA checked by expect_pda just above, which only this program can create, as its own token account: see this branch's note in docs/09). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test that invalid limits and asset settings are refused at every entry Three bound checks had no test; deleting any one left every program test green (tasks/error-codes/mutants.log, base cede705): - genesis.rs:171, create_vault: config_is_valid (bps <= 100%, windows > 0). Without it, a vault is created with a 100.01%-an-hour burst. - governance.rs:40, parse of a SetLimits change (at proposal and again at execution, :254 and :478): the same rule for a proposed change. - governance.rs:47, parse of an UpsertAsset change: valid_asset (haircut <= 100%, liability >= 100%, the oracle bounds of finding 88). Only the genesis upsert's copy (genesis.rs:296) was tested, by oracle_settings_of_a_priced_asset_are_bounded. New test invalid_limits_and_assets_are_refused_wherever_they_enter: before the seal, create_vault with short_bps 10,001 -> INVALID_LIMITS; after it, a SetLimits proposal with long_window_s 0 -> INVALID_LIMITS, and an UpsertAsset proposal with a 10,001 bps haircut -> INVALID_ASSET. Evidence (tasks/error-codes/mutants.log, STATE=c3, full program suite): E06 genesis.rs:171 deleted -> KILLED ("expected failure, got success") E07 governance.rs:40 deleted -> KILLED ("expected failure, got success") E08 governance.rs:47 deleted -> KILLED ("expected failure, got success") each by the new test only. All three were SURVIVED at base. Vault hash 7086148b unchanged (tests only). Sibling search: every config_is_valid / valid_asset call in programs/ vault/src: genesis.rs:170 (create_vault), :295 (upsert_asset, KILLED by the oracle test), governance.rs:39 and :46. All now killed. Counts: Vault 105 -> 106, programs 155 -> 156 (claims.py OK). The execution copy (:478) is not reached by any transaction: it parses the bytes the proposal already checked, and the bounds read nothing else. The test comment says so (code-review fix: it had claimed execution was covered). oracle_settings_of_a_priced_asset_are_ bounded now asserts INVALID_ASSET for each of its four cases, not any error (all four return it; E09 stays KILLED). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test that an import into a ninth mint is refused genesis.rs:410, import_backing: a backing ledger holds MAX_SLOTS (8) mints, and an import into a ninth is refused with NoSlot. Deleting the check left every program test green (tasks/error-codes/mutants.log, base cede705). From the code, without it the import goes through: the vault's imported total grows (:414), and the seal must cover that, but the owner's ledger is saved unchanged, so the owner has no backed claim for it. New test an_import_into_a_ninth_mint_is_refused: nine vaults before the seal; one owner gets an import into eight (ledger_of shows 8 positions), then the ninth -> NO_SLOT. Evidence (tasks/error-codes/mutants.log, STATE=c4, full program suite): E14 genesis.rs:410 deleted -> KILLED by an_import_into_a_ninth_mint_is_ refused ("expected failure, got success"); SURVIVED at base. Vault hash 7086148b unchanged (tests only). Sibling search: the only other ledger credit is deposit's (custody.rs:128). There a full ledger is by design: the deposit is taken unbacked (Ledger::credit's doc, ledger.rs:187-189), and credit's result is the backed flag, not an error. Counts: Vault 106 -> 107, programs 156 -> 157 (claims.py OK). import_ix now calls import_into_ix, which takes the vault, so the new test shares the one import layout (code-review fix). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pin six Vault refusals no test reached Six reachable checks had no test that failed when the check was deleted (tasks/error-codes/mutants.log, base cede705). Each now has an assertion on its code, and each was seen to fail with the check deleted: E02 genesis.rs:185 create_vault of an existing vault after the seal: ALREADY_INITIALIZED (with the check gone: 8043 GenesisOnly, so the "says so first" order in the comment above it was untested) E12 governance.rs:426 veto of an executed change: CHANGE_EXECUTED (gone: the veto succeeds and marks an executed change vetoed) E13 governance.rs:517 cancel of an executed change: CHANGE_EXECUTED (gone: the cancel succeeds) E16 lifeboat.rs:256 execute_exit with no exit proposed: NOTHING_TO_EXIT (gone: 8033 ChangeNotExecutable) E17 custody.rs:84 a zero deposit: ZERO_AMOUNT (gone: it succeeds) E18 custody.rs:272 a zero release: ZERO_AMOUNT (gone: it succeeds; a zero release has no unbacked part, and the limiter, whose own zero check custody.rs:242 maps, is called only when unbacked > 0, :361) Where: E02 in after_the_seal_a_new_vault_starts_with_its_unbacked_lane_ closed, E12/E13 in an_unvetoed_loosening_executes_after_the_delay, E16 in the_lifeboat_stays_shut_until_the_seal (after the move in, before an exit is proposed), E17/E18 in the new a_zero_deposit_or_release_is_ refused. Evidence (tasks/error-codes/mutants.log, STATE=c5, full program suite): all six KILLED by the tests named above; all six SURVIVED at base. The vault suite is green (108 passed). Vault hash 7086148b unchanged. Counts: Vault 107 -> 108, programs 157 -> 158 (claims.py OK). Helpers (code-review fixes): veto_change_ix beside cancel_change_ix; create_vault_for_ix, create_vault_ix with a given mint and token program, so E02's case names USDC's own accounts and creates no mint. E02's comment now repeats the program's ("whatever the seal's state"): create_vault checks the limits first (genesis.rs:170). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pin four Vault checks another layer backstops Four checks survived deletion (tasks/error-codes/mutants.log, base cede705) because something later still refused the case, with its own error, so no test could tell. Each now has a test that sees the Vault's own code, and each was seen to fail with the check deleted: E01 lib.rs:230, create_pda's guard: a second registration of an existing protocol (register_protocol has no check of its own) -> ALREADY_INITIALIZED. Gone: the system program refuses it, with no custom code (custom_code 0). W10 lifeboat.rs:71, move_in naming another mint or token program -> WRONG_VAULT. Gone: the token program refuses (code 3, MintMismatch). W12 lifeboat.rs:247, execute_exit naming another mint or token program -> WRONG_VAULT. Gone: the same token-program error. T01 token.rs:37, account_info on an account no token program owns: a copy of a real USDC account's bytes under the system program. Gone: the quote SUCCEEDS (it promises a payout), and the release fails only in the runtime at the transfer ("instruction modified data of an account it does not own"). The quote is asserted first so the mutant is seen at the quote (STATE=c6b). New tests: a_protocol_registers_once, a_lifeboat_move_names_the_vaults_ own_mint (also completes a move in and an exit, as a control: the destination receives 1,000 USDC), a_payout_goes_only_to_an_account_a_ token_program_owns. Evidence (tasks/error-codes/mutants.log, STATE=c6 and c6b, full program suite): E01, W10, W12, T01 KILLED by the tests named above; all four SURVIVED at base. Full program suite green: Floor 50, Vault 111. Vault hash 7086148b unchanged (tests only). Counts: Vault 108 -> 111, programs 158 -> 161 (claims.py OK). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test the bounds on ticket extensions and a closed lane's code B's review of vault-error-tests-r3 (R1) found two Vault codes no test asserted by value that the earlier commits missed: ExtensionTooLong and LaneClosed. Deleting the 7-day total extension cap (brakes.rs:86, B24) left every program test green, so a live guardian, or a stolen guardian key, could hold a queued payout back without end, 24h at a time, against docs/05 row 11. The wide differential models the total (differential_wide.rs:406), but its default run never extends one ticket that far. Mapping LaneClosed to QueueFull (custody.rs:240, B23) also survived. The code review of the fix found a worse gap at the same check: no test sends a negative extension (the differential draws secs from 0, 1, 3600, 86400, 86401). With `secs <= 0` weakened to `secs == 0` (A3, brakes.rs:84) every program test passed, and one guardian key could pull a queued payout's ETA into the past, skipping the queue delay, and refund its own 7-day budget. - a_ticket_is_extended_later_by_at_most_a_day_a_call_and_seven_in_total: -1h, 0 and 24h + 1 s are each refused with EXTENSION_TOO_LONG, ETA unchanged; six 24h extensions and one of 23h pass; 2h more is refused, ETA unchanged; the last hour is granted; then 1 s more is refused. The split is unaligned so that a total that ignores this call's `secs` fails too. - a_closed_lane_refuses_an_unbacked_payout: a vault with short_bps 0, and one with long_bps 0, each refuse an unbacked admin payout with LANE_CLOSED; the same payout from a vault with both buckets open is taken (the control, so a capacity base of 0 cannot pass it). Evidence (tasks/vault-error-tests-r4/, B's runner copied: each mutant applied alone, programs rebuilt, whole program suite run): - without these tests (551e519; run-without.out, run-base2.out, run-base3.out): B23, B24, A1 (`tk.extended_s >= MAX`, this call's secs ignored), A2 (`>=` for `>`) and A3 SURVIVED. A4 (the 24h per call deleted), A5 and A6 (either half of limiter.rs:172's closed-lane test deleted) were KILLED by the differential only, not by value; - with them (run-final.out, at the branch tip): B23 KILLED by a_closed_lane_refuses_an_unbacked_payout; B24 A1 A2 A3 KILLED by the extension test only; A4 by it and the differential; A5 A6 by the closed-lane test and the differential. No program change: the runner's closing build gives 8ca8a608 / 543be60b / 7086148b, as at cede705. Counts: Vault 111 -> 113, programs 161 -> 163 (claims.py OK). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pin each half of create_pda's existing-account guard on its own B's review of vault-error-tests-r3 (R2): each half of lib.rs:229 (`!is_data_empty() || !owned_by(SYSTEM)`) survived deleted on its own (B20, B21), since an existing protocol account fails both, so a_protocol_registers_once pinned them only jointly. The test now also plants, at a fresh protocol address, an account that fails just one half: one with data owned by the system program, one with no data owned by another program (a fresh address). Each registration is refused with ALREADY_INITIALIZED. The comment says no transaction is known to make either account: data or a new owner at a PDA needs its signature, which only the Vault gives, and create_pda gives both in one instruction. Evidence (tasks/vault-error-tests-r4/): at 551e519 B20 and B21 SURVIVED (run-base2.out); at the branch tip both are KILLED by a_protocol_registers_once only (run-final.out). No count change (claims.py OK: vault_programs 113, programs 163). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Record the Vault refusal deletions and what each became docs/09 §3 gains a row for the deletion run this branch answers (tasks/error-codes/mutants.log): 38 refusals, across 14 of the 18 Vault codes no test asserted by value at cede705, each deleted on its own with the whole program suite run against it, on the AUD-2 build. The 18 (each VaultError variant with no `verr::` use in programs/tests at cede705): AlreadyInitialized NotRegistrar WrongVault WrongTokenProgram AlreadySealed VaultCountMismatch InvalidLimits InvalidAsset LaneClosed TicketNotPending ExtensionTooLong ChangeExecuted LedgerUnsound LedgerCorrupt NoSlot NothingToExit ZeroAmount Overflow. The run missed LaneClosed and ExtensionTooLong; B's review deleted one check of each (B23 custody.rs:240, B24 brakes.rs:86), and both survived until 1797ae5. That commit also pins a negative extension (A3, which survived every test before it), the 24h per call, and a lane closed by either bucket. At base: 13 KILLED, 25 SURVIVED. The commits before this one add tests that kill 18 of the 25 and both of B's, each asserting the Vault's own code (W02 W06, E04 E05, E06 E07 E08, E14, E02 E12 E13 E16 E17 E18, E01 W10 W12 T01; B23 B24). The row names the 7 that still survive and why: - six guards no transaction is known to reach: lib.rs:324 (vault PDA address; create_vault creates a vault only at its PDA), genesis.rs:519 and :523 (the lifeboat, checked by expect_pda first, created only by lifeboat_move, move_in requires the seal, lifeboat.rs:53, and the seal happens once, so no lifeboat holds data at it), brakes.rs:50 and custody.rs:541 (a pending-ticket count below zero), token.rs:235 (GetAccountDataSize's return data from an already-checked program); - custody.rs:399, stated as not shown unreachable: properties.rs:588-592 debits backed_amount and one more on random ledgers, and only crates/bulkhead-core/tests/units.rs:375 debits less, on a one-mint ledger with no liability. properties.rs:606 asserts backed(a) == min(a, backed(MAX)) for requests below 2^52 (:613), so any smaller amount is backed too, and debit_backed(b) refuses only when backed_amount(b) != b (ledger.rs:289). The known exception is a collateral value that overflows u128 (ledger.rs:262): the valid amounts need not form an interval there, so the consent or freeze cap (custody.rs:337, :347) can lower the amount to an invalid one and :399 refuses the release (fail closed). The search's own result is always valid. Not verified by a tool beyond that property test. (A full ledger cannot reach :399 through set's NoSlot: backed_amount's valid() refuses a new liability slot on a full ledger, ledger.rs:237.) Not deleted, as the row says: the two core-error mappings to ChangeExecuted and ZeroAmount, token.rs:233, and the codes Overflow and LedgerCorrupt (checked arithmetic, decoding). After this branch five codes are still asserted by no test (same grep at this commit): NotRegistrar (E03, genesis.rs:108, was KILLED at base by another test), TicketNotPending, LedgerUnsound, LedgerCorrupt, Overflow. Docs only: no count change (claims.py OK: vault_programs 113, programs 163). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Run the exclusion check's main in its self-test A's review of finding 105 (REVIEWS/auto-B-mutants-exclusions-r7.md, finding 1, Low): CI's `mutants` job acts on main's exit status, but the self-test called problems() only, and check.sh never runs the tool on real lists. With main returning 0 on a mismatch (mutant A10), every in-tree check passed, and CI would have printed the problems and gone green. This was not a defect at fb771c6, since the job fails today. The gap was that a later regression would pass. Fix: main takes its arguments and, for the self-test, a config path (default: the repo's .cargo/mutants.toml, as before), and its lines name the config it read. The self-test runs main in-process on a planted config and checks the exit status and every line on each stream: - the right lists: exit 0, "1 of 3 mutants excluded" on stdout, nothing on stderr; - the lists the other way round: exit 1, nothing on stdout, and exactly the count and "full list lacks" problems, so KEPT is read as KEPT; - coloured lists: exit 1 with exactly one problem, so a main that fails only on several problems is caught; - one argument and three: exit 2 and the two usage lines. One subprocess (python3 -I, 60 s timeout, in the temporary directory so the config must be found from the tool's own path) runs the command line with two lists and the repo's config, as scripts/mutants.sh does: on a list no pattern can match, it exits 1 with an empty stdout and problem lines that each name .cargo/mutants.toml. Which problems those are depends on the config, so only their form is checked. Self-test: 16 cases -> 22. No doc states the number. Evidence (tasks/exclusions-main-exit/): - old-A10.log: the fb771c6 self-test with A10 applied prints "self-test OK: 16 cases" and exits 0. - mutants-r6.log (mutants.py applies each mutant to a scratch copy of the tool, which reads the real config through a symlink): unmutated, "self-test OK: 22 cases", exit 0. All 18 are KILLED (exit 1): A10 (return 0), A10b (fail only on more than one problem), A0 (always 1), U1 (usage returns 0), U2/U3 (argument check `< 2`, `< 1`), K1 (KEPT ignored), K2 (ALL read twice), C1 (planted config ignored), C2 (no default config), R1 (the default config read from the cwd), E1/E2 (sys.argv sliced as [:] or [1:2]), P1 (problems not printed), S1 (problems on stdout), L1 (the planted config not named), N1 (only the first problem printed), G1 (the success line printed on failure too). Code-review passes at high shaped this. The first version ran three subprocesses against the real config, and A0, K1 and U3 survived it. The second left E2, C2 and G1 alive. Declined from the third: catching exceptions in the self-test (a traceback already exits 1) and moving the config hook out of main. The fourth showed that requiring one "matches 0 mutants" line per live pattern tied the self-test to the config's contents. The subprocess now checks the form of the lines only, and the 17 stay killed. The fifth showed that R1 survived while the child ran in the repo root (it does in check.sh); now it runs in the temporary directory. Its other points repeat declined ones. - real-lists.log: on the real lists, made with scripts/mutants.sh's commands: ".cargo/mutants.toml: 22 of 795 mutants excluded, each pattern matching its count", exit 0. - python3 -I tools/mutant_exclusions.py --self-test: 22 cases OK. - scripts/check.sh: ALL CHECKS PASSED (check-final.log); bin/check-each-commit.sh green (each-commit.log). Sibling search: the other tools check.sh runs (tools/claims.py, tools/floor_check.py) also gate CI through main's exit status. claims.py has the same gap: check.sh runs it on the real tree, where it finds no problems, and its self-test calls self_test(), not the path in main that returns 1. With that return changed to 0, `--self-test` and the real run both still exit 0 (claims-C1.log). Not changed here: it is its own commit (BACKLOG-B). floor_check.py runs in CI only as --self-test, whose result main returns as is. Its RPC mode is an operator's tool that no CI job runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Record finding 106: Vault liveness and exit rules had no failing test The tests landed in 944f5e8 (lifeboat-exit-tests-r3, 15 Oct). Their 09 §4 row waited for 104 and 105 so the list stays numbered without gaps; 105 is now on integration (fb771c6), so the row lands here. Docs only: 09 §4 row 106; README and 12's count and post-submission range (96–106, 106 fixed). Two code-review passes changed the row as written on 15 Oct: a sentence for today's re-run, the nine single-guardian sites named, G5 quoted verbatim, and the exit veto attributed to T18 (G6 has no veto), as is the veto test's doc comment (vault.rs:3996, comment only). Evidence, re-run today on bf8ba41 (the row's claims are from 15 Oct, and AUD-2 changed the Vault since): B's re-review list of 33 line- neutral mutants (the ten rules, the nine single-guardian sites L1-L9, the boundaries, the six quorums, R1-R6), each applied, programs rebuilt, all program tests run: 33 KILLED, 0 survived, 0 not applied, 0 build failures. Vault hashes after the run match the tip (Vault 7086148b, Floor 8ca8a608, Harbor 543be60b). Log: autopilot tasks/finding-106-row/mutants.log. - python3 -I tools/claims.py: OK, findings 106. - git grep for "96–105", "105 findings": no other stale range. - scripts/mutants.sh still mutates the core only (SCOPE, line 11), so the row's last sentence holds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Cite PR #100's tree, not the unpublished bf8ba41, in finding 106 Finding 106's row in docs/09 said its 33 mutants were re-run on 16 Oct at bf8ba41. That commit is not an ancestor of this branch or of the stack above it and is on no remote branch, so the citation would point at nothing once the stack lands. The owner's review of #99-#109 found it. bf8ba41 and 86b777b (the head of PR #100, vault-error-tests-d2, the D2 replay of vault-error-tests-r4) have the same tree, 36187b49 (`git rev-parse bf8ba41^{tree} 86b777b^{tree}` prints it twice). So the run was a run on 86b777b's tree, and the row now says "on tree 36187b49, the tree of 86b777b (PR #100's head)". Why not just "at 86b777b": it was committed on 17 Oct, after the run; and main squash-merges (1045cd9 has one parent), so after PR #100 lands 86b777b is reachable only as GitHub's refs/pull/100/head. The tree id stays checkable from main: squashed bottom first, main's commit for PR #100 has that tree. "PR" keeps "#100" from reading as finding 100, the row above. These points came from two code-review passes. Sibling search: every 7-12 hex token in tracked .md/.py/.sh/.mjs/.rs/ .toml files at autopilot/integration 3e9f58d that names a local commit (51 tokens) was checked with git merge-base --is-ancestor against integration. bf8ba41 was the only one that is not an ancestor. Docs only. tools/claims.py: claims OK (26 files). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Warn about a Floor guardian set short of the margin (finding 107) The guardian checks the Vault protocol's guardian set against finding 107's margin (main.mjs checkGuardianSet), but never a Floor config's. A Floor set registered before the rule, say 2-of-3, keeps working and keeps AUD-2's exposure: one lost key and one rogue guardian lock its governance for good. The guardian said nothing, and docs/07 §6 said so ("not yet a Floor set"). The rule moves to policy.mjs as guardianSetWarning, a pure function: the same text as before, a warn line that alerts on mainnet only. In main.mjs, checkGuardianSet now remembers the set shape per set ("protocol", or the protected program), and checkFloor calls it for each watched Floor config it reads. floorAlerts is unchanged. The margin is not a readiness code there: "floor not ready" alerts on every cluster, and only the owner's new set clears a margin problem, so "floor ready" would never be logged again. Evidence (tasks/floor-guardian-margin/r2/): - node --test cloud/test/*.test.mjs: 180/180 pass (one new test). - Mutants of guardianSetWarning, each run on policy.test.mjs and reverted; each fails the new test: N1 never warn, N2 threshold fixed at 2, N3 notify always, N4 notify never, N5 GUARDIAN_MARGIN = 1. - tools/claims.py: OK, client 180 (docs/09:40 updated). - Not tested: the main.mjs wiring (the per-set Map and the checkFloor call). No test runs checkFloor; deleting the call leaves every test green. Sibling search: guardianMarginProblem's callers are vault-check.mjs (an info line) and guardianSetWarning. The status snapshot (snapshot.mjs:33) prints live guardians and the quorum, not the margin; left as it is. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Judge each guardian set's margin on its own, and say when it is fixed The parent's per-set memory (which set shapes were already reported) lived in main.mjs, so no test could fail on it: dropping the once-per-shape check, or keying every set the same, left every test green. The warning did not say which set it was about (only a program tag on Floor lines), and it gave the same text for a set ejected down to its threshold, or registered n-of-n before finding 96. There one rogue guardian alone locks governance; a lost key alone does not, since dead guardians cannot veto the owner's new set (docs/16 §4.4), but it leaves no quorum until that set executes. Nothing was said when the owner's new set fixed the problem. And a delivered alert did not say which Floor program it was about: the program went only into the line's `program` field, which alertFor uses in its repeat key but does not show. Moving the program name into the text is a trap: alertFor keys repeats on the first 60 characters of the text, and "the Floor guardians of <44-character id>" pushes the shape past them. A 2-of-3 set ejected to 2-of-2 within ten minutes would then give the same key, and the worse warning would not alert. So the program goes after the shape: "the Floor guardians are 2-of-3 for <program>: ...". The `program` field stays for the repeat key. guardianSetWarning becomes guardianSetCheck(shapes, which, set, mainnet) in policy.mjs. It keeps each set's last shape and whether it was short in the Map, keyed by "protocol" or the protected program, and returns the line to log or null. The warning names the set and has its own text for n-of-n. When a set that was short gets the margin, it returns one line saying so, which alerts on mainnet like the warning, so whoever was paged hears that it is fixed. main.mjs only logs what it returns. docs/07 §6 says so. The advice also had a hole: a set registered as 7-of-8 was told to "run at least 7-of-9", and no set can have 9 keys (core MAX_GUARDIANS is 8). When threshold + 2 is over 8, the advice is now to lower the threshold to at most 6-of-8. That needs MAX_GUARDIANS in JS, so lib/solana.mjs gets it next to GUARDIAN_MARGIN; the next commit ties both to the core. Evidence (tasks/floor-guardian-margin/): - node --test cloud/test/*.test.mjs: 180/180 pass (the test is extended, not added). - 19 mutants, each run on policy.test.mjs and reverted (r6/mutants.py, r6/mutants.log, r6/N5-margin1.log). Each fails the extended test, and only it: never warn, threshold fixed at 2, notify always, notify never, GUARDIAN_MARGIN = 1, no once-per-shape, one Map key for every set, the n-of-n text for threshold + 1, no fixed line, a fixed line for a set that was never short, the fixed line never alerting, no program tag, every set named as the protocol's, the short flag not stored, the program before the shape, the program left out of the text, `need < 8` for `need <= 8` (6-of-7), the 8-key limit ignored (7-of-8), 7-of-8 as the lowered advice. - The mainnet alerts are checked on their own in a scratch copy of the test without the earlier text assertions (r5/C1-C3-alone.log): with the program left out of the text, the alert's detail check fails; with a long name pushing the shape past character 60, "2-of-2 alerts" fails. - A's review of the previous version (branch auto/B/floor-guardian-margin-r7, kept) found two survivors there: X3, the shape keyed on the count only, and X5, the advice threshold-of-(n+1). It also found that the n-of-n text said a lost key alone locks governance. The test now has a threshold-only fix (7-of-8 -> 6-of-8, the advice's own step) and pins the n-of-n text and advice. r9/mutants.log: X3, X5, the r7 n-of-n text and the text without its lost-key clause are all KILLED. r9/mutants-r7.log: X3 and X5 SURVIVED on the r7 code and test. The 18 mutants of r6/mutants.py, re-run on this code: all KILLED (r9/r6-mutants-rerun.log). Two code-review passes on the fix (kept as -r8 and -r9) added the lost-key clause: a lost key alone leaves no quorum (it never approves, and core `quorum` needs threshold approvals) until the owner's new set executes. A third pass (-r10) fixed the comments' wording only. - tools/claims.py: OK, client 180. - Still not tested: the checkFloor call in main.mjs (no test runs checkFloor). Not changed (code review): the "fixed" line needs the short shape to have been seen since the guardian started (the Map is in memory, as the old per-process shape was; docs/07 says so). A protocol that abdicated SetGuardians can't take a new set, so the advice to run 2-of-4 can't be followed there; REPORT lists that case for the owner. The fault text assumes a margin of 2, the owner's decision (16 §4.4). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Tie the JS guardian margin and set size limit to the core constants cloud/lib/solana.mjs keeps its own GUARDIAN_MARGIN = 2, which the guardian and the deploy tooling use to judge a guardian set, and now MAX_GUARDIANS = 8, which the guardian's advice uses. Their comments say they are core governance.rs's constants, but nothing checked that. Had the programs' margin moved to 3, every JS test would have stayed green while the guardian passed sets the programs refuse. deploy.test.mjs now reads governance.rs and requires exactly one `pub const GUARDIAN_MARGIN: u8 = N;` and one `pub const MAX_GUARDIANS: usize = N;`, each N equal to the JS constant. It reads the source text, like the build-script test above it. If a declaration changes form, the test fails rather than passing. Evidence (tasks/floor-guardian-margin/r6/tie-mutants.log): - node --test cloud/test/*.test.mjs: 181/181 pass (one new test). - Mutants of governance.rs, each reverted with its mtime kept (else the limiter differential asks for a worker rebuild): GUARDIAN_MARGIN = 3, MAX_GUARDIANS = 9, and a trailing comment on MAX_GUARDIANS. Each fails the new test, naming the constant. deploy.test.mjs is green again. - tools/claims.py: OK, client 181 (docs/09:40 updated). Sibling search: the other JS copies of core rules (the limiter port, the account layouts) are checked by the differential and by decoding real accounts. MIN_GUARDIAN_THRESHOLD has no JS copy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Reach the 7-day extension total in the wide differential differential_wide.rs drew extension seconds only from {0, 1, 3600, 86400, 86401} on a random ticket and lumped every refusal into one counter. A ticket never got near the 7-day total, no negative value was sent, and a refusal was not checked by its code. So the default run could not tell the total bound, a negative extension or the error code from a bug: only the unit tests at vault.rs:3382-3401 pinned them. Now one extension draw in three pushes the chosen ticket by up to 24h a call to exactly the 604,800 s total, then asks for 1 s more, so the edge itself is hit on every push, not only when earlier draws line up. The other draws are half legal sizes (1, 3600, 86400) and half out of range for one call (i64::MIN, -86400, -1, 0, 86401, i64::MAX). Each refusal must fail with EXTENSION_TOO_LONG. The counters are split into extend (a random legal size), extend_push, extend_refused_call and extend_refused_total, all required >= 10, so pushes cannot stand in for the random sizes. The model reads both bounds from bulkhead-core (consts.rs:37-38), as the module says it is built, rather than restating 86,400 and 604,800. docs/09 §7's row says so, and the 16 Oct "Vault refusals deleted" row, which says the default run never extended a ticket that far, now says it does since this change (its differential_wide.rs:406 cite had drifted). Default run (24 seeds x 250 steps): extend 16, extend_push 126, extend_refused_call 16, extend_refused_total 30 (was extend 40, extend_refused 27); every other required counter still >= 10 (loosen_executed 12, as before). Evidence: mutants of brakes.rs extend_ticket, each built and run against the old and the new differential_wide only: T1 drop the total check old SURVIVED new KILLED T2 total > to >= old SURVIVED new KILLED T3 drop secs <= 0 old KILLED new KILLED T4 secs <= 0 to < 0 old KILLED new KILLED T5 total ignores this call's secs old SURVIVED new KILLED T6 extended_s += secs / 2 old SURVIVED new KILLED T7 refusal returns WrongTicket old SURVIVED new KILLED T8 secs <= 0 to == 0 (negatives) old SURVIVED new KILLED T9 total > to > total + 1 old SURVIVED new KILLED (T9: a draft that pushed by 24h until refused killed it only through chance 1 s and 3600 s draws on the same ticket; the push to exactly 604,800 then 1 s more reaches the edge every time.) sBPF sha256 after the run unchanged (Floor 8ca8a608, Harbor 543be60b, Vault 7086148b): tests only. Sibling search: the other refusal counters in this file (reset_refused, lifeboat_refused_*, loosen_*) are each split by cause already. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Check settle and lifeboat refusals by code in the wide differential The same gap as the extension refusal, found in the sibling search for the previous commit: a refused settle and the three lifeboat-move refusals asserted only r.is_err(). Any refusal passed, so the order of the checks and the program's own codes went unchecked by the differential. Settle's comment (custody.rs:520-521) says it returns its own error, not the token program's, when the vault is short; nothing in this file held it to that. Now a settle refused while frozen must fail with FROZEN, and one that is only short with INSUFFICIENT_BALANCE (frozen is checked first). A lifeboat move inside the delay after the seal must fail with NOT_SEALED, one while frozen with FROZEN, and one beyond the balance with the token program's InsufficientFunds (1). The two settle refusals get their own counters: settle_refused_frozen is required >= 10 like the rest; settle_refused_short needs lifeboat moves to drain a due ticket's tokens first and occurs 7 times in the default run, so it is required >= 1. Evidence: mutants of programs/vault/src, each built and run against the previous head's differential_wide and this one, nothing else: S1 settle: drop the InsufficientBalance check old SURVIVED new KILLED S2 settle: balance check before the freeze old SURVIVED new KILLED L1 require_sealed returns Frozen old SURVIVED new KILLED L2 move_in's freeze returns NotSealed old SURVIVED new KILLED The token-program code (1) on a move beyond the balance is pinned but not shown red by a mutant. sBPF sha256 unchanged (Floor 8ca8a608, Harbor 543be60b, Vault 7086148b): tests only. Deep run of both commits together (BULKHEAD_PROGRAM_SEEDS=600 BULKHEAD_PROGRAM_STEPS=500, 85 s): 300,000 steps, all identical; extend 876, extend_push 6,462, extend_refused_call 1,175, extend_refused_total 1,453, settle 2,024, settle_refused_frozen 921, settle_refused_short 566, lifeboat refusals 3,854 + 5,768 + 1,171. docs/09 §7's row gets that run as a 16 Oct result next to the 5 Oct one, which predates these checks. Sibling search: in differential_wide.rs, reset (TICKETS_PENDING) and execute change already asserted codes, and a release or quote refused for want of tokens asserts INSUFFICIENT_BALANCE. A release or quote the limiter refuses (frozen, lane closed, queue full) is still compared only as a refusal: mapping the core's refusal to the program's code is left for its own change. Lifeboat returns only send legal amounts. docs/09's row now says which refusals are checked by code. differential.rs:327 has the same is_err settle check; left as it is, since the wide run now pins both settle codes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Check limiter refusals by code in the wide differential A release or quote that the limiter refused (lane closed, queue full) was compared with the model only as Err. Any refusal code passed: a program that swapped LaneClosed and QueueFull in custody.rs's limiter_error, or checked the backed part against the balance before the limiter and so answered InsufficientBalance for a full queue, still matched the model. About 670 such refusals happen in a default run. The model now maps core's LimiterError to the program's code with verr::limiter in the test harness (all four variants, as custody.rs's limiter_error does), and every refused quote and release must fail with the model's code. Lane-closed and queue-full refusals have their own counters, each required at least 10 times; a refusal code with no counter panics rather than being counted as want of tokens. docs/09 row 362 says so and records the deep run. Evidence (autopilot tasks/differential-limiter-codes/): - mutants.log: C1 (LaneClosed and QueueFull swapped in limiter_error), C2 (QueueFull -> InsufficientBalance), C3 (LaneClosed -> ZeroAmount) and C4 (backed > balance checked before the limiter, same line) each SURVIVED the old test and are KILLED by the new one. Hashes back to 8ca8a608/543be60b/7086148b after. - c4-full-suite.log: under C4 the whole program suite (Vault 113, Floor 50, the fixed differential) passes; only this test fails. C1-C3 are also caught by vault.rs's code tests (:434, :3428). - Default run: refused_lane_closed 254, refused_queue_full 414, insufficient 118. - deep-600x500.log: 300,000 steps, 15,561 lane closed, 18,853 queue full, 6,345 want of tokens, all identical. Sibling search: every other Err arm in differential_wide.rs already asserts a code (extend, settle, reset, lifeboat in, execute change). Not here: lifeboat returns are never refused (backlog), and lifeboat move_in beyond the balance fails with the token program's code (backlog, a program change). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Check release refusals by code in the fixed differential differential.rs reduced every refused release to Err(()) on both sides, so a program that refused for the wrong reason still matched the model. The same gap the previous commit closed in the wide differential (found by its code review). The model's refusal now carries the program's code: verr::limiter for a limiter refusal, INSUFFICIENT_BALANCE when the vault holds less than it would pay. A refused release must fail with that code, and the assert prints the program's error. Refusals for a full queue get their own counter, required at least 10 times like the others. Want of tokens and a closed lane are counted and printed but not required: the wide differential requires both. A refusal with any other code panics, as in the wide differential, so the counts always add up. docs/09 row 361 says so; its undated figure (4,843 releases) no longer matched a run, so it is replaced by a dated one. Evidence (autopilot tasks/differential-limiter-codes/): - mutants-narrow.log (mutants.sh <wt> differential): C1 (LaneClosed and QueueFull swapped), C2 (QueueFull -> InsufficientBalance) and C4 (backed > balance checked before the limiter) SURVIVED the old test and are KILLED by the new one. C3 (LaneClosed -> ZeroAmount) survives both: no release in this test's runs is refused for a closed lane. The wide differential kills it, and docs/09 says so. Hashes back to 8ca8a608/543be60b/7086148b after. - r2/default-run.log: 1,000 steps, 82 releases refused, all for a full queue (want of tok…
shuv-amp
added a commit
that referenced
this pull request
Oct 19, 2026
Row 106 said its 16 Oct re-run was on tree 36187b4, "the tree of 86b777b (PR #100's head)". PR #100 was pushed again before the owner merged it: refs/pull/100/head is now 7ded086, 86b777b is not an ancestor of it, and no ref on origin (branches or refs/pull/*/head) contains 86b777b or a commit with tree 36187b4. The citation named something nobody can fetch. What the re-run used is still on main. git diff 86b777b 04ad61e -- programs crates touches one file, programs/tests/tests/vault.rs, 2 lines: a doc comment of a_guardian_silent_past_its_interval_can_no_ longer_act rewrapped. So the row now cites main at 04ad61e (PR #100's squash) and says that one difference. Sibling search: every 7-12 hex token in README.md and docs/*.md (47) resolved with git cat-file; each commit checked with for-each-ref --contains against refs/remotes/origin and refs/pull/*/head. Only 86b777b was unpublished; 36187b4 was the only tree id. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Tests only. Agent A deleted the Vault's refusals one at a time to find checks that no test pins by value, then added tests for those checks. There are nine commits (adbd3b3..9f552fa). No program code changes: the sBPF hashes stay 8ca8a608 / 543be60b / 7086148b.
custody.rs:399, is stated as not shown unreachable. If it does fire, it fails closed.brakes.rs:86): deleting it left every program test green. A live guardian, or a stolen guardian key, could have held a queued payout back indefinitely, 24h at a time.brakes.rs:84): no test sent a negative extension. Weakeningsecs <= 0tosecs == 0passed every test. One guardian key could then have pulled a payout's ETA into the past.custody.rs:240): mapping it to QueueFull passed every test.create_pda's existing-account guard: each half is now pinned on its own.Proof
bin/check-each-commit.sh ea09733: 9/9 green.scripts/check.shat 86b777b: ALL CHECKS PASSED.Limits
custody.rs:399argument rests on a random property test (properties.rs:606, requests below 2^52). It is not a proof.Hashes
The owner's D2 split (17 Oct) replayed this unit with the same tree at every commit (autopilot tasks/pr/d2-hashmap.sed; checked with git rev-parse ^{tree} ^{tree}). The reviews and runs above were done at the old hashes and are quoted here at the new ones: f794f72..bf8ba41 = c263ff4..86b777b (c0f2b7d = 5874687, c6f41ff = 6293531, f8e2098 = cf960c9), and the base fb771c6 = ea09733.
Hashes after the sync onto main (17 Oct)
sync-main-97 replayed the stack onto main d7b9bdb (#97), after the owner's README redesign (#115). Each commit's own diff is unchanged, except README hunks ported to the new README. Outside README, every old/new pair differs only by main's own changes (autopilot tasks/sync-main-115/per-commit-tree.log treebad=0, tasks/sync-main-97/pairs.log bad=0). The ids under What changed are the new ones; the runs and reviews quoted elsewhere were done at the old ones. Old = new: c263ff4 = adbd3b3, 632d6b5 = 8687dba, bc72605 = dc993dd, b531e1d = 533562a, 3d75717 = 7931020, 5874687 = e4bd08d, 6293531 = e6bb61e, cf960c9 = ae52fe4, 86b777b = 9f552fa. On the synced chain:
bin/check-each-commit.sh60/60 green, andscripts/check.shALL CHECKS PASSED at its tip 2fa7a7f (tasks/sync-main-97/each-commit.log, check-tip.log). Agent B reviewed the sync: REVIEWS/auto-A-sync-main-97.md (APPROVE).🤖 Generated with Claude Code