Skip to content

Guardian config: refuse unknown alert and liveness keys; whole numbers - #128

Merged
shuv-amp merged 16 commits into
mainfrom
auto/B/config-nested-keys-r2
Oct 19, 2026
Merged

shuv-amp merged 16 commits into
mainfrom
auto/B/config-nested-keys-r2

Conversation

@shuv-amp

@shuv-amp shuv-amp commented Oct 18, 2026 •

Copy link
Copy Markdown
Owner

Opened by the Bulkhead autopilot agents: one reviewed unit of work. Every commit was reviewed adversarially by the other agent, with mutants, and passes tools/claims.py on its own.
Stacked on #127: merge that first; this PR is then rebased onto main.
The agents never merge. The owner reviews and merges.

What changed

The guardian's config check (cloud/guardian/policy.mjs), ALERT_FORMATS exported from alert.mjs, main.mjs passing the alerter only url and format, tests, and one sentence in docs/07. No program change. Two commits.

  1. 5035839: refuse unknown alert and liveness settings and alert formats. checkConfig refused an unknown top-level setting but never looked inside alert or liveness. A typo there fell back to a default without a word. "format": "slak" posted the JSON body, which has no text (Slack) or content (Discord) field, so the alert was likely not shown (the services' documented behaviour; not verified, localhost only). "every_s": 600 pinged every 60 s. docs/07 already said an unknown setting stops the start. Now alert takes only url and format, liveness only url and everyS, and format must be a listed format; "" is refused too. The alerter's undocumented options (windowMs, timeoutMs, retryDelaysMs) are now refused by name.
  2. 0f70133: take only whole numbers for counts and durations. docs/07 says seconds are whole numbers and checkPolicy refuses fractions, but the config's range loop took 10.5. No harm is shown (for example, ingestConcurrency: 2.5 works as 3). This is consistency with the documented rule, not a shown defect. minBalanceSol stays a fraction by design.

Proof

Autopilot tasks/config-nested-keys/:

  • repro-old.log: on the old code these configs gave no warning, and webhookBody(..., "slak") gave the JSON body. red-old.log and red-old-whole.log: the new cases fail on the previous code.
  • Mutants: 13/13 KILLED (mutants-r2.log) and 6/6 KILLED (mutants-whole.log). Each of the seven whole-number settings is tried at its lower bound + 0.5, inside its range, so only the integer test refuses it.
  • scripts/check.sh ALL CHECKS PASSED on the head, client 228 (r3/check.log). bin/check-each-commit.sh green over the whole stack of four units (r3/each-commit.log).
  • Agent A reviewed it: APPROVE with one nit. "format": "" had no test, and A's mutant X3 (a blank format let through) survived. "" is now refused, and X3 fails 1 test (r3/x3.log). A approved again (REVIEWS/auto-B-config-nested-keys-r2.md).

Limits

  • main.mjs builds the alerter before checkConfig runs (finding 71), so the fatal alert for a refused format is itself sent as the JSON body. Before this change, that guardian ran with every alert unseen. Now it stops, which shows on its console and in systemd. Picking the format from the URL host is backlogged.
  • This unit adds no finding row: it enforces rules docs/07 already states.

Hashes after the sync onto main (19 Oct)

The sync replayed the stack onto main bb55c15 (#114), whose tree is auto/B/failover-snapshot-r2's (8506759); #113's is auto/B/floor-follow-each's (51d20d3), both built on the main they merged into. The owner's #107 edited one sentence in README.md, docs/08 and docs/09 (the devnet job, and with it ci, fails for Floor and Vault until finding 107 is deployed); and #99's merge changed README's devnet line and the guardians figure (3-8 -> 4-8 keys). Each commit's own message is unchanged. Every old/new pair's trees differ by exactly that: README.md, docs/08, docs/09, docs/assets/src/figures.py and the two PNGs, the same +/- lines for all 38 synced commits (autopilot tasks/sync-main-114/per-commit-tree.log and map.txt). The ids under What changed are the new ones; the runs and reviews quoted elsewhere were done at the old ones. Old = new: c81f2be = 5035839, 0b233e9 = 0f70133. On the synced chain: bin/check-each-commit.sh bb55c15 41/41 green (the 38 synced commits, then the murmur3 exclusion fix, finding 106's citation and the full-ledger fuzz target), and scripts/check.sh ALL CHECKS PASSED at their tip (tasks/sync-main-114/each-commit.log, check-tip.log). Agent B reviewed the sync: REVIEWS/auto-A-sync-main-114.md.

🤖 Generated with Claude Code

@shuv-amp
shuv-amp force-pushed the auto/B/cluster-own-names-r2 branch from af7cd0d to 5454d71 Compare October 18, 2026 20:06
@shuv-amp
shuv-amp force-pushed the auto/B/config-nested-keys-r2 branch from 0b233e9 to 0f70133 Compare October 18, 2026 20:06
@shuv-amp
shuv-amp changed the base branch from auto/B/cluster-own-names-r2 to main October 19, 2026 17:25
shuv-amp and others added 16 commits October 19, 2026 23:10
The Vault's deposit refuses any caller but the protocol's, signed
(custody.rs:88, check_caller: `!caller.is_signer() || key(caller) !=
p.caller` -> WrongCaller). No test could fail if that call went: with
it replaced by `let _ = ...` (M1, from an independent review of
sweep-own-codes-r2), the whole program suite passed, the substitution
sweep included. In the sweep a deposit goes through Harbor, and a
swapped caller slot is refused by Harbor's own seeds check
(InvalidSeeds) before the Vault runs. The release's call of the same
check (:277) had a test, only_the_protocol_itself_can_release, but
only for an unsigned caller: with :277 keeping only the signer test
(R4k), every test passed too.

Without the deposit check, another protocol's integration, or anyone
calling the Vault directly with a key in slot 0, could deposit into
this protocol's vault and open a backing record the protocol never
saw. Without the release's key half, any signer in slot 0 could
release from this protocol's vaults.

New test only_the_protocol_itself_can_deposit: a second protocol on the
same Vault program, with its own Harbor (Market::with_ids), sends this
protocol's deposit, signed by its own caller PDA -> WRONG_CALLER; this
protocol's caller PDA unsigned, by a direct call (tag 7) ->
WRONG_CALLER; then the same deposit through this protocol's Harbor goes
through, and the vault holds it. only_the_protocol_itself_can_release
gains the second protocol's Harbor sending the user's own withdrawal ->
WRONG_CALLER, then the real withdrawal pays.

Evidence (tasks/sweep-own-codes/r3/mutants.log, whole program suite per
mutant; mutants-t2.txt):
  M1  :88  call deleted                    KILLED by the deposit test
  M1k :88  signer test only (key half gone) KILLED (the other Harbor's
           deposit succeeds)
  M1s :88  key test only (signer half gone) KILLED (the direct call
           succeeds)
  R4k :277 signer test only                 KILLED by the release test
           (the other Harbor's withdrawal succeeds)
  R4s :277 key test only                    KILLED by the release test
M1, M1k and M1s SURVIVED with the parent's tests (STATE=t1parent), and
R4k with only the deposit test added (STATE=t1sib). Floor, Harbor and
Vault sha256 8ca8a608, 543be60b, 7086148b, unchanged (tests only).

Sibling search: the other Vault checks behind a slot where Harbor or
another program refuses first in the sweep. The release's recipient
checks (:291, both halves of :295, :306) each fail a direct test
(R1 R2 R3 KILLED, STATE=t1sib; :291 is the review's M6). The deposit's
source check (:98) has a_deposit_cannot_come_from_the_vault_token_
account. Harbor's slot 1 in a registration is not a Vault account (the
registrar slot is swept), import_backing has no Vault check of its
system slot, and Floor has none of the loader's slot (each called by
its fixed id).

docs/05 §9 names the deposit test. Counts: Vault 115 -> 116, programs
173 -> 174 (claims.py OK).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An independent review of sweep-own-codes-r2 deleted the Vault deposit's
call of check_caller (custody.rs:88) and the whole program suite
passed: the substitution sweep sends deposits through Harbor, whose own
seeds check refuses a swapped caller PDA first. The sibling search of
the previous commit found the release's call (:277) tested only with
its caller unsigned; its key half also had no failing test. Finding
106's class, enforced but untested rules, so a finding of its own
(reserved 17 Oct 16:58Z, FINDINGS-RESERVED).

The previous commit adds the tests; this one records the finding in
09 §4 and moves the counts: README 111 -> 112 findings, "96–112 were
recorded" and 112 among the Vault-test fixes, docs/12's range 96–112
with 112 fixed (Vault tests). The two test names cited each exist once
in programs/tests/tests/vault.rs.

Docs only: claims.py OK, findings 112.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The account-substitution sweep (programs/tests/tests/common/sweep.rs)
counted any failed transaction as a refusal. So a program check whose
deletion leaves a CPI or the runtime to refuse the swap was invisible
to it: finding 111's eight Floor check calls (check_program_accounts,
check_authority) each passed the sweep with the call deleted, because
the loader then refused the swapped account (before finding 111's
test, the two in a stop failed another test and the other six none).

Measured first, on 47c096f (floor-check-calls before its restack;
the same programs and sweep as this commit's parent) with the sweep's
errors logged: of 7,075 refused swaps, 6,239 carried the swept
program's own code; the other 836 all fell at 31 slots (run, account):
loader-v3's slot in Floor's eight loader calls and the system
program's in import_backing (MissingAccount: each is invoked by its
fixed id), the token program at a deposit's source and authority
(Custom 1/3/4, InvalidAccountData, PrivilegeEscalation), and Harbor's
own checks before its CPI: its program and caller-PDA slots in every
Harbor run, its signer, a withdrawal's recipient and a payout's
protocol account (IncorrectProgramId, InvalidSeeds,
MissingRequiredSignature).

Fix: a Sweep knows its program's code block (from its first code,
err::/verr::INVALID_INSTRUCTION, 1000 wide: Floor 7000.., Vault
8000..; the token, system and loader programs use codes below 100). A
refusal with any other error is reported like an accepted swap,
unless the slot is listed with the new backstop(at, errors, why): every
swap there must still fail, and only with one of the named errors.
A key sent unsigned is excused only by MissingRequiredSignature, and
only if one of its own backstop slots names it. A named error that no
account swap in the run produced fails the sweep, so a backstop cannot
outlive its cause. A reported refusal carries the log line of the
program that failed first. The 31 slots above are listed. The sweep
prints how many refusals fell at backstop slots (Floor 163, Vault
673).

Stated limits (docs/09 §2 and the sweep's doc comment):
- At a backstop slot the sweep cannot see the program's own check
  deleted. The Vault's caller check behind Harbor's caller slot
  (custody.rs:88 in a deposit, :277 in a release) is one: the previous
  two commits pin it with direct tests (finding 112). Likewise the
  Vault's recipient checks behind Harbor's in a withdrawal (each has a
  direct test, and admin payouts reach them in the sweep) and the
  Vault's own checks of the key at Harbor's signer slot.
- A refusal is matched by its error and code block, not by which check
  or program returned it. A check whose deletion lets a later check of
  the same program refuse still passes: the release's recipient ==
  vault_tokens (custody.rs:291) deleted, the owner check after it
  returns the same DuplicateAccount, and only a_release_cannot_pay_any_
  account_the_vault_owns fails (M6 below).

docs/09 rows 110 and 111 and the §3 Floor deletion row said the sweep
checks only that a step fails. That was true at their dates; each now
says so ("then") and points to §2 for the change.

Evidence (autopilot tasks/sweep-own-codes/r3/; old = sweep tests built
at this commit's parent, new = at this commit, both in this worktree):
- 90 recorded line-neutral deletions on these programs (the lists of
  findings 108, 110 and 111; git diff 43bfa31 HEAD -- programs/floor/src
  programs/vault/src programs/harbor/src crates/ is empty, though
  43bfa31 is not in this history) and V01 below, each rebuilt and run
  against the old and the new sweep (plants.sh, plants-r3.out): 20 pass
  the old sweep and fail the new one (V01; K1-K8, the eight check calls
  of finding 111; F05 F12 F14 F14a F14b F15 F15a F15b F31 in Floor; W10
  W12 in the Vault lifeboat), each failure a "refused by ..., not the
  program's own code" line; 17 fail both; 54 neither; none fails only
  the old one. The same split as the run on r2 (r2/plants-r2.out) and
  as a first run on working-tree versions (plants-p2.out, -p4.out).
- V01, the deposit's own require_signer(authority) deleted
  (custody.rs:101): passes the old sweep and a first version of this
  one that excused an unsigned key at any backstop slot of that key
  (the token program then refuses, PrivilegeEscalation); fails this
  one ("deposit: account 8 ... unsigned ... refused by
  PrivilegeEscalation, not the program's own code").
- The review's six mutants (review-M1-M6.txt; review-sweeps.out for the
  sweeps, mutants.log STATE=r3rev for the whole program suite):
    M1 :88  caller check deleted       old PASS new PASS, suite KILLED
                                        (only_the_protocol_itself_can_deposit)
    M2 :98  source == vault_tokens     old PASS new FAIL, suite KILLED
    M3 :97  check_vault_accounts       old FAIL new FAIL, suite KILLED
    M4 stop.rs:295 programdata         old FAIL new FAIL, suite KILLED
    M5 :277 caller check deleted       old PASS new PASS, suite KILLED
                                        (only_the_protocol_itself_can_release)
    M6 :291 recipient == vault_tokens  old PASS new PASS, suite KILLED
                                        (a_release_cannot_pay_any_account_the_vault_owns)
  M3 and M4 fail the old sweep here as accepted swaps (a stranger's
  USDC account as the deposit's account 5; a fresh address and other
  accounts as close_buffer's account 5). The review's old column read PASS for every mutant
  because its old binaries were built in this worktree and so() loads
  target/deploy from CARGO_MANIFEST_DIR: they ran this worktree's clean
  programs, not the review's mutated build.
- With backstops ignored (a scratch build of this commit, the backstop
  match made `false &&`; nobackstop-*.log), the sweep fails at exactly
  the 31 listed (run, slot) pairs, 836 swaps. With an extra Custom(99)
  named at the loader backstop (scratch build, stale99-floor.log), it
  fails with "backstop at account N never refused with Custom(99)" in
  all 8 Floor runs.
- Counts unchanged: 5,365 Vault and 1,710 Floor substitutions over 37
  and 18 transactions. No test added or removed; programs untouched
  (Floor 8ca8a608, Vault 7086148b, Harbor 543be60b).
- programs/tests: cargo test --release 177 passed, 0 failed; clippy
  --release --tests -D warnings clean.

Sibling search (grep is_err() in programs/tests/tests): six single
refusals of a transaction still accept any failure, floor.rs:839 and
vault.rs:468, :1349, :2290, :4750, and differential.rs:341 (a settle
refused "frozen, or not enough tokens"). Each needs its expected code
worked out; left for a follow-up, not changed here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Six refusals in programs/tests asserted only `is_err()`. A deleted
program check that something later still refuses (another Floor check,
the token program, the system program) passed them. That is findings
110-112's class, here at single test sites rather than in the sweep.

Each now asserts the code the check under test returns. Codes were
read first from the clean programs (Floor 8ca8a608, Vault 7086148b) by
printing each error:
- floor.rs orphaned_buffers_...: close_buffer of the programdata or
  the program -> WRONG_BUFFER (loader.rs:17)
- vault.rs a_ticket_settles_...: settle to another's account ->
  RECIPIENT_OWNER_MISMATCH (custody.rs:503)
- vault.rs the_lifeboat_moves_...: exit to a thief's account ->
  RECIPIENT_OWNER_MISMATCH (lifeboat.rs:270). That case is also
  refused by :274 (the signer is not the account's owner) with the
  same code, so a new case has the thief sign for its own account,
  which only :270 refuses.
- vault.rs queued_releases_...: a payer reusing its nonce ->
  ALREADY_INITIALIZED (lib.rs:229)
- vault.rs a_pre_seal_thief_...: the thief's seal after the recovery
  key's admin change -> SEAL_NOT_EXECUTABLE (genesis.rs:483)
- differential.rs, settle: frozen -> FROZEN, else short ->
  INSUFFICIENT_BALANCE (custody.rs:516, :522, in that order)
The differential also counts the two settle refusals and prints them
(counted, not required, as for a release refused for want of tokens).
Its docs/09 row says so, with the counts: the default run refuses 6
settles while frozen and none for want of tokens; the 24 x 500 run
(re-run 17 Oct, every earlier figure in the row identical) refuses 68
and 8. No test added; test counts unchanged.

Evidence (tasks/refusal-codes/run-both-3.log, run on this commit's
tests and on its parent's, 801e199 (tree 8cc19ea, before the sync
onto main), whole program suite per mutant):
- F1 loader.rs:17 `.ok_or(WrongBuffer)?` -> `.unwrap_or(None)` (a
  non-buffer reads as authority None and fails as BUFFER_NOT_FROZEN):
  SURVIVED old, KILLED new by orphaned_buffers_... alone.
- V6 lib.rs:229 create_pda's check -> `if false` (the system program
  refuses instead): KILLED old by a_protocol_registers_once only; new
  adds queued_releases_built_from_the_same_state_do_not_collide.
- V3 custody.rs:522 -> `if false` (the token program's Custom(1)):
  narrow differential at 20 seeds SURVIVED old, KILLED new ("settle
  refused with InstructionError(0, Custom(1))"); at the default 4
  seeds neither reaches a short settle. The wide differential and
  a_ticket_settled_while_the_vault_is_short_... kill it in both
  states, so in CI's default run the narrow differential pins only
  the frozen code and those two pin the short one.
- V4 (lifeboat.rs:270 owner half): KILLED old by
  a_lifeboat_exit_goes_only_to_a_destination_committed_72h_before_it
  only; new adds the_lifeboat_moves_... (the thief-signs case).
- V1 (custody.rs:503 recipient half), V2 (:516), V5 (lifeboat.rs:274),
  V7 (governance.rs:318 seal veto): KILLED in both states by the same
  tests; deleting each lets the call through or another test fails.
  For these the code only rules out a masked refusal.
- V8 (custody.rs:503 payer half, the rent receiver), checked after a
  review asked: KILLED by every_account_substitution_is_refused (the
  sweep swaps account 6), not by these tests (mutants.log STATE=p1).
Program sources unchanged; sBPF Floor/Harbor/Vault 8ca8a608 /
543be60b / 7086148b after the runs.

Sibling search: `grep -n "is_err()" programs/tests/tests/*.rs` at
this commit leaves only floor.rs's four `env.call(&f.program).is_err()`,
calls of the protected program itself (its abort ELF), not a refusal
by Floor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Finding 109 left elf_extent and snapshot_restorable (finding 88's
length rule, which Floor runs on the live programdata) with no fuzz
target: only unit tests and the core's mutation testing checked them.

The loader_parsers target now restates the extent's rules in u128, so
no sum can overflow and no read leaves the input: the header, both
tables, each segment's file bytes and each non-NOBITS section; None
past the end, on a bad header, or over the count and entry-size
limits. The rules are the same as the core's, so this checks the
arithmetic and bounds, not the rules. On both inputs it checks
elf_extent against it and that a copy cut at the extent has the same
extent, and it checks snapshot_restorable as snapshot_matches plus the
length. Three seeds carry an ELF with both tables (a 600-byte region,
extent 576): a buffer of exactly 576 bytes, one of 575 (finding 88's
cut), and the region alone (tasks/fuzz-elf-extent/seeds.py).

Evidence (tasks/fuzz-elf-extent/ in the autopilot directory):
- SECONDS_PER=120 scripts/fuzz.sh loader_parsers: "Done 53153777 runs
  in 121 second(s)", no crash.
- 14 hand mutants of floor.rs (mutants.py), each from the seeds in a
  fresh corpus for at most 60 s, three rounds (seed-timing.log).
  KILLED in every round, each within 5.4 s: phentsize < 50,
  shentsize < 60, NOBITS != 7, end >= len, no segment loop, no
  section loop, p_memsz for p_filesz, sh_addr for sh_offset, either
  table's end dropped, restorable with > and with no length check
  (again on the final target, mutants-final.log: each within 4.7 s).
  From the four header seeds alone (OLD_SEEDS=1), no length check
  SURVIVED all three rounds and > took up to 48.9 s, so the seeds stay.
  SURVIVED both ways (mutants-r2.log): phnum > 65 and shnum > 1025.
  Only a count of exactly 65 or 1,025 tells those apart; a crafted
  input with phnum 65 fails the target (Some(3704) vs None), a
  4,000-byte seed at 64 segments did not get the fuzzer there in 60 s
  (not kept), and the unit test elf_extent_rules_at_their_edges pins
  both limits.
- cd fuzz && cargo clippy --all-targets -- -D warnings: clean.
The core and the programs are unchanged.

docs/09 §2 (scope and results) and row 109 keep their 17 Oct
statements, saying what came after; is_canonical_abort is still
unfuzzed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sweep swaps one account at a time. Another protocol's vault alone
at a vault slot is refused by the vault-token check
(check_vault_accounts) or by a ticket's vault check, with WrongVault,
the same code as load_vault's protocol check (lib.rs:319). So that
check was beyond the sweep's reach at most of its calls: with it
skipped at one call at a time (the account loaded with `load` alone),
13 of its 18 calls passed the whole program suite, the deposit
(custody.rs:89) and the release (:283) among them. With the twin sets
below swapped in and the check skipped at one call, a deposit
(custody.rs:89), a withdrawal, both admin payouts and a quote from a
funded vault of another protocol (:283), a lifeboat move
(lifeboat.rs:54) and a seal listing the other protocol's vault
(genesis.rs:489) are accepted outright.

The sweep now takes twin sets: accounts that other checks tie to each
other, each paired with its twin at a vault of the same mint in a second
protocol. Every run that names a set's first account (the vault) gets
the whole set swapped, in the accounts and in the instruction data (a
change names its target vault), except at the run's free slots, and must
be refused with the given code. Each distinct tampered transaction is
sent once, and a set no run swaps in fails the sweep, as a stale
backstop does. The Vault sweep registers the USDC vault with its token
and lifeboat accounts, twinned with a vault the second market creates
for the same mint (and in the second world, whose seal lists both, the
SOL vault too); and, for the runs that use a queued ticket, the USDC
vault with that ticket, twinned with a ticket the second market queues
on its USDC vault. The required code is WrongVault, the code of the
check the sets are for. Any Vault code would not do: with the sweep
taking any code from 8000 to 8999 (s2-run.out), the check skipped at
three calls passes the whole suite. A backlog reset (brakes.rs:241) is
then refused with TicketsPending (the twin vault has a queued ticket),
and a lifeboat exit's proposal and execution (lifeboat.rs:194, :239)
with NothingToExit.

Evidence (tasks/harbor-masks/, mutants.log; programs unchanged, Vault
sha256 7086148b…, Floor 8ca8a608…, Harbor 543be60b… before and after
every run):
- Single swaps unchanged: 5,365 substitutions over 37 transactions, 673
  refused at a backstop slot, on the old and the new test
  (base-sweep.log). 26 transactions sent with a twin set.
- load_vault skipped at each call (state base, then r3; the 8 below
  again at this commit, test-head-*.log): the 13 that passed now: 8
  KILLED by every_account_substitution_is_refused (custody 89, 283;
  brakes 241; lifeboat 54, 140, 194, 239; genesis 489). The other 5
  SURVIVE, each because a second check refuses the same swap with the
  same code; each pair skipped together is KILLED: settle (custody 490 +
  the ticket protocol half of :500), cancel (brakes 36 + :38's), propose
  (governance 257 + 131). Execution's load (governance 171) is checked
  again against the target the proposal stored (:168).
- The twin match loosened to any Vault code (sweep.rs:543) passes the
  suite alone; with load_vault skipped at each of the 8 calls as well,
  5 fail and 3 pass, brakes 241 and lifeboat 194, 239 (s2-run.out).
- Each half of the ticket checks on its own: the vault halves KILLED
  (brakes :38 by the sweep, custody :500 by
  a_ticket_settles_after_its_eta_to_its_stored_recipient_only); the
  protocol halves SURVIVE (covered only jointly, above).
- The new asserts fail when the t1 ticket set is left out
  (assert-plant.log: "cancel ticket: the USDC vault and a ticket on it
  not swapped") and when a set no run names is registered
  (stale-plant.log).

Sibling search: the release's own checks (custody.rs:290, :291, both
halves of :295, :306) each already failed a named test, so Harbor's
own checks hide none of them. Not pursued: load_vault's and
load_protocol's address re-derivations (lib.rs:323, :306) and an
asset record's protocol and mint in a release's price table
(custody.rs:224-225) fail no test either, but each repeats a PDA
address or discriminator check only forged state gets past. Floor
compares each account with its config on its own: each half of
loader.rs:140 deleted fails three tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
load_vault's check that a vault belongs to the given protocol
(lib.rs:319), skipped at one of its 18 calls at a time, passed the
whole program suite at 13 of them. The sweep swapped one account at a
time, and another protocol's vault alone meets the vault-token or
ticket check first, with the same WrongVault. The previous commit
swaps each vault's accounts whole for another protocol's: 8 of the 13
now fail it on their own, the other 5 only jointly with a second check
that refuses the same swap. With the check skipped, that whole set is
accepted by a deposit, a withdrawal, both admin payouts, a quote, a
lifeboat move and a seal (test-head-*.log); three calls need the exact
WrongVault the sweep requires (s2-run.out).

docs/09 §4 row 113, and the findings count and list in README and
docs/12 (113, "96–113", Vault tests).

Evidence: tasks/harbor-masks/mutants.log (state base and final),
test-head-*.log and s2-run.out, as quoted in the previous commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
followConfigs (cloud/guardian/follow.mjs) ingested a watched Floor
config as a replay (act=false) until the config was in `followed`, which
happens only once a poll's walk and ingest succeed. So the line between
replay and live was the first good poll, not the start of the process.
Two cases read events after the start silently:
- A restart whose replay failed (a refused walk, a transaction
  requireTransaction cannot get) stayed a replay on every later poll,
  and all that landed in the meantime was replayed once a poll got
  through.
- A restart whose first poll succeeded still replayed what landed
  between the start and that poll. The first followFloor runs after the
  Vault startup: its replay (up to 50,000 signatures, no budget), the
  chain sync and the startup judgement.
Replay sends nothing and logs no event (consider in main.mjs). A
replaced or removed recovery key, a restore and its approvals are
reported only through these events, so they were never said. A stop
and an ejection showed only in the minute's config check (readiness
"stopped", the guardian set), without who did it. The account scans
still veto pending proposals, so the loss is the reports, not vetoes.
The test added in 221b13f pins that a failed replay stays a replay; it
did not ask what lands after the start.

Fix: the line is the slot the process started at, on every poll.
- main() reads it once, after the cluster check, with startSlot
  (follow.mjs): the highest confirmed slot that any of a FailoverRpc's
  endpoints gives, so one endpoint behind cannot put the line early. If
  none answers (an endpoint asked directly leaves at once when busy:
  `hurry`, 08b1822), or for a plain Rpc, the RPC itself is asked, and
  FailoverRpc.call waits for a busy one once. With nothing answering it
  is fatal (must), like the cluster check. followFloor passes the slot
  to followConfigs as `liveAfterSlot`.
- The node lists signatures newest-first by slot, and the walk already
  relies on that (afterCursor, `until`). So one cut splits a poll: the
  first row in a slot at or before the start slot, and all listed after
  it, are replayed, then the newer rows are ingested live. A row with no
  slot (null over JSON-RPC) never ends the newer part, so one among the
  newest is acted on (twice at worst, never unsaid). That holds for a
  followed config too, so a node whose listing had not reached the
  start slot at the first poll cannot make older rows live at the next;
  a followed config that lists such rows logs a warning once they are
  read (a lagging node, or a start slot from an endpoint ahead of its
  chain), with the start slot and their slots. The warning alerts
  (alert.mjs URGENT_WARNINGS), as Floor's stop and ejection do since
  finding 76: with a late line, the events in those rows are never
  reported. Its text gives both causes, the count, the lowest and
  highest slot and the start slot, since the webhook shows only the
  text and the program.
Slots, not block times: a block time against the host clock moves with
clock skew, and block times need not grow. Without a start slot (only
when no Floor config is watched) the old rule stands: resuming replays,
followed acts. `ingest` is now called only with something to ingest:
the empty calls did nothing in main.mjs, and two existing assertions
that pinned an empty replay call now pin no call.

Evidence (tasks/floor-replay-boundary/ in the autopilot directory):
- New test "a Floor config resuming acts on what lands after the slot
  the process started at" (restart.test.mjs), start slot 4 unless said:
  a failed replay, then s4-s7 land (s4 in the start slot, s6 with an
  earlier block time); the next poll replays s4, s3 and acts on s7, s6,
  s5 in that order, and the config is followed. A clean first poll
  replays s3 and acts on s5, and logs nothing. All after the start (2):
  all live, no empty replay call. All at or before it (3): all replayed,
  no live call. A followed config listing s5, s4, s3: s4 and s3
  replayed, s5 live, one warning naming 2 rows, start slot 4 and slots
  3-4; the same with the replay failing: no warning. A null-slot row on
  top: live. One between s7 and s5: live with them. One below s4:
  replayed with s4 and s3 in one pass, the cursor at s4.
- New test "the start slot is the highest confirmed slot any endpoint
  gives, and none is fatal": highest of four endpoints (one behind, one
  refusing), each asked for its confirmed slot, and the RPC's own call
  not made; a plain Rpc asked itself; every endpoint busy but the RPC's
  own call answering; nothing answering throws.
- red-r9.log, the r9 test file: (a) on the parent's follow.mjs with
  only startSlot appended: the boundary test acts on nothing, and the
  empty-call pin fails (11 pass, 2 fail). (b) on the version before the
  sixth code review (branch auto/B/floor-replay-boundary-r7, kept): the
  warning has no start slot or slots (12 pass, 1 fail). red-r8.log: the
  previous test file on the version before the fifth (-r6): the null
  row below s4 makes it act on sn and stop with s4 unread.
  clean-restart-probe-r5.log: the clean-first-poll case alone on the
  parent's follow.mjs: [["P1",false,["s5","s3"]]], s5 never acted on.
- mutants-r9.py / mutants-r9.log (full client suite each, on the r9
  tests): M1-M16, M19-M21 and S1-S9 KILLED: the first-good-poll line,
  the start slot live, no slot as all live, no slot as all replay on a
  followed config, the line only while resuming, a null slot ending the
  newer part, all newer as none, an empty replay call, an empty live
  call, live as replay, replay as live, live before replay, the replay
  part from the front, no warning, a warning on a resuming config, the
  warning's row count, the warning before the replay is read, without
  the start slot, its slots newest first; startSlot lowest, no fallback
  to the RPC, the RPC asked always, a refusal not caught, none not
  fatal, processed commitment, first answer, endpoint refusals kept, the
  RPC's refusal kept. M17 and M18 (main.mjs does not pass or does not
  read the slot) SURVIVE: no test drives main() or followFloor, as for
  the rest of their wiring (221b13f); the spawn test in restart.test.mjs
  stops at the key file, before main(), and reaching followFloor would
  take a fake of the whole startup RPC surface. Baseline green; both
  files restored (cmp). Earlier rounds are kept beside it;
  mutants-r4-INVALID.log and mutants-r7-OVERLAP-INVALID.log are void and
  say why.
- Review by A (CHANGES): the warning was level warn and matched no
  URGENT_WARNINGS pattern, so alertFor returned null and it went only
  to stdout. Now a pattern; the lagging-node case in restart.test.mjs
  passes the logged line itself to alertFor and expects a warning
  whose detail ends with the count, slots, start slot and program.
  A followed config whose replay part is out of slot order with a
  slot-less row last pins the slots as the lowest and highest of the
  rows that have one (it logged [null, 4]). Its lowest row sits in the
  middle, so taking the first and last rows instead fails too.
  mutants-r12.py, mut-r12-N1..N6.log, the whole client suite each on
  the final tests: the lowest slot as the last row's (N1), no numbers
  in the text (N2), the pattern removed (N3), the lowest as the
  highest (N4), the highest as the first row's (N5), the old wording
  (N6): all KILLED, 203 pass, 1 fail; files restored (cmp);
  green-r12.log 204/204. red-r9 and mutants-r9 (above) ran on the
  tests before this round.
- Second review by A (CHANGES, low): its mutant R8 (the slots as the
  first and last rows', sorted) survived, since that case had its
  extremes at the ends. The case now has slot 1 between slots 3 and 4.
  r14/mutants.log, A's mut.py on the whole client suite: R1, R2, R4,
  R8, R9 KILLED; follow.mjs restored (cmp); r14/green.log 204/204.
- Client suite 204/204 (202 on the parent, two new tests); docs/09:40
  says 204 in both columns.
- scripts/check.sh at this commit on 1646a1a: ALL CHECKS PASSED,
  client 204/204, claims OK (r14/check.log); check-each-commit.sh
  1646a1a: green (r14/each-commit.log).

Limits: the main.mjs wiring is not pinned (M17, M18). The split
assumes the node's slot order, as the walk does; a listing out of slot
order is split at its first row at or before the line. With a single
endpoint that lags, or every endpoint lagging, the start slot is early,
and what landed shortly before the start (while no guardian ran) is
acted on as new: alerts for real events, and a veto for each proposal
among them that is not approved (the account scans send those anyway
while it is pending; for one already executed the veto fails, and how
its retry ends was not checked here). An endpoint whose slot is ahead
of its chain, or a backup on another cluster (checkCluster asks only
the active endpoint), puts the line late, and it holds for the life of
the process: until the watched chain reaches it, what a followed
config lists is replayed. That is worse than the old rule, which ended
at the first good poll. The followed config's warning shows it, and
it alerts, at most once per 10 minutes per program (alertFor's
window) (backlog c8: check every endpoint's cluster). A lagging node
alerts the same way, though its rows landed before the start and a
resuming replay would also replay them silently: the two cannot be
told apart. Under a late line the first poll, which is a resuming
replay, replays what landed after the start and below the line
without a warning: only rows a followed config lists alert. A replay
part that checkpoints some rows and then fails logs nothing, and the
next poll's warning counts only what is left. A replay part still has
no budget (main.mjs: `live ? ingestBudgetMs : Infinity`), now on a
followed config too when its listing reaches back before the start
(backlog c3p; a slot line makes budgeting the replay safe, since the
rest is still a replay next poll). The replay part uses
requireTransaction: one old transaction the RPC will not return blocks
the poll (the stall alert fires; c7). A config with no cursor (a first
start, a lost state file) still anchors at its newest signature at the
first poll, so what lands between the start and that poll is not read
at all (finding 71's "the account scans cover what came before"; c6).
A row with no slot is checkpointed at slot 0, as before, which drops
the next walk's minContextSlot guard; afterCursor drops such rows
whenever the cursor has a slot. startSlot waits for its slowest
endpoint once at startup (an endpoint's timeout is 10 s) and repeats
crossCheck's ask-every-endpoint pattern rather than share a helper.
Rows in the seconds around the slot read can fall either side: those
landing before it are replayed, and some processed before the
confirmed slot read but confirmed after it are acted on. The line is
this process's start: a row the previous process listed live but did
not finish (a transaction that kept failing, a crash) is replayed after
a restart, as the old code did (backlog c9).

Sibling search: the only other replay/live line is the Vault startup
ingest (main.mjs, act=false). Its line is its own startup listing (the
live loop reads on from `until`, the newest signature listed), and a
failure there ends the process (main().catch(fatal)), so it never runs
on as a replay. refreshPendingTickets and syncFromChain take `act` from
their caller, and spendExecution's `live` and assess's `act` consume the
flag; none ties it to a poll succeeding. The no-cursor anchor above is
the one sibling found; it is left for its own change (c6).

Code review (high), eight passes.
- Pass 1, seven points. Taken: the slot from every endpoint; fatal when
  none answers (it was a warning and a silent fallback to the old
  rule); no empty replay call; the followFloor comment. Stated: the
  wiring, the no-cursor anchor. Not taken: "must() retries for minutes
  before the slot is read": must (main.mjs:118) ends the process on the
  first failure.
- Pass 2, nine points. Taken: the line on every poll (a lagging listing
  made a followed config act on older rows); the RPC's own call in
  startSlot (all endpoints busy was fatal); slot-less rows; the doc
  header. Stated: a blocked replay, a cross-cluster backup, the shared
  helper, the startup wait.
- Pass 3, five points. Taken: the RPC's own call only as a fallback (it
  asked the active endpoint twice); the cross-cluster Limit corrected
  (it said "the old behaviour"). Not taken: making `liveAfterSlot`
  required; without it the old rule is kept for the existing tests, and
  main.mjs always sets it when a Floor config is watched. Its other
  point (replay rows first could checkpoint past a live row in an
  out-of-order listing) led to runs in listing order, undone in pass 5.
- Pass 4, seven points. Taken: a null slot is live (`null <= n` is true
  in JS, so it was replayed); the followed config's warning (a bad line
  was silent); `.catch`. Stated: the followed config's unbudgeted
  replay, the slot-0 checkpoint. Its per-run budget and cut-run points
  were about the runs, gone in pass 5.
- Pass 5, four points, all taken: the runs (with a stop after a live
  run) dropped an older replay row for good in an out-of-order listing,
  left rows that the next poll took for a lagging node, and pinned a
  cut-replay path main.mjs never produces. Since the walk assumes slot
  order anyway, one cut, replay part then live part, replaces them.
- Pass 6, five points. Taken: the warning only once the replay is read
  (a failing one repeated it each poll), with the start slot and slots;
  the Limits on rows around the slot read. Stated: rows a previous
  process left unfinished (c9; not a regression). Not taken: pinning
  the main.mjs wiring by spawning main.mjs (see the mutants above).
- Pass 7, after A's review, eight points. Taken: the alert carries
  the count and slots; the slots skip slot-less rows; the alert.mjs
  comment had the line backwards; the evidence cited the r9 runs.
  Stated: a lagging node alerts; a failed partial replay is
  undercounted. Not taken: `notify: true` instead of a pattern
  (URGENT_WARNINGS is how a warn line alerts, and the test feeds it the
  logged line, so a reworded message fails it: N2/N3); a rate-limit
  test (alertFor's window and per-program key are alert.test.mjs's).
- Pass 8, nine points. Taken: the text said the rows were from before
  the start, which is false with a late line (now "at or before the
  start slot", with both causes); the highest slot (it was the first
  row's); the slot test matches newerThan's (`!= null`); the alert.mjs
  comment now says a first poll's replay is silent; three commit
  message slips. Stated: a first resuming poll replays rows under a
  late line silently (below); the startSlot/crossCheck copy. Not
  taken: level error. The hidden events alert as warnings themselves
  (a key replaced or removed), so a warning matches them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The guardian's startup check (finding 83) asked rpc.call for the genesis
hash, which the active endpoint answers. A backup in `rpcs` on another
cluster was never asked, and mainnet requires backups. A failover moved
reads and sends onto such a backup blind. The freshness check (finding
76) moves by slot alone: with stub endpoints on 801e199, a mainnet
primary and a devnet backup reporting a higher slot passed the startup
check, and the first crossCheck switched the guardian to the devnet
endpoint (tasks/cluster-each-endpoint/demo.log; the slots are made up).

The gate is in FailoverRpc, where every move of `active` happens. With
`genesis`, an endpoint is used only once its genesis hash has matched
(admit): asked before its first use in call(), and before each
crossCheck until it answers, one ask shared by overlapping calls. One on
another cluster is never asked again or read from; an active endpoint
found there is left for the endpoint that answered. A missing hash or
any failed ask (a refusal included) is no answer: it keeps a busy
endpoint's wait, and call()'s one rest now also runs when the last
endpoint was skipped. Without `genesis` nothing changes.

The guardian passes `genesis` for a named cluster. At start it asks
every endpoint, and once one has matched gives the rest CLUSTER_GRACE_MS
(2 s), so a silent backup does not hold up a restart by its 10 s
timeout. One on another cluster refuses the start, named by host and
config entry with its hash. A silent one is reported once and stays
unused until it answers on the right cluster (so every endpoint must
serve getGenesisHash); a silent primary is left for a checked backup at
once. With none answering it asks once more through call() (a busy
endpoint's wait is waited out) and stops on that error, as before. After
the start, a backup that comes up or turns out wrong is logged.

Tests: 8 in delivery.test.mjs on FailoverRpc, 6 in restart.test.mjs that
start main.mjs against local stub endpoints. Old main.mjs (801e199, the
same file at 2fa7a7f): 5 of the 6 red; the none-answering test passes on
both (it pins that the start is still refused). Mutants, one at a time,
restored by cmp (tasks/cluster-each-endpoint/r3/): FailoverRpc 16/17
killed, the survivor equivalent (admit's early return is reached only in
state ok or wrong); guardian gating and labels 12/12 killed (one more
mutant was void, recorded; mutants-final.log has the last 5); 3 mutants
of the after-start log lines survive (no test). scripts/check.sh: ALL
CHECKS PASSED on b6bf1d7 with the row, client 218, findings 114
(r8/check.log).
Review by A (CHANGES, docs only): the row had no "Fixed (...)" status,
and it named finding 117, which is not recorded at this commit. Both
fixed: "Fixed (off-chain).", and the stop's gap is "not fixed here".

Limits: an endpoint that serves everything but getGenesisHash is never
used (fail closed). A backup that answers after the grace and is on
another cluster is logged, not refused. Copies of guardian transactions
still go to every backup (extraRpcs), one on another cluster included.
The finding is recorded as 114 in docs/09 §4, and the counts in README
and docs/12 follow it.

Siblings: this was the only getGenesisHash call in cloud/. One is left
for its own change: cloud/guardian/stop.mjs, the emergency stop, reads
the same config into a FailoverRpc and checks no cluster at all, not
even the active endpoint's. It can now pass `genesis`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
stop.mjs, the tool a guardian runs to stop a program under Floor, took
`cluster` from the guardian's config but never checked it: it read
Floor's config from whichever endpoint answered. With a primary on
another cluster (a testnet or devnet copy of the program, already
stopped there), it read that copy's config, saw "stopped", checked that
a call failed with FLOR there, and exited 0 while the real program ran
on. Finding 114 closed the guardian's side; its row named this gap.

Root cause: the stop built its FailoverRpc without `genesis`, and with
one endpoint used a plain Rpc that nothing checked.

Fix, with `cluster` set (not localnet):
- With backups, the FailoverRpc gets `genesis`, so each endpoint is
  read only once its genesis hash has matched (finding 114's gate).
  Every endpoint is asked at start without waiting for the answers, so
  one on another cluster is reported even while the primary answers
  everything, and the stop is not held up. One on another cluster is
  logged by its config entry (a backup still gets copies of the stop);
  with every endpoint there, the stop exits 2 at once. The switch line
  says the endpoint was "left", not "failing": a node on another
  cluster may be healthy (the cause field says which).
- With one endpoint (a plain Rpc, which keeps its patient retry), its
  hash is asked before anything else. Another cluster: exit 2. An
  answer with no hash, or no such method (-32601): exit 2 at once,
  since a retry gives the same and waiting out --max-seconds (900 s by
  default) only delays the operator. Any other error (unhealthy,
  internal error, no answer): asked again until --max-seconds, then
  exit 1. The time limit is checked between asks, and one ask on a
  plain Rpc may take its retries (as each read already did).
- An unknown `cluster` exits 2, as the guardian's checkConfig refuses
  it (a misspelt one used to check nothing); it is looked up with
  Object.hasOwn, so "constructor" is unknown too.
No `cluster`: a warning, as the guardian gives, and no check.
"localnet": no check, as before. The deploy rehearsal's stop config
has no cluster; its parser reads JSON lines, which the warning is.

Evidence (tasks/stop-cluster/, after five code-review passes):
- cloud/test/stop.test.mjs, 10 tests spawning stop.mjs against local
  stub endpoints (10 green in each of 10 runs): on the old stop.mjs 9
  fail (r6/red-old.log; the wrong-cluster cases exit 0 "stopped: calls
  now fail with FLOR"); the right-cluster case passes on both
  (r6/green.log).
- ../mutants-r7.py: 24 mutants of the new rules, 24 KILLED, none
  failing every test (r7/mutants.log); stop.mjs restored (cmp).
- Review by A (CHANGES, low): (1) with onCluster reporting a matching
  endpoint too (X1), a good backup was logged "serves another cluster
  ... remove it from the config" during a stop, and no test failed.
  The own-cluster test now asserts no such line: X1 survives the
  previous tests (r7/mutants-old-tests.log) and is killed now. (2) M23
  left a dangling `if`, a syntax error that failed all 10 tests, so its
  kill was void; it now leaves `;`, and the runner calls a mutant that
  fails every test void.
- scripts/check.sh on 0cb3202: ALL CHECKS PASSED (r7/check.log).
- tools/claims.py: client 228; docs/09's count 218 -> 228.

Not changed:
- Transaction copies (`backups`) still go to every backup, one on
  another cluster included; a copy carries this cluster's blockhash,
  so it cannot land there (not tested; the guardian has the same).
- With backups that all refuse getGenesisHash, the stop retries until
  --max-seconds (FailoverRpc does not keep why an endpoint gave no
  hash); a wrong primary with a backup that is down does too (tested).
- A list such as `"cluster": ["devnet"]` is taken as "devnet"
  (Object.hasOwn stringifies it) and that hash is checked.
- Finding 117's row is added to docs/09 after 115 and 116.

Sibling search: `git grep -n "new FailoverRpc\|new Rpc(" cloud`:
main.mjs (finding 114), stop.mjs (this); the deploy tools and
status/snapshot.mjs take the URL as an argument and read no config
with a `cluster`. policy.mjs:375 checks `cluster in GENESIS` as this
did (backlog; the guardian then refuses with a wrong message).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
checkConfig took config.cluster when `cfg.cluster in GENESIS`. `in`
turns a list into its text, so "cluster": ["mainnet-beta"], valid JSON,
passed. Every later test compares with === "mainnet-beta", so the backup
endpoint and the alert webhook were not required and a guardian set
without the margin was logged but not alerted. The genesis check looked
up GENESIS[["mainnet-beta"]], which is mainnet's hash, so the guardian
started on mainnet with none of finding 83's mainnet rules. `in` also
took names every object inherits ("constructor", "__proto__",
"toString"): those started and then refused with a wrong "serves
another cluster".

Fix: cluster must be one of a Set of GENESIS's own names and
"localnet". Set.has neither converts a list to text nor sees inherited
names, and the error message is built from the same Set.

Evidence (tasks/cluster-own-names/):
- repro-old.log: on the old code ["mainnet-beta"] is accepted with only
  the "no config.alert" warning and resolves to mainnet's genesis hash;
  the three inherited names are accepted too.
- red-old.log: the new prod.test.mjs case fails on the old code
  ("Missing expected exception: [\"mainnet-beta\"]"); green after.
- mutants.log C1-C4: 4 mutants killed. The old `in` test back; an
  Object.hasOwn test without a string test (only the list fails it); an
  `in` test with a string test (only the inherited names fail it);
  "localnet" dropped from the Set (the existing localnet case).

Sibling search: grep for `in` tests on names in cloud/. checkPolicy
(policy.mjs:212) has the same class: an inherited policy name is
refused, but with "function is not iterable". Fixed in the next commit.
stop.mjs on auto/B/stop-cluster uses Object.hasOwn: a list there
resolves to the same name's hash and the endpoint is checked against it,
and stop.mjs has no mainnet rules to skip, so it is harmless; not
changed here.

Finding 118 (reserved); its docs/09 row follows 114-117.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
checkPolicy looked each policy name up with `name in POLICY_RULES`,
which also takes names every object inherits. A policy key such as
"constructor", "__proto__" or "toString" (each is an own key once the
config is parsed from JSON) passed that test, and checkField then
destructured the inherited function as a rule: the start was refused,
but with "function is not iterable" instead of naming the setting. It
failed closed; the message was wrong. Same class as the previous
commit's cluster check.

Fix: Object.hasOwn(POLICY_RULES, name).

Evidence (tasks/cluster-own-names/):
- red-old-policy.log: the new prod.test.mjs case fails on the old code
  (actual "TypeError: function is not iterable", expected /not a policy
  setting/); green after.
- mutants.log P1: `in` back for Object.hasOwn is killed.

Sibling search: the other `in` test in checkPolicy,
`"approvedChangeIds" in policy`, asks for a name no object inherits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
checkConfig never looked at the type of config.policy. main.mjs starts
with checkPolicy({ ...DEFAULT_POLICY, ...(cfg.policy ?? {}) }), and
spreading true, 5, [], null or "" adds no keys, so a config with such a
policy started on DEFAULT_POLICY alone: the operator's thresholds and
approvals were dropped without an error or a warning. (A non-empty list
or string spreads index keys, which checkPolicy already refuses.) Found
by the code review of the previous two commits; same class as the
cluster check: a value of the wrong type passed the config check.

Fix: config.policy, when set, must be an object that is not null or a
list. This refuses "policy": null, which started on the defaults
before: null is also what a template with an empty value writes, so it
does not say the operator wants no policy. No config in the tree sets
it (git grep).

Evidence (tasks/cluster-own-names/):
- repro-old-policy-type.log: on the old code each of true, 5, [], null,
  "" and 0 passes checkConfig and the checked policy equals
  DEFAULT_POLICY.
- red-old-policy-type.log: the new prod.test.mjs case fails on the old
  code ("Missing expected exception: true"); green after.
- mutants.log T1-T4: 4 mutants killed: the typeof test dropped, the
  null test dropped, the list test dropped, and every policy refused
  (the new `policy: {}` case).

Sibling search: the other object settings in checkConfig, alert and
liveness, read `?.url`, so a value of the wrong type has no URL and is
refused. liveness.everyS had no type test: the next commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
checkConfig refused config.liveness.everyS when !(everyS >= 10), on its
own line: no type test and no upper bound, unlike the settings in its
range loop. A string "600" or a list [600] passed, since >= turns each
into a number, and main.mjs multiplied it by 1000 the same way: so
these ran as 600 s, but by accident. 1e9, or 1e400 (Infinity once
parsed), passed as well: pingLiveness pings once and not again while
the guardian runs, so the dead-man's switch pages a false outage. No
case pinned the 10 s bound. Raised by the code review of this branch:
the same class as config.policy and config.cluster.

Fix: "liveness.everyS" is an entry of the range loop, from 10 to 86400
(86400 is the cap stallAlertS and lagAlertS have); the loop reads a
dotted name through the nested object. A config with a string, a list
or more than a day there now refuses to start until it is fixed; no
config in the tree sets one (git grep). The loop's message now prints
a number as itself, so 1e400 shows as Infinity rather than JSON's
"null". The URL check is a line of its own, and its message says
liveness must be an object with an http(s) url, everyS optional: the
old one ("{ url, everyS ≥ 10 }") named only the shape.

Evidence (tasks/cluster-own-names/):
- red-old-everys.log: the new prod.test.mjs case fails on the old code
  ("Missing expected exception: \"600\""); green after.
- mutants.log L1-L9, U1-U2: 11 mutants killed. The entry dropped; each
  bound moved by one either way (the 9, 10, 86400 and 86401 cases);
  the dotted read replaced by cfg[k]; its `?.` dropped (liveness: null
  throws a TypeError); the loop's typeof test dropped ("600"); the
  number shown as JSON (Infinity); the URL check made always true and
  its `?.` dropped (the no-url, no-scheme, true, null and bare-URL
  cases).

Not changed (backlog): docs/07:132 says seconds are whole numbers and
that an unknown setting stops the start. checkConfig takes fractions
for its own second and count settings (a fraction works as a number
there: `inflight.length < 2.5` caps at 3), and does not look for
unknown keys inside alert or liveness.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
checkConfig refused an unknown top-level setting but never looked inside
`alert` or `liveness`, which are read by name only. A typo there fell
back to a default without a word: "format": "slak" or "fromat": "slack"
posted the JSON body, which has no `text` (Slack) or `content` (Discord)
field, so a Slack or Discord webhook would not show it (that rejection
is the services' documented behaviour; not verified here, localhost
only); "every_s": 600 under liveness pinged every 60 s. docs/07 and the
main.mjs header already say an unknown setting stops the start.

Fix: `alert` takes only url and format, `liveness` only url and everyS,
each object its own names, checked before the URLs so a misspelt url
is named as such; `format` must be one of ALERT_FORMATS, now exported
from alert.mjs next to webhookBody, the code that reads it. main.mjs
hands alerter only url and format, so what is checked is what is used.

Deliberately refused now: alerter's other options (windowMs, timeoutMs,
retryDelaysMs) were taken from cfg.alert by the old spread. They are
documented nowhere (no doc, header or config in the tree names them),
so an operator who set one learns of it at start with the key's name.
If any of them should be a setting, it needs a range check here first.

Limit: main.mjs builds the alerter before checkConfig runs (finding 71),
so the fatal alert for a refused format is itself sent as the JSON body.
Before, that guardian ran with every alert unseen; now it stops, loudly
on its console and in systemd. Choosing a format from the URL host would
change valid configs too; left as a backlog item.

Evidence (tasks/config-nested-keys/):
- repro-old.log: on the parent (0d18757 when run; its policy.mjs and
  prod.test.mjs are af7cd0d's, alert.mjs gained only an unrelated
  warning pattern) the configs above returned [] (no warning);
  webhookBody(..., "slak") gave the JSON body.
- red-old.log: the new prod.test.mjs cases fail on the old policy.mjs
  and alert.mjs (exit 1). On this commit prod and alert tests 16/16.
- mutants-r2.log: 13 of 13 KILLED (each key list dropped, liveness
  taking alert's names, the key test off, the format test off, loosened
  to a lower-case string, null let through, an extra format, `in` for
  includes, the plain-object guard and its list part off, the key loop
  moved after the URL checks, a format listed with no body of its own).
- Review by A (APPROVE, one nit): `"format": ""` had no test, and
  A's mutant X3 (a blank format let through) survived; a blank format
  posts the JSON body. "" is now in the refused list: X3 fails 1 test
  (r3/x3.log), and the suite is 228/228 (r3/green.log).
- code-review high: its points 2-5 adopted (the alerter spread, a doc
  comment I had split, the format/body test, the order). Not adopted:
  sharing checkField with checkConfig, and moving ALERT_FORMATS out of
  alert.mjs (alert.mjs imports nothing, so no cycle): refactors only.

Sibling search: every other object in the config is a list (rpcs,
mints, floorProtected) checked element by element, or `policy`, whose
names checkPolicy checks against POLICY_RULES. git grep for "alert"
and "liveness" objects finds only url/format/everyS in use (docs/07,
alert.mjs and main.mjs headers, restart.test.mjs).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/07 says "seconds are whole numbers", and checkPolicy refuses a
fraction for every count and duration in the policy. checkConfig's range
loop took any number in range: "stallAlertS": 10.5, "ingestConcurrency":
2.5 or "startupMaxSignatures": 1000.5 started without a word. No harm is
shown: ingestConcurrency is a `<` bound (follow.mjs:249), so 2.5 works
as 3; the others were not traced one by one. This is consistency with
the documented rule, not a demonstrated defect.

Fix: every setting in the loop is a whole number except minBalanceSol,
which is a fraction of a SOL by design (a `whole` flag per setting);
the error says which ("a whole number from 10 to 86400"). docs/07 now
says counts, seconds and milliseconds, not only seconds.

Evidence (tasks/config-nested-keys/):
- red-old-whole.log: the new cases fail on the previous commit's
  policy.mjs (exit 1); prod.test.mjs passes 6/6 here.
- mutants-whole.log: 6 of 6 KILLED (integer test off, minBalanceSol made
  whole, every setting not whole by default, isFinite for isInteger, the
  message's kind fixed, the flag inverted). Each of the seven settings
  is tried at its lower bound + 0.5, inside its range, so only the
  integer test refuses it.

Sibling search: checkPolicy's "int" and "price" kinds already use
Number.isInteger (policy.mjs:200, :204). `git grep "is not a number
from"` finds only this commit's minBalanceSol test; no doc quotes it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant