Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
0651ce0
Run the tests on 2-of-4 guardian sets; test recovery from two faults
shuv-amp Oct 17, 2026
4ee0a4e
Rehearse deploys with four guardians, and read only the keys there are
shuv-amp Oct 16, 2026
3d46770
Refuse a guardian set without two keys beyond its threshold (107)
shuv-amp Oct 15, 2026
ad3eed4
State the guardian margin in the docs, with its costs (AUD-2)
shuv-amp Oct 15, 2026
2b6ba81
Say the weekly CI run skips mutants, as its condition does
shuv-amp Oct 15, 2026
ee4e02c
Check that the mutant exclusions still match; repin murmur3 (finding …
shuv-amp Oct 15, 2026
adbd3b3
Test that a ticket stays in its own protocol and vault
shuv-amp Oct 16, 2026
8687dba
Test that the seal names every vault and happens once
shuv-amp Oct 16, 2026
dc993dd
Test that invalid limits and asset settings are refused at every entry
shuv-amp Oct 16, 2026
533562a
Test that an import into a ninth mint is refused
shuv-amp Oct 16, 2026
7931020
Pin six Vault refusals no test reached
shuv-amp Oct 16, 2026
e4bd08d
Pin four Vault checks another layer backstops
shuv-amp Oct 16, 2026
e6bb61e
Test the bounds on ticket extensions and a closed lane's code
shuv-amp Oct 16, 2026
ae52fe4
Pin each half of create_pda's existing-account guard on its own
shuv-amp Oct 16, 2026
9f552fa
Record the Vault refusal deletions and what each became
shuv-amp Oct 16, 2026
cc30bfa
Run the exclusion check's main in its self-test
shuv-amp Oct 16, 2026
b0fcc42
Record finding 106: Vault liveness and exit rules had no failing test
shuv-amp Oct 16, 2026
0269fd1
Cite PR #100's tree, not the unpublished bf8ba41, in finding 106
shuv-amp Oct 17, 2026
6915801
Warn about a Floor guardian set short of the margin (finding 107)
shuv-amp Oct 16, 2026
e7b2aec
Judge each guardian set's margin on its own, and say when it is fixed
shuv-amp Oct 16, 2026
032cb1a
Tie the JS guardian margin and set size limit to the core constants
shuv-amp Oct 16, 2026
ef765dc
Reach the 7-day extension total in the wide differential
shuv-amp Oct 16, 2026
0e02121
Check settle and lifeboat refusals by code in the wide differential
shuv-amp Oct 16, 2026
c6eba2b
Check limiter refusals by code in the wide differential
shuv-amp Oct 16, 2026
3048d93
Check release refusals by code in the fixed differential
shuv-amp Oct 17, 2026
618d728
Pin the deposit and settle duplicate-account checks
shuv-amp Oct 17, 2026
adbc6dc
Let the findings range say recorded after the submission
shuv-amp Oct 16, 2026
f26717f
Record finding 107: a lost key and a rogue locked a 2-of-3 set
shuv-amp Oct 16, 2026
2825222
Say devnet runs the build of PR #85, not this tree's
shuv-amp Oct 16, 2026
449a19a
List every on-chain change since the engagement in 16 §6
shuv-amp Oct 16, 2026
9a03ced
Key an alert's repeats on its heartbeat target too
shuv-amp Oct 16, 2026
297e688
Log what each sent or failed action acts on, so vetoes don't mute
shuv-amp Oct 16, 2026
e389a6d
Name in each alert the program, mint or change it is about
shuv-amp Oct 16, 2026
72de79c
Pin that an alert keeps and names an id of 0
shuv-amp Oct 17, 2026
1873d91
Run the claims check's main in its self-test
shuv-amp Oct 16, 2026
dd8ccf7
Say in a failed action's alert why the action was sent
shuv-amp Oct 17, 2026
c15a3b8
Say in Floor's not-ready alert which pieces are missing
shuv-amp Oct 17, 2026
2f5a15e
Merge main into #111's branch
shuv-amp Oct 18, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 3 additions & 7 deletions cloud/guardian/main.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ import { basename, dirname } from "node:path";
import { Rpc, FailoverRpc, b58encode, decodeTicket, key, decodeProtocol, decodeFloor, decodeVault } from "../lib/solana.mjs";
import { Keypair, sendWithFallback, sendEscalating, priorityPrice } from "../lib/signer.mjs";
import { VaultClient, FloorClient } from "../lib/bulkhead.mjs";
import { assess, watchedEvents, floorAlerts, guardianSetCheck, newState, observeProtocolFreeze, pendingCancels, rememberTicket, retirePendingFreeze, decisionTime, judgeReplayedWindow, queueFailedVeto, openIncident, observeFreezeEnd, incidentHold, guardianStanding, afterHeartbeat, heartbeatEveryAfterAccount, HEARTBEAT_ALERT_AFTER, pendingChangeVeto, pendingExitVeto, pendingUpgradeVeto, dueApprovedChange, dueApprovedFloor, vetoRetryDisposition, readChangeVeto, readPendingChange, checkPolicy, checkConfig, GENESIS, checkProposedChange, spendExecution, observeExecutions, unpinnedChangeApprovals, parseApproval, changeApproval, floorApproval, afterCancelFailure, CANCEL_RETRY_MS, DEFAULT_POLICY, DAY_S, outflowAlerts, OUTFLOW_KEEP_S, ticketAlreadyGone, freezeAlreadyCovers, replayedBreachTime, ticketScanDue } from "./policy.mjs";
import { assess, watchedEvents, floorCheck, guardianSetCheck, newState, observeProtocolFreeze, pendingCancels, rememberTicket, retirePendingFreeze, decisionTime, judgeReplayedWindow, queueFailedVeto, openIncident, observeFreezeEnd, incidentHold, guardianStanding, afterHeartbeat, heartbeatEveryAfterAccount, HEARTBEAT_ALERT_AFTER, pendingChangeVeto, pendingExitVeto, pendingUpgradeVeto, dueApprovedChange, dueApprovedFloor, vetoRetryDisposition, readChangeVeto, readPendingChange, checkPolicy, checkConfig, GENESIS, checkProposedChange, spendExecution, observeExecutions, unpinnedChangeApprovals, parseApproval, changeApproval, floorApproval, afterCancelFailure, CANCEL_RETRY_MS, DEFAULT_POLICY, DAY_S, outflowAlerts, OUTFLOW_KEEP_S, ticketAlreadyGone, freezeAlreadyCovers, replayedBreachTime, ticketScanDue } from "./policy.mjs";
import { ingestBudgeted, ingestLagS, signatureWalk, signatureWalkSinceSlot, followAccount, cursorGone, PAGE_LOG_INTERVAL, requireTransaction, transactionOrSkip } from "./follow.mjs";
import { executeChangeIx as executeChangeIxFor } from "./keeper.mjs";
import { alerter, stallWatch, scanAll, actionLine } from "./alert.mjs";
Expand Down Expand Up @@ -1078,12 +1078,8 @@ async function checkFloor() {
const acc = await rpc.account(fc.config);
const fl = acc ? decodeFloor(acc.bytes) : null;
if (fl) checkGuardianSet(program, fl);
const alerts = fl ? floorAlerts(fl) : [{ code: "missing", msg: "no Floor config for this program" }];
const codes = alerts.map((a) => a.code).join(",");
if (floorReported.get(program) === codes) continue;
floorReported.set(program, codes);
if (alerts.length) log({ level: "warn", msg: "floor not ready", program, alerts });
else log({ msg: "floor ready", program });
const line = floorCheck(floorReported, program, fl);
if (line) log(line);
}
}

Expand Down
18 changes: 18 additions & 0 deletions cloud/guardian/policy.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -408,6 +408,24 @@ export function floorAlerts(fl) {
return alerts;
}

/**
* What to log about a Floor program's readiness, once per change, or null.
* `fl` is its decoded config, or null when there is none; `reported`
* remembers each program's last answer. The message names the missing
* pieces, codes first, so they stay inside the first 60 characters that
* alertFor keys its repeats on. It used to be "floor not ready" alone: the
* alert didn't say what was missing, and a second missing piece within 10
* minutes of the first was muted.
*/
export function floorCheck(reported, program, fl) {
const alerts = fl ? floorAlerts(fl) : [{ code: "missing", msg: "no Floor config for this program" }];
const codes = alerts.map((a) => a.code).join(", ");
if (reported.get(program) === codes) return null;
reported.set(program, codes);
if (!alerts.length) return { msg: "floor ready", program };
return { level: "warn", msg: `floor not ready (${codes}): ${alerts.map((a) => a.msg).join("; ")}`, program, alerts };
}

/**
* What to log about a guardian set against finding 107's margin, once per set
* shape, or null. `which` is "protocol" for the Vault protocol's set, or the
Expand Down
2 changes: 1 addition & 1 deletion cloud/test/alert.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,7 @@ test("an alert names what it is about when its text does not", () => {
const P = "Prog1111111111111111111111111111111111111111";
const M = "Mint1111111111111111111111111111111111111111";
const detail = (line) => alertFor(line, new Map(), 0).detail;
assert.equal(detail({ level: "warn", msg: "floor not ready", program: P, alerts: [{ code: "unarmed" }] }), `floor not ready | program ${P}`);
assert.equal(detail({ level: "warn", msg: "floor not ready (unarmed): no abort buffer staged", program: P, alerts: [{ code: "unarmed" }] }), `floor not ready (unarmed): no abort buffer staged | program ${P}`);
assert.equal(detail({ level: "warn", msg: "the admin moved tokens into the lifeboat: depositors may be short", mint: M, amount: 5 }), `the admin moved tokens into the lifeboat: depositors may be short | mint ${M}`);
const change = { type: "veto_change", changeId: 4, reason: "loosening 4 (kind 1) is not pre-approved" };
assert.equal(detail(actionLine(change, { signature: "S" })), "loosening 4 (kind 1) is not pre-approved | change 4 | tx S");
Expand Down
34 changes: 33 additions & 1 deletion cloud/test/policy.test.mjs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { afterHeartbeat, assess, guardianSetCheck, breach, decode, decisionTime, floorAlerts, guardianStanding, heartbeatEveryAfterAccount, heartbeatEverySeconds, judgeReplayedWindow, newState, observeProtocolFreeze, openIncident, observeFreezeEnd, incidentHold, pendingCancels, pendingChangeVeto, pendingExitVeto, pendingUpgradeVeto, dueApprovedChange, dueApprovedFloor, queueFailedVeto, readChangeVeto, readPendingChange, rememberTicket, changeApproval, floorApproval, checkPolicy, retirePendingFreeze, scaleBy, vetoRetryDisposition, consumeApproval, parseApproval, checkProposedChange, afterCancelFailure, programErrorCode, CANCEL_RETRY_MS, DEFAULT_POLICY, HEARTBEAT_RECHECK_MS } from "../guardian/policy.mjs";
import { afterHeartbeat, assess, guardianSetCheck, breach, decode, decisionTime, floorAlerts, floorCheck, guardianStanding, heartbeatEveryAfterAccount, heartbeatEverySeconds, judgeReplayedWindow, newState, observeProtocolFreeze, openIncident, observeFreezeEnd, incidentHold, pendingCancels, pendingChangeVeto, pendingExitVeto, pendingUpgradeVeto, dueApprovedChange, dueApprovedFloor, queueFailedVeto, readChangeVeto, readPendingChange, rememberTicket, changeApproval, floorApproval, checkPolicy, retirePendingFreeze, scaleBy, vetoRetryDisposition, consumeApproval, parseApproval, checkProposedChange, afterCancelFailure, programErrorCode, CANCEL_RETRY_MS, DEFAULT_POLICY, HEARTBEAT_RECHECK_MS } from "../guardian/policy.mjs";
import { u64le, i64le, concat, key, b58encode as b58encodeForTest } from "../lib/solana.mjs";

const USDC = 1_000_000n;
Expand Down Expand Up @@ -195,6 +195,38 @@ test("floor readiness: an unarmed stop button and an unpinned snapshot are repor
assert.deepEqual(floorAlerts({ status: "aborted", abortStaged: false, snapshotPinned: false }).map((a) => a.code), ["stopped"]);
});

test("floor readiness is logged once per change, names what is missing, and each change alerts (A5)", async () => {
const { alertFor } = await import("../guardian/alert.mjs");
const [P, P2] = [PROTOCOL_ID, owner];
const ready = { status: "live", abortStaged: true, snapshotPinned: true };
const reported = new Map(), seen = new Map();
assert.deepEqual(floorCheck(reported, P, ready), { msg: "floor ready", program: P });
assert.equal(floorCheck(reported, P, ready), null, "said once");
const unarmed = floorCheck(reported, P, { ...ready, abortStaged: false });
assert.equal(unarmed.level, "warn");
assert.equal(unarmed.msg, "floor not ready (unarmed): no abort buffer staged: no guardian can stop the program until one is");
assert.deepEqual(unarmed.alerts.map((a) => a.code), ["unarmed"]);
assert.equal(floorCheck(reported, P, { ...ready, abortStaged: false }), null, "said once per change");
assert.equal(floorCheck(reported, P2, { ...ready, abortStaged: false })?.msg, unarmed.msg, "each program is remembered on its own");
// The alert says what is missing; it used to be "floor not ready | program P".
assert.equal(alertFor(unarmed, seen, 0).detail, `${unarmed.msg} | program ${P}`);
// A second piece missing a minute later is news: it used to be muted for
// 10 minutes as the same "floor not ready".
const both = floorCheck(reported, P, { ...ready, abortStaged: false, snapshotPinned: false });
assert.equal(both.msg, "floor not ready (unarmed, no_snapshot): no abort buffer staged: no guardian can stop the program until one is; no snapshot pinned: after a stop, restoring needs a new frozen buffer and a quorum");
assert.notEqual(alertFor(both, seen, 60_000), null);
const gone = floorCheck(reported, P, null);
assert.equal(gone.msg, "floor not ready (missing): no Floor config for this program");
assert.notEqual(alertFor(gone, seen, 120_000), null);
const stopped = floorCheck(reported, P, { status: "aborted", abortStaged: false, snapshotPinned: false });
assert.match(stopped.msg, /^floor not ready \(stopped\): the program is stopped: /);
assert.notEqual(alertFor(stopped, seen, 180_000), null);
// Ready again is logged, and is not an alert.
const back = floorCheck(reported, P, ready);
assert.deepEqual(back, { msg: "floor ready", program: P });
assert.equal(alertFor(back, seen, 240_000), null);
});

test("a guardian set without two keys beyond its threshold is warned about once per shape, alerting on mainnet only (finding 107)", async () => {
const { alertFor } = await import("../guardian/alert.mjs");
const set = (threshold, n) => ({ threshold, guardians: Array.from({ length: n }, (_, i) => `g${i}`) });
Expand Down
2 changes: 1 addition & 1 deletion docs/09-VERIFICATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ The programs are checked against the core as a model (§7). Two implementations
| **Wide program differential** | The same, with a limit configuration drawn per seed (floors and caps far above the balance, a closed lane, a queue delay past the horizon) and every instruction that changes what can leave: protocol and vault freezes, quorum unfreezes, ticket extensions, backlog resets, lifeboat moves both ways, limit changes; `quote_release` against the release, every `attest` field against the core (§7) | 5 Oct: 200,000 steps across 400 configurations, then 100,000 with limit changes after finding 30, identical |
| **Floor differential** | Random sequences of every Floor instruction but `register`, `release_authority` and `close_buffer` (those three are tested in `floor.rs` and the substitution sweep) on the real loader, against a model built from the core's governance and restore rules; stale guardians, strangers, a former owner, wrong and malformed veto ids, and a recovery key that comes and goes (§7) | 8 Oct, with the recovery key: 200,000 steps across 400 seeds, every config field and the live code identical; by default 24,000 steps in CI |
| **End-to-end** | `solana-test-validator` + the programs + the Cloud guardian + an attacker (`cloud/e2e/run.sh`) | Attacker ordered 1.2M and got 100k: the burst is 2% of what is in the vault at each payout, so the second 100k no longer fit (200k before the 29 Sep fix); guardian froze 2–3 s after the breaching payout (`09`, finding 82; 2–6 s before it); every queued payout cancelled; page numbers equal to `attest`; the run fails unless all of this holds |
| **Client tests** | 184 JavaScript tests: base58, legacy messages, PDA derivation against an on-chain address, 268 curve25519 vectors from `curve25519-dalek`, RFC 8032 test vector 1, the guardian's policy, follower, delivery and keeper, a paced attacker against the policy, the deploy scripts, the status page's pending-change view and account decoders, the limiter port, the low-water balance included, against the Python reference and the Rust core (40,000 cases, 10,000 of them full-range: u64 capacities, windows up to u32::MAX seconds, any duration; every available-now view in 1,500 limiter sessions, a third of them full-range; 10,000 arbitrary bucket states against Python only), approvals pinned and spent, restarts, a 30-day patient attacker, bounded ingest, RPC freshness, cursor-safe walks, the freeze's escalating send, config checks | 184/184 |
| **Client tests** | 185 JavaScript tests: base58, legacy messages, PDA derivation against an on-chain address, 268 curve25519 vectors from `curve25519-dalek`, RFC 8032 test vector 1, the guardian's policy, follower, delivery and keeper, a paced attacker against the policy, the deploy scripts, the status page's pending-change view and account decoders, the limiter port, the low-water balance included, against the Python reference and the Rust core (40,000 cases, 10,000 of them full-range: u64 capacities, windows up to u32::MAX seconds, any duration; every available-now view in 1,500 limiter sessions, a third of them full-range; 10,000 arbitrary bucket states against Python only), approvals pinned and spent, restarts, a 30-day patient attacker, bounded ingest, RPC freshness, cursor-safe walks, the freeze's escalating send, config checks | 185/185 |
| **`floor check` self-test** | The Python on-curve test against the same 268 dalek vectors | Agrees on all 268 |
| **Lint** | `cargo fmt --check`, `cargo clippy -D warnings` on every crate and program | Clean |

Expand Down