Repository navigation
Write log lines and webhook bodies as well-formed JSON - #137
Merged
Merged
Conversation
Row 106 said its 16 Oct re-run was on tree 36187b4, "the tree of 86b777b (PR #100's head)". PR #100 was pushed again before the owner merged it: refs/pull/100/head is now 7ded086, 86b777b is not an ancestor of it, and no ref on origin (branches or refs/pull/*/head) contains 86b777b or a commit with tree 36187b4. The citation named something nobody can fetch. What the re-run used is still on main. git diff 86b777b 04ad61e -- programs crates touches one file, programs/tests/tests/vault.rs, 2 lines: a doc comment of a_guardian_silent_past_its_interval_can_no_ longer_act rewrapped. So the row now cites main at 04ad61e (PR #100's squash) and says that one difference. Sibling search: every 7-12 hex token in README.md and docs/*.md (47) resolved with git cat-file; each commit checked with for-each-ref --contains against refs/remotes/origin and refs/pull/*/head. Only 86b777b was unpublished; 36187b4 was the only tree id. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ledger target had two gaps. It mapped every mint byte to one of 4 mints, so its ledgers never filled (MAX_SLOTS = 8) and backed_amount's full-ledger refusal (ledger.rs:237) was never reached; docs/09 listed this. And it never checked a grant against the rule: it asserted backed <= amount, that a re-ask grants the same, and that debit_backed accepts it, but debit_backed re-runs backed_amount and only a missing slot could make it fail, so no unsound grant could fail the target. The target now draws from MAX_SLOTS + 1 mints, each with its own price, keeps the first MAX_SLOTS distinct nonzero positions (it took the first 8 raw tuples before), and checks every grant against sound_after, restated from g3_backed_amount_is_sound_and_maximal (tests/properties.rs): the grant is sound, and one unit more and the whole request are not. from_positions can no longer refuse the ledger it builds, so it is expected to succeed. Its prices keep hi >= 1, as conservative() does (price > 0, hi >= max(price, ema)). With hi = 0 a debt is worth 0, and backed_amount's no-collateral shortcut (ledger.rs:254, "exactly what the search below returns") grants 0 where the rule allows the whole amount: both 120 s runs of the first version found it (tasks/fuzz-full-ledger/artifacts- hi0: one debt in the asked mint, priced 0). No price the Vault reads is 0, and liability_bps >= 10,000 (genesis.rs:36, classify.rs:83), so a debt is worth at least 1 on chain; the shortcut is exact there. sound_after returns c > 0 when the health sum overflows, where the core fails closed; not reachable: value_v caps a value at u128::MAX / BPS (a 2e38 collateral counts 0, f1-overflow.log), and 9 of those fit i128. Evidence (autopilot tasks/fuzz-full-ledger/, r6-mutants.sh/.log at this head), each mutant 3 rounds of 60 s from a fresh corpus, and cargo test -p bulkhead-core: - M1 full-ledger refusal dropped (:237 `if false &&`): fuzz 3/2/2 s. The old target survived all three rounds (14.1M-14.8M runs each). - M2 full at > MAX_SLOTS (:225): 5/8/3 s. - M3 full at >= MAX_SLOTS - 1, M4 `!present` dropped (refuse too much): "maximal" in 4/2/1 and 3/2/2 s. - M5 health bound t >= -1 (:246): "sound" in 2/2/2 s. No core test fails; check.sh's Python differential does (21/17/16 mismatches at seeds 11/12/13, 50,000 cases; m5-pyrust.log). - M6 unknown health counted healthy (:242), M7 unallowlisted paid past its balance, M8 never paid (:231), M9 skip valid(amount) (:259): each fails within 2 s in all rounds. - The whole-request check alone (the others disabled) kills M9 in 2/1/1 s (allofit-M9-r*.log). - Core tests catch M1-M4 and M6-M9 too; M5 is a first catch for cargo test only, and the differential has it. - Real core: SECONDS_PER=300 scripts/fuzz.sh ledger, 45.6M runs, no crash. The input layout changed, so saved ledger corpora (local and CI's fuzz-corpus cache) decode into other values; libFuzzer reloads them as plain inputs. Sibling: docs/09 lists one other refusal no target asks for, the closed limiter lane (capacity at least 1); left as is. The shortcut comment's unstated hi >= 1 premise is a backlog item, not changed here (core line numbers). No program or core source change; sBPF hashes unaffected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
crossCheck noted `active`, then awaited every endpoint's signature list. A call() that failed over in that wait moved `active`, and the check went on to switch away from the new endpoint on the old one's list. A signature first seen missing on the old primary also kept its clock across such a move, so the new primary was switched off at its first check instead of after SIGNATURE_LAG_SWITCH_MS. Root cause: the `missing` map recorded which backup had a signature but not which endpoint lacked it, and the switch did not check that the endpoint the lists judged was still active. Fix: `missing` is one map per endpoint, filled from the lists of the endpoint that was active when they were asked, so an endpoint call() moved to gets its own 30 s. The switch happens only if that endpoint is still active. A switch by crossCheck itself still clears every map, as the single map was cleared before (`restartWaits`, which also makes the maps lazily, so they follow `endpoints`). The crossCheck comment says all this. Two earlier versions were replaced after code review: clearing the map on any move (call() moving e0 -> e1 -> e0 between checks would restart the wait every time and keep the guardian on lagging endpoints, where the old code switched), and dropping the whole round when `switches` moved during it (e0 -> e1 -> e0 in flight, and the lists still judge e0). Evidence (tasks/crosscheck-switch/ in the autopilot dir): - Nine tests in delivery.test.mjs against stub endpoints. r6/red-old.log: on the old code (0b233e9) three of 66 fail: the in-flight test ("no verdict on lists from the old primary"), the moved-to test ("e2 not judged on e0's wait") and the filed-under test ("e2's wait started at 50 s"). The other five pass on the old code and pin what it got right: the flap test and the moved-back test fail on the two earlier versions (M5, M3); the two clear tests pin the old clears of every map, which had no test (mutants-r2-partial.log: three such mutants survived before them); the left-in-flight test (from A's review) pins that lists asked of an endpoint call() left while they were in flight still record what it lacks, from every other endpoint, the one now active included: it runs once with S listed by e2 (now active) and once by e1. The ninth (A's, from the r3 review) pins the other half: lists asked of an endpoint call() left in flight do not prune the wait of the one now active. - r6/mutants13.log: 13/13 KILLED, none void, 1-9 of 66 fail (switch guard off/inverted, whole round dropped on any move, one shared map, clear on any move, evidence filed under the endpoint active after the await; on a slot switch: keep all maps, clear only the left, only the joined, or both; on a signature switch: keep all, clear only the left, or left and joined). - r6/mutants-loop.log: A's X3 (record nothing when `active` moved in flight), X4 (backup filter reads this.active) and Y1 (when moved, record only from the endpoint now active) each fail the left-in-flight test alone; r4/mutants.log: X3 and X4 survive without it. r6/mutants-committed.log: Y1 survives the test with only e2 listing S, hence the second case. - r7/ (on 859ed7e, after the ninth test): mutants.log and mutants-rest.log, M1-M13, X1-X4, X6-X8, Y1, Z1, Z2, Z4 KILLED, 1-14 of 67 fail. A's Z4 (the prune loop reads this.missing[this.active]) survived all 66 earlier tests (tasks/review-crosscheck-switch-r3/ z4.log) and fails the ninth alone now. X5 survives (equivalent, A's r2 review), and so does Z3 (A's first, wrong form of Z4). - node --test cloud/test: 237/237 (docs/09 client cell, 228 + 9). Limits (backlog c16, c17, c18): the switch goes to the backup that listed the signature when it was first seen missing, even if that backup is down now. The old code did that too, but kept fewer such entries: one became void when that backup was active and listed the signature, and an endpoint's entries now outlive call()'s moves. An inactive endpoint's wait keeps ageing (to the 600 s expiry), so one call() returns to after a long absence is left at its first check. An endpoint call() moves to is judged from the first check that finds it active, so a lagging one is left a check later than before: up to about two minutes at the default 60 s freshnessCheckS (before, it was left at once, on the old primary's wait). Sibling search: call() already dates its evidence with `switches` (finding 84 family). crossCheck's slot step reads `active` after the slot await and compares that same endpoint's slot, with no await before its switch, so a move there is judged on the right endpoint. No other reader of `missing` (git grep in cloud/). Finding 119 reserved; its row waits on rows 114-118. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An emergency stop whose primary `rpc` does not parse as a URL (a typo
such as "http//host") gave up, though its backup was good. fetch
refuses the primary, so FailoverRpc moves to the backup and calls the
stop's onSwitch, which built its log line with `new URL(from).host`.
That threw inside call()'s try: the backup's answer was taken for its
failure, every read failed the same way, and the stop logged "gave up:
Floor's config could not be read within the time limit" with "Invalid
URL", exit 1 (finding 121).
Root cause: the stop kept its own URL parsing for logs, while the
guardian's copy (main.mjs, hostOf) had long caught the error; the two
drifted. Fix: one hostOf, exported from cloud/lib/solana.mjs and used
by both. It also gives "(endpoint)" for a URL with no host
("localhost:8899" parses with "localhost:" as its scheme), which
gave an empty name. The stop's switch line now names the config
entry, as its cluster lines do ("RPC (endpoint) (rpc) left; now using
host (rpcs[0])"); onSwitch is passed both indexes for that, since a
URL listed twice cannot name its entry. Why FailoverRpc let a hook's
error discard an answer is the next commit.
Evidence (tasks/switch-outside-try/; r2/red1.log, r3/):
- Test "a stop whose primary URL does not parse still stops through a
backup" (stop.test.mjs). With the parent's stop.mjs, main.mjs and
solana.mjs it fails, the stop "gave up" with "Invalid URL" (red1.log;
r6/red-parent.log on this test as committed). Now exit 0, "stopped",
and "RPC (endpoint) (rpc) left; now using host (rpcs[1])": the bad URL
is listed twice, so the stop skips rpcs[0] and the line must name
where it went.
- Tests "onSwitch is told each endpoint's index, a URL listed twice
too" and "hostOf names a host, and only a host, for logs"
(delivery.test.mjs).
- r3/mutants1.py (mutants1.log; re-run on this test: r6/mutants1.log,
same), none void: S1 the switch line parses
URLs itself, S2 hostOf rethrows, S3 the line names hosts only, S4
hostOf gives an empty host, S5 switchTo gives no indexes: KILLED. S6
the stop maps URLs back to indexes with indexOf SURVIVED, as
expected: the stop never leaves a URL listed twice, so the stop's
use of the indexes is covered only jointly with the lib test that
they are passed (the test's comment says so). Y4 (A's review: the
new entry named endpointName(fromAt + 1)) survived the first version
of the test, with one backup; KILLED now (r6/mut-f1.log).
- node --test cloud/test/*.test.mjs: 240/240. claims OK, client 240.
(The red and mutant logs were taken on the parent's earlier base,
5a7c6b2, with 239 tests; A's ninth freshness test came in since.)
Sibling search: `git grep "new URL(" cloud` outside tests: the two
hostOf copies (now one), and status/snapshot.mjs:22, which builds an
error message only after a read of that same URL has answered, so it
parses there. cloud/status/index.html:248 has its own helper that
falls back to the raw URL; a URL it refuses would not have fetched
(backlog c19). Not done here: the stop does not refuse an unparseable
URL at start, as checkConfig does for the guardian; with good backups
it can still stop, so that is a choice (backlog c21).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FailoverRpc called its report hooks (onSwitch, onCluster) where their
errors were read as the endpoint's. onSwitch ran inside call()'s try:
a report that threw made the endpoint that had just answered look
failed, its answer was dropped, and the request, a sendTransaction
too, went on to the next endpoint while the dead one stayed active.
That is how the stop's log line (previous commit) turned one bad URL
into a stop that read nothing. onCluster ran in askGenesis after the
cluster was recorded, so a throw there passed over an endpoint on the
right cluster. In crossCheck, a throwing onSwitch rejected the check
before anything moved, so the guardian's loop (main.mjs, "rpc
freshness") saw a failed step instead of the switch.
Fix: both hooks go through report(), which passes over a hook's error
and writes one line to stderr ("FailoverRpc: onSwitch threw: ...", at
most 160 characters of the message, or of the value thrown when the
message is empty), so the failure still leaves a trace. The trace is
guarded too: String() of some thrown values (an object with no
prototype) throws. A hook for that failure (onHookError) was weighed and
not added: the hook that failed is the log. The shipped hooks no longer
throw (hostOf, previous commit); this keeps call() and crossCheck right
for any hook. Hooks are synchronous; a rejected promise from one is not
caught (none is async). Throwing the hook's error to the caller was
tried first (branch auto/B/switch-outside-try-r1) and dropped: it still
loses an accepted transaction's signature, and signer.mjs's send() would
then report a freeze that landed as failed.
Evidence (tasks/switch-outside-try/r4/):
- Four tests (delivery.test.mjs): a throwing onSwitch neither drops
the answer nor sends the request on (three endpoints, the first
down; also checks the stderr line); crossCheck's switch with a
throwing onSwitch is made and returned to its caller (slot and
signature causes); a throwing onCluster does not pass over an
endpoint on the right cluster; a hook whose error cannot be printed,
or is long, is passed over too (an object with no prototype, a
Symbol, an Error with no message, a 161-character message). With
the parent's solana.mjs: 4/243 fail (red2.log).
- mutants2.py, 6/6 KILLED, none void (mutants2.log): G1 switchTo calls
onSwitch itself (3 fail), G2 askGenesis calls onCluster itself (1),
G3 report rethrows (4; mutants2-g3.log, as first written it hit the
guarded trace and was G4 again), G4 report leaves no trace (2), G5
the trace is not guarded (1), G6 an empty message is printed empty
(1). Y1 (A's review: the trace cut at 159, 161 or 1000 characters)
survived the first version of the tests; KILLED now, 1 fail each
(r6/y1.log).
- node --test cloud/test/*.test.mjs: 244/244. claims OK, client 244.
(The red and mutant logs were taken on the earlier base, 5a7c6b2,
with 243 tests; A's ninth freshness test came in since.)
Sibling search: FailoverRpc has no other callbacks; Rpc has none.
alerter reports its own failures through onError, outside any request.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PR #129 (859ed7e) went red in CI's `mutants` job: finding 115 moved murmur3_32 in crates/bulkhead-core/src/floor.rs by 28 lines, and the murmur3 tail's exclusion is pinned to lines (finding 105), so `floor\.rs:(327|330|333):11:` matched 0 mutants against a count of 3. The guard refused as designed, but only in CI: scripts/check.sh ran tools/mutant_exclusions.py's self-test, not the check on the real config, so the author's and the reviewer's check.sh were both green. scripts/mutants.sh gets `--exclusions-only`: the check it already runs first (the same lines, not a copy), then exit. It takes no other argument (exit 2), so a --config can't turn it into a silent pass. The counts are the mutants of the version CI installs, so that version is read from ci.yml's `cargo install --locked cargo-mutants@X` (exit 2 unless that names one X.Y.Z), and with another version installed, or none, the mode prints that it did not check and passes. check.sh runs it right after the claims step (0.3 s). CI's `check` job has no cargo-mutants, so it passes unchecked there; the `mutants` job still runs the check first, as before. Both agents' machines have 27.1.0. check.sh's header and docs/09 (command list, CI table, §6 "In CI.") say so. Evidence, tasks/exclusions-in-check/r5/ in the autopilot directory: - check-red.log: check.sh on this tree, rc 1 at the new step, CI's message ("matches 0 mutants, its count says 3"). - check-green-with-pattern-fix.log: pattern repointed to (355|358|361) in the worktree only: "22 of 800 mutants excluded, each pattern matching its count", ALL CHECKS PASSED, rc 0. - paths.log, `scripts/mutants.sh --exclusions-only` with JOBS=auto: stale rc 1; an extra argument rc 2; repointed rc 0; ci.yml edited to 27.1.1 prints "not checked, they need cargo-mutants 27.1.1" rc 0; the install line twice rc 0; 27.1.0-rc.1 rc 2; no version rc 2. Each edit was reverted (git status). Code review (5 passes) shaped this. Pass 2: the first draft called `scripts/mutants.sh --list`, which lists again after the check with the run's options, so JOBS=auto failed check.sh after a passing check. Pass 3: the flag was tested twice and reached cargo-mutants when combined with others. Pass 4: the version was a fourth copy of CI's; a version line with a suffix was read as another version. Pass 5: a pre-release pin was read as its release, and the same install line twice failed; docs/09 named 27.1.0 again. Kept as is: the step runs early (fast feedback, like claims); another version passes with a printed line, since its counts can differ; a full mutants.sh run is not gated (as before). Not done here: ci.yml's cache key repeats the version (bumping only the install line would keep CI on the cached old one), cargo-mutants in CI's `check` job (a draft PR or the weekly run gets no exclusion check), and the root fix, a murmur3 tail pattern not pinned to lines (e.g. the tail in its own function), which edits core code the programs build from (floor.rs:454 calls murmur3_32); proposed to A. tools/mutant_exclusions.py's self-test runs a copy of mutants.sh and itself in a temporary tree with a planted config and ci.yml (the repo's target/ is not written: shasum same before and after), and a planted `cargo` in $HOME/.cargo/bin that logs its calls, reports a given version or fails as if missing, and serves two lists. Ten runs: `--config x` and `--config=x` rc 2, no cargo call; another version, and none, "not checked" rc 0 after only `--version`; two install lines, none, and a pre-release, rc 2; the same line twice is one version; CI's version with lists the config matches rc 0 after exactly the version call and the two lists (so it stops after the check); with lists it does not match rc 1. python3 is linked beside the shim, since a pyenv or asdf python3 can fail under a planted $HOME (not tried here). check.sh now runs this self-test before the exclusions step, so a broken check fails under its own name first; docs/09's command list and `check` row follow the new order. Mutants of mutants.sh, each applied, red, restored (tasks/exclusions-in-check/r6/mutants.log; rc 1 each, failing cases in brackets): M1 `!=` -> `==` on the version test, review F1's first mutant [4]; M2 the `$#` guard dropped, F1's second [1]; M3 `exit 1` after the check [1]; M3b that line deleted, so a passing check goes on to the full run [1]; M4 the skip exits 1 [2]; M5 the version read always matches [2]; M6 the check's status ignored [1]; M7 only the first of two install lines read [1]; M8 the X.Y.Z guard deleted [2]; M9 `sort -u` dropped [1]; M10 the X.Y.Z regex loosened to any word [1]; M11 the guard loosened to `$# < 2` [1]. Restored: self-test OK, 32 cases. r2, after A's review (F1: M1 and M2 survived, no in-tree test) and four more code-review passes. Taken: the first test draft ran the real mutants.sh and wrote fake lists into the repo's target/ (now a copied tree); a planted $HOME could break a pyenv python3 (linked); the stop after a passing check was untested (M3, M3b); the self-test ran after the check it guards; no-install-line case (M8); the duplicate line and pre-release cases were tested only out of tree (M9, M10); one extra argument was untested (M11); docs order, and docs/09 §3 lists `--exclusions-only`. Not taken: an install line in a ci.yml comment fails closed, rc 2 naming the cause; `--exclusions-only` is read only as the first argument (as documented; elsewhere cargo-mutants refuses the flag); mutants.sh:3 repeats 27.1.0 (it was there before); no TimeoutExpired handling (as the existing command-line run); the two lists run one after the other; a tree without ci.yml fails at the step (a git checkout or archive has it); check.sh rewrites target/mutants-*.txt, as any mutants.sh run in that tree does (one run per tree at a time). Sibling search: the only line-pinned pattern in .cargo/mutants.toml is the murmur3 tail (its header says so). Every other pattern has `\d+` for the line; limiter.rs's `:(27|26)` is two columns, not lines. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The guardian builds its alerter before checkConfig, so a start refused
for its config still alerts (finding 71). It took alert.format as
given: a config refused for "format": "slak" or "Slack", or for another
misspelt key beside a correct url, posted the JSON body, and a Slack or
Discord webhook does not show it, so the alert saying the guardian did
not start was lost. A config with no format and a Slack URL posted
JSON on every alert the same way.
formatFor (alert.mjs): a valid format is kept; otherwise a Slack
incoming webhook (hooks.slack.com/services/...) gets Slack's body and a
Discord webhook (discord.com or discordapp.com and their subdomains,
/api[/vN]/webhooks/<id>/<token>) Discord's; on any other URL a format
refused only for its case or spacing ("Slack") is read as meant (a
string only: an object format must not throw before the fatal handler
exists); else JSON. Slack workflow triggers (hooks.slack.com/triggers/...) take the
JSON body's keys as variables, so they keep it, and Discord's
Slack-compatible .../slack endpoint is not guessed. docs/07 and
alert.mjs's header say so. checkConfig still refuses "Slack": only the
alert reads it.
Evidence (tasks/alert-format-host/):
- red-old.log: the new end-to-end test (main.mjs with "format": "slak",
fetch redirected from hooks.slack.com to a local server by a
--require preload) against the old alert.mjs: the body's keys were
severity, title, detail, at, protocol, not text.
- mutants.log: 18 mutants of formatFor and its use, all killed by the
unit test; the end-to-end test also kills the two on its path.
- Review (A): the Slack host's exact compare was unpinned
(host.endsWith("slack.com") survived); a notslack.com case kills it
(r6/n1-red.log: fail 1 of 11; r6/green.log 11/11). The Discord
subdomains, /api/vN and trailing slash the code takes are now in the
JSDoc and docs/07, and docs/07 says a Discord webhook's /slack or
/github endpoint gets JSON.
- node --test cloud/test/*.test.mjs 246/246; tools/claims.py OK
(client 246; docs/09 line 40 updated from 244).
Sibling search: alerter is the only webhook sender with a format
(liveness GETs its URL with no body; cloud/e2e posts to a local hook,
still JSON); main.mjs is its only caller. Not covered: a refused config
whose url key is itself misspelt has no URL to alert (backlog).
Finding 122 (reserved); its docs/09 row waits for the rows batch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Finding 123. An alert's detail can hold an RPC's own words: solana.mjs puts the endpoint's JSON error in the Error's message, and a failed scan or send logs it as an error line, which alerts. webhookBody passed that text through as chat markup. In Slack, "<!channel>" from the endpoint pinged the operators' channel and "<url|label>" showed a link under a label of the endpoint's choosing, in the channel the operators act from. In Discord, "@everyone" pinged, "[label](url)" was a masked link, "||" could hide the rest of the alert (the tx signature) in a spoiler, and a URL unfurled into a preview. Discord's cut at 2000 UTF-16 units could also keep half a surrogate pair. Fix: both chat bodies put the detail in a code block (none when it is empty), its backticks swapped for ˋ (U+02CB) so it cannot close the block and half pairs cut upstream (main.mjs cuts errors to 160 or 200 units) for U+FFFD. Slack's text also escapes &, < and > (Slack's control characters, which code blocks do not disable) and sets unfurl_links and unfurl_media false. Discord's body sets allowed_mentions { parse: [] } and flags 4 (SUPPRESS_EMBEDS), and cuts the detail, not the whole text, so the block stays closed, dropping a high surrogate left at the cut. The title line is fixed wording and unchanged; the JSON body is unchanged (its consumer decides). docs/07 says what the chat bodies now do, and that a bare URL may still show as a link. The two tests that pin Slack's body keys (alert.test.mjs:123, restart.test.mjs:396) list the new ones. docs/09's client count 246 -> 247. Evidence (tasks/alert-text-escape/): - red-parent.log: the tests against the parent's alert.mjs, fail 2 (the new test, and the key list at alert.test.mjs:123). - mutants.log: 22 mutants (each escape dropped, escape set without &, not global, allowed_mentions dropped or parse ["users"], flags dropped or 0, unfurl_links dropped, unfurl_media true, no backtick swap, no toWellFormed, a block for an empty detail, no surrogate trim, trim unanchored or of the low half, room without CLOSE, one over or from the title alone (a protocol label pins it), each block dropped) each fail the test; restored, pass 12 fail 0. r2/r14.log: trimming either half at the cut (A's R14) fails 1/32 on a pair that ends exactly at the cut ("d\ud83d" left); restored 32/32, file cmp-equal. An earlier draft's `length > room` guard was equivalent under `>=` (the slice is then a no-op), so the cut is unconditional. - check.log: scripts/check.sh ALL CHECKS PASSED. Not verified against the services (no network): Slack's escaping, code-block rendering and unfurl keys and Discord's allowed_mentions and flags are from their documented APIs. Sibling search: the only other chat or HTML sink is the status page, whose interpolations of chain or RPC text all go through esc (index.html:89, :97, :265; the rest are numbers or fixed strings); the liveness ping is a GET with no body; e2e's fetch is test-only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The guardian and the stop cut an RPC's error text by UTF-16 units
(`.slice(0, 160)`, `.slice(0, 200)`). When the cut falls inside a
surrogate pair, the line keeps the first half alone, and JSON.stringify
writes it as "\ud83d". Strict parsers refuse the whole text: serde_json
1.0.151 answers "unexpected end of hex escape" for {"detail":"ab\ud83d"}
and parses {"detail":"ab�"}. So a log shipper or a JSON webhook
receiver written that way drops that log line or alert, and the alert
is retried and then lost (finding 124).
Fix at the writer, not at each cut: jsonText (alert.mjs) makes every
string well-formed (U+FFFD for a half pair) and writes a bigint as its
digits, as main.mjs's replacer did. It writes the guardian's log lines
(main.mjs log, the undelivered-alert line, the state-save line), the
stop's log lines, and every webhook body. The Slack and Discord bodies
were already well-formed (webhookBody, finding 123); the JSON body was
not.
Tests (each fails at the parent):
- alert.test.mjs: jsonText on a half pair, a lone low half, a bigint
and a whole emoji; the body posted in each ALERT_FORMATS has no
\ud8xx-\udfxx escape and shows U+FFFD.
- stop.test.mjs: a getGenesisHash error whose emoji's first half is
unit 159 of the 160 kept; the stop's line shows U+FFFD.
- restart.test.mjs: the same for the guardian's start-up line about a
backup that does not answer (fakeRpc can now answer `{ error }`).
At the parent: alert 0/1 pass, stop and restart 0/2.
Mutants (tasks/json-well-formed/mutants.py): jsonText without
toWellFormed, without the bigint branch, the body via JSON.stringify,
main.mjs's log via JSON.stringify, stop.mjs's log via JSON.stringify:
5/5 KILLED, restored green. The undelivered-alert and state-save lines
in main.mjs have no test of their own; they use the same jsonText.
Sibling search: every JSON.stringify under cloud/ outside tests. The
others build error text (re-escaped as text, so well-formed), RPC
request bodies from our own values, key and state files, or
deploy-tool output; e2e/attack.mjs's demo log cuts e.message the same
way and is left as is (a demo, not run by operators).
docs/09: client tests 247 -> 250.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
shuv-amp
force-pushed
the
auto/B/json-well-formed
branch
from
October 19, 2026 17:45
18b2e53 to
865821b
Compare
This was referenced Oct 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
The JSON writer for log lines and webhook bodies (cloud/guardian/alert.mjs, main.mjs, stop.mjs), their tests and the client test count in docs/09 (247 -> 250). No program change. One commit, 18b2e53. Finding 124.
Log lines and webhook bodies are now well-formed JSON. The guardian and the stop cut an RPC's error text by UTF-16 units (
.slice(0, 160),.slice(0, 200)). When the cut falls inside a surrogate pair, the line keeps the first half alone, and JSON.stringify writes it as "\ud83d". Strict parsers refuse the whole text: serde_json 1.0.151 answers "unexpected end of hex escape" for {"detail":"ab\ud83d"}. So a log shipper or JSON webhook receiver written that way drops that line or alert, and the alert is retried and then lost.Fix at the writer, not at each cut: jsonText (alert.mjs) makes every string well-formed (U+FFFD for a half pair) and writes a bigint as its digits, as main.mjs's replacer did. It writes the guardian's log lines, the undelivered-alert line, the state-save line, the stop's log lines and every webhook body. The Slack and Discord bodies were already well-formed (finding 123); the JSON body was not.
Proof
Autopilot tasks/json-well-formed/ (runs taken before the rebase onto 804e297, at earlier ids of this commit):
scripts/check.shALL CHECKS PASSED. each-commit.log: green.bin/check-each-commit.sh 804e297all 7 green (tasks/review-rebase-35/). APPROVE.bin/integrate.shranbin/check-each-commit.shgreen on this unit's commit (autopilot tasks/integrate-it12/6.log), andscripts/check.shat the top of this batch, fb44f95, which contains this unit unchanged: ALL CHECKS PASSED (tasks/integrate-it12/check-fb44f95.log).Limits
🤖 Generated with Claude Code