Skip to content

Latest commit

 

History

History
59 lines (51 loc) · 14.5 KB

File metadata and controls

59 lines (51 loc) · 14.5 KB

Changelog: Parameters

A focused ledger of deliberate changes to tunable parameters in this fork — constants, config defaults, timeouts, window/limit sizes, and congestion-control coefficients.

This complements CHANGELOG.md. The changelog records user-visible behavior in prose; this file is a compact table of every parameter value we have re-tuned, so reviewers and operators can see — at a glance — what changed, where it lives, and why.

How to use this file

When a PR changes a tunable parameter, add a row to the table below in the same PR. A "tunable parameter" is any value chosen for behavior or performance rather than correctness — a constant, a Config default, a timeout, a window or limit, or a backoff/growth coefficient.

Keep entries newest-first. Each row records:

  • Parameter — the constant or config field name.
  • Location — the file where it is defined (crate-relative path).
  • Old → New — the previous value and the new value.
  • PR — a link to the pull request that made the change.
  • Why — a one-line rationale.

Parameters

Parameter Location Old → New PR Why
EOS_PANIC_AFTER zakurad/src/components/sync/end_of_support.rs 7 days → 18 days #234 v1.0.1 releases 2026-07-17, ~11 days before the expected Ironwood activation (height 3,428,143, ~2026-07-28); keeping the v1.0.0-era 7-day window would halt every 1.0.1 node ~4 days before activation. 18 days puts the halt one week after activation (~2026-08-04, height ~3,436,436), so 1.0.1 spans the network upgrade while still forcing migration to the post-Ironwood release. Warnings keep their 3-day lead via EOS_WARN_AFTER = EOS_PANIC_AFTER - 3 (start ~2026-08-01).
DEFAULT_BS_MAX_REORDER_LOOKAHEAD_BYTES zakura-network/src/zakura/block_sync/config.rs 6 GiB − 32 MiB (wire ×4 semantics) → 1.5 GiB (resident, wire-honest) #190 The apply backlog now stays serialized (decoded bodies are bounded by the submission window) and admission charges serialized pools at wire size, so the look-ahead budget is the byte ceiling of the backlog itself and is sized as a memory target: total sync RSS tracks budget + decode window + process overhead, independent of block era. 1.5 GiB holds nearly two checkpoint ranges of maximum-size blocks and about a minute of worst-case committed throughput of look-ahead; deeper speculation buys nothing the committer can consume. The checkpoint-range clamp floor drops from range wire bytes ×4 to range wire bytes for the same reason (the commit-window admission exemption guarantees range assembly).
EOS_WARN_AFTER zakurad/src/components/sync/end_of_support.rs EOS_PANIC_AFTER - 5EOS_PANIC_AFTER - 3 #179 Give operators three days of upgrade warnings before v1.0.0 reaches end of support.
EOS_PANIC_AFTER zakurad/src/components/sync/end_of_support.rs 18 days → 7 days #179 Keeping the v1.0.0 support window short to encourage migration to the next release.
GLIBC_MALLOC_ARENA_MAX zakurad/src/bin/zakurad/main.rs glibc default → 16 #148 Mainnet sync nodes repeatedly approached or reached OOM on the default allocator configuration, especially during sandblast: one 32 GiB node reached 30.8 GiB RSS with no swap, severe memory/I/O pressure, and repeated health-probe timeouts. Live eBPF allocation profiles identified block-body deserialization—not headers—as the allocation workload, dominated by variable-length transaction bytes, Orchard actions, raw block buffers, and transaction collections. An A/B run over the first 100k heights also ruled out the header-sync update in #115 as the cause. A live malloc_trim(0) reduced RSS from 12.1 GiB to 3.0 GiB, reclaiming 9.1 GiB (75%) and confirming that most excess RSS was fragmented allocator-retained free memory rather than a leak. The initial tuning value of four arenas kept RSS near 3.7 GiB during a 30-minute full-concurrency run and improved throughput around the lighter blocks above height 2M, where fragmentation was severe. Arena count is a probabilistic tuning parameter, however, so four arenas do not imply a fixed RSS ceiling or throughput reduction; setting it too low can increase contention among worker threads performing the larger allocations common at early heights. Comparative runs with 8, 16, and 64 arenas over the first 100k heights found 16 to provide the best balance, while 8 delivered about 15% less throughput. Limiting glibc to 16 arenas therefore reduces allocator-retained memory without imposing the early-height contention observed at lower values. This remains a Linux-specific mitigation; the durable cross-platform pipeline fix is to account decoded applying memory directly, or keep excess bodies serialized until a bounded applying slot is available.
DEFAULT_BS_BBR_CWND_GAIN_PERCENT zebra-network/src/zakura/block_sync/config.rs 200300 #361 Ramp a proven peer up faster (1 → 3 → 9 … per round) from the smaller cold-start floor; the reliability discount and delay-gradient ceiling pull it back if the extra concurrency costs drops or standing queue.
DEFAULT_BS_BBR_MIN_CWND_BYTES zebra-network/src/zakura/block_sync/config.rs 4 MiBMAX_BLOCK_BYTES + 512 KiB (≈2.5 MB) #361 Conservative cold start: size the window floor to one max block plus headroom so a just-proven peer rides its measured BDP up via the higher gain instead of bursting to multiple megabytes.
DEFAULT_BS_INITIAL_BLOCK_PROBE_REQUESTS zebra-network/src/zakura/block_sync/config.rs new → 1 #361 Probe-first: an unproven peer gets a single request before its first accepted body, so a peer that accepts requests but never serves bodies cannot spend a full cold-start burst.
DEFAULT_BS_MAX_REQUESTS_WITHOUT_BLOCK_PROGRESS zebra-network/src/zakura/block_sync/config.rs new → 64 #361 Hard cap on requests to a proven peer without an accepted body before the no-progress liveness deadline disconnects it.
DEFAULT_BS_BBR_RELIABILITY_WEIGHT_PERCENT zebra-network/src/zakura/block_sync/config.rs new → 100 #361 Weight of the per-peer goodput discount on the BDP-derived cwnd (0 = plain BBR, 100 = full discount), folding the cost of dropped requests into the window so a request-dropping carrier holds proportionally less in flight.
BBR_RELIABILITY_EWMA_ALPHA zebra-network/src/zakura/block_sync/bbr.rs new → 0.1 #361 EWMA weight for the per-peer reliability estimate (~10–20 request outcomes), so a brief blip does not collapse a peer but sustained dropping does.
max_reorder_lookahead_blocks zebra-network/src/zakura/block_sync/config.rsadmission.rs config knob (default 262144) → removed; the cap is the fixed admission::LOOKAHEAD_BLOCK_HARD_CAP = 262144 [#413] Pure defense-in-depth on bookkeeping size (it binds before the byte gate only for tiny bodies averaging < ~6.1 KB wire); it never needed operator tuning, and the resident-memory budget is the primary bound. Configs carrying the key no longer parse (deny_unknown_fields).
fanout zebra-network/src/zakura/block_sync/config.rs config knob (default 1) → removed [#413] Legacy reservation multiplier: no scheduler ever sends the same range to multiple peers, so it only inflated a validation floor. Configs carrying the key no longer parse (deny_unknown_fields).
COMMIT_WINDOW_EXEMPT_SPAN_BLOCKS (look-ahead gate exemption) zebra-network/src/zakura/block_sync/admission.rs single commit-frontier block (verified_tip + 1) → one checkpoint range (MIN_BS_CHECKPOINT_SUBMITTED_BLOCK_APPLIES = 401 blocks) above the verified tip #386 Checkpoint sync resolves a range only once the whole range is co-resident while verified_tip stays pinned, so a single-block exemption deadlocked whenever the gate filled mid-range (exact-fit clamped configs wedged on one next-range body or reservation). Exempting the whole commit window makes range assembly independent of budget size; the plateau gains at most one worst-case window (~3.2 GB). Deliberately a constant, not max_submitted_block_applies, which has no ceiling and would let a huge submit window disable the gate.
Block-sync look-ahead resident accounting zebra-network/src/zakura/block_sync/admission.rs stage-aware (applying + input × factor, reorder ×1, reservations ×0) → flat (reorder + applying + input + reserved) × DESERIALIZED_MEM_FACTOR #386 Reverts the short-lived stage-aware model: charging reservations 0× made in-flight volume invisible to the byte gate (a commit stall could fill the whole in-flight wire budget, then decode ×4 to ~24 GiB, and the reorder→applying drain decodes unconditionally with no re-gate, so under-charging reorder had the same tail. Every pool is charged at its eventual decoded cost; stage-aware charging becomes sound once decode-at-submit lands.
max_reorder_lookahead_bytes / max_reorder_lookahead_blocks (checkpoint-range floor) zebra-network/src/zakura/block_sync/config.rs effective budget min(max_reorder, max_inflight) (wire) → min(max_reorder, max_inflight × DESERIALIZED_MEM_FACTOR) (resident); added a config-load clamp up to one checkpoint range (BS_CHECKPOINT_RANGE_BYTE_FLOOR × 4 bytes, MIN_BS_CHECKPOINT_SUBMITTED_BLOCK_APPLIES blocks) #386 A wire-denominated effective budget starved look-ahead depth ~4× under the resident gate, and a sub-range budget would refuse nearly all above-window work. Defense-in-depth sizing: checkpoint-sync liveness itself is guaranteed by the commit-window exemption, not this clamp.
DEFAULT_BS_MAX_REORDER_LOOKAHEAD_BLOCKS (max_reorder_lookahead_blocks default) zebra-network/src/zakura/block_sync/config.rs 4096262144 #386 The 4096 block cap bound before the resident-memory byte budget, making the block-sync applying buffer shallow (~one checkpoint range) and masking max_reorder_lookahead_bytes — it also risked starving the bursty VCT committer. Raised so the memory budget is the binding cap: a deep (~100k-block) buffer for burst absorption, still bounded to the resident-memory budget. Block count remains defense-in-depth.
max_reorder_lookahead_bytes (effective meaning) zebra-network/src/zakura/block_sync/config.rs (enforced in admission.rs) wire bytes → resident bytes (≈¼ the effective block headroom); now also gates the floor lane #386 Previously counted wire bytes and was bypassed by floor-priority requests, so the applying queue reached ~569k decoded blocks (~26 GiB RSS, OOM). Now enforced in resident memory (× DESERIALIZED_MEM_FACTOR) and on the floor path, with only the commit window exempt. Same default value, ~4× less effective headroom.
DESERIALIZED_MEM_FACTOR zebra-network/src/zakura/block_sync/admission.rs (new) → 4 #386 Block-sync look-ahead now bounds resident memory (serialized × 4), a calibration of the measured ~3.3–4× wire→decoded ratio, so a small-block backlog can't OOM the node. Approximate; a precise per-block heap estimate is a TODO.
ZAKURA_BLOCK_SYNC_CHECKPOINT_FRONTIER_REFRESH_INTERVAL zebrad/src/commands/start/zakura/block_sync_driver.rs 5s200ms #374 Recycle the checkpoint apply window promptly during checkpoint sync so the finalized writer is not left idle for ~5s between frontier refreshes.
DEFAULT_ZAKURA_BOOTSTRAP_PEERS zebra-network/src/zakura/handler.rs empty default → 9 native bootstrap peers #376 Let Zakura nodes discover the native P2P network without requiring every operator to configure bootstrap peers manually.
DEFAULT_ZAKURA_MAX_CONNECTIONS zebra-network/src/zakura/handler.rs 32256 #376 Raise the native P2P connection envelope for production sync and peer diversity.
DEFAULT_ZAKURA_MAX_PENDING_HANDSHAKES zebra-network/src/zakura/handler.rs 832 #376 Allow more simultaneous native control handshakes during bootstrap and peer churn.
DEFAULT_ZAKURA_STREAM_OPEN_RATE_PER_SECOND zebra-network/src/zakura/handler.rs 1632 #376 Permit higher stream-open churn across the larger default peer set.
DEFAULT_ZAKURA_STREAM_RECEIVE_WINDOW zebra-network/src/zakura/handler.rs 3 MiB32 MiB #376 Avoid throttling high-throughput native streams with the earlier conservative per-stream receive window.
DEFAULT_ZAKURA_RECEIVE_WINDOW zebra-network/src/zakura/handler.rs 16 MiB32 MiB #376 Match the connection receive window to the larger stream window used for production sync.
DEFAULT_ZAKURA_SEND_WINDOW zebra-network/src/zakura/handler.rs 16 MiB32 MiB #376 Keep the native QUIC send window from becoming the bottleneck for larger receive windows.
OUTBOUND_WINDOW_FLOOR_TIMEOUTS_BEFORE_DISCONNECT zebra-network/src/zakura/block_sync/state.rs 32 * OUTBOUND_WINDOW_REDUCTION_EPOCH_TIMEOUTS (32) #303 Tolerate two full reduction epochs (~256s at the 8s request timeout) of floor-pinned timeouts before disconnecting a block-sync peer, instead of ~24s, so briefly-congested peers are not churned. Any successful response resets the streak.
RocksDB max_total_wal_size zebra-state/src/service/finalized_state/disk_db.rs 0 (unbounded) → 4 GiB #383 Bound WAL growth during heavy sync so restarts do not spend minutes replaying tens of GiB of logs.