Skip to content

Commit aa5bd30

Browse files
authored
perf(l1): move per-tx BAL validation into the par_iter closure (#6677)
## Summary `execute_block_parallel` previously returned `(current_state, codes, shadow_touched, shadow_reads)` per tx and validated them in a serial post-`par_iter` loop across all txs. Validation is per-tx pure work; only the marking of `unread_storage_reads` / `unaccessed_pure_accounts` mutates cross-tx state. This PR moves `validate_tx_execution` + shadow-touched / shadow-reads checks into the rayon closure (parallel). The closure also precomputes the small `(Vec<(Address, H256)>, Vec<Address>)` inputs the serial pass uses to update the shared sets, so `current_state` + `codes` no longer cross the rayon boundary. BAL validation errors are deferred via `Option<EvmError>` in the result tuple so the post-`par_iter` gas-limit check still takes priority (preserves `GAS_USED_OVERFLOW` > BAL mismatch on blocks exceeding the gas limit; the BAL is built assuming rejected txs, so miner balance in the BAL won't match execution that ran all txs). ## Changes - `TxExecResult` tuple shrinks from 8 fields (with two heavy `FxHashMap`s) to 7 fields with `Vec`s only. - Inside the closure, after `execute_tx_in_block`: - Compute `reads_satisfied: Vec<(Address, H256)>` (touched storage keys per non-destroyed account) and `destroyed: Vec<Address>` (selfdestructed accounts whose remaining BAL storage_reads are wiped) from `current_state`. - Run `validate_tx_execution` + shadow-touched + shadow-reads checks; capture the first error as `Option<EvmError>` (deferred). - Drop `current_state` and `codes` before returning. - After `par_iter`: - Existing 2D gas allowance / breakdowns / block-overflow loop unchanged. - **New step**: surface the first deferred BAL error in tx order (only reached if the gas-limit check passes). - Apply `reads_satisfied` / `destroyed` to `unread_storage_reads` and `tracked_accounts` to `unaccessed_pure_accounts` (cheap hash-set ops). ## Invariants 1. **Gas-limit error priority preserved**: the serial pass runs gas / 2D allowance / block-overflow before surfacing any deferred BAL error. 2. **Tx-order error reporting preserved**: errors are surfaced in sorted `tx_idx` order; the first failing tx still wins. 3. **Destroyed-account semantics preserved**: the `Destroyed | DestroyedModified` branch in the unread-reads marking is now an explicit `Vec<Address>` from the closure; the serial pass does the same `retain(|&(a, _)| a != addr)` it did before. 4. **No semantic change to validation**: same checks, same error strings, only the location moves. ## Benchmark Fixture: `bal-devnet-7-mainnet-mix-460` (460 blocks, ~30 Ggas, transfer/EVM mix). `release-with-debug`, `import-bench --with-bal`. Baseline = `feat/import-bench-bal-tooling` (bench tooling only, no perf change). | metric | baseline | this PR | delta | |------------------|---------:|--------:|--------------------| | wall time | 8.575 s | 7.871 s | **-704 ms (-8.2%)** | | agg Ggas/s | 3.901 | 4.291 | **+10.0%** | | avg ms / block | 16.884 | 15.349 | **-1.54 ms (-9.1%)** | | p95 ms / block | 17.730 | 16.050 | -1.68 ms (-9.5%) | | max ms / block | 90.060 | 85.000 | -5.06 ms | | **exec avg** | **15.574 ms** | **14.172 ms** | **-1.40 ms (-9.0%)** | | merkle | 0.479 ms | 0.412 ms | flat (run variance) | | store | 0.667 ms | 0.610 ms | flat (run variance) | | warmer | 1.365 ms | 1.315 ms | flat | The win is concentrated in `exec` (-9%), which is exactly where the serial post-`par_iter` validation pass used to live. The merkle / store / warmer deltas are within run-to-run variance; this PR doesn't touch those phases. Compare dashboard: <https://edgl.dev/share/compare-feat-import-bench-bal-tooling@02a5663c-vs-perf-bal-validation-into-par-iter@cc8ba27c.html> ## Test plan - [x] `cargo check --bin ethrex` (default features) - [x] `cargo fmt --all --check` - [x] `cargo clippy --workspace --no-deps -- -D warnings` - [ ] EF blockchain tests (Amsterdam; two-pass parallel runner) - [ ] Hive `bal` group
1 parent d1f8733 commit aa5bd30

2 files changed

Lines changed: 134 additions & 96 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@
2323
### 2026-05-19
2424

2525
- Lazy BAL cursor for per-tx parallel execution [#6669](https://github.com/lambdaclass/ethrex/pull/6669)
26+
- Move per-tx BAL validation into the rayon par_iter closure on the parallel execution path [#6677](https://github.com/lambdaclass/ethrex/pull/6677)
2627

2728
### 2026-05-15
2829

crates/vm/backends/levm/mod.rs

Lines changed: 133 additions & 96 deletions
Original file line numberDiff line numberDiff line change
@@ -1097,17 +1097,26 @@ impl LEVM {
10971097
let arc_idx = Arc::new(validation_index.clone());
10981098

10991099
// 2. Execute all txs in parallel (embarrassingly parallel, BAL-seeded).
1100-
// BAL validation is deferred to after the gas limit check (step 3) so that
1101-
// blocks exceeding gas limit produce GAS_USED_OVERFLOW before BAL mismatch.
1100+
// BAL validation runs INSIDE the par_iter closure (parallel) but its
1101+
// errors are deferred via Option<EvmError> so the post-par_iter
1102+
// gas-limit check still takes priority (GAS_USED_OVERFLOW must beat
1103+
// BAL mismatch on blocks exceeding the gas limit; the BAL is built
1104+
// assuming rejected txs, so miner balance in the BAL won't match
1105+
// execution that ran all txs).
1106+
//
1107+
// The closure also precomputes the small (Vec<(Address, H256)>,
1108+
// Vec<Address>) inputs needed to update the shared
1109+
// `unread_storage_reads` / `unaccessed_pure_accounts` sets, so the
1110+
// serial pass after par_iter is just hash-set ops; current_state
1111+
// and codes never cross the rayon boundary.
11021112
type TxExecResult = (
11031113
usize,
11041114
TxType,
11051115
ExecutionReport,
1106-
FxHashMap<Address, LevmAccount>,
1107-
FxHashMap<H256, ethrex_common::types::Code>,
11081116
FxHashSet<Address>, // accessed_accounts tracker (coarse)
1109-
Vec<Address>, // shadow recorder touched_addresses (EIP-7928 exact)
1110-
Vec<(Address, U256)>, // shadow recorder storage_reads (EIP-7928 exact)
1117+
Vec<(Address, H256)>, // reads_satisfied: (addr, slot) loaded during this tx
1118+
Vec<Address>, // destroyed: accounts selfdestructed during this tx
1119+
Option<EvmError>, // deferred BAL validation error
11111120
);
11121121

11131122
let exec_results: Result<Vec<TxExecResult>, EvmError> = (0..n_txs)
@@ -1173,37 +1182,120 @@ impl LEVM {
11731182
.take()
11741183
.map(|mut r| (r.take_touched_addresses(), r.take_storage_reads()))
11751184
.unwrap_or_default();
1185+
1186+
// Precompute the per-tx inputs the serial pass uses to update
1187+
// the shared unread_storage_reads set. Selfdestruct clears
1188+
// storage from the final state, so destroyed accounts
1189+
// satisfy ALL their BAL storage_reads regardless of which
1190+
// slots remain in `current_state`.
1191+
// Rough avg storage slots per touched account; over-allocation
1192+
// is cheap compared to 2-3 reallocations on the hot path.
1193+
let mut reads_satisfied: Vec<(Address, H256)> =
1194+
Vec::with_capacity(current_state.len() * 4);
1195+
// `destroyed` stays empty on the typical block (selfdestruct
1196+
// is rare post-EIP-6780), so `Vec::new()` (no allocation) is
1197+
// optimal here.
1198+
let mut destroyed: Vec<Address> = Vec::new();
1199+
for (addr, acct) in &current_state {
1200+
if matches!(
1201+
acct.status,
1202+
AccountStatus::Destroyed | AccountStatus::DestroyedModified
1203+
) {
1204+
destroyed.push(*addr);
1205+
} else {
1206+
for key in acct.storage.keys() {
1207+
reads_satisfied.push((*addr, *key));
1208+
}
1209+
}
1210+
}
1211+
1212+
// Run BAL validation inline. Errors are DEFERRED: stored in
1213+
// Option<EvmError> so the serial gas-limit check below still
1214+
// takes priority. Borrow current_state / codes during the
1215+
// validation closure, then drop them before returning so
1216+
// they don't cross the rayon boundary.
1217+
let deferred_bal_err: Option<EvmError> = (|| -> Result<(), EvmError> {
1218+
let bal_idx = u32::try_from(tx_idx + 1).unwrap_or(u32::MAX);
1219+
let seed_idx = u32::try_from(tx_idx).unwrap_or(u32::MAX);
1220+
Self::validate_tx_execution(
1221+
bal_idx,
1222+
seed_idx,
1223+
&current_state,
1224+
&codes,
1225+
bal,
1226+
validation_index,
1227+
&system_seed,
1228+
&store,
1229+
)
1230+
.map_err(|e| {
1231+
EvmError::Custom(format!("BAL validation failed for tx {tx_idx}: {e}"))
1232+
})?;
1233+
1234+
// EIP-7928 (Group B): missing-access detection via shadow recorder.
1235+
for addr in &shadow_touched {
1236+
if !validation_index.addr_to_idx.contains_key(addr) {
1237+
return Err(EvmError::Custom(format!(
1238+
"BAL validation failed for tx {tx_idx}: account {addr:?} was \
1239+
accessed during execution but is missing from BAL"
1240+
)));
1241+
}
1242+
}
1243+
for (addr, slot) in &shadow_reads {
1244+
let Some(&bal_acct_idx) = validation_index.addr_to_idx.get(addr) else {
1245+
// Already caught by the touched-address check above.
1246+
continue;
1247+
};
1248+
let acct = &bal.accounts()[bal_acct_idx];
1249+
let in_changes = acct
1250+
.storage_changes
1251+
.binary_search_by(|sc| sc.slot.cmp(slot))
1252+
.is_ok();
1253+
let in_reads = acct.storage_reads.contains(slot);
1254+
if !in_changes && !in_reads {
1255+
return Err(EvmError::Custom(format!(
1256+
"BAL validation failed for tx {tx_idx}: storage slot {slot} of \
1257+
account {addr:?} was read during execution but is missing from \
1258+
BAL (no storage_changes or storage_reads entry)"
1259+
)));
1260+
}
1261+
}
1262+
Ok(())
1263+
})()
1264+
.err();
1265+
1266+
drop(current_state);
1267+
drop(codes);
1268+
11761269
Ok((
11771270
tx_idx,
11781271
tx.tx_type(),
11791272
report,
1180-
current_state,
1181-
codes,
11821273
tracked,
1183-
shadow_touched,
1184-
shadow_reads,
1274+
reads_satisfied,
1275+
destroyed,
1276+
deferred_bal_err,
11851277
))
11861278
})
11871279
.collect();
11881280

11891281
let mut exec_results = exec_results?;
11901282

1191-
// Sort so gas accounting and validation happen in tx order.
1192-
exec_results.sort_unstable_by_key(|(idx, _, _, _, _, _, _, _)| *idx);
1193-
1194-
// 3. Gas limit check — must happen BEFORE BAL validation so that blocks
1195-
// exceeding the gas limit produce GAS_USED_OVERFLOW instead of a BAL
1196-
// mismatch error (the BAL is built assuming rejected txs, so the miner
1197-
// balance in the BAL won't match execution that ran all txs).
1198-
//
1199-
// EIP-8037 PR #2703: also enforce the per-tx 2D inclusion check
1200-
// against running block totals. A tx whose worst-case regular or
1201-
// state contribution exceeds the remaining budget at its inclusion
1202-
// position invalidates the block with GAS_ALLOWANCE_EXCEEDED.
1283+
// `IndexedParallelIterator` (via `(0..n_txs).into_par_iter()`) preserves
1284+
// source-index order through `.map().collect()`, so `exec_results` is
1285+
// already sorted. The sort is kept as a defensive guard against a future
1286+
// refactor swapping in an unordered iterator; `sort_unstable_by_key` on
1287+
// an already-sorted slice is near-linear via pdqsort, so the cost is
1288+
// negligible.
1289+
exec_results.sort_unstable_by_key(|(idx, _, _, _, _, _, _)| *idx);
1290+
1291+
// 3. Gas limit check — must happen BEFORE BAL validation errors so that
1292+
// blocks exceeding the gas limit produce GAS_USED_OVERFLOW instead of
1293+
// a BAL mismatch error. EIP-8037 PR #2703: also enforce the per-tx
1294+
// 2D inclusion check against running block totals.
12031295
let mut block_regular_gas_used = 0_u64;
12041296
let mut block_state_gas_used = 0_u64;
12051297
let mut tx_gas_breakdowns: Vec<TxGasBreakdown> = Vec::with_capacity(exec_results.len());
1206-
for (tx_idx, _, report, _, _, _, _, _) in &exec_results {
1298+
for (tx_idx, _, report, _, _, _, _) in &exec_results {
12071299
let (tx, _) = txs_with_sender
12081300
.get(*tx_idx)
12091301
.ok_or_else(|| EvmError::Custom(format!("tx index {tx_idx} out of bounds")))?;
@@ -1234,48 +1326,25 @@ impl LEVM {
12341326
)));
12351327
}
12361328

1237-
// 4. Per-tx BAL validation — now safe to run after gas limit is confirmed OK.
1238-
// Also mark off storage_reads that appear in per-tx execution state.
1239-
for (tx_idx, _, _, current_state, codes, tracked_accounts, shadow_touched, shadow_reads) in
1240-
&exec_results
1241-
{
1242-
let bal_idx = u32::try_from(*tx_idx + 1).unwrap_or(u32::MAX);
1243-
let seed_idx = u32::try_from(*tx_idx).unwrap_or(u32::MAX);
1244-
Self::validate_tx_execution(
1245-
bal_idx,
1246-
seed_idx,
1247-
current_state,
1248-
codes,
1249-
bal,
1250-
validation_index,
1251-
&system_seed,
1252-
&store,
1253-
)
1254-
.map_err(|e| EvmError::Custom(format!("BAL validation failed for tx {tx_idx}: {e}")))?;
1255-
1256-
// Mark storage_reads that were actually loaded during this tx.
1257-
// storage_reads slots are NOT in storage_changes (conflict check ensures this),
1258-
// so they're not seeded. If a slot appears in the per-tx state's storage,
1259-
// the tx genuinely read it via SLOAD.
1260-
// Special case: selfdestruct clears storage from the final state, so reads
1261-
// that happened before destruction are no longer visible. For destroyed
1262-
// accounts, mark ALL their BAL storage_reads as satisfied.
1329+
// 4. Surface the first deferred BAL validation error (in tx order) now
1330+
// that the gas-limit check has passed.
1331+
for (_, _, _, _, _, _, deferred) in &mut exec_results {
1332+
if let Some(err) = deferred.take() {
1333+
return Err(err);
1334+
}
1335+
}
1336+
1337+
// 5. Apply per-tx reads_satisfied / destroyed / tracked to the shared
1338+
// sets (cheap hash-set ops; preserves prior semantics).
1339+
for (_, _, _, tracked_accounts, reads_satisfied, destroyed, _) in &exec_results {
12631340
if !unread_storage_reads.is_empty() {
1264-
for (addr, acct) in current_state {
1265-
if matches!(
1266-
acct.status,
1267-
AccountStatus::Destroyed | AccountStatus::DestroyedModified
1268-
) {
1269-
unread_storage_reads.retain(|&(a, _)| a != *addr);
1270-
} else {
1271-
for key in acct.storage.keys() {
1272-
unread_storage_reads.remove(&(*addr, *key));
1273-
}
1274-
}
1341+
for addr in destroyed {
1342+
unread_storage_reads.retain(|&(a, _)| a != *addr);
1343+
}
1344+
for pair in reads_satisfied {
1345+
unread_storage_reads.remove(pair);
12751346
}
12761347
}
1277-
1278-
// Mark pure-access accounts that were accessed during this tx.
12791348
// The coinbase is always accessed during fee finalization (geth's
12801349
// readerTracker records it), even when the miner fee is zero and
12811350
// ethrex skips the load_account call.
@@ -1285,44 +1354,12 @@ impl LEVM {
12851354
unaccessed_pure_accounts.remove(addr);
12861355
}
12871356
}
1288-
1289-
// EIP-7928 (Group B): missing-access detection using the shadow recorder.
1290-
// For each address the per-tx shadow recorder marked as touched, the header
1291-
// BAL must contain an entry for it. For each storage read, the header BAL
1292-
// must carry the slot either in storage_changes or storage_reads.
1293-
for addr in shadow_touched {
1294-
if !validation_index.addr_to_idx.contains_key(addr) {
1295-
return Err(EvmError::Custom(format!(
1296-
"BAL validation failed for tx {tx_idx}: account {addr:?} was \
1297-
accessed during execution but is missing from BAL"
1298-
)));
1299-
}
1300-
}
1301-
for (addr, slot) in shadow_reads {
1302-
let Some(&bal_acct_idx) = validation_index.addr_to_idx.get(addr) else {
1303-
// Already caught by the touched-address check above.
1304-
continue;
1305-
};
1306-
let acct = &bal.accounts()[bal_acct_idx];
1307-
let in_changes = acct
1308-
.storage_changes
1309-
.binary_search_by(|sc| sc.slot.cmp(slot))
1310-
.is_ok();
1311-
let in_reads = acct.storage_reads.contains(slot);
1312-
if !in_changes && !in_reads {
1313-
return Err(EvmError::Custom(format!(
1314-
"BAL validation failed for tx {tx_idx}: storage slot {slot} of \
1315-
account {addr:?} was read during execution but is missing from \
1316-
BAL (no storage_changes or storage_reads entry)"
1317-
)));
1318-
}
1319-
}
13201357
}
13211358

1322-
// 5. Build receipts in tx order.
1359+
// 6. Build receipts in tx order.
13231360
let mut receipts = Vec::with_capacity(n_txs);
13241361
let mut cumulative_gas_used = 0_u64;
1325-
for (_, tx_type, report, _, _, _, _, _) in exec_results {
1362+
for (_, tx_type, report, _, _, _, _) in exec_results {
13261363
cumulative_gas_used += report.gas_spent;
13271364
let receipt = Receipt::new(
13281365
tx_type,

0 commit comments

Comments
 (0)