Skip to content

test: Huge portion of unit circuit tests, fuzz and prover-verifier tests for V2 Airbender#302

Open
Fitznik wants to merge 8 commits into
devfrom
oo_unrolled_tests
Open

test: Huge portion of unit circuit tests, fuzz and prover-verifier tests for V2 Airbender#302
Fitznik wants to merge 8 commits into
devfrom
oo_unrolled_tests

Conversation

@Fitznik

@Fitznik Fitznik commented May 18, 2026

Copy link
Copy Markdown
Contributor

What ❔

Adds a test suite for the unrolled-circuits path (per-circuit, fuzzer, and end-to-end) and fixes the debug witness evaluator so is_satisfied doesn't panic on transient unresolved variables.

Debug witness graph fix ( for making test purposes)

In cs/src/cs/witness_placer/cs_debug_evaluator.rs and cs/src/cs/cs_reference.rs:1042:

  • The evaluator now tracks which variables are actually assigned (assigned: Vec). Before, reading an unassigned variable returned F::ZERO and that zero got fed into downstream resolvers, producing garbage.
  • Resolvers run through evaluate_resolver. If a resolver reads an unassigned variable, the writes it just made get rolled back to unassigned.
  • is_satisfied re-runs the witness graph in a loop until the assigned-count stops growing. Out-of-order resolution graphs can now converge instead of failing on the first pass.
  • The unsatisfied-constraint case went from panic! to println!, and switched from get_value to get_assigned_value. One bad row no longer takes the whole test binary down.

Per-circuit tests (no prover, no verifier)

New prover/src/tests/circuit/ module. Each family's circuit goes through BasicAssembly::is_satisfied directly with fixed inputs. Shared helpers in prover/src/tests/circuit/mod.rs.

  • prover/src/tests/circuit/add_sub.rs — ADD, SUB, ADDI, LUI, AUIPC, ADDMOD/SUBMOD/MULMOD
  • prover/src/tests/circuit/jump_branch.rs — JAL, JALR, BEQ/BNE/BLT/BGE/BLTU/BGEU, SLT/SLTU/SLTI/SLTIU
  • prover/src/tests/circuit/shift_binop.rs — SLL/SRL/SRA, SLLI/SRLI/SRAI, XOR/OR/AND, XORI/ORI/ANDI, CSRRW
  • prover/src/tests/circuit/mul_div.rs — MUL/MULH/MULHU/MULHSU, DIV/DIVU/REM/REMU
  • prover/src/tests/circuit/load_store.rs — LW/SW, aligned + edge addresses
  • prover/src/tests/circuit/load_store_subword.rs — LB/LH/LBU/LHU/SB/SH, all alignments + sign-ext corners
  • prover/src/tests/circuit/decoder_negative.rs — malformed encodings (wrong opcode, bad funct3/funct7, illegal SYSTEM custom-op) against each family decoder, asserts the slot stays empty
  • prover/src/tests/circuit/encoding.rs — bit-packing helpers (encode_r, encode_i, encode_b, …) shared across the other files
  • prover/src/tests/circuit/compliance_vectors.rs — ~20k lines of vectors lifted from the RISC-V compliance suite (riscv-compliance/riscv_ctg)

Fuzzer

prover/src/tests/circuit/fuzz.rs. Seeded StdRng, random operands fed through the same circuit harness. Tests: fuzz_add_sub, fuzz_slt, fuzz_shift_binop, fuzz_mul_div, fuzz_i_type, fuzz_u_type, fuzz_load_store_word, fuzz_load_store_subword, fuzz_csrrw, fuzz_jal, fuzz_jalr, fuzz_branches, and fuzz_negative_all_r_type for malformed encodings. Just random inputs to catch what the fixed vectors miss.

End-to-end prover + verifier

execution_utils/tests/e2e_prove_verify.rs — 47 tests, one per opcode. Each one assembles a tiny RV32IM program with lib_rv32_asm, transpiles + runs it, generates a proof, verifies it, and checks x10..x25. Covers: ADD/SUB/ADDI/LUI/AUIPC, SLT/SLTU/SLTI/SLTIU, all six branches, JAL/JALR, the shift+binop set, the M-extension set, all loads/stores, and CSRRW with the non-determinism CSR.

@Fitznik Fitznik requested a review from popzxc May 18, 2026 23:45
@popzxc popzxc changed the title Huge portion of unit circuit tests, fuzz and prover-verifier tests for V2 Airbender test: Huge portion of unit circuit tests, fuzz and prover-verifier tests for V2 Airbender May 19, 2026
Fitznik and others added 6 commits May 19, 2026 20:24
## What ❔

<!-- What are the changes this PR brings about? -->
<!-- Example: This PR adds a PR template to the repo. -->
<!-- (For bigger PRs adding more context is appreciated) -->

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- The `Why` has to be clear to non-Matter Labs entities running their
own ZK Chain -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Is this a breaking change?
- [ ] Yes
- [ ] No

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted.

@popzxc popzxc left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also CI if failing still -- please format the code

Comment thread .github/workflows/ci.yml
Comment thread cs/src/cs/cs_reference.rs Outdated
Comment on lines +106 to +109
#[test]
#[ignore = "heavy prove+verify; run with `cargo test -- --ignored`"]
#[serial_test::serial]
fn test_prover_pipeline_add() {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. Since these tests mostly check chunks of instructions, maybe we can create test program compiled to RISC-V (with asm blocks)?
We already have a smoke e2e flow for proving and verification, moreover for both 80/100 bit security and cpu/gpu.

So in theory we can replace whatever we run there (dynamic fibonacci AFAIR) with this compiled set of tests, and it will get the best of two worlds: full test matrix & really good coverage.

WDYT?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make sure I understand: you mean take small program chunks already in this file (e2e_prove_verify.rs) and stitch them into a single RISC-V binary, then run that through the smoke matrix (80/100 × CPU/GPU) in place of fibonacci — right? Or you want to work around the compliance vector binary, what is in zksync-airbender/prover/src/tests/circuit/compliance_vectors.rs?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you mean take small program chunks already in this file (e2e_prove_verify.rs) and stitch them into a single RISC-V binary, then run that through the smoke matrix (80/100 × CPU/GPU) in place of fibonacci

This is what I meant, yeah -- I think it will give us better coverage on multiple fronts, while reusing existing infra.

Comment on lines +776 to +779
/// Positive fuzz for ADD/SUB/ADDMOD/SUBMOD/MULMOD (Family 1 R-type).
#[test]
#[ignore = "unbounded fuzz loop; run explicitly with --ignored"]
fn fuzz_add_sub() {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Usually fuzzing is not shaped as a test, it's shaped as a dedicated runnable binary.
Right now we have a bunch of heavy tests that are marked ignored -- if someone will try to run cargo test --workspace --ignored -- --test-threads 1, fuzz tests will hang the process.
So I think a dedicated binary will be better here

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also I'm not sure, but maybe something like AFL will be more efficient, since it's typically structural and kind find regressions better

Comment on lines +268 to +290
let outcome = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
with_csrrw_cs(
rd_reg,
rs1_reg,
UNSUPPORTED_CSR,
0,
// not a non-determinism cycle, so the oracle returns 0 and the rd
// field is irrelevant - what we want to trip is the CSR table
// lookup, not value flow.
0,
|mut cs| {
assert!(
!cs.is_satisfied(),
"CSRRW on an unsupported CSR must violate the SpecialCSRProperties \
lookup constraint"
);
},
)
}));

// The add-time debug constraint checker may reject this intentionally
// invalid case via a panic before is_satisfied() runs.
let _ = outcome;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So basically we are catching any panic and then dropping the result, meaning that we have no assertion in this test, no? Not sure if I understand the idea behind this test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants