Skip to content

Commit e990431

Browse files
TlatoaniHJAlex Zhaobfan05zlangleyjonathanpwang
committed
Feat/Verify Aggregation VM with binary (#89)
* Create stark-vm crate * Add lib.rs, add afs-chips dependency, add CPU chip * Make and use OpCode enum * Refactor CPU tests Refactor CPU tests, add positive tests for CPU, fix relevant bugs in AIR and trace generation * Add program chip Add program chip, move instruction frequency calculation to CPU trace, add positive tests for program * Actually add program chip Include program chip (+ positive tests) in commit (forgot to last time), add negative tests, change cpu::trace::Memory to return 0 for cells that haven't been written to, change CPU's generate_trace to terminate when pc goes out of bounds (previously was only when pc = -1) * feat: imp arithmetic unit, minus tests * Use destructuring in cpu/air.rs * Add negative test for CPU * Reformat cpu and program * chore: fix generate_trace, remove AUChip and replace with AUAir * Add explanations for test programs * chore: resolve conflict * Derive new for instruction Add dependency on derive-new, derive new for Instruction, use that in cpu/tests/mod.rs, add comment explaining program used in test_negative. * chore: change Operation to MemoryAccess * Align MemoryAccess with memory chip Refactor MemoryAccess for eventual merge with memory::MemoryAccess. Analogously change MemoryAccessCols.value to .data for consistency. * fix(au): width misalignment, incorrect trace generation * fix(au bridge): positive tests run * fix(trace): fix bug in AUCols::new, add negative tests * chore: rename AUAir -> FieldArithmeticAir, remove AUChip * Add terminate instruction Add terminate instruction, and use to make sure CPU trace height is power of 2. Also make sure that program trace height is power of 2 by adding jumps to same instruction. Refactor tests using ::from_size to construct Instruction, MemoryAccess, ArithmeticOperation. * chore: un-expand constraints using AB::Expr::one() * Address some zlangley comments * docs(au) * Commit zlangley suggestion Co-authored-by: Zach Langley <[email protected]> * Commit zlangley suggestion Co-authored-by: Zach Langley <[email protected]> * Make inst_width.clone() unnecessary * Address zlangley comment * Rename as_b, as_c to d, e * feat: Memory op interactions * Add negative test for termination check * Add failing test for secret write vulnerability * Add failing test for disable write vulnerability * Add failing test for disable read vulnerability * Check access enabled in AIR * Actually fix access enabled check * Simplify cpu/tests/mod.rs * Don't mock arithmetic chip when testing with is_field_arithmetic = false * chore: add test for invalid read * Attempt to write integration test * Conditionally use arith chip in integration test * Fix integration test wrt range checker * Lint * Use unique memory timestamp instead of clock cycle * chore: rename MemoryAccess clock field to timestamp * feat: change default FRI config to blowup factor 4 (#73) * feat: Offline Checker delete support (#74) * added new columns * squashing inner_join * feat: supporting delete in Offline Checker * removing pub * adding option in controller to provide initial ProverData * keeping page_controller as a directory for simplicity * adding option to generate ProverTraceData * addressing comments * deriving new in PageCols * feat: add engines for BabyBear with blake3 and keccak256 (#78) * chore: auto-implement `Rap` when builder is not `Sync` (#80) * [feat] Query binary (#46) * Add MVP bin * Add MVP bin * Update cache command * Refactor cmd line items * Mock subcommand + logical interface updates * Update interface for reading db * Change interface to read * Change interface to read * Add write command * Add output command for write * Feat/m1 execution air (#66) * feat: sorted_limbs chip checking each limb less than limb_bits bits * wip * feat: completed sorted_limbs chip with tests * wip * feat: SortedLimbsChip with LessThan subchip * feat: less_than subchip refactored * feat: rename SortedLimbsChip to AssertSortedChip and write LessThanChip tests * chore: change name of assert sorted chip * chore: fix names in tests for AssertSortedChip * chore: address comments * chore: cleanup * chore: change MAX from generic to instance field for LessThanChip and AssertSortedChip * wip: added extra bits to middle chip trace * wip * adding connection to IsEqualVec chip * feat: IsLessThanChip to compare two numbers * feat: IsLessThanTuple subchip for different limb_bits * test: added tests for partially and non-allocated pages * test: added negative tests * added constraints * adding comments * feat: IsLessThanTupleChip subchip in AssertSortedChip * renaming * removing TODO comments * fixing clippy * chore: renaming to idx and data * chore: address comments first pass * chore: moving page_controller inside page_read * chore: refactor AssertSorted, IsEqual, IsLessThan, and IsLessThanTuple chips * chore: address comments * chore: eliminate high dim poly from IsLessThanTupleChip * chore: fix tests * chore: address comments for AssertSortedChip * chore: cleanup AssertSorted * chore: cleanup * chore: include roundtrip flatten and from_slice tests * feat: flatten and from_slice for IO and Aux columns * wip: final page chip * remove txt file * feat: final_page_chip * feat: integrating all chips for page_read_write * fixing lib * using field bits() * adding extra communication between checker and final chip * muting clippy * renaming to AIR * adding comments * wip * wip * feat: execution air finished --------- Co-authored-by: bfan <[email protected]> Co-authored-by: Osama Alkhodairy <[email protected]> Co-authored-by: Jonathan Wang <[email protected]> * Add print flag to afi command, update readmes * Update warnings * wip keygen * wip: start prove * Predicate * wip: prove and verify * change branches * Add to_page for Table * Use btreemap * Update functions * Update page to/from * Update tests * Add predicate binary * wip: proof not serializable * chore: patch stuff * add serde bounds * feat: zk commands (not debugged yet) * update mockdb * wip: mold into new format * chore: remove extraneous stuff * [WIP: query-binary] Merge r/w changes (#69) * Add is_less_than_bits chip (#60) * Add is_less_than_bits chip * Address zlangely's comments * Remove chip and use from_bool * Use row_mut * Add comments explaining comparison_check * Add alt version of is_less_than_bits chip * Replace is_less_than_bits with alt version, address comments * Fix lint issues * Feat/is less than bits tuple chip (#63) * Add is_less_than_tuple_bits chip * Incorporate is_less_than_bits feedback into is_less_than_tuple_bits * Add is_less_than_tuple_bits chip * Use match and cmp instead of else if * Adjust for new is_less_than_bits chip * chore: remove range_max field from IsLessThanAir and IsLessThanTupleAir (#65) * chore: derive serialize for proof (#67) * feat: page read write checker (#45) * feat: sorted_limbs chip checking each limb less than limb_bits bits * wip * feat: completed sorted_limbs chip with tests * wip * feat: SortedLimbsChip with LessThan subchip * feat: less_than subchip refactored * feat: rename SortedLimbsChip to AssertSortedChip and write LessThanChip tests * chore: change name of assert sorted chip * chore: fix names in tests for AssertSortedChip * chore: address comments * chore: cleanup * chore: change MAX from generic to instance field for LessThanChip and AssertSortedChip * wip: added extra bits to middle chip trace * wip * adding connection to IsEqualVec chip * feat: IsLessThanChip to compare two numbers * feat: IsLessThanTuple subchip for different limb_bits * test: added tests for partially and non-allocated pages * test: added negative tests * added constraints * adding comments * feat: IsLessThanTupleChip subchip in AssertSortedChip * renaming * removing TODO comments * fixing clippy * chore: renaming to idx and data * chore: address comments first pass * chore: moving page_controller inside page_read * chore: refactor AssertSorted, IsEqual, IsLessThan, and IsLessThanTuple chips * chore: address comments * chore: eliminate high dim poly from IsLessThanTupleChip * chore: fix tests * chore: address comments for AssertSortedChip * chore: cleanup AssertSorted * chore: cleanup * chore: include roundtrip flatten and from_slice tests * feat: flatten and from_slice for IO and Aux columns * wip: final page chip * remove txt file * feat: final_page_chip * feat: integrating all chips for page_read_write * fixing lib * using field bits() * adding extra communication between checker and final chip * muting clippy * renaming to AIR * adding comments * comment fix * optimization to one less bus * comments * moving old page_controller inside page_read * updateing comment * feat: more general FinalPageAir * making IsEqualVecAuxCols clonable * enforcing unallocated rows to be zero * aligning with new less than chip * using Air::eval instead of SubAir::eval * fix comment * removed arguments from load_page for readability * addressing comments and adding common/ directory with Page struct --------- Co-authored-by: bfan <[email protected]> Co-authored-by: Jonathan Wang <[email protected]> * Update to/from page and table tests * Update tests * chore: satisfy clippy (#68) * Fix execution_air test * Update --------- Co-authored-by: TlatoaniHJ <[email protected]> Co-authored-by: bfan05 <[email protected]> Co-authored-by: Jonathan Wang <[email protected]> Co-authored-by: Osama Alkhodairy <[email protected]> Co-authored-by: Zach Langley <[email protected]> * Tmp: remove predicate * Update readme * Update commands for newest rows * feat: add tracing and switch default fri params to log_blowup_factor = 3 * chore: update logging * Fix readme typo * Fix clippy warnings * Update to handle input byte lengths * Remove extraneous code * Revert 9f02efc^..9bad23e * Update logical interface to add byte lengths * Remove generics from logical-interface * merging new Offline Checker changes * Revert "change branches" This reverts commit ec8639f. * Update mock read/write commands * chore: add `mock describe` command * chore: do not overwrite db if exists * chore: remove schema config * chore: switch config back to data_bytes=1024 --------- Co-authored-by: MonkeyKing-1 <[email protected]> Co-authored-by: bfan <[email protected]> Co-authored-by: Osama Alkhodairy <[email protected]> Co-authored-by: Jonathan Wang <[email protected]> Co-authored-by: Zach Langley <[email protected]> Co-authored-by: TlatoaniHJ <[email protected]> * perf: Offline Checker Optimization (#82) * perf: removed 2*data_len columns from the offline checker * renaming to air * Add macros for testing * Additional tests * Remove unused function * Update naming and add comment * Feat/rename page air update from slice (#84) * feat: rename my page air and update from slice * add renamed files * feat: rename final page air * chore: fix lint and rename idx_limb_bits --------- Co-authored-by: Jonathan Wang <[email protected]> * VM binary * Resolve merge conflicts * Add back fibonacci integration test * Redo CLI test * Remove extraneous au directory * Fix lint * Remove keys and proof * Remove commented code * Use FromStr to parse OpCode * VM* -> Vm*, VM -> VirtualMachine * Return Result from VmConfig::read_config_file * Minor zlangley comments * Remove 8 * Use zip for chips and traces Co-authored-by: Zach Langley <[email protected]> * zip_eq -> zip * Use Path and replace unwrap with ? * Removed unnecessary casts * Fix lint * Address jonathan comments, fix proof file path * Delete prove.bin * Use zip_eq and remove unnecessary clones * Use eyre::Result in asm/mod.rs * Fix lint --------- Co-authored-by: Alex Zhao <[email protected]> Co-authored-by: bfan <[email protected]> Co-authored-by: Zach Langley <[email protected]> Co-authored-by: Jonathan Wang <[email protected]> Co-authored-by: Osama Alkhodairy <[email protected]> Co-authored-by: Yu Jiang Tham <[email protected]> Co-authored-by: MonkeyKing-1 <[email protected]> Co-authored-by: Yu Jiang Tham <[email protected]>
1 parent 8d9a470 commit e990431

File tree

20 files changed

+748
-88
lines changed

20 files changed

+748
-88
lines changed

Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ members = [
88
"bin/afs",
99
"logical-interface",
1010
"vm",
11+
"vm/bin",
1112
]
1213
resolver = "2"
1314

config.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,4 +4,4 @@ data_bytes = 1024
44
bits_per_fe = 16
55
height = 1048576
66
mode = "ReadWrite" # options: "ReadOnly", "ReadWrite"
7-
max_rw_ops = 65536
7+
max_rw_ops = 65536

vm/Cargo.toml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,11 @@ itertools = "0.13.0"
2929
getset = "0.1.2"
3030
derive-new = "0.5.1"
3131

32+
serde = "1.0.203"
33+
serde_derive = "1.0.203"
34+
toml = "0.8.14"
35+
enum-utils = "0.1.1"
36+
3237
[dev-dependencies]
3338
p3-uni-stark = { workspace = true }
3439
p3-baby-bear = { workspace = true }

vm/bin/Cargo.toml

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
[package]
2+
name = "cli"
3+
version.workspace = true
4+
authors.workspace = true
5+
edition.workspace = true
6+
description = "CLI for the STARK VM"
7+
8+
[dependencies]
9+
p3-air = { workspace = true }
10+
p3-baby-bear = { workspace = true, optional = true }
11+
p3-commit = { workspace = true }
12+
p3-field = { workspace = true }
13+
p3-keccak = { workspace = true }
14+
p3-keccak-air = { workspace = true }
15+
p3-matrix = { workspace = true }
16+
p3-maybe-rayon = { workspace = true }
17+
p3-symmetric = { workspace = true }
18+
p3-util = { workspace = true }
19+
p3-uni-stark = { workspace = true }
20+
rand = "0.8"
21+
22+
afs-chips = { path = "../../chips" }
23+
afs-stark-backend = { path = "../../stark-backend" }
24+
afs-derive = { path = "../../derive" }
25+
afs-test-utils = { path = "../../test-utils" }
26+
stark-vm = { path = "../" }
27+
parking_lot = "0.12.2"
28+
tracing = "0.1.40"
29+
itertools = "0.13.0"
30+
getset = "0.1.2"
31+
derive-new = "0.5.1"
32+
33+
clap = { version = "4.5.4", features = ["derive"] }
34+
color-eyre = "0.6.3"
35+
bincode = "1.3.3"
36+
logical-interface = { path = "../../logical-interface" }
37+
serde = "1.0.203"
38+
serde_derive = "1.0.203"
39+
toml = "0.8.14"
40+
41+
[dev-dependencies]
42+
p3-uni-stark = { workspace = true }
43+
p3-baby-bear = { workspace = true }
44+
p3-challenger = { workspace = true }
45+
p3-dft = { workspace = true }
46+
p3-fri = { workspace = true }
47+
p3-goldilocks = { workspace = true }
48+
p3-keccak = { workspace = true }
49+
p3-merkle-tree = { workspace = true }
50+
p3-poseidon2 = { workspace = true }
51+
p3-symmetric = { workspace = true }
52+
tracing-subscriber = { version = "0.3.17", features = ["std", "env-filter"] }
53+
tracing-forest = { version = "0.1.6", features = ["ansi", "smallvec"] }
54+
test-case = "3.3.1"
55+
56+
[features]
57+
default = ["test-traits"]
58+
test-traits = ["p3-baby-bear"]

vm/bin/resources/config.toml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
[vm]
2+
field_arithmetic_enabled = true
3+
limb_bits = 8
4+
decomp = 4

vm/bin/resources/fibonacci.asm

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# (-0 indicates unused value)
2+
3+
# initialize register 0 as n = 9
4+
STOREW 9 0 0 0 1
5+
# output will be stored in register 1
6+
7+
# initialize register 2 as 1, will be used as index
8+
STOREW 1 0 2 0 1
9+
10+
11+
# initialize register 3 as 1 (utility)
12+
STOREW 1 0 3 0 1
13+
14+
15+
# initialize [0]_2 as 0 and [1]_2 as 1 (initial values)
16+
STOREW 0 0 0 0 2
17+
18+
19+
20+
STOREW 1 0 1 0 2
21+
22+
23+
# once index = n, terminate (pc += 7)
24+
BEQ 2 0 7 1 1
25+
26+
27+
# increment index
28+
FADD 2 2 3 1 1
29+
30+
31+
# load [index - 2]_2, [index - 1]_2 into registers 4, 5
32+
LOADW 4 -2 2 1 2
33+
34+
35+
36+
LOADW 5 -1 2 1 2
37+
38+
39+
# calculate new value in register 6
40+
FADD 6 4 5 1 1
41+
42+
43+
# store new value in [index]_2
44+
STOREW 6 0 2 1 2
45+
46+
47+
# jump to beginning of loop
48+
JAL 7 -6 -0 1 -0
49+
50+
51+
# terminate
52+
TERMINATE -0 -0 -0 -0 -0

vm/bin/src/asm/mod.rs

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
use std::fs::File;
2+
3+
use std::io::{BufRead, BufReader};
4+
use std::path::Path;
5+
6+
use color_eyre::eyre::{Report, Result};
7+
use p3_field::PrimeField64;
8+
use stark_vm::cpu::trace::Instruction;
9+
10+
pub fn parse_asm_file<F: PrimeField64>(path: &Path) -> Result<Vec<Instruction<F>>> {
11+
let file = File::open(path)?;
12+
let reader = BufReader::new(file);
13+
14+
let mut result = vec![];
15+
for line in reader.lines() {
16+
if let Some(instruction) = instruction_from_line::<F>(&line?)? {
17+
result.push(instruction);
18+
}
19+
}
20+
21+
Ok(result)
22+
}
23+
24+
fn instruction_from_line<F: PrimeField64>(line: &str) -> Result<Option<Instruction<F>>> {
25+
let parts: Vec<&str> = line.split_whitespace().collect();
26+
if parts.is_empty() {
27+
return Ok(None);
28+
}
29+
if parts[0].starts_with('#') {
30+
return Ok(None);
31+
}
32+
if parts.len() != 6 {
33+
return Err(Report::msg(
34+
"Instruction should have opcode followed by 5 arguments",
35+
));
36+
}
37+
let opcode = parts[0]
38+
.parse()
39+
.map_err(|_| Report::msg("Invalid opcode"))?;
40+
let mut ints = vec![];
41+
for part in parts.iter().skip(1) {
42+
ints.push(
43+
part.parse::<isize>()
44+
.map_err(|_| Report::msg("Opcode argument should be int"))?,
45+
);
46+
}
47+
48+
Ok(Some(Instruction::from_isize(
49+
opcode, ints[0], ints[1], ints[2], ints[3], ints[4],
50+
)))
51+
}

vm/bin/src/cli/mod.rs

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
use crate::commands::keygen::KeygenCommand;
2+
use crate::commands::{keygen, prove, verify};
3+
use clap::Parser;
4+
use clap::Subcommand;
5+
use stark_vm::vm::config::VmConfig;
6+
7+
#[derive(Debug, Parser)]
8+
#[command(author, version, about = "VM STARK CLI")]
9+
#[command(propagate_version = true)]
10+
pub struct Cli {
11+
#[command(subcommand)]
12+
pub command: CliCommand,
13+
}
14+
15+
#[derive(Debug, Subcommand)]
16+
pub enum CliCommand {
17+
#[command(name = "keygen", about = "Generate partial proving and verifying keys")]
18+
/// Generate partial proving and verifying keys
19+
Keygen(keygen::KeygenCommand),
20+
21+
#[command(name = "prove", about = "Generates a multi-STARK proof")]
22+
/// Generates a multi-STARK proof
23+
Prove(prove::ProveCommand),
24+
25+
#[command(name = "verify", about = "Verifies a multi-STARK proof")]
26+
/// Verifies a multi-STARK proof
27+
Verify(verify::VerifyCommand),
28+
}
29+
30+
impl Cli {
31+
pub fn run(config: VmConfig) -> Self {
32+
let cli = Self::parse();
33+
match &cli.command {
34+
CliCommand::Keygen(keygen) => {
35+
let cmd = KeygenCommand {
36+
output_folder: keygen.output_folder.clone(),
37+
asm_file_path: keygen.asm_file_path.clone(),
38+
};
39+
cmd.execute(config).unwrap();
40+
}
41+
CliCommand::Prove(prove) => {
42+
prove.execute(config).unwrap();
43+
}
44+
CliCommand::Verify(verify) => {
45+
verify.execute(config).unwrap();
46+
}
47+
}
48+
cli
49+
}
50+
}

vm/bin/src/commands/keygen.rs

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
use std::{
2+
fs::{self},
3+
path::Path,
4+
time::Instant,
5+
};
6+
7+
use afs_test_utils::{
8+
config::{self},
9+
engine::StarkEngine,
10+
};
11+
use clap::Parser;
12+
use color_eyre::eyre::Result;
13+
use itertools::Itertools;
14+
use p3_matrix::Matrix;
15+
use stark_vm::vm::{config::VmConfig, VirtualMachine};
16+
17+
use crate::asm::parse_asm_file;
18+
19+
use super::write_bytes;
20+
21+
/// `afs keygen` command
22+
/// Uses information from config.toml to generate partial proving and verifying keys and
23+
/// saves them to the specified `output-folder` as *.partial.pk and *.partial.vk.
24+
#[derive(Debug, Parser)]
25+
pub struct KeygenCommand {
26+
#[arg(
27+
long = "asm-file",
28+
short = 'f',
29+
help = "The .asm file input",
30+
required = true
31+
)]
32+
pub asm_file_path: String,
33+
#[arg(
34+
long = "output-folder",
35+
short = 'o',
36+
help = "The folder to output the keys to",
37+
required = false,
38+
default_value = "keys"
39+
)]
40+
pub output_folder: String,
41+
}
42+
43+
impl KeygenCommand {
44+
/// Execute the `keygen` command
45+
pub fn execute(self, config: VmConfig) -> Result<()> {
46+
let start = Instant::now();
47+
self.execute_helper(config)?;
48+
let duration = start.elapsed();
49+
println!("Generated keys in {:?}", duration);
50+
Ok(())
51+
}
52+
53+
fn execute_helper(self, config: VmConfig) -> Result<()> {
54+
let instructions = parse_asm_file(Path::new(&self.asm_file_path.clone()))?;
55+
let vm = VirtualMachine::new(config, instructions);
56+
let engine = config::baby_bear_poseidon2::default_engine(vm.max_log_degree());
57+
let mut keygen_builder = engine.keygen_builder();
58+
59+
let chips = vm.chips();
60+
let traces = vm.traces();
61+
62+
for (chip, trace) in chips.into_iter().zip_eq(traces) {
63+
keygen_builder.add_air(chip, trace.height(), 0);
64+
}
65+
66+
let partial_pk = keygen_builder.generate_partial_pk();
67+
let partial_vk = partial_pk.partial_vk();
68+
let encoded_pk: Vec<u8> = bincode::serialize(&partial_pk)?;
69+
let encoded_vk: Vec<u8> = bincode::serialize(&partial_vk)?;
70+
fs::create_dir_all(Path::new(&self.output_folder))?;
71+
let pk_path = Path::new(&self.output_folder).join("partial.pk");
72+
let vk_path = Path::new(&self.output_folder).join("partial.vk");
73+
fs::create_dir_all(self.output_folder)?;
74+
write_bytes(&encoded_pk, &pk_path)?;
75+
write_bytes(&encoded_vk, &vk_path)?;
76+
Ok(())
77+
}
78+
}

vm/bin/src/commands/mod.rs

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
use color_eyre::eyre::Result;
2+
use std::{
3+
fs::File,
4+
io::{BufReader, BufWriter, Read, Write},
5+
path::Path,
6+
};
7+
8+
pub mod keygen;
9+
pub mod prove;
10+
pub mod verify;
11+
12+
fn read_from_path(path: &Path) -> Result<Vec<u8>> {
13+
let file = File::open(path)?;
14+
let mut reader = BufReader::new(file);
15+
let mut buf = vec![];
16+
reader.read_to_end(&mut buf)?;
17+
Ok(buf)
18+
}
19+
20+
fn write_bytes(bytes: &[u8], path: &Path) -> Result<()> {
21+
let file = File::create(path)?;
22+
let mut writer = BufWriter::new(file);
23+
writer.write_all(bytes)?;
24+
Ok(())
25+
}

0 commit comments

Comments
 (0)