dd: Batch propose for building in chunks #74

spalladino · 2025-07-15T21:06:34Z

No description provided.

joeandrews · 2025-07-16T07:47:54Z

docs/building-in-chunks/batch-propose.md

+
+If we land at lower TPS or less bytes per tx, this can lead to worse UX overall, since a proposer would be incentivized to only post a single block within their checkpoint, effectively increasing block times to 72s. If we assume 300 bytes per tx, which is the current lower bound for a transfer with out-of-band messaging (which should be the preference for app developers who want to reduce tx costs) at 6 TPS, a single blob takes the full 72s to be filled.
+
+On the other hand, if we consider 2400 bytes per tx effect, which is the size of a transfer with fpc support with all notes posted on-chain, then each blob can carry about 50 txs only. Assuming 10 TPS, this means we fill 2 blobs per L1 slot, so checkpoints cannot be longer than 3 L1 slots (36s).


Why is this? We have up to 9 blobs per block currently and this should increase by end of year to 12 - 18 when peerDas goes in.

We have up to 9 blobs per block currently

Using 9 blobs leads to increased blob gas price pretty quickly, so I don't want to assume we can use them consistently.

this should increase by end of year

If we think we can rely on this, then I can update the numbers!

joeandrews · 2025-07-16T07:48:13Z

docs/building-in-chunks/batch-propose.md

+
+Since we are not making changes to circuits, each block still takes up at least one blob. If we target 6 blobs per tx, which may reduce our chances at L1 inclusion, we can squeeze no more than 6 slots per checkpoint. Assuming 1200 bytes per tx effect, each 128kb blob can carry about 100 txs. Assuming 10 TPS, we require 12kb per second for DA, which is 1 blob every 10.6 seconds, roughly 1 blob per L1 slot. This means the duration of a checkpoint cannot be more than 6 L1 slots (72s), and the duration of each L2 slot should be equivalent to one L1 slot (12s).
+
+Note that the above assumes 10TPS. If we go below this, then each block would not fully utilize its blob, leading to wasted blobspace and increased DA cost per tx. We should expect proposers to **not** produce a block for a slot if they have not filled the block, and instead accumulate txs into the following block in the slot within their checkpoint. This leads to less DA cost, which is borne by the proposer, but worse UX, since users now have to wait longer for L2 inclusion.


Can we not incentivise production via the block reward?

We can, but doesn't that mean more inflation?

Only if we incentivise at a rate where the gas / second is above the current.

joeandrews · 2025-07-16T07:49:50Z

docs/building-in-chunks/batch-propose.md

+
+If we land at lower TPS or less bytes per tx, this can lead to worse UX overall, since a proposer would be incentivized to only post a single block within their checkpoint, effectively increasing block times to 72s. If we assume 300 bytes per tx, which is the current lower bound for a transfer with out-of-band messaging (which should be the preference for app developers who want to reduce tx costs) at 6 TPS, a single blob takes the full 72s to be filled.
+
+On the other hand, if we consider 2400 bytes per tx effect, which is the size of a transfer with fpc support with all notes posted on-chain, then each blob can carry about 50 txs only. Assuming 10 TPS, this means we fill 2 blobs per L1 slot, so checkpoints cannot be longer than 3 L1 slots (36s).


Why is this so high? The private refund flow is not needed here. This should be max 3 notes, and one is transient I beleive.

See @iAmMichaelConnor's model here, txs seem to oscillate between 600 and 4500 bytes.

joeandrews · 2025-07-16T07:59:37Z

docs/building-in-chunks/batch-propose.md

+
+Note that the above assumes 10TPS. If we go below this, then each block would not fully utilize its blob, leading to wasted blobspace and increased DA cost per tx. We should expect proposers to **not** produce a block for a slot if they have not filled the block, and instead accumulate txs into the following block in the slot within their checkpoint. This leads to less DA cost, which is borne by the proposer, but worse UX, since users now have to wait longer for L2 inclusion.
+
+If we land at lower TPS or less bytes per tx, this can lead to worse UX overall, since a proposer would be incentivized to only post a single block within their checkpoint, effectively increasing block times to 72s. If we assume 300 bytes per tx, which is the current lower bound for a transfer with out-of-band messaging (which should be the preference for app developers who want to reduce tx costs) at 6 TPS, a single blob takes the full 72s to be filled.


Even if we are at 300 bytes per TX and 48 second checkpoint time and < 5 TPS, over 96 secods, our costs for the pending chain are 2 * (277,000 + 2*(122,000 + 131,000)) = 1.56M.

Assuming a 24s block time (small UX win), we would post a total of 2 blobs, 1 per block.

Comparing this to the current system, we would have costs of 1.2M gas for the pending chain and three blobs equaling the same gas cost, but better UX.

I also want to understand the costs per block more in the table as these seem very high.

I also want to understand the costs per block more in the table as these seem very high.

These are sourced from a doc where @just-mitch was experimenting with the code. It's possible there is room for further optimization, sure.

joeandrews · 2025-07-16T08:02:19Z

docs/building-in-chunks/batch-propose.md

+| Store block log                | x         |           | 30k      |
+| Consume inbox                  | x         |           | 10k      |
+| Push messages to outbox        | x         |           | 9k       |
+| Emit log                       | x         |           | 3k       |


Why is this needed?

We use logs for discovering new blocks being posted

joeandrews · 2025-07-16T08:02:55Z

docs/building-in-chunks/batch-propose.md

+| Validate block header          | x         |           | 20k      |
+| Post and validate attestations |           | x         | 200k     |
+| Update pending chain tip       |           | x         | 3k       |
+| Store archive root             | x         |           | 22k      |


Is this not a warm slot?

Not the archive root. We keep the entire history of archive roots in storage (as opposed to other fields, which are kept in circular storage, that's why they are cheaper). This allows any client to easily detect if their current state is valid or not with a single CALL. We could probably replace this with circular storage at the expense of more complexity in the archiver.

Also, a nit: cold/warm refers to whether the storage slot has been accessed at some point throughout the tx. I understand that here you meant whether the slot was zero initially or not.

joeandrews · 2025-07-16T08:13:42Z

docs/building-in-chunks/batch-propose.md

+| Update L1 gas fee oracle       |           | x         | 8k       |
+| Validate blob commitments      | x         |           | 8k       |
+| Compute base fee               |           | x         | 22k      |
+| Validate block header          | x         |           | 20k      |


Is this not also warm?

I understand we do multiple reads during this validation, which would account for this cost. I'd need to go deeper in gas analysis, but it's possible this number could go slightly down.

joeandrews · 2025-07-16T08:17:07Z

docs/building-in-chunks/batch-propose.md

+
+- With 72s checkpoints and 12s blocks (best for UX), total gas cost per checkpoint is 1M, or 14k gas per second (worse than status quo)
+- Decreasing block frequency to once per 36s (same UX as today), total gas cost per checkpoint is 521k gas, or 7k gas per second (33% better than status quo)
+- Halving checkpoints to 36s to support 2400-byte txs, with one block every 6s (best for UX), total gas cost is 1M, or 28k gas per second (much worse than status quo)


I don't understand this. Why would checkpoints be every 36s here, and why is the data so high, what needs 4 notes and nullifiers with data on chain? At most a FPC for a private transaction needs 3 notes, and one is transient. A refund is not needed for any private transaction as the cost of the refund will be > than the refund.

Why would checkpoints be every 36s here

This is assuming we consume 24kb/s in blob data and there are only 6 usable blobs per L1 blocks.

why is the data so high,

Can I defer to you @iAmMichaelConnor on this?

LHerskind

Thanks for writing it here 👍 Not sure that I think it would take 8 weeks, but the benefit is too small anyway it seems from targets.

LHerskind · 2025-07-16T12:58:02Z

docs/building-in-chunks/batch-propose.md

+| Emit log                       | x         |           | 3k       |
+| Unaccounted for                |           | x         | 20k      |
+
+The total gas cost from `propose` today is then 400k gas every 36s, or 11k gas per second. Under this new model, the cost per block is `20 + 8 + 20 + 22 + 30 + 10 + 9 + 3` or 122k gas, and the cost per batch is `21 + 3 + 8 + 22 + 200 + 3 + 20` or 277k gas. Assuming 72s checkpoints:


Had some quick mafs I don't think I posted in the original comment. As you always say right after this, mainly a cost reduction compared to status que UX, and depending on what other changes is crammed in savings can go down as %.

// New (48 slots of 24 s) full_epoch_cost = 6 * (8 * 200K + 50K) + 4M 14,100,000 gas (1,223 gas per L2 tx) // New (32 slots of 36 s) full_epoch_cost = 4 * (8 * 200K + 50K) + 4M 10,600,000 gas (920 gas per L2 tx) // Old (32 slots of 36 s) full_epoch_cost = 32 * 250K + 4M 12,000,000 gas (1,041 gas per L2 tx)

dd: Batch propose for building in chunks

9631092

spalladino force-pushed the palla/batch-block-propose branch from 590402d to 9631092 Compare July 15, 2025 21:10

joeandrews reviewed Jul 16, 2025

View reviewed changes

LHerskind reviewed Jul 16, 2025

View reviewed changes


		If we land at lower TPS or less bytes per tx, this can lead to worse UX overall, since a proposer would be incentivized to only post a single block within their checkpoint, effectively increasing block times to 72s. If we assume 300 bytes per tx, which is the current lower bound for a transfer with out-of-band messaging (which should be the preference for app developers who want to reduce tx costs) at 6 TPS, a single blob takes the full 72s to be filled.

		On the other hand, if we consider 2400 bytes per tx effect, which is the size of a transfer with fpc support with all notes posted on-chain, then each blob can carry about 50 txs only. Assuming 10 TPS, this means we fill 2 blobs per L1 slot, so checkpoints cannot be longer than 3 L1 slots (36s).


		Since we are not making changes to circuits, each block still takes up at least one blob. If we target 6 blobs per tx, which may reduce our chances at L1 inclusion, we can squeeze no more than 6 slots per checkpoint. Assuming 1200 bytes per tx effect, each 128kb blob can carry about 100 txs. Assuming 10 TPS, we require 12kb per second for DA, which is 1 blob every 10.6 seconds, roughly 1 blob per L1 slot. This means the duration of a checkpoint cannot be more than 6 L1 slots (72s), and the duration of each L2 slot should be equivalent to one L1 slot (12s).

		Note that the above assumes 10TPS. If we go below this, then each block would not fully utilize its blob, leading to wasted blobspace and increased DA cost per tx. We should expect proposers to not produce a block for a slot if they have not filled the block, and instead accumulate txs into the following block in the slot within their checkpoint. This leads to less DA cost, which is borne by the proposer, but worse UX, since users now have to wait longer for L2 inclusion.

dd: Batch propose for building in chunks #74

Are you sure you want to change the base?

dd: Batch propose for building in chunks #74

Uh oh!

Conversation

spalladino commented Jul 15, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joeandrews Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joeandrews Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joeandrews Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LHerskind left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

joeandrews Jul 16, 2025 •

edited

Loading

joeandrews Jul 16, 2025 •

edited

Loading

joeandrews Jul 16, 2025 •

edited

Loading