Skip to content

Simplify updateBase #3449

Closed
Closed
@arnetheduck

Description

@arnetheduck

updateBase currently receives base update jobs that may involve hundreds of blocks - in order to act cooperatively, it splits up the job into chunks of 4 blocks:

for x in base.everyNthBlock(4):

Instead of chunking inside updateBase, we can perform the chunking outside, thus making updateBase itself synchronous and easier to reason about. Further, we can perform the base update lazily. It would work like this:

  • A single block or an FCU is processed by the queue worker
  • finalization is run eagerly, ie the stale head cleanup runs removing any histories that are not relevant
  • updateBase is scheduled but not run
  • queue worker yields
  • updateBase runs with a maximum step of 4 (achieved by changing the default persist batch size to 4.
  • queue worker yields
  • repeat

In the above flow, if finalized moves by 100 blocks, it will take 25 iterations of the loop to update the base to its new value interleaving the work with block processing - as long as the persist batch size is > 1, base will catch up (we should probably clamp the value to at least 2) - alternatively, updateBase can be run in a loop by the worker until it's done - either strategy is fine though there's some value in interleaving blocks and base updates.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions