Skip to content

Bound min size of dynamic processor queues #6466

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Oct 10, 2024

Conversation

dapplion
Copy link
Collaborator

@dapplion dapplion commented Oct 5, 2024

Issue Addressed

A discord user notes that in networks of < SLOTS_PER_EPOCH the beacon node did not process attestations.

It's due to integer division here

// Capacity for a full slot's worth of attestations if subscribed to all subnets
attestation_queue: active_validator_count / slots_per_epoch,
// Capacity for a full slot's worth of attestations if subscribed to all subnets
unknown_block_attestation_queue: active_validator_count / slots_per_epoch,

Proposed Changes

Min against a const

Closes #6463

@dapplion dapplion added the ready-for-review The code is ready for review label Oct 5, 2024
Copy link
Member

@michaelsproul michaelsproul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this could easily regress with the beacon processor refactors. Wanna add a test?

Copy link
Member

@chong-he chong-he left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we should use std::cmp::max here? Happy to stand corrected.

@@ -155,9 +160,15 @@ impl BeaconProcessorQueueLengths {
aggregate_queue: 4096,
unknown_block_aggregate_queue: 1024,
// Capacity for a full slot's worth of attestations if subscribed to all subnets
attestation_queue: active_validator_count / slots_per_epoch,
attestation_queue: std::cmp::min(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may be wrong here, I don't understand why it is std::cmp::min. From my understanding, it takes the minimum value of the two in comparison. I thought it should be std::cmp::max.

Because if it is min, then when active_validator_count / slots_per_epoch is, say 16, then the queue would be still 0? But if we use max, then for small validator count, it will return MIN_QUEUE_LEN which is 128.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you're correct!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ups, good catch

// Capacity for a full slot's worth of attestations if subscribed to all subnets
unknown_block_attestation_queue: active_validator_count / slots_per_epoch,
unknown_block_attestation_queue: std::cmp::min(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above

@michaelsproul michaelsproul added v6.0.0 New major release for hierarchical state diffs waiting-on-author The reviewer has suggested changes and awaits thier implementation. and removed ready-for-review The code is ready for review labels Oct 7, 2024
@dapplion
Copy link
Collaborator Author

dapplion commented Oct 8, 2024

I feel like this could easily regress with the beacon processor refactors. Wanna add a test?

To not conflict with the refactor, what API should the test use? i.e.

let manager = spawn_manager(event_rx, work_journal_tx, BeaconProcessorQueueLengths::empty());
event_rx.send(Work::GossipAttestation {});
// assert work_journal_tx recv GossipAttestation work

@michaelsproul
Copy link
Member

I think it doesn't matter too much if the test conflicts, as long as it exists and can be translated over.

I would even just test initialising the Lengths struct in isolation so that it's minimally entangled with the rest of the beacon processor

Copy link
Member

@michaelsproul michaelsproul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed a test. This is ready to go.

@michaelsproul michaelsproul added ready-for-merge This PR is ready to merge. and removed waiting-on-author The reviewer has suggested changes and awaits thier implementation. labels Oct 10, 2024
@michaelsproul
Copy link
Member

@mergify queue

Copy link

mergify bot commented Oct 10, 2024

queue

✅ The pull request has been merged automatically

The pull request has been merged automatically at 244a460

mergify bot added a commit that referenced this pull request Oct 10, 2024
@mergify mergify bot merged commit 244a460 into sigp:unstable Oct 10, 2024
29 checks passed
@dapplion dapplion deleted the min-queue-len branch October 10, 2024 13:49
chong-he pushed a commit to chong-he/lighthouse that referenced this pull request Nov 26, 2024
* Bound min size of dynamic processor queues

* Use max

* Add test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready-for-merge This PR is ready to merge. v6.0.0 New major release for hierarchical state diffs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants