scheduler: exit early on count=0 and filter out server-terminal #26292
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
While working on reconciler behaviors around stopped allocations, I found it extremely hard to follow the intent of the disconnected client tests because many of the fields in the table-driven test are switches for more complex behavior or just tersely named. Attempt to make this a little more legible by moving some branches directly into fields, renaming some fields, and flattening out some branching.
When a task group is removed from a jobspec, the reconciler stops all allocations and immediately returns from `computeGroup`. We can do the same for when the group has been scaled-to-zero, but doing so runs into an inconsistency in the way that server-terminal allocations are handled. Prior to this change server-terminal allocations fall through `computeGroup` without being marked as `ignore`, unless they are terminal canaries, in which case they are marked `stop` (but this is a no-op). This inconsistency causes a tiny amount of extra Raft traffic, but more importantly makes it more difficult to make test assertions for `stop` vs `ignore` vs fallthrough. Remove this inconsistency by filtering out server-terminal allocations early in `computeGroup`. This brings the cluster reconciler's behavior closer to the node reconciler's behavior, except that the node reconciler discards _all_ terminal allocations because it doesn't support rescheduling.
|
The |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When a task group is removed from a jobspec, the reconciler stops all allocations and immediately returns from
computeGroup. We can do the same for when the group has been scaled-to-zero, but doing so runs into an inconsistency in the way that server-terminal allocations are handled.Prior to this change server-terminal allocations fall through
computeGroupwithout being marked asignore, unless they are terminal canaries, in which case they are markedstop(but this is a no-op). This inconsistency causes a tiny amount of extraPlan.Submit/Raft traffic, but more importantly makes it more difficult to make test assertions forstopvsignorevs fallthrough. Remove this inconsistency by filtering out server-terminal allocations early incomputeGroup.This brings the cluster reconciler's behavior closer to the node reconciler's behavior, except that the node reconciler discards all terminal allocations because it doesn't support rescheduling.
This changeset required adjustments to two tests, but the tests themselves were a bit of a mess:
There's a good bit here, but I've broken this PR up into commits that should stand alone.
Ref: https://hashicorp.atlassian.net/browse/NMD-819