Repository navigation
feat(sc): add elastic scheduler design, scheduling policy and replay tool - #5107
Draft
LoganVegnaSHOP wants to merge 4 commits into
Draft
LoganVegnaSHOP wants to merge 4 commits into
LoganVegnaSHOP wants to merge 4 commits into
Conversation
Add a design doc for elastic cells that switch between rollout and training as the rollout tail drains (QwenGyre, section 4), and the first building block: a pure scheduling policy implementing the launch rule, the waterlevel and supply-hold rule for moving the next cell to training, and core retention at burst end. The policy has no Ray, GPU or asyncio dependency and nothing in the SingleController calls it yet, so there is no behavior change. Signed-off-by: Logan Vegna <logan.vegna@shopify.com>
…ndary The policy counts executions, as the paper does, while the SingleController admits, launches and commits whole prompt groups. Give the policy the group size and steps per burst instead of a precomputed burst dispatch, make ElasticSchedulerPolicy.executions the single groups-to-executions conversion, and add group-unit counters and launchable_groups, which never splits a group across a launch decision. Signed-off-by: Logan Vegna <logan.vegna@shopify.com>
Add tools/elastic_scheduler_replay.py, the offline go/no-go measurement for elastic cells. It reads the per-group rl.sc.generate_and_push spans a SingleController run records (OTLP/JSON from a collector file exporter, or the console exporter's output), replays the durations through ElasticScheduler, a fixed rollout/training split (every split tried unless one is given) and colocate on the same cells, and reports projected step time, idle cell-seconds and switching overhead for each. Signed-off-by: Logan Vegna <logan.vegna@shopify.com>
…r risks Document the offline replay as the go/no-go measurement with its trace format, command and assumptions, plus a synthetic example. Explain the group-to-execution boundary. Replace the open communicator-memory risk with an estimate from NCCL default buffer sizes and five concrete alternatives, recommending shared-resource splits. Spell out what the router and the colocated SingleController checks already allow for MInf cells and what each phase must change. Signed-off-by: Logan Vegna <logan.vegna@shopify.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do ?
Adds a design doc, a pure scheduling policy and an offline replay tool for elastic train/rollout cells on the SingleController. The training path is unchanged: nothing in the SC calls the policy yet, and there is no config.
Motivation
In agentic RL, rollout durations have a long tail. With a fixed split (async), rollout GPUs sit idle while the tail drains, and training GPUs wait for a batch. Colocate makes every GPU wait for the slowest rollout. QwenGyre (arXiv 2609.33848, §4) groups GPUs into cells. Each cell moves to training as soon as the remaining rollout work fits without it, and later cells join the batch that is already training.
Reported results:
What it does
docs/design-docs/elastic-scheduler.md: the full design and an 8-phase plan. It covers rerouting, offload/onload, the satellite weight pull, joining a running step, gradient reduction with changing membership (with an NCCL memory estimate and alternatives), staleness, checkpointing, failures, and where it doesn't help.nemo_rl/algorithms/single_controller_utils/elastic_scheduler.py: a pure policy in the style offleet_health.py. It implements the paper's eqs. 3–6: the capacity-gated launch rule, the waterlevel plus held-supply rule for moving cells one at a time with the core first, and core retention at burst end. The SC calls it in prompt groups;ElasticSchedulerPolicy.executions(groups)is the one place groups become executions, and launches never split a group.tools/elastic_scheduler_replay.py: the policy's first consumer and the go/no-go measurement. It replays therl.sc.generate_and_pushspan durations a real run recorded (OTLP/JSON or console exporter) through elastic cells, the best fixed split and colocate on the same cells. It reports step time, idle cell-time and switching overhead for each.Usage
uv run python tools/elastic_scheduler_replay.py --trace spans.jsonl \ --num-cells 8 --cell-capacity 144 --group-size 16 --batch-groups 64 \ --train-step-s 3600 --num-steps 12Planned opt-in config (Phase 3, not in this PR):
async_rl.elastic_scheduler.{enabled: false, num_cells, min_supply_fraction, supply_hold_s}.Testing
--nemo-gym-onlylanes), compared with cleanmainon the same machine: no new failures. The failures that remain also happen onmainhere (no triton/NCCL/GPU, missing Gym extras, Ray resource limits).Additional Information
splitShare=1) and notes that this would change how the existing communicator abort code behaves.🤖 Generated with Claude Code