Repository navigation
[Fix] simulator: Keep CPU bootstrap and allocator compatibility generic - #2
Closed
Jeff010101 wants to merge 1 commit into
Closed
Jeff010101 wants to merge 1 commit into
Jeff010101 wants to merge 1 commit into
Conversation
This was referenced Nov 8, 2026
Jeff010101
force-pushed
the
codex/review-layer-p1
branch
from
November 9, 2026 02:04
3b806b4 to
b3a27e1
Compare
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Jeff010101
force-pushed
the
codex/review-layer-p1
branch
from
November 9, 2026 03:56
b3a27e1 to
1d00322
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Superseded by the upstream Draft PR: sgl-project#49240.
This fork-only review is closed to keep one active PR for the feature. Its discussion and source branches are retained. Continue review and dependency tracking in the upstream PR above.
Head:
codex/review-layer-p1at1d003221f2ba9be949de2ddd95bf6c9145697e66.Scoped base:
codex/review-base-mainat4003bf1077debcca3fccfa6e0d7b56ed584aa727; 1 commit, 13 files. Fixed comparison.Motivation
The CPU simulator must import SGLang and initialize its native allocator without probing a physical accelerator or importing an optional AITER kernel before it is used. This PR fixes bootstrap and allocator compatibility independently of any optional cost provider. It does not claim that every unpatched CPU installation fails.
Delivered capability after prerequisites: the existing simulator can bootstrap CPU parent and spawned workers, preserve native quantization configuration validation, and translate an explicit logical token capacity through the native pool allocator.
Dependencies: the current SGLang base and the simulator's existing installation requirements. No SimOpus SDK, native staged adapter, closed-loop transport, or later resource-budget PR is required. This fork PR is a scoped review layer against the pinned upstream baseline.
Modifications
SGLANG_SIMULATOR_BOOTSTRAP=1andSGLANG_USE_CPU_ENGINE=1are set by the simulator entry point. Other flag combinations leave these wrappers inactive.For a basic CPU run, use the documented simulator entry point, an existing replay configuration, and an explicit
--max-total-tokenscapacity. Later PRs add other policies; this layer does not claim a new latency model.The isolated Quark utility fix is submitted upstream in #49231, with the real CPU-container reproducer in #49230. This review layer retains the same bytes until that prerequisite lands and is rebased.
Scope boundary
P1 adds no SimOpus imports, resource policies, exact-capacity option or SDK-specific rejection rules. The exact SDK capacity helper and
test_cpu_pool_capacity.pybelong to P4. P1 retains onlytest_cpu_pool_bounds.pyfor the generic existing allocator path.Accuracy Tests
The unchanged Quark fix passes all 13 registered CPU tests against core
2ad35d45, including deferred import, preserved dependency errors, argument/result delegation and one-time successful binding. Five fresh-process controls cover the four CPU flag combinations and repeat activation. Native-capacity and allocator compatibility controls cover bounded failure and import identity. The new generic capacity file passes all five cases on this head and on the assembled stack; stacking P4 passes those five plus its nine exact-capacity cases. A pure CPU import baseline also passes, so the reproducible Quark read-only-HOME failure is kept as a narrower optional-dependency issue. Whole-stack differential evidence is maintained in the native integration PR; it is not a separate full-cohort accuracy claim for this layer.Speed Tests and Profiling
No physical GPU throughput result is claimed. The cached loader avoids a repeated Python import on successful quantization calls. CPU simulator checks validate functionality rather than inference performance.
Checklist
Review and Merge Process
Fork merge changes only the fork; upstream landing requires dependency rebase, separate submission and normal required reviews/CI.
Review this foundation separately from simulator cost and transport features. Follow CODEOWNERS, required CI and branch protection; this review-layer draft does not waive them.
CI States
Latest PR Test (Base): ❌ Run #42559064312
Latest PR Test (Extra): ❌ Run #42559064428