Skip to content

[Fix] simulator: Keep CPU bootstrap and allocator compatibility generic - #2

Closed
Jeff010101 wants to merge 1 commit into
codex/review-base-mainfrom
codex/review-layer-p1
Closed

Jeff010101 wants to merge 1 commit into
codex/review-base-mainfrom
codex/review-layer-p1

Conversation

@Jeff010101

@Jeff010101 Jeff010101 commented Nov 8, 2026 •

Copy link
Copy Markdown
Owner

Superseded by the upstream Draft PR: sgl-project#49240.

This fork-only review is closed to keep one active PR for the feature. Its discussion and source branches are retained. Continue review and dependency tracking in the upstream PR above.


Head: codex/review-layer-p1 at 1d003221f2ba9be949de2ddd95bf6c9145697e66.
Scoped base: codex/review-base-main at 4003bf1077debcca3fccfa6e0d7b56ed584aa727; 1 commit, 13 files. Fixed comparison.

Motivation

The CPU simulator must import SGLang and initialize its native allocator without probing a physical accelerator or importing an optional AITER kernel before it is used. This PR fixes bootstrap and allocator compatibility independently of any optional cost provider. It does not claim that every unpatched CPU installation fails.

Delivered capability after prerequisites: the existing simulator can bootstrap CPU parent and spawned workers, preserve native quantization configuration validation, and translate an explicit logical token capacity through the native pool allocator.

Dependencies: the current SGLang base and the simulator's existing installation requirements. No SimOpus SDK, native staged adapter, closed-loop transport, or later resource-budget PR is required. This fork PR is a scoped review layer against the pinned upstream baseline.

Modifications

  • Activate CPU compatibility only when both SGLANG_SIMULATOR_BOOTSTRAP=1 and SGLANG_USE_CPU_ENGINE=1 are set by the simulator entry point. Other flag combinations leave these wrappers inactive.
  • Install the minimal loader surface before worker imports while retaining native configuration validation.
  • Bound the existing ordinary token-capacity search to 64 expansion probes while preserving its at-least-capacity rule and undersized-probe behavior. Use native allocator APIs at their supported import locations without swallowing transitive import errors.
  • Defer the optional Quark AITER import until the first quantization call and cache the successful binding. The actual quantizer, arguments, reshape and results remain delegated. This separate change affects import timing in all execution modes; it does not replace the GPU quantizer.
  • Add registered Quark unit coverage and explicit simulator CPU workflow selection.

For a basic CPU run, use the documented simulator entry point, an existing replay configuration, and an explicit --max-total-tokens capacity. Later PRs add other policies; this layer does not claim a new latency model.

The isolated Quark utility fix is submitted upstream in #49231, with the real CPU-container reproducer in #49230. This review layer retains the same bytes until that prerequisite lands and is rebased.

Scope boundary

P1 adds no SimOpus imports, resource policies, exact-capacity option or SDK-specific rejection rules. The exact SDK capacity helper and test_cpu_pool_capacity.py belong to P4. P1 retains only test_cpu_pool_bounds.py for the generic existing allocator path.

Accuracy Tests

The unchanged Quark fix passes all 13 registered CPU tests against core 2ad35d45, including deferred import, preserved dependency errors, argument/result delegation and one-time successful binding. Five fresh-process controls cover the four CPU flag combinations and repeat activation. Native-capacity and allocator compatibility controls cover bounded failure and import identity. The new generic capacity file passes all five cases on this head and on the assembled stack; stacking P4 passes those five plus its nine exact-capacity cases. A pure CPU import baseline also passes, so the reproducible Quark read-only-HOME failure is kept as a narrower optional-dependency issue. Whole-stack differential evidence is maintained in the native integration PR; it is not a separate full-cohort accuracy claim for this layer.

Speed Tests and Profiling

No physical GPU throughput result is claimed. The cached loader avoids a repeated Python import on successful quantization calls. CPU simulator checks validate functionality rather than inference performance.

Checklist

  • Add focused CPU tests, including registered SGLang unit coverage.
  • Document activation, dependencies and nonselected behavior.
  • Keep allocator arithmetic and quantizer delegation native.
  • Complete required repository CI and maintainer review.

Review and Merge Process

Fork merge changes only the fork; upstream landing requires dependency rebase, separate submission and normal required reviews/CI.

Review this foundation separately from simulator cost and transport features. Follow CODEOWNERS, required CI and branch protection; this review-layer draft does not waive them.


CI States

Latest PR Test (Base): ❌ Run #42559064312
Latest PR Test (Extra): ❌ Run #42559064428

Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
@Jeff010101
Jeff010101 force-pushed the codex/review-layer-p1 branch from b3a27e1 to 1d00322 Compare November 9, 2026 03:56
@Jeff010101 Jeff010101 changed the title [Feat] simulator: Isolate CPU bootstrap and optional accelerator imports [Fix] simulator: Keep CPU bootstrap and allocator compatibility generic Nov 9, 2026
@Jeff010101 Jeff010101 closed this Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant