Repository navigation
[Feat] simulator: Add source-bound staged execution - #48814
Draft
Jeff010101 wants to merge 13 commits into
Draft
Jeff010101 wants to merge 13 commits into
Jeff010101 wants to merge 13 commits into
Conversation
Jeff010101
marked this pull request as ready for review
November 6, 2026 05:47
Jeff010101
requested review from
1am9trash,
BowenBao,
HaiShaw,
bingxche,
hubertlu-tw,
hzh0425,
kkHuang-amd,
michaelzhang-ai and
yichiche
as code owners
November 6, 2026 05:47
Jeff010101
marked this pull request as draft
November 6, 2026 05:52
Jeff010101
force-pushed
the
codex/task4-native-public-followup-20261106
branch
2 times, most recently
from
November 8, 2026 19:34
896e14f to
b9fec12
Compare
This was referenced Nov 8, 2026
Closed
Closed
This was referenced Nov 8, 2026
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Jeff010101
force-pushed
the
codex/task4-native-public-followup-20261106
branch
from
November 9, 2026 02:04
b9fec12 to
9962b2a
Compare
3 of 4 tasks
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Jeff010101
force-pushed
the
codex/task4-native-public-followup-20261106
branch
from
November 9, 2026 03:56
9962b2a to
cb8a9e8
Compare
This was referenced Nov 9, 2026
Jeff010101
force-pushed
the
codex/task4-native-public-followup-20261106
branch
from
November 9, 2026 05:37
cb8a9e8 to
0a37a2b
Compare
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Jeff010101
force-pushed
the
codex/task4-native-public-followup-20261106
branch
from
November 9, 2026 06:11
0a37a2b to
b7b7db6
Compare
Jeff010101
added a commit
to Jeff010101/sglang
that referenced
this pull request
Nov 13, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
P5: Source-bound native ADP1 staged execution
Source head:
codex/task4-native-public-followup-20261106atb0bc21178bcd0ee4f6f0007968c44f979bafbb76.Focused feature base:
2acb83cec0c576b362a1eb2d13e5e75603e4d43e; 2 feature commits, 101 files. Review this feature's fixed diff.Target:
sgl-project/sglang:main. GitHub's full diff also includes unmerged prerequisites; use the focused comparison above to review this layer. Dependencies must land and the branch must be rebased before merge.Dependencies: Requires P2, P7 (which includes P1) and shared SDK support (an internal shared commit, not a separate feature PR). It does not require the ordinary latency-adapter or automatic KV-budget features.
Motivation
Native staged simulation needs SDK-owned costs and simulator-owned selection, progress and request lifecycle, without changing real-serving algorithms.
Modifications
Delivered capability after prerequisites
This opt-in route executes source-bound ADP1 worker stages with runnable SourceGolden progress, result-owned timing, FIFO publication, overlap/FutureMap bookkeeping and KV retirement.
All costs resolve before progress is drawn. Invalid/missing costs cannot advance modeled history, forward ID or the virtual clock. The 19 native records now use immutable/mutable Structs as appropriate.
msgspec>=0.21.0preserves replacement validation; explicit JSON projections retain the original fields and ordering. SDK-owned records are unchanged.Dependencies and enablement
Requires generic closed loop, packed KV support, its CPU prerequisite, and shared SDK support. That support is the first internal commit of the ordinary SDK PR, also included here; it is not a separate feature PR. Whichever SDK feature lands first supplies it. Native does not wait for ordinary R1 runtime behavior.
Configure the exact SDK/model/source/capture/runtime pins, explicit
source_golden351and seed, then use the native launch documented indocs/simopus-native.md. Matching external SDK assets and model configuration are required. Scope is TP2/TP4, ADP1, PP1/EP1/CP1, static Q5/Q6 and length-only synthetic completion. A run starts cold. ADP2/joint, unsupported source domains and semantic answer quality are excluded. Empirical/given policies and storage/numeric optimizations remain separate.Dependency overview
The ordinary adapter and resource-budget features are independent of native execution. Progress policies and evidence storage are siblings. Shared SDK support is carried by the first SDK feature to land, then reused by rebased branches.
Storage-independent SDK correction
Native source-binding roots are opaque SDK inputs. Configuration resolution calls
sdk.resolve_native_source_binding(binding, base_dir=...)rather than interpreting the root as a filesystem path. The existing native cost path already delegates tosdk.load_native_region_assets; it remains unchanged. Missing SDK interfaces fail withprovider_initialization_failed, without a filesystem fallback or a SQLite/Parquet branch.Requires a compatible SDK including SimOpus #541 at
11052146027a53283987a9ae0a3acf69d38a16f4or its successor. Native configuration, real public SDK factory/cost controls (TP2 and TP4), and canonical SDK loading: 25 passed withSGLANG_SIMULATOR_REQUIRE_SIMOPUS_SDK=1. No full-cohort replay or new timing/accuracy claim.Accuracy Tests
The P1 prerequisite has been rebased onto Quark #49231, advancing the inherited core to
2ad35d45. Its runtime source files are unchanged. Execution evidence below predates this core rebase; matching new-core integration has not been rerun.Require the installed SDK with
SGLANG_SIMULATOR_REQUIRE_SIMOPUS_SDK=1fortest_native_public_sdk.pyandtest_simopus_native_factory.py; native lifecycle and representation controls also run without timing qualification.Whole-stack status:
cc2a569bon SGLang core tree2a540f80, SDK284f19f7and cliente0c2b3d6matched closed38bb0c0dand R4: C04 211 complete/1 cancel; C05 475 complete. Each point's 53 metric objects, payloads, trajectories, usage, integer times, snapshots, token commits and windows matched; owners/readers closed.R5 used SourceGolden351, metrics/off, seed 42, trace-gap cap 300 seconds, 60 virtual seconds plus 30 seconds grace and ten extra warmups per lane, explicitly
submission_valid=false. Native cancelled partial tokens/cutoff matched; cancelled HTTP UUID joining and full query-ledger verification remain false. Formal promotions: zero; full nine-point/171-point MAPE was not rerun.Earlier assembled source preserves every production-file byte of frozen source. Its only tree differences are the new generic pool-boundary test and the CPU workflow selection for that test. Existing E2E observations retain their original source identity.
Speed Tests and Profiling
No new GPU timing or real-inference speedup claim.
Checklist
Review and Merge Process
Keep this upstream PR Draft until dependency rebases, applicable CI and required maintainer reviews are complete. Existing execution results retain their recorded source/runtime identities; this PR publication adds no new execution or accuracy result.
CI States
Latest PR Test (Base): ⊘ Skipped -- the PR is a draft; mark it ready, then
/rerun-failed-ci.Latest PR Test (Extra): ⊘ Skipped -- the PR is a draft; mark it ready, then
/rerun-failed-ci.Latest PR Test (AMD ROCm 10): ⊘ Skipped -- the PR is a draft; mark it ready, then
/rerun-failed-ci.Tip
Base CI only covers the core cases, and its workload may not exercise your change. Use
/rerun-testto run the tests your change touches, or copy this prompt for an agent: