Skip to content

[Feat] simulator: Add source-bound staged execution - #48814

Draft
Jeff010101 wants to merge 13 commits into
sgl-project:mainfrom
Jeff010101:codex/task4-native-public-followup-20261106
Draft

Jeff010101 wants to merge 13 commits into
sgl-project:mainfrom
Jeff010101:codex/task4-native-public-followup-20261106

Conversation

@Jeff010101

@Jeff010101 Jeff010101 commented Nov 6, 2026 •

Copy link
Copy Markdown

P5: Source-bound native ADP1 staged execution

Source head: codex/task4-native-public-followup-20261106 at b0bc21178bcd0ee4f6f0007968c44f979bafbb76.
Focused feature base: 2acb83cec0c576b362a1eb2d13e5e75603e4d43e; 2 feature commits, 101 files. Review this feature's fixed diff.

Target: sgl-project/sglang:main. GitHub's full diff also includes unmerged prerequisites; use the focused comparison above to review this layer. Dependencies must land and the branch must be rebased before merge.

Dependencies: Requires P2, P7 (which includes P1) and shared SDK support (an internal shared commit, not a separate feature PR). It does not require the ordinary latency-adapter or automatic KV-budget features.

Motivation

Native staged simulation needs SDK-owned costs and simulator-owned selection, progress and request lifecycle, without changing real-serving algorithms.

Modifications

  • Own this adapter's CPU initialization and unselected-provider import controls in this layer. Generic prerequisite tests do not prepare or name this adapter.

Delivered capability after prerequisites

This opt-in route executes source-bound ADP1 worker stages with runnable SourceGolden progress, result-owned timing, FIFO publication, overlap/FutureMap bookkeeping and KV retirement.

flowchart LR
  B[Native before-work state] --> C[Public SDK stage costs]
  C --> R[Result-owned timing]
  G[SourceGolden progress] --> R
  R --> L[Publication and KV lifecycle]
Loading

All costs resolve before progress is drawn. Invalid/missing costs cannot advance modeled history, forward ID or the virtual clock. The 19 native records now use immutable/mutable Structs as appropriate. msgspec>=0.21.0 preserves replacement validation; explicit JSON projections retain the original fields and ordering. SDK-owned records are unchanged.

Dependencies and enablement

Requires generic closed loop, packed KV support, its CPU prerequisite, and shared SDK support. That support is the first internal commit of the ordinary SDK PR, also included here; it is not a separate feature PR. Whichever SDK feature lands first supplies it. Native does not wait for ordinary R1 runtime behavior.

Configure the exact SDK/model/source/capture/runtime pins, explicit source_golden351 and seed, then use the native launch documented in docs/simopus-native.md. Matching external SDK assets and model configuration are required. Scope is TP2/TP4, ADP1, PP1/EP1/CP1, static Q5/Q6 and length-only synthetic completion. A run starts cold. ADP2/joint, unsupported source domains and semantic answer quality are excluded. Empirical/given policies and storage/numeric optimizations remain separate.

Dependency overview

flowchart TD
  P1[CPU simulator compatibility] --> P7[Packed KV metadata]
  P7 --> P5[Native ADP1 staged execution]
  P2[Generic closed loop] --> P5
  C0[Shared SDK support: internal commit] --> P3[Whole-forward adapter]
  C0 --> P4[Resource budgets]
  C0 --> P5
  P1 --> P4
  P5 --> P6[Progress policies]
  P5 --> P8[Evidence storage]
  P8 --> P9[Metrics-only numeric execution]
Loading

The ordinary adapter and resource-budget features are independent of native execution. Progress policies and evidence storage are siblings. Shared SDK support is carried by the first SDK feature to land, then reused by rebased branches.

Storage-independent SDK correction

Native source-binding roots are opaque SDK inputs. Configuration resolution calls sdk.resolve_native_source_binding(binding, base_dir=...) rather than interpreting the root as a filesystem path. The existing native cost path already delegates to sdk.load_native_region_assets; it remains unchanged. Missing SDK interfaces fail with provider_initialization_failed, without a filesystem fallback or a SQLite/Parquet branch.

Requires a compatible SDK including SimOpus #541 at 11052146027a53283987a9ae0a3acf69d38a16f4 or its successor. Native configuration, real public SDK factory/cost controls (TP2 and TP4), and canonical SDK loading: 25 passed with SGLANG_SIMULATOR_REQUIRE_SIMOPUS_SDK=1. No full-cohort replay or new timing/accuracy claim.

Accuracy Tests

The P1 prerequisite has been rebased onto Quark #49231, advancing the inherited core to 2ad35d45. Its runtime source files are unchanged. Execution evidence below predates this core rebase; matching new-core integration has not been rerun.

Require the installed SDK with SGLANG_SIMULATOR_REQUIRE_SIMOPUS_SDK=1 for test_native_public_sdk.py and test_simopus_native_factory.py; native lifecycle and representation controls also run without timing qualification.

Whole-stack status:

  • Earlier CPU matrix: 1,016/1,036 passed; targeted supplement resolved two fixtures and 12 codec skips. Unique coverage is 1,030 passed, one unresolved 51.117 ms offline/blocking discrepancy and five ordinary R1 E2E cases unexecuted.
  • Struct-focused controls: 430 passed/2 runtime-dependent skips. Three new representation controls passed on 0.21.0 and 0.22.0; 23 full-export/codec controls passed without skips. This was not another full-matrix run.
  • R5 cc2a569b on SGLang core tree 2a540f80, SDK 284f19f7 and client e0c2b3d6 matched closed 38bb0c0d and R4: C04 211 complete/1 cancel; C05 475 complete. Each point's 53 metric objects, payloads, trajectories, usage, integer times, snapshots, token commits and windows matched; owners/readers closed.

R5 used SourceGolden351, metrics/off, seed 42, trace-gap cap 300 seconds, 60 virtual seconds plus 30 seconds grace and ten extra warmups per lane, explicitly submission_valid=false. Native cancelled partial tokens/cutoff matched; cancelled HTTP UUID joining and full query-ledger verification remain false. Formal promotions: zero; full nine-point/171-point MAPE was not rerun.

Earlier assembled source preserves every production-file byte of frozen source. Its only tree differences are the new generic pool-boundary test and the CPU workflow selection for that test. Existing E2E observations retain their original source identity.

Speed Tests and Profiling

No new GPU timing or real-inference speedup claim.

Checklist

  • Bounded contracts, source guards, formatting and documentation included.
  • Applicable upstream CI and required reviews.

Review and Merge Process

Keep this upstream PR Draft until dependency rebases, applicable CI and required maintainer reviews are complete. Existing execution results retain their recorded source/runtime identities; this PR publication adds no new execution or accuracy result.


CI States

Latest PR Test (Base): ⊘ Skipped -- the PR is a draft; mark it ready, then /rerun-failed-ci.
Latest PR Test (Extra): ⊘ Skipped -- the PR is a draft; mark it ready, then /rerun-failed-ci.
Latest PR Test (AMD ROCm 10): ⊘ Skipped -- the PR is a draft; mark it ready, then /rerun-failed-ci.

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

@github-actions github-actions Bot added documentation Improvements or additions to documentation dependencies Pull requests that update a dependency file deepseek labels Nov 6, 2026
@Jeff010101
Jeff010101 marked this pull request as ready for review November 6, 2026 05:47

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
@Jeff010101
Jeff010101 force-pushed the codex/task4-native-public-followup-20261106 branch from 9962b2a to cb8a9e8 Compare November 9, 2026 03:56
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
@Jeff010101
Jeff010101 force-pushed the codex/task4-native-public-followup-20261106 branch from 0a37a2b to b7b7db6 Compare November 9, 2026 06:11
@Jeff010101 Jeff010101 changed the title [Feat] simulator: Route source-bound native staged execution through SimOpus [Feat] simulator: Add source-bound native staged execution Nov 9, 2026
@Jeff010101 Jeff010101 changed the title [Feat] simulator: Add source-bound native staged execution [Feat] Add source-bound staged execution Nov 9, 2026
Jeff010101 added a commit to Jeff010101/sglang that referenced this pull request Nov 13, 2026
@Jeff010101 Jeff010101 changed the title [Feat] Add source-bound staged execution [Feat] simulator: Add source-bound staged execution Nov 18, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deepseek dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant