Repository navigation
[Perf] simulator: Add an opt-in metrics-only numeric cost path - #7
Closed
Jeff010101 wants to merge 1 commit into
Closed
Jeff010101 wants to merge 1 commit into
Jeff010101 wants to merge 1 commit into
Conversation
Jeff010101
force-pushed
the
codex/review-layer-p9
branch
from
November 9, 2026 02:04
44fd133 to
c179657
Compare
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Jeff010101
force-pushed
the
codex/review-layer-p8
branch
from
November 9, 2026 03:56
220fc47 to
d60943f
Compare
Jeff010101
force-pushed
the
codex/review-layer-p9
branch
from
November 9, 2026 03:56
c179657 to
275b24a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Superseded by the upstream Draft PR: sgl-project#49245.
This fork-only review is closed to keep one active PR for the feature. Its discussion and source branches are retained. Continue review and dependency tracking in the upstream PR above.
Head:
codex/review-layer-p9atc179657fb2e031a5d24bf4a911fab71468ef86fd.Scoped base:
codex/review-layer-p8at220fc474d1127bd03acdfac7c8b911329dc45f93; 1 commit, 9 files. Fixed comparison.Motivation
A metrics-only run should not construct full per-phase cost documents that its retention policy deliberately omits, while preserving the SDK's preparation and query identities.
Modifications
Delivered capability after prerequisites
Enable both settings explicitly:
Only this combination selects public
estimate_prepared_forward_ms. Defaults remain full evidence and CPU detail on; detail-off with full retention is rejected. The native factory initializes the SDK model/assets pair exactly once and reuses that pair for prepared estimates. It does not reload a second source binding or replace the SDK with a simulator predictor.Source, profile, topology, supported-domain and every-cost validation still apply before progress or history can advance. Missing numeric API or an incompatible pair fails initialization; there is no relaxed fallback. Exact model/assets types and the retained private pair-identity check are documented pinned compatibility requirements, not a newly claimed public SDK interface.
Omitted evaluated documents, classifications and full-ledger digests remain explicitly unavailable. Numeric serving metrics cannot stand in for per-query custody or full independent cost verification.
Dependencies and enablement
Requires evidence retention/storage, which includes the native core. Empirical/given progress is optional. Provide the same exact external SDK/model/source/capture/data inputs as the bounded native route. The documented SDK must additionally supply the numeric API and the single verified model/assets pairing; unsupported installations fail clearly.
Struct-backed simulator records require
msgspec>=0.21.0so replacement keeps post-construction validation. SDK-owned records and all cost/progress implementations are unchanged.Accuracy Tests
CI explicitly selects
test_metrics_cpu_detail.py. Minimal required-SDK acceptance setsSGLANG_SIMULATOR_REQUIRE_SIMOPUS_SDK=1and runs that file withtest_native_public_sdk.pyandtest_simopus_native_factory.py.The 40 metrics CPU controls passed with the SDK present, including float-bit and leaf-query equivalence, one initialization, strict pair/domain/API rejection and late-failure ordering. They also passed as affected focused controls after record migration. Current matched-runtime status, complete software counts and unexecuted cases are centralized in the native PR; no full-MAPE result is relabeled onto this source.
Speed Tests and Profiling
This removes optional simulator CPU work, not real GPU inference work. No new GPU speed benchmark or quantified speedup is claimed.
Checklist
Review and Merge Process
This fork PR is review-only. A fork merge does not change SGLang upstream. After native/storage prerequisites land upstream, rebase and submit the scoped optimization through normal CODEOWNERS review and CI. It remains Draft.
CI States
Latest PR Test (Base): ❌ Run #42559063260
Latest PR Test (Extra): ❌ Run #42559064221