Skip to content

[Perf] simulator: Add an opt-in metrics-only numeric cost path - #7

Closed
Jeff010101 wants to merge 1 commit into
codex/review-layer-p8from
codex/review-layer-p9
Closed

Jeff010101 wants to merge 1 commit into
codex/review-layer-p8from
codex/review-layer-p9

Conversation

@Jeff010101

@Jeff010101 Jeff010101 commented Nov 8, 2026 •

Copy link
Copy Markdown
Owner

Superseded by the upstream Draft PR: sgl-project#49245.

This fork-only review is closed to keep one active PR for the feature. Its discussion and source branches are retained. Continue review and dependency tracking in the upstream PR above.


Head: codex/review-layer-p9 at c179657fb2e031a5d24bf4a911fab71468ef86fd.
Scoped base: codex/review-layer-p8 at 220fc474d1127bd03acdfac7c8b911329dc45f93; 1 commit, 9 files. Fixed comparison.

Motivation

A metrics-only run should not construct full per-phase cost documents that its retention policy deliberately omits, while preserving the SDK's preparation and query identities.

Modifications

Delivered capability after prerequisites

Enable both settings explicitly:

export SGLANG_SIMULATOR_EVIDENCE_LEVEL=metrics
export SGLANG_SIMULATOR_METRICS_CPU_DETAIL=off

Only this combination selects public estimate_prepared_forward_ms. Defaults remain full evidence and CPU detail on; detail-off with full retention is rejected. The native factory initializes the SDK model/assets pair exactly once and reuses that pair for prepared estimates. It does not reload a second source binding or replace the SDK with a simulator predictor.

Source, profile, topology, supported-domain and every-cost validation still apply before progress or history can advance. Missing numeric API or an incompatible pair fails initialization; there is no relaxed fallback. Exact model/assets types and the retained private pair-identity check are documented pinned compatibility requirements, not a newly claimed public SDK interface.

Omitted evaluated documents, classifications and full-ledger digests remain explicitly unavailable. Numeric serving metrics cannot stand in for per-query custody or full independent cost verification.

Dependencies and enablement

Requires evidence retention/storage, which includes the native core. Empirical/given progress is optional. Provide the same exact external SDK/model/source/capture/data inputs as the bounded native route. The documented SDK must additionally supply the numeric API and the single verified model/assets pairing; unsupported installations fail clearly.

Struct-backed simulator records require msgspec>=0.21.0 so replacement keeps post-construction validation. SDK-owned records and all cost/progress implementations are unchanged.

Accuracy Tests

CI explicitly selects test_metrics_cpu_detail.py. Minimal required-SDK acceptance sets SGLANG_SIMULATOR_REQUIRE_SIMOPUS_SDK=1 and runs that file with test_native_public_sdk.py and test_simopus_native_factory.py.

The 40 metrics CPU controls passed with the SDK present, including float-bit and leaf-query equivalence, one initialization, strict pair/domain/API rejection and late-failure ordering. They also passed as affected focused controls after record migration. Current matched-runtime status, complete software counts and unexecuted cases are centralized in the native PR; no full-MAPE result is relabeled onto this source.

Speed Tests and Profiling

This removes optional simulator CPU work, not real GPU inference work. No new GPU speed benchmark or quantified speedup is claimed.

Checklist

  • Explicit selection/defaults, required-API documentation and focused controls.
  • Formatting, English documentation and local links checked.
  • Required upstream reviews and applicable CI.

Review and Merge Process

This fork PR is review-only. A fork merge does not change SGLang upstream. After native/storage prerequisites land upstream, rebase and submit the scoped optimization through normal CODEOWNERS review and CI. It remains Draft.


CI States

Latest PR Test (Base): ❌ Run #42559063260
Latest PR Test (Extra): ❌ Run #42559064221

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Nov 8, 2026
@Jeff010101
Jeff010101 force-pushed the codex/review-layer-p9 branch from 44fd133 to c179657 Compare November 9, 2026 02:04
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
@Jeff010101
Jeff010101 force-pushed the codex/review-layer-p8 branch from 220fc47 to d60943f Compare November 9, 2026 03:56
@Jeff010101
Jeff010101 force-pushed the codex/review-layer-p9 branch from c179657 to 275b24a Compare November 9, 2026 03:56
@Jeff010101 Jeff010101 closed this Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant