Skip to content

[Feat] simulator: Bound record memory and add retention codecs - #49244

Draft
Jeff010101 wants to merge 13 commits into
sgl-project:mainfrom
Jeff010101:codex/review-layer-p8
Draft

Jeff010101 wants to merge 13 commits into
sgl-project:mainfrom
Jeff010101:codex/review-layer-p8

Conversation

@Jeff010101

@Jeff010101 Jeff010101 commented Nov 9, 2026 •

Copy link
Copy Markdown

P8: Bounded evidence memory and explicit retention/codecs

Source head: codex/review-layer-p8 at 2b600081cc24f24bd0d9de597e8177a10a4ae632.
Focused feature base: b7b7db61ba6eee55fae54e57012c890a04281a87; 1 feature commit, 12 files. Review this feature's fixed diff.

Target: sgl-project/sglang:main. GitHub's full diff also includes unmerged prerequisites; use the focused comparison above to review this layer. Dependencies must land and the branch must be rebased before merge.

Dependencies: Requires P5 for its native sink integration. P6 is a parallel sibling, not a prerequisite. Numeric SDK execution is supplied separately by P9.

Motivation

Verbose iteration and evaluated-cost documents should not grow the scheduler's Python heap or force every simulation to retain the same debug artifacts.

Modifications

Delivered capability after prerequisites

This PR spools full iteration evidence to disk, adds explicit lossless gzip/zstd choices and separates full from metrics retention. Defaults remain full and uncompressed JSONL; existing runs are not silently converted.

Logical JSONL ordering, bytes and SHA/size are independent of compressed-file identity. Failed writes/exports preserve the spool and do not establish a completed profile. Full retention keeps committed event/evaluated-cost linkage; metrics retention reports verbose records and digests as omitted rather than claiming an independent full cost ledger.

Dependencies and enablement

Its native sink integration follows the native core. Empirical/given progress is a parallel sibling, not a prerequisite. This PR does not add the numeric SDK execution path.

For explicit Zstandard output, install ./tools/sglang-simulator[zstd] and set SGLANG_SIMULATOR_EVIDENCE_COMPRESSION=zstd before starting an existing workload. gzip also remains available without that extra. Set SGLANG_SIMULATOR_EVIDENCE_LEVEL=metrics only when the required deliverable does not need complete per-step cost evidence. With variables unset, the original full/plain output remains selected.

Provide a writable output directory with capacity for spool and export coexistence; compression does not justify deleting earlier originals or changing a live run's policy. Client recording and token-event recording remain separately configured. Neither retention nor codec selection changes model costs, request progress, RNG, scheduling or throughput denominators.

Accuracy Tests

The P1 prerequisite has been rebased onto Quark #49231, advancing the inherited core to 2ad35d45. Its runtime source files are unchanged. Execution evidence below predates this core rebase; matching new-core integration has not been rerun.

Minimal acceptance covers test_iteration_stats.py, test_evidence_storage.py, test_native_evidence_hooks.py and public-SDK row controls. CI explicitly installs the zstd extra. With the SDK installed, SGLANG_SIMULATOR_REQUIRE_SIMOPUS_SDK=1 makes SDK-backed controls required rather than skipped.

After the Struct representation change, the three directly affected storage/export files passed 23 tests with zero skips, covering plain/gzip/zstd, default-full output, metrics retention, spool CRC/truncation and failed exports. The new software record fixture also preserves the old JSON representation. These are serialization/storage checks, not a new full-MAPE or GPU-timing validation. Whole-stack status and diagnostic limits are centralized in the native PR.

Speed Tests and Profiling

No real-inference or GPU speedup is claimed. Storage savings depend on actual selected evidence and workload; provision from measured peak use.

Checklist

  • Explicit defaults, codec dependencies, failure contracts and focused tests.
  • Formatting, English documentation and local links checked.
  • Required upstream reviews and applicable CI.

Review and Merge Process

Keep this upstream PR Draft until dependency rebases, applicable CI and required maintainer reviews are complete. Existing execution results retain their recorded source/runtime identities; this PR publication adds no new execution or accuracy result.


CI States

Latest PR Test (Base): ⊘ Skipped -- the PR is a draft; mark it ready, then /rerun-failed-ci.
Latest PR Test (Extra): ⊘ Skipped -- the PR is a draft; mark it ready, then /rerun-failed-ci.
Latest PR Test (AMD ROCm 10): ⊘ Skipped -- the PR is a draft; mark it ready, then /rerun-failed-ci.

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
@Jeff010101
Jeff010101 force-pushed the codex/review-layer-p8 branch from fbf0cc0 to 2b60008 Compare November 9, 2026 06:11
@Jeff010101 Jeff010101 changed the title [Feat] simulator: Bound evidence memory and support explicit retention codecs [Feat] simulator: Add bounded evidence storage and retention codecs Nov 9, 2026
@Jeff010101 Jeff010101 changed the title [Feat] simulator: Add bounded evidence storage and retention codecs [Feat] Bound record memory and add retention codecs Nov 9, 2026
@Jeff010101 Jeff010101 changed the title [Feat] Bound record memory and add retention codecs [Feat] simulator: Bound record memory and add retention codecs Nov 18, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant