Repository navigation
[Feat] simulator: Bound record memory and add retention codecs - #49244
Draft
Jeff010101 wants to merge 13 commits into
Draft
Jeff010101 wants to merge 13 commits into
Jeff010101 wants to merge 13 commits into
Conversation
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Jeff010101
force-pushed
the
codex/review-layer-p8
branch
from
November 9, 2026 05:37
d60943f to
fbf0cc0
Compare
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Signed-off-by: Jeff010101 <Jifang.Xing@amd.com>
Jeff010101
force-pushed
the
codex/review-layer-p8
branch
from
November 9, 2026 06:11
fbf0cc0 to
2b60008
Compare
This was referenced Nov 13, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
P8: Bounded evidence memory and explicit retention/codecs
Source head:
codex/review-layer-p8at2b600081cc24f24bd0d9de597e8177a10a4ae632.Focused feature base:
b7b7db61ba6eee55fae54e57012c890a04281a87; 1 feature commit, 12 files. Review this feature's fixed diff.Target:
sgl-project/sglang:main. GitHub's full diff also includes unmerged prerequisites; use the focused comparison above to review this layer. Dependencies must land and the branch must be rebased before merge.Dependencies: Requires P5 for its native sink integration. P6 is a parallel sibling, not a prerequisite. Numeric SDK execution is supplied separately by P9.
Motivation
Verbose iteration and evaluated-cost documents should not grow the scheduler's Python heap or force every simulation to retain the same debug artifacts.
Modifications
Delivered capability after prerequisites
This PR spools full iteration evidence to disk, adds explicit lossless gzip/zstd choices and separates
fullfrommetricsretention. Defaults remainfulland uncompressed JSONL; existing runs are not silently converted.Logical JSONL ordering, bytes and SHA/size are independent of compressed-file identity. Failed writes/exports preserve the spool and do not establish a completed profile. Full retention keeps committed event/evaluated-cost linkage; metrics retention reports verbose records and digests as omitted rather than claiming an independent full cost ledger.
Dependencies and enablement
Its native sink integration follows the native core. Empirical/given progress is a parallel sibling, not a prerequisite. This PR does not add the numeric SDK execution path.
For explicit Zstandard output, install
./tools/sglang-simulator[zstd]and setSGLANG_SIMULATOR_EVIDENCE_COMPRESSION=zstdbefore starting an existing workload.gzipalso remains available without that extra. SetSGLANG_SIMULATOR_EVIDENCE_LEVEL=metricsonly when the required deliverable does not need complete per-step cost evidence. With variables unset, the original full/plain output remains selected.Provide a writable output directory with capacity for spool and export coexistence; compression does not justify deleting earlier originals or changing a live run's policy. Client recording and token-event recording remain separately configured. Neither retention nor codec selection changes model costs, request progress, RNG, scheduling or throughput denominators.
Accuracy Tests
The P1 prerequisite has been rebased onto Quark #49231, advancing the inherited core to
2ad35d45. Its runtime source files are unchanged. Execution evidence below predates this core rebase; matching new-core integration has not been rerun.Minimal acceptance covers
test_iteration_stats.py,test_evidence_storage.py,test_native_evidence_hooks.pyand public-SDK row controls. CI explicitly installs the zstd extra. With the SDK installed,SGLANG_SIMULATOR_REQUIRE_SIMOPUS_SDK=1makes SDK-backed controls required rather than skipped.After the Struct representation change, the three directly affected storage/export files passed 23 tests with zero skips, covering plain/gzip/zstd, default-full output, metrics retention, spool CRC/truncation and failed exports. The new software record fixture also preserves the old JSON representation. These are serialization/storage checks, not a new full-MAPE or GPU-timing validation. Whole-stack status and diagnostic limits are centralized in the native PR.
Speed Tests and Profiling
No real-inference or GPU speedup is claimed. Storage savings depend on actual selected evidence and workload; provision from measured peak use.
Checklist
Review and Merge Process
Keep this upstream PR Draft until dependency rebases, applicable CI and required maintainer reviews are complete. Existing execution results retain their recorded source/runtime identities; this PR publication adds no new execution or accuracy result.
CI States
Latest PR Test (Base): ⊘ Skipped -- the PR is a draft; mark it ready, then
/rerun-failed-ci.Latest PR Test (Extra): ⊘ Skipped -- the PR is a draft; mark it ready, then
/rerun-failed-ci.Latest PR Test (AMD ROCm 10): ⊘ Skipped -- the PR is a draft; mark it ready, then
/rerun-failed-ci.Tip
Base CI only covers the core cases, and its workload may not exercise your change. Use
/rerun-testto run the tests your change touches, or copy this prompt for an agent: