Repository navigation
Conversation
Contributor
Preview approval requiredThis pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit |
CaoE
marked this pull request as ready for review
November 6, 2026 05:59
CaoE
requested review from
Fridge003,
JustinTong0323,
Ying1123,
hnyls2002,
ispobock,
merrymercy,
sogalin,
wisclmy0611,
xiezhq-hermann and
zijiexia
as code owners
November 6, 2026 05:59
Contributor
Author
|
/tag-run-ci-label |
CaoE
force-pushed
the
ecao/xpu_graph_improve_rebase
branch
from
November 9, 2026 06:24
f00d7ce to
434c983
Compare
…raphBackend XPU device graphs record raw device addresses rather than holding Python reference counts. Retain capture_inputs in FullXPUGraphBackend to ensure capture-time tensors (e.g. dummy batch metadata under data parallelism) are preserved across the graph's lifetime rather than being collected by GC. Also ensure captured graphs are cleanly reset upon backend cleanup. Update XPU graph limitations in xpu.mdx. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CaoE
force-pushed
the
ecao/xpu_graph_improve_rebase
branch
from
November 9, 2026 07:39
fd6c435 to
f91dd9b
Compare
This comment has been minimized.
This comment has been minimized.
airMeng
pushed a commit
to airMeng/sglang
that referenced
this pull request
Nov 9, 2026
airMeng
pushed a commit
to airMeng/sglang
that referenced
this pull request
Nov 10, 2026
airMeng
pushed a commit
to airMeng/sglang
that referenced
this pull request
Nov 11, 2026
airMeng
pushed a commit
to airMeng/sglang
that referenced
this pull request
Nov 12, 2026
airMeng
pushed a commit
to airMeng/sglang
that referenced
this pull request
Nov 13, 2026
airMeng
pushed a commit
to airMeng/sglang
that referenced
this pull request
Nov 15, 2026
airMeng
pushed a commit
to airMeng/sglang
that referenced
this pull request
Nov 16, 2026
…raphBackend (sgl-project#48609) sgl-project#48609 Source-head: 058e06a
airMeng
pushed a commit
to airMeng/sglang
that referenced
this pull request
Nov 17, 2026
…raphBackend (sgl-project#48609) sgl-project#48609 Source-head: 058e06a
airMeng
pushed a commit
to airMeng/sglang
that referenced
this pull request
Nov 18, 2026
…raphBackend (sgl-project#48609) sgl-project#48609 Source-head: 058e06a
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
In SGLang, device graphs capture and record raw GPU virtual addresses rather than retaining Python object references. During graph capture (e.g., under data parallelism where dummy batches instantiate capture-only metadata tensors such as
global_num_tokens_gpu), these temporary tensors must be preserved across the graph's lifetime to prevent Python GC from collecting them. If collected, the PyTorch caching allocator may reassign their underlying memory while the graph actively replays, resulting in dangling pointers, silent memory corruption, or illegal memory access errors.While
BreakableCudaGraphBackendtracks and retainscapture_inputs,FullXPUGraphBackendpreviously accepted thecapture_inputsparameter incapture_one()but omitted storing it. Additionally, captured graphs were not explicitly reset upon backendcleanup().Modifications
Retain Capture Inputs in
FullXPUGraphBackend(python/sglang/srt/hardware_backend/xpu/graph_runner/xpu_full_graph_backend.py):self._capture_inputs: Dict[Any, Any] = {}in__init__.self._capture_inputs[shape_key] = capture_inputsincapture_one()so that capture-time tensor lifetimes are preserved across the backend lifecycle.self._capture_inputs.clear()incleanup().Explicitly Reset Graphs upon Cleanup (
xpu_full_graph_backend.py):graph.reset()on all capturedXPUGraphinstances duringcleanup()to ensure allocated memory pool segments are cleanly released back to the driver.Accuracy Tests
XPU Graph Metadata & Capture Inputs Retention Tests (
test/registered/xpu/test_xpu_graph_metadata.py):test_full_xpu_graph_backend_retains_capture_inputs_and_cleans_upalong with SWA page table and chunk-size rejection tests.XPU Full Graph End-to-End Integration Tests (
test/registered/xpu/test_xpu_graph.py):Speed Tests and Profiling
Checklist
Review and Merge Process
/tag-and-rerun-ci,/tag-run-ci-label,/rerun-failed-ciCI States
Latest PR Test (Base): ✅ Run #42778802437
Latest PR Test (Extra): ❌ Run #42778802070
Latest PR Test (AMD ROCm 10): ✅ Run #42778802192
Tip
Base CI only covers the core cases, and its workload may not exercise your change. Use
/rerun-testto run the tests your change touches, or copy this prompt for an agent: