Skip to content

fix: resolve correctness_test issue - #1002

Merged
zhyncs merged 1 commit into
sgl-project:mainfrom
zhyncs:fix
Aug 9, 2024
Merged

zhyncs merged 1 commit into
sgl-project:mainfrom
zhyncs:fix

Conversation

@zhyncs

@zhyncs zhyncs commented Aug 9, 2024

Copy link
Copy Markdown
Contributor

Motivation

as titled cc @merrymercy @Ying1123 @hnyls2002 @min-xu-et

tested with

python -m sglang.bench_latency --model-path internlm/internlm2-chat-7b --load-format dummy --trust-remote-code
python -m sglang.bench_latency --model-path internlm/internlm2-chat-7b --batch 1 12 14 --input-len 256 512 --output-len 32 256 --result-filename out.jsonl --trust-remote-code
python -m sglang.bench_latency --model-path internlm/internlm2-chat-7b --batch 1 12 14 --input-len 256 512 --output-len 32 256 --result-filename out.jsonl --run-name after --trust-remote-code
python -m sglang.bench_latency --result-filename out.jsonl --graph-sql="select run_name, batch_size, prefill_throughput from results" 
python -m sglang.bench_latency --model-path internlm/internlm2-chat-7b --correct --trust-remote-code

Modification

Briefly describe the changes made in this PR.

Checklist

  1. Ensure pre-commit pre-commit run --all-files or other linting tools are used to fix potential lint issues.
  2. Confirm that modifications are covered by complete unit tests. If not, please add more unit tests for correctness.
  3. Modify documentation as needed, such as docstrings or example tutorials.

@zhyncs
zhyncs merged commit b568df5 into sgl-project:main Aug 9, 2024
@zhyncs
zhyncs deleted the fix branch August 9, 2024 13:21

@min-xu-et min-xu-et left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix!

timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 29, 2026
Boot 51 died on "sgl-project#789 OUTPUT READINESS TIMEOUT: mb_id=2: upstream (rank 1)
posted 9 dict message(s) on dict|output (entered 9), this rank has consumed 9".
Nine posted, nine consumed, nothing outstanding -- and the rank waited for a
tenth, then killed the instance for its absence.

The arithmetic in the readiness gate is right; the caller is not. `_do_recv`
enters the blocking output receive on `mbs[next_mb_id] is not None and not
prebuilt and not can_skip` -- this rank's OWN slot state -- while the upstream
forwards on `if pp_outputs:`. Two predicates on one stream, and the raise's
own hint says so: "this rank decided to receive from its OWN slot state while
its upstream decided to forward on `if pp_outputs:`, and nothing publishes
this rank's per-slot expectation to that upstream".

So the gate now reports its verdict instead of only ever killing on it. When
`consumed == posted == attempted`, the upstream has stated in-band that it
forwarded nothing for this slot, and the caller takes the SAME no-output exit
it already takes for `target is None` two lines up -- an existing, handled
path, not a new one.

What this deliberately is not: nothing is dropped, because nothing was sent;
nothing is refused, because there is nothing to refuse. Those two forms are
both metal-falsified on this path already (corpse R 2026-08-09, and sgl-project#995 in
this window, boot 15). And no token is fabricated -- `_pp_make_skip_output_
result` is NOT reused here, because its zero placeholder is legitimate only
for a chunk that really ran and produced no token, which is not this case.

Opt-in, default unchanged. The proxy alias keeps raising: a missing proxy is a
different fact, since those hidden states are the input to this rank's own
forward and declining them would compute on nothing.

Evidence: desk, executed. py_compile; both `soft` parameters present; the
proxy alias reaches no soft path; the declined case emits a named line with
posted/entered/consumed so a decline can never become quiet. Belegstufe:
DESK-BEWIESEN -- the metal proof is the boot.
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 31, 2026
… visible

Boot 631row3 wedged in a three-arc ring 60 s after READY: the chain hop
precedes the frame by the upstream's whole forward, so the probe was
correctly False at plan time, the loop then parked in the BLOCKING chain
receive, and PP0 parked in the output receive downstream never fed
(py-spy: PP1 in PpChainReceiver.recv <- recv_requests, PP0 in _do_recv <-
_pp_commit_send_output_work). The pre-row world never hit this because a
downstream rank with work planned locally and parked on the PROXY wire.
Row mode now does the same parking on the same wire: probe False + visible
work (queue/chunked/slots/running) -> the budgeted soft readiness wait
(30 s, declines loudly per sgl-project#1002), then re-probe and route by stamp. An
idle rank has no visible work and still parks on the chain -- the PP3-solo
shape stays deadlock-free. Matched check: py_compile. Metal proof is the
boot.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants