Skip to content

Fix llava parallelism/fork bug - #315

Merged
merrymercy merged 4 commits into
sgl-project:mainfrom
lockon-n:main
Mar 29, 2024
Merged

merrymercy merged 4 commits into
sgl-project:mainfrom
lockon-n:main

Conversation

@lockon-n

Copy link
Copy Markdown
Contributor

Associated with #260

@merrymercy merrymercy self-assigned this Mar 22, 2024
@merrymercy
merrymercy merged commit cb389c9 into sgl-project:main Mar 29, 2024
timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025
efschu added a commit to efschu/htsglang that referenced this pull request Jul 30, 2026
…s15 consumer regexes realigned to the real English emitters (6 dead German markers fixed across 9+ files, incl. one check s13 asserted but nothing ever harvested), fixtures REGENERATED from emitter format strings via ast (_bar1_marker_source.py) with provenance notes, new coupling test feeds synthetic emissions through live consumer regexes (14 tests) so the next rename breaks loudly; battery locks now disk-owned — heartbeat_pid in the lock info file, exec -a identity, release scans by step tag with bracket-idiom pid identity check against recycling, idempotent, cross-process proven (7 tests, acquire in A release in B); 299 battery-check tests unchanged, s12 4 failures pre-existing on base
efschu added a commit to efschu/htsglang that referenced this pull request Jul 31, 2026
…h markers match live (9/9/9 hits for setup/ledger/ACHIEVED in harvested evidence, three real lines quoted; same regexes score 0/0/0 against the 2026-07-30 German logs — coupling measured both directions), s11 PASS in 238s with all 9 gate cases, ACHIEVED=bar1 on dcp/tp/world, accept 3.10, no residual find (gloo mentions are hint text inside the ACHIEVED line); sgl-project#314 card-proven — locks acquired in shell A released from fresh shell B via disk-owned state (empty process memory verified), heartbeat dead, idempotent, no orphans
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 5, 2026
…by line number

REGRESSION I CAUSED, and the fragility that let one line do so much damage.

_bar1_marker_source.py parses the real emitter sources with `ast` and pulls
the format strings out of them -- a good design, built precisely so BAR1 log
fixtures cannot drift into hand-typed copies (sgl-project#315, the sgl-project#380 family). But it
located each call by ABSOLUTE LINE NUMBER (LINE_BAR1_SETUP = 2236, ...).

That coupled the module to the LAYOUT of files it does not own. Adding the
collective-census counter to parallel_state.py shifted line 787, the AST
lookup found no call there, and the assertion fired at IMPORT -- taking the
whole BAR1 marker suite down at COLLECTION (test_gpu_battery_checks_bar1.py
+ test_bar1_marker_coupling.py, 136 tests).

Note the actual trigger was the census BUMPS in parallel_state.py, not the
abort-handler edit in barlink_bar1.py: the failing assertion names
parallel_state.py:787.

A line number is the same fragility class as a hand-typed fixture string: a
second, unmaintained copy of a fact the source already states. So the
anchors are now the MARKER TEXT itself (MARK_* instead of LINE_*), resolved
by `_call_with_marker`: the unique call whose first argument contains the
fragment. For the exception class the anchor is its DOCSTRING, not its name
-- anchoring on the name would make `bar1_unavailable_class_name` a
tautology, when its whole job is to report a rename to the consumers.

This is strictly better coupling, not weaker: it survives every edit that
does not touch the marker, and still fails loudly on the edit that does --
which is the drift the module exists to catch. Ambiguity (a marker matching
two calls) is a loud error naming both, never a silent first-match.

Tests:
  * test_gpu_battery_checks_bar1.py + test_bar1_marker_coupling.py: 136
    passed (were: collection error)
  * robustness proof: inserting 40 unrelated lines above the old pin in
    parallel_state.py -- the exact edit class that broke it -- still 136
    passed
  * can-fail proof: renaming the emitted marker ("barlink enabled for
    group " -> "barlink ACTIVATED for group ") fails loudly with a message
    naming the file, the marker and both things to update
  * WHOLE test/registered/unit/distributed directory: 2370 collected, 0
    collection errors, 2358 passed / 12 skipped / 726 subtests passed
  * the remaining 3 subtest failures (test_dcp_context_ceiling.py,
    '_PoolRowStub' object has no attribute 'server_args') are PRE-EXISTING,
    not from this branch: verified identical at 38e7c69 in a throwaway
    worktree (3 failed, 17 passed there too). The call site is
    flashinfer_backend.py:691, which this branch does not touch.

Process note: this slipped through because I ran only my own test files.
The directory-wide run is the check that would have caught it, and it is
what I ran here.
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 17, 2026
…ards

test_bar1_strand_is_opt_in.py's import probe parsed stdout with
set(proc.stdout.split()) -- every whitespace token became a "module". On a
heterogeneous rig the runtime prints

  WARNING [cuda.py:962] Detected different devices in the system: NVIDIA
  GeForce RTX 3080, NVIDIA GeForce RTX 5090, NVIDIA GeForce RTX 3080. Please
  make sure to set `CUDA_DEVICE_ORDER=PCI_BUS_ID` ...

to stdout, so "Detected", "WARNING" and "3080," entered the set and
assertEqual(..., set()) could never hold. The guard could not pass on the one
rig it matters for, which means it protected nothing there: a real eager
import of the transport would have been indistinguishable from the noise.
_clean_env() sets CUDA_VISIBLE_DEVICES=99, but the warning is emitted from the
NVML enumeration regardless, so the existing isolation did not suppress it.

The sgl-project#315 lesson applied: anchor the consumer regex to the STRUCTURE of the
line, not to whatever the producer happened to print. The probe now emits one
module per line under a MODULE: prefix and the parser is
re.compile(r"^MODULE:(\S+)$", re.MULTILINE) -- prose cannot produce a match at
line start. All three call sites go through one _probe_code() builder so the
report format has a single definition.

Five tests added, three of them can-fail proofs rather than assertions of
success:
- the rig's warning is planted VERBATIM and must contribute nothing;
- the OLD parser is reconstructed and asserted to turn that same line into
  20+ phantom modules, so a future cleanup back to .split() cannot pass
  silently on a quiet rig;
- a module name embedded in prose, indented, or prefixed must NOT count, so
  the anchor is proved to be an anchor;
- and, most importantly, a subprocess that genuinely imports the transport is
  asserted to come back NON-EMPTY through the same parser -- the obvious way
  to make a red test green is to blind it, and this rules that out. The
  == set() assertions are load-bearing, not vacuous.

14 passed on this rig, up from 1 failed. Ruff and codespell clean. Found while
running related suites after 8fa8b67; pre-existing, verified failing at
8bb26dd in a scratch worktree, so not introduced by the peer-transport work.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants