Repository navigation
Fix RuntimeEndpoint - #279
Merged
Merged
Conversation
timethink
pushed a commit
to timethink/sglang
that referenced
this pull request
Mar 9, 2025
Garrybest
pushed a commit
to Garrybest/sglang
that referenced
this pull request
Jan 9, 2026
vschandramourya
pushed a commit
to vschandramourya/sglang
that referenced
this pull request
Feb 3, 2026
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
liuxianglong17
pushed a commit
to shun8686/sglang
that referenced
this pull request
Apr 8, 2026
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Jul 28, 2026
…weiter Comm-Arbiter — lane-keyed, prioritaetsbewusst, graph-safe
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Jul 29, 2026
…spitze auf NEXTN, konfigurierbar (Sensor aus D1/sgl-project#279, Aktor Plan-Flip, ein Politik-Objekt mit Aktoren Algo/K/topk, sgl-project#141 verwandt)
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Jul 29, 2026
…register.py — generisches VRAM-Posten-Tiering (Klassen graph_rungs/drafter_heads/lane_workspaces/cold_lane/experts, Regler resident|ram|auto + fraction, Presets --lane-offload-profile, phase_mask + Zeitkonstanten-Stufen wave|phase|turn, on_phase_boundary-No-Op-Planer mit injizierbarer Overlap-Kostenfunktion, Latenz-Term-API fuer sgl-project#279); Invarianten (Hysterese/Hot-Ablehnung/Prio-Schutz/Fraction-Kappe) in park() und Planer erzwungen; Adapter hinter SGLANG_OFFLOAD_REGISTER=1, Default-Pfad unveraendert; Audit-Tabelle Experten-Maschinerie in INTEGRATION_R3_VALIDATION; Tests 42 neu + 152 Concurrency gruen, ruff/codespell sauber
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Jul 29, 2026
…t): htccl_path_dispatcher.py (PathProfile affine ms(bytes) + Kapazitaet + effektive Apertur + measured|placeholder je Eintrag; decide() je Klasse/Groesse/Lane/Prio mit Saettigungs-Ueberlauf, Latch je log2-Band, Flips nur an Runden-/Capture-Grenzen) + htccl_path_rates.py (Parser p2p_readiness NUR Effektivwerte, sgl-project#278-TSV, NCCL-Referenz-Schema v1 mit Pflicht-p99+Last-Arm); harte Regeln je an einer Stelle erzwungen: Placeholder-Neutralitaet (ohne measured-Daten byte-identische sgl-project#240-Wahl, per Identitaets-Test ueber 3x3x4x4 belegt), Graph-Safety, Prio-Schutz, Offload-Latenz-Term-Anschluss; Bus-Begriff mit sgl-project#286-Arbiter geteilt via benanntem Sensor-Adapter; SGLANG_HTCCL_PATH_DISPATCHER Default aus; 46 neue Tests, distributed-Suite ohne neuen Fehlschlag
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Jul 29, 2026
…terie (s00 preflight -> s01 P2P-Re-Probe -> s02-s05 R7c-Boots mit Report-Gate nach A -> s06 NCCL-Referenz -> s07 Register-GPU -> s08 Dispatcher-Tabellen+Neutralitaet -> s09 Sensor-Smoke; ~3h28 erwartet, harte Budgets je Schritt), run_step.sh mit Resume-Gate/Korridor/Locks/Timeout/py-spy/Verdikt, battery_state.py (state.json, Artefakt- statt Reihenfolge-Abhaengigkeiten, BLOCKED statt stillem Skip, --force/--only/--from mit history), 10 inhaltliche Checks (Positionskurve+Referenzspalte, echte sgl-project#279-Lader, CudaDeviceOps-Identitaet + parked-Zustandsfolge, werfende Hooks als Neutralitaetsbeweis), EXECUTOR_PROTOCOL (haiku Skript-/sonnet Boot-Schritte, FAIL/STOP = anhalten ohne Debug) + HANDOFF_TEMPLATE fuer den opus-Fixer; 121 hermetische Trockentests (Checks als Subprozess gegen PASS/FAIL/STOP-Fixtures, no-P2P-Rig-Toleranz gefixt); Vorbehalt dokumentiert: s06/s07/s09 sind neue Messskripte, Erstlauf beaufsichtigen
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Jul 29, 2026
… integriert + all_gather-Stopper geschlossen — Merge-Ordnung in _select ist die Substanz (handles() sgl-project#240 -> refine sgl-project#279 -> lauter Riegel zuletzt, sonst entkaeme HINT_GLOO-in-Capture); all_gather als AG-Phase des a2a-Kerns (kein neuer Kernel: konstanter Sende-Offset, erbt Byte-Beleg/Runden-Paritaet/ragged-tail; Shard > Slot laeuft in ceil(shard/slot) Runden statt Ablehnung — 10.600.448 B = 2 Runden; graph-safe weil Rundenzahl gruppen-uniform eingebrannt, Direktmodus bewusst NICHT), uneven-Shards als Laengenvektor; reduce_scatter/broadcast bleiben hinterm Riegel mit begruendeter Meldung aus HTCCL_OPS statt Literalliste; drei Fork-Bugfixes als erste Commits (--help %-Maskierung, scheduler hicache, test_utils CVD=leer) mit Falsifikatoren; Bonus belegt offline: bar1_netz_kernel STACK 64->0 (ring/a2a byte-identisch); F811-Fund + Signatur-Fix; distributed 949 passed / 16 vorbestehend, Dispatcher-46 unveraendert; GPU-Checkliste in INTEGRATION_R3_VALIDATION §BAR1 + Runbook §4.15
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 2, 2026
Desk design only, no cards (CUDA_VISIBLE_DEVICES=99), no implementation. Verdict: sgl-project#407 is not a new abstraction, it is a NODE layer plus a reconciliation. The edge half — what does it cost to move a byte from here to there — already exists FOUR times with four readers and four absent-value rules: cost_model.PairMatrix (#348b, UUID-keyed), barlink_path_rates. PathProfile (sgl-project#279, affine per route), offload_movement.PeerPathCapability (sgl-project#286, CUDA-ordinal-keyed) and card_probe h2d/d2h (sgl-project#271). E2 and E3 both want the effective BAR1 aperture from p2p_readiness/capability_matrix.json and only E2 has a loader. What exists nowhere is the node layer, and in its place the tree carries four disjoint tier vocabularies (PARK_TARGETS sgl-project#286; LOCAL_DESTINATION/SUPPORTED_PARK_BACKENDS sgl-project#224; StorageMedium telemetry; HiCache choices=). Expert offload, the largest byte mover in the fork, is in none of them: its tier is device="cpu" + .pin_memory() written out five times. Two shipped contradictions the design bends around, both stated up front: C1 — two opposite total orders over one axis. sgl-project#224's destinations_error makes 'local' first a hard validation law ("a remote tier is a tier BELOW local, never above it"); sgl-project#286's ladder puts peer_vram ABOVE host_ram. Resolved by making the order a shortest path over a STAGING GRAPH: 'local first' is recovered as reachability (the only edge out of a device tier toward a blob tier runs through host), peer_vram is recovered because vram->vram is a direct edge. Neither consumer's shipped behaviour changes. C2 — the one existing tier ledger is positionally keyed. CapacityLedger keys peer-VRAM budgets by CUDA ordinal; AUDIT_331 calls that SCOPED and legitimate until it crosses a process boundary, which is exactly what sgl-project#305 §6 ("one ledger, no second accounting") asks for. On this rig the 5090 is CUDA ordinal 0 and NVML index 1, so the failure is silent card-swapping. Honesty correction on three numbers the brief assumed measured: the peer-VRAM 1-3 us posted-write class is an explicit ASSUMPTION (EVAL_p2p §"Tragende Annahmen"), NVMe latency does not exist in the tree in any unit, and host DRAM 38 GB/s is derived from an ASSUMED DDR4-3200 peak (the VM hides SPD). They ship as ESTIMATE/ABSENT under #348b's measured-or-named rule. Everything else in the inventory is sourced: 5090 32607 MiB / 3080 20480 MiB, membw 1558/723 GB/s, BAR1 256 MiB nominal but 96 MiB contiguous with 1.13-1.34x vs NCCL at 20 KiB-16 MiB, H2D 6.4/13/13 GB/s gen4 x4/x8/x8, NVMe 1.8 GB/s cold (ARC rows excluded), rig-2 2.07-2.83 GB/s with 1.47 us at 8 B and ~89/202 us serving collectives. Consumer survey, eight consumers read in their target-selection code: sgl-project#77/sgl-project#123 expert offload MECHANICAL for the UUID-vector half (it revives sgl-project#394's merged-but-starved link-proportional sharding, 145 -> 86 ms/token, 1.69x) and REBUILD for a third rung (two dicts, binary resident|spill, UVA aliasing); HiCache MECHANICAL for backends, ladder EXCLUDED (it is the type signature of every controller method, not configuration); sgl-project#224 MECHANICAL for the list, REBUILD for the ordering law, victim half untouched; sgl-project#89 MECHANICAL-PLUS (needs a persistence class or a tmpfs --hibernate-dir is a silent regression, and a driver-free arg-parse-time query); sgl-project#305 NO MIGRATION, it is the awaited mechanism; sgl-project#286 MECHANICAL and the donor — lift _select_target and CapacityLedger rather than write new ones; sgl-project#389 and sgl-project#306 greenfield. Cut plan, no big bang: 1 registry + read-only enumeration + GateRow dashboard row (S); 2 publish the rank->card-UUID vector without a collective so sgl-project#394 comes alive (S, the only cut whose yield is already measured); 3 sgl-project#224 consumes it (S-M); 4 CapacityLedger becomes a view of registry/ledger.py, keys widen to TierId (M); 5-9 expert offload, hibernate, sgl-project#305 transitions, HiCache backends, NVMe/sgl-project#306 declaration. Exclusions with hard reasons only: the HiCache ladder, GDN state (DEVICE_BOUND_ONLY admission, a refusal not an omission), cross-rig GPU-to-GPU (driver guard, sgl-project#280 closed in userspace), compute placement (sgl-project#302). Measurement plan names six gaps, five of them S, three of them new arms in the existing comm-suite which today has no BAR1, storage or host-DRAM arm. codespell clean; ruff not applicable (documentation only).
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 2, 2026
…r plus a reconciliation, not a new abstraction — edge costs already exist FOUR times (PairMatrix #348b, PathProfile sgl-project#279, PeerPathCapability sgl-project#286 on CUDA ordinals, card_probe sgl-project#271) with four absent-value rules and no reconciler, while four disjoint tier vocabularies coexist and the largest byte mover (expert offload) is in none of them; honesty corrections shipped as ESTIMATE/ABSENT (peer-VRAM us-class is an assumption, NVMe latency unmeasured in any unit, DRAM derived from assumed DDR4-3200); contradictions resolved in design: local-first vs peer-vram-above-host becomes shortest-path reachability over a staging graph (neither consumer changes), the positionally-keyed CapacityLedger named as the silent-card-swap hazard at process boundaries; TenantState frozen (client-visible enum mixing tiers into policy) with orthogonal resident_tier; cut plan S-first with cut 2 = publish rank->UUID vector WITHOUT a collective — revives sgl-project#394, the only cut with measured yield (1.69x) (sgl-project#407)
This was referenced Oct 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix bugs after #230