Skip to content

Improve logging & add logit cap - #471

Merged
merrymercy merged 1 commit into
mainfrom
grok
May 24, 2024
Merged

merrymercy merged 1 commit into
mainfrom
grok

Conversation

@merrymercy

Copy link
Copy Markdown
Collaborator

No description provided.

@merrymercy
merrymercy merged commit 2cea614 into main May 24, 2024
@merrymercy
merrymercy deleted the grok branch May 24, 2024 10:48
timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025
vschandramourya pushed a commit to vschandramourya/sglang that referenced this pull request Feb 3, 2026
…ject#471)

* Copy OSS code from f0e948a on 20251209

* update ci_installl_dependency.sh

* update test time

* upd

* fix from oss

* update perf test time for new node

* apply oss fix

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: gongwei-130 <weigong@research-secure-b200-01.cloud.together.ai>
efschu added a commit to efschu/htsglang that referenced this pull request Aug 3, 2026
…utation, sgl-project#471/sgl-project#33407/sgl-project#33312/sgl-project#33098/sgl-project#33271 ports, sgl-project#481 PP fixes

Eight commits, largest bundle this session:

- sgl-project#479 (two commits): the GGUF-MoE offload coverage test and its refusal
  message named MXFP4/type 39 as THE example of a ggml type with no MoE
  kernel -- sgl-project#398 made that false (type 39 has a full native kernel set on
  a wheel that carries it). Traced the ACTIVE DSV4-Flash MXFP4 path end to
  end and confirmed it is native, not an untraced fallback; the refusal
  message now names the layer's own declared types instead of a stale
  hardcoded example, so it cannot go stale again the same way.
  docs/dev/NOTE_479_mxfp4_active_driver_path.md has the trace.
- sgl-project#471: ports upstream sglang#32320 -- mask the SM120 SWA page-split to
  touched pages only, instead of the full page range.
  docs/dev/TICKET_471_masked_page_split.md.
- sgl-project#33407 port: SM120 sparse-MLA buckets the topk width instead of using
  it unbucketed, falling back to Triton outside the bucketed range.
- sgl-project#33312 port: DSpark draft head resolves and loads fused shared experts.
- sgl-project#33098 port: DSpark draft fills the EP token-accounting fields.
- sgl-project#481: three pipeline-parallel defects from the sgl-project#445 window fixed in
  server_args.py/resident_fraction.py/expert_stats.py.
- sgl-project#33271 port: auto-round GPTQ MoE completes the MoeWNA16 delegation.

FEATURE_CATALOG.md §8 conflict-risk as flagged, but no actual conflict --
this branch's base (a5eff26) is now several merges behind (sgl-project#520, sgl-project#523,
design-453 landed since), yet none of those touched §8, so git's 3-way
merge combined everything automatically. All 17 sections and all 22
previously-preserved additive paragraphs verified present.

**Explicit verification requested and confirmed**: the two "ancient
pre-existing" failures in tests/moe_offload/test_gguf_moe_offload.py
(test_materialize_declines_uncovered_ggml_type,
test_guard_still_refuses_an_unstaged_gguf_layer) that this session
bisected back to before this session even began -- sgl-project#479's fix to
expert_offload.py and the type-39 assumption in gguf.py directly
addresses their root cause. Full file run: 20 passed, 0 failed, both
confirmed GREEN now.

One test-isolation quirk found and run to ground, not a code bug: the
three flash_mla test files this branch adds/touches
(test_flash_mla_backends.py, test_flash_mla_page_split_mask_471.py,
test_flash_mla_sm120_topk_buckets.py) interfere with each other when run
in the same pytest process -- a shared Triton-decorated function object
resolves to JITFunction instead of the expected InterpretedFunction once
another file in the same process has already imported flash_mla_sm120.py
under real (non-interpret) conditions. Verified this is NOT a merge
regression: reproduced identically on the unmerged branch tip in
isolation (fresh archive, no other code involved), and every one of the
30 tests across the three files passes cleanly when its file is run
alone. Branch-inherent test-ordering fragility, not in scope for this
merge to fix.

Tests (CUDA_VISIBLE_DEVICES=99, PYTHONPATH pinned to this worktree):
- tests/moe_offload/test_gguf_moe_offload.py alone: 20 passed (was 18
  passed/2 failed before this merge, per this session's own bisection).
- The eight new/touched test files, run individually where the isolation
  quirk above requires it: all pass in full (dspark 442: 18; auto_round:
  17; dspark shared-expert 33312: 21; gguf mxfp4 479: 18; pp defects 481:
  26; flash_mla backends/topk-buckets/page-split-mask: 10/25/10 each
  alone).
- test/registered/unit/quantization/ + unit/models/ + spec/dspark/: 28
  failed both before and after (diffed by test ID, not count -- the
  pre-merge archive showed 35 because 7 of its failures were an artifact
  of an incomplete git-archive extraction missing sgl-kernel/, confirmed
  by re-checking those 7 IDs against the working tree directly). The
  remaining 28 are dspark_kernel_parity's CUDA-only subtests plus
  test_gguf_dequant_scratch_budget.py, both pre-named as pre-existing.
- test/registered/unit/layers/ + tests/moe_offload/: 3 failed (the
  pre-named test_deterministic_fp8_gemm.py x2 +
  test_flashinfer_workspace_zeroing.py x1), 1168 passed -- confirms
  test_gguf_moe_offload.py's contribution to this count is now 0 failures.
- test/registered/unit/distributed/ (full, server_args.py touched): 24
  failed, byte-identical established baseline.
- ruff check --select=F401,F821,UP037: 15 findings, all in
  expert_offload.py, all confirmed pre-existing (same set already
  verified at cc5a92e in the earlier sgl-project#494 merge) and outside this
  branch's two diff hunks (2112-2138, 2616-2648) -- auto_round.py and
  every other touched file clean.
- codespell: clean on all 21 touched/added files.
Amrutha-M05 added a commit to Amrutha-M05/sglang that referenced this pull request Nov 2, 2026
…rence test

The XPU shim no longer copies K/V, probes softmax_lse, or rejects flash-attn
options the kernel already handles; sgl-kernel-xpu honours the row stride since
sgl-project#471, checks the device on every call, and accepts causal and window_size.

The VMOBA_ATTN branch of _build_attn_metadata never called build(), so forward
received attn_metadata=None. Wan 2.1 T2V-1.3B now generates end to end.

test_vmoba_selection_gpu runs on any accelerator at head_dim 64, so its torch
reference catches an unwritten softmax_lse on XPU where the shape-only cases
cannot.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant