Skip to content

[Test] runners: Fix the HF embedding reference for sentence-transformers 6 - #48362

Closed
dayanandav wants to merge 1 commit into
sgl-project:mainfrom
dayanandav:st6-embedding-reference
Closed

dayanandav wants to merge 1 commit into
sgl-project:mainfrom
dayanandav:st6-embedding-reference

Conversation

@dayanandav

@dayanandav dayanandav commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

With sentence-transformers 6, test_embedding_models compares against a wrong HF reference:

  • ST 6 renders plain text through the backbone's chat template, while SGLang embeds the raw text.
  • The non-ST branch builds models.Transformer without is_causal=True, so gte-Qwen2 runs bidirectionally. The kwarg was renamed (config_args to config_kwargs) in v6.

Modifications

  • test/runners.py: drop the chat-template ("message") modality for ST >= 6; pass is_causal=True with the version's kwarg name. No-op for ST < 6.
  • test_embedding_models.py: mem_fraction_static=0.85 on XPU only, so the 8B model has enough KV for the ~6.4k-token prompt on a 24 GB card.

Test-only change.

Accuracy Tests

test_embedding_models::test_prefill_logits (gte-Qwen2-1.5B, Qwen3-Embedding-8B, e5-mistral-7b) and test_embedding_encoding_format pass on Arc Pro B60 with sentence-transformers 6.0.1.

Speed Tests and Profiling

N/A (test-only).

🤖 Generated with Claude Code


CI States

Latest PR Test (Base): ❌ Run #41800310855
Latest PR Test (Extra): ❌ Run #41800310771
Latest PR Test (AMD ROCm 10): ❌ Run #41800310767

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

…ers 6

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@dayanandav

Copy link
Copy Markdown
Contributor Author

Superseded by #48372, #48373 and #48374 (regrouped into 3 PRs).

@dayanandav dayanandav closed this Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant