Upstream sync #17

tlrmchlsmth · 2025-05-01T21:11:32Z

No description provided.

Signed-off-by: youkaichao <[email protected]>

Signed-off-by: Michal Adamczyk <[email protected]> Signed-off-by: Chendi Xue <[email protected]> Co-authored-by: Michal Adamczyk <[email protected]>

Signed-off-by: reidliu41 <[email protected]> Co-authored-by: reidliu41 <[email protected]>

…16902) Signed-off-by: vllmellm <[email protected]>

…ht (vllm-project#16807) Signed-off-by: Nick Hill <[email protected]>

…shape (metadata_size) (vllm-project#16998) Signed-off-by: Lucas Wilkinson <[email protected]>

…7024) Signed-off-by: gitover22 <[email protected]>

…ed for Mistral tokenizers (vllm-project#16964) Signed-off-by: chaunceyjiang <[email protected]>

Signed-off-by: Russell Bryant <[email protected]>

Signed-off-by: mgoin <[email protected]>

…ect#17042) Signed-off-by: windsonsea <[email protected]>

…llm-project#17051) Signed-off-by: Harry Mellor <[email protected]>

Signed-off-by: Russell Bryant <[email protected]>

Signed-off-by: Harry Mellor <[email protected]>

…ct#17052) Signed-off-by: Harry Mellor <[email protected]>

…m-project#17053) Signed-off-by: Harry Mellor <[email protected]>

…y with xgrammar (vllm-project#15949) Signed-off-by: Travis Johnson <[email protected]>

…llm-project#16806)

…roject#16899) Signed-off-by: Woosuk Kwon <[email protected]>

Signed-off-by: csy1204 <[email protected]> Co-authored-by: Michael Goin <[email protected]>

…7017)

…t#17073) Signed-off-by: Woosuk Kwon <[email protected]>

Signed-off-by: mgoin <[email protected]> Signed-off-by: Nick Hill <[email protected]> Co-authored-by: Nick Hill <[email protected]>

…16988) Signed-off-by: Harry Mellor <[email protected]>

…project#17043) Signed-off-by: sydarb <[email protected]>

…when s… (vllm-project#16472) Signed-off-by: 开哲 <[email protected]> Co-authored-by: 开哲 <[email protected]>

Signed-off-by: reidliu41 <[email protected]> Co-authored-by: reidliu41 <[email protected]>

Signed-off-by: Gregory Shtrasberg <[email protected]>

Signed-off-by: Yan Cangang <[email protected]>

…ect#17510) Signed-off-by: mgoin <[email protected]>

…m-project#17500) Signed-off-by: mgoin <[email protected]>

Signed-off-by: DarkLight1337 <[email protected]>

Signed-off-by: Alexei V. Ivanov <[email protected]>

Signed-off-by: Russell Bryant <[email protected]>

…d line (vllm-project#17229) Signed-off-by: Hongxia Yang <[email protected]>

…abled (vllm-project#15506) Signed-off-by: Noah Yoshida <[email protected]>

…project#17494) Signed-off-by: Lucas Wilkinson <[email protected]>

Signed-off-by: Keyun Tong <[email protected]>

) Signed-off-by: Russell Bryant <[email protected]>

…llm-project#17515) Signed-off-by: chaunceyjiang <[email protected]>

Signed-off-by: Harry Mellor <[email protected]>

…ct a model (vllm-project#17285) Signed-off-by: Teruaki Ishizaki <[email protected]>

Signed-off-by: DarkLight1337 <[email protected]>

…vllm-project#17530) Signed-off-by: tjtanaa <[email protected]>

Signed-off-by: Huy Do <[email protected]>

Signed-off-by: reidliu41 <[email protected]> Co-authored-by: reidliu41 <[email protected]>

…ig (vllm-project#17535) Signed-off-by: tjtanaa <[email protected]>

Signed-off-by: chaunceyjiang <[email protected]>

…orted list (vllm-project#17536) Signed-off-by: Hongxia Yang <[email protected]>

…subsequent scaled_fp8_quant operations (vllm-project#10867) Signed-off-by: Sage Moore <[email protected]>

Signed-off-by: reidliu41 <[email protected]> Co-authored-by: reidliu41 <[email protected]>

Signed-off-by: Tyler Michael Smith <[email protected]>

…tion_merge_main

github-actions · 2025-05-01T21:11:42Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

chaunceyjiang and others added 30 commits April 22, 2025 19:22

Revert "[Misc] Add S3 environment variables for better support of Min…

6bc1e30

…IO." (vllm-project#17021)

[misc] tune some env vars for GB200 (vllm-project#16992)

e1cf90e

Signed-off-by: youkaichao <[email protected]>

[INTEL-HPU][v0] Port delayed sampling to upstream (vllm-project#16949)

56a7352

Signed-off-by: Michal Adamczyk <[email protected]> Signed-off-by: Chendi Xue <[email protected]> Co-authored-by: Michal Adamczyk <[email protected]>

[doc] add download path tips (vllm-project#17013)

eb8ef42

Signed-off-by: reidliu41 <[email protected]> Co-authored-by: reidliu41 <[email protected]>

[Bugfix] Triton FA function takes no keyword arguments (vllm-project#…

047797e

…16902) Signed-off-by: vllmellm <[email protected]>

[V1] Avoid socket errors during shutdown when requests are in in-flig…

b2f195c

…ht (vllm-project#16807) Signed-off-by: Nick Hill <[email protected]>

[BugFix] llama4 fa3 fix - RuntimeError: scheduler_metadata must have …

d0da99f

…shape (metadata_size) (vllm-project#16998) Signed-off-by: Lucas Wilkinson <[email protected]>

[Misc] Improve readability of get_open_port function. (vllm-project#1…

ec69124

…7024) Signed-off-by: gitover22 <[email protected]>

[Bugfix] Fix AssertionError: skip_special_tokens=False is not support…

8c87a9a

…ed for Mistral tokenizers (vllm-project#16964) Signed-off-by: chaunceyjiang <[email protected]>

[CI] Run v1/test_serial_utils.py in CI (vllm-project#16996)

ce17db8

Signed-off-by: Russell Bryant <[email protected]>

Mistral-format support for compressed-tensors (vllm-project#16803)

aa72d9a

Signed-off-by: mgoin <[email protected]>

Categorize tests/kernels/ based on kernel type (vllm-project#16799)

6317a51

Signed-off-by: mgoin <[email protected]>

[Doc] Add top anchor and a note to quantization/bitblas.md (vllm-proj…

f7912cb

…ect#17042) Signed-off-by: windsonsea <[email protected]>

Ensure that pid passed to kill_process_tree is int for mypy (v…

53c0fa1

…llm-project#17051) Signed-off-by: Harry Mellor <[email protected]>

[CI] Update structured-output label automation (vllm-project#17055)

af869f6

Signed-off-by: Russell Bryant <[email protected]>

Improve Transformers backend model loading QoL (vllm-project#17039)

8e630d6

Signed-off-by: Harry Mellor <[email protected]>

CacheConfig.block_size should always be int when used (vllm-proje…

f3a21e9

…ct#17052) Signed-off-by: Harry Mellor <[email protected]>

Use @property and private field for data_parallel_rank_local (vll…

bdb3660

…m-project#17053) Signed-off-by: Harry Mellor <[email protected]>

[Frontend] Support guidance:no-additional-properties for compatibilit…

3cde34a

…y with xgrammar (vllm-project#15949) Signed-off-by: Travis Johnson <[email protected]>

[BugFix][V1] Fix int32 token index overflow when preparing input ids (v…

32d4b66

…llm-project#16806)

[V1][Spec Decode] Always use argmax for sampling draft tokens (vllm-p…

41fb013

…roject#16899) Signed-off-by: Woosuk Kwon <[email protected]>

[CI/Build] workaround for CI build failure (vllm-project#17070)

b07d741

Signed-off-by: csy1204 <[email protected]> Co-authored-by: Michael Goin <[email protected]>

[Quantization]add prefix for commandA quantized model (vllm-project#1…

6b2427f

…7017)

[Minor] Use larger batch sizes for A100/B100/B200/MI300x (vllm-projec…

46e678b

…t#17073) Signed-off-by: Woosuk Kwon <[email protected]>

[Bugfix] Enable V1 usage stats (vllm-project#16986)

ed50f46

Signed-off-by: mgoin <[email protected]> Signed-off-by: Nick Hill <[email protected]> Co-authored-by: Nick Hill <[email protected]>

More informative error when using Transformers backend (vllm-project#…

2c8ed8e

…16988) Signed-off-by: Harry Mellor <[email protected]>

Addendum Fix to support FIPS enabled machines with MD5 hashing (vllm-…

ed2e464

…project#17043) Signed-off-by: sydarb <[email protected]>

[Bugfix][Core] add seq_id_to_seq_group clearing to avoid memory leak …

6167c0e

…when s… (vllm-project#16472) Signed-off-by: 开哲 <[email protected]> Co-authored-by: 开哲 <[email protected]>

[V1] Update structured output (vllm-project#16812)

db2f8d9

Signed-off-by: reidliu41 <[email protected]> Co-authored-by: reidliu41 <[email protected]>

[doc] update to hyperlink (vllm-project#17096)

9c1244d

Signed-off-by: reidliu41 <[email protected]> Co-authored-by: reidliu41 <[email protected]>

gshtras and others added 27 commits April 30, 2025 19:51

[Bugfix][ROCm] Fix import error on ROCm (vllm-project#17495)

08fb558

Signed-off-by: Gregory Shtrasberg <[email protected]>

[Bugfix] Temporarily disable gptq_bitblas on ROCm (vllm-project#17411)

1144a8e

Signed-off-by: Yan Cangang <[email protected]>

[CI][TPU] Skip structured outputs+spec decode tests on TPU (vllm-proj…

17b4d85

…ect#17510) Signed-off-by: mgoin <[email protected]>

[CI][Bugfix] Fix failing V1 Test due to missing 'cache_salt' arg (vll…

aa4502e

…m-project#17500) Signed-off-by: mgoin <[email protected]>

[CI/Build] Reorganize models tests (vllm-project#17459)

afb4429

Signed-off-by: DarkLight1337 <[email protected]>

FIxing the AMD test failures caused by PR#16457 (vllm-project#17511)

7ab643e

Signed-off-by: Alexei V. Ivanov <[email protected]>

[Build] Require setuptools >= 77.0.3 for PEP 639 (vllm-project#17389)

7a0a146

Signed-off-by: Russell Bryant <[email protected]>

[ROCm] Effort to reduce the number of environment variables in comman…

90d0a54

…d line (vllm-project#17229) Signed-off-by: Hongxia Yang <[email protected]>

[BugFix] fix speculative decoding memory leak when speculation is dis…

13cf6b6

…abled (vllm-project#15506) Signed-off-by: Noah Yoshida <[email protected]>

[BugFix] Fix mla cpu - missing 3 required positional arguments (vllm-…

3c3d767

…project#17494) Signed-off-by: Lucas Wilkinson <[email protected]>

Avoid overwriting vllm_compile_cache.py (vllm-project#17418)

26bc4bb

Signed-off-by: Keyun Tong <[email protected]>

[Core] Enable IPv6 with vllm.utils.make_zmq_socket() (vllm-project#16506

fbefc8a

) Signed-off-by: Russell Bryant <[email protected]>

[Misc] Optimize the Qwen3_ReasoningParser extract_reasoning_content (v…

015069b

…llm-project#17515) Signed-off-by: chaunceyjiang <[email protected]>

Improve configs - ObservabilityConfig (vllm-project#17453)

a257d9b

Signed-off-by: Harry Mellor <[email protected]>

[Bugfix][Benchmarks] Allow benchmark of deepspeed-mii backend to sele…

86a1f67

…ct a model (vllm-project#17285) Signed-off-by: Teruaki Ishizaki <[email protected]>

[Frontend] Show progress bar for adding requests (vllm-project#17525)

1903c0b

Signed-off-by: DarkLight1337 <[email protected]>

[Misc] Clean up test docstrings and names (vllm-project#17521)

48e925f

Signed-off-by: DarkLight1337 <[email protected]>

[FEAT] [ROCm]: Add Qwen/Qwen3-30B-A3B-FP8 fused moe config for MI300X (…

2007d4d

…vllm-project#17530) Signed-off-by: tjtanaa <[email protected]>

Fix more broken speculative decode tests (vllm-project#17450)

b74d888

Signed-off-by: Huy Do <[email protected]>

[doc] add streamlit integration (vllm-project#17522)

7169f87

Signed-off-by: reidliu41 <[email protected]> Co-authored-by: reidliu41 <[email protected]>

[FEAT] [ROCm]: Add Qwen/Qwen3-235B-A22B-FP8 TP4 triton fused moe conf…

f5a3c65

…ig (vllm-project#17535) Signed-off-by: tjtanaa <[email protected]>

[Feature][Frontend]: Deprecate --enable-reasoning (vllm-project#17452)

98060b0

Signed-off-by: chaunceyjiang <[email protected]>

[ROCm] remove unsupported archs from rocm triton flash-attention supp…

28566d7

…orted list (vllm-project#17536) Signed-off-by: Hongxia Yang <[email protected]>

[torch.compile] Add torch inductor pass for fusing silu_and_mul with …

460a2b1

…subsequent scaled_fp8_quant operations (vllm-project#10867) Signed-off-by: Sage Moore <[email protected]>

[Misc] refactor example - cpu_offload_lmcache (vllm-project#17460)

7423cf0

Signed-off-by: reidliu41 <[email protected]> Co-authored-by: reidliu41 <[email protected]>

Merge branch 'main' into nixl_integration_merge_main

e785a42

Signed-off-by: Tyler Michael Smith <[email protected]>

Merge remote-tracking branch 'tms/nixl_integration' into nixl_integra…

28833b2

…tion_merge_main

tlrmchlsmth requested a review from robertgshaw2-redhat as a code owner May 1, 2025 21:11

tlrmchlsmth merged this pull request into nixl_integration May 1, 2025
2 of 4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Upstream sync #17

Upstream sync #17

Uh oh!

tlrmchlsmth commented May 1, 2025 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented May 1, 2025

Uh oh!

Uh oh!

Uh oh!

Upstream sync #17

Upstream sync #17

Uh oh!

Conversation

tlrmchlsmth commented May 1, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 1, 2025

Uh oh!

Uh oh!

Uh oh!

tlrmchlsmth commented May 1, 2025 •

edited by github-actions bot

Loading