[Misc] mv chat_utils.py to renderers. by noooop · Pull Request #60522 · vllm-project/vllm

noooop · 2026-10-08T02:23:59Z

Overview

mv chat_utils.py to renderers.

After this PR, vllm/entrypoints is nice and clean — let's keep it.

Claims

Validation

Details

Pull Request Checklist

I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).
AI assistance was used during the creation of this PR.
Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

mergify · 2026-10-08T02:24:36Z

Documentation preview: https://vllm--60522.org.readthedocs.build/en/60522/

mergify · 2026-10-08T10:59:38Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @noooop.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

mergify · 2026-10-10T03:49:25Z

Documentation preview: https://vllm--60522.org.readthedocs.build/en/60522/

noooop · 2026-10-10T04:19:32Z

/ci run

github-actions · 2026-10-10T04:19:53Z

✅ Triggered Buildkite CI #93969 for commit 80efa4d84d54.

DarkLight1337

As discussed offline please split up chat_utils.py in the next PR

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

mergify · 2026-10-10T05:32:22Z

Hi @noooop, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

noooop · 2026-10-10T05:33:04Z

/ci run

github-actions · 2026-10-10T05:33:24Z

✅ Triggered Buildkite CI #93987 for commit a5d79ae9d4b0.

… chat_utils helpers from vllm.renderers.chat_utils Root cause: vllm#60522 (c1692c85f1) moved vllm/entrypoints/chat_utils.py to vllm/renderers/chat_utils.py; two vllm-gaudi imports still use the old path Upstream: vllm-project/vllm#60522 Fix: switch load_chat_template and make_tool_call_id imports to vllm.renderers.chat_utils (both defined at pinned sha) Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>

… chat_utils helpers, Drop HPU SeedOss (+5 more) (#1880) This PR consolidates 7 hourly-CI fixes against vllm@`8d7b93fec2eb72ce01f2217472ac1f51b9bce600`. ## Bug 1: Import chat_utils helpers from vllm.renderers.chat_utils - **State machine id**: missing_module_vllm_entrypoints_chat_utils - **Commit**: 6607e1b ### Root cause vllm#60522 (c1692c85f1) moved vllm/entrypoints/chat_utils.py to vllm/renderers/chat_utils.py; two vllm-gaudi imports still use the old path ### Culprit Probable culprit: [PR #60522](vllm-project/vllm#60522) - pinned by symbol archaeology, bisect not run. Candidate range: [compare](vllm-project/vllm@ac7f3e1...8d7b93f). ### Fix switch load_chat_template and make_tool_call_id imports to vllm.renderers.chat_utils (both defined at pinned sha) ## Bug 2: Drop HPU SeedOss override and kernel_block_sizes removed upstream - **State machine id**: missing_module_vllm_model_executor_models_seed_oss - **Commit**: d4aece7 ### Root cause vllm#59701 (6e4efafc6e) deleted vllm.model_executor.models.seed_oss, which vllm_gaudi.models.seed_oss subclassed, so register_model() fails; with that fixed, engine init fails because vllm#57169 (02e55d2af3) removed kernel_block_sizes and the KV-to-kernel block split from MultiGroupBlockTable ### Culprit Probable culprit: [PR #59701](vllm-project/vllm#59701) - pinned by symbol archaeology, bisect not run. Candidate range: [compare](vllm-project/vllm@ac7f3e1...8d7b93f). ### Fix delete pass-through HpuSeedOssForCausalLM and its registration (upstream serves SeedOss via the Transformers backend); stop passing kernel_block_sizes, and raise NotImplementedError when a group's kernel block size differs from its KV block size, the split upstream no longer performs ## Bug 3: Stub get_load_source in offloading connector test mock - **State machine id**: hpu_unit_tests - **Commit**: 56ef9c0 ### Root cause vllm#56318 wraps manager.get_load_source() in CacheHitSource(); our MagicMock manager returns a Mock, so ValueError ### Culprit Probable culprit: [PR #56318](vllm-project/vllm#56318) - pinned by symbol archaeology, bisect not run. ### Fix MockOffloadingSpec manager returns CacheHitSource.EXTERNAL_UNSPECIFIED from get_load_source, mirroring upstream utils.py ## Bug 4: Use vLLM LayerNorm in the regional compilation unit test - **State machine id**: hpu_unit_tests_2 - **Commit**: aa162a6 ### Root cause vllm#57172 replaced torch nn.LayerNorm in OPT with vllm's CustomOp LayerNorm, so the test's torch LayerNorm filter matches nothing ### Culprit Regression introduced by [PR #57172](vllm-project/vllm#57172). ### Fix import LayerNorm from vllm.model_executor.layers.layernorm, which OPT now builds its norms from ## Bug 5: Split KV blocks into kernel blocks in an HPU block table - **State machine id**: multigroup_block_table_kernel_block_sizes_removed - **Commit**: 31c5fa1 ### Root cause vllm#57169 dropped kernel_block_sizes and virtual block splitting from MultiGroupBlockTable/BlockTable; HPU InputBatch still passes it ### Culprit Probable culprit: [PR #57169](vllm-project/vllm#57169) - pinned by symbol archaeology, bisect not run. ### Fix add HPUMultiGroupBlockTable: tables built at kernel block size, manager block ids expanded to kernel ids on append/add_row, replacing the NotImplementedError split guard ## Bug 6: Accept list block ids in the HPU block table - **State machine id**: pre_commit - **Commit**: 8abe42f ### Root cause HPUMultiGroupBlockTable typed add_row/append_row block_ids as tuple[list[int], ...] like upstream, but HPU InputBatch passes the request's list[list[int]], which mypy rejects now that the subclass is local ### Culprit Upstream culprit: not identified - no candidate range recovered (see investigation notes). ### Fix type block_ids as Sequence[list[int]] in add_row, append_row and _to_kernel_blocks; runtime is unchanged ## Bug 7: Accept kv_cache_spec in HPU get_supported_kernel_block_sizes - **State machine id**: hpu_pd_tests - **Commit**: c78bc81 ### Root cause vllm#53175 added kv_cache_spec to get_supported_kernel_block_sizes; attention.py passes it positionally, HPU overrides take none ### Culprit Probable culprit: [PR #53175](vllm-project/vllm#53175) - pinned by symbol archaeology, bisect not run. ### Fix add optional kv_cache_spec=None parameter to both HPU backend overrides, matching the upstream base signature --------- Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

vLLM 0.32 moved chat_utils from vllm.entrypoints to vllm.renderers (vllm-project/vllm#60522) with no shim, so --custom-jinja-template on a chat-style embedding request raised ImportError. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: cloudnathan5 <83739056+cloudnathan5@users.noreply.github.com>

Conflict resolutions and follow-ups for Kimi K3.1: - vllm-project#57169 moved kernel block splitting into AttentionGroup and removed storage_block_size. MergedMLAIndexerSpec drops storage_block_size; FLASHMLA_SPARSE and ROCM_AITER_MLA_SPARSE now report the merged spec's 64-token unit as their kernel block size and align block strides to whole padded units (MultipleOf(64)). MRv2 binds the kernel-block views before splitting the merged MLA / indexer sections. - MergedMLAIndexerSpec.merge carries the new MLAAttentionSpec fields (block_stride_alignment, dcp_sharded, max_tp_shards, topk_buffer_width, cache_role, is_index_group_leader). - Hybrid block size: keep the merged-unit anchor and apply the new backend_block_alignment (lcm with 64) in place of the removed indexer alignment helper. - vllm-project#61455 replaced the trailing-MTP heuristic with draft-layer tags, so the kimi_k31 entry there is dropped. - Mooncake: the hetero-TP fields (kernel block size, kernel blocks per logical block, prefix caching) are sent on every pull, including the layout-less pulls added by vllm-project#59825. - chat_utils moved to vllm/renderers/chat_utils (vllm-project#60522): kimi_k31 added to the tool-call id model types there. - Kimi K3 model: drop the removed DeepSeek-V4 MegaMoE imports and process_weights_after_loading (vllm-project#61538); keep adapt_drafter. - ROCm sparse MLA: init_topk_indices_buffer moved into the base class. - rust multimodal models table: keep upstream's Step3-VL spec and the kimi_k31 alias. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-project-automation Bot added this to Tool Calling Oct 8, 2026

mergify Bot added the needs-rebase label Oct 8, 2026

xijiaat mentioned this pull request Oct 9, 2026

[Bugfix][Frontend] Align DeepSeek V4.1 images with rendered content #60792

Merged

6 tasks done

noooop closed this Oct 10, 2026

noooop force-pushed the mv_chat_utils branch from 762f06c to 2d69083 Compare October 10, 2026 03:18

init

ec8d0bc

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

noooop reopened this Oct 10, 2026

mergify Bot removed the needs-rebase label Oct 10, 2026

noooop marked this pull request as ready for review October 10, 2026 03:50

noooop requested review from DarkLight1337, patrickvonplaten and ywang96 as code owners October 10, 2026 03:50

Merge branch 'main' into mv_chat_utils

80efa4d

DarkLight1337 approved these changes Oct 10, 2026

View reviewed changes

DarkLight1337 enabled auto-merge (squash) October 10, 2026 04:26

github-actions Bot added the ready ONLY add when PR is ready to merge/full CI is needed label Oct 10, 2026

noooop added 2 commits October 10, 2026 13:28

Merge branch 'main' into mv_chat_utils

8f93c75

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

refine

a5d79ae

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

DarkLight1337 merged commit c1692c8 into vllm-project:main Oct 10, 2026
140 checks passed

noooop deleted the mv_chat_utils branch October 10, 2026 06:52

noooop mentioned this pull request Oct 10, 2026

[Misc] Break chat_utils into more maintainable modules. #60972

Merged

6 tasks

pctablet505 mentioned this pull request Oct 10, 2026

[Test][Renderer] Cover the reasoning -> reasoning_content mirror in chat message parsing #56129

Merged

pawel-olejniczak mentioned this pull request Oct 14, 2026

[FIX_FOR_VLLM_CUSTOM=8d7b93fec2eb72ce01f2217472ac1f51b9bce600] Import chat_utils helpers, Drop HPU SeedOss (+5 more) vllm-project/vllm-gaudi#1880

Merged

cloudnathan5 mentioned this pull request Oct 14, 2026

feat(embeddings): accept image, video and audio input on vLLM embedding workers ai-dynamo/dynamo#16447

Open

michael-gpu pushed a commit to practical-inference/vllm-ampere-extended that referenced this pull request Oct 20, 2026

[Misc] mv chat_utils.py to renderers. (vllm-project#60522)

cbcfb38

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

richard-for-bugen Bot mentioned this pull request Oct 20, 2026

[Rust Frontend] Support chat message names BugenZhao/vllm#17

Draft

arpera removed this from Tool Calling Oct 21, 2026

zkawfanx mentioned this pull request Oct 22, 2026

[Start here] Classify the pinned DSv4.1 timeline and prepare chronological backport batches zkawfanx/vllm#18

Open

3 of 5 tasks

Navigation Menu

Repository navigation

[Misc] mv chat_utils.py to renderers. - #60522

[Misc] mv chat_utils.py to renderers.#60522
DarkLight1337 merged 6 commits into
vllm-project:mainfrom
noooop:mv_chat_utils

noooop commented Oct 8, 2026 •

edited

Loading

Uh oh!

mergify Bot commented Oct 8, 2026

Uh oh!

mergify Bot commented Oct 8, 2026

Uh oh!

mergify Bot commented Oct 10, 2026

Uh oh!

noooop commented Oct 10, 2026

Uh oh!

github-actions Bot commented Oct 10, 2026

Uh oh!

DarkLight1337 left a comment

Uh oh!

mergify Bot commented Oct 10, 2026

Uh oh!

noooop commented Oct 10, 2026

Uh oh!

github-actions Bot commented Oct 10, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

noooop commented Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Claims

Validation

Details

Uh oh!

mergify Bot commented Oct 8, 2026

Uh oh!

mergify Bot commented Oct 8, 2026

Uh oh!

mergify Bot commented Oct 10, 2026

Uh oh!

noooop commented Oct 10, 2026

Uh oh!

github-actions Bot commented Oct 10, 2026

Uh oh!

DarkLight1337 left a comment

Choose a reason for hiding this comment

Uh oh!

mergify Bot commented Oct 10, 2026

Uh oh!

noooop commented Oct 10, 2026

Uh oh!

github-actions Bot commented Oct 10, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

noooop commented Oct 8, 2026 •

edited

Loading