Repository navigation
[Model Runner V2] Migration to support quantized model by default [5/N] - #44446
Merged
Merged
Conversation
Signed-off-by: yewentao256 <zhyanwentao@126.com>
yewentao256
requested review from
ProExpertProg,
WoosukKwon,
hmellor,
houseroad,
mgoin,
robertgshaw2-redhat,
tlrmchlsmth and
youkaichao
as code owners
June 3, 2026 19:56
yewentao256
added
ready
48 of 49 tasks
Contributor
|
This pull request has merge conflicts that must be resolved before it can be |
Signed-off-by: yewentao256 <zhyanwentao@126.com>
yewentao256
commented
Jun 16, 2026
Member
Author
There was a problem hiding this comment.
buildkite/ci/pr/attention-benchmarks-smoke-test-b200 has been fixed in main
buildkite/ci/pr/amd-kernels-quantization-test-2-mi325-1
buildkite/ci/pr/amd-v1-sample-plus-logits-mi325-1
are unrelated, we can remove the run all tests label now.
yewentao256
removed
the
ready-run-all-tests
njhill
approved these changes
Jun 18, 2026
1 task done
nkzhenhua
pushed a commit
to nkzhenhua/vllm
that referenced
this pull request
Jun 24, 2026
…N] (vllm-project#44446) Signed-off-by: yewentao256 <zhyanwentao@126.com>
MengqingCao
pushed a commit
to vllm-project/vllm-ascend
that referenced
this pull request
Jul 4, 2026
### What this PR does / why we need it? This PR upgrades the verified vLLM main commit to `b9a7cd464c9ae9b1b450f8982b76d7be4de73724` and adapts vLLM Ascend to upstream API changes introduced after the previous verified commit. `vllm_ascend/__init__.py` - Set `VLLM_USE_V2_MODEL_RUNNER=0` for non-`0.23.0`. - Reason: this branch keeps the main compatibility path on V1 model runner. V2 still has partial compatibility updates, but it is not enabled as the default path here. - Related upstream changes: vLLM Model Runner V2 migration work, including [vllm#42667](vllm-project/vllm#42667) and [vllm#44446](vllm-project/vllm#44446). `tests/e2e/pull_request/one_card/test_guided_decoding.py` - For non-`0.23.0` and `xgrammar` / `outlines`, set `VLLM_REGEX_COMPILATION_TIMEOUT_S=30` during the test. - Keep `0.23.0` behavior unchanged. - Reason: upstream added a structured-output regex compilation timeout. The existing complex JSON schema can exceed the default 5s timeout during grammar compilation. - Upstream source: [vllm#45118](vllm-project/vllm#45118). `vllm_ascend/core/scheduler_profiling_chunk.py` - Add `throttle_prefills: bool = False` to `ProfilingChunkScheduler.schedule`. - Reason: upstream scheduler `schedule()` now accepts `throttle_prefills`. - Upstream source: [vllm#44558](vllm-project/vllm#44558). `vllm_ascend/patch/platform/patch_balance_schedule.py` - Add `throttle_prefills` to `BalanceScheduler.schedule`. - For `0.23.0`, continue calling `super().schedule()` with no argument. - For non-`0.23.0`, forward `throttle_prefills` to upstream scheduler. - Reason: keep the balance scheduler wrapper compatible with both old and new upstream scheduler signatures. - Upstream source: [vllm#44558](vllm-project/vllm#44558). `vllm_ascend/patch/platform/patch_profiling_chunk.py` - Change the schedule wrapper to accept `*args, **kwargs`. - Reason: profiling chunk patch wraps scheduler implementations whose signatures now differ between `0.23.0` and upstream main. - Upstream source: [vllm#44558](vllm-project/vllm#44558). `vllm_ascend/spec_decode/ngram_proposer.py` - Split `AscendNgramProposer.propose` by `vllm_version_is("0.23.0")`. - `0.23.0` keeps the old Ascend behavior using `runner.input_batch`. - Non-`0.23.0` follows upstream Dynamic SD signature: `propose(num_speculative_tokens, sampled_token_ids, num_tokens_no_spec, token_ids_cpu, ...)`. - Pass `num_speculative_tokens` into `batch_propose`. - Reason: upstream Dynamic Speculative Decoding changed ngram proposer signatures and made the per-step draft-token budget explicit. - Upstream source: [vllm#32374](vllm-project/vllm#32374). `vllm_ascend/spec_decode/suffix_proposer.py` - Add version-aware wrapper for suffix decoding proposer. - `0.23.0` keeps calling upstream as: `super().propose(input_batch, sampled_token_ids)`. - Non-`0.23.0` calls upstream as: `super().propose(num_speculative_tokens, input_batch, sampled_token_ids, ...)`. - Reason: upstream Dynamic SD added `num_speculative_tokens` to suffix decoding proposer. - Upstream source: [vllm#32374](vllm-project/vllm#32374). `vllm_ascend/worker/model_runner_v1.py` - For `AscendNgramProposer`, pass `scheduler_output.num_spec_tokens_to_schedule`, `self.input_batch.num_tokens_no_spec`, and `self.input_batch.token_ids_cpu` on non-`0.23.0`. - For `AscendSuffixDecodingProposer`, pass `num_speculative_tokens=scheduler_output.num_spec_tokens_to_schedule` on non-`0.23.0`. - For Eagle-style proposers, pass `self.speculative_config.num_speculative_tokens` on non-`0.23.0`. - Reason: upstream Dynamic SD changed proposer call signatures and scheduler output now carries the dynamic speculative token budget. - Upstream source: [vllm#32374](vllm-project/vllm#32374). `vllm_ascend/worker/encoder_acl_graph.py` - For `0.23.0`, continue using `self.budget_graphs[token_budget]`. - For non-`0.23.0`, use upstream graph-set storage through `self._get_graph_set("default")`. - Reason: upstream encoder CUDA graph manager changed from a flat budget graph map to graph sets for multiple capture paths. - Upstream source: [vllm#43586](vllm-project/vllm#43586). `vllm_ascend/ops/fused_moe/fused_moe.py` - Keep `0.23.0` routed-experts capture config lookup from `layer.vllm_config.model_config`. - For non-`0.23.0`, use `get_current_vllm_config()` to obtain `model_config`. - Reason: upstream routed experts capture is now driven by current vLLM config context instead of relying only on the layer-local config path. - Upstream source: [vllm#28284](vllm-project/vllm#28284). `vllm_ascend/worker/v2/aclgraph_utils.py` - Add `lora_capture_cases` to `ModelAclGraphManager.__init__` and forward it upstream. - Add `lora_capture_hook` to `capture_model` and forward it by keyword. - Reason: upstream MRV2 LoRA warmup/capture now passes LoRA capture cases and hook through CUDA graph manager APIs. - Upstream source: [vllm#35536](vllm-project/vllm#35536). `vllm_ascend/worker/v2/model_runner.py` - Update the graph manager factory to accept `lora_capture_cases`. - Forward `lora_capture_cases` into `ModelAclGraphManager`. - Reason: without this factory-level argument, upstream MRV2 LoRA warmup fails before reaching Ascend ACL graph manager. - Upstream source: [vllm#35536](vllm-project/vllm#35536). `vllm_ascend/worker/v2/spec_decode/eagle/aclgraph.py` - Replace the misleading `AttentionStatePair as CapturedAttentionState` alias with direct `AttentionStatePair` usage. - Rename `full_cg_attn_states` to `attn_states`. - Forward `progress_bar_desc` by keyword. - Reason: upstream Eagle graph capture split/renamed captured attention state flow. - Upstream source: [vllm#44253](vllm-project/vllm#44253). `vllm_ascend/patch/platform/__init__.py` - Only import GLM4.7 / MiniMax M2 parser patches and MiniMax usage accounting patch on `0.23.0`. - Reason: upstream main has refactored these parser internals, so the old Ascend patches are only valid for the `0.23.0` parser implementation. - Upstream sources: [vllm#45701](vllm-project/vllm#45701), [vllm#45915](vllm-project/vllm#45915). `tests/ut/patch/platform/test_patch_glm47_tool_call_parser.py` - Skip the module on non-`0.23.0` before importing incompatible upstream parser internals. - Reason: upstream GLM parser internals were migrated/refactored and `_extract_tool_call_regions` is no longer available in the old form. - Upstream source: [vllm#45915](vllm-project/vllm#45915). `tests/ut/patch/platform/test_patch_minimax_m2_tool_call_parser.py` - Skip on non-`0.23.0`. - Reason: upstream MiniMax M2 tool parser moved to the new streaming parser engine and old attributes such as `tool_call_start_token` are no longer available. - Upstream source: [vllm#45701](vllm-project/vllm#45701). `tests/ut/patch/platform/test_patch_minimax_usage_accounting.py` - Skip on non-`0.23.0`. - Reason: upstream MiniMax M2 reasoning/parser internals changed and old usage accounting patch assumptions no longer hold on main. - Upstream source: [vllm#45701](vllm-project/vllm#45701). - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@a30addc --------- Signed-off-by: MrZ20 <2609716663@qq.com> Signed-off-by: Mrz20 <2609716663@qq.com>
wangyichao1999
pushed a commit
to wangyichao1999/vllm-ascend
that referenced
this pull request
Jul 9, 2026
### What this PR does / why we need it? This PR upgrades the verified vLLM main commit to `b9a7cd464c9ae9b1b450f8982b76d7be4de73724` and adapts vLLM Ascend to upstream API changes introduced after the previous verified commit. `vllm_ascend/__init__.py` - Set `VLLM_USE_V2_MODEL_RUNNER=0` for non-`0.23.0`. - Reason: this branch keeps the main compatibility path on V1 model runner. V2 still has partial compatibility updates, but it is not enabled as the default path here. - Related upstream changes: vLLM Model Runner V2 migration work, including [vllm#42667](vllm-project/vllm#42667) and [vllm#44446](vllm-project/vllm#44446). `tests/e2e/pull_request/one_card/test_guided_decoding.py` - For non-`0.23.0` and `xgrammar` / `outlines`, set `VLLM_REGEX_COMPILATION_TIMEOUT_S=30` during the test. - Keep `0.23.0` behavior unchanged. - Reason: upstream added a structured-output regex compilation timeout. The existing complex JSON schema can exceed the default 5s timeout during grammar compilation. - Upstream source: [vllm#45118](vllm-project/vllm#45118). `vllm_ascend/core/scheduler_profiling_chunk.py` - Add `throttle_prefills: bool = False` to `ProfilingChunkScheduler.schedule`. - Reason: upstream scheduler `schedule()` now accepts `throttle_prefills`. - Upstream source: [vllm#44558](vllm-project/vllm#44558). `vllm_ascend/patch/platform/patch_balance_schedule.py` - Add `throttle_prefills` to `BalanceScheduler.schedule`. - For `0.23.0`, continue calling `super().schedule()` with no argument. - For non-`0.23.0`, forward `throttle_prefills` to upstream scheduler. - Reason: keep the balance scheduler wrapper compatible with both old and new upstream scheduler signatures. - Upstream source: [vllm#44558](vllm-project/vllm#44558). `vllm_ascend/patch/platform/patch_profiling_chunk.py` - Change the schedule wrapper to accept `*args, **kwargs`. - Reason: profiling chunk patch wraps scheduler implementations whose signatures now differ between `0.23.0` and upstream main. - Upstream source: [vllm#44558](vllm-project/vllm#44558). `vllm_ascend/spec_decode/ngram_proposer.py` - Split `AscendNgramProposer.propose` by `vllm_version_is("0.23.0")`. - `0.23.0` keeps the old Ascend behavior using `runner.input_batch`. - Non-`0.23.0` follows upstream Dynamic SD signature: `propose(num_speculative_tokens, sampled_token_ids, num_tokens_no_spec, token_ids_cpu, ...)`. - Pass `num_speculative_tokens` into `batch_propose`. - Reason: upstream Dynamic Speculative Decoding changed ngram proposer signatures and made the per-step draft-token budget explicit. - Upstream source: [vllm#32374](vllm-project/vllm#32374). `vllm_ascend/spec_decode/suffix_proposer.py` - Add version-aware wrapper for suffix decoding proposer. - `0.23.0` keeps calling upstream as: `super().propose(input_batch, sampled_token_ids)`. - Non-`0.23.0` calls upstream as: `super().propose(num_speculative_tokens, input_batch, sampled_token_ids, ...)`. - Reason: upstream Dynamic SD added `num_speculative_tokens` to suffix decoding proposer. - Upstream source: [vllm#32374](vllm-project/vllm#32374). `vllm_ascend/worker/model_runner_v1.py` - For `AscendNgramProposer`, pass `scheduler_output.num_spec_tokens_to_schedule`, `self.input_batch.num_tokens_no_spec`, and `self.input_batch.token_ids_cpu` on non-`0.23.0`. - For `AscendSuffixDecodingProposer`, pass `num_speculative_tokens=scheduler_output.num_spec_tokens_to_schedule` on non-`0.23.0`. - For Eagle-style proposers, pass `self.speculative_config.num_speculative_tokens` on non-`0.23.0`. - Reason: upstream Dynamic SD changed proposer call signatures and scheduler output now carries the dynamic speculative token budget. - Upstream source: [vllm#32374](vllm-project/vllm#32374). `vllm_ascend/worker/encoder_acl_graph.py` - For `0.23.0`, continue using `self.budget_graphs[token_budget]`. - For non-`0.23.0`, use upstream graph-set storage through `self._get_graph_set("default")`. - Reason: upstream encoder CUDA graph manager changed from a flat budget graph map to graph sets for multiple capture paths. - Upstream source: [vllm#43586](vllm-project/vllm#43586). `vllm_ascend/ops/fused_moe/fused_moe.py` - Keep `0.23.0` routed-experts capture config lookup from `layer.vllm_config.model_config`. - For non-`0.23.0`, use `get_current_vllm_config()` to obtain `model_config`. - Reason: upstream routed experts capture is now driven by current vLLM config context instead of relying only on the layer-local config path. - Upstream source: [vllm#28284](vllm-project/vllm#28284). `vllm_ascend/worker/v2/aclgraph_utils.py` - Add `lora_capture_cases` to `ModelAclGraphManager.__init__` and forward it upstream. - Add `lora_capture_hook` to `capture_model` and forward it by keyword. - Reason: upstream MRV2 LoRA warmup/capture now passes LoRA capture cases and hook through CUDA graph manager APIs. - Upstream source: [vllm#35536](vllm-project/vllm#35536). `vllm_ascend/worker/v2/model_runner.py` - Update the graph manager factory to accept `lora_capture_cases`. - Forward `lora_capture_cases` into `ModelAclGraphManager`. - Reason: without this factory-level argument, upstream MRV2 LoRA warmup fails before reaching Ascend ACL graph manager. - Upstream source: [vllm#35536](vllm-project/vllm#35536). `vllm_ascend/worker/v2/spec_decode/eagle/aclgraph.py` - Replace the misleading `AttentionStatePair as CapturedAttentionState` alias with direct `AttentionStatePair` usage. - Rename `full_cg_attn_states` to `attn_states`. - Forward `progress_bar_desc` by keyword. - Reason: upstream Eagle graph capture split/renamed captured attention state flow. - Upstream source: [vllm#44253](vllm-project/vllm#44253). `vllm_ascend/patch/platform/__init__.py` - Only import GLM4.7 / MiniMax M2 parser patches and MiniMax usage accounting patch on `0.23.0`. - Reason: upstream main has refactored these parser internals, so the old Ascend patches are only valid for the `0.23.0` parser implementation. - Upstream sources: [vllm#45701](vllm-project/vllm#45701), [vllm#45915](vllm-project/vllm#45915). `tests/ut/patch/platform/test_patch_glm47_tool_call_parser.py` - Skip the module on non-`0.23.0` before importing incompatible upstream parser internals. - Reason: upstream GLM parser internals were migrated/refactored and `_extract_tool_call_regions` is no longer available in the old form. - Upstream source: [vllm#45915](vllm-project/vllm#45915). `tests/ut/patch/platform/test_patch_minimax_m2_tool_call_parser.py` - Skip on non-`0.23.0`. - Reason: upstream MiniMax M2 tool parser moved to the new streaming parser engine and old attributes such as `tool_call_start_token` are no longer available. - Upstream source: [vllm#45701](vllm-project/vllm#45701). `tests/ut/patch/platform/test_patch_minimax_usage_accounting.py` - Skip on non-`0.23.0`. - Reason: upstream MiniMax M2 reasoning/parser internals changed and old usage accounting patch assumptions no longer hold on main. - Upstream source: [vllm#45701](vllm-project/vllm#45701). - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@a30addc --------- Signed-off-by: MrZ20 <2609716663@qq.com> Signed-off-by: Mrz20 <2609716663@qq.com>
xqchen7
pushed a commit
to nv-action/vllm-benchmarks
that referenced
this pull request
Jul 15, 2026
### What this PR does / why we need it? This PR upgrades the verified vLLM main commit to `b9a7cd464c9ae9b1b450f8982b76d7be4de73724` and adapts vLLM Ascend to upstream API changes introduced after the previous verified commit. `vllm_ascend/__init__.py` - Set `VLLM_USE_V2_MODEL_RUNNER=0` for non-`0.23.0`. - Reason: this branch keeps the main compatibility path on V1 model runner. V2 still has partial compatibility updates, but it is not enabled as the default path here. - Related upstream changes: vLLM Model Runner V2 migration work, including [vllm#42667](vllm-project/vllm#42667) and [vllm#44446](vllm-project/vllm#44446). `tests/e2e/pull_request/one_card/test_guided_decoding.py` - For non-`0.23.0` and `xgrammar` / `outlines`, set `VLLM_REGEX_COMPILATION_TIMEOUT_S=30` during the test. - Keep `0.23.0` behavior unchanged. - Reason: upstream added a structured-output regex compilation timeout. The existing complex JSON schema can exceed the default 5s timeout during grammar compilation. - Upstream source: [vllm#45118](vllm-project/vllm#45118). `vllm_ascend/core/scheduler_profiling_chunk.py` - Add `throttle_prefills: bool = False` to `ProfilingChunkScheduler.schedule`. - Reason: upstream scheduler `schedule()` now accepts `throttle_prefills`. - Upstream source: [vllm#44558](vllm-project/vllm#44558). `vllm_ascend/patch/platform/patch_balance_schedule.py` - Add `throttle_prefills` to `BalanceScheduler.schedule`. - For `0.23.0`, continue calling `super().schedule()` with no argument. - For non-`0.23.0`, forward `throttle_prefills` to upstream scheduler. - Reason: keep the balance scheduler wrapper compatible with both old and new upstream scheduler signatures. - Upstream source: [vllm#44558](vllm-project/vllm#44558). `vllm_ascend/patch/platform/patch_profiling_chunk.py` - Change the schedule wrapper to accept `*args, **kwargs`. - Reason: profiling chunk patch wraps scheduler implementations whose signatures now differ between `0.23.0` and upstream main. - Upstream source: [vllm#44558](vllm-project/vllm#44558). `vllm_ascend/spec_decode/ngram_proposer.py` - Split `AscendNgramProposer.propose` by `vllm_version_is("0.23.0")`. - `0.23.0` keeps the old Ascend behavior using `runner.input_batch`. - Non-`0.23.0` follows upstream Dynamic SD signature: `propose(num_speculative_tokens, sampled_token_ids, num_tokens_no_spec, token_ids_cpu, ...)`. - Pass `num_speculative_tokens` into `batch_propose`. - Reason: upstream Dynamic Speculative Decoding changed ngram proposer signatures and made the per-step draft-token budget explicit. - Upstream source: [vllm#32374](vllm-project/vllm#32374). `vllm_ascend/spec_decode/suffix_proposer.py` - Add version-aware wrapper for suffix decoding proposer. - `0.23.0` keeps calling upstream as: `super().propose(input_batch, sampled_token_ids)`. - Non-`0.23.0` calls upstream as: `super().propose(num_speculative_tokens, input_batch, sampled_token_ids, ...)`. - Reason: upstream Dynamic SD added `num_speculative_tokens` to suffix decoding proposer. - Upstream source: [vllm#32374](vllm-project/vllm#32374). `vllm_ascend/worker/model_runner_v1.py` - For `AscendNgramProposer`, pass `scheduler_output.num_spec_tokens_to_schedule`, `self.input_batch.num_tokens_no_spec`, and `self.input_batch.token_ids_cpu` on non-`0.23.0`. - For `AscendSuffixDecodingProposer`, pass `num_speculative_tokens=scheduler_output.num_spec_tokens_to_schedule` on non-`0.23.0`. - For Eagle-style proposers, pass `self.speculative_config.num_speculative_tokens` on non-`0.23.0`. - Reason: upstream Dynamic SD changed proposer call signatures and scheduler output now carries the dynamic speculative token budget. - Upstream source: [vllm#32374](vllm-project/vllm#32374). `vllm_ascend/worker/encoder_acl_graph.py` - For `0.23.0`, continue using `self.budget_graphs[token_budget]`. - For non-`0.23.0`, use upstream graph-set storage through `self._get_graph_set("default")`. - Reason: upstream encoder CUDA graph manager changed from a flat budget graph map to graph sets for multiple capture paths. - Upstream source: [vllm#43586](vllm-project/vllm#43586). `vllm_ascend/ops/fused_moe/fused_moe.py` - Keep `0.23.0` routed-experts capture config lookup from `layer.vllm_config.model_config`. - For non-`0.23.0`, use `get_current_vllm_config()` to obtain `model_config`. - Reason: upstream routed experts capture is now driven by current vLLM config context instead of relying only on the layer-local config path. - Upstream source: [vllm#28284](vllm-project/vllm#28284). `vllm_ascend/worker/v2/aclgraph_utils.py` - Add `lora_capture_cases` to `ModelAclGraphManager.__init__` and forward it upstream. - Add `lora_capture_hook` to `capture_model` and forward it by keyword. - Reason: upstream MRV2 LoRA warmup/capture now passes LoRA capture cases and hook through CUDA graph manager APIs. - Upstream source: [vllm#35536](vllm-project/vllm#35536). `vllm_ascend/worker/v2/model_runner.py` - Update the graph manager factory to accept `lora_capture_cases`. - Forward `lora_capture_cases` into `ModelAclGraphManager`. - Reason: without this factory-level argument, upstream MRV2 LoRA warmup fails before reaching Ascend ACL graph manager. - Upstream source: [vllm#35536](vllm-project/vllm#35536). `vllm_ascend/worker/v2/spec_decode/eagle/aclgraph.py` - Replace the misleading `AttentionStatePair as CapturedAttentionState` alias with direct `AttentionStatePair` usage. - Rename `full_cg_attn_states` to `attn_states`. - Forward `progress_bar_desc` by keyword. - Reason: upstream Eagle graph capture split/renamed captured attention state flow. - Upstream source: [vllm#44253](vllm-project/vllm#44253). `vllm_ascend/patch/platform/__init__.py` - Only import GLM4.7 / MiniMax M2 parser patches and MiniMax usage accounting patch on `0.23.0`. - Reason: upstream main has refactored these parser internals, so the old Ascend patches are only valid for the `0.23.0` parser implementation. - Upstream sources: [vllm#45701](vllm-project/vllm#45701), [vllm#45915](vllm-project/vllm#45915). `tests/ut/patch/platform/test_patch_glm47_tool_call_parser.py` - Skip the module on non-`0.23.0` before importing incompatible upstream parser internals. - Reason: upstream GLM parser internals were migrated/refactored and `_extract_tool_call_regions` is no longer available in the old form. - Upstream source: [vllm#45915](vllm-project/vllm#45915). `tests/ut/patch/platform/test_patch_minimax_m2_tool_call_parser.py` - Skip on non-`0.23.0`. - Reason: upstream MiniMax M2 tool parser moved to the new streaming parser engine and old attributes such as `tool_call_start_token` are no longer available. - Upstream source: [vllm#45701](vllm-project/vllm#45701). `tests/ut/patch/platform/test_patch_minimax_usage_accounting.py` - Skip on non-`0.23.0`. - Reason: upstream MiniMax M2 reasoning/parser internals changed and old usage accounting patch assumptions no longer hold on main. - Upstream source: [vllm#45701](vllm-project/vllm#45701). - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@a30addc --------- Signed-off-by: MrZ20 <2609716663@qq.com> Signed-off-by: Mrz20 <2609716663@qq.com> Signed-off-by: xqchen7 <chenxueqing7@huawei.com>
philippesic
pushed a commit
to philippesic/vllm-semantic-cache
that referenced
this pull request
Jul 19, 2026
…N] (vllm-project#44446) Signed-off-by: yewentao256 <zhyanwentao@126.com>
Alex-stack-hub
pushed a commit
to 0moyi0-2024/vllm-ascend_tp
that referenced
this pull request
Jul 27, 2026
### What this PR does / why we need it? This PR upgrades the verified vLLM main commit to `b9a7cd464c9ae9b1b450f8982b76d7be4de73724` and adapts vLLM Ascend to upstream API changes introduced after the previous verified commit. `vllm_ascend/__init__.py` - Set `VLLM_USE_V2_MODEL_RUNNER=0` for non-`0.23.0`. - Reason: this branch keeps the main compatibility path on V1 model runner. V2 still has partial compatibility updates, but it is not enabled as the default path here. - Related upstream changes: vLLM Model Runner V2 migration work, including [vllm#42667](vllm-project/vllm#42667) and [vllm#44446](vllm-project/vllm#44446). `tests/e2e/pull_request/one_card/test_guided_decoding.py` - For non-`0.23.0` and `xgrammar` / `outlines`, set `VLLM_REGEX_COMPILATION_TIMEOUT_S=30` during the test. - Keep `0.23.0` behavior unchanged. - Reason: upstream added a structured-output regex compilation timeout. The existing complex JSON schema can exceed the default 5s timeout during grammar compilation. - Upstream source: [vllm#45118](vllm-project/vllm#45118). `vllm_ascend/core/scheduler_profiling_chunk.py` - Add `throttle_prefills: bool = False` to `ProfilingChunkScheduler.schedule`. - Reason: upstream scheduler `schedule()` now accepts `throttle_prefills`. - Upstream source: [vllm#44558](vllm-project/vllm#44558). `vllm_ascend/patch/platform/patch_balance_schedule.py` - Add `throttle_prefills` to `BalanceScheduler.schedule`. - For `0.23.0`, continue calling `super().schedule()` with no argument. - For non-`0.23.0`, forward `throttle_prefills` to upstream scheduler. - Reason: keep the balance scheduler wrapper compatible with both old and new upstream scheduler signatures. - Upstream source: [vllm#44558](vllm-project/vllm#44558). `vllm_ascend/patch/platform/patch_profiling_chunk.py` - Change the schedule wrapper to accept `*args, **kwargs`. - Reason: profiling chunk patch wraps scheduler implementations whose signatures now differ between `0.23.0` and upstream main. - Upstream source: [vllm#44558](vllm-project/vllm#44558). `vllm_ascend/spec_decode/ngram_proposer.py` - Split `AscendNgramProposer.propose` by `vllm_version_is("0.23.0")`. - `0.23.0` keeps the old Ascend behavior using `runner.input_batch`. - Non-`0.23.0` follows upstream Dynamic SD signature: `propose(num_speculative_tokens, sampled_token_ids, num_tokens_no_spec, token_ids_cpu, ...)`. - Pass `num_speculative_tokens` into `batch_propose`. - Reason: upstream Dynamic Speculative Decoding changed ngram proposer signatures and made the per-step draft-token budget explicit. - Upstream source: [vllm#32374](vllm-project/vllm#32374). `vllm_ascend/spec_decode/suffix_proposer.py` - Add version-aware wrapper for suffix decoding proposer. - `0.23.0` keeps calling upstream as: `super().propose(input_batch, sampled_token_ids)`. - Non-`0.23.0` calls upstream as: `super().propose(num_speculative_tokens, input_batch, sampled_token_ids, ...)`. - Reason: upstream Dynamic SD added `num_speculative_tokens` to suffix decoding proposer. - Upstream source: [vllm#32374](vllm-project/vllm#32374). `vllm_ascend/worker/model_runner_v1.py` - For `AscendNgramProposer`, pass `scheduler_output.num_spec_tokens_to_schedule`, `self.input_batch.num_tokens_no_spec`, and `self.input_batch.token_ids_cpu` on non-`0.23.0`. - For `AscendSuffixDecodingProposer`, pass `num_speculative_tokens=scheduler_output.num_spec_tokens_to_schedule` on non-`0.23.0`. - For Eagle-style proposers, pass `self.speculative_config.num_speculative_tokens` on non-`0.23.0`. - Reason: upstream Dynamic SD changed proposer call signatures and scheduler output now carries the dynamic speculative token budget. - Upstream source: [vllm#32374](vllm-project/vllm#32374). `vllm_ascend/worker/encoder_acl_graph.py` - For `0.23.0`, continue using `self.budget_graphs[token_budget]`. - For non-`0.23.0`, use upstream graph-set storage through `self._get_graph_set("default")`. - Reason: upstream encoder CUDA graph manager changed from a flat budget graph map to graph sets for multiple capture paths. - Upstream source: [vllm#43586](vllm-project/vllm#43586). `vllm_ascend/ops/fused_moe/fused_moe.py` - Keep `0.23.0` routed-experts capture config lookup from `layer.vllm_config.model_config`. - For non-`0.23.0`, use `get_current_vllm_config()` to obtain `model_config`. - Reason: upstream routed experts capture is now driven by current vLLM config context instead of relying only on the layer-local config path. - Upstream source: [vllm#28284](vllm-project/vllm#28284). `vllm_ascend/worker/v2/aclgraph_utils.py` - Add `lora_capture_cases` to `ModelAclGraphManager.__init__` and forward it upstream. - Add `lora_capture_hook` to `capture_model` and forward it by keyword. - Reason: upstream MRV2 LoRA warmup/capture now passes LoRA capture cases and hook through CUDA graph manager APIs. - Upstream source: [vllm#35536](vllm-project/vllm#35536). `vllm_ascend/worker/v2/model_runner.py` - Update the graph manager factory to accept `lora_capture_cases`. - Forward `lora_capture_cases` into `ModelAclGraphManager`. - Reason: without this factory-level argument, upstream MRV2 LoRA warmup fails before reaching Ascend ACL graph manager. - Upstream source: [vllm#35536](vllm-project/vllm#35536). `vllm_ascend/worker/v2/spec_decode/eagle/aclgraph.py` - Replace the misleading `AttentionStatePair as CapturedAttentionState` alias with direct `AttentionStatePair` usage. - Rename `full_cg_attn_states` to `attn_states`. - Forward `progress_bar_desc` by keyword. - Reason: upstream Eagle graph capture split/renamed captured attention state flow. - Upstream source: [vllm#44253](vllm-project/vllm#44253). `vllm_ascend/patch/platform/__init__.py` - Only import GLM4.7 / MiniMax M2 parser patches and MiniMax usage accounting patch on `0.23.0`. - Reason: upstream main has refactored these parser internals, so the old Ascend patches are only valid for the `0.23.0` parser implementation. - Upstream sources: [vllm#45701](vllm-project/vllm#45701), [vllm#45915](vllm-project/vllm#45915). `tests/ut/patch/platform/test_patch_glm47_tool_call_parser.py` - Skip the module on non-`0.23.0` before importing incompatible upstream parser internals. - Reason: upstream GLM parser internals were migrated/refactored and `_extract_tool_call_regions` is no longer available in the old form. - Upstream source: [vllm#45915](vllm-project/vllm#45915). `tests/ut/patch/platform/test_patch_minimax_m2_tool_call_parser.py` - Skip on non-`0.23.0`. - Reason: upstream MiniMax M2 tool parser moved to the new streaming parser engine and old attributes such as `tool_call_start_token` are no longer available. - Upstream source: [vllm#45701](vllm-project/vllm#45701). `tests/ut/patch/platform/test_patch_minimax_usage_accounting.py` - Skip on non-`0.23.0`. - Reason: upstream MiniMax M2 reasoning/parser internals changed and old usage accounting patch assumptions no longer hold on main. - Upstream source: [vllm#45701](vllm-project/vllm#45701). - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@a30addc --------- Signed-off-by: MrZ20 <2609716663@qq.com> Signed-off-by: Mrz20 <2609716663@qq.com>
MmMmaru
pushed a commit
to jiaqi-lee/vllm-ascend
that referenced
this pull request
Aug 19, 2026
### What this PR does / why we need it? This PR upgrades the verified vLLM main commit to `b9a7cd464c9ae9b1b450f8982b76d7be4de73724` and adapts vLLM Ascend to upstream API changes introduced after the previous verified commit. `vllm_ascend/__init__.py` - Set `VLLM_USE_V2_MODEL_RUNNER=0` for non-`0.23.0`. - Reason: this branch keeps the main compatibility path on V1 model runner. V2 still has partial compatibility updates, but it is not enabled as the default path here. - Related upstream changes: vLLM Model Runner V2 migration work, including [vllm#42667](vllm-project/vllm#42667) and [vllm#44446](vllm-project/vllm#44446). `tests/e2e/pull_request/one_card/test_guided_decoding.py` - For non-`0.23.0` and `xgrammar` / `outlines`, set `VLLM_REGEX_COMPILATION_TIMEOUT_S=30` during the test. - Keep `0.23.0` behavior unchanged. - Reason: upstream added a structured-output regex compilation timeout. The existing complex JSON schema can exceed the default 5s timeout during grammar compilation. - Upstream source: [vllm#45118](vllm-project/vllm#45118). `vllm_ascend/core/scheduler_profiling_chunk.py` - Add `throttle_prefills: bool = False` to `ProfilingChunkScheduler.schedule`. - Reason: upstream scheduler `schedule()` now accepts `throttle_prefills`. - Upstream source: [vllm#44558](vllm-project/vllm#44558). `vllm_ascend/patch/platform/patch_balance_schedule.py` - Add `throttle_prefills` to `BalanceScheduler.schedule`. - For `0.23.0`, continue calling `super().schedule()` with no argument. - For non-`0.23.0`, forward `throttle_prefills` to upstream scheduler. - Reason: keep the balance scheduler wrapper compatible with both old and new upstream scheduler signatures. - Upstream source: [vllm#44558](vllm-project/vllm#44558). `vllm_ascend/patch/platform/patch_profiling_chunk.py` - Change the schedule wrapper to accept `*args, **kwargs`. - Reason: profiling chunk patch wraps scheduler implementations whose signatures now differ between `0.23.0` and upstream main. - Upstream source: [vllm#44558](vllm-project/vllm#44558). `vllm_ascend/spec_decode/ngram_proposer.py` - Split `AscendNgramProposer.propose` by `vllm_version_is("0.23.0")`. - `0.23.0` keeps the old Ascend behavior using `runner.input_batch`. - Non-`0.23.0` follows upstream Dynamic SD signature: `propose(num_speculative_tokens, sampled_token_ids, num_tokens_no_spec, token_ids_cpu, ...)`. - Pass `num_speculative_tokens` into `batch_propose`. - Reason: upstream Dynamic Speculative Decoding changed ngram proposer signatures and made the per-step draft-token budget explicit. - Upstream source: [vllm#32374](vllm-project/vllm#32374). `vllm_ascend/spec_decode/suffix_proposer.py` - Add version-aware wrapper for suffix decoding proposer. - `0.23.0` keeps calling upstream as: `super().propose(input_batch, sampled_token_ids)`. - Non-`0.23.0` calls upstream as: `super().propose(num_speculative_tokens, input_batch, sampled_token_ids, ...)`. - Reason: upstream Dynamic SD added `num_speculative_tokens` to suffix decoding proposer. - Upstream source: [vllm#32374](vllm-project/vllm#32374). `vllm_ascend/worker/model_runner_v1.py` - For `AscendNgramProposer`, pass `scheduler_output.num_spec_tokens_to_schedule`, `self.input_batch.num_tokens_no_spec`, and `self.input_batch.token_ids_cpu` on non-`0.23.0`. - For `AscendSuffixDecodingProposer`, pass `num_speculative_tokens=scheduler_output.num_spec_tokens_to_schedule` on non-`0.23.0`. - For Eagle-style proposers, pass `self.speculative_config.num_speculative_tokens` on non-`0.23.0`. - Reason: upstream Dynamic SD changed proposer call signatures and scheduler output now carries the dynamic speculative token budget. - Upstream source: [vllm#32374](vllm-project/vllm#32374). `vllm_ascend/worker/encoder_acl_graph.py` - For `0.23.0`, continue using `self.budget_graphs[token_budget]`. - For non-`0.23.0`, use upstream graph-set storage through `self._get_graph_set("default")`. - Reason: upstream encoder CUDA graph manager changed from a flat budget graph map to graph sets for multiple capture paths. - Upstream source: [vllm#43586](vllm-project/vllm#43586). `vllm_ascend/ops/fused_moe/fused_moe.py` - Keep `0.23.0` routed-experts capture config lookup from `layer.vllm_config.model_config`. - For non-`0.23.0`, use `get_current_vllm_config()` to obtain `model_config`. - Reason: upstream routed experts capture is now driven by current vLLM config context instead of relying only on the layer-local config path. - Upstream source: [vllm#28284](vllm-project/vllm#28284). `vllm_ascend/worker/v2/aclgraph_utils.py` - Add `lora_capture_cases` to `ModelAclGraphManager.__init__` and forward it upstream. - Add `lora_capture_hook` to `capture_model` and forward it by keyword. - Reason: upstream MRV2 LoRA warmup/capture now passes LoRA capture cases and hook through CUDA graph manager APIs. - Upstream source: [vllm#35536](vllm-project/vllm#35536). `vllm_ascend/worker/v2/model_runner.py` - Update the graph manager factory to accept `lora_capture_cases`. - Forward `lora_capture_cases` into `ModelAclGraphManager`. - Reason: without this factory-level argument, upstream MRV2 LoRA warmup fails before reaching Ascend ACL graph manager. - Upstream source: [vllm#35536](vllm-project/vllm#35536). `vllm_ascend/worker/v2/spec_decode/eagle/aclgraph.py` - Replace the misleading `AttentionStatePair as CapturedAttentionState` alias with direct `AttentionStatePair` usage. - Rename `full_cg_attn_states` to `attn_states`. - Forward `progress_bar_desc` by keyword. - Reason: upstream Eagle graph capture split/renamed captured attention state flow. - Upstream source: [vllm#44253](vllm-project/vllm#44253). `vllm_ascend/patch/platform/__init__.py` - Only import GLM4.7 / MiniMax M2 parser patches and MiniMax usage accounting patch on `0.23.0`. - Reason: upstream main has refactored these parser internals, so the old Ascend patches are only valid for the `0.23.0` parser implementation. - Upstream sources: [vllm#45701](vllm-project/vllm#45701), [vllm#45915](vllm-project/vllm#45915). `tests/ut/patch/platform/test_patch_glm47_tool_call_parser.py` - Skip the module on non-`0.23.0` before importing incompatible upstream parser internals. - Reason: upstream GLM parser internals were migrated/refactored and `_extract_tool_call_regions` is no longer available in the old form. - Upstream source: [vllm#45915](vllm-project/vllm#45915). `tests/ut/patch/platform/test_patch_minimax_m2_tool_call_parser.py` - Skip on non-`0.23.0`. - Reason: upstream MiniMax M2 tool parser moved to the new streaming parser engine and old attributes such as `tool_call_start_token` are no longer available. - Upstream source: [vllm#45701](vllm-project/vllm#45701). `tests/ut/patch/platform/test_patch_minimax_usage_accounting.py` - Skip on non-`0.23.0`. - Reason: upstream MiniMax M2 reasoning/parser internals changed and old usage accounting patch assumptions no longer hold on main. - Upstream source: [vllm#45701](vllm-project/vllm#45701). - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@a30addc --------- Signed-off-by: MrZ20 <2609716663@qq.com> Signed-off-by: Mrz20 <2609716663@qq.com>
shiqiangA
pushed a commit
to shiqiangA/vllm-ascend
that referenced
this pull request
Aug 20, 2026
### What this PR does / why we need it? This PR upgrades the verified vLLM main commit to `b9a7cd464c9ae9b1b450f8982b76d7be4de73724` and adapts vLLM Ascend to upstream API changes introduced after the previous verified commit. `vllm_ascend/__init__.py` - Set `VLLM_USE_V2_MODEL_RUNNER=0` for non-`0.23.0`. - Reason: this branch keeps the main compatibility path on V1 model runner. V2 still has partial compatibility updates, but it is not enabled as the default path here. - Related upstream changes: vLLM Model Runner V2 migration work, including [vllm#42667](vllm-project/vllm#42667) and [vllm#44446](vllm-project/vllm#44446). `tests/e2e/pull_request/one_card/test_guided_decoding.py` - For non-`0.23.0` and `xgrammar` / `outlines`, set `VLLM_REGEX_COMPILATION_TIMEOUT_S=30` during the test. - Keep `0.23.0` behavior unchanged. - Reason: upstream added a structured-output regex compilation timeout. The existing complex JSON schema can exceed the default 5s timeout during grammar compilation. - Upstream source: [vllm#45118](vllm-project/vllm#45118). `vllm_ascend/core/scheduler_profiling_chunk.py` - Add `throttle_prefills: bool = False` to `ProfilingChunkScheduler.schedule`. - Reason: upstream scheduler `schedule()` now accepts `throttle_prefills`. - Upstream source: [vllm#44558](vllm-project/vllm#44558). `vllm_ascend/patch/platform/patch_balance_schedule.py` - Add `throttle_prefills` to `BalanceScheduler.schedule`. - For `0.23.0`, continue calling `super().schedule()` with no argument. - For non-`0.23.0`, forward `throttle_prefills` to upstream scheduler. - Reason: keep the balance scheduler wrapper compatible with both old and new upstream scheduler signatures. - Upstream source: [vllm#44558](vllm-project/vllm#44558). `vllm_ascend/patch/platform/patch_profiling_chunk.py` - Change the schedule wrapper to accept `*args, **kwargs`. - Reason: profiling chunk patch wraps scheduler implementations whose signatures now differ between `0.23.0` and upstream main. - Upstream source: [vllm#44558](vllm-project/vllm#44558). `vllm_ascend/spec_decode/ngram_proposer.py` - Split `AscendNgramProposer.propose` by `vllm_version_is("0.23.0")`. - `0.23.0` keeps the old Ascend behavior using `runner.input_batch`. - Non-`0.23.0` follows upstream Dynamic SD signature: `propose(num_speculative_tokens, sampled_token_ids, num_tokens_no_spec, token_ids_cpu, ...)`. - Pass `num_speculative_tokens` into `batch_propose`. - Reason: upstream Dynamic Speculative Decoding changed ngram proposer signatures and made the per-step draft-token budget explicit. - Upstream source: [vllm#32374](vllm-project/vllm#32374). `vllm_ascend/spec_decode/suffix_proposer.py` - Add version-aware wrapper for suffix decoding proposer. - `0.23.0` keeps calling upstream as: `super().propose(input_batch, sampled_token_ids)`. - Non-`0.23.0` calls upstream as: `super().propose(num_speculative_tokens, input_batch, sampled_token_ids, ...)`. - Reason: upstream Dynamic SD added `num_speculative_tokens` to suffix decoding proposer. - Upstream source: [vllm#32374](vllm-project/vllm#32374). `vllm_ascend/worker/model_runner_v1.py` - For `AscendNgramProposer`, pass `scheduler_output.num_spec_tokens_to_schedule`, `self.input_batch.num_tokens_no_spec`, and `self.input_batch.token_ids_cpu` on non-`0.23.0`. - For `AscendSuffixDecodingProposer`, pass `num_speculative_tokens=scheduler_output.num_spec_tokens_to_schedule` on non-`0.23.0`. - For Eagle-style proposers, pass `self.speculative_config.num_speculative_tokens` on non-`0.23.0`. - Reason: upstream Dynamic SD changed proposer call signatures and scheduler output now carries the dynamic speculative token budget. - Upstream source: [vllm#32374](vllm-project/vllm#32374). `vllm_ascend/worker/encoder_acl_graph.py` - For `0.23.0`, continue using `self.budget_graphs[token_budget]`. - For non-`0.23.0`, use upstream graph-set storage through `self._get_graph_set("default")`. - Reason: upstream encoder CUDA graph manager changed from a flat budget graph map to graph sets for multiple capture paths. - Upstream source: [vllm#43586](vllm-project/vllm#43586). `vllm_ascend/ops/fused_moe/fused_moe.py` - Keep `0.23.0` routed-experts capture config lookup from `layer.vllm_config.model_config`. - For non-`0.23.0`, use `get_current_vllm_config()` to obtain `model_config`. - Reason: upstream routed experts capture is now driven by current vLLM config context instead of relying only on the layer-local config path. - Upstream source: [vllm#28284](vllm-project/vllm#28284). `vllm_ascend/worker/v2/aclgraph_utils.py` - Add `lora_capture_cases` to `ModelAclGraphManager.__init__` and forward it upstream. - Add `lora_capture_hook` to `capture_model` and forward it by keyword. - Reason: upstream MRV2 LoRA warmup/capture now passes LoRA capture cases and hook through CUDA graph manager APIs. - Upstream source: [vllm#35536](vllm-project/vllm#35536). `vllm_ascend/worker/v2/model_runner.py` - Update the graph manager factory to accept `lora_capture_cases`. - Forward `lora_capture_cases` into `ModelAclGraphManager`. - Reason: without this factory-level argument, upstream MRV2 LoRA warmup fails before reaching Ascend ACL graph manager. - Upstream source: [vllm#35536](vllm-project/vllm#35536). `vllm_ascend/worker/v2/spec_decode/eagle/aclgraph.py` - Replace the misleading `AttentionStatePair as CapturedAttentionState` alias with direct `AttentionStatePair` usage. - Rename `full_cg_attn_states` to `attn_states`. - Forward `progress_bar_desc` by keyword. - Reason: upstream Eagle graph capture split/renamed captured attention state flow. - Upstream source: [vllm#44253](vllm-project/vllm#44253). `vllm_ascend/patch/platform/__init__.py` - Only import GLM4.7 / MiniMax M2 parser patches and MiniMax usage accounting patch on `0.23.0`. - Reason: upstream main has refactored these parser internals, so the old Ascend patches are only valid for the `0.23.0` parser implementation. - Upstream sources: [vllm#45701](vllm-project/vllm#45701), [vllm#45915](vllm-project/vllm#45915). `tests/ut/patch/platform/test_patch_glm47_tool_call_parser.py` - Skip the module on non-`0.23.0` before importing incompatible upstream parser internals. - Reason: upstream GLM parser internals were migrated/refactored and `_extract_tool_call_regions` is no longer available in the old form. - Upstream source: [vllm#45915](vllm-project/vllm#45915). `tests/ut/patch/platform/test_patch_minimax_m2_tool_call_parser.py` - Skip on non-`0.23.0`. - Reason: upstream MiniMax M2 tool parser moved to the new streaming parser engine and old attributes such as `tool_call_start_token` are no longer available. - Upstream source: [vllm#45701](vllm-project/vllm#45701). `tests/ut/patch/platform/test_patch_minimax_usage_accounting.py` - Skip on non-`0.23.0`. - Reason: upstream MiniMax M2 reasoning/parser internals changed and old usage accounting patch assumptions no longer hold on main. - Upstream source: [vllm#45701](vllm-project/vllm#45701). - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@a30addc --------- Signed-off-by: MrZ20 <2609716663@qq.com> Signed-off-by: Mrz20 <2609716663@qq.com>
Leetrytry
pushed a commit
to Leetrytry/vllm-ascend
that referenced
this pull request
Sep 11, 2026
### What this PR does / why we need it? This PR upgrades the verified vLLM main commit to `b9a7cd464c9ae9b1b450f8982b76d7be4de73724` and adapts vLLM Ascend to upstream API changes introduced after the previous verified commit. `vllm_ascend/__init__.py` - Set `VLLM_USE_V2_MODEL_RUNNER=0` for non-`0.23.0`. - Reason: this branch keeps the main compatibility path on V1 model runner. V2 still has partial compatibility updates, but it is not enabled as the default path here. - Related upstream changes: vLLM Model Runner V2 migration work, including [vllm#42667](vllm-project/vllm#42667) and [vllm#44446](vllm-project/vllm#44446). `tests/e2e/pull_request/one_card/test_guided_decoding.py` - For non-`0.23.0` and `xgrammar` / `outlines`, set `VLLM_REGEX_COMPILATION_TIMEOUT_S=30` during the test. - Keep `0.23.0` behavior unchanged. - Reason: upstream added a structured-output regex compilation timeout. The existing complex JSON schema can exceed the default 5s timeout during grammar compilation. - Upstream source: [vllm#45118](vllm-project/vllm#45118). `vllm_ascend/core/scheduler_profiling_chunk.py` - Add `throttle_prefills: bool = False` to `ProfilingChunkScheduler.schedule`. - Reason: upstream scheduler `schedule()` now accepts `throttle_prefills`. - Upstream source: [vllm#44558](vllm-project/vllm#44558). `vllm_ascend/patch/platform/patch_balance_schedule.py` - Add `throttle_prefills` to `BalanceScheduler.schedule`. - For `0.23.0`, continue calling `super().schedule()` with no argument. - For non-`0.23.0`, forward `throttle_prefills` to upstream scheduler. - Reason: keep the balance scheduler wrapper compatible with both old and new upstream scheduler signatures. - Upstream source: [vllm#44558](vllm-project/vllm#44558). `vllm_ascend/patch/platform/patch_profiling_chunk.py` - Change the schedule wrapper to accept `*args, **kwargs`. - Reason: profiling chunk patch wraps scheduler implementations whose signatures now differ between `0.23.0` and upstream main. - Upstream source: [vllm#44558](vllm-project/vllm#44558). `vllm_ascend/spec_decode/ngram_proposer.py` - Split `AscendNgramProposer.propose` by `vllm_version_is("0.23.0")`. - `0.23.0` keeps the old Ascend behavior using `runner.input_batch`. - Non-`0.23.0` follows upstream Dynamic SD signature: `propose(num_speculative_tokens, sampled_token_ids, num_tokens_no_spec, token_ids_cpu, ...)`. - Pass `num_speculative_tokens` into `batch_propose`. - Reason: upstream Dynamic Speculative Decoding changed ngram proposer signatures and made the per-step draft-token budget explicit. - Upstream source: [vllm#32374](vllm-project/vllm#32374). `vllm_ascend/spec_decode/suffix_proposer.py` - Add version-aware wrapper for suffix decoding proposer. - `0.23.0` keeps calling upstream as: `super().propose(input_batch, sampled_token_ids)`. - Non-`0.23.0` calls upstream as: `super().propose(num_speculative_tokens, input_batch, sampled_token_ids, ...)`. - Reason: upstream Dynamic SD added `num_speculative_tokens` to suffix decoding proposer. - Upstream source: [vllm#32374](vllm-project/vllm#32374). `vllm_ascend/worker/model_runner_v1.py` - For `AscendNgramProposer`, pass `scheduler_output.num_spec_tokens_to_schedule`, `self.input_batch.num_tokens_no_spec`, and `self.input_batch.token_ids_cpu` on non-`0.23.0`. - For `AscendSuffixDecodingProposer`, pass `num_speculative_tokens=scheduler_output.num_spec_tokens_to_schedule` on non-`0.23.0`. - For Eagle-style proposers, pass `self.speculative_config.num_speculative_tokens` on non-`0.23.0`. - Reason: upstream Dynamic SD changed proposer call signatures and scheduler output now carries the dynamic speculative token budget. - Upstream source: [vllm#32374](vllm-project/vllm#32374). `vllm_ascend/worker/encoder_acl_graph.py` - For `0.23.0`, continue using `self.budget_graphs[token_budget]`. - For non-`0.23.0`, use upstream graph-set storage through `self._get_graph_set("default")`. - Reason: upstream encoder CUDA graph manager changed from a flat budget graph map to graph sets for multiple capture paths. - Upstream source: [vllm#43586](vllm-project/vllm#43586). `vllm_ascend/ops/fused_moe/fused_moe.py` - Keep `0.23.0` routed-experts capture config lookup from `layer.vllm_config.model_config`. - For non-`0.23.0`, use `get_current_vllm_config()` to obtain `model_config`. - Reason: upstream routed experts capture is now driven by current vLLM config context instead of relying only on the layer-local config path. - Upstream source: [vllm#28284](vllm-project/vllm#28284). `vllm_ascend/worker/v2/aclgraph_utils.py` - Add `lora_capture_cases` to `ModelAclGraphManager.__init__` and forward it upstream. - Add `lora_capture_hook` to `capture_model` and forward it by keyword. - Reason: upstream MRV2 LoRA warmup/capture now passes LoRA capture cases and hook through CUDA graph manager APIs. - Upstream source: [vllm#35536](vllm-project/vllm#35536). `vllm_ascend/worker/v2/model_runner.py` - Update the graph manager factory to accept `lora_capture_cases`. - Forward `lora_capture_cases` into `ModelAclGraphManager`. - Reason: without this factory-level argument, upstream MRV2 LoRA warmup fails before reaching Ascend ACL graph manager. - Upstream source: [vllm#35536](vllm-project/vllm#35536). `vllm_ascend/worker/v2/spec_decode/eagle/aclgraph.py` - Replace the misleading `AttentionStatePair as CapturedAttentionState` alias with direct `AttentionStatePair` usage. - Rename `full_cg_attn_states` to `attn_states`. - Forward `progress_bar_desc` by keyword. - Reason: upstream Eagle graph capture split/renamed captured attention state flow. - Upstream source: [vllm#44253](vllm-project/vllm#44253). `vllm_ascend/patch/platform/__init__.py` - Only import GLM4.7 / MiniMax M2 parser patches and MiniMax usage accounting patch on `0.23.0`. - Reason: upstream main has refactored these parser internals, so the old Ascend patches are only valid for the `0.23.0` parser implementation. - Upstream sources: [vllm#45701](vllm-project/vllm#45701), [vllm#45915](vllm-project/vllm#45915). `tests/ut/patch/platform/test_patch_glm47_tool_call_parser.py` - Skip the module on non-`0.23.0` before importing incompatible upstream parser internals. - Reason: upstream GLM parser internals were migrated/refactored and `_extract_tool_call_regions` is no longer available in the old form. - Upstream source: [vllm#45915](vllm-project/vllm#45915). `tests/ut/patch/platform/test_patch_minimax_m2_tool_call_parser.py` - Skip on non-`0.23.0`. - Reason: upstream MiniMax M2 tool parser moved to the new streaming parser engine and old attributes such as `tool_call_start_token` are no longer available. - Upstream source: [vllm#45701](vllm-project/vllm#45701). `tests/ut/patch/platform/test_patch_minimax_usage_accounting.py` - Skip on non-`0.23.0`. - Reason: upstream MiniMax M2 reasoning/parser internals changed and old usage accounting patch assumptions no longer hold on main. - Upstream source: [vllm#45701](vllm-project/vllm#45701). - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@a30addc --------- Signed-off-by: MrZ20 <2609716663@qq.com> Signed-off-by: Mrz20 <2609716663@qq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Part of #41286
Tests coverd by CI