Repository navigation
fix(plugin): start DeepSeek-V4.1 on the vLLM plugin again after #3136 - #3189
Conversation
#3136 made AttentionMetadataBuilder.h2d_group_members a method, but the V4.1 publication wrapper in the plugin still read it as a dict, so every DeepSeek-V4.1 serve on the vLLM plugin failed in profile_run: v41_core_overrides.py:87, in publish for name, names in self.h2d_group_members.items(): AttributeError: 'function' object has no attribute 'items' The wrapper only needs the groups V4.1 publishes by name outside TBO (step_metadata from the builder, the producer groups from the buffers), which the method returns. The new test checks the plugin source, since the non-GPU unit run cannot import the module that broke. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#3136 changed native `_prepare_step` from `verifying = tentative and not batch.is_dummy_run and bool(spans)` to `verifying = tentative and bool(spans)`: native capture batches are not dummies. The plugin's warmup before a DSpark capture is: its dummy batch is fabricated on private scratch with every request at position 0, and `_v41_prepare_kwargs` marks a uniform verify-width batch tentative so the plan keeps the verify write grid. `begin_step` now refuses that step and every DeepSeek-V4.1 DSpark serve on the plugin fails at capture: ValueError: Tentative verification needs a prefix and sufficient window slack Begin that one step as non-verifying, as before #3136; the plan still gets the verify write grid from `tentative`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
🏷️ CI GuideRuns automatically on every eligible PR before approval:
Heavy model tests:
|
| ) | ||
| with ( | ||
| _unverified_steps(cache) | ||
| if tentative and batch.is_dummy_run |
There was a problem hiding this comment.
forcing tentative=False also makes the verify-width dummy a prefill step (prefill ffn instead of decode_ffn), intended? if so please say it in the docstring..
There was a problem hiding this comment.
Intended; it ran this way before #3136 too. Dummies only feed eager forwards whose output is discarded (capture warmups, dummy runs before the pool binds). Captured batches (_v41_capture_batch, _v41_profile_capture_batch) are not dummies, start a full window in and stay verifying, so the graphs record decode_ffn. Docstring updated in 74d8ea9.
Review on #3189: forcing the dummy's step to non-verifying also makes it a prefill step (`BatchStep.decode`), so it runs the prefill FFN rather than `decode_ffn`. That is how it ran before #3136 and is intended: dummy batches only feed eager forwards whose output is discarded, while the batches captures record are not dummies and stay verifying. Docstring only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Round 2 pushed 74d8ea9 (docstring only, per the inline review). Results at ee37864: DSpark TP4 0.9287 (MTP acceptance passes), eager TP4 0.9257, LMCache MP two-pass 0.9257 then 0.9272 with 84.96% from the tier. @valar-cop |
|
ci triage for 74d8ea9:
|
Fixes a regression from #3136 (bf65756). Ready for review. No blocking issues.
Since #3136, every DeepSeek-V4.1-Flash serve on the vLLM plugin fails before serving. #3136 changed two native contracts that the plugin's V4.1 path depends on; the plugin was not updated for either. One commit fixes each; native code is unchanged.
Dev record
AttentionMetadataBuilder.h2d_group_membersa method and updatedModelRunner._init_h2d_publication, but the plugin's V4.1 publication wrapper, which builds those groups itself because vLLM's runner does not, still read it as a dict ('function' object has no attribute 'items'inprofile_run). Fix: call it.not batch.is_dummy_runfromverifyinginDeepseekV41MetadataBuilder._prepare_step; native capture batches are not dummies. The plugin's eager warmup before each DSpark capture is a dummy verify-width batch on private scratch with requests at position 0, sobegin_stepnow refuses it. Fix: begin that one step non-verifying, as before refactor(tbo): stage, publish and finish attention metadata once per micro-batch #3136.tentativefor the plan, so the dummy keeps the verify write grid; its step is a prefill step and runs the prefill FFN, as before refactor(tbo): stage, publish and finish attention metadata once per micro-batch #3136. Dummies only feed eager forwards whose output is discarded; the batches captures record are not dummies and still recorddecode_ffn. Restoring the native condition was not chosen because refactor(tbo): stage, publish and finish attention metadata once per micro-batch #3136 removed it on purpose; a real verify on scratch has no state to verify against.Named result
Plugin CI V4.1 cells, local repro at ee37864 (74d8ea9 adds a docstring only): DSpark TP4 0.9287 (threshold 0.91, MTP acceptance passes), eager TP4 0.9257. LMCache two-pass through the MP server: 0.9257 then 0.9272, with 84.96% of the second pass's prompt tokens served by the tier.
Gates
tests/test_vllm_plugin_h2d_group_members.py: passes; on main's source it flags the read.Evidence
On main @ 1676569, worker traceback in
profile_run:With only the first commit, eager passes and DSpark fails in the warmup forward of
capture:The test reads the plugin source because the module cannot be imported in the non-GPU unit run.
On the earlier base (main @ 1676569) the same two commits scored DSpark 0.9151 and 0.9166, eager 0.9257, LMCache 0.9310 then 0.9287.
🤖 Generated with Claude Code