Repository navigation
Add a script for serving experiments & Collect system stats in scheduler - #30
Merged
Merged
Conversation
1 task done
1 task done
1 task done
zyongye
added a commit
to zyongye/vllm
that referenced
this pull request
Aug 5, 2025
Signed-off-by: simon-mo <xmo@berkeley.edu> Co-authored-by: simon-mo <xmo@berkeley.edu>
zyongye
added a commit
to zyongye/vllm
that referenced
this pull request
Aug 6, 2025
Signed-off-by: simon-mo <xmo@berkeley.edu> Co-authored-by: simon-mo <xmo@berkeley.edu>
heheda12345
pushed a commit
to heheda12345/vllm
that referenced
this pull request
Sep 29, 2025
…cache [NIXL] fix unify kv cache spec
1 task done
inkcherry
pushed a commit
to inkcherry/vllm
that referenced
this pull request
Nov 6, 2025
dik654
pushed a commit
to dik654/vllm-for-study
that referenced
this pull request
Nov 18, 2025
New Industry Use Cases (vllm-project#21-30): - vllm-project#21 Game Development: AI game testing + balance tuning - vllm-project#22 Construction: Vision AI safety inspection - vllm-project#23 Agriculture/Smart Farm: Crop monitoring + pest detection - vllm-project#24 Government/Public: Document automation + citizen services - vllm-project#25 Energy/Utilities: Grid monitoring + anomaly detection - vllm-project#26 Environment/Sustainability: Carbon tracking + ESG reporting - vllm-project#27 Fashion/Apparel: Trend analysis + inventory optimization - vllm-project#28 Sports/Fitness: Performance analytics + tactical analysis - vllm-project#29 Automotive/Mobility: Autonomous driving simulation - vllm-project#30 Space/Aerospace: Satellite image analysis Advanced Architecture Patterns: 1. Event-Driven Pattern: Webhook → Event Bus → Agent triggers 2. Streaming Pattern: Large dataset processing with chunking 3. Batch Processing Pattern: Celery-based parallel processing 4. Circuit Breaker Pattern: Fault tolerance + auto recovery 5. CQRS + Event Sourcing: Command/Query separation 6. Saga Pattern: Distributed transaction management Guide now covers: - 30+ industry-specific MCP implementations - 6 production-ready architecture patterns - Real-world scalability solutions - Enterprise integration strategies - Total: 8,672 lines (from 7,249)
soodoshll
pushed a commit
to soodoshll/vllm
that referenced
this pull request
Jan 30, 2026
* add implementation Signed-off-by: Max Hu <hyoung2991@gmail.com> * add impl Signed-off-by: Max Hu <hyoung2991@gmail.com> * add flashinfer * fix tp Signed-off-by: Max Hu <hyoung2991@gmail.com> * Temporary change for ViT * fix workspace_buffer device. * change max_seqlen to 128k. * remove duplicate multiplier. * fix accuracy and refactor * more fix * change dockerfile * format Signed-off-by: Max Hu <hyoung2991@gmail.com> * fix version Signed-off-by: Max Hu <hyoung2991@gmail.com> * change python version * remove qwen25 transformer support * change dockerfile * add build versions * chagne version * change version * change * change * change * change * change * build image * change back * change to 10.0f * fix fi import Signed-off-by: Max Hu <hyoung2991@gmail.com> * change to build in dev image Signed-off-by: Max Hu <hyoung2991@gmail.com> * change location Signed-off-by: Max Hu <hyoung2991@gmail.com> * change location Signed-off-by: Max Hu <hyoung2991@gmail.com> * change Signed-off-by: Max Hu <hyoung2991@gmail.com> * change cubin and jitcache to wheels Signed-off-by: Max Hu <hyoung2991@gmail.com> * change Signed-off-by: Max Hu <hyoung2991@gmail.com> * add comment Signed-off-by: Max Hu <hyoung2991@gmail.com> --------- Signed-off-by: Max Hu <hyoung2991@gmail.com> Co-authored-by: Anerudhan Gopal <agopal@nvidia.com> Co-authored-by: Baorun Mu <bmu@nvidia.com>
chopper0126
pushed a commit
to chopper0126/vllm
that referenced
this pull request
Feb 2, 2026
add multistream and core limitation of communication stream
1 task done
1 task done
amd-xavierwang
pushed a commit
to amd-xavierwang/vllm
that referenced
this pull request
Apr 8, 2026
…roject#30) Reverts mgehre-amd#24; zeros shape is wrongly assumed
amd-xavierwang
pushed a commit
to amd-xavierwang/vllm
that referenced
this pull request
Apr 8, 2026
Relands the zero-point support that was reverted in PR vllm-project#30, with the correct zero-point tensor layout handling. The original implementation assumed a specific shape for zero points that didn't match what AWQ and compressed-tensors models provide. The fix uses permute_param_layout_() to normalize the zero-point tensor layout from any source format to [M, num_groups] before unpacking, the same approach used by HipW4A16LinearKernel. This enables asymmetric AWQ models (zero_points=True) to use the fast skinny GEMM kernels instead of falling back to the slower ConchLinearKernel. Kernel-level changes: - Added HAS_ZERO_POINTS template parameter to wvSplitK_int4_hf_sml_ and wvSplitK_int4_hf_ (default false, zero codegen impact) - New wvSplitK_int4_g_zp C++ function with per-group zero-point subtraction after dequant: (nibble - 8) - (zp - 8) = nibble - zp - Pre-adjusted zero points stored as (zp_raw - 8) in act_type Python-level changes: - HipW4A16SkinnyLinearKernel.can_implement() now accepts zero_points with per-group quantization (group_size=32 or 128) - process_weights_after_loading() normalizes zero-point layout via permute_param_layout_() and prepares pre-adjusted fp16/bf16 tensor - Custom op signature updated to pass optional w_zp tensor Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
zhewenl
added a commit
to zhewenl/vllm
that referenced
this pull request
Apr 29, 2026
Co-authored-by: Zhewen Li <zhewenli@inferact.ai> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 19, 2026
matteso1
added a commit
to thaw-ai/vllm
that referenced
this pull request
Jul 27, 2026
…municators The capability flags merged in vllm-project#44074 (preserves_communicators, preserves_graphs_with_communicators) document a contract nothing in-tree enforces: Worker.sleep assumes the cumem world where communicators survive suspend untouched. Checkpoint-style backends (RFC vllm-project#34303 phase 2) break that assumption. cuda-checkpoint still cannot restore a 2-GPU vLLM server (NVIDIA/cuda-checkpoint#27) and hangs on multi-process NCCL jobs (NVIDIA/cuda-checkpoint#45, vllm-project#30). vllm-project#46877 owns the checkpoint lifecycle itself: the communicator-owned hooks in device_communicators, the registry walk, and the checkpoint_prepare/ checkpoint_restore entry points on Worker and AsyncLLM. This change does not duplicate any of that. It adds the one thing the lifecycle cannot express: a fail-closed refusal, before any state is destroyed, for the configuration the hooks do not cover. The communicator hooks remain no-ops for NCCL pending an NCCLCheckpoint shim, so a multi-process suspend under a non-preserving backend would otherwise walk cleanly and leave NCCL state live underneath. The guard keys on world_size_across_dp rather than world_size because a TP=1, DP>1 deployment is still multi-process. A single-process deployment cannot hold a device communicator at all (GroupCoordinator builds one only when world_size > 1), so it is deliberately not refused. Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
heliubj18
added a commit
to heliubj18/vllm
that referenced
this pull request
Aug 10, 2026
It passed in vllm-project#30 and again in vllm-project#31. On a gpu_slots: 1 queue a passing step is not free: 38 minutes of it sat in front of the two steps still under debug, and the GPU concurrency group serialises everything behind it. Leaves Async Engine and Multi-Modal Processor as the only auto steps, which is what the next round needs to measure. Co-authored-by: Claude Opus 5 Signed-off-by: he-liu <he.liu@ppio.cn>
sharvil10
added a commit
to sharvil10/vllm
that referenced
this pull request
Oct 15, 2026
Kueue was removed from the vllm-ci cluster (vllm-ci-fluxCD vllm-project#30), so the kueue.x-k8s.io/queue-name and priority-class pod labels no longer do anything. Pods go straight to the default scheduler. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Chloe-orz
pushed a commit
to Chloe-orz/vllm-maintomain
that referenced
this pull request
Oct 26, 2026
[Bugfix][Lwd] LWD multi-request full-pipeline fixes — stable branch (vllm side)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Example usage:
python benchmark/benchmark_text_completion.py --dataset alpaca_opt_text_completion.pkl --model facebook/opt-13b --request-rate 1.0 --duration 3600 --n1 1.0python benchmark/benchmark_text_completion.py --dataset alpaca_opt_text_completion.pkl --model facebook/opt-13b --request-rate 1.0 --duration 3600 --n2 1.0python benchmark/benchmark_text_completion.py --dataset alpaca_opt_text_completion.pkl --model facebook/opt-13b --request-rate 1.0 --duration 3600 --n2-beam 1.0