Skip to content

Sync 0.8.4 tag of NM midstream repo to RHOAI 2.19.1 #116

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
ckhordiasma merged 1,140 commits into from
Apr 24, 2025
Merged

Sync 0.8.4 tag of NM midstream repo to RHOAI 2.19.1 #116

ckhordiasma merged 1,140 commits into from
Apr 24, 2025

Conversation

@heyselbi
Copy link

@heyselbi heyselbi commented Apr 23, 2025 •
edited by github-actions bot
Loading

In preparation for 2.19.1 release.
Addresses: https://issues.redhat.com/browse/INFERENG-612

bigPYJ1151 and others added 30 commits April 2, 2025 07:46
@bigPYJ1151
[CPU][Bugfix] Using custom allreduce for CPU backend (vllm-project#15934
550b280 
)

Signed-off-by: jiang1.li <[email protected]>
@lengrongfu
[Model] use AutoWeightsLoader in model load_weights (vllm-project#15770)
e86c414 
Signed-off-by: rongfu.leng <[email protected]>
@jeejeelee
[Misc] V1 LoRA support CPU offload (vllm-project#15843)
58e234a 
Signed-off-by: Jee Jee Li <[email protected]>
@npanpaliya
Restricted cmake to be less than version 4 as 4.x breaks the build of… (
8bd651b 
vllm-project#15859)

Signed-off-by: Nishidha Panpaliya <[email protected]>
@youkaichao
[misc] instruct pytorch to use nvml-based cuda check (vllm-project#15951
1cab43c 
)

Signed-off-by: youkaichao <[email protected]>
@mgoin
[V1] Support Mistral3 in V1 (vllm-project#15950)
f021b97 
Signed-off-by: mgoin <[email protected]>
@hmellor
Fix huggingface-cli[hf-xet] -> huggingface-cli[hf_xet] (vllm-proj…
55acf86 
…ect#15969)

Signed-off-by: Harry Mellor <[email protected]>
@hyeygit
[V1][TPU] TPU-optimized top-p implementation (avoids scattering). (vl…
1b84eff 
…lm-project#15736)

Signed-off-by: Hyesoo Yang <[email protected]>
Co-authored-by: root <root@t1v-n-822696b7-w-0.us-central2-b.c.tpu-prod-env-large-adhoc.internal>
@yaochengji
[TPU] optimize the all-reduce performance (vllm-project#15903)
01b6113 
Signed-off-by: Chengji Yao <[email protected]>
@NickLucche
[V1][TPU] Do not compile sampling more than needed (vllm-project#15883)
bd7599d 
Signed-off-by: NickLucche <[email protected]>
@maleksan85
[ROCM][KERNEL] Paged attention for V1 (vllm-project#15720)
e73ff24 
Signed-off-by: Aleksandr Malyshev <[email protected]>
Signed-off-by: root <[email protected]>
Co-authored-by: Aleksandr Malyshev <[email protected]>
Co-authored-by: root <[email protected]>
@yihong0618
fix: better error message for get_config close vllm-project#13889 (vl…
37bfee9 
…lm-project#15943)

Signed-off-by: yihong0618 <[email protected]>
@youkaichao
[bugfix] add seed in torchrun_example.py (vllm-project#15980)
8b66470 
Signed-off-by: youkaichao <[email protected]>
@maleksan85
[ROCM][V0] PA kennel selection when no sliding window provided (vllm-…
57a810d 
…project#15982)

Signed-off-by: Aleksandr Malyshev <[email protected]>
Co-authored-by: Aleksandr Malyshev <[email protected]>
@StevenShi-23
[Benchmark] Add AIMO Dataset to Benchmark (vllm-project#15955)
06f21ce 
Signed-off-by: Ziji Shi <[email protected]>
Signed-off-by: StevenShi-23 <[email protected]>
@youkaichao
[misc] improve error message for "Failed to infer device type" (vllm-…
5e125e7 
…project#15994)

Signed-off-by: youkaichao <[email protected]>
@wwl2755
[Bugfix][V1] Fix bug from putting llm_engine.model_executor in a back…
463bbb1 
…ground process (vllm-project#15367)

Signed-off-by: wwl2755 <[email protected]>
@reidliu41
[doc] update contribution link (vllm-project#15922)
a43aa18 
Signed-off-by: reidliu41 <[email protected]>
Co-authored-by: reidliu41 <[email protected]>
@yihong0618
fix: tiny fix make format.sh excutable (vllm-project#16015)
84884cd 
Signed-off-by: yihong0618 <[email protected]>
@kylesayrs
[SupportsQuant] Bert, Blip, Blip2, Bloom (vllm-project#15573)
421c462 
Signed-off-by: Kyle Sayers <[email protected]>
@kylesayrs
[SupportsQuant] Chameleon, Chatglm, Commandr (vllm-project#15952)
82e7e19 
Signed-off-by: Kyle Sayers <[email protected]>
@andy-neuma
upstreamsync-20250330 (#203)
849c32e 
SUMMARY:

upstream sync to `bb103b29bf3d26fa831458a8ca5af1bb820c4712` this is as
of 2025-03-30 Boston AM.

```bash
> git log
commit f44d253 (HEAD -> upstreamsync-20250330, origin/upstreamsync-20250330)
Merge: 2602d9d bb103b2
Author: andy-neuma <[email protected]>
Date:   Sun Mar 30 11:17:52 2025 -0400

    Merge remote-tracking branch 'upstream/main'

commit bb103b2 (upstream/main)
Author: kYLe <[email protected]>
Date:   Sun Mar 30 05:45:08 2025 -0500

    [Bugfix] Added `embed_is_patch` mask for fuyu model (vllm-project#15731)
    
    Signed-off-by: Kyle Huang <[email protected]>
...
```

created via "merge" not cherry-pick

TEST PLAN:
accept sync is green -
https://github.com/neuralmagic/nm-cicd/actions/runs/14225725956
@liangfu
[Neuron][kernel] Fuse kv cache into a single tensor (vllm-project#15911)
d2b58ca 
Signed-off-by: Liangfu Chen <[email protected]>
@bnellnm
[Minor] Fused experts refactor (vllm-project#15914)
15ba07e 
Signed-off-by: Bill Nell <[email protected]>
@yarongmu-google
[Misc][Performance] Advance tpu.txt to the most recent nightly torch … (
45b1ff7 
vllm-project#16024)
@Alexei-V-Ivanov-AMD
Re-enable the AMD Testing for the passing tests. (vllm-project#15586)
03a70ea 
Signed-off-by: Alexei V. Ivanov <[email protected]>
@vanbasten23
[TPU] Support sliding window and logit soft capping in the paged atte…
b6be6f8 
…ntion kernel for TPU. (vllm-project#15732)

Signed-off-by: Xiongfei Wei <[email protected]>
@robertgshaw2-redhat
[TPU] Switch Test to Non-Sliding Window (vllm-project#15981)
f15e70d 
Signed-off-by: Robert Shaw <[email protected]>
Co-authored-by: Robert Shaw <[email protected]>
@bnellnm
[Bugfix] Fix function names in test_block_fp8.py (vllm-project#16033)
dcc56d6 
Signed-off-by: Bill Nell <[email protected]>
@huydhn
[ROCm] Tweak the benchmark script to run on ROCm (vllm-project#14252)
092475f
@ckhordiasma
Copy link

ckhordiasma commented Apr 23, 2025

/build-cuda

@ckhordiasma
Copy link

ckhordiasma commented Apr 23, 2025

/build-cpu

@ckhordiasma
Copy link

ckhordiasma commented Apr 23, 2025

/build-powerz

@ckhordiasma ckhordiasma mentioned this pull request Apr 23, 2025
Closed
@heyselbi
Copy link
Author

heyselbi commented Apr 23, 2025

/build-cuda

@heyselbi
Copy link
Author

heyselbi commented Apr 24, 2025 •
edited
Loading

Pre-commit fail can be ignored. One of the fails is in the example directory, the other one suggests importing AsyncGenerator from collections.abc --> but that is recognized only in Python 3.9+. Importing AsyncGenerator from typing is recognized across all Python versions.

@heyselbi
Copy link
Author

heyselbi commented Apr 24, 2025

/build-powerz

@heyselbi
Copy link
Author

heyselbi commented Apr 24, 2025

Cuda build is successful. Now working on the power/z images, ensuring they build. Cherry-picked two commits for power/z that are necessary.

@heyselbi
Copy link
Author

heyselbi commented Apr 24, 2025

/build-powerz

@heyselbi
Copy link
Author

heyselbi commented Apr 24, 2025

/build-powerz

@heyselbi
Copy link
Author

heyselbi commented Apr 24, 2025

/build-powerz

@heyselbi
Copy link
Author

heyselbi commented Apr 24, 2025

/build-powerz

@heyselbi
Copy link
Author

heyselbi commented Apr 24, 2025

Power/Z image builds are both successful despite the failed result shown. Fail is due to some pipeline error unrelated to build of the images. PR is good to merge.

@ckhordiasma ckhordiasma merged commit 184a37f into rhoai-2.19 Apr 24, 2025
3 of 7 checks passed
@ckhordiasma ckhordiasma deleted the 084-sync-rhoai-219 branch May 15, 2025 14:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.