Maintainers & Contributors: AMD is giving out 100 Strix Halo desktops for developers

[Mushoz]

For all the wonderful maintainers and contributors here, AMD is offering 100 free Strix Halo desktop PCs for developers. It would be wonderful if some of the developers working on llamacpp could get access to this machine for optimization purposes :)

https://www.amd.com/en/forms/sign-up/framework-desktop-giveaway.html

[Mushoz]

@JohannesGaessler @ggerganov @ngxson @CISC @0cc4m @jeffbolznv

My apologies if I missed anyone! You are all doing an amazing job. Hoping you will get selected should you apply. My apologies if you are not interested in the offer. Just wanted to make sure it reached the right people here :)

[0cc4m]

Yeah, I wanted to apply for that. I could use it for optimization work, but I'm in Germany, so not eligible.

[JohannesGaessler]

I am not aware of any American/Canadian devs having worked on HIP performance (with the exception of AMD engineers who presumably already have access to hardware anyways).

[Mushoz]

It's a bummer it's so restricted. This offer was posted in the lemonade Discord which runs llamacpp as its backend. A llamacpp developer getting one of these would be absolutely perfect for them.

I have shared a link to this topic in the discord. Hoping the offer can be extended outside the US/Canada.

[Mushoz]

So I have spoken with an AMD employee, and this is what they said:

So if anyone is interested in working on optimizing llamacpp for Strix Halo, feel free to send that provided email address an email. I know you said you wanted to apply for optimization work @0cc4m , but obviously this would apply to anyone here that had similar plans.

Really hoping this can get some of these machines in the hands of developers!

[netrunnereve]

IMO it would make more sense for them to send us FAEs for ROCM and Vulkan and help out directly. Nvidia, Intel, Qualcomm, IBM, and other smaller companies have been doing this already for a long time. I think AMD helps out with vllm already, so why not us?

If they want to provide hardware instead then I think those should go to the ggml company as a whole (rather than individual devs who may run off) to be used for CI and assigned to contributors if they need it. Like if someone needs more memory for supporting a big new model or wants to test an optimization they can ssh into the machine and use it for that.

[ExtReMLapin]

Btw the European Commission/europol can gives €€€ if you work on innovatives project like llama.cpp

[JohannesGaessler]

Quite frankly, I already have sufficient funds for the "cheap" hardware that I'm interested in and I have income independent of llama.cpp/ggml from my PhD (which typically also makes me ineligible for open-source fellowship programs). I don't have anything in my private life that I would want to spend significant amounts of money on either. I will accept hardware relevant to my interests if it's given without strings attached but unless it's on the level of H100s I could just buy it myself if I wanted to.