Support for GPU acceleration on Linux hosts

[unsuman]

Currently GPU accelerated Lima VMs are only possible on macOS using the Krunkit driver. We should consider adding native GPU passthrough(VFIO) to Linux hosts using the Qemu driver to provide near native CUDA/ROCm performance for AI workloads and possibly some gaming(when we support Windows guests in the future)🕹️.

However, there’s a caveat: PCI passthrough only works with multiple graphics cards. A workaround does exists for passing a single graphics card. The downside is that you will loose display on the host side(if running a desktop env) and have to use SSH to control it from the guest unless you have a pair of dGPUs or a single dGPU with iGPU.

A rough workflow

• Step 1: Enabling IOMMU on the host for device isolation (Requires sudo)
• Step 2: Identifying the target GPU's PCI address. The whole group has to be passed like audio controller and any other devices present (No sudo)
• Step 3: Unbinding from the host and binding it to VFIO(/dev/vfio/<group_number>), also changing the ownership (Requires sudo)
• Step 4: Launching the Lima VM by adding the PCI address of their GPU to a new field pciAddresses under QEMUOpts (No sudo)
• Step 5: Teardown and restoration of the GPU from vfio-pci (Requires sudo)

Workaround for the privileged needs

The steps 1, 3 & 5 are the main hurdles here. Maybe we can

• Create a separate standalone executable like lima-vfio-helper like how we currently do for socket_vmnet?

$ limactl sudoers | sudo tee /etc/sudoers.d/lima
$ limactl start --name=default template://vmnet

• Use Polkit?
• Provide a detailed doc and make the user configure the sudo parts?

[afbjorklund]

I think there were also some early experiments with other technologies, if that helps...
https://medium.com/nttlabs/accelerating-llama-on-lima-with-wasi-nn-rpc-06b84bcbbe5c

[unsuman]

Thanks, this is easy to implement and doesn't require root privileges but it seems not to support Windows Guests and ofc doesn’t offer near-native performance

[afbjorklund]

and possibly some gaming(when we support Windows guests in the future)

You don't need Windows for gaming (any more), as proven by Steam and others

[unsuman]

Yeah that is very true but some games which require their anti-cheat software to run in kernel-level like Vanguard for Valorant, Ricochet for Call of Duty etc. require Windows

[afbjorklund]

I somehow feel that I need to link https://www.youtube.com/watch?v=CPRvc2UMeMI

[unsuman]

This video isn't available anymore

[afbjorklund]

Fixed the link, apparently "share" was broken for some reason (it's a little about Pong)

[jandubois]

We built Pong (well, TV Tennis) with TTL chips when I was in high school. Took us a whole year to learn all about it (debugging TV signals with an oscilloscope), but we did everything from scratch, including etching our own circuit boards¹. I think only 2 of the 5 machines we built worked by the end of the year.

Footnotes

1. I know it sounds like "We walked barefoot to school, in snow, against the wind, both ways...", but it is true. ↩