Custom CUDA extensions

[msaroufim]

This is the mergaeble version of #130 - some updates I have to make

• Add a skip test unless pytorch 2.4+ is used and Add a skip test if cuda is not available
• Add ninja to dev dependencies
• Locally had to ensure conda install nvidia/label/cuda-12.1.0::cuda-toolkit to get things working
• Added continue-on-error: true to CI otherwise was difficult to debug
• Add a new pyproject.toml for build time dependencies
• Do everything from within a docker container and then everything within a conda env
• Add a pyproject.toml with torch as a build time dependency to fix CI issue (even though this was working locally) or do python setup.py install instead
• Add verbose pip logging so we can tell that the cuda extension was not built
• Move kernels to a test namespace so we can keep testing in CI
• Using both a custom docker image and conda environment is more than doubling the total CI runtime, not great

OK for manylinux I managed to repro the missing torch error locally (FINALLY) using this

CIBW_BUILD=cp310-macosx_* python -m cibuildwheel --output-dir wheelhouse

Follow up PRs

• Setup manylinux in github action CI: should i use cibuildwheel? in which case how do I configure CUDA? If I instead use the pytorch images, how do I get setup?
• Binary tests, how do i ensure that the cuda kernels we built actually work
• Add an actually useful kernel as a first example and remove the rms norm kernel like paged attention

[weifengpy]

do we need to update README? for building from source, is it python setup.py developer now? I spent some time to understand why pip install -e . does not work anymore and find this PR

[msaroufim]

argh, lemme update now

[liangan1]

@msaroufim whether CPU kernel extension works with this PR?

[msaroufim]

Yup cpu extensions work perfectly fine