Skip to content

[Request] nice-ifying build failures around hardware acceleration #793

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jamesbraza opened this issue Oct 5, 2023 · 0 comments
Open
Labels
enhancement New feature or request

Comments

@jamesbraza
Copy link
Contributor

Build scripts support many types of hardware acceleration: https://github.com/abetlen/llama-cpp-python/blob/v0.2.11/README.md#installation-with-hardware-acceleration

# cuBLAS
CMAKE_ARGS="-DLLAMA_CUBLAS=on" pip install llama-cpp-python
# Metal
CMAKE_ARGS="-DLLAMA_METAL=on" pip install llama-cpp-python

If one is missing OS-level dependencies, the errors thrown can be somewhat arcane, to those not informed. The request is to nice-ify these messages, providing context and/or solutions.

For example, when installing on an AWS EC2 g4dn.4xlarge, without nvcc installed via nvidia-cuda-toolkit:

      CMake Error at /tmp/pip-build-env-xoh3dnhj/normal/lib/python3.11/site-packages/cmake/data/share/cmake-3.27/Modules/CMakeDetermineCUDACompiler.cmake:603 (message):
        Failed to detect a default CUDA architecture.

This can be nice-ified to be:

        Failed to detect a default CUDA architecture.
        cuBLAS backend failed: do you have nvcc installed?

Or when accidentally pasting in the wrong CMAKE_ARGS (here I mixed up Metal and cuBLAS):

      CMake Error at vendor/llama.cpp/CMakeLists.txt:174 (find_library):
        Could not find FOUNDATION_LIBRARY using the following names: Foundation

This can be nice-ified to be:

        Could not find FOUNDATION_LIBRARY using the following names: Foundation
        Metal backend failed: are you on a Mac?

In Python, Exceptions can be re-raised using from: raise Exception("Nice message.") from arcane_exc. I am not sure if this is something supported by CMakeLists.txt, but I think it would be really helpful to provide better failure messages for the hardware acceleration specifics.

For context, these errors were encountered in ggml-org/llama.cpp#3459.

@abetlen abetlen added the enhancement New feature or request label Dec 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants