Build failed with cuBLAS #205

AlexiaChen · 2023-05-14T09:43:09Z

mathxh@mathxh-server:~$ CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip install llama-cpp-python
Defaulting to user installation because normal site-packages is not writeable
Collecting llama-cpp-python
  Downloading https://mirrors.aliyun.com/pypi/packages/82/2c/9614ef76422168fde5326095559f271a22b1926185add8ae739901e113b9/llama_cpp_python-0.1.50.tar.gz (1.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 4.2 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: typing-extensions>=4.5.0 in ./.local/lib/python3.10/site-packages (from llama-cpp-python) (4.5.0)
Building wheels for collected packages: llama-cpp-python
  Building wheel for llama-cpp-python (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for llama-cpp-python (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [109 lines of output]


      --------------------------------------------------------------------------------
      -- Trying 'Ninja' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      -- The C compiler identification is GNU 11.3.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- The CXX compiler identification is GNU 11.3.0
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Configuring done (0.6s)
      -- Generating done (0.0s)
      -- Build files have been written to: /tmp/pip-install-qzc9tklk/llama-cpp-python_6fcd1f65f0b54bae869069faa39cdc1f/_cmake_test_compile/build
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'Ninja' generator - success
      --------------------------------------------------------------------------------

      Configuring Project
        Working directory:
          /tmp/pip-install-qzc9tklk/llama-cpp-python_6fcd1f65f0b54bae869069faa39cdc1f/_skbuild/linux-x86_64-3.10/cmake-build
        Command:
          /tmp/pip-build-env-t04dq_3v/overlay/local/lib/python3.10/dist-packages/cmake/data/bin/cmake /tmp/pip-install-qzc9tklk/llama-cpp-python_6fcd1f65f0b54bae869069faa39cdc1f -G Ninja --no-warn-unused-cli -DCMAKE_INSTALL_PREFIX:PATH=/tmp/pip-install-qzc9tklk/llama-cpp-python_6fcd1f65f0b54bae869069faa39cdc1f/_skbuild/linux-x86_64-3.10/cmake-install -DPYTHON_VERSION_STRING:STRING=3.10.6 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-t04dq_3v/overlay/local/lib/python3.10/dist-packages/skbuild/resources/cmake -DPYTHON_EXECUTABLE:PATH=/usr/bin/python3 -DPYTHON_INCLUDE_DIR:PATH=/usr/include/python3.10 -DPYTHON_LIBRARY:PATH=/usr/lib/x86_64-linux-gnu/libpython3.10.so -DPython_EXECUTABLE:PATH=/usr/bin/python3 -DPython_ROOT_DIR:PATH=/usr -DPython_FIND_REGISTRY:STRING=NEVER -DPython_INCLUDE_DIR:PATH=/usr/include/python3.10 -DPython3_EXECUTABLE:PATH=/usr/bin/python3 -DPython3_ROOT_DIR:PATH=/usr -DPython3_FIND_REGISTRY:STRING=NEVER -DPython3_INCLUDE_DIR:PATH=/usr/include/python3.10 -DCMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-t04dq_3v/overlay/local/lib/python3.10/dist-packages/ninja/data/bin/ninja -DLLAMA_CUBLAS=on -DCMAKE_BUILD_TYPE:STRING=Release -DLLAMA_CUBLAS=on

      Not searching for unused variables given on the command line.
      -- The C compiler identification is GNU 11.3.0
      -- The CXX compiler identification is GNU 11.3.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      CMake Warning (dev) in CMakeLists.txt:
        A logical block opening on the line

          /tmp/pip-install-qzc9tklk/llama-cpp-python_6fcd1f65f0b54bae869069faa39cdc1f/CMakeLists.txt:9 (if)

        closes on the line

          /tmp/pip-install-qzc9tklk/llama-cpp-python_6fcd1f65f0b54bae869069faa39cdc1f/CMakeLists.txt:31 (endif)

        with mis-matching arguments.
      This warning is for project developers.  Use -Wno-dev to suppress it.

      -- Found Git: /usr/bin/git (found version "2.34.1")
      fatal: not a git repository (or any of the parent directories): .git
      fatal: not a git repository (or any of the parent directories): .git
      CMake Warning at vendor/llama.cpp/CMakeLists.txt:105 (message):
        Git repository not found; to enable automatic generation of build info,
        make sure Git is installed and the project is a Git repository.


      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
      -- Found Threads: TRUE
      -- Found CUDAToolkit: /usr/local/cuda/include (found version "12.1.105")
      -- cuBLAS found
      -- The CUDA compiler identification is unknown
      CMake Error at /tmp/pip-build-env-t04dq_3v/overlay/local/lib/python3.10/dist-packages/cmake/data/share/cmake-3.26/Modules/CMakeDetermineCUDACompiler.cmake:603 (message):
        Failed to detect a default CUDA architecture.



        Compiler output:

      Call Stack (most recent call first):
        vendor/llama.cpp/CMakeLists.txt:188 (enable_language)


      -- Configuring incomplete, errors occurred!
      Traceback (most recent call last):
        File "/tmp/pip-build-env-t04dq_3v/overlay/local/lib/python3.10/dist-packages/skbuild/setuptools_wrap.py", line 666, in setup
          env = cmkr.configure(
        File "/tmp/pip-build-env-t04dq_3v/overlay/local/lib/python3.10/dist-packages/skbuild/cmaker.py", line 356, in configure
          raise SKBuildError(msg)

      An error occurred while configuring with CMake.
        Command:
          /tmp/pip-build-env-t04dq_3v/overlay/local/lib/python3.10/dist-packages/cmake/data/bin/cmake /tmp/pip-install-qzc9tklk/llama-cpp-python_6fcd1f65f0b54bae869069faa39cdc1f -G Ninja --no-warn-unused-cli -DCMAKE_INSTALL_PREFIX:PATH=/tmp/pip-install-qzc9tklk/llama-cpp-python_6fcd1f65f0b54bae869069faa39cdc1f/_skbuild/linux-x86_64-3.10/cmake-install -DPYTHON_VERSION_STRING:STRING=3.10.6 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-t04dq_3v/overlay/local/lib/python3.10/dist-packages/skbuild/resources/cmake -DPYTHON_EXECUTABLE:PATH=/usr/bin/python3 -DPYTHON_INCLUDE_DIR:PATH=/usr/include/python3.10 -DPYTHON_LIBRARY:PATH=/usr/lib/x86_64-linux-gnu/libpython3.10.so -DPython_EXECUTABLE:PATH=/usr/bin/python3 -DPython_ROOT_DIR:PATH=/usr -DPython_FIND_REGISTRY:STRING=NEVER -DPython_INCLUDE_DIR:PATH=/usr/include/python3.10 -DPython3_EXECUTABLE:PATH=/usr/bin/python3 -DPython3_ROOT_DIR:PATH=/usr -DPython3_FIND_REGISTRY:STRING=NEVER -DPython3_INCLUDE_DIR:PATH=/usr/include/python3.10 -DCMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-t04dq_3v/overlay/local/lib/python3.10/dist-packages/ninja/data/bin/ninja -DLLAMA_CUBLAS=on -DCMAKE_BUILD_TYPE:STRING=Release -DLLAMA_CUBLAS=on
        Source directory:
          /tmp/pip-install-qzc9tklk/llama-cpp-python_6fcd1f65f0b54bae869069faa39cdc1f
        Working directory:
          /tmp/pip-install-qzc9tklk/llama-cpp-python_6fcd1f65f0b54bae869069faa39cdc1f/_skbuild/linux-x86_64-3.10/cmake-build
      Please see CMake's output for more information.

      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for llama-cpp-python
Failed to build llama-cpp-python
ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects

AlexiaChen · 2023-05-14T09:44:05Z

Seems can not detect cuda arch. but I installed cuda as follows and CUDA toolkit already are found from CMAKE above:

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.0-1_all.deb
sudo dpkg -i cuda-keyring_1.0-1_all.deb
sudo apt-get update
sudo apt-get -y install cuda

My Display card:

*-display
       description: VGA compatible controller
       product: GA106M [GeForce RTX 3060 Mobile / Max-Q]
       vendor: NVIDIA Corporation
       physical id: 0
       bus info: pci@0000:03:00.0
       logical name: /dev/fb0
       version: a1
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress vga_controller bus_master cap_list rom fb
       configuration: depth=32 driver=nouveau latency=0 mode=1920x1080 resolution=1920,1080 visual=truecolor xres=1920 yres=1080
       resources: iomemory:383c0-383bf iomemory:383e0-383df irq:33 memory:fa000000-faffffff memory:383c00000000-383dffffffff memory:383e00000000-383e01ffffff ioport:e000(size=128) memory:fb000000-fb07ffff

gjmulder · 2023-05-14T19:26:55Z

    Failed to detect a default CUDA architecture.

Did you install CUDA?

AlexiaChen · 2023-05-14T23:57:04Z

    Failed to detect a default CUDA architecture.

Did you install CUDA?

yes. i already installed cuda i mentioned above. @gjmulder

gjmulder · 2023-05-15T07:08:20Z

Apologies. So you did.

I saw this error with cmake builds a while back. Can you try building from source as per my attached Dockerfile? This Dockerfile works with the latest code.

Dockerfile.txt

AlphaAtlas · 2023-05-23T21:23:48Z

I think this is related to a warning/issue I ran into building vanilla llama.cpp with cmake. I fixed it by manually specifying the CUDA architecture: https://cmake.org/cmake/help/latest/variable/CMAKE_CUDA_ARCHITECTURES.html
set(CMAKE_CUDA_ARCHITECTURES 75)

In fact
set(CMAKE_CUDA_ARCHITECTURES native)
Is probably a good thing to put in this repo's cmake file.

abetlen · 2023-05-23T21:42:29Z

@AlphaAtlas I can add it here temporarily but do you mind reporting this issue in llama.cpp as that's the more appropriate place to set that flag.

gjmulder · 2023-05-24T07:23:18Z

set(CMAKE_CUDA_ARCHITECTURES native)

This will potentially need a lot of regression testing against different CUDA installs, as I do not have this problem.

EDIT:

Try running:

nvcc --version

I suspect the problem may be the build can't find nvcc:

if (GGML_CUDA_SOURCES)
    message(STATUS "GGML CUDA sources found, configuring CUDA architecture")
    set_property(TARGET ggml PROPERTY CUDA_ARCHITECTURES OFF)
    set_property(TARGET ggml PROPERTY CUDA_SELECT_NVCC_ARCH_FLAGS "Auto")
    set_property(TARGET llama PROPERTY CUDA_ARCHITECTURES OFF)
endif()

gjmulder · 2023-06-02T13:57:00Z

Closing. Reopen if the issue is not resolved upstream in llama.cpp

gjmulder added build hardware Hardware specific issue labels May 15, 2023

mindwellsolutions mentioned this issue May 19, 2023

Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects #244

Closed

gjmulder closed this as completed Jun 2, 2023

jamesbraza mentioned this issue Oct 3, 2023

Installing Llama.cpp without nvcc from nvidia-cuda-toolkit ggml-org/llama.cpp#3459

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build failed with cuBLAS #205

Build failed with cuBLAS #205

AlexiaChen commented May 14, 2023

AlexiaChen commented May 14, 2023 •

edited

Loading

gjmulder commented May 14, 2023

AlexiaChen commented May 14, 2023

gjmulder commented May 15, 2023

AlphaAtlas commented May 23, 2023 •

edited

Loading

abetlen commented May 23, 2023

gjmulder commented May 24, 2023 •

edited

Loading

gjmulder commented Jun 2, 2023

Build failed with cuBLAS #205

Build failed with cuBLAS #205

Comments

AlexiaChen commented May 14, 2023

AlexiaChen commented May 14, 2023 • edited Loading

gjmulder commented May 14, 2023

AlexiaChen commented May 14, 2023

gjmulder commented May 15, 2023

AlphaAtlas commented May 23, 2023 • edited Loading

abetlen commented May 23, 2023

gjmulder commented May 24, 2023 • edited Loading

gjmulder commented Jun 2, 2023

AlexiaChen commented May 14, 2023 •

edited

Loading

AlphaAtlas commented May 23, 2023 •

edited

Loading

gjmulder commented May 24, 2023 •

edited

Loading