Issue with compiling torch sparse #8
Description
Hi @Looong01,
Thank you so much for creating the pyg-rocm-build. Apart from torch sparse, everything else is working fine for me. However when I am compiling the torch-sparse, it is failing. I am using torch 2.4.1+rocm6.1 and its on LUMI supercomputer. Below is the error:
Processing /pfs/lustrep1/projappl/project_462000xxx/py_env/OCP_env/pyg-rocm-build-7/pytorch_sparse-0.6.18
Preparing metadata (setup.py) ... done
Requirement already satisfied: scipy in /opt/miniconda3/envs/pytorch/lib/python3.12/site-packages (from torch_sparse==0.6.18) (1.13.1)
Requirement already satisfied: numpy<2.3,>=1.22.4 in /users/bhatiani/.local/lib/python3.12/site-packages (from scipy->torch_sparse==0.6.18) (1.26.4)
Building wheels for collected packages: torch_sparse
Building wheel for torch_sparse (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [99 lines of output]
running bdist_wheel
running build
running build_py
running egg_info
writing torch_sparse.egg-info/PKG-INFO
writing dependency_links to torch_sparse.egg-info/dependency_links.txt
writing requirements to torch_sparse.egg-info/requires.txt
writing top-level names to torch_sparse.egg-info/top_level.txt
reading manifest file 'torch_sparse.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*' under directory 'third_party'
warning: no previously-included files matching '*' found under directory 'third_party/parallel-hashmap/css'
warning: no previously-included files matching '*' found under directory 'third_party/parallel-hashmap/html'
warning: no previously-included files matching '*' found under directory 'third_party/parallel-hashmap/tests'
warning: no previously-included files matching '*' found under directory 'third_party/parallel-hashmap/examples'
warning: no previously-included files matching '*' found under directory 'third_party/parallel-hashmap/benchmark'
warning: no previously-included files matching '*' found under directory 'benchmark'
adding license file 'LICENSE'
writing manifest file 'torch_sparse.egg-info/SOURCES.txt'
running build_ext
building 'torch_sparse._neighbor_sample_cpu' extension
Emitting ninja build file /pfs/lustrep1/projappl/project_462000612/Nitik_py_env/OCP_env/pyg-rocm-build-7/pytorch_sparse-0.6.18/build/temp.linux-x86_64-cpython-312/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/1] g++-12 -MMD -MF /pfs/lustrep1/projappl/project_462000612/Nitik_py_env/OCP_env/pyg-rocm-build-7/pytorch_sparse-0.6.18/build/temp.linux-x86_64-cpython-312/csrc/cpu/neighbor_sample_cpu.o.d -pthread -B /opt/miniconda3/envs/pytorch/compiler_compat -fno-strict-overflow -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /opt/miniconda3/envs/pytorch/include -fPIC -O2 -isystem /opt/miniconda3/envs/pytorch/include -fPIC -DWITH_PYTHON -Icsrc -I/pfs/lustrep1/projappl/project_462000612/Nitik_py_env/OCP_env/pyg-rocm-build-7/pytorch_sparse-0.6.18/third_party/parallel-hashmap -I/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/torch/include -I/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/torch/include/torch/csrc/api/include -I/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/torch/include/TH -I/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/torch/include/THC -I/user-software/venv/pytorch/include -I/opt/miniconda3/envs/pytorch/include/python3.12 -c -c /pfs/lustrep1/projappl/project_462000612/Nitik_py_env/OCP_env/pyg-rocm-build-7/pytorch_sparse-0.6.18/csrc/cpu/neighbor_sample_cpu.cpp -o /pfs/lustrep1/projappl/project_462000612/Nitik_py_env/OCP_env/pyg-rocm-build-7/pytorch_sparse-0.6.18/build/temp.linux-x86_64-cpython-312/csrc/cpu/neighbor_sample_cpu.o -fPIC -D__HIP_PLATFORM_AMD__=1 -DUSE_ROCM=1 -DHIPBLAS_V2 -fPIC -D__HIP_PLATFORM_AMD__=1 -DUSE_ROCM=1 -DHIPBLAS_V2 -O3 -Wno-sign-compare -DAT_PARALLEL_OPENMP -fopenmp -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_neighbor_sample_cpu -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
FAILED: /pfs/lustrep1/projappl/project_462000612/Nitik_py_env/OCP_env/pyg-rocm-build-7/pytorch_sparse-0.6.18/build/temp.linux-x86_64-cpython-312/csrc/cpu/neighbor_sample_cpu.o
g++-12 -MMD -MF /pfs/lustrep1/projappl/project_462000612/Nitik_py_env/OCP_env/pyg-rocm-build-7/pytorch_sparse-0.6.18/build/temp.linux-x86_64-cpython-312/csrc/cpu/neighbor_sample_cpu.o.d -pthread -B /opt/miniconda3/envs/pytorch/compiler_compat -fno-strict-overflow -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /opt/miniconda3/envs/pytorch/include -fPIC -O2 -isystem /opt/miniconda3/envs/pytorch/include -fPIC -DWITH_PYTHON -Icsrc -I/pfs/lustrep1/projappl/project_462000612/Nitik_py_env/OCP_env/pyg-rocm-build-7/pytorch_sparse-0.6.18/third_party/parallel-hashmap -I/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/torch/include -I/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/torch/include/torch/csrc/api/include -I/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/torch/include/TH -I/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/torch/include/THC -I/user-software/venv/pytorch/include -I/opt/miniconda3/envs/pytorch/include/python3.12 -c -c /pfs/lustrep1/projappl/project_462000612/Nitik_py_env/OCP_env/pyg-rocm-build-7/pytorch_sparse-0.6.18/csrc/cpu/neighbor_sample_cpu.cpp -o /pfs/lustrep1/projappl/project_462000612/Nitik_py_env/OCP_env/pyg-rocm-build-7/pytorch_sparse-0.6.18/build/temp.linux-x86_64-cpython-312/csrc/cpu/neighbor_sample_cpu.o -fPIC -D__HIP_PLATFORM_AMD__=1 -DUSE_ROCM=1 -DHIPBLAS_V2 -fPIC -D__HIP_PLATFORM_AMD__=1 -DUSE_ROCM=1 -DHIPBLAS_V2 -O3 -Wno-sign-compare -DAT_PARALLEL_OPENMP -fopenmp -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_neighbor_sample_cpu -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
In file included from /pfs/lustrep1/projappl/project_462000612/Nitik_py_env/OCP_env/pyg-rocm-build-7/pytorch_sparse-0.6.18/csrc/cpu/neighbor_sample_cpu.cpp:3:
/pfs/lustrep1/projappl/project_462000612/Nitik_py_env/OCP_env/pyg-rocm-build-7/pytorch_sparse-0.6.18/csrc/cpu/utils.h:4:10: fatal error: parallel_hashmap/phmap.h: No such file or directory
4 | #include "parallel_hashmap/phmap.h"
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 2105, in _run_ninja_build
subprocess.run(
File "/opt/miniconda3/envs/pytorch/lib/python3.12/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/pfs/lustrep1/projappl/project_462000612/Nitik_py_env/OCP_env/pyg-rocm-build-7/pytorch_sparse-0.6.18/setup.py", line 147, in <module>
setup(
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/setuptools/__init__.py", line 117, in setup
return distutils.core.setup(**attrs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 183, in setup
return run_commands(dist)
^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 199, in run_commands
dist.run_commands()
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 954, in run_commands
self.run_command(cmd)
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/setuptools/dist.py", line 995, in run_command
super().run_command(command)
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
cmd_obj.run()
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/setuptools/command/bdist_wheel.py", line 381, in run
self.run_command("build")
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
self.distribution.run_command(command)
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/setuptools/dist.py", line 995, in run_command
super().run_command(command)
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
cmd_obj.run()
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/setuptools/_distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
self.distribution.run_command(command)
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/setuptools/dist.py", line 995, in run_command
super().run_command(command)
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
cmd_obj.run()
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 99, in run
_build_ext.run(self)
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
self.build_extensions()
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 866, in build_extensions
build_ext.build_extensions(self)
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 476, in build_extensions
self._build_extensions_serial()
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 502, in _build_extensions_serial
self.build_extension(ext)
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 264, in build_extension
_build_ext.build_extension(self, ext)
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 557, in build_extension
objects = self.compiler.compile(
^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 679, in unix_wrap_ninja_compile
_write_ninja_file_and_compile_objects(
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 1785, in _write_ninja_file_and_compile_objects
_run_ninja_build(
File "/opt/miniconda3/envs/pytorch/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 2121, in _run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for torch_sparse
Running setup.py clean for torch_sparse
Failed to build torch_sparse
ERROR: Could not build wheels for torch_sparse, which is required to install pyproject.toml-based projects
´´´
Could you please help me with this? Thank you.