Running examples with DPC++ SYCL GPU-only

[frenchwr]

Hello, I am attempting to create a debian package (.deb) of oneDNN built with the DPC++ compiler. I am able to build oneDNN and am now attempting to build and run the examples distributed with this repo. I am targeting GPU-only SYCL devices, as Intel's OpenCL CPU implementation is not packaged in Debian or Ubuntu.

I have picked a few examples from the 3.9.1 release that seem relevant. In both cases I am able to build the examples and link with the SYCL-enabled libdnn, but both examples fail with different (but potentially related?) errors. I was hoping you might be able to shed some light.

Building oneDNN

I built the library with the following options:

-DCMAKE_CXX_COMPILER=clang++-dpcpp \
-DCMAKE_C_COMPILER=clang-dpcpp \
-DCMAKE_AR=/usr/lib/llvm-dpcpp-21/bin/llvm-ar \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DDNNL_EXPERIMENTAL_SPARSE=ON \
-DDNNL_EXPERIMENTAL_UKERNEL=${BUILD_UKERNEL} \
-DONEDNN_BUILD_TESTS=OFF \
-DONEDNN_BUILD_EXAMPLES=OFF \
-DONEDNN_BUILD_DOC=OFF \
-DONEDNN_ARCH_OPT_FLAGS="" \
-DONEDNN_CPU_RUNTIME=SYCL \
-DONEDNN_GPU_RUNTIME=SYCL \
-DDNNL_LIBRARY_NAME=dnnl-sycl

Note that clang++-dpcpp is the name of the C++ version of DPC++ that we are submitting to the Ubuntu archive (but it is not yet there). Also note I'm using dnnl-sycl for the library name to distinguish from the existing libdnnl (built with the GCC toolchain) in Debian and Ubuntu.

gpu_opencl_interop.cpp

$ clang++-dpcpp -DCL_TARGET_OPENCL_VERSION=300 -fsycl -fsycl-targets=spir64 -ldnnl-sycl -ltbb -lOpenCL gpu_opencl_interop.cpp -o gpu_opencl_interop

$ ldd ./gpu_opencl_interop
	linux-vdso.so.1 (0x00007263d9787000)
	libdnnl-sycl.so.3.6 => /lib/x86_64-linux-gnu/libdnnl-sycl.so.3.6 (0x00007263d4400000)
	libtbb.so.12 => /lib/x86_64-linux-gnu/libtbb.so.12 (0x00007263d9722000)
	libOpenCL.so.1 => /lib/x86_64-linux-gnu/libOpenCL.so.1 (0x00007263d970b000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007263d4000000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007263d95fe000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007263d43d3000)
	libsycl.so.8 => /lib/x86_64-linux-gnu/libsycl.so.8 (0x00007263d3a00000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007263d3600000)
	/lib64/ld-linux-x86-64.so.2 (0x00007263d9789000)
	libur_loader.so.0 => /lib/x86_64-linux-gnu/libur_loader.so.0 (0x00007263d2400000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007263d43b3000)

$ ./gpu_opencl_interop 
oneDNN error caught: 
	Status: invalid_arguments
	Message: could not get an OpenCL context from an engine
Example failed on GPU.

Note according to sycl-ls (see section below) I have a opencl:gpu device. I have tried filtering devices with export SYCL_DEVICE_FILTER=gpu, export ONEAPI_DEVICE_SELECTOR=opencl:gpu, and export DNNL_CPU_RUNTIME=NONE at runtime without any change.

sycl_interop_buffer.cpp

$ clang++-dpcpp -DCL_TARGET_OPENCL_VERSION=300 -fsycl -fsycl-targets=spir64 -ldnnl-sycl -ltbb -lOpenCL sycl_interop_buffer.cpp -o sycl_interop_buffer

$ ldd ./sycl_interop_buffer
	linux-vdso.so.1 (0x000071ad0c514000)
	libdnnl-sycl.so.3.6 => /lib/x86_64-linux-gnu/libdnnl-sycl.so.3.6 (0x000071ad07200000)
	libtbb.so.12 => /lib/x86_64-linux-gnu/libtbb.so.12 (0x000071ad0c4af000)
	libOpenCL.so.1 => /lib/x86_64-linux-gnu/libOpenCL.so.1 (0x000071ad0c498000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x000071ad06e00000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x000071ad070f3000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x000071ad0c469000)
	libsycl.so.8 => /lib/x86_64-linux-gnu/libsycl.so.8 (0x000071ad06800000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x000071ad06400000)
	/lib64/ld-linux-x86-64.so.2 (0x000071ad0c516000)
	libur_loader.so.0 => /lib/x86_64-linux-gnu/libur_loader.so.0 (0x000071ad05200000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x000071ad0c449000)

$ ./sycl_interop_buffer 
onednn_verbose,v1,info,oneDNN v3.9.1 (commit N/A)
onednn_verbose,v1,info,cpu,runtime:DPC++,nthr:22
onednn_verbose,v1,info,cpu,isa:Intel AVX2 with Intel DL Boost
onednn_verbose,v1,info,gpu,runtime:DPC++
onednn_verbose,v1,info,cpu,engine,sycl cpu device count:0 - no devices available.
onednn_verbose,v1,info,gpu,engine,sycl gpu device count:1 
onednn_verbose,v1,info,gpu,engine,0,backend:OpenCL,name:Intel(R) Arc(TM) Graphics,driver_version:25.31.34666,binary_kernels:enabled
onednn_verbose,v1,info,graph,backend,0:dnnl_backend
onednn_verbose,v1,primitive,info,template:operation,engine,primitive,implementation,prop_kind,memory_descriptors,attributes,auxiliary,problem_desc,exec_time
onednn_verbose,v1,graph,info,template:operation,engine,partition_id,partition_kind,op_names,data_formats,logical_tensors,fpmath_mode,implementation,backend,exec_time
onednn_verbose,v1,common,error,runtime,DPC++ cpu devices queried but not found,src/common/engine.cpp:102
oneDNN error caught: 
	Status: invalid_arguments
	Message: could not create an engine
Example failed on CPU.

I also tried device filtering like in the previous example without any luck.

Host system

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 25.10
Release:	25.10
Codename:	questing

$ sycl-ls
[level_zero:gpu][level_zero:0] Intel(R) oneAPI Unified Runtime over Level-Zero, Intel(R) Arc(TM) Graphics 12.71.4 [1.6.34666]
[opencl:gpu][opencl:0] Intel(R) OpenCL Graphics, Intel(R) Arc(TM) Graphics OpenCL 3.0 NEO  [25.31.034666]

$ lspci -nn | grep -i graphics
00:02.0 VGA compatible controller [0300]: Intel Corporation Meteor Lake-P [Intel Arc Graphics] [8086:7d55] (rev 08)

$ lscpu  | grep "Model name"
Model name:                              Intel(R) Core(TM) Ultra 7 155H

$ apt list --installed | grep -E 'ocl|intel|libze|dpcpp'

clang-dpcpp-21/questing,now 6.2.0-0ubuntu1~25.10~ppa2 amd64 [installed]
dpcpp-sycl-runtime-libs/questing,now 6.2.0-0ubuntu1~25.10~ppa2 amd64 [installed,automatic]
intel-microcode/questing-updates,questing-security,questing-security,now 3.20250812.0ubuntu0.25.10.1 amd64 [installed,automatic]
intel-opencl-icd-legacy/questing,now 24.35.30872.32-2 amd64 [installed,automatic]
intel-opencl-icd/questing,now 25.31.34666.3-1ubuntu1 amd64 [installed,automatic]
libclang-dpcpp-common-21-dev/questing,now 6.2.0-0ubuntu1~25.10~ppa2 amd64 [installed]
libclang-dpcpp21/questing,now 6.2.0-0ubuntu1~25.10~ppa2 amd64 [installed,automatic]
libdrm-intel1/questing-updates,now 2.4.125-1ubuntu0.1 amd64 [installed,automatic]
libze-intel-gpu-legacy1-1/questing,now 24.35.30872.32-2 amd64 [installed,automatic]
libze-intel-gpu1/questing,now 25.31.34666.3-1ubuntu1 amd64 [installed,automatic]
libze1/questing,now 1.24.1-2 amd64 [installed,automatic]
llvm-dpcpp/questing,now 6.2.0-0ubuntu1~25.10~ppa2 amd64 [installed,automatic]
ocl-icd-libopencl1/questing,now 2.3.3-1 amd64 [installed,automatic]
ocl-icd-opencl-dev/questing,now 2.3.3-1 amd64 [installed]

[frenchwr]

One thing I realized as I was writing this issue is that -DONEDNN_CPU_RUNTIME=SYCL may be the issue. I will try building with that set to TBB or even SEQ to see if that makes a difference.

[vpirogov]

Glad to hear we are finally packaging GPU version of oneDNN! Thank you for the effort.

One thing I realized as I was writing this issue is that -DONEDNN_CPU_RUNTIME=SYCL may be the issue. I will try building with that set to TBB or even SEQ to see if that makes a difference.

You are right, this is what causes tests to fail. SYCL CPU runtime has dependency on Intel OpenCL implementation for CPU which is not available in your case. I would suggest building this configuration with -DONEDNN_CPU_RUNTIME=NONE to limit support to GPU only.

[vpirogov]

Few more notes:

• DNNL_EXPERIMENTAL_SPARSE=ON is not needed. Sparse functionality is enabled in production builds in oneDNN v3.9 and later releases.
• Experimental features in oneDNN do not guarantee API compatibility. Enabling these in packages may cause undesired ABI compatibility issues down the road.
• Examples run on CPU by default. To run these on GPU you need to pass gpu argument. This is handled automatically by ctest.
• gpu_opencl_interop is OpenCL specific example. It does not work in SYCL configurations. There are some other configuration-specific exceptions. CMake handles this automatically, but if you are building these on your own you'll have to exclude these manually.

[frenchwr]

Hi @vpirogov , thanks for the feedback!

I've added -DONEDNN_CPU_RUNTIME=NONE as you suggest and when I pass gpu as an arg to sycl_interop_buffer the example runs successfully on the Meteor Lake machine where I'm testing:

$ clang++-dpcpp -fsycl -ldnnl-sycl -ltbb -lOpenCL sycl_interop_buffer.cpp -o sycl_interop_buffer
$ ./sycl_interop_buffer gpu
Example passed on GPU.

I'm wondering if you might suggest the best examples for validation of a SYCL build (running on Intel-only GPU for now, maybe in the future we'll try to include NVIDIA/AMD)? Our build-time environment will not include Intel GPUs, so most likely I'll hand pick 3-4 examples and run those through some sort of automation to validate the packaging. Perhaps I could even include a subset of the examples in a debian package to make the validation simpler. I'm imaging something like:

$ apt install libdnnl-sycl-examples
$ cd /usr/share/doc/libdnnl-sycl-examples/
$ clang++-dpcpp -fsycl -ldnnl-sycl ... foo.cpp -o foo

DNNL_EXPERIMENTAL_SPARSE=ON is not needed. Sparse functionality is enabled in production builds in oneDNN v3.9 and later releases.

Nice - I have disabled this in the build.

Experimental features in oneDNN do not guarantee API compatibility. Enabling these in packages may cause undesired ABI compatibility issues down the road.

The reason I originally kept DNNL_EXPERIMENTAL_SPARE=ON and -DDNNL_EXPERIMENTAL_UKERNEL=ON was for consistency with the GCC build in Debian and Ubuntu (Ubuntu just syncs the package from Debian), specifically here. Line 18 in that file also mentions "# ukernel is only implemented (and used by PyTorch) on amd64", so it sounds like that UKERNEL experimental feature is important for PyTorch. For that reason I have kept that flag (-DDNNL_EXPERIMENTAL_UKERNEL=ON) in the SYCL build for now.

I have not been in touch with the debian maintainers of the existing onednn package in Ubuntu/Debian, but one other thing I'm wondering about is their naming of the binary packages:

• libdnnl3.6
• libdnnl-dev

Specifically I'm wondering why libdnnl3.6 is used for the binary package name for the .so, since they are actually packaging version 3.9. Does that mean something to you, or is it just an oversight on their part? Generally versions are only embedded in an apt package name if you want to deliver multiple versions simultaneously in a single OS release or if there is some issue related to ABI compatibility and upgrade path.

gpu_opencl_interop is OpenCL specific example. It does not work in SYCL configurations. There are some other configuration-specific exceptions. CMake handles this automatically, but if you are building these on your own you'll have to exclude these manually.

Ah ok - noted. The DPC++ compiler includes an OpenCL adapter but I guess the OpenCL engine in oneDNN refers to direct OpenCL within oneDNN itself.

[frenchwr]

#2514 provides context on the libdnnl3.6 naming in Debian/Ubuntu, it is indeed due to ABI changes in the experimental ukernel feature that were introduced in that version of oneDNN.

From some quick research, it appears that the microkernel feature is CPU-only, in which case I will go ahead and disable it for the SYCL-enabled packaging work.