[GPU][WIP] Add L0 runtime support #30789

jkasprza · 2025-05-29T09:56:36Z

Details:

Based on [GPU][WIP] Initial L0 runtime support #26874
Add L0 Immediate Command List
Specify -DGPU_RT_TYPE=L0 for cmake to build with L0 support. Current default is OCL

p-durandin · 2025-06-16T14:13:50Z

build_jenkins

…into gpu_l0

* removed opencl from targets: openvino_intel_gpu_kernels, openvino_intel_gpu_runtime * opencl will still be linked to openvino gpu plugin even with L0 rt

…into gpu_l0

isanghao · 2025-12-10T08:32:09Z

.gitmodules

 	url = https://github.com/herumi/xbyak_riscv.git
+[submodule "src/plugins/intel_gpu/thirdparty/l0_onednn_gpu"]
+	path = src/plugins/intel_gpu/thirdparty/l0_onednn_gpu
+	url = https://github.com/jkasprza/oneDNN.git


for the record: I guess we will have single copy of onednn_gpu. Please update that before it is merged.

Need to wait for uxlfoundation/oneDNN#4499 to be merged first.

isanghao · 2025-12-10T08:36:04Z

src/plugins/intel_gpu/include/intel_gpu/runtime/device_info.hpp

    bool has_separate_cache;                    ///< Does the target hardware has separate cache for usm_device and usm_host

+    bool supports_cp_offload;                   ///< [L0] Does the command queue support copy offload
+    bool supports_cb_events;                    ///< [L0] Does the target runtime support counter based events


nit: cb_events sounds ambiguous because cb usually means callback.

Renamed in 8c9e855

isanghao · 2025-12-10T09:07:47Z

src/plugins/intel_gpu/include/intel_gpu/runtime/event.hpp

        _profiling_info.clear();
    }
+    // Set event profiling data instead of retrieving it from event object
+    void set_profiling(uint64_t duration_nsec) {


What about set_profiling_duration instead? I think the name is difficult to understand.

Renamed in d377c45

isanghao · 2025-12-10T09:08:14Z

src/plugins/intel_gpu/include/intel_gpu/runtime/event.hpp

 #include <utility>
 #include <utility>
 #include <functional>
+#include <optional>


nit: do we need this header?

Removed in d377c45

isanghao · 2025-12-12T07:16:09Z

src/plugins/intel_gpu/src/runtime/stream.cpp


-QueueTypes stream::detect_queue_type(engine_types engine_type, void* queue_handle) {
-    switch (engine_type) {
-        case engine_types::sycl:


I think you need create_surfaces_lock for sycl_stream as well?

Here sycl_stream derives create_surfaces_lock from ocl_stream. It will create ocl::ocl_surfaces_lock for sycl_stream the same way It was implemented before changes.

isanghao · 2025-12-16T09:19:57Z

src/plugins/intel_gpu/src/runtime/ze/ze_event.cpp

+}
+
+void ze_event::wait_impl() {
+    OV_ZE_EXPECT(zeEventHostSynchronize(m_event, default_timeout));


what about endless_wait instead of default_timeout to make it explicit?

Renamed in 5ccd52c

isanghao · 2025-12-16T09:21:40Z

src/plugins/intel_gpu/src/runtime/ze/ze_event.hpp

+            // Ensure event handle is not null
+            if (ev == nullptr) {
+                OPENVINO_THROW("[GPU] Trying to create event with null handle");
+            }


nit: what about OPENVINO_ASSERT?

Done in 8621d68

isanghao · 2025-12-16T09:23:02Z

src/plugins/intel_gpu/src/runtime/ze/ze_event.hpp

+    void set_impl() override;
+    bool is_set_impl() override;
+    // TODO: Implement add_event_handler_impl
+    // bool add_event_handler_impl(event_handler, void*) override;


not sure, but is it required?

isanghao · 2025-12-16T09:27:33Z

src/plugins/intel_gpu/src/runtime/ze/ze_event_factory.cpp

+        ze_event_pool_flags_t flags = is_profiling_enabled() ? ZE_EVENT_POOL_FLAG_KERNEL_TIMESTAMP : 0;
+        flags |= ZE_EVENT_POOL_FLAG_HOST_VISIBLE;
+        m_current_pool = std::make_shared<ze_event_pool>(m_engine, m_capacity, flags);
+    }


this assume single thread execution, right? isn't it necessary to have some synchronization here?

isanghao · 2025-12-16T09:31:29Z

src/plugins/intel_gpu/src/runtime/ze/ze_event_factory.hpp

+struct ze_event_factory : public ze_base_event_factory {
+public:
+    ze_event_factory(const ze_engine &engine, bool enable_profiling, uint32_t capacity = 255);
+    event::ptr create_event(uint64_t queue_stamp) override;


what does queue_stamp mean here? is it relevant to queue? does it mean unique id of event? What about event_stamp then?

Here queue_stamp is analogous to already existing queue_stamp from ocl_base_event. It is just a value provided by the stream when creating events and is used for ordering the events for example in ocl_stream::sync_events. It should be unique if the events were created by the same stream. I do not see variable named event_stamp anywhere in the code.

isanghao · 2025-12-17T06:56:01Z

src/plugins/intel_gpu/src/runtime/ze/ze_events.hpp

+
+    ze_event_handle_t m_last_event = nullptr;
+    std::vector<event::ptr> m_events;
+    const ze_engine &m_engine;


nit: please use same convention for naming as ocl_event.

isanghao · 2025-12-26T11:00:25Z

src/plugins/intel_gpu/src/runtime/ze/ze_device.cpp

+
+    auto mem_properties = std::find_if(device_memory_properties.begin(), device_memory_properties.end(), [](const ze_device_memory_properties_t& p) {
+        auto name = std::string(p.name);
+        return name == "DDR" || name == "HBM";


out of curiosity, what other name does it have other than DDR and HBM? e.g. does it have property like "SRAM"?

Am I correct that intel GPUs have only 1 type of memory?

Here I am not sure. So far, I only encountered single type that is either DDR or HBM

isanghao · 2025-12-26T11:06:03Z

src/plugins/intel_gpu/src/runtime/ze/ze_memory.cpp

+        }
+        _mapped_ptr = nullptr;
+    }
+}


These code is very similar to the one in ocl_memory.cpp. Could you refactor it to reduce duplication?

isanghao · 2025-12-26T11:06:13Z

src/plugins/intel_gpu/src/runtime/ze/ze_memory.cpp

+        ev_ze,
+        ze_dep_events.size(),
+        ze_dep_events.data()));
+    // FIXME: when not blocking pattern goes out of scope


what does this mean?

Pointer to local variable pattern is passed to the zeCommandListAppendMemoryFill. To avoid use after free, memory fill must finish execution before this variable gets destroyed.

isanghao · 2025-12-26T11:19:09Z

I finished 1st round of review. Could you check my comment? I see code duplication in many places. Could you review whether it can be reduced?

…into gpu_l0

jkasprza added the category: GPU OpenVINO GPU plugin label May 29, 2025

jkasprza requested a review from sshlyapn May 29, 2025 09:56

Add L0 support for gpu

38f889e

jkasprza force-pushed the gpu_l0 branch from 4104a1a to 38f889e Compare June 30, 2025 12:59

jkasprza added 11 commits July 10, 2025 16:05

Fix L0 DPAS check

ce58599

Merge remote-tracking branch 'upstream/master' into gpu_l0

d8283b1

Use copy offload only when supported

f38ae58

Remove dlopen for L0

8aaec53

Merge branch 'master' of https://github.com/openvinotoolkit/openvino …

51e581b

…into gpu_l0

Add dep_events for l0 mem fill

f8eb99a

Add OneDNN with L0 support

44d0a79

Adjust to L0 OneDNN

c3b1a67

Disable OneDNN pooling

7f79b07

Merge remote-tracking branch 'upstream/master' into gpu_l0

2bbe030

Update L0 OneDNN submodule

0e70375

jkasprza marked this pull request as ready for review August 13, 2025 09:19

jkasprza requested review from a team as code owners August 13, 2025 09:19

jkasprza added 3 commits August 13, 2025 16:54

Adjust to new OneDNN

35e492d

Fix include

6b586d3

Add new OneDNN for L0

afe606b

jkasprza added the do_not_merge label Nov 6, 2025

jkasprza added 10 commits November 7, 2025 09:40

Fix style

9ba3c9c

Remove OpenCL dependency when running L0

d0f157c

* removed opencl from targets: openvino_intel_gpu_kernels, openvino_intel_gpu_runtime * opencl will still be linked to openvino gpu plugin even with L0 rt

Fix OCL program build

4ec431e

Add gfx_ver parsing for L0

3e6b368

Skip L0 symbol table kernel

cfdbb02

Avoid cl_mem usage with L0 engine

04b086a

Fix windows build

6c6664d

Remove unnecessary vector during L0 mem fill

a3eac10

Merge branch 'master' of https://github.com/openvinotoolkit/openvino …

e6480ac

…into gpu_l0

Fix memory

c76cdb2

github-actions bot removed the category: Core OpenVINO Core (aka ngraph) label Nov 28, 2025

jkasprza added 3 commits November 28, 2025 14:08

Set runtime interface for kernel_selector

6473aa0

Remove onednn ocl include

93bb04b

Fix backend compilation and remote tensor

c621131

isanghao reviewed Dec 16, 2025

View reviewed changes

isanghao reviewed Dec 26, 2025

View reviewed changes

jkasprza added 12 commits January 7, 2026 12:38

Merge branch 'master' of https://github.com/openvinotoolkit/openvino …

fadca80

…into gpu_l0

Rename counter based events

8c9e855

Adjust event set_profiling_duration

d377c45

Remove unnecessary engine parameter

ca7f6cf

Update L0 OneDNN submodule

674556c

Update onednn L0 submodule and fix profiling

863ea10

Restore cpu checks in primitive_base

0cdb6f5

Reuse check_allocatable for ocl and l0

86c81b4

Use asserts instead of throw

8621d68

Fix get_user_context

8d0b5bf

Move common code to engine class

d64e6b6

Rename L0 timeout

5ccd52c

[GPU][WIP] Add L0 runtime support #30789

Are you sure you want to change the base?

[GPU][WIP] Add L0 runtime support #30789

Conversation

jkasprza commented May 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Details:

Uh oh!

p-durandin commented Jun 16, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jkasprza Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jkasprza Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

isanghao commented Dec 26, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jkasprza commented May 29, 2025 •

edited

Loading

jkasprza Jan 7, 2026 •

edited

Loading

jkasprza Jan 7, 2026 •

edited

Loading