vulkan: add UMA dual-access mode for zero-copy CPU access on iGPUs#1
Closed
essentialols wants to merge 1 commit into
Closed
vulkan: add UMA dual-access mode for zero-copy CPU access on iGPUs#1essentialols wants to merge 1 commit into
essentialols wants to merge 1 commit into
Conversation
On UMA (Unified Memory Architecture) devices like AMD APU iGPUs, CPU and GPU share the same physical memory. This patch enables the CPU to directly access GPU buffer contents without copies by: 1. Reporting Vulkan device buffers as host-accessible via the is_host callback when the device is UMA and dual-access mode is enabled 2. Returning the real host-mapped pointer from get_base instead of the sentinel vk_ptr_base, enabling direct CPU reads/writes 3. Computing tensor offsets from the actual buffer base address rather than the sentinel, so CPU-side pointer arithmetic is correct Gated behind GGML_VK_UMA_DUAL_ACCESS env var. When unset, behavior is identical to before (is_host returns false, get_base returns the sentinel). Only effective on UMA devices (integrated GPUs). This eliminates unnecessary GPU-to-CPU transfer operations on iGPUs where both processors already see the same physical memory.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Changes (+38 lines, 1 file: ggml-vulkan.cpp)
is_hostcallback: reports buffers as host-accessible on UMA when dual-access enabled (was NULL)get_base: returns real host-mapped pointer instead of sentinelvk_ptr_basevk_tensor_offset: computes offsets from actual buffer baseGating
GGML_VK_UMA_DUAL_ACCESSenv var (opt-in). Only on UMA devices. Zero change when unset.Context
PR ggml-org#22930 already allocates buffers with DEVICE_LOCAL+HOST_VISIBLE+HOST_COHERENT on UMA, so the memory IS host-mapped. But get_base returns a sentinel and is_host is NULL, preventing upper layers from using direct access. This patch bridges that gap.