Proposal for mapping resource attributes to DXIL and SPIR-V #76

bogner · 2024-10-08T18:21:39Z

No description provided.

s-perron · 2024-10-08T20:13:45Z

proposals/NNNN-resource-attributes-in-dxil-and-spirv.md

+|                                 | Dim    | Depth       | Arrayed | MS    | Sampled | Access |
+|---------------------------------|--------|-------------|---------|-------|---------|--------|
+| Texture1D                       | 1D     | Unknown (2) | false   | false | r/o (1) | r/o    |
+| Texture1DArray                  | 1D     | Unknown (2) | true    | false | r/o (1) | r/o    |
+| Texture2D                       | 2D     | Unknown (2) | false   | false | r/o (1) | r/o    |
+| Texture2DArray                  | 2D     | Unknown (2) | true    | false | r/o (1) | r/o    |
+| Texture2DMS                     | 2D     | Unknown (2) | false   | true  | r/o (1) | r/o    |
+| Texture2DMSArray                | 2D     | Unknown (2) | true    | true  | r/o (1) | r/o    |
+| Texture3D                       | 3D     | Unknown (2) | false   | false | r/o (1) | r/o    |
+| TextureCUBE                     | Cube   | Unknown (2) | false   | false | r/o (1) | r/o    |
+| TextureCUBEArray                | Cube   | Unknown (2) | true    | false | r/o (1) | r/o    |
+| RWTexture1D                     | 1D     | Unknown (2) | false   | false | r/w (2) | r/w    |
+| RWTexture1DArray                | 1D     | Unknown (2) | true    | false | r/w (2) | r/w    |
+| RWTexture2D                     | 2D     | Unknown (2) | false   | false | r/w (2) | r/w    |
+| RWTexture2DArray                | 2D     | Unknown (2) | true    | false | r/w (2) | r/w    |
+| RWTexture2DMS                   | 2D     | Unknown (2) | false   | true  | r/w (2) | r/w    |
+| RWTexture2DMSArray              | 2D     | Unknown (2) | true    | true  | r/w (2) | r/w    |
+| RWTexture3D                     | 3D     | Unknown (2) | false   | false | r/w (2) | r/w    |
+| Buffer                          | Buffer | Unknown (2) | false   | false | r/o (1) | r/o    |
+| RWBuffer                        | Buffer | Unknown (2) | false   | false | r/w (2) | r/w    |


The Access Qualifier is for Kernel (think OpenCL) only. The value after "Sampled" is the Image Format.

Updated the table to add an Image format column but largely leave it blank. See my replies to your other comments

s-perron · 2024-10-08T20:16:33Z

proposals/NNNN-resource-attributes-in-dxil-and-spirv.md

+DXC lowers [Constant/Texture/Structured/Byte Buffers] to [OpTypeStruct] in
+SPIR-V with a uniform storage class and various layout decorations. DXC tracks


The storage class should depend on the version of Vulkan that is targeted. Starting with Vulkan 1.2 the storage class is StorageBuffer.

Updated to be somewhat handwavey about this but mention the version dependency

s-perron

We need to add a section discussing the "Image format" for spir-v. This has been discussed a lot in DXC over the years. The Image format indicates the format of the underlying data. I don't know what the expectations are for DXIL, but many Vulkan users want flexibility. The most flexibility is to use Unknown as much as possible bey default. However for Vulkan we cannot always use Unknown.

There are two feature bits that we care about: shaderStorageImageReadWithoutFormat and shaderStorageImageWriteWithoutFormat. I'm trying to check how available these feature are. However, many devices are still running Vulkan 1.1, and probably do not support it.

DXC guesses at the image format based on the sampled type, and developers can use the vk::image_format attribute to set it to something else. We should document what we will be doing.

Vulkan documentation:

https://docs.vulkan.org/spec/latest/appendices/spirvenv.html#spirvenv-format-type-matching
https://docs.vulkan.org/spec/latest/appendices/spirvenv.html#spirvenv-image-formats

DXC Issues:

microsoft/DirectXShaderCompiler#4941
microsoft/DirectXShaderCompiler#4773
microsoft/DirectXShaderCompiler#2498
microsoft/DirectXShaderCompiler#3395
microsoft/DirectXShaderCompiler#4868

proposals/NNNN-resource-attributes-in-dxil-and-spirv.md

s-perron · 2024-10-09T14:42:37Z

proposals/NNNN-resource-attributes-in-dxil-and-spirv.md

+| Texture1D                          | SRV  | vec4   | -   | 1D   | -   | -        | -     | -   | -   |
+| Texture1DArray                     | SRV  | vec4   | -   | 1D   | -   | -        | yes   | -   | -   |
+| Texture2D                          | SRV  | vec4   | -   | 2D   | -   | -        | -     | -   | -   |
+| Texture2DArray                     | SRV  | vec4   | -   | 2D   | -   | -        | yes   | -   | -   |
+| Texture2DMS                        | SRV  | vec4   | -   | 2D   | yes | -        | -     | -   | -   |
+| Texture2DMSArray                   | SRV  | vec4   | -   | 2D   | yes | -        | yes   | -   | -   |
+| Texture3D                          | SRV  | vec4   | -   | 3D   | -   | -        | -     | -   | -   |
+| TextureCUBE                        | SRV  | vec4   | -   | Cube | -   | -        | -     | -   | -   |
+| TextureCUBEArray                   | SRV  | vec4   | -   | Cube | -   | -        | yes   | -   | -   |
+| RWTexture1D                        | UAV  | vec4   | -   | 1D   | -   | -        | -     | -   | -   |
+| RWTexture1DArray                   | UAV  | vec4   | -   | 1D   | -   | -        | yes   | -   | -   |
+| RWTexture2D                        | UAV  | vec4   | -   | 2D   | -   | -        | -     | -   | -   |
+| RWTexture2DArray                   | UAV  | vec4   | -   | 2D   | -   | -        | yes   | -   | -   |
+| RWTexture2DMS                      | UAV  | vec4   | -   | 2D   | yes | -        | -     | -   | -   |
+| RWTexture2DMSArray                 | UAV  | vec4   | -   | 2D   | yes | -        | yes   | -   | -   |
+| RWTexture3D                        | UAV  | vec4   | -   | 3D   | -   | -        | -     | -   | -   |


These look good. The individual attributes map nicely to individual operands to spirv.Image.

s-perron · 2024-10-09T15:29:57Z

proposals/NNNN-resource-attributes-in-dxil-and-spirv.md

+need to keep track of that information in new SPIR-V target extension types in
+our implementation.


I agree we will need a new target type.

I've been thinking about this a bit. SPIR-V does not have a "handle" for structured buffers. There is a global variable, which is accessed using the same instructions used to access function scope variables.

In Vulkan SPIR-V, a structured buffer StructuredBuffer<T> is represented as struct { T[] }. We could make the handle a pointer to the struct or a pointer to the array. In any case, you want it to be a pointer type.

One option is to have CreateHandleFromBinding return a pointer. Then use use standard llvm-ir instruction to access it. The problem with that is that we will have to expose the layout of T, and the type information may be hard to recover. That is why I agree we need a target type.

We should be able to define a target type to represent a spir-v pointer.

s-perron · 2024-10-09T15:37:32Z

proposals/NNNN-resource-attributes-in-dxil-and-spirv.md

+| ByteAddressBuffer                  | SRV  | -      | -   | -    | -   | -        | -     | yes | -   |
+| RWByteAddressBuffer                | UAV  | -      | -   | -    | -   | -        | -     | yes | -   |
+| RasterizerOrderedByteAddressBuffer | UAV  | -      | yes | -    | -   | -        | -     | yes | -   |
+| StructuredBuffer                   | SRV  | struct | -   | -    | -   | -        | -     | yes | -   |
+| RWStructuredBuffer                 | UAV  | struct | -   | -    | -   | -        | -     | yes | -   |
+| RasterizerOrderedStructuredBuffer  | UAV  | struct | yes | -    | -   | -        | -     | yes | -   |
+| AppendStructuredBuffer             | UAV  | struct | -   | -    | -   | -        | -     | yes | -   |
+| ConsumeStructuredBuffer            | UAV  | struct | -   | -    | -   | -        | -     | yes | -   |


To generate or use these types, the SPIR-V backend will have to know the layout of any structs. In DXC, we have a few different layout options. I feel like it should be the FE's job to specify the layout. Let me know what you think.

s-perron · 2024-10-09T15:44:30Z

proposals/NNNN-resource-attributes-in-dxil-and-spirv.md

+| ByteAddressBuffer                  | SRV  | -      | -   | -    | -   | -        | -     | yes | -   |
+| RWByteAddressBuffer                | UAV  | -      | -   | -    | -   | -        | -     | yes | -   |


ByteAddressBuffers are hard in SPIR-V. Without untyped pointers, we cannot cast from one pointer type to another. This make it very hard to do some thing, and impossible to do others. For, example, it is impossible to have both 32-bit atomic operations and 64-bit atomic operations.

For now, we cannot have a common implementation for all of the raw buffer resources.

damyanp · 2024-10-10T16:20:30Z

proposals/NNNN-resource-attributes-in-dxil-and-spirv.md

+  View), CBV (Constant buffer view), or Sampler. The only difference between
+  SRV and UAV is whether the object is writeable.


The only difference between SRV and UAV is whether the object is writeable.

The resource class also has implications on the register type that the resource can be bound to. I think I'd expect to see some reference to that (maybe just a link to the other documentation we have about it).

I almost wonder if it would be fair to say that the only reason we track this resource class is to know which register types it belongs to. It's notably absent from any discussion about the SPIR-V implementation, where SPIR-V doesn't distinguish between different types of register.

@bogner - thoughts on this?

I reworded this to mention resource class but not really go into detail, WDYT?

- The Resource class is SRV (Shader resource view), UAV (Unordered Access View), CBV (Constant buffer view), or Sampler. This is largely only used for determining which register class a resource belongs to, and is mostly valuable in differentiating between (writeable) UAVs and (read-only) SRVs.

Instead of storing an auxilliary structure with the information from the DXIL resource target extension types duplicated, access the information that we can via the type itself. This also means we need to handle some of the target extension types we haven't fully defined yet, like Texture and CBuffer. For now we make an educated guess to what those should look like based on llvm/wg-hlsl#76, and we can update them fairly easily when we've defined them more thoroughly. First part of llvm#118400

Instead of storing an auxilliary structure with the information from the DXIL resource target extension types duplicated, access the information that we can via the type itself. This also means we need to handle some of the target extension types we haven't fully defined yet, like Texture and CBuffer. For now we make an educated guess to what those should look like based on llvm/wg-hlsl#76, and we can update them fairly easily when we've defined them more thoroughly. First part of #118400

bogner · 2025-02-04T08:53:29Z

We need to add a section discussing the "Image format" for spir-v. This has been discussed a lot in DXC over the years. The Image format indicates the format of the underlying data. I don't know what the expectations are for DXIL, but many Vulkan users want flexibility. The most flexibility is to use Unknown as much as possible bey default. However for Vulkan we cannot always use Unknown.

I added some of these links and basically portrayed all of this as a work-in-progress. I think we need to have a discussion about where we want the language to go in this regard (it's fairly clear that vk::image_format, while it does the job, is probably not sufficiently forward looking)

damyanp

LGTM - don't forget to assign a proper proposal number to this before merging.

s-perron

LGTM. I'll follow up with more SPIR-V specific details in #98.

* [0002] RootSignatures - Versioning proposal (llvm#115) This PR is adding the versioning information to Root Signature specs. It includes: - adding a new attribute to clang driver to specify root signature versions. - updating metadata representation to hold root signature versions. Closes: llvm#113 * [0002] RootSignatures - Add default values to proposal (llvm#173) - adds default value information to the Root Signature specification Resolves llvm#172 * Proposal for mapping resource attributes to DXIL and SPIR-V (llvm#76) * Constant buffers design document (llvm#94) Design document describing how HLSL constant buffers will be handled in Clang. Contains the design for parsing, codegen a lowering of `cbuffer` declarations, including the default constant buffer `$Globals`. Design for the `ConstantBuffer<T>` resource class is still work in progress. * Update proposal number on constant buffers proposal (llvm#188) * [NNNN] Target Extension Types for Inline SPIR-V and Decorated Types (llvm#105) This adds target extension types to represent `SpirvType` from [HLSL 0011, Inline SPIR-V](https://github.com/microsoft/hlsl-specs/blob/main/proposals/0011-inline-spirv.md). * [0018] SPIRV resource representation (llvm#98) Adds a proposal for how HLSL resources will be represented in llvm-ir when targeting SPIR-V. * [SPIR-V] Add proposal for I/O builtin in Clang (llvm#97) Initial proposal to implement Input/Output built ins with both semantic & inline SPIR-V * [SPIR-V] Add proposal for global & local variable address spaces (llvm#111) This commit adds a proposal on how to implement local and global variables in the SPIR-V backend given HLSL put them in the same address space while SPIR-V requires them to be in 2 distinct ones. --------- Signed-off-by: Nathan Gauër <[email protected]> Co-authored-by: Steven Perron <[email protected]> * [0002] RootSignatures - Add missing Descriptor Ranges `offset` (llvm#191) - it appears the `offset` was mistakenly dropped as discarding this information no longer allows the user to provide a manual offset from the start of the descriptors heap and allow for register aliasing - additionally, we do a small fix-up to update the flag values to be their correct default as we adjust the parameter values Resolves llvm#190 --------- Co-authored-by: Finn Plummer <[email protected]> * [0002] RootSignatures - Misc updates to spec from testing (llvm#195) This issue addresses the discrepancies found from testing [here](llvm/llvm-project#130826). The pr has been broken up to commits that address the bullet points in the original issue. Briefly: - First 2 commits work to have a concrete EBNF description of the grammar that denotes that all parameter args can be specified in any order - It is also addressed that `NUMBER` can be more restrictive with respect to what the dxc/clang compiler will accept - Missing documentation for `unbounded` is added - Add additional validation of some parameters that surface as errors in dxc - Remove the specification of 'AllowLowTierReservedHwCbLimit' Resolves llvm#192 --------- Co-authored-by: Finn Plummer <[email protected]> * [0021] Proposal for handling member function address spaces. (llvm#187) As we expose new address spaces in HLSL, we will have a problem with the address space for the `this` pointer in member functions. * Resource Instance Analysis (llvm#207) Fixes llvm#179 Proposes a generic architecture for organizing, resolving, and validating Resource Instance metadata --------- Co-authored-by: Damyan Pepper <[email protected]> Co-authored-by: Helena Kotas <[email protected]> * [0023] Proposal for separate counter resource handle (llvm#208) This is a proposal to create a separte handle for the counter resource in RWStructureBuffer and other that have associated counters. * [0002] RootSignatures - NFC Formatting Grammar to EBNF (llvm#198) Update the formatting of the root signature grammar to use EBNF in a consistent manner: - The first commit just goes through to use `=` instead of `:` and denote statements with ';' to comply with EBNF - The second commit comes from a suggestion [here](llvm#195 (comment)) to just use the `|` notation to specify the parameters in any order but not explicitly denote that they can only occur once Clean up pr to resolve llvm#192 --------- Co-authored-by: Finn Plummer <[email protected]> * Implicit resource binding - document current DXC behavior (llvm#193) Initial commit of the Implicit Resource Binding proposal that includes: - brief overview section on resource bindings - number of examples documenting how are implicit resource bindings are assigned in DXC Closes llvm#176 * Implicit resource bindings design (llvm#196) Proposed design for implicit resource bindings in Clang. Closes llvm#177 * Resource constructors proposal (llvm#197) Proposal for resource classes constructors. Closes llvm#209 * Remove tools/hlslpm (llvm#228) As the process evolves, it becomes more effort to keep tools/hlslpm up to date. This change removes tools/hlslpm rather than keep a stale version around. * Update issue_tracking.md for latest thinking (llvm#229) Describe epics, scenarios, deliverables and tasks, and the custom fields used in the HLSL Support project. --------- Co-authored-by: Finn Plummer <[email protected]> * Add "Review" value for the "blocked" field (llvm#257) Some issues are blocked on design review. This value can be used to identify them. * Resource binding: add another example and notes about resources in structs (llvm#214) Add one more example showing binding of resource arrays in user-defined structs. Add a note to re-evaluate whether to bind resource arrays in structs individually or as a continuous descriptor range later on when we start working on the resources in structs design (llvm/wg-hlsl#llvm#212). * [0026] Add proposal for symbol visibility. (llvm#272) Section 3.6 of the [HLSL specification](https://microsoft.github.io/hlsl-specs/specs/hlsl.pdf) defined the possible linkages for names. This proposal updates how these linkages are represented in LLVM IR. The current implementation presents challenges for the SPIR-V backend due to inconsistencies with OpenCL. In HLSL, a name can have external linkage and program linkage, among others. If a name has external linkage, it is visible outside the translation unit, but not outside a linked program. A name with program linkage is visible outside a partially linked program. We propose that names with program linkage in HLSL should have external linkage and default visibility in LLVM IR, while names with external linkage in HLSL should have external linkage and hidden visibility in LLVM IR. They both have external linkage because they are visible outside the translation unit. Default visibility means the name is visible outside a shared library (program). Hidden visibility means the name is not visible outside the shared library (program). * Update proposals for implicit binding and resource constructors to match implementation (llvm#282) * Add missing titles to proposals (llvm#283) Adding title to proposals that were missing one. * Unify new line endings to LF (llvm#286) Add .gitattributes and set new line endings to LF. Normalize existing files by running `git add --renormalize`. * [NNNN] Implementation for `vk::constant_id` (llvm#287) Design for the Clang implmentation of`vk::constant_id`. --------- Signed-off-by: Nathan Gauër <[email protected]> Co-authored-by: Finn Plummer <[email protected]> Co-authored-by: Justin Bogner <[email protected]> Co-authored-by: Helena Kotas <[email protected]> Co-authored-by: Cassandra Beckley <[email protected]> Co-authored-by: Steven Perron <[email protected]> Co-authored-by: Nathan Gauër <[email protected]> Co-authored-by: Finn Plummer <[email protected]> Co-authored-by: Ashley Coleman <[email protected]> Co-authored-by: Damyan Pepper <[email protected]> Co-authored-by: Finn Plummer <[email protected]>

bogner force-pushed the 2024-10-08-resource-attributes branch from 57e9383 to c412b60 Compare October 8, 2024 18:28

Proposal for mapping resource attributes to DXIL and SPIR-V

d0b6a34

bogner force-pushed the 2024-10-08-resource-attributes branch from c412b60 to d0b6a34 Compare October 8, 2024 18:54

s-perron reviewed Oct 8, 2024

View reviewed changes

s-perron reviewed Oct 9, 2024

View reviewed changes

proposals/NNNN-resource-attributes-in-dxil-and-spirv.md Outdated Show resolved Hide resolved

s-perron reviewed Oct 9, 2024

View reviewed changes

damyanp reviewed Oct 10, 2024

View reviewed changes

s-perron mentioned this pull request Oct 17, 2024

[202x] Propose adding vk::SampledTexture* types microsoft/hlsl-specs#343

Merged

hekota mentioned this pull request Nov 21, 2024

[HLSL] Add ByteAddressBuffer, RWByteAddressBuffer and RasterizerOrderedByteAddressBuffer definitions to HLSLExternalSemaSource #113477 llvm/llvm-project#116699

Merged

bogner mentioned this pull request Dec 12, 2024

[DirectX] Get resource information via TargetExtType llvm/llvm-project#119772

Merged

bogner linked an issue Jan 28, 2025 that may be closed by this pull request

Document the plan for how resource attributes map to DXIL and SPIR-V #68

Closed

bogner added 2 commits February 4, 2025 00:44

Editorial changes based on review feedback

df37890

Add examples of the DirectX target extension types

0a44839

damyanp assigned tex3d Feb 10, 2025

damyanp self-requested a review February 10, 2025 22:16

damyanp assigned damyanp and pow2clk and unassigned tex3d Feb 10, 2025

damyanp approved these changes Feb 11, 2025

View reviewed changes

damyanp removed their assignment Feb 11, 2025

s-perron approved these changes Feb 12, 2025

View reviewed changes

Number proposal and move to accepted

b5040e2

bogner merged commit 06455bf into llvm:main Feb 12, 2025

bogner deleted the 2024-10-08-resource-attributes branch February 12, 2025 21:56

damyanp moved this from Needs Review to Closed in HLSL Support Apr 22, 2025

damyanp removed this from HLSL Support Jun 25, 2025

		DXC lowers [Constant/Texture/Structured/Byte Buffers] to [OpTypeStruct] in
		SPIR-V with a uniform storage class and various layout decorations. DXC tracks

		need to keep track of that information in new SPIR-V target extension types in
		our implementation.

		\| ByteAddressBuffer \| SRV \| - \| - \| - \| - \| - \| - \| yes \| - \|
		\| RWByteAddressBuffer \| UAV \| - \| - \| - \| - \| - \| - \| yes \| - \|

		View), CBV (Constant buffer view), or Sampler. The only difference between
		SRV and UAV is whether the object is writeable.

Proposal for mapping resource attributes to DXIL and SPIR-V #76

Proposal for mapping resource attributes to DXIL and SPIR-V #76

Uh oh!

Conversation

bogner commented Oct 8, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

s-perron left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bogner commented Feb 4, 2025

Uh oh!

damyanp left a comment

Choose a reason for hiding this comment

Uh oh!

s-perron left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

s-perron left a comment •

edited

Loading