Refactor tests for Flash Attention Prefill #446
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
joeatodd
approved these changes
Jun 27, 2025
| @@ -187,6 +252,27 @@ struct TestbedImpl { | |||
| // Methods | |||
| // | |||
|
|
|||
| template <typename SrcT, typename DstT> | |||
| void convert_fp8_to_fp16(const SrcT* d_src, DstT* d_dst, size_t size) { | |||
There was a problem hiding this comment.
Assuming this PR lands after #351, we could use a single definition for convert_fp8_to_fp16 which I suggested to move to sycl_common.hpp
There was a problem hiding this comment.
I'll rename it as convert_dtype & move it to sycl_common.hpp. Thanks!
Comment on lines
+266
to
+267
| using outType = cute::conditional_t<is_fp8_v<Tin>, half_t, Tin>; | ||
| if constexpr(is_fp8_v<Tin>) { |
There was a problem hiding this comment.
Suggested change
| using outType = cute::conditional_t<is_fp8_v<Tin>, half_t, Tin>; | |
| if constexpr(is_fp8_v<Tin>) { | |
| if constexpr(is_fp8_v<Tin>) { | |
| using outType = half_t; |
* Add comment on final output type conversion
muhammad-tanvir-1211
force-pushed
the
flash_prefill_separate_out_tests
branch
from
be16975 to
ab0e187
Compare
aacostadiaz
added a commit
that referenced
this pull request
Jun 28, 2025
This PR separates the output type and accumulator type for Flash Attention Prefill. Combinations supported are: * bf16 inputs, fp32 accumulator, bf16 | fp32 output * fp16 inputs, fp32 accumulator, fp16 | fp32 output * fp8 inputs, fp32 accumulator, fp8 | fp32 output Tests added in: #446 Benchmarks added in: #447 --------- Co-authored-by: Alejandro Acosta <[email protected]>
…tlass-fork into flash_prefill_separate_out_tests
aacostadiaz
merged commit 725aab4
into
codeplaysoftware:sycl-develop
28 of 54 checks passed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds tests for all the different data types supported with Flash Attention Prefill. It is a continuation of PR #443