Skip to content

FA3 with flash_attn_varlen_func#1

Merged
varunneal merged 50 commits intomasterfrom
backup-3
Sep 4, 2025
Merged

FA3 with flash_attn_varlen_func#1
varunneal merged 50 commits intomasterfrom
backup-3

Conversation

@varunneal
Copy link
Copy Markdown
Owner

See README

varunneal and others added 30 commits August 28, 2025 18:02
This commit replaces the linear increase of window size with a discrete
schedule, simplifying the logic and potentially improving performance.
The new approach uses a predefined tuple of window sizes and selects
the appropriate size based on the current training progress.
@varunneal varunneal merged commit 8386811 into master Sep 4, 2025
varunneal pushed a commit that referenced this pull request Sep 5, 2025
varunneal added a commit that referenced this pull request Sep 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant