Repository navigation
[Fix] Honor strides in vector activations - #1369
Merged
yzhangcs merged 2 commits intoOct 11, 2026
Merged
Conversation
4 of 6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix incorrect activation values and gradients for one-dimensional strided views, such as
x[::3]. The layout guard previously accepted every rank-one tensor even though these kernels use unit column offsets; the existing contiguous fallback now also handles strided vector inputs and upstream gradients for sigmoid, logsigmoid, swish, SwiGLU, SigmoidGLU and PowGLU.Contiguous vectors and the higher-dimensional row-stride path are unchanged. This follows the non-contiguous support in #765; it does not extend scalar support or change the separate Ascend implementation.
Test plan
FLA_CI_ENV=0.1e-3tolerance.f3bec720f7847d4f176872b34f002cc580e2cc6e, the identical regression tests give 6 passes and 19 failures (18 numerical failures plus the layout-helper assertion). On this head all 25 pass. Independent public-call probes also reproduce the numerical failures before the fix and pass after it.FLA_CI_ENV=0 pytest tests/modules/test_activation.py -q --tb=short: 103 cases passed across two bounded runs. The first completed 72 cases before a 900-second cold-compilation limit; the next deselected exactly those 72 and passed the remaining 31. This is not a single uninterrupted full-suite pass. No cases failed or skipped; the second run reported three PyTorch/pytest setup advisories, not tolerance warnings.pre-commit run --files fla/modules/activations/ops.py tests/modules/test_activation.py: passed.python scripts/find_dependent_tests.py fla/modules/activations/ops.pyand audited shared callers. Its broad transitive layer/model test list was not run; this change is limited to rank-one layout admission, and the activation suite covers the existing dense/chunked and compiled paths. No full-model or non-NVIDIA validation claim.Benchmark / NCU
N/A for kernel benchmarks: no kernel code or higher-rank dispatch changed. This restores correctness through the existing copy fallback for previously misread vector views, so the incorrect no-copy baseline is not a valid performance comparison for those views. No performance-neutrality, throughput or speedup claim.
Breaking changes
None. Public signatures, dtypes and tolerances are unchanged.
Checklist