Skip to content

Slightly expand the usage of VNNI256#1764

Open
XZiar wants to merge 1 commit into
ikawrakow:mainfrom
XZiar:more-vnni256
Open

Slightly expand the usage of VNNI256#1764
XZiar wants to merge 1 commit into
ikawrakow:mainfrom
XZiar:more-vnni256

Conversation

@XZiar

@XZiar XZiar commented May 9, 2026

Copy link
Copy Markdown
Contributor

I am applying the changes on common patterns like _mm256_madd_epi16(_mm256_maddubs_epi16(a, b), ones) and _mm256_add_epi32(c, _mm256_madd_epi16(a, b)).

The original logics inside and outside HAVE_FANCY_SIMD are the same, so these changes should be safe (unless there's overflow handling outside of those regions...).

Actually there's already ultility functions like DotHelper::dot, using them could be neater...

@ikawrakow

Copy link
Copy Markdown
Owner

Can you provide evidence that

  1. The change does not break anything on an AVX2 system with VNNI
  2. The change actually improves performance

As this change affects quite a few quantization types, I would appreciate if you tested all affected types.

Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants