Skip to content

ggml-cpu : set openmp wait time if not set #13758

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 24, 2025
Merged

Conversation

slaren
Copy link
Member

@slaren slaren commented May 24, 2025

Sets the OpenMP wait policy to active if not set. On some implementations this can significantly increase performance when using OpenMP.

Edit: changed to setting the thread block time instead, otherwise threads will continue spinning indefinitely. This is not a standard OpenMP setting, but it works in clang/windows. In my system setting it to 200ms results in the same performance as active wait policy.

@github-actions github-actions bot added the ggml changes relating to the ggml tensor library for machine learning label May 24, 2025
@slaren slaren changed the title ggml-cpu : set openmp wait policy to active if not set ggml-cpu : set openmp wait time if not set May 24, 2025
@CISC
Copy link
Collaborator

CISC commented May 24, 2025

Strange, according to the LLVM docs 200ms is the default?

@slaren
Copy link
Member Author

slaren commented May 24, 2025

I noticed that too. However I don't think the documentation is accurate. Actually even setting it to just 1us is enough to improve generation performance by 50% on my system.

@slaren
Copy link
Member Author

slaren commented May 24, 2025

On hybrid CPUs the default value is actually zero.
https://github.com/llvm/llvm-project/blob/7ff0cf6138f30db4f361a20eda7d18692337d351/openmp/runtime/src/kmp.h#L1278

@slaren slaren merged commit 2bd1b30 into master May 24, 2025
45 of 46 checks passed
@slaren slaren deleted the sl/openmp-wait-policy branch May 24, 2025 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ggml changes relating to the ggml tensor library for machine learning
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants