Skip to content

Improve multithreaded support and testing #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ngoldbaum opened this issue Apr 9, 2025 · 2 comments
Open

Improve multithreaded support and testing #9

ngoldbaum opened this issue Apr 9, 2025 · 2 comments
Labels
enhancement help wanted zstd For issues related to Zstandard in CPython

Comments

@ngoldbaum
Copy link

Proposal:

The zstandard wrappers should be thread-safe and should not allow any multithreaded use (particularly under the free-threaded build) that is not considered safe by the zstandard C library.

Another thing to look at is using TSAN to verify thread safety in the C extension. I don't know offhand about if you have to do anything special to use TSAN with new tests in the CPython test suite.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

Rogdham/pyzstd#32 (comment)

@emmatyping
Copy link
Owner

To follow up here, I pushed 6d36249 a few days ago with some freethreaded tests. I want to expand this to (de)compress and test sharing a ZstdDict across threads.

@emmatyping
Copy link
Owner

I ran the tests with tsan on and the GIL disabled and they passed. I'm not sure if I need to do anything else to have the tests use tsan specially, but the tests did print out == sanitizers: thread, so it picked up that it was available.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement help wanted zstd For issues related to Zstandard in CPython
Projects
None yet
Development

No branches or pull requests

2 participants