Skip to content

fix: Improve python lib detection #1093

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

BLaZeKiLL
Copy link

On windows the vars in sysconfig return None so as a fallback we try to enumerate possible library candidates in the installation base to see if we can find the library

This change get's rid of the following warning

2025-05-31 23:15:15,755 - scikit_build_core - WARNING - Can't find a Python library, got libdir=None, ldlibrary=None, multiarch=None, masd=None
2025-05-31 23:15:15,755 - scikit_build_core - WARNING - Can't find a Python library, got libdir=None, ldlibrary=None, multiarch=None, masd=None

@BLaZeKiLL BLaZeKiLL changed the title Improve python lib detection fix: Improve python lib detection Jun 1, 2025
@BLaZeKiLL BLaZeKiLL force-pushed the dev/dlal/libpython-detection-improvements branch from c5d54ca to da8aac9 Compare June 1, 2025 06:34
@LecrisUT
Copy link
Collaborator

LecrisUT commented Jun 1, 2025

For the lib part Python_ROOT_DIR and FindPython should handle that fallback logic. I actually am not sure why include hint is passed, but in that case it is directly exposed in the sysconfig.

@BLaZeKiLL
Copy link
Author

For the lib part Python_ROOT_DIR and FindPython should handle that fallback logic. I actually am not sure why include hint is passed, but in that case it is directly exposed in the sysconfig.

Pushed the fix just noticed in the get_library function if abi3=True we don't look for the .lib with minor version

@BLaZeKiLL
Copy link
Author

@LecrisUT let me know if any other changes are required here

@LecrisUT
Copy link
Collaborator

LecrisUT commented Jun 5, 2025

I thought that the original issue was more about what library variant gets detected and what gets reported by python3-conf (forgot what the name of the shell script is). @henryiii can clarify the issue? I believe that if we do such library guess work we are not much different than how FindPython works it out?

One key thing is that FindPython has 2 modes of detection based on if Interpreter is passed or not 1 and if the CMake module is inconsistent depending on this, we should pass the helper to override that.

Footnotes

  1. somewhere around here: https://gitlab.kitware.com/cmake/cmake/-/blob/master/Modules/FindPython/Support.cmake?ref_type=heads#L581

@henryiii
Copy link
Collaborator

henryiii commented Jun 5, 2025

The reason I didn't add this yet was I felt the old logic just tried "everthing possible", and wanted a more principled approach. Hard-coding things like pypy means this might miss graalpy, for example. It seems like "where's the library" shouldn't be that hard of question. But I added printouts for libdir=None, ldlibrary=None, multiarch=None, masd=None, and they are all None on Windows, so this is probably what we'll have to do.

Have you looked at the logic in CMake's FindPython? That's been good, replicating that is likely ideal.

There are two concerns; the one voiced above, with running into a new situtation like a new interpreter implementation, and the the other is cross-compiling. We might end up picking up a completely incorrect library if we try to hunt too hard.

I'll try to carefully review soon, I've been distracted with pybind11 3.0 and cibuildwheel 3.0 release preparations, and I'm teaching a summer course.

@henryiii henryiii force-pushed the dev/dlal/libpython-detection-improvements branch from 0fa0ac4 to 13579e0 Compare June 12, 2025 05:19
On windows the vars in sysconfig return None so as
a fallback we try to enumerate possible library
candidates in the installation base to see if we
can find the library

Signed-off-by: Devashish Lal <[email protected]>
incase of sabi abi3=True is passed to get library
which looks for the .lib without minor version

Signed-off-by: Devashish Lal <[email protected]>
@henryiii henryiii force-pushed the dev/dlal/libpython-detection-improvements branch from 13579e0 to 6c290c2 Compare June 12, 2025 20:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants