Compilation on windows Fails after PR #2053

[MonkeyBreaker]

Hello everyone,

First, thank you everyone for this great tool !

Issue description

Current system:

• Windows 10
• MinGW
• conda 4.8.1 installed on windows
• python version 3.7.4 (the one in conda)
• cmake 3.14.0 installed on windows
• compiler MSVC 14.24.28314

After PR #2053 was merged, our building pipelines on windows started failing. The output message error is fatal error LNK1181: cannot open input file 'python37.lib.
After tracking the problem, I found out that PYTHON_LIBRARY in file tools/FindPythonLibsNew.cmake was not set to the correct path.

• Before PR Fix compilation with MinGW only #2053 PYTHON_LIBRARY = C:/Users/user/Anaconda3/libs/python37.lib
• After PR Fix compilation with MinGW only #2053 PYTHON_LIBRARY = python37.lib

Debugging each step of the previous file resulted that:

• PYTHON_LIBDIR is empty
• After the patch, it tries to look in a Linux location for the library
• PYTHON_LIBRARY is set to python37.lib

I found a solution changing some lines in tools/FindPythonLibsNew.cmake, without changing the behaviour for Linux or Mac.

I would like to know if you'll be interested for a PR, if yes the only thing is I don't have many more windows system to test on it. To I don't know if other Windows user with their build system could test ?

Or maybe do you have any other suggestion that could help us of course :) ?

Reproducible example code

1. windows system
2. install mingw
3. install conda for windows
4. import a C++ project using cmake for building process, and of course pybind

In the CmakeLists.txt file, we use add_subdirectory() to indicate where the folder of pybind11 is located.

Expected error: fatal error LNK1181: cannot open input file 'python37.lib

Let me know if you have any questions/suggestion.

Cheers,
Julián

[Baljak]

I can't reproduce your issue. Furthermore, the intention of the PR you mention, which I authored, was precisely to fix problems like this.

If you setup a bare bones CMakeLists.txt like this:

project(test)

add_subdirectory(pybind11)

and use the following command:

cmake -G "MinGW Makefiles" -DPYTHON_EXECUTABLE=C:\Users\user\AppData\Local\Continuum\anaconda3\python.exe

Everything works fine, CMake is perfectly able to find the required .lib file, see my log here:

-- Found PythonInterp: C:/Users/user/AppData/Local/Continuum/anaconda3/python.exe (found version "3.7.5")
-- Found PythonLibs: C:/Users/user/AppData/Local/Continuum/anaconda3/libs/Python37.lib

Maybe you have set the PYTHON_LIBRARY variable manually? This isn't required on Windows, except if you're using a Unix emulator of some kind like MSYS or Cygwin.

[MonkeyBreaker]

Hi and thank you for your reply,

The setup CMakeLists.txt is as you described, the only difference is project(test LANGUAGES CXX).

Maybe I wasn't clear enough, the installation works perfectly fine when running from a windows shell, in our case it doesn't work when running from our shell (MinGW).

Your second solution makes it work, setting manually PYTHON_LIBRARY, but I don't like this solution because the users of our library that would like to contribute, will need more setup before make it work. I know that setting a variable isn't much, but it's already more than just make nothing and it works ^^.

BTW, is it written somewhere about setting PYTHON_LIBRARYfor a Unix emulator ? I couldn't find myself.

I'll prepare a PR to just discuss my solution, if it doesn't provide anything useful, we'll drop it and indicates to our users the steps to make it work if they develop on Windows. Our objective would be that users should not indicate any paths, and the "finding" scripts could provide everything needed to make it compile. Maybe I dream to much of unicorns, I don't know.

Thank you for your time and the indications your provide us :)

Cheers,
Julián

[Baljak]

Maybe I wasn't clear enough, the installation works perfectly fine when running from a windows shell, in our case it doesn't work when running from our shell (MinGW).

Then, what shell are you using? Is it Cygwin, MSYS, WSL or another one? In this last case, what shell is it? If you provide enough information, we can expand the check I changed recently in FindPythonLibsNew.cmake to include the specific shell your are using.

BTW, is it written somewhere about setting PYTHON_LIBRARY for a Unix emulator ? I couldn't find myself.

I don't think it's written somewhere, I said that because it's the way FindPythonLibsNew.cmake is working: for Unix, it expects a library in a Unix format, i.e. ending with a .a extension. Because of this, on Windows, a Unix emulator won't look for the .lib library since it expects a .a one, so we need to set PYTHON_LIBRARY manually.

[MonkeyBreaker]

Then, what shell are you using? Is it Cygwin, MSYS, WSL or another one? In this last case, what shell is it? If you provide enough information, we can expand the check I changed recently in FindPythonLibsNew.cmake to include the specific shell your are using.

Okay, now I understand why I was giving you wrong information. I use directly the shell which comes when downloading git for windows, which I cannot find if it's related to Cygwin, MSYS. etc. It seems that it uses bash shell. Sorry I was telling you MinGW, because the window name of the shell is MinGW64, my mistake.

I don't think it's written somewhere, I said that because it's the way FindPythonLibsNew.cmake is working: for Unix, it expects a library in a Unix format, i.e. ending with a .a extension. Because of this, on Windows, a Unix emulator won't look for the .lib library since it expects a .a one, so we need to set PYTHON_LIBRARY manually.

Well, in my case, CMAKE_FIND_LIBRARY_SUFFIXES is set automatically to .lib. Then, in FindPythonLibsNew.cmake, find_library can find the good file (python37.lib) if we provide the correct path to look at.

Maybe if it can help you more, MSYSTEM is set to MINGW64 on my setup.

Let me know if you want more information, I have some time for this issue.

[Baljak]

Ok I understand your issue now. One more information, are you using MSVC on Git Bash, or are you using GCC?

[MonkeyBreaker]

I'm using MSVC, compiler version 14.24.28314

[MonkeyBreaker]

In attachment a solution I found which works on my side without providing library path location, with my setup. The file is a patch, but Github does not support *.patch extension.

FindPython.txt

[Baljak]

I made a PR which will hopefully fix your issue, can you check against it?