Loss of some environment state in Windows build?

[jbarlow83]

For Windows 32-bit, I need to call vcvars32.bat to switch to 32-bit compilation to link against a third party 32-bit library. This seems to work fine for the first attempted build (Python 3.5), but whatever happens before attempting Python 3.6, some environment variables change and the state set by vcvars32.bat seem to be lost. It appears to be trying to compile using mingw64's link.exe instead of msvc.

Any ideas? Perhaps you've seen something like this before and I'm going about it the wrong way entirely.

Here is the full build.

https://dev.azure.com/jim0585/pikepdf/_build/results?buildId=55

  C:\Program Files\Git\usr\bin\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:c:\hostedtoolcache\windows\python\3.6.8\x86\libs /LIBPATH:c:\hostedtoolcache\windows\python\3.6.8\x86\PCbuild\win32 "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\lib\um\x86" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\ucrt\x86" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x86" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\LIB" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\LIB" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\ucrt\x86" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\lib\um\x86" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x86" /LIBPATH:c:\qpdf\lib qpdf.lib /EXPORT:PyInit__qpdf build\temp.win32-3.6\Release\src/qpdf\annotation.obj build\temp.win32-3.6\Release\src/qpdf\object.obj build\temp.win32-3.6\Release\src/qpdf\object_convert.obj build\temp.win32-3.6\Release\src/qpdf\object_repr.obj build\temp.win32-3.6\Release\src/qpdf\pikepdf.obj build\temp.win32-3.6\Release\src/qpdf\qpdf.obj build\temp.win32-3.6\Release\src/qpdf\qpdf_pagelist.obj build\temp.win32-3.6\Release\src/qpdf\shims.obj build\temp.win32-3.6\Release\src/qpdf\utils.obj /OUT:build\lib.win32-3.6\pikepdf\_qpdf.cp36-win32.pyd /IMPLIB:build\temp.win32-3.6\Release\src/qpdf\_qpdf.cp36-win32.lib
  /usr/bin/link: extra operand '/LTCG'
  Try '/usr/bin/link --help' for more information.
  error: command 'C:\\Program Files\\Git\\usr\\bin\\link.exe' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for pikepdf
  Running setup.py clean for pikepdf

[YannickJadoul]

That is really weird, indeed. I can't seem to immediately think of what could be wrong, though.

Have you tried printing the related environment variables? And what if you skip 3.5 and start with 3.6; does 3.6 then succeed?

[jbarlow83]

This skips 3.5 begins with 3.6. 3.6 seems to fail all on its own.

https://dev.azure.com/jim0585/pikepdf/_build/results?buildId=59

[YannickJadoul]

Hmm, OK, so it's not the case that some environment gets lost or so; the environment is just different on 3.6? That's sómething, but not really a solution yet :-(

[YannickJadoul]

Oh, but maybe ... I see you are calling this vcvars32.bat in CIBW_BEFORE_BUILD. I don't think changes to the environment in CIBW_BEFORE_BUILD are kept.

@joerick Any idea how to keep environment variables set from a batch file?

[joerick]

Ah, not using CIBW_BEFORE_BUILD :/ environment variables tend to only flow downwards, not up into the parent process. The only way I could think would be for cibuildwheel to run all its build commands in a pty, but that module only works on Linux apparently.

I'm wondering if this might be a use case for the (currently non-existent) CIBW_BUILD_WHEEL_COMMAND option, that's been talked about once before, where a default value would be-

{pip} wheel {project_dir} -w {output_dir} --no-deps {build_verbosity_flag}

and then you could set that value to

CIBW_BUILD_WHEEL_COMMAND="call vcvars32.bat && {pip} wheel {project_dir} -w {output_dir} --no-deps {build_verbosity_flag}"

... I don't love it, though. The arguments the cibuildwheel passes to pip wheel feel like implementation details of cibuildwheel, it's going to make it inconvenient to change those down the line if they're written into users' options.

Perhaps setuptools can help us here? Or could you decant the variables set in vcvars32.bat into CIBW_ENVIRONMENT? Or could you run cibuildwheel twice, once for 32-bit builds and once for 64-bit? And then you could load the vcvars32.bat before calling cibuildwheel for the 32-bit builds.

[YannickJadoul]

Damn, that's what I expected. This separation of CIBW_BEFORE_BUILD is a bit annoying, but I think it also contributes to the abstraction of cibuildwheel. And I agree and feel the same towards CIBW_BUILD_WHEEL_COMMAND, @joerick.

But back to @jbarlow83's problem. Indeed; I see that you are calling vcvars32.bat before starting cibuildwheel? So normally, there should be no reason to call it again in CIBW_BEFORE_BUILD, since on Windows, cibuildwheel does not require any docker image or so? Is there a reason you also added it to CIBW_BEFORE_BUILD?

[YannickJadoul]

@jbarlow83 I've forked your project and started investigating what happens on Azure. Seems that somehow the mingw executables get added to PATH when you call bash:

cmd_before.txt
cmd_after.txt
bash.txt

EDIT: When I comment out call "%VS140COMNTOOLS%\..\..\VC\bin\vcvars32.bat" in win-vcvars-build.bat, the build does succeed. Could you shortly explain why you need this vcvars32.bat?

[jbarlow83]

@YannickJadoul Thank you for taking time to look, that help me get unstuck.

I found out I could remove vcvars32.bat entirely. I believe that the UsePythonVersion@0 task also selects MSVC at the appropriate architecture. I added vcvars32.bat because it was necessary in my appveyor build and used in qpdf's azure-pipelines based build.

Because I need a 32-bit and 64-bit version of a third party DLL. It wouldn't surprise me other people who get errors with cibuildwheel like need to split Windows into 32 or 64 bit.

fatal error LNK1112: module machine type 'X86' conflicts with target machine type 'x64'

Anyway thanks for your help.

[YannickJadoul]

Geat; thanks for letting me know! :-)