gh-90108: Disable LTO on _freeze_module and _testembed

[vstinner]

LTO optimization is nice to make Python faster, but _freeze_module and _testembed performance is not important. Using LTO to build these two programs make a whole Python build way slower, especially combined with a sanitizer (like ASAN).

• Issue: Reintroduce bootstrap_python for freezing #90108

[vstinner]

cc @erlend-aasland @gpshead @corona10

[vstinner]

vstinner added the skip news label

I don't think that this change is important to document in the Changelog, since it has no impact on users, it only makes the build a little bit faster when LTO is used.

[vstinner]

I noticed this issue when working on PR #109576.

[corona10]

Did you check that build is passed with the clang(thinLTO)?

[corona10]

PTAL: #96761 (comment)

I am not sure that it will happen again, but you need check.

[vstinner]

Before, I only tested my PR with GCC.

It seems like in practice, this change has no effect on clang with Python is built with ./configure --with-lto=<any value>. clang always use LTO in this case.

Did you check that build is passed with the clang(thinLTO)?

Sure.

git clean -fdx
./configure --with-lto=thin CC=clang LD=clang 
make

It just works, but there is a trick: CONFIGURE_LDFLAGS_NOLTO=-flto=thin in Makefile, because of this line in configure.ac:

      dnl Clang linker requires -flto in order to link objects with LTO information.
      dnl Thin LTO is faster and works for object files with full LTO information, too.
      AX_CHECK_COMPILE_FLAG([-flto=thin],[LDFLAGS_NOLTO="-flto=thin"],[LDFLAGS_NOLTO="-flto"])

Code added by you in commit 83d84e6 :-)

In short, "NOLTO" does not disable LTO but enable LTO instead :-)

---

I also tried:

• ./configure --with-lto=full CC=clang LD=clang.
• ./configure --with-lto CC=clang LD=clang: alias to --with-lto=thin in practice.

Both commands produce CONFIGURE_LDFLAGS_NOLTO=-flto=thin as well in the Makefile.

Python build works (with my PR) in both cases.

---

If I modify manually Makefile to use CONFIGURE_LDFLAGS_NOLTO=-fno-lto, Python build fails with my change.

clang   -fno-lto  -o Programs/_freeze_module Programs/_freeze_module.o  (...)

Programs/_freeze_module.o: file not recognized: file format not recognized

But... Python also fails on the main branch (without this PR) if manually edit Makefile to use CONFIGURE_LDFLAGS_NOLTO=-fno-lto. Build fails somewhere else:

clang   -fno-lto  -o _bootstrap_python Modules/getbuildinfo.o (...)

Modules/getbuildinfo.o: file not recognized: file format not recognized

Well:

$ file Modules/getbuildinfo.o
Modules/getbuildinfo.o: LLVM IR bitcode

[vstinner]

I discovered "NOLTO" variables when I worked on PR #109576: "Don't export -fsanitize compiler options".

[corona10]

LGTM

[vstinner]

Thanks for reviews @corona10 and @gpshead.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Debian 3.x has failed when building commit 3e3a7da.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/49/builds/6610) and take a look at the build logs.
4. Check if the failure is related to this commit (3e3a7da) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/49/builds/6610

Failed tests:

• test_tools

Failed subtests:

• test_freeze_simple_script - test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/test/test_tools/test_freeze.py", line 28, in test_freeze_simple_script
    outdir, scriptfile, python = helper.prepare(script, outdir)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Tools/freeze/test/freeze.py", line 146, in prepare
    copy_source_tree(srcdir, SRCDIR)
  File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Tools/freeze/test/freeze.py", line 95, in copy_source_tree
    shutil.copytree(oldroot, newroot, ignore=ignore_non_src)
  File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/shutil.py", line 588, in copytree
    return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/shutil.py", line 542, in _copytree
    raise Error(errors)
shutil.Error: [('/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/build/test_python_2763169æ/test_python_t7c7j57j.sock', '/tmp/test_python_a4o1bjnm/tmphlyck6y3/cpython/build/test_python_2763169æ/test_python_t7c7j57j.sock', "[Errno 6] No such device or address: '/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/build/test_python_2763169æ/test_python_t7c7j57j.sock'")]

[erlend-aasland]

Nice, thanks Victor!