bpo-41818: test_openpty succeed on Gentoo, don't expect to fail on this platform

[asvetlov]

Fix buildbot failures introduced after #22962 merging.

https://bugs.python.org/issue41818

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @asvetlov for commit 4ba515d 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

[vstinner]

See https://bugs.python.org/issue28468

[asvetlov]

@vstinner @tiran I happy to use freedesktop_os_release() when it lands but want to merge this PR ASAP to fix the broken buildbot.

[vstinner]

@vstinner @tiran I happy to use freedesktop_os_release() when it lands but want to merge this PR ASAP to fix the broken buildbot.

Sure.

[vstinner]

I don't understand the rationale for skipping the test only on Gentoo. Would you mind to write a comment to explaining the rationale? Why not skipping the test on Ubuntu? Is it a temporary workaround until someone can investigate the root issue? Please add a reference to bpo-41818 in the command.

[asvetlov]

No, it is the reverse: Gentoo passes the test, all other tested Linux distributions fail.
That's why expectedFailure() is applied for each platform where stdin is a TTY except Gentoo.

I'll add a comment.

[miss-islington]

@asvetlov: Status check is done, and it's a success ✅ .

[miss-islington]

@asvetlov: Status check is done, and it's a success ✅ .

[miss-islington]

Sorry, I can't merge this PR. Reason: 2 of 4 required status checks are expected..

[miss-islington]

Sorry, I can't merge this PR. Reason: 2 of 4 required status checks are expected..

[8vasu]

I just noticed this. I have a 1-2 hour(s) long meeting with my advisor that is scheduled to start in an hour. After that, I will install Gentoo in a VM to inspect what exactly is happening. Thank you for being patient.

[asvetlov]

@8vasu take your time.
The bots are green now, the investigation can be as long as needed.
Without this quick-fix I should apply our policy and revert #22962 entirely.

[8vasu]

@asvetlov Thank you for being so kind. Also, thank you for preparing this quick-fix. My advisor has postponed the meeting because of thanksgiving preparation. I will look into this matter now.

[8vasu]

@ethanfurman @asvetlov I created a Gentoo VM which is currently running emerge; it looks like it might take a lot more time for that to finish. I will let it run. I created a second Gentoo VM, and performed the tests using the live cd.

There are a few things to note. I realized that by default, the live cd does not include a C compiler; their "stage tarballs" include a C compiler. Since I already have one VM currently performing a full installation of Gentoo, I decided to avoid performing those steps again. As an initial phase of the investigation, I ran the tests using the python3.7 that was already available on the live cd.

To be able to run the tests using Python 3.7, I had to comment out the 2 import_module lines. Both of them ran as I had expected them to run: test_openpty() resulted in an OK (expected failures=1); test_master_read() passed. I do not exactly know how the bot was able to make test_openpty() pass.

I will try compiling Python 3.10 after Gentoo is fully installed in the first VM.

Edit: I ran the tests without applying this quick-fix.

[zware]

FTR, I have serious doubts that this is a Gentoo-specific issue. Note that this bot is run using buildbot's usePTY option so that curses tests will run; see

https://github.com/python/buildmaster-config/blob/db57e24feb7c41c07a84b502838e7b849f251661/master/master.cfg#L143-L148

and

https://github.com/python/buildmaster-config/blob/db57e24feb7c41c07a84b502838e7b849f251661/master/custom/factories.py#L125-L132

[zware]

In fact, running this test on both of my Gentoo machines (including the one running the buildbot worker in question) in an interactive SSH session, there is a failure with this change:

0:00:00 load avg: 9.92 [1/1] test_pty
test test_pty failed -- Traceback (most recent call last):
  File "/python/cpython/master/Lib/test/test_pty.py", line 201, in test_openpty
    self.assertEqual(_get_term_winsz(slave_fd), new_stdin_winsz,
AssertionError: b'\x00\x00\x00\x00\x00\x00\x00\x00' != b'\r\x006\x00\x00\x00\x00\x00' : openpty() failed to set slave window size

There is no such failure without this change, or when run in a non-interactive SSH session.

[8vasu]

@zware I also have doubts about that. Thank you for the help. I can now reinforce your doubts. I just finished installing Gentoo; it took half a day for the process to complete.

Some details:

1. as suggested in the devguide, I compiled Python 3.10 WITHOUT including this quick-fix using ./configure --with-pydebug && make -j
2. I did not install X or Wayland; the commands were invoked on the terminal emulator provided by the kernel (/dev/tty1); $TERM was linux

Everything went as expected when I ran ./python -m test -j3: there was one expected failure, since stdin was a tty. Then, for further inspection,

1. I ran ./python Lib/test/test_pty.py PtyTest.test_openpty to obtain OK (expected failures=1); stdin was a tty in this case
2. I ran ./python Lib/test/test_pty.py PtyTest.test_openpty < /dev/null to obtain OK; stdin was not a tty in this case

Both the above results were as expected. Therefore, I am still not sure how the buildbot was able to make the test pass earlier.

[8vasu]

Update: #23526