gh-108303: Move config parser data to `Lib/test/configparserdata/` #111879

sobolevn · 2023-11-09T12:56:13Z

There are two options:

a data dir (because it is also used by idlelib)
test_configparser package with these files and test_configparser.py (it is more complex to add, since requires several build-step changes, while in my opinion does not add much)

What do others think?

Issue: Move test files into test subdirectories #108303

miss-islington-app · 2023-11-09T13:52:36Z

Thanks @sobolevn for the PR, and @ambv for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖

…a/` (pythongh-111879) (cherry picked from commit cc18b88) Co-authored-by: Nikita Sobolev <[email protected]>

bedevere-app · 2023-11-09T13:52:50Z

GH-111882 is a backport of this pull request to the 3.12 branch.

bedevere-app · 2023-11-09T13:52:55Z

GH-111883 is a backport of this pull request to the 3.11 branch.

…ta/` (gh-111879) (gh-111882) gh-108303: Move config parser data to `Lib/test/configparserdata/` (gh-111879) (cherry picked from commit cc18b88) Co-authored-by: Nikita Sobolev <[email protected]>

…ta/` (gh-111879) (gh-111883) gh-108303: Move config parser data to `Lib/test/configparserdata/` (gh-111879) (cherry picked from commit cc18b88) Co-authored-by: Nikita Sobolev <[email protected]>

bedevere-bot · 2023-11-09T15:28:57Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Fedora Stable Clang Installed 3.12 has failed when building commit e11ea5a.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1154/builds/358) and take a look at the build logs.
Check if the failure is related to this commit (e11ea5a) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1154/builds/358

Failed tests:

test_configparser
test_idle

Failed subtests:

test_load_cfg_files - idlelib.idle_test.test_config.IdleConfTest.test_load_cfg_files
test_parse_errors - test.test_configparser.SortedTestCase.test_parse_errors
test_parse_errors - test.test_configparser.StrictTestCase.test_parse_errors
test_parse_errors - test.test_configparser.ConfigParserTestCaseNoInterpolation.test_parse_errors
test_unicode_failure - test.test_configparser.ConfigParserTestCaseTrickyFile.test_unicode_failure
test_parse_errors - test.test_configparser.MultilineValuesTestCase.test_parse_errors
test_read_returns_file_list_with_bytestring_path - test.test_configparser.ConfigParserTestCaseNoValue.test_read_returns_file_list_with_bytestring_path
test_read_returns_file_list - test.test_configparser.ConfigParserTestCaseNonStandardDefaultSection.test_read_returns_file_list
test_parse_errors - test.test_configparser.RawConfigParserTestCase.test_parse_errors
test_read_returns_file_list - test.test_configparser.ConvertersTestCase.test_read_returns_file_list
test_parse_errors - test.test_configparser.ConfigParserTestCaseNonStandardDefaultSection.test_parse_errors
test_read_returns_file_list - test.test_configparser.StrictTestCase.test_read_returns_file_list
test_read_returns_file_list - test.test_configparser.CopyTestCase.test_read_returns_file_list
test_read_returns_file_list - test.test_configparser.ConfigParserTestCaseNoInterpolation.test_read_returns_file_list
test_read_returns_file_list_with_bytestring_path - test.test_configparser.StrictTestCase.test_read_returns_file_list_with_bytestring_path
test_parse_errors - test.test_configparser.ConvertersTestCase.test_parse_errors
test_cfgparser_dot_3 - test.test_configparser.ConfigParserTestCaseTrickyFile.test_cfgparser_dot_3
test_read_returns_file_list_with_bytestring_path - test.test_configparser.ConfigParserTestCase.test_read_returns_file_list_with_bytestring_path
test_file - test.test_configparser.ReadFileTestCase.test_file
test_read_returns_file_list - test.test_configparser.RawConfigParserTestCase.test_read_returns_file_list
test_parse_errors - test.test_configparser.RawConfigParserTestCaseNonStandardDelimiters.test_parse_errors
test_parse_errors - test.test_configparser.ConfigParserTestCaseLegacyInterpolation.test_parse_errors
test_read_returns_file_list_with_bytestring_path - test.test_configparser.ConfigParserTestCaseExtendedInterpolation.test_read_returns_file_list_with_bytestring_path
test_read_returns_file_list - test.test_configparser.ConfigParserTestCaseExtendedInterpolation.test_read_returns_file_list
test_read_returns_file_list_with_bytestring_path - test.test_configparser.CopyTestCase.test_read_returns_file_list_with_bytestring_path
test_read_returns_file_list_with_bytestring_path - test.test_configparser.MultilineValuesTestCase.test_read_returns_file_list_with_bytestring_path
test_read_returns_file_list - test.test_configparser.MultilineValuesTestCase.test_read_returns_file_list
test_read_returns_file_list - test.test_configparser.SortedTestCase.test_read_returns_file_list
test_load_file - idlelib.idle_test.test_config.IdleConfParserTest.test_load_file
test_parse_errors - test.test_configparser.ConfigParserTestCaseExtendedInterpolation.test_parse_errors
test_parse_errors - test.test_configparser.ConfigParserTestCase.test_parse_errors
test_read_returns_file_list_with_bytestring_path - test.test_configparser.ConfigParserTestCaseLegacyInterpolation.test_read_returns_file_list_with_bytestring_path
test_read_returns_file_list_with_bytestring_path - test.test_configparser.RawConfigParserTestCase.test_read_returns_file_list_with_bytestring_path
test_read_returns_file_list_with_bytestring_path - test.test_configparser.SortedTestCase.test_read_returns_file_list_with_bytestring_path
test_read_returns_file_list - test.test_configparser.ConfigParserTestCase.test_read_returns_file_list
test_read_returns_file_list_with_bytestring_path - test.test_configparser.ConfigParserTestCaseNonStandardDefaultSection.test_read_returns_file_list_with_bytestring_path
test_parse_errors - test.test_configparser.CopyTestCase.test_parse_errors
test_read_returns_file_list_with_bytestring_path - test.test_configparser.ConfigParserTestCaseNoInterpolation.test_read_returns_file_list_with_bytestring_path
test_read_returns_file_list - test.test_configparser.ConfigParserTestCaseNoValue.test_read_returns_file_list
test_read_returns_file_list_with_bytestring_path - test.test_configparser.ConvertersTestCase.test_read_returns_file_list_with_bytestring_path
test_parse_errors - test.test_configparser.ConfigParserTestCaseNonStandardDelimiters.test_parse_errors
test_read_returns_file_list - test.test_configparser.ConfigParserTestCaseLegacyInterpolation.test_read_returns_file_list
test_reading - test.test_configparser.RawConfigParserTestSambaConf.test_reading

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.12.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.12/test/test_configparser.py", line 1540, in test_readline_generator
    parser.read_file(FakeFile())
                     ^^^^^^^^^^
  File "/home/buildbot/buildarea/3.12.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.12/test/test_configparser.py", line 1493, in __init__
    with open(file_path, encoding="utf-8") as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'configdata/cfgparser.1'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.12.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.12/test/test_configparser.py", line 1361, in test_cfgparser_dot_3
    self.assertEqual(len(cf.read(tricky, encoding='utf-8')), 1)
AssertionError: 0 != 1


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.12.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.12/idlelib/idle_test/test_config.py", line 93, in test_load_file
    self.assertEqual(parser.Get('Foo Bar', 'foo'), 'newbar')
AssertionError: None != 'newbar'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.12.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.12/test/test_configparser.py", line 1168, in test_reading
    self.assertEqual(parsed_files, [smbconf])
AssertionError: Lists differ: [] != ['configdata/cfgparser.2']


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.12.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.12/test/test_configparser.py", line 1520, in test_file
    with open(file_path, encoding="utf-8") as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'configdata/cfgparser.1'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.12.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.12/test/test_configparser.py", line 758, in test_read_returns_file_list_with_bytestring_path
    self.assertEqual(parsed_files, [file1_bytestring])
AssertionError: Lists differ: [] != [b'configdata/cfgparser.1']


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.12.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.12/test/test_configparser.py", line 1393, in test_unicode_failure
    with self.assertRaises(UnicodeDecodeError):
AssertionError: UnicodeDecodeError not raised


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.12.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.12/idlelib/idle_test/test_config.py", line 308, in test_load_cfg_files
    eq(conf.defaultCfg['foo'].Get('Foo Bar', 'foo'), 'newbar')
AssertionError: None != 'newbar'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.12.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.12/test/test_configparser.py", line 555, in test_parse_errors
    with open(tricky, encoding='utf-8') as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'configdata/cfgparser.3'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.12.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.12/test/test_configparser.py", line 725, in test_read_returns_file_list
    self.assertEqual(parsed_files, [file1])
AssertionError: Lists differ: [] != ['configdata/cfgparser.1']

sobolevn · 2023-11-09T15:38:58Z

Failures look related

ambv · 2023-11-09T15:39:59Z

They are indeed related because looks like the data files don't get installed.

sobolevn · 2023-11-09T15:44:49Z

Fixing

vstinner · 2023-11-09T16:23:46Z

They are indeed related because looks like the data files don't get installed.

Ooooh, right. It's good that we have buildbots to test that :-)

…a/` (pythongh-111879)

jaraco · 2024-02-24T15:49:35Z

They are indeed related because looks like the data files don't get installed.

Ooooh, right. It's good that we have buildbots to test that :-)

Better would be to fail faster (#81224).

This change is also implicated In jaraco/configparser#70.

It's probably good to learn that the backport was inadvertently depending on fixtures in the Python test suite.

Additionally, it's an indication that this change has implications for the backport's test suite and it would be good for me to get notified when there are such changes. I think what that means is that I just need to add my username to the codeowners for the relevant paths.

…a/` (pythongh-111879)

pythongh-108303: Move config parser data to Lib/test/configparserdata/

45b6eca

sobolevn requested a review from vstinner November 9, 2023 12:56

sobolevn requested a review from terryjreedy as a code owner November 9, 2023 12:56

bedevere-app bot added the awaiting review label Nov 9, 2023

bedevere-app bot mentioned this pull request Nov 9, 2023

Move test files into test subdirectories #108303

Closed

sobolevn added skip news needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Nov 9, 2023

ambv merged commit cc18b88 into python:main Nov 9, 2023

bedevere-app bot removed the awaiting review label Nov 9, 2023

bedevere-app bot removed the needs backport to 3.12 only security fixes label Nov 9, 2023

bedevere-app bot removed the needs backport to 3.11 only security fixes label Nov 9, 2023

sobolevn added a commit to sobolevn/cpython that referenced this pull request Nov 9, 2023

pythongh-111879: Install Lib/test/configdata

263451d

sobolevn mentioned this pull request Nov 9, 2023

gh-108303: Install Lib/test/configdata #111899

Merged

aisk pushed a commit to aisk/cpython that referenced this pull request Feb 11, 2024

pythongh-108303: Move config parser data to `Lib/test/configparserdat…

41d5e74

…a/` (pythongh-111879)

jaraco mentioned this pull request Feb 24, 2024

Tests failing jaraco/configparser#70

Closed

Glyphack pushed a commit to Glyphack/cpython that referenced this pull request Sep 2, 2024

pythongh-108303: Move config parser data to `Lib/test/configparserdat…

8e206a4

…a/` (pythongh-111879)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-108303: Move config parser data to `Lib/test/configparserdata/` #111879

gh-108303: Move config parser data to `Lib/test/configparserdata/` #111879

sobolevn commented Nov 9, 2023 •

edited by bedevere-app bot

Loading

miss-islington-app bot commented Nov 9, 2023

bedevere-app bot commented Nov 9, 2023

bedevere-app bot commented Nov 9, 2023

bedevere-bot commented Nov 9, 2023

sobolevn commented Nov 9, 2023

ambv commented Nov 9, 2023

sobolevn commented Nov 9, 2023

vstinner commented Nov 9, 2023

jaraco commented Feb 24, 2024

gh-108303: Move config parser data to Lib/test/configparserdata/ #111879

gh-108303: Move config parser data to Lib/test/configparserdata/ #111879

Conversation

sobolevn commented Nov 9, 2023 • edited by bedevere-app bot Loading

miss-islington-app bot commented Nov 9, 2023

bedevere-app bot commented Nov 9, 2023

bedevere-app bot commented Nov 9, 2023

bedevere-bot commented Nov 9, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

sobolevn commented Nov 9, 2023

ambv commented Nov 9, 2023

sobolevn commented Nov 9, 2023

vstinner commented Nov 9, 2023

jaraco commented Feb 24, 2024

gh-108303: Move config parser data to `Lib/test/configparserdata/` #111879

gh-108303: Move config parser data to `Lib/test/configparserdata/` #111879

sobolevn commented Nov 9, 2023 •

edited by bedevere-app bot

Loading