Skip to content

Crashes in containers on Cgroupsv2 based hosts #10103

Closed
@DominicLavery

Description

@DominicLavery

Bug description

In containers running on cgroupv2 systems _query_cpu currently returns None. This results in sched_getaffinity being used, which will normally return all installed CPUs of the host. This can result in crashes with the error:

.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

Configuration

Command used

pylint

Pylint output

Traceback (most recent call last):
  File "/home/circleci/.local/bin/pylint", line 8, in <module>
    sys.exit(run_pylint())
  File "/home/circleci/.local/lib/python3.10/site-packages/pylint/__init__.py", line 34, in run_pylint
    PylintRun(argv or sys.argv[1:])
  File "/home/circleci/.local/lib/python3.10/site-packages/pylint/lint/run.py", line 215, in __init__
    linter.check(args)
  File "/home/circleci/.local/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 679, in check
    check_parallel(
  File "/home/circleci/.local/lib/python3.10/site-packages/pylint/lint/parallel.py", line 153, in check_parallel
    for (
  File "/home/circleci/.pyenv/versions/3.10.15/lib/python3.10/concurrent/futures/process.py", line 575, in _chain_from_iterable_of_lists
    for element in iterable:
  File "/home/circleci/.pyenv/versions/3.10.15/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator
    yield _result_or_cancel(fs.pop())
  File "/home/circleci/.pyenv/versions/3.10.15/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel
    return fut.result(timeout)
  File "/home/circleci/.pyenv/versions/3.10.15/lib/python3.10/concurrent/futures/_base.py", line 458, in result
    return self.__get_result()
  File "/home/circleci/.pyenv/versions/3.10.15/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

Expected behavior

The run of pylint to succeed rather than crash. _query_cpu should return the restricted number of cpus rather than None so that the number of jobs is calculated correctly for the container environment

Pylint version

pylint 3.3

OS / Environment

No response

Additional dependencies

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions