`eb` command fails outside of framework source folder

[lexming]

I installed easybuild-framework v5.0.x in development mode pip install -e and the resulting installation fails to execute the eb command with any combination of options (or none) with the following error:

Traceback (most recent call last):
  File "/home/lexming/src/EB/easybuild-framework/easybuild/tools/options.py", line 1492, in parse_options
    eb_go = EasyBuildOptions(usage=usage, description=description, prog='eb', envvar_prefix=CONFIG_ENV_VAR_PREFIX,
  File "/home/lexming/src/EB/easybuild-framework/easybuild/tools/options.py", line 249, in __init__
    super(EasyBuildOptions, self).__init__(*args, **kwargs)
  File "/home/lexming/src/EB/easybuild-framework/easybuild/base/generaloption.py", line 975, in __init__
    self.main_options()
  File "/home/lexming/src/EB/easybuild-framework/easybuild/base/generaloption.py", line 1051, in main_options
    fn()
  File "/home/lexming/src/EB/easybuild-framework/easybuild/tools/options.py", line 604, in config_options
    'choice', 'store', DEFAULT_REPOSITORY, sorted(avail_repositories().keys())),
  File "/home/lexming/src/EB/easybuild-framework/easybuild/tools/repository/repository.py", line 151, in avail_repositories
    raise EasyBuildError("avail_repositories: FileRepository missing from list of repositories")
easybuild.tools.build_log.EasyBuildError: 'avail_repositories: FileRepository missing from list of repositories'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib64/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib64/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/lexming/src/EB/easybuild-framework/easybuild/main.py", line 781, in <module>
    main_with_hooks()
  File "/home/lexming/src/EB/easybuild-framework/easybuild/main.py", line 764, in main_with_hooks
    init_session_state, eb_go, cfg_settings = prepare_main(args=args)
  File "/home/lexming/src/EB/easybuild-framework/easybuild/main.py", line 758, in prepare_main
    eb_go, cfg_settings = set_up_configuration(args=args, logfile=logfile, testing=testing)
  File "/home/lexming/src/EB/easybuild-framework/easybuild/tools/options.py", line 1535, in set_up_configuration
    eb_go = parse_options(args=args)
  File "/home/lexming/src/EB/easybuild-framework/easybuild/tools/options.py", line 1496, in parse_options
    raise EasyBuildError("Failed to parse configuration options: %s" % err)
easybuild.tools.build_log.EasyBuildError: "Failed to parse configuration options: 'avail_repositories: FileRepository missing from list of repositories'"

The eb command does work from the root of the easybuild-framework folder though. This mode of installation used to work prior to v5.0.x

[boegel]

@lexming Can you check if this is new in 5.0.x branch? I think we have the same problem in develop already...

[lexming]

This is indeed not related to 5.0.x but caused by changes in pip. It is either related to the deprecation of eggs, the deprecation of setup.py or changes to how editable installations are carried out, or everything all at once, not sure.

The issue raises from easybuild.tools.utilities.import_available_modules which depends on sys.path to load all available submodules of a given python module. However, editable installations with recent pip do not get their paths injected into sys.path.

This is what happens in Rocky 8.8: python v3.6 and old pip v9.0.3

# Create new venv, activate it and install easybuild-framework
$ python3 -m venv ~/.local/venv/ebtest
$ source ~/.local/venv/ebtest/bin/activate
(ebtest) $ cd ~/src/easybuild-framework/
(ebtest) $ pip install --no-deps -e .
Obtaining file:///rhea/scratch/brussel/101/vsc10122/easybuild/easybuild-framework
Installing collected packages: easybuild-framework
  Running setup.py develop for easybuild-framework
Successfully installed easybuild-framework
You are using pip version 9.0.3, however version 24.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
# Installation is done with `egg-link` files that point to source dir
(ebtest) $ ls ~/.local/venv/ebtest/lib/python3.6/site-packages/
easybuild-framework.egg-link  easy-install.pth  easy_install.py  pip  pip-9.0.3.dist-info  pkg_resources  __pycache__  setuptools  setuptools-39.2.0.dist-info
(ebtest) $ cat ~/.local/venv/ebtest/lib/python3.6/site-packages/easybuild-framework.egg-link
/rhea/scratch/brussel/101/vsc10122/easybuild/easybuild-framework
.
# Paths in `egg-link` files populate `sys.path`
python -c "import sys; print(sys.path)"
(ebtest) $ ['', '/usr/lib64/python36.zip', '/usr/lib64/python3.6', '/usr/lib64/python3.6/lib-dynload', '/user/brussel/101/vsc10122/.local/venv/ebtest/lib64/python3.6/site-packages', '/rhea/scratch/brussel/101/vsc10122/easybuild/easybuild-framework', '/user/brussel/101/vsc10122/.local/venv/ebtest/lib/python3.6/site-packages']

This is what happens in Fedora 38: python v3.10 and pip v23.2.1

# Create new venv, activate it and install easybuild-framework
$ python3.10 -m venv ~/.local/venv/
$ source ~/.local/venv/ebtest/bin/activate
(ebtest) $ cd ~/src/easybuild-framework/
(ebtest) $ pip install --no-deps -e .
Obtaining file:///home/lexming/src/EB/easybuild-framework
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Building wheels for collected packages: easybuild-framework
  Building editable for easybuild-framework (pyproject.toml) ... done
  Created wheel for easybuild-framework: filename=easybuild_framework-5.0.0.dev0-0.editable-py3-none-any.whl size=25377 sha256=ad97550d366b89607fba98da8a7ec6df33786ed3714d93f16f966c4feabcd5fb
  Stored in directory: /tmp/pip-ephem-wheel-cache-akxj_hly/wheels/0c/3e/e2/92f1d587382290451654a30193e17ee7820c1db2d7e25722de
Successfully built easybuild-framework
Installing collected packages: easybuild-framework
Successfully installed easybuild-framework-5.0.0.dev0
# Installation is done with a regular dist-info package and some __editable__ files
(ebtest) $ ls ~/.local/venv/ebtest/lib/python3.10/site-packages/
  __pycache__/        easybuild_framework-5.0.0.dev0.dist-info/    pip-23.2.1.dist-info/    setuptools/                     __editable__.easybuild_framework-5.0.0.dev0.pth         distutils-precedence.pth
  _distutils_hack/    pip/                                         pkg_resources/           setuptools-67.7.2.dist-info/   __editable___easybuild_framework_5_0_0_dev0_finder.py
# The __editable__ files have some code to append a placeholder path to python sys.path
# But it is not working, sys.path does not contain any reference to framework install dir or source dir
python -c "import sys; print(sys.path)"
['', '/usr/lib64/python310.zip', '/usr/lib64/python3.10', '/usr/lib64/python3.10/lib-dynload', '/home/lexming/.local/venv/ebtest/lib64/python3.10/site-packages', '/home/lexming/.local/venv/ebtest/lib/python3.10/site-packages']

[lexming]

RTFM: https://setuptools.pypa.io/en/latest/userguide/development_mode.html

So this stems from changes in setuptools. Old behaviour can be still achieved by installing in compat mode:

pip install -e . --config-settings editable_mode=compat

[boegel]

@lexming Any idea if we should also consider something else than sys.path to make sure it also works with newer setuptools versions?

[Crivella]

I am not sure this is tied to setuptools but pip itself, i tested using setuptools==78.0.2
the error can be reproduced with any version of pip >= 23.1
If i try with pip < 23.1 eg 23.0.1 it works fine

[Crivella]

I guess it is tied to this change (https://discuss.python.org/t/announcement-pip-23-1-release/25844):

Further work has been done on removing the “legacy” behaviour of running setup.py install directly. The standard behaviour of building a wheel locally and installing from that wheel is used instead.

[Crivella]

Also this pypa/pip#12094