Skip to content

Conversation

@Flamefire
Copy link
Contributor

@Flamefire Flamefire commented Jun 21, 2021

(created using eb --new-pr)

virtualenv seems to bundle a custom site.py which in versions prior to 20.something does not have site.getsitepackages(), see e.g. pypa/virtualenv#737

The error in that case is AttributeError: module 'site' has no attribute 'getsitepackages'

The workaround is a bit hacky but seems to work across various versions of Python/virtualenv.

Some problems considered/solved:

  • base_prefix is supposed to contain the python prefix, but does not in early virtualenv versions, it instead contains the virtualenv prefix
  • real_prefix was used instead but later removed
  • Especially Python2 adds more than 1 folder to the getsitepackages
  • Not all such entries end in site-packages

Hence the very conservative approach of trying getting the first of all possible prefixes and moving all paths starting with that after the EBPYTHONPREFIXES

I also added a sanity check to check the base case, i.e. site-customize does work at all and adds the new site-packages path before the core site-packages path. (conservative check, we might want to limit to moving paths only ending with site-packages)

Testing that this works with virtualenv is hard, as we'd actually need to install a virtualenv. Not sure if we really want to do that.

The script I used to verify the new site-customize.py: checkPythonModule.sh

@Flamefire Flamefire force-pushed the 20210621094832_new_pr_kEvSgkVNYw branch from 99543f0 to 4d53333 Compare June 21, 2021 07:50
@boegel boegel changed the title Add support for older virtualenv versions enhance sitecustomize.py script installed by Python easyblock to add support for older virtualenv versions Jun 21, 2021
Copy link
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Flamefire Can you edit the PR description to mention the error messages that pop up unless this is included?

@Flamefire Flamefire marked this pull request as draft June 21, 2021 10:05
@Flamefire Flamefire changed the title enhance sitecustomize.py script installed by Python easyblock to add support for older virtualenv versions Enhance sitecustomize.py script installed by Python easyblock to fix support for virtualenv Jun 21, 2021
@Flamefire Flamefire marked this pull request as ready for review June 21, 2021 12:08
@boegel boegel added the bug fix label Jun 21, 2021
@boegel boegel added this to the next release (4.4.1) milestone Jun 21, 2021
@boegel
Copy link
Member

boegel commented Jun 30, 2021

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS Python-2.7.15-foss-2018b.eb
  • SUCCESS Python-3.6.4-foss-2018a.eb
  • SUCCESS Python-3.8.6-GCCcore-10.2.0.eb
  • SUCCESS Python-3.9.5-GCCcore-10.3.0.eb

Build succeeded for 4 out of 4 (4 easyconfigs in total)
node3102.skitty.os - Linux centos linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz, Python 3.6.8
See https://gist.github.com/a96c0da98ec645218d49263c1390ee2d for a full test report.

Copy link
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@boegel boegel merged commit ab11933 into easybuilders:develop Jun 30, 2021
@Flamefire Flamefire deleted the 20210621094832_new_pr_kEvSgkVNYw branch July 1, 2021 08:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants