-
Notifications
You must be signed in to change notification settings - Fork 772
{lang}[GCCcore/6.4.0] Python v2.7.13 #4962
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
verdurin
merged 1 commit into
easybuilders:develop
from
JackPerdue:20170730111353_new_pr_Python2713
Jul 30, 2017
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
56 changes: 56 additions & 0 deletions
56
easybuild/easyconfigs/p/Python/Python-2.7.13-GCCcore-6.4.0-bare.eb
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| name = 'Python' | ||
| version = '2.7.13' | ||
| versionsuffix = '-bare' | ||
|
|
||
| homepage = 'http://python.org/' | ||
|
|
||
| description = """ | ||
| Python is a programming language that lets you work more quickly and | ||
| integrate your systems more effectively. | ||
|
|
||
| Note: This module is meant to provide a builddependency for other Python | ||
| modules while using EasyBuild's --minimaltoolchain option. Modules | ||
| built with it will require the full Python later | ||
| """ | ||
|
|
||
| toolchain = {'name': 'GCCcore', 'version': '6.4.0'} | ||
| toolchainopts = {'pic': True} | ||
|
|
||
| source_urls = ['http://www.python.org/ftp/%(namelower)s/%(version)s/'] | ||
| sources = [SOURCE_TGZ] | ||
| checksums = ['a4f05a0720ce0fd92626f0278b6b433eee9a6173ddf2bced7957dfb599a5ece1'] | ||
|
|
||
| builddependencies = [ | ||
| ('binutils', '2.28'), | ||
| ] | ||
|
|
||
| dependencies = [ | ||
| ('bzip2', '1.0.6'), | ||
| ('libreadline', '7.0'), | ||
| ('ncurses', '6.0'), | ||
| ('SQLite', '3.19.3'), | ||
| ('zlib', '1.2.11'), | ||
| ] | ||
|
|
||
| osdependencies = [ | ||
| # rely upon distribution for timely security updates | ||
| ('openssl-devel', 'libssl-dev', 'libopenssl-devel'), | ||
| ] | ||
|
|
||
| # We hide this by default since users should not use it in production, | ||
| # high-performance Python should be delivered at compiler level with | ||
| # a default extension set | ||
| # hidden = True | ||
|
|
||
| # bare installation: only known module deps for GCCcore tools included | ||
| exts_list = [] | ||
|
|
||
| # Until such time that EasyBuild accepts 'rpath' as a toolchainopt... | ||
| # add RPATH so this can be used as a builddependency for "bare" PythonPackages | ||
| preconfigopts = """ | ||
| sed -e\ | ||
| 's:$(LINKCC) $(LDFLAGS) $(LINKFORSHARED):$(LINKCC) $(LDFLAGS) $(LINKFORSHARED) -Wl,-rpath=%(installdir)s/lib:'\ | ||
| -i.eb Makefile.pre.in | ||
| """ | ||
|
|
||
| moduleclass = 'lang' | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bleh, not too happy with this part...
At the very list, this should be done properly via an
rpathtoolchain option, and it should be clearly explained why the RPATH is really needed.This is just another sign that using a bare Python as a build dep later to be replaced by a 'full' Python installation higher up the hierarchy is a bad idea imho...
Thoughts @damianam?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To be honest, I am not sure why is this even needed. Why can't this be used without rpathing?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@boegel well, AFAIK, there is no way to tell Travis to use --expirmental. Why should I be FORCED to make three different versions (iomkl/intel/foss) for a module when the GCCcore one will suffice as a builddep that can be used with any of those later?
@damianam because pythonpackage.py doesn't set LD_LIBRARY_PATH that includes builddeps when it does a "python -c 'import 'foo'" check.. So the check fails because python can't find libpythonX.Y.so. The alternative is disable that check in everything that uses Python-bare, which seems much worse than any concern expressed thus far.
Y'all could fix this by updating pythonpackage.py to set the LD_LIBRARY_PATH for the Python-bare that was used as a builddep. Not sure if that's the best solution. I just know this one works).
Like I said, I have some 50+ modules using Python bare. I"ve spent a lot of time working through the issues. Multiply that by 2 versions of Python. Y'all are basically saying I should have to multiply that by 3 (iomkl/intel/foss). So I end up with 300 modules when 100 would have sufficed. That's just a waste of time and space.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In our setup, we have Python in
GCCcoreand a bunch of packages on top. On that sense, it is similar to what you say. Some, likenumpy,scipy, etc are in the appropriate toolchain (top level with math libraries), which needed to rebuild the interpreter to get fast exponential and trigonometric functions. Others likempi4py, were simply built on top of the python inGCCcore. In the latter case, I haven't run into the issues you described. The only difference I see is that I've listed Python independencies, and notbuilddependencies. In cases were python is just used for building, without any particular package, we simply rely on the system python. I guess that explains why that hasn't been a problem for us. In our case the setup you are proposing (python-bare forbuilddependenciesof some packages) is not really necessary.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Certainly you have to step up to a MPI toolchain if you need MPI or a toolchain with math for numpy/scipy. eg we have these
Boost-1.61.0-gompi-2017A-Python-2.7.12-bare.eb
Boost-1.61.0-iimpi-2017A-Python-2.7.12-bare.eb
Boost-1.61.0-iimpi-2017A-Python-3.5.2-bare.eb
Boost-1.61.0-iompi-2017A-Python-2.7.12-bare.eb
Boost-1.61.0-iompi-2017A-Python-3.5.2-bare.eb
because Boost needs MPI for some parts.
How does a system Python (e.g. 2.6.6 on RHEL6) provide you with usuable packages for 2.7.13 and 3.6.2? Somebody tried that trick with X11 and it doesn't work in the long run (causes all kinds of weird problems later on).
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, it doesn't. What I meant is that if python (as in the python interpreter to be used to run a simple python script) is needed for building, we use the system python. Anything more convoluted than that will use our
Python-2.7.13-GCCcore-5.4.0.eborPython-3.6.1-GCCcore-5.4.0.eb, which are listed asdependencies, notbuilddependencies. I haven't found any case where python should be inbuilddependencies, but the system python isn't enough for it. X11 needs python just forxcb-proto, as stated in the easyconfig file:so we simply use that and don't introduce any Python dependency.