Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions easybuild/easyconfigs/p/Python/Python-2.7.13-GCCcore-6.4.0-bare.eb
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
name = 'Python'
version = '2.7.13'
versionsuffix = '-bare'

homepage = 'http://python.org/'

description = """
Python is a programming language that lets you work more quickly and
integrate your systems more effectively.

Note: This module is meant to provide a builddependency for other Python
modules while using EasyBuild's --minimaltoolchain option. Modules
built with it will require the full Python later
"""

toolchain = {'name': 'GCCcore', 'version': '6.4.0'}
toolchainopts = {'pic': True}

source_urls = ['http://www.python.org/ftp/%(namelower)s/%(version)s/']
sources = [SOURCE_TGZ]
checksums = ['a4f05a0720ce0fd92626f0278b6b433eee9a6173ddf2bced7957dfb599a5ece1']

builddependencies = [
('binutils', '2.28'),
]

dependencies = [
('bzip2', '1.0.6'),
('libreadline', '7.0'),
('ncurses', '6.0'),
('SQLite', '3.19.3'),
('zlib', '1.2.11'),
]

osdependencies = [
# rely upon distribution for timely security updates
('openssl-devel', 'libssl-dev', 'libopenssl-devel'),
]

# We hide this by default since users should not use it in production,
# high-performance Python should be delivered at compiler level with
# a default extension set
# hidden = True

# bare installation: only known module deps for GCCcore tools included
exts_list = []

# Until such time that EasyBuild accepts 'rpath' as a toolchainopt...
# add RPATH so this can be used as a builddependency for "bare" PythonPackages
preconfigopts = """
sed -e\
's:$(LINKCC) $(LDFLAGS) $(LINKFORSHARED):$(LINKCC) $(LDFLAGS) $(LINKFORSHARED) -Wl,-rpath=%(installdir)s/lib:'\
-i.eb Makefile.pre.in
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bleh, not too happy with this part...

At the very list, this should be done properly via an rpath toolchain option, and it should be clearly explained why the RPATH is really needed.

This is just another sign that using a bare Python as a build dep later to be replaced by a 'full' Python installation higher up the hierarchy is a bad idea imho...

Thoughts @damianam?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest, I am not sure why is this even needed. Why can't this be used without rpathing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@boegel well, AFAIK, there is no way to tell Travis to use --expirmental. Why should I be FORCED to make three different versions (iomkl/intel/foss) for a module when the GCCcore one will suffice as a builddep that can be used with any of those later?

@damianam because pythonpackage.py doesn't set LD_LIBRARY_PATH that includes builddeps when it does a "python -c 'import 'foo'" check.. So the check fails because python can't find libpythonX.Y.so. The alternative is disable that check in everything that uses Python-bare, which seems much worse than any concern expressed thus far.

Y'all could fix this by updating pythonpackage.py to set the LD_LIBRARY_PATH for the Python-bare that was used as a builddep. Not sure if that's the best solution. I just know this one works).

Like I said, I have some 50+ modules using Python bare. I"ve spent a lot of time working through the issues. Multiply that by 2 versions of Python. Y'all are basically saying I should have to multiply that by 3 (iomkl/intel/foss). So I end up with 300 modules when 100 would have sufficed. That's just a waste of time and space.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In our setup, we have Python in GCCcore and a bunch of packages on top. On that sense, it is similar to what you say. Some, like numpy, scipy, etc are in the appropriate toolchain (top level with math libraries), which needed to rebuild the interpreter to get fast exponential and trigonometric functions. Others like mpi4py, were simply built on top of the python in GCCcore. In the latter case, I haven't run into the issues you described. The only difference I see is that I've listed Python in dependencies, and not builddependencies. In cases were python is just used for building, without any particular package, we simply rely on the system python. I guess that explains why that hasn't been a problem for us. In our case the setup you are proposing (python-bare for builddependencies of some packages) is not really necessary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Certainly you have to step up to a MPI toolchain if you need MPI or a toolchain with math for numpy/scipy. eg we have these

Boost-1.61.0-gompi-2017A-Python-2.7.12-bare.eb
Boost-1.61.0-iimpi-2017A-Python-2.7.12-bare.eb
Boost-1.61.0-iimpi-2017A-Python-3.5.2-bare.eb
Boost-1.61.0-iompi-2017A-Python-2.7.12-bare.eb
Boost-1.61.0-iompi-2017A-Python-3.5.2-bare.eb

because Boost needs MPI for some parts.

How does a system Python (e.g. 2.6.6 on RHEL6) provide you with usuable packages for 2.7.13 and 3.6.2? Somebody tried that trick with X11 and it doesn't work in the long run (causes all kinds of weird problems later on).
.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does a system Python (e.g. 2.6.6 on RHEL6) provide you with usuable packages for 2.7.13 and 3.6.2? Somebody tried that trick with X11 and it doesn't work in the long run (causes all kinds of weird problems later on).

Well, it doesn't. What I meant is that if python (as in the python interpreter to be used to run a simple python script) is needed for building, we use the system python. Anything more convoluted than that will use our Python-2.7.13-GCCcore-5.4.0.eb or Python-3.6.1-GCCcore-5.4.0.eb, which are listed as dependencies, not builddependencies. I haven't found any case where python should be in builddependencies, but the system python isn't enough for it. X11 needs python just for xcb-proto, as stated in the easyconfig file:

# Python is required for xcb-proto
allow_system_deps = [('Python', SYS_PYTHON_VERSION)]

so we simply use that and don't introduce any Python dependency.

"""

moduleclass = 'lang'