look into HashDist's jailtool

[boegel]

We should look whether the jailtool that HashDist provides can help us to avoid missing dependencies when building (new) software packages.

https://github.com/hashdist/hdist-jail

This should increase the confidence that all dependencies are captured when pull requests are issues for new software.

[fgeorgatos]

and perhaps also look at combining it wiht http://docker.io/,
since that might help to deliver 100% reproducible builds and even runs (out of a common original template).

[fgeorgatos]

following a suggestion of Kenneth, here is some reference info for docker-io,
which could be a good candidate for reinforcing total build reproducibility, esp. if combined with hashdist
(and among other benefits, permit the parallel builds of blact/ncbi-toolkit and other such software):

Date: Fri, 6 Dec 2013 15:04:14 -0600
From: Lokesh Mandvekar <lsm5@fedoraproject.org>
To: devel-announce@lists.fedoraproject.org
Cc: golang@lists.fedoraproject.org
Subject: docker-io rpm and docs
Message-ID: <20131206210414.GB18031@naruto.usersys.redhat.com>
Content-Type: text/plain; charset="utf-8"

Hi lists,

docker-io 0.7.0 has made its way into yum stable repos for 19, 20 and epel
and 0.7.1 is being pushed out to testing by Vincent Batts currently.
Please test and add karma here:
https://admin.fedoraproject.org/updates/docker-io

The fedora docs have been merged in the master branch and can be found here:
http://docs.docker.io/en/master/installation/fedora/

This will probably be pushed to the 'latest' branch when 0.7.2 is released,
so until then, if you click on the fedora icon here:
http://www.docker.io/gettingstarted/#h_installation, all you'll see is
http://docs.docker.io/en/latest/installation/fedora/

Comments, suggestions welcome :)
-- 
Lokesh

[fgeorgatos]

btw. Just for the record, Docker v0.7 (26/11/2013) seems to be of major importance because
a) it no longer requires a patched Linux kernel and seems to run fine on a plenty of unmodified envs
b) it has been now adopted by a number of other parties, including Google Compute Engine, RedHat v6.5 etc
http://www.redhat.com/about/news/press-archive/2013/11/red-hat-launches-latest-version-of-red-hat-enterprise-linux-6

[wpoely86]

Docker seems very cool 😎 and much better then HashDist. LXC contains are really 'waterproof': the even separate the process trees.

[wpoely86]

Using docker will be quite a bit of work. We will need a basic Linux image (with the same glibc version as our install target). Maybe using unionfs or something similar will help? The builds hosts of then needs to have a very basic Linux version.

[fgeorgatos]

Getting a default image is no different than, say, downloading the source
of one intel compiler.

Then, the next step is how to spawn the whole machinery is user-space,
obtaining proper builds,
presumably allowing for the first time checkinstall-like features (ie.
auto-create .rpm, .deb etc).

hashdist still has a role, fi. when you play with vars like DISPLAY, TERM,
LC_* etc;
this aspect has not yet been confined all that much within easybuild
and I'm not 100% sure that docker-io is the final say as regards that
aspect...

In fact, IMHO, the cake of these layers should give you very very
reproducible and rich buildsets:

• docker-io -(vm/core image) > hashist -(env. tuning) > easybuild -(build
  process) > hpcbios (composite sw stacks)
  I can see how someone could obtain a major "repro" objective with just two
  of the above layers,
  yet I am not certain that all possible use-cases & needs are met in such a
  case.

[boegel]

the Nix package manager looks very interested for providing a jail tool for EasyBuild...

[boegel]

see https://nixos.org, http://chromaticleaves.com/posts/nix-in-2-days.html, http://lethalman.blogspot.be/2014/07/nix-pill-1-why-you-should-give-it-try.html