add support for --show-config (REVIEW) #1611

boegel · 2016-02-12T13:13:59Z

Current output only shows values for configuration options if they're different from the default values.

$ eb --show-current-config --prefix /tmp --installpath=$HOME/eb --job-cores=4 
#
# Current EasyBuild configuration
# [C: command line option, E: environment variable, F: configuration file]
#
buildpath      = /tmp/eb-build           [F]
installpath    = /Users/kehoste/eb       [C]
job-cores      = 4                       [C]
modules-tool   = Lmod                    [E]
optarch        = ''                      [E]
packagepath    = /tmp/packages           [C]
prefix         = /tmp                    [C]
repositorypath = /tmp/ebfiles_repo       [C]
sourcepath     = /tmp/sources            [C]

I'm considering to always mention some key settings, regardless of whether they still have the default value or not, e.g. installpath, buildpath, sourcepath, module naming scheme, etc.

Also needs tests.

gppezzi · 2016-02-12T14:41:11Z

Cool

+1 for (optionally) always displaying that before the builds

hpcugentbot · 2016-02-12T14:44:48Z

EasyBuild framework unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2672/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

damianam · 2016-02-12T14:48:52Z

I like this, specially since I am working in an environment that I didn't set up. So +1 too. A couple of suggestion though:

I would show also the default values. I can't see a reason for not displaying them, and seeing them can help to improve set ups (eg: "Oh, I forgot about this config option, I should set it to XYZ").
I would show if the displayed value is the default or not. Perhaps with (default) at the end of the value or something similar.

boegel · 2016-02-12T15:34:32Z

@damianam: the full list of configuration options is going to be *long (and thus unreadable), see below.

We should figure out a list of 'important' configuration options to always mention (even if they're default)?

#
# Current EasyBuild configuration
# [D: default value, c: command line option, e: environment variable, f: configuration file]
#
add-dummy-to-minimal-toolchains = 0                                                                                                                                          [D]
aggregate-regtest               = None                                                                                                                                       [D]
allow-modules-tool-mismatch     = 0                                                                                                                                          [D]
amend                           = None                                                                                                                                       [D]
avail-cfgfile-constants         = 0                                                                                                                                          [D]
avail-easyconfig-constants      = 0                                                                                                                                          [D]
avail-easyconfig-licenses       = 0                                                                                                                                          [D]
avail-easyconfig-params         = None                                                                                                                                       [D]
avail-easyconfig-templates      = 0                                                                                                                                          [D]
avail-module-naming-schemes     = 0                                                                                                                                          [D]
avail-modules-tools             = 0                                                                                                                                          [D]
avail-repositories              = 0                                                                                                                                          [D]
buildpath                       = /tmp/eb-build                                                                                                                              [f]
cleanup-builddir                = 1                                                                                                                                          [D]
cleanup-tmpdir                  = 1                                                                                                                                          [D]
color                           = 1                                                                                                                                          [D]
configfiles                     = /Users/kehoste/.config/easybuild/config.cfg                                                                                                [D]
debug                           = 0                                                                                                                                          [D]
default-opt-level               = defaultopt                                                                                                                                 [D]
dep-graph                       = None                                                                                                                                       [D]
deprecated                      = None                                                                                                                                       [D]
download-timeout                = None                                                                                                                                       [D]
dry-run                         = 0                                                                                                                                          [D]
dry-run-short                   = 0                                                                                                                                          [D]
dump-autopep8                   = 0                                                                                                                                          [D]
dump-test-report                = None                                                                                                                                       [D]
easyblock                       = None                                                                                                                                       [D]
experimental                    = 0                                                                                                                                          [D]
extended-dry-run                = 0                                                                                                                                          [D]
extended-dry-run-ignore-errors  = 1                                                                                                                                          [D]
external-modules-metadata       =                                                                                                                                            [D]
filter-deps                     = None                                                                                                                                       [D]
fixed-installdir-naming-scheme  = 0                                                                                                                                          [D]
force                           = 0                                                                                                                                          [D]
from-pr                         = None                                                                                                                                       [D]
git-working-dirs-path           = None                                                                                                                                       [D]
github-user                     = None                                                                                                                                       [D]
group                           = None                                                                                                                                       [D]
group-writable-installdir       = 0                                                                                                                                          [D]
hidden                          = 0                                                                                                                                          [D]
hide-deps                       = None                                                                                                                                       [D]
ignore-dirs                     = .git, .svn                                                                                                                                 [D]
ignore-osdeps                   = 0                                                                                                                                          [D]
ignoreconfigfiles               = None                                                                                                                                       [D]
include-easyblocks              =                                                                                                                                            [D]
include-module-naming-schemes   =                                                                                                                                            [D]
include-toolchains              =                                                                                                                                            [D]
info                            = 0                                                                                                                                          [D]
installpath                     = /Users/kehoste/eb                                                                                                                          [c]
installpath-modules             = None                                                                                                                                       [D]
installpath-software            = None                                                                                                                                       [D]
job                             = 0                                                                                                                                          [D]
job-backend                     = PbsPython                                                                                                                                  [D]
job-backend-config              = None                                                                                                                                       [D]
job-cores                       = 4                                                                                                                                          [c]
job-max-walltime                = 24                                                                                                                                         [D]
job-output-dir                  = /Users/kehoste/work/easybuild-framework                                                                                                    [D]
job-polling-interval            = 30.0                                                                                                                                       [D]
job-target-resource             = None                                                                                                                                       [D]
last-log                        = 0                                                                                                                                          [D]
list-easyblocks                 = None                                                                                                                                       [D]
list-toolchains                 = 0                                                                                                                                          [D]
logfile-format                  = ('easybuild', 'easybuild-%(name)s-%(version)s-%(date)s.%(time)s.log')                                                                      [D]
logtostdout                     = 0                                                                                                                                          [D]
minimal-toolchains              = 0                                                                                                                                          [D]
module-naming-scheme            = EasyBuildMNS                                                                                                                               [D]
module-only                     = 0                                                                                                                                          [D]
module-syntax                   = Tcl                                                                                                                                        [D]
moduleclasses                   = base, bio, cae, chem, compiler, data, debugger, devel, geo, ide, lang, lib, math, mpi, numlib, perf, phys, system, toolchain, tools, vis   [D]
modules-footer                  = None                                                                                                                                       [D]
modules-header                  = None                                                                                                                                       [D]
modules-tool                    = Lmod                                                                                                                                       [e]
new-pr                          = 0                                                                                                                                          [D]
only-blocks                     = None                                                                                                                                       [D]
optarch                         = ''                                                                                                                                         [e]
package                         = 0                                                                                                                                          [D]
package-naming-scheme           = EasyBuildPNS                                                                                                                               [D]
package-release                 = 1                                                                                                                                          [D]
package-tool                    = fpm                                                                                                                                        [D]
package-type                    = rpm                                                                                                                                        [D]
packagepath                     = /tmp/packages                                                                                                                              [c]
parallel                        = None                                                                                                                                       [D]
pr-branch-name                  = None                                                                                                                                       [D]
pr-commit-msg                   = None                                                                                                                                       [D]
pr-descr                        = None                                                                                                                                       [D]
pr-target-account               = hpcugent                                                                                                                                   [D]
pr-target-branch                = develop                                                                                                                                    [D]
pr-target-repo                  = easybuild-easyconfigs                                                                                                                      [D]
pr-title                        = None                                                                                                                                       [D]
prefix                          = /tmp                                                                                                                                       [c]
pretend                         = 0                                                                                                                                          [D]
quiet                           = 0                                                                                                                                          [D]
read-only-installdir            = 0                                                                                                                                          [D]
rebuild                         = 0                                                                                                                                          [D]
recursive-module-unload         = 0                                                                                                                                          [D]
regtest                         = 0                                                                                                                                          [D]
regtest-output-dir              = None                                                                                                                                       [D]
repository                      = FileRepository                                                                                                                             [D]
repositorypath                  = /tmp/ebfiles_repo                                                                                                                          [c]
review-pr                       = None                                                                                                                                       [D]
robot                           = None                                                                                                                                       [D]
robot-paths                     = /Users/kehoste/work/easybuild-easyconfigs/easybuild/easyconfigs                                                                            [D]
search                          = None                                                                                                                                       [D]
search-filename                 = None                                                                                                                                       [D]
search-short                    = None                                                                                                                                       [D]
sequential                      = 0                                                                                                                                          [D]
set-gid-bit                     = 0                                                                                                                                          [D]
show-current-config             = 1                                                                                                                                          [c]
show-default-configfiles        = 0                                                                                                                                          [D]
show-default-moduleclasses      = 0                                                                                                                                          [D]
skip                            = 0                                                                                                                                          [D]
skip-test-cases                 = 0                                                                                                                                          [D]
software                        = None                                                                                                                                       [D]
software-name                   = None                                                                                                                                       [D]
software-version                = None                                                                                                                                       [D]
sourcepath                      = /tmp/sources                                                                                                                               [c]
sticky-bit                      = 0                                                                                                                                          [D]
stop                            = None                                                                                                                                       [D]
strict                          = warn                                                                                                                                       [D]
subdir-modules                  = modules                                                                                                                                    [D]
subdir-software                 = software                                                                                                                                   [D]
subdir-user-modules             = None                                                                                                                                       [D]
suffix-modules-path             = all                                                                                                                                        [D]
terse                           = 0                                                                                                                                          [D]
test-report-env-filter          = None                                                                                                                                       [D]
testoutput                      = None                                                                                                                                       [D]
tmp-logdir                      = None                                                                                                                                       [D]
tmpdir                          = None                                                                                                                                       [D]
toolchain                       = None                                                                                                                                       [D]
toolchain-name                  = None                                                                                                                                       [D]
toolchain-version               = None                                                                                                                                       [D]
try-amend                       = None                                                                                                                                       [D]
try-software                    = None                                                                                                                                       [D]
try-software-name               = None                                                                                                                                       [D]
try-software-version            = None                                                                                                                                       [D]
try-toolchain                   = None                                                                                                                                       [D]
try-toolchain-name              = None                                                                                                                                       [D]
try-toolchain-version           = None                                                                                                                                       [D]
umask                           = None                                                                                                                                       [D]
unittest-file                   = None                                                                                                                                       [D]
update-modules-tool-cache       = 0                                                                                                                                          [D]
update-pr                       = None                                                                                                                                       [D]
upload-test-report              = 0                                                                                                                                          [D]
use-existing-modules            = 0                                                                                                                                          [D]

damianam · 2016-02-12T15:38:19Z

You have a good point there. What about having --show-current-config and --show-complete-config?

boegel · 2016-02-12T15:40:42Z

@damianam: support both makes sense, yeah

I would still like to include a set of 'important' settings in the 'short' version though... Question is which ones.

These? http://easybuild.readthedocs.org/en/latest/Configuration.html#mandatory-configuration-settings

gppezzi · 2016-02-12T17:32:10Z

The options you propose to always show are a good start. My vote is:

installpath
buildpath
sourcepath
(+) repositorypath
(+) robot-paths
(-) module naming scheme: maybe only if not the default?

The column with [F], [C], [E] should go on the left of the actual variable content.

boegel · 2016-02-12T17:39:05Z

@gppezzi you mean like this?

$ eb --show-current-config --prefix /tmp --installpath=$HOME/eb --job-cores=4 
#
# Current EasyBuild configuration
# [C: command line option, E: environment variable, F: configuration file]
#
buildpath      [F] = /tmp/eb-build
installpath    [C] = /Users/kehoste/eb
job-cores      [C] = 4
modules-tool   [E] = Lmod
optarch        [E] = ''
packagepath    [C] = /tmp/packages
prefix         [C] = /tmp
repositorypath [C] = /tmp/ebfiles_repo
sourcepath     [C] = /tmp/sources

gppezzi · 2016-02-12T17:56:55Z

👍 Much better, or else that would float around in case of long paths, which is very annoying ;)

boegel · 2016-02-12T21:04:42Z

reworked according to feedback, unit test add

always included (even when default)

--buildpath
--installpath
--repositorypath
--robot-paths
--sourcepath

current output:

$  eb --show-config --prefix /tmp --installpath=$HOME/eb --job-cores=4
#
# Current EasyBuild configuration
# (C: command line argument, D: default value, E: environment variable, F: configuration file)
#
buildpath      (F) = /tmp/eb-build
installpath    (C) = /Users/kehoste/eb
job-cores      (C) = 4
modules-tool   (E) = Lmod
optarch        (E) = ''
packagepath    (C) = /tmp/packages
prefix         (C) = /tmp
repositorypath (C) = /tmp/ebfiles_repo
robot-paths    (D) = /Users/kehoste/work/easybuild-easyconfigs/easybuild/easyconfigs
sourcepath     (C) = /tmp/sources

$  eb --show-full-config --prefix /tmp --installpath=$HOME/eb --job-cores=4
#
# Current EasyBuild configuration
# (C: command line argument, D: default value, E: environment variable, F: configuration file)
#
add-dummy-to-minimal-toolchains (D) = False
aggregate-regtest               (D) = None
allow-modules-tool-mismatch     (D) = False
amend                           (D) = None
avail-cfgfile-constants         (D) = False
avail-easyconfig-constants      (D) = False
avail-easyconfig-licenses       (D) = False
avail-easyconfig-params         (D) = None
avail-easyconfig-templates      (D) = False
avail-module-naming-schemes     (D) = False
avail-modules-tools             (D) = False
avail-repositories              (D) = False
buildpath                       (F) = /tmp/eb-build
cleanup-builddir                (D) = True
cleanup-tmpdir                  (D) = True
color                           (D) = True
configfiles                     (D) = /Users/kehoste/.config/easybuild/config.cfg
debug                           (D) = False
default-opt-level               (D) = defaultopt
dep-graph                       (D) = None
deprecated                      (D) = None
download-timeout                (D) = None
dry-run                         (D) = False
dry-run-short                   (D) = False
dump-autopep8                   (D) = False
dump-test-report                (D) = None
easyblock                       (D) = None
experimental                    (D) = False
extended-dry-run                (D) = False
extended-dry-run-ignore-errors  (D) = True
external-modules-metadata       (D) = 
filter-deps                     (D) = None
fixed-installdir-naming-scheme  (D) = False
force                           (D) = False
from-pr                         (D) = None
git-working-dirs-path           (D) = None
github-user                     (D) = None
group                           (D) = None
group-writable-installdir       (D) = False
hidden                          (D) = False
hide-deps                       (D) = None
ignore-dirs                     (D) = .git, .svn
ignore-osdeps                   (D) = False
ignoreconfigfiles               (D) = None
include-easyblocks              (D) = 
include-module-naming-schemes   (D) = 
include-toolchains              (D) = 
info                            (D) = False
installpath                     (C) = /Users/kehoste/eb
installpath-modules             (D) = None
installpath-software            (D) = None
job                             (D) = False
job-backend                     (D) = PbsPython
job-backend-config              (D) = None
job-cores                       (C) = 4
job-max-walltime                (D) = 24
job-output-dir                  (D) = /Users/kehoste/work/easybuild-framework
job-polling-interval            (D) = 30.0
job-target-resource             (D) = None
last-log                        (D) = False
list-easyblocks                 (D) = None
list-toolchains                 (D) = False
logfile-format                  (D) = ('easybuild', 'easybuild-%(name)s-%(version)s-%(date)s.%(time)s.log')
logtostdout                     (D) = False
minimal-toolchains              (D) = False
module-naming-scheme            (D) = EasyBuildMNS
module-only                     (D) = False
module-syntax                   (D) = Tcl
moduleclasses                   (D) = base, bio, cae, chem, compiler, data, debugger, devel, geo, ide, lang, lib, math, mpi, numlib, perf, phys, system, toolchain, tools, vis
modules-footer                  (D) = None
modules-header                  (D) = None
modules-tool                    (E) = Lmod
new-pr                          (D) = False
only-blocks                     (D) = None
optarch                         (E) = ''
package                         (D) = False
package-naming-scheme           (D) = EasyBuildPNS
package-release                 (D) = 1
package-tool                    (D) = fpm
package-type                    (D) = rpm
packagepath                     (C) = /tmp/packages
parallel                        (D) = None
pr-branch-name                  (D) = None
pr-commit-msg                   (D) = None
pr-descr                        (D) = None
pr-target-account               (D) = hpcugent
pr-target-branch                (D) = develop
pr-target-repo                  (D) = easybuild-easyconfigs
pr-title                        (D) = None
prefix                          (C) = /tmp
pretend                         (D) = False
quiet                           (D) = False
read-only-installdir            (D) = False
rebuild                         (D) = False
recursive-module-unload         (D) = False
regtest                         (D) = False
regtest-output-dir              (D) = None
repository                      (D) = FileRepository
repositorypath                  (C) = /tmp/ebfiles_repo
review-pr                       (D) = None
robot                           (D) = None
robot-paths                     (D) = /Users/kehoste/work/easybuild-easyconfigs/easybuild/easyconfigs
search                          (D) = None
search-filename                 (D) = None
search-short                    (D) = None
sequential                      (D) = False
set-gid-bit                     (D) = False
show-default-configfiles        (D) = False
show-default-moduleclasses      (D) = False
skip                            (D) = False
skip-test-cases                 (D) = False
software                        (D) = None
software-name                   (D) = None
software-version                (D) = None
sourcepath                      (C) = /tmp/sources
sticky-bit                      (D) = False
stop                            (D) = None
strict                          (D) = warn
subdir-modules                  (D) = modules
subdir-software                 (D) = software
subdir-user-modules             (D) = None
suffix-modules-path             (D) = all
terse                           (D) = False
test-report-env-filter          (D) = None
testoutput                      (D) = None
tmp-logdir                      (D) = None
tmpdir                          (D) = None
toolchain                       (D) = None
toolchain-name                  (D) = None
toolchain-version               (D) = None
try-amend                       (D) = None
try-software                    (D) = None
try-software-name               (D) = None
try-software-version            (D) = None
try-toolchain                   (D) = None
try-toolchain-name              (D) = None
try-toolchain-version           (D) = None
umask                           (D) = None
unittest-file                   (D) = None
update-modules-tool-cache       (D) = False
update-pr                       (D) = None
upload-test-report              (D) = False
use-existing-modules            (D) = False

hpcugentbot · 2016-02-12T21:05:30Z

EasyBuild framework unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2673/console for more details.