Remove duplicate extensions in R 3.5.x easyconfigs, and add test to detect such issues

[Flamefire]

A user reported a failure when building with: eb --package R-3.5.1-foss-2018b.eb --skip --force --ignore-checksum --debug --info

== 2021-01-28 18:36:09,232 run.py:222 INFO running cmd:  R CMD INSTALL /tmp/centos/R/3.5.1/foss-2018b/later/later   --library=/mnt/eb/apps/R/3.5.1-foss-2018b/lib64/R/library --no-clean-on-error
== 2021-01-28 18:36:10,920 build_log.py:169 ERROR EasyBuild crashed with an error (at easybuild/base/exceptions.py:124 in __init__): cmd " R CMD INSTALL /tmp/centos/R/3.5.1/foss-2018b/later/later   --library=/mnt/eb/apps/R/3.5.1-foss-2018b/lib64/R/library --no-clean-on-error " exited with exit code 1 and output:
* installing *source* package later ...
[...]
C11-style threads.h support not detected. Using tinycthread library.
[...]
/mnt/eb/apps/binutils/2.30-GCCcore-7.3.0/bin/ld.gold: error: ./tinycthread/tinycthread.o: multiple definition of 'mtx_init'
/mnt/eb/apps/binutils/2.30-GCCcore-7.3.0/bin/ld.gold: tinycthread.o: previous definition here
[...]
/mnt/eb/apps/binutils/2.30-GCCcore-7.3.0/bin/ld.gold: error: ./tinycthread/tinycthread.o: multiple definition of 'tss_set'
/mnt/eb/apps/binutils/2.30-GCCcore-7.3.0/bin/ld.gold: tinycthread.o: previous definition here
collect2: error: ld returned 1 exit status
make: *** [later.so] Error 1
ERROR: compilation failed for package later
 (at easybuild/tools/run.py:533 in parse_cmd_output)
== 2021-01-28 18:36:10,921 filetools.py:1684 INFO Removing lock /mnt/eb/apps/.locks/_mnt_eb_apps_R_3.5.1-foss-2018b.lock...
== 2021-01-28 18:36:10,922 filetools.py:330 INFO Path /mnt/eb/apps/.locks/_mnt_eb_apps_R_3.5.1-foss-2018b.lock successfully removed.
== 2021-01-28 18:36:10,923 filetools.py:1688 INFO Lock removed: /mnt/eb/apps/.locks/_mnt_eb_apps_R_3.5.1-foss-2018b.lock
== 2021-01-28 18:36:10,923 easyblock.py:3340 WARNING build failed (first 300 chars): cmd " R CMD INSTALL /tmp/centos/R/3.5.1/foss-2018b/later/later   --library=/mnt/eb/apps/R/3.5.1-foss-2018b/lib64/R/library --no-clean-on-error " exited with exit code 1 and output:

Reason seems to be duplicated packages in the R ECs.

Removed the older versions and added a test case to detect this

[algiar]

It seems that promises-1.0.1 is also duplicated in the easyconfig. In this case it is not causing the build to fail though.

[Flamefire]

Yeah, there were in fact 3 duplicate packages but only 1 with different versions which caused the problem

[lexming]

Test report by @lexming
FAILED
Build succeeded for 11 out of 18 (4 easyconfigs in total)
node370.hydra.os - Linux centos linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz, Python 2.7.5
See https://gist.github.com/bc631e263b396a0cc07f5fea5d67047c for a full test report.

[lexming]

@Flamefire you opened Pandora's box here

== 2021-02-01 21:18:07,667 easyblock.py:603 INFO sha256 checksum for /user/brussel/101/vsc10122/.local/easybuild-skylake/sources/r/R/extensions/pkgmaker_0.27.tar.gz: e5eb9058e68702c15e8dc6fee6ede41cdbe8e3914d190a5a1113079db26fb003
== 2021-02-01 21:18:07,675 build_log.py:169 ERROR EasyBuild crashed with an error (at easybuild/base/exceptions.py:124 in init): Checksum verification for extension source pkgmaker_0.27.tar.gz failed (at easybuild/framework/easyblock.py:613 in fetch_extension_sources)

[Flamefire]

I hate R/CRAN!
Luckily we already fixed that in newer ECs, so I updated that. Checked with --fetch and works now

[lexming]

Test report by @lexming
SUCCESS
Build succeeded for 7 out of 7 (4 easyconfigs in total)
node374.hydra.os - Linux centos linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz, Python 2.7.5
See https://gist.github.com/3ca2123143fed6f1b8c98a2e18bca83c for a full test report.

[boegel]

lgtm

[boegel]

I also have a test report coming up, but it'll need a couple hours more... Should show up today.

[lexming]

LGTM

[lexming]

@boegel too slow 🐢

Going in, thanks @Flamefire !

[boegel]

Test report by @boegel
FAILED
Build succeeded for 18 out of 19 (4 easyconfigs in total)
node2606.swalot.os - Linux centos linux 7.9.2009, x86_64, Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHz (haswell), Python 3.6.8
See https://gist.github.com/c43a5404c5797345dd08f196419e6a3c for a full test report.

[Flamefire]

@boegel Yours shows an unrelated issue but might be worth fixing:

checking CUDA_HOME environment variable... not set
checking CUDA home path... /usr/local/cuda-11.0
checking setting CUDA library path... -L/usr/local/cuda-11.0/lib64
checking whether CUDA is available... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating src/Makevars
config.status: creating R/hegy-test.R
** libs
icpc [....]
/usr/local/cuda-11.0/bin/nvcc -O3 -G --shared -Xcompiler -fPIC -I/user/gent/400/vsc40023/eb_scratch/CO7/haswell-ib/software/R/3.5.0-iomkl-2018a-X11-20180131/lib64/R/include -rdc=true -arch=sm_30 -I/usr/local/cuda-11.0/include -c dev-fnc.cu -o dev-fnc.o
make: /usr/local/cuda-11.0/bin/nvcc: Command not found

In an iomkl build!

[boegel]

What the... Please open an issue on that.