Python 2.7 compatibility?

[boegel]

@samcmill As discussed during our conf call, one thing currently blocking us from leveraging hpccm in EasyBuild (cfr. #20) is that it currently requires Python 3.x; EasyBuild is not compatible with Python 3.x yet, but we're working on it (see easybuilders/easybuild-framework#133).

I took a quick look at this, by enabling tests for Python 2.7 in Travis (cfr. #24). Here's what I ran into:

• the use of from enum import Enum implies that enum34 needs to be installed (https://pypi.org/project/enum34/), which is fine imho

• with enum34 installed, a bunch of tests fail with:

  File "hpccm/recipe.py", line 75
    raise e from e
               ^
  SyntaxError: invalid syntax
  

  Changing raise e from e back to just raise e fixes that, but I'm not sure if that's OK to do?

• With the above changes, just two tests fail on top of Python 2.7:

  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "/tmp/hpc-container-maker/test/test_sed.py", line 37, in test_basic
      r's/FOO = BAR/FOO = BAZ/g']),
    File "hpccm/sed.py", line 54, in sed_step
      quoted_patterns = ['-e {}'.format(shlex.quote(patterns[0]))]
  AttributeError: 'module' object has no attribute 'quote'
  
  ======================================================================
  ERROR: test_verify (test_git.Test_git)
  git with verification enabled
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "/tmp/hpc-container-maker/test/test_git.py", line 96, in test_verify
      branch=valid_branch, verify=True),
    File "hpccm/git.py", line 104, in clone_step
      fatal=fatal)
    File "hpccm/git.py", line 53, in __verify
      p = subprocess.Popen(command, shell=True, stdout=subprocess.DEVNULL,
  AttributeError: 'module' object has no attribute 'DEVNULL'
  
  ----------------------------------------------------------------------
  

I'm happy to help out with restoring Python 2 compatibility if that's desired. If so, it would be useful to have #24 merged first, so we can rely on Travis to check Python 2.7 compatibility, and to ensure the codebase stays compatible with both Python 2.7 & 3.x.

[samcmill]

At first glance:

• enums are a minor annoyance with Python 2.7 since they are not default, but not a big deal to install the enum package.
• shlex.quote() looks like it can be handled by six
• subprocess.DEVNULL can probably be replaced by os.devnull.
• Will need to take a closer look at raise e from e. This was added to support --print-exceptions, which was a user requested feature.

[boegel]

@samcmill Hmm, relying on six to resolve the shlex.quote issue seems overkill to me, assuming we're OK with compatibility with only Python 2.7 and 3.3 or newer, since in Python 2.7 we can use pipes.quote instead, cfr. https://stackoverflow.com/a/26791164/1467322 ?

[samcmill]

My impression, perhaps incorrect, is that pipes.quote is deprecated in Python 3 and may be removed at any time. https://docs.python.org/3/library/pipes.html does not mention it, although it's (indirectly) available in the implementation https://github.com/python/cpython/blob/3.6/Lib/pipes.py due to from shlex import quote.

[boegel]

@samcmill I wasn't suggesting to switch to pipes.quote, but rather to use a construct like:

try:
    from shlex import quote  # python 3.3
except ImportError:
    from pipes import quote  # python 2.7

[samcmill]

Closing. Resolved with merge #28.