Add `--base-dir` option #287

swryan · 2021-05-06T15:55:09Z

add --base-dir option
add --submit option

When testing an installed python package, the path to source files appears as the full site-packages path.

We originally addressed this using the --output option to capture and post-process the data, which then we could submit with the new --submit option.

A more direct solution was to add a --base_dir argument and do the path replacement within coveralls-python.

Both of these new options are included here for your consideration.

* add option to upload previously generated file * cleanup * fix * add --base_dir arg * result=None? * fix * win fix * posix base_dir * big * fix tests

TheKevJames · 2021-05-17T23:48:43Z

Hi @swryan, thanks for the contribution! Could you expand a bit upon why a user might make use of this feature? I'm having a bit of trouble understanding what this flag would be used for. A quick example would be absolutely wonderful!

swryan · 2021-05-18T02:08:59Z

Sure... we use it in our workflow here:
https://github.com/OpenMDAO/OpenMDAO/blob/master/.github/workflows/openmdao_test_workflow.yml#L191

We like to run our CI tests with the installed package and in a directory outside the repo to catch any packaging errors (e.g. missing files), so coveralls.io would include the full installed path including site-packages, etc. The installed path of course varies across platforms, which also messes up the merged results.

So we generate the coverage and use the --base-dir arg to strip the site-packages part of the path:

          pip install git+https://github.com/swryan/coveralls-python
          SITE_DIR=`python -c 'import site; print(site.getsitepackages()[-1])'`
          coveralls --base_dir $SITE_DIR

As I mentioned before.. our first approach to this problem was to --output and post-process the data and then --submit it (we had originally named the option --upload):
https://github.com/OpenMDAO/OpenMDAO/blob/197e2810c30895719c5cdb51988b67308c63e715/.travis.yml#L175

    coveralls --rcfile=../../.coveragerc --output=coveralls.json;
    SITE=`python -c 'import site; print(site.getsitepackages()[0])'`;
    sed "s/${SITE//\//\\/}\///g" < coveralls.json > coveralls-upd.json;
    coveralls --upload=coveralls-upd.json;

These are just the solutions we came up with. If there's a better way, I'm all ears.

TheKevJames

Thanks for the example, that was very clear. I can't think of any other decent way to support this with existing options, so I'm definitely in favour of introducing a new approach.

I actually like both of these (--basedir and --upload) -- I can't think of a good reason not to introduce both. Added comments, mostly on the stylistic side of things, but overall the implementation looks great.

TheKevJames · 2021-05-18T02:40:54Z

coveralls/reporter.py

+    def __init__(self, cov, conf, base_dir=None):
        self.coverage = []
+        if base_dir:
+            self.base_dir = base_dir.replace(os.path.sep, '/')
+            if self.base_dir[-1] != '/':
+                self.base_dir += '/'
+        else:
+            self.base_dir = None


nit:

Suggested change

def __init__(self, cov, conf, base_dir=None):

self.coverage = []

if base_dir:

self.base_dir = base_dir.replace(os.path.sep, '/')

if self.base_dir[-1] != '/':

self.base_dir += '/'

else:

self.base_dir = None

def __init__(self, cov, conf, base_dir=''):

self.coverage = []

self.base_dir = base_dir.replace(os.path.sep, '/')

if self.base_dir[-1] != '/':

self.base_dir += '/'

Keeping base_dir unconditionally as a string is a bit easier to reason about (fewer cases to enumerate) and will allow us to simplify the handling both here and in parse_file(), which no longer needs to have as many conditional checks (see comment below).

You need the if base_dir test regardless or the index operation will fail, so I thought it better to just do the minimal amount of work (the assignment) in the common case... no need to do the replace or the indexing operations 99% of the time

>>> base_dir = '' >>> if base_dir[-1] != '/': ... base_dir += '/' ... Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: string index out of range

Ah, good call, makes sense to leave as you have it (though the above example could be replaced with if not base_dir.endswith('/') to avoid that IndexError, FYI!).

ya that's a good point.. I guess I thought checking that single last char was more direct than using a string func, but I see the advantage

thanks for the thorough review

FWIW you are correct and the direct indexing is faster:

~ » python3 -m timeit -s "x = 'asdf'" "x.endswith('f')" 2000000 loops, best of 5: 120 nsec per loop ~ » python3 -m timeit -s "x = 'asdf'" "x[-1] == 'f'" 5000000 loops, best of 5: 46.3 nsec per loop ~ » python3 -m timeit -s "x = 'asdf'" "x and x[-1] == 'f'" 5000000 loops, best of 5: 54.3 nsec per loop ~ » python3 -m timeit -s "x = ''" "x.endswith('f')" 2000000 loops, best of 5: 112 nsec per loop ~ » python3 -m timeit -s "x = ''" "x and x[-1] == 'f'" 20000000 loops, best of 5: 16.7 nsec per loop

...however at least personally I find that it's not always worth picking this most performant code for a block like this if another option would be more readable etc. Given this app tends to run once on CI with no real latency concerns, a couple hundred nanoseconds here and there is effectively ignorable :)

coveralls/reporter.py

TheKevJames · 2021-05-18T02:46:58Z

coveralls/cli.py

                              config_file=options['--rcfile'],
-                              service_name=options['--service'])
+                              service_name=options['--service'],
+                              base_dir=options.get('--base_dir', None))


To keep up with the below null vs string comments:

Suggested change

base_dir=options.get('--base_dir', None))

base_dir=options.get('--base_dir') or '')

TheKevJames · 2021-05-18T02:47:37Z

coveralls/cli.py

+            log.info(result.get('message', None))
+            log.info(result.get('url', None))


nit: dict.get already returns None by default, no need to specify it.

Suggested change

log.info(result.get('message', None))

log.info(result.get('url', None))

log.info(result.get('message'))

log.info(result.get('url'))

TheKevJames · 2021-05-18T02:48:20Z

coveralls/api.py

        workman.get_data()

-        return CoverallReporter(workman, workman.config).coverage
+        base_dir = self.config.get('base_dir', None)


(see below)

Suggested change

base_dir = self.config.get('base_dir', None)

base_dir = self.config.get('base_dir') or ''

TheKevJames · 2021-05-18T02:51:20Z

coveralls/cli.py

 Global options:
    --service=<name>  Provide an alternative service name to submit.
    --rcfile=<file>   Specify configuration file. [default: .coveragerc]
+    --base_dir=<dir>  Base directory that is removed from reported paths.


Please rename to:

Suggested change

--base_dir=<dir> Base directory that is removed from reported paths.

--basedir=<dir> Base directory that is removed from reported paths.

I was going to think through a convention for hyphens vs underscores for this, only to realize that was a silly level of overkill when we can just side-step any future consistency issues.

Enables users to dynamically remove a given base path from the generated config failes and to upload files generated in previous runs of the tool.

add --base-dir option (#1)

5b1cd92

* add option to upload previously generated file * cleanup * fix * add --base_dir arg * result=None? * fix * win fix * posix base_dir * big * fix tests

swryan requested a review from TheKevJames as a code owner May 6, 2021 15:55

swryan and others added 5 commits May 6, 2021 13:30

Merge branch 'master' into master

f6993bc

tests

77b8c4d

new file

a8a6b53

more tests for --base_dir

e0811ab

lint

4e65569

TheKevJames requested changes May 18, 2021

View reviewed changes

swryan and others added 2 commits May 18, 2021 14:54

change --base_dir to --basedir, default to ''

e71c7e4

Merge branch 'master' into master

d7a384b

TheKevJames approved these changes May 24, 2021

View reviewed changes

TheKevJames merged commit 165a5cd into TheKevJames:master May 24, 2021

	base_dir=options.get('--base_dir', None))
	base_dir=options.get('--base_dir') or '')

		log.info(result.get('message', None))
		log.info(result.get('url', None))

	base_dir = self.config.get('base_dir', None)
	base_dir = self.config.get('base_dir') or ''

	--base_dir=<dir> Base directory that is removed from reported paths.
	--basedir=<dir> Base directory that is removed from reported paths.

Uh oh!

Add --base-dir option #287

Add --base-dir option #287

Uh oh!

Conversation

swryan commented May 6, 2021

Uh oh!

TheKevJames commented May 17, 2021

Uh oh!

swryan commented May 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TheKevJames left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Add `--base-dir` option #287

Add `--base-dir` option #287

swryan commented May 18, 2021 •

edited

Loading