Skip to content

Use functools.lru_cache with _getconftest_pathlist #4227

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

blueyed
Copy link
Contributor

@blueyed blueyed commented Oct 24, 2018

For pytest's own suite the cache_info() looks as follows:

> session.config._getconftest_pathlist.cache_info()
CacheInfo(hits=231, misses=19, maxsize=None, currsize=19)

While it does not really make a difference for me this might help with
larger test suites / the case mentioned in
#2206 (comment).

/cc @boxed

For pytest's own suite the `cache_info()` looks as follows:

    > session.config._getconftest_pathlist.cache_info()
    CacheInfo(hits=231, misses=19, maxsize=None, currsize=19)

While it does not really make a difference for me this might help with
larger test suites / the case mentioned in
pytest-dev#2206 (comment).
@boxed
Copy link
Contributor

boxed commented Oct 24, 2018

(Moving the discussion here) Yep. It helps, but there is some other big issue that overshadows this.

@nicoddemus
Copy link
Member

AttributeError: 'module' object has no attribute 'lru_cache' on py27.

Any objections to applying this only for PY3?

@blueyed
Copy link
Contributor Author

blueyed commented Oct 24, 2018

Any objections to applying this only for PY3?

No.
But is not clear to me how to approach this - please feel free to edit / post a diff.

@nicoddemus
Copy link
Member

Done! 👍

@codecov
Copy link

codecov bot commented Oct 24, 2018

Codecov Report

Merging #4227 into features will increase coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@             Coverage Diff              @@
##           features    #4227      +/-   ##
============================================
+ Coverage     95.82%   95.82%   +<.01%     
============================================
  Files           109      109              
  Lines         24236    24313      +77     
  Branches       2390     2401      +11     
============================================
+ Hits          23224    23298      +74     
- Misses          718      719       +1     
- Partials        294      296       +2
Flag Coverage Δ
#docs 28.51% <33.33%> (+0.01%) ⬆️
#doctesting 28.51% <33.33%> (+0.01%) ⬆️
#linting 28.51% <33.33%> (+0.01%) ⬆️
#linux 95.6% <100%> (ø) ⬆️
#nobyte 91.86% <33.33%> (-0.02%) ⬇️
#numpy 92.91% <100%> (-0.01%) ⬇️
#pexpect 41.28% <100%> (-0.03%) ⬇️
#py27 93.95% <33.33%> (ø) ⬆️
#py34 92.24% <66.66%> (-0.01%) ⬇️
#py35 92.25% <66.66%> (-0.01%) ⬇️
#py36 93.91% <66.66%> (-0.05%) ⬇️
#py37 92.28% <66.66%> (+0.01%) ⬆️
#trial 92.91% <100%> (-0.01%) ⬇️
#windows 94.01% <100%> (-0.04%) ⬇️
#xdist 93.76% <100%> (-0.03%) ⬇️
Impacted Files Coverage Δ
src/_pytest/config/__init__.py 94.94% <100%> (+0.02%) ⬆️
src/_pytest/capture.py 93.36% <0%> (-0.46%) ⬇️
src/_pytest/terminal.py 92.96% <0%> (-0.06%) ⬇️
src/_pytest/assertion/rewrite.py 95.68% <0%> (ø) ⬆️
testing/logging/test_reporting.py 100% <0%> (ø) ⬆️
testing/test_collection.py 99.76% <0%> (ø) ⬆️
testing/python/fixture.py 99.24% <0%> (ø) ⬆️
testing/test_conftest.py 99.6% <0%> (ø) ⬆️
src/_pytest/fixtures.py 97.31% <0%> (ø) ⬆️
testing/test_recwarn.py 99.55% <0%> (ø) ⬆️
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2a45851...1786ad1. Read the comment docs.

@@ -907,6 +908,10 @@ def _getconftest_pathlist(self, name, path):
values.append(relroot)
return values

if six.PY3:
# once we drop Python 2, please change this to use the normal decorator syntax (#4227)
_getconftest_pathlist = functools.lru_cache(maxsize=None)(_getconftest_pathlist)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Old school style! 😎

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

\o/

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw

if six.PY3:

is going to break when python4 comes out -- usually prefer if six.PY2: ... / if not six.PY2: ...

I've seen other projects do this in a different way:

if six.PY2:
    def lru_cache(*_, **__):
        def dec(fn):
            return fn
        return dec
else:
    from functools import lru_cache

neat trick though (saw this from twitter, missed the PR notification)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@asottile
Please consider creating a follow up PR - otherwise I might do it for the follow up via #4224 (the removed, 2nd commit).
It should go into pytest.compat probably, so that it is easier to use everywhere.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All fair points. I don't mind leaving the code as is, and also don't mind to change to the ones suggested here too. 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will move it to compat now.

@RonnyPfannschmidt RonnyPfannschmidt merged commit 65b97c2 into pytest-dev:features Oct 25, 2018
@blueyed blueyed deleted the _getconftest_pathlist-lru branch October 25, 2018 10:50
blueyed added a commit to blueyed/pytest that referenced this pull request Oct 25, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants