Skip to content

document a get_version pattern for editable installs staying as correct as sensible #273

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
blueyed opened this issue Jun 30, 2018 · 14 comments

Comments

@blueyed
Copy link
Contributor

blueyed commented Jun 30, 2018

Is the following reasonable, e.g. for output of --version?

I am using this to get the current version with editable installs, useful
during development:

def get_version():
    try:
        from setuptools_scm import get_version
        return get_version(root='..', relative_to=__file__)
    except (ImportError, LookupError):
        from pkg_resources import get_distribution
        return get_distribution(__package__).version

setup.py uses the basic:

    use_scm_version=True,
    setup_requires=['setuptools_scm'],

btw: what is a use case for write_to? The file itself says that the file
should not be tracked in the SCM itself.

Ref: Vimjas/covimerage#43

@RonnyPfannschmidt
Copy link
Contributor

@blueyed ´write_tois for when people dont want to pay the cost ofpkg_resources.get_distribution`

personally i think its perfectly valid to work with stale metadata on development, just doing semi-regular pip install -e ...

@blueyed
Copy link
Contributor Author

blueyed commented Jun 30, 2018

Thanks for clarification!

blueyed added a commit to blueyed/setuptools_scm that referenced this issue Jun 30, 2018
@blueyed
Copy link
Contributor Author

blueyed commented Jun 30, 2018

Closing.
README update in #274.

@blueyed blueyed closed this as completed Jun 30, 2018
@majuscule
Copy link
Contributor

Just want to voice my support for availability of access to updated version metadata from within an editable install. I work from within an editable install constantly, and having the version data up to date would be very helpful to avoid confusion when interacting with others who are using experimental releases of my packages. I don't feel that forcing developers to remember to repeatedly pip install ... is a valid solution.

Please reopen this issue.

@blueyed blueyed reopened this Dec 12, 2018
@taldcroft
Copy link

For the get_version() function shown in the description, will the initial attempt to run setuptools_scm.get_version(..) be lightweight / fast? In particular that it doesn't actually end up running a git executable if there is no git repo in the installed package directory. I looked into the code a bit but confess to finding the code that detects the repo to be a bit inscrutable.

Running in debug mode I see for a successful detection it says:

found ep .git = setuptools_scm.git:parse  # found a .git directory??
cmd 'git rev-parse --show-toplevel'  # runs a git command now
...

While for unsuccessful:

root '/Users/aldcroft/git/sparkles'
looking for ep setuptools_scm.parse_scm /Users/aldcroft/git/sparkles
looking for ep setuptools_scm.parse_scm_fallback /Users/aldcroft/git/sparkles
Traceback (most recent call last):
  File "setup.py", line 21, in <module>
    cmdclass=cmdclass,

So it all looks good, but again just don't know what's going on under the hood.

@RonnyPfannschmidt
Copy link
Contributor

without much more tooling this is hard to solve
personally i wont go and search for patterns to enable this in combination with build tools

@RonnyPfannschmidt RonnyPfannschmidt changed the title question: get_version pattern for editable installs document a get_version pattern for editable installs staying as correct as sensible May 26, 2019
@ashwinvis
Copy link

At fluidsim, we have a use-case which has a need for the scm version (or local version as we call it) in an editable install. Being a research code, we always use it as an editable install and the local version is useful to know which commit the code was executed with. We have a module fluidsim._version which contains:

__version__ = "0.3.0"
_loc_version = None

def get_local_version():
    """Get a long "local" version."""


    global _loc_version


    if _loc_version is None:
        from setuptools_scm import get_version


        try:
            _loc_version = get_version(root="..", relative_to=__file__)
        except (LookupError, AssertionError):
            _loc_version = __version__


    return _loc_version

The if condition and the variable _loc_version avoids the lag of using get_version, and the try-except is a fallback option in case no version control repository is detected.

I wonder if this is a good pattern, or can be improved upon? Any feedback is welcome.

cc:@vezeli

@RonnyPfannschmidt
Copy link
Contributor

its possible to do this better using pkg_resources and/or impotzlib_metadata

by determining the distribution and its location its possible to infer the location and use static or dynamic determination of the version

the patternn with loc_version is slightly undesirable

@ashwinvis
Copy link

When you mention "by determining the distribution and its location", do you mean something like this:

In [1]: from importlib import resources

In [2]: from pathlib import Path

In [3]: with resources.path("fluidsim", "__init__.py") as path:
   ...:      repo = Path(path).parent.parent
   ...:      print(repo)
   ...:
/home/avmo/src/gfdyn/fluidsim

In [4]: from setuptools_scm import get_version

In [5]: get_version(repo)
Out[5]: '0.3.2.dev19+h58ca00423a36'

@RonnyPfannschmidt
Copy link
Contributor

@ashwinvis that would be a very rough starting point, but it needs some more sophistication to work out of the box in more general terms

mhw-at-yg added a commit to JamesRamm/archook that referenced this issue May 14, 2020
@embray
Copy link

embray commented Sep 22, 2020

Also related are my proposals here:

#143 (comment)

and here

#143 (comment)

james-smith-za pushed a commit to ska-sa/katgpucbf that referenced this issue Jul 8, 2021
The [setuptools documentation](https://setuptools.readthedocs.io/en/latest/pkg_resources.html)
discourages the use of pkg_resources directly as there's performance overhead
at runtime. Similarly, [usage in Sphinx](https://github.com/pypa/setuptools_scm#usage-from-sphinx)
is discouraged. Both references suggest `importlib.metadata` instead.

It also seems strange to me to require setuptools-scm to be installed, as
this is something that pip pulls in to the build environment when it is
needed. So I have removed requirements for it, and references to it in the
code, opting for `importlib` as recommended.

I've also removed the `setup_requires` kwarg from `setup.py`, in favour of
the list of build requirements in `pyproject.toml`.

Identifiers for editable installs were [rejected from PEP660](https://www.python.org/dev/peps/pep-0660/#editable-local-version-identifier),
at least as far as annotating the `version` specifier goes. This breaks
things with pip.

For versioning from within editable installs, some others seem to have
implemented something [vaguely similar to katversion](pypa/setuptools-scm#273 (comment)).

The maintainer of setuptools_scm says that [there's no good solution to
the editable installs problem right now](pypa/setuptools-scm#518).

We could perhaps consider a (clunky) workaround like [this](https://stackoverflow.com/questions/43348746/how-to-detect-if-module-is-installed-in-editable-mode#57854737).

My personal feeling is that an editable install is an exceptional case
that we shouldn't optimise for - I think the onus is on the developer to
remember that they are working in one, and not let the ambiguity get the
better of them. But I am willing to consider other points of view on this.
@sidekick-eimantas
Copy link

sidekick-eimantas commented Nov 10, 2022

@blueyed ´write_tois for when people dont want to pay the cost ofpkg_resources.get_distribution`

personally i think its perfectly valid to work with stale metadata on development, just doing semi-regular pip install -e ...

What ballpark numbers are we lookin at when talking about the cost of pkg_resources.get_distribution?
I'm seeing added latency of ~0.05s
Is this a problem seen with larger packages?

@RonnyPfannschmidt
Copy link
Contributor

In the early days and when pkg_resources was not in use it potentially meant additional 0.2 seconds

Which makes for a noticeable delay

@RonnyPfannschmidt
Copy link
Contributor

closing this as not planned - until there is editable hooks wit scm intgration, updates are not safe

happy to pick this up once someone provides the hooks

@RonnyPfannschmidt RonnyPfannschmidt closed this as not planned Won't fix, can't repro, duplicate, stale Jul 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants