Skip to content

Version specification in main text of tutorials/pyproject-toml.md #191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Apr 3, 2024
Merged

Version specification in main text of tutorials/pyproject-toml.md #191

Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
a832d9b
More description on version specification in main text, model version…
sneakers-the-rat Mar 1, 2024
10fb76a
dummy commit because github is not receiving my pushes?
sneakers-the-rat Mar 1, 2024
2b1cd51
undo dummy commit, github received
sneakers-the-rat Mar 1, 2024
c59600b
Merge branch 'main' into dependency-spec
sneakers-the-rat Mar 4, 2024
15f6040
run precommit
sneakers-the-rat Mar 4, 2024
9ce4bc1
merge main
sneakers-the-rat Mar 19, 2024
972dbf2
Apply suggestions from code review
sneakers-the-rat Mar 19, 2024
ba9f79c
fix poetry link
sneakers-the-rat Mar 19, 2024
713ac6a
lint
sneakers-the-rat Mar 19, 2024
20f4bef
finish footnote
sneakers-the-rat Mar 19, 2024
6bb0f9d
Update tutorials/pyproject-toml.md
sneakers-the-rat Mar 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 54 additions & 18 deletions tutorials/pyproject-toml.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -296,7 +296,10 @@ license = {file = "LICENSE"}
```
### Step 3: Specify Python version with `requires-python`

Finally, add the `requires-python` field to your `pyproject.toml` `[project]` table. The `requires-python` field, helps pip understand the lowest version of Python that you package supports when it's installed. It is thus a single value.
Add the `requires-python` field to your `pyproject.toml` `[project]` table.
The `requires-python` field helps pip identify which Python versions that your package supports.
It is set to a single value.
The [packaging specification](https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata-requires-python) defines`requires-python` as a string that uses version specifiers. Most projects will specify the oldest Python version supported by the package. In some advanced cases, an upper bound is set to indicate which future Python versions, if any, will be supported.


{emphasize-lines="22"}
Expand Down Expand Up @@ -334,8 +337,42 @@ The `dependencies =` section contains a list (or array in the toml language) of
[build-system] # <- this is a table
requires = ["hatchling"] # this is an array (or list) of requirements
```

dependencies are added in an array (similar to a Python list) structure.

```toml
dependencies = ["numpy", "requests", "pandas", "pydantic"]
```

A dependency can be limited to specific versions using a **version specifier.**
If the dependency has no version specifier after the dependency name, your package can use any version of the dependent package.
Code changes over time, bugs are fixed, APIs change, and so it's good to be clear about which version of the dependency you wrote your code to be compatible with - a package you wrote this year probably isn't compatible with numpy v0.0.1!

[Learn more about various ways to specify ranges of package versions here.](https://packaging.python.org/en/latest/specifications/version-specifiers/#id5)

The most common version specifier is a **lower bound,** allowing any version higher than the specified version.
Ideally you should set this to the lowest version that is still compatible with your package, but in practice for new packages this is often set at the version that was current at the time the package was written[^lowerbound].

[^lowerbound]: Some packaging tools will do this for you when you add a dependency using their cli interface. For example [`poetry add`](https://python-poetry.org/docs/cli/#add) will add the most recent version with a `^` specifier, and [`pdm add`](https://pdm-project.org/latest/reference/cli/#add) will add the most recent version with `>=`.

Lower bounds look like this:

```toml
dependencies = [ "numpy>=1.0" ]
```

Commas are used to separate individual dependencies, and each package in your `dependencies` section can use different types of version specifiers:

```toml
dependencies = [
"numpy>=1.0", # Greater than or equal to 1.0
"requests==10.1", # Exactly 10.1
"pandas", # Any version
"pydantic>=1.7,<2" # Greater than or equal to 1.7, but less than 2
]
```

Your `pyproject.toml` file will now look like this:

{emphasize-lines="24"}
```toml
Expand All @@ -362,31 +399,30 @@ readme = "README.md"
license = {file = 'LICENSE'}
requires-python = ">=3.10"

dependencies = ["numpy", "requests", "pandas", "pydantic"]
dependencies = ["numpy>=1.0", "requests==10.1", "pandas", "pydantic>=1.7,<2"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sneakers-the-rat one thing i think users will get stuck on here is how do they know what minimum version of a tool a package supports? i'm not sure how we'd explain that. it's not trivial because normally if you are new to software dev you are just installing tools and you won't do any pinning or lower or upper bound entries at all (unless you use poetry which has defaults). So how would i know (as a scientist who is a beginner to creating a package) that my tool can't support numpy < 1.0

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so this comment is really about adding some direction to a scientist on how to figure this out. rather than telling them to use a lower bounds with out any guidance as to how to do it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I typically set to the lower bound to the version that I create the package with. Described in another comment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I initially had that in here but took it out bc i didn't want to be too verbose, but if this is a sticking point then yes :). poetry at least does this automatically with poetry add, same with pdm add. i'm sort of surprised that hatch doesn't seem to have a cli command to add a dependency.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sneakers-the-rat I think if we choose the greater than current numpy version and make the comment: "Greater than or equal to the current version" that might clarify for @lwasser's concern.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes this would absolutely clarify my concern. i think this is important to add but also guiding a user through how to pick the minimum version would be great. Currently using hatch, it does not support this feature (yet i think anyway) but i'm pretty sure ofek plans to add something like this? or that locking and deps are in the future of hatch's tooling. if you could add this clarification that would be great. @sneakers-the-rat

```

:::{admonition} Pin dependencies with caution
Pinning dependencies refers to specifying a specific version of a dependency like this `numpy == 1.0`. In some specific cases, you may chose to pin a package version for a specific package dependency.
"Pinning" a dependency means setting it to a specific version, like this:

Declaring lower bounds involves ensuring that a user has at least a specific version (or greater) of a package installed. This is important as often your package is not backwards compatible with an older version of a tool - for example a version of Pandas that was released 5 years ago.

You can declare a lower bound using syntax like this:

`ruamel-yaml>=0.17.21`

[Learn more about various ways to specify ranges of package versions here.](https://packaging.python.org/en/latest/specifications/version-specifiers/#id5)

Note that unless you are building an application, you want to be cautious about pinning dependencies to precise versions. For example:

`numpy == 1.0.2`
`numpy == 1.0`.

If you are building a library package that other developers will depend upon, you must be cautious before pinning to a precise dependency version. Applications, such as production websites, will often pin their dependencies since other packages will not depend on their project.
This is because
users will be installing your package into various environments.
A dependency pinned to a single specific version can make
resolving a Python environment more challenging. As such only
pin dependencies to a specific version if you absolutely need to
do so.

Similarly, you should be cautious when specifying an upper bound on a package.
These two specifications are equivalent:

```
pydantic>=1.10,<2
pydantic^1.10
```

One build tool that you should be aware of that pins dependencies to an upper bound by default is Poetry. [Read more about how to safely add dependencies with Poetry, here.](challenges-with-poetry)
:::

Expand Down Expand Up @@ -438,7 +474,7 @@ readme = "README.md"
license = {file = 'LICENSE'}
requires-python = ">=3.10"

dependencies = ["numpy", "requests", "pandas", "pydantic"]
dependencies = ["numpy>=1.0", "requests==10.1", "pandas", "pydantic>=1.7,<2"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes so here - i know pydantic well at this point. below 2.0 makes sense. but why did you pick 1.7 as the lower bound? and why doesn't pandas have any bounds?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just random numbers to illustrate the version syntax :) - showing you can do exact pins, lower bounds, ranges, or unspecified - let me add some annotations to make that explicit

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome. yes that clarification would be great. for instance the pydantic <2 makes a ton of sense


classifiers = [
"Development Status :: 4 - Beta",
Expand Down Expand Up @@ -488,7 +524,7 @@ readme = "README.md"
license = {file = 'LICENSE'}
requires-python = ">=3.10"

dependencies = ["ruamel-yaml>=0.17.21", "requests", "python-dotenv", "pydantic"]
dependencies = ["numpy>=1.0", "requests==10.1", "pandas", "pydantic>=1.7,<2"]

classifiers = [
"Development Status :: 4 - Beta",
Expand Down Expand Up @@ -539,7 +575,7 @@ readme = "README.md"
license = {file = 'LICENSE'}
requires-python = ">=3.10"

dependencies = ["ruamel-yaml>=0.17.21", "requests", "python-dotenv", "pydantic"]
dependencies = ["numpy>=1.0", "requests==10.1", "pandas", "pydantic>=1.7,<2"]

classifiers = [
"Development Status :: 4 - Beta",
Expand Down Expand Up @@ -612,7 +648,7 @@ classifiers = [
]


dependencies = ["xarray", "requests"]
dependencies = ["numpy>=1.0", "requests==10.1", "pandas", "pydantic>=1.7,<2"]
# This is the metadata that pip reads to understand what versions your package supports
requires-python = ">=3.10"
readme = "README.md"
Expand Down
Loading