From 4f9394598f362ff70570c3d03c563679843ff616 Mon Sep 17 00:00:00 2001 From: Jeremy Paige Date: Mon, 8 Apr 2024 13:59:09 -0700 Subject: [PATCH 1/7] hatch backend compat --- .../complex-python-package-builds.md | 27 ++++++++++++++++++- .../python-package-build-tools.md | 2 +- .../python-package-versions.md | 2 +- 3 files changed, 28 insertions(+), 3 deletions(-) diff --git a/package-structure-code/complex-python-package-builds.md b/package-structure-code/complex-python-package-builds.md index 58183c4e..8a5cfc3e 100644 --- a/package-structure-code/complex-python-package-builds.md +++ b/package-structure-code/complex-python-package-builds.md @@ -13,7 +13,32 @@ back-end tools. 1. **Pure-python packages:** these are packages that only rely on Python to function. Building a pure Python package is simpler. As such, you can chose a tool below that has the features that you want and be done with your decision! -2. **Python packages with non-Python extensions:** These packages have additional components called extensions written in other languages (such as C or C++). If you have a package with non-python extensions, then you need to select a build back-end tool that allows you to add additional build steps needed to compile your extension code. Further, if you wish to use a front-end tool to support your workflow, you will need to select a tool that supports additional build setups. In this case, you could use setuptools. However, we suggest that you chose build tool that supports custom build steps such as Hatch with Hatchling or PDM. PDM is an excellent choice as it allows you to also select your build back-end of choice. We will discuss this at a high level on the complex builds page. +2. **Python packages with non-Python extensions:** These packages have additional components called extensions written in other languages (such as C or C++). If you have a package with non-python extensions, then you need to select a build back-end tool that allows you to add additional build steps needed to compile your extension code. Further, if you wish to use a front-end tool to support your workflow, you will need to select a tool that supports additional build setups. We suggest that you chose build tool that supports custom build steps like Hatch. 3. **Python packages that have extensions written in different languages (e.g. Fortran and C++) or that have non Python dependencies that are difficult to install (e.g. GDAL)** These packages often have complex build steps (more complex than a package with just a few C extensions for instance). As such, these packages require tools such as [scikit-build](https://scikit-build.readthedocs.io/en/latest/) or [meson-python](https://mesonbuild.com/Python-module.html) to build. NOTE: you can use meson-python with PDM. + +## Mixing front-end and back-end projects + +It is sometimes necessary or desirable to use a build front-end with an alternative build-backend. +This is because some front-ends do not have a default back-end (`build`), and this choice is placed on the maintainer. +Other back-ends (`hatch`) have a preferred back-end (`hatchling`) but allow the maintainer to migrate to another, while +some back-ends (`poetry`) only work with a single back-end (`poetry-core`). Refer to (#python-package-build-tools) for +more information about front-end and back-end compatibility. + +In this packaging guide we recommend using `hatch` along with it's preferred back-end `hatchling`. One reason to use +hatch with an alternate back-end is for creating an extension module. A Python extension module is one that is make up, +either in part or entirely, of compiled code. In this case the back-end chosen (such as `meson-python`) must know how to +compile the extension language and bind it to Python. + +In order to use a different back-end you will need to edit your project's `pyproject.toml`. If you have a +`pyproject.toml` generated by the `hatch` command, or from following the packaging tutorial, you may have +to make a change like this + +```diff + [build-system] +- requires = ["hatchling"] ++ requires = ["meson-python"] +- build-backend = "hatchling.build" ++ build-backend = "mesonpy" +``` diff --git a/package-structure-code/python-package-build-tools.md b/package-structure-code/python-package-build-tools.md index 1017d571..a1188db5 100644 --- a/package-structure-code/python-package-build-tools.md +++ b/package-structure-code/python-package-build-tools.md @@ -344,7 +344,7 @@ as building your documentation locally. This means that you could potentially dr :widths: 20,5,50 :delim: "|" -Use Other Build Backends|✅ | Switching out build back-ends is not currently an option with Hatch. However, this feature is planned for a future release. +Use Other Build Backends|✅ | Hatch is used with the back-end Hatchling by default, but allows you to use another back-end by switching the declaration in pyproject.toml. Dependency management|✖| Currently you have to add dependencies manually with Hatch. However a feature to support dependencies management may be added in a future release. Environment Management |✅ | Hatch supports Python virtual environments. If you wish to use other types of environments such as Conda, you will need to [install a plugin such as hatch-conda for conda support](https://github.com/OldGrumpyViking/hatch-conda). Publish to PyPI and test PyPI|✅|Hatch supports publishing to both test PyPI and PyPI diff --git a/package-structure-code/python-package-versions.md b/package-structure-code/python-package-versions.md index 6a20e40c..27a4e1d9 100644 --- a/package-structure-code/python-package-versions.md +++ b/package-structure-code/python-package-versions.md @@ -179,7 +179,7 @@ your package version. To use **hatch_vcs** you will need to use the **hatchling** build back end. ```{tip} -Hatchling can also be used any of the modern build tools +Hatchling can also be used with any of the modern build tools including **Flit** and **PDM** if you prefer those for your day to day workflow. ``` From dc9d07816a1e3cfa8383f9de69e7eec2ea72633d Mon Sep 17 00:00:00 2001 From: Jeremy Paige Date: Tue, 9 Apr 2024 09:26:41 -0700 Subject: [PATCH 2/7] complex package cleanup --- .../complex-python-package-builds.md | 12 ++++++------ package-structure-code/python-package-build-tools.md | 2 +- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/package-structure-code/complex-python-package-builds.md b/package-structure-code/complex-python-package-builds.md index 8a5cfc3e..c685755d 100644 --- a/package-structure-code/complex-python-package-builds.md +++ b/package-structure-code/complex-python-package-builds.md @@ -29,16 +29,16 @@ more information about front-end and back-end compatibility. In this packaging guide we recommend using `hatch` along with it's preferred back-end `hatchling`. One reason to use hatch with an alternate back-end is for creating an extension module. A Python extension module is one that is make up, either in part or entirely, of compiled code. In this case the back-end chosen (such as `meson-python`) must know how to -compile the extension language and bind it to Python. +compile the extension language and bind it to Python but `hatchling` is only capable of building pure Python modules. In order to use a different back-end you will need to edit your project's `pyproject.toml`. If you have a `pyproject.toml` generated by the `hatch` command, or from following the packaging tutorial, you may have to make a change like this ```diff - [build-system] -- requires = ["hatchling"] -+ requires = ["meson-python"] -- build-backend = "hatchling.build" -+ build-backend = "mesonpy" + [build-system] +-requires = ["hatchling"] ++requires = ["meson-python"] +-build-backend = "hatchling.build" ++build-backend = "mesonpy" ``` diff --git a/package-structure-code/python-package-build-tools.md b/package-structure-code/python-package-build-tools.md index a1188db5..9fe9fdda 100644 --- a/package-structure-code/python-package-build-tools.md +++ b/package-structure-code/python-package-build-tools.md @@ -344,7 +344,7 @@ as building your documentation locally. This means that you could potentially dr :widths: 20,5,50 :delim: "|" -Use Other Build Backends|✅ | Hatch is used with the back-end Hatchling by default, but allows you to use another back-end by switching the declaration in pyproject.toml. +Use Other Build Backends|✅ | Hatch is used with the back-end Hatchling by default, but allows you to use another back-end by switching the declaration in pyproject.toml. Dependency management|✖| Currently you have to add dependencies manually with Hatch. However a feature to support dependencies management may be added in a future release. Environment Management |✅ | Hatch supports Python virtual environments. If you wish to use other types of environments such as Conda, you will need to [install a plugin such as hatch-conda for conda support](https://github.com/OldGrumpyViking/hatch-conda). Publish to PyPI and test PyPI|✅|Hatch supports publishing to both test PyPI and PyPI From 189be092bb24f6eac471b462d9b46e81a14e3bb1 Mon Sep 17 00:00:00 2001 From: Jeremy Paige Date: Tue, 9 Apr 2024 15:44:56 -0700 Subject: [PATCH 3/7] Update package-structure-code/complex-python-package-builds.md Co-authored-by: Carol Willing --- package-structure-code/complex-python-package-builds.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/package-structure-code/complex-python-package-builds.md b/package-structure-code/complex-python-package-builds.md index c685755d..3ca885c5 100644 --- a/package-structure-code/complex-python-package-builds.md +++ b/package-structure-code/complex-python-package-builds.md @@ -29,7 +29,7 @@ more information about front-end and back-end compatibility. In this packaging guide we recommend using `hatch` along with it's preferred back-end `hatchling`. One reason to use hatch with an alternate back-end is for creating an extension module. A Python extension module is one that is make up, either in part or entirely, of compiled code. In this case the back-end chosen (such as `meson-python`) must know how to -compile the extension language and bind it to Python but `hatchling` is only capable of building pure Python modules. +compile the extension language and bind it to Python since `hatchling` is only capable of building pure Python modules. In order to use a different back-end you will need to edit your project's `pyproject.toml`. If you have a `pyproject.toml` generated by the `hatch` command, or from following the packaging tutorial, you may have From 215ba598ec9e3ff22daca6fe134a08c1d6f01e36 Mon Sep 17 00:00:00 2001 From: Jeremy Paige Date: Tue, 9 Apr 2024 15:46:22 -0700 Subject: [PATCH 4/7] Update package-structure-code/complex-python-package-builds.md Co-authored-by: Carol Willing --- package-structure-code/complex-python-package-builds.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/package-structure-code/complex-python-package-builds.md b/package-structure-code/complex-python-package-builds.md index 3ca885c5..c6442259 100644 --- a/package-structure-code/complex-python-package-builds.md +++ b/package-structure-code/complex-python-package-builds.md @@ -27,7 +27,7 @@ some back-ends (`poetry`) only work with a single back-end (`poetry-core`). Refe more information about front-end and back-end compatibility. In this packaging guide we recommend using `hatch` along with it's preferred back-end `hatchling`. One reason to use -hatch with an alternate back-end is for creating an extension module. A Python extension module is one that is make up, +hatch with an alternate back-end is for creating an extension module. A Python extension module is one that is made up, either in part or entirely, of compiled code. In this case the back-end chosen (such as `meson-python`) must know how to compile the extension language and bind it to Python since `hatchling` is only capable of building pure Python modules. From e9f101829ce214c50794f4b328e3253433802159 Mon Sep 17 00:00:00 2001 From: Jeremy Paige Date: Tue, 9 Apr 2024 15:47:59 -0700 Subject: [PATCH 5/7] Update package-structure-code/complex-python-package-builds.md Co-authored-by: Carol Willing --- package-structure-code/complex-python-package-builds.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/package-structure-code/complex-python-package-builds.md b/package-structure-code/complex-python-package-builds.md index c6442259..9368ed1c 100644 --- a/package-structure-code/complex-python-package-builds.md +++ b/package-structure-code/complex-python-package-builds.md @@ -26,7 +26,7 @@ Other back-ends (`hatch`) have a preferred back-end (`hatchling`) but allow the some back-ends (`poetry`) only work with a single back-end (`poetry-core`). Refer to (#python-package-build-tools) for more information about front-end and back-end compatibility. -In this packaging guide we recommend using `hatch` along with it's preferred back-end `hatchling`. One reason to use +In this packaging guide we recommend using `hatch` along with it's preferred back-end `hatchling`. While this will be suitable for most packages, an alternate back-end may be used with Hatch if needed when creating an extension module. hatch with an alternate back-end is for creating an extension module. A Python extension module is one that is made up, either in part or entirely, of compiled code. In this case the back-end chosen (such as `meson-python`) must know how to compile the extension language and bind it to Python since `hatchling` is only capable of building pure Python modules. From 0c367f38227bab06351024f4b0890daff9e5dde0 Mon Sep 17 00:00:00 2001 From: Jeremy Paige Date: Tue, 9 Apr 2024 16:02:45 -0700 Subject: [PATCH 6/7] clarify that hatchling plugins can build extension modules --- .../complex-python-package-builds.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/package-structure-code/complex-python-package-builds.md b/package-structure-code/complex-python-package-builds.md index 9368ed1c..7fe91171 100644 --- a/package-structure-code/complex-python-package-builds.md +++ b/package-structure-code/complex-python-package-builds.md @@ -13,7 +13,7 @@ back-end tools. 1. **Pure-python packages:** these are packages that only rely on Python to function. Building a pure Python package is simpler. As such, you can chose a tool below that has the features that you want and be done with your decision! -2. **Python packages with non-Python extensions:** These packages have additional components called extensions written in other languages (such as C or C++). If you have a package with non-python extensions, then you need to select a build back-end tool that allows you to add additional build steps needed to compile your extension code. Further, if you wish to use a front-end tool to support your workflow, you will need to select a tool that supports additional build setups. We suggest that you chose build tool that supports custom build steps like Hatch. +2. **Python packages with non-Python extensions:** These packages have additional components called extensions written in other languages (such as C or C++). If you have a package with non-Python extensions, then you need to select a build back-end tool that allows additional build steps needed to compile your extension code. Further, if you wish to use a front-end tool to support your workflow, you will need to select a tool that supports additional build setups. We suggest that you chose build tool that supports custom build steps like Hatch. 3. **Python packages that have extensions written in different languages (e.g. Fortran and C++) or that have non Python dependencies that are difficult to install (e.g. GDAL)** These packages often have complex build steps (more complex than a package with just a few C extensions for instance). As such, these packages require tools such as [scikit-build](https://scikit-build.readthedocs.io/en/latest/) or [meson-python](https://mesonbuild.com/Python-module.html) to build. NOTE: you can use meson-python with PDM. @@ -26,10 +26,13 @@ Other back-ends (`hatch`) have a preferred back-end (`hatchling`) but allow the some back-ends (`poetry`) only work with a single back-end (`poetry-core`). Refer to (#python-package-build-tools) for more information about front-end and back-end compatibility. -In this packaging guide we recommend using `hatch` along with it's preferred back-end `hatchling`. While this will be suitable for most packages, an alternate back-end may be used with Hatch if needed when creating an extension module. -hatch with an alternate back-end is for creating an extension module. A Python extension module is one that is made up, -either in part or entirely, of compiled code. In this case the back-end chosen (such as `meson-python`) must know how to -compile the extension language and bind it to Python since `hatchling` is only capable of building pure Python modules. +In this packaging guide we recommend using `hatch` along with its preferred back-end `hatchling`. While this will be +suitable for most packages, an alternate back-end may be used with Hatch if needed when creating an extension module. +A Python extension module is one that is made up, either in part or entirely, of compiled code. In this case the +back-end chosen (such as `meson-python`) must know how to compile the extension language and bind it to Python. +`hatchling` does not know how to do this all on its own and must either make use of +[plugins](https://hatch.pypa.io/1.9/plugins/about/) or be replaced by a backend that is already capable of building +extension modules. In order to use a different back-end you will need to edit your project's `pyproject.toml`. If you have a `pyproject.toml` generated by the `hatch` command, or from following the packaging tutorial, you may have From 969e0b1e05ace23c841df035bd20f2d561792789 Mon Sep 17 00:00:00 2001 From: Jeremy Paige Date: Thu, 11 Apr 2024 09:40:42 -0700 Subject: [PATCH 7/7] de-hyphen tool descriptors --- .../complex-python-package-builds.md | 26 +++++++++---------- .../python-package-build-tools.md | 4 +-- 2 files changed, 15 insertions(+), 15 deletions(-) diff --git a/package-structure-code/complex-python-package-builds.md b/package-structure-code/complex-python-package-builds.md index 7fe91171..cc9be2f8 100644 --- a/package-structure-code/complex-python-package-builds.md +++ b/package-structure-code/complex-python-package-builds.md @@ -8,33 +8,33 @@ In the future, we want to provide resources for packaging workflows that require ## Pure Python Packages vs. packages with extensions in other languages You can classify Python package complexity into three general categories. These -categories can in turn help you select the correct package front-end and -back-end tools. +categories can in turn help you select the correct package frontend and +backend tools. 1. **Pure-python packages:** these are packages that only rely on Python to function. Building a pure Python package is simpler. As such, you can chose a tool below that has the features that you want and be done with your decision! -2. **Python packages with non-Python extensions:** These packages have additional components called extensions written in other languages (such as C or C++). If you have a package with non-Python extensions, then you need to select a build back-end tool that allows additional build steps needed to compile your extension code. Further, if you wish to use a front-end tool to support your workflow, you will need to select a tool that supports additional build setups. We suggest that you chose build tool that supports custom build steps like Hatch. +2. **Python packages with non-Python extensions:** These packages have additional components called extensions written in other languages (such as C or C++). If you have a package with non-Python extensions, then you need to select a build backend tool that allows additional build steps needed to compile your extension code. Further, if you wish to use a frontend tool to support your workflow, you will need to select a tool that supports additional build setups. We suggest that you chose build tool that supports custom build steps like Hatch. 3. **Python packages that have extensions written in different languages (e.g. Fortran and C++) or that have non Python dependencies that are difficult to install (e.g. GDAL)** These packages often have complex build steps (more complex than a package with just a few C extensions for instance). As such, these packages require tools such as [scikit-build](https://scikit-build.readthedocs.io/en/latest/) or [meson-python](https://mesonbuild.com/Python-module.html) to build. NOTE: you can use meson-python with PDM. -## Mixing front-end and back-end projects +## Mixing frontend and backend projects -It is sometimes necessary or desirable to use a build front-end with an alternative build-backend. -This is because some front-ends do not have a default back-end (`build`), and this choice is placed on the maintainer. -Other back-ends (`hatch`) have a preferred back-end (`hatchling`) but allow the maintainer to migrate to another, while -some back-ends (`poetry`) only work with a single back-end (`poetry-core`). Refer to (#python-package-build-tools) for -more information about front-end and back-end compatibility. +It is sometimes necessary or desirable to use a build frontend with an alternative build-backend. +This is because some frontends do not have a default backend (`build`), and this choice is placed on the maintainer. +Other backends (`hatch`) have a preferred backend (`hatchling`) but allow the maintainer to migrate to another, while +some backends (`poetry`) only work with a single backend (`poetry-core`). Refer to (#python-package-build-tools) for +more information about frontend and backend compatibility. -In this packaging guide we recommend using `hatch` along with its preferred back-end `hatchling`. While this will be -suitable for most packages, an alternate back-end may be used with Hatch if needed when creating an extension module. +In this packaging guide we recommend using `hatch` along with its preferred backend `hatchling`. While this will be +suitable for most packages, an alternate backend may be used with Hatch if needed when creating an extension module. A Python extension module is one that is made up, either in part or entirely, of compiled code. In this case the -back-end chosen (such as `meson-python`) must know how to compile the extension language and bind it to Python. +backend chosen (such as `meson-python`) must know how to compile the extension language and bind it to Python. `hatchling` does not know how to do this all on its own and must either make use of [plugins](https://hatch.pypa.io/1.9/plugins/about/) or be replaced by a backend that is already capable of building extension modules. -In order to use a different back-end you will need to edit your project's `pyproject.toml`. If you have a +In order to use a different backend you will need to edit your project's `pyproject.toml`. If you have a `pyproject.toml` generated by the `hatch` command, or from following the packaging tutorial, you may have to make a change like this diff --git a/package-structure-code/python-package-build-tools.md b/package-structure-code/python-package-build-tools.md index 9fe9fdda..3bbe3954 100644 --- a/package-structure-code/python-package-build-tools.md +++ b/package-structure-code/python-package-build-tools.md @@ -344,7 +344,7 @@ as building your documentation locally. This means that you could potentially dr :widths: 20,5,50 :delim: "|" -Use Other Build Backends|✅ | Hatch is used with the back-end Hatchling by default, but allows you to use another back-end by switching the declaration in pyproject.toml. +Use Other Build Backends|✅ | Hatch is used with the backend Hatchling by default, but allows you to use another backend by switching the declaration in pyproject.toml. Dependency management|✖| Currently you have to add dependencies manually with Hatch. However a feature to support dependencies management may be added in a future release. Environment Management |✅ | Hatch supports Python virtual environments. If you wish to use other types of environments such as Conda, you will need to [install a plugin such as hatch-conda for conda support](https://github.com/OldGrumpyViking/hatch-conda). Publish to PyPI and test PyPI|✅|Hatch supports publishing to both test PyPI and PyPI @@ -355,7 +355,7 @@ Install your package in editable mode|✅| Hatch will install your package into Build your sdist and wheel distributions|✅| Hatch will build the sdist and wheel distributions ✨Matrix environment creation to support testing across Python versions✨|✅| The matrix environment creation is a feature that is unique to Hatch in the packaging ecosystem. This feature is useful if you wish to test your package locally across Python versions (instead of using a tool such as tox). ✨[Nox / MAKEFILE like functionality](https://hatch.pypa.io/latest/environment/#selection)✨| ✅| This feature is also unique to Hatch. This functionality allows you to create workflows in the **pyproject.toml** configuration to do things like serve docs locally and clean your package build directory. This means you may have one less tool in your build workflow. -✨A flexible build back-end: **hatchling**✨| ✅| **The hatchling build back-end offered by the maintainer of Hatch allows developers to easily build plugins to support custom build steps when packaging. +✨A flexible build backend: **hatchling**✨| ✅| **The hatchling build backend offered by the maintainer of Hatch allows developers to easily build plugins to support custom build steps when packaging. ```