From 91995a20df98f7ac2f6cc50ddb5f270cb8915efd Mon Sep 17 00:00:00 2001 From: Tzu-ping Chung Date: Thu, 11 Feb 2021 15:38:49 +0800 Subject: [PATCH 01/13] Propose musllinux --- pep-9999.rst | 108 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 108 insertions(+) create mode 100644 pep-9999.rst diff --git a/pep-9999.rst b/pep-9999.rst new file mode 100644 index 00000000000..26a560c279e --- /dev/null +++ b/pep-9999.rst @@ -0,0 +1,108 @@ +PEP: 9999 +Title: Platform Tag for Linux Distributions Using Musl +Author: Tzu-ping Chung +Sponsor: TBD +PEP-Delegate: TBD +Discussions-To: https://discuss.python.org/t/7165 +Status: Draft +Type: Informational +Content-Type: text/x-rst +Created: TBA + + +Abstract +======== + +This PEP proposes a new platfrom tag series ``musllinux`` for +binary Python package distributions for a Python installation linked +against musl on a Linux distribution. The tag works similarly to the +"perennial manylinux" platform tags specified in :pep:`600`, but +targeting platforms based on musl instead. + + +Motivation +========== + +With the wide use of containers, distributions such as `Alpine Linux`_ +have been gaining more popularity than ever. Many of them based on +`musl`_, a different libc implementation from glibc, and therefore +cannot use the existing ``manylinux`` platform tags. This means that +Python package projects cannot deploy binary distributions on PyPI +for them. Users of such projects demand build-time constraints from +those projects, putting unnecessary burden on project maintainers. + + +Rationale +========= + +Logic behind the new platform tag largely follows :pep:`600`, and +require wheels using this tag make similar promises. Please refer to +the PEP for more details on rationale and reasoning behind the design. + + +Specification +============= + +Tags using the new scheme will take the form:: + + musllinux_${MUSLMAJOR}_${MUSLMINOR}_${ARCH} + +Distributions using the tag make similar promises to those discribed +in :pep:`600`, including: + +1. The distribution works on any mainstream Linux distributions with + musl version ``${MUSLMAJOR}.${MUSLMINOR}`` or later. +2. The distribution's ``${ARCH}`` matches the return value of + ``sysconfig.get_platform()`` on the host system. + +One exception is made to the first promise: If a project chooses to +publish a distribution using musl 1.2 or later on, it does not need to +promise distributions on the same architecture built against musl 1.1 +or earlier to work on musl 1.2 or later. This is due to a potentially +ABI incompatible change in musl 1.2 that switched ``time_t`` to a +64-bit type, and may cause breakage to libraries built on musl 1.1 and +earlier. [1]_ + + +Backwards Compatibility +======================= + +There are no backwards compatibility concerns in this PEP. + + +Rejected Ideas +============== + +Create a platform tag based specifically for Alpine Linux +--------------------------------------------------------- + +Past experience on the ``manylinux`` tag series shows this approach +would be too costly time-wise. The author feels the "works well with +others" rule both is more inclusive and works well enough in practice. + + +References +========== + +.. _`Alpine Linux`: https://alpinelinux.org/ + +.. _musl: https://musl.libc.org + +.. [1] https://musl.libc.org/time64.html + + +Copyright +========= + +This document is placed in the public domain or under the +CC0-1.0-Universal license, whichever is more permissive. + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: From fccea0babb26302193da11dcfc828c4fbafa9dad Mon Sep 17 00:00:00 2001 From: Tzu-ping Chung Date: Thu, 11 Feb 2021 15:49:34 +0800 Subject: [PATCH 02/13] Switch to named references --- pep-9999.rst | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index 26a560c279e..2b367e68764 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -23,12 +23,12 @@ targeting platforms based on musl instead. Motivation ========== -With the wide use of containers, distributions such as `Alpine Linux`_ -have been gaining more popularity than ever. Many of them based on -`musl`_, a different libc implementation from glibc, and therefore -cannot use the existing ``manylinux`` platform tags. This means that -Python package projects cannot deploy binary distributions on PyPI -for them. Users of such projects demand build-time constraints from +With the wide use of containers, distributions such as Alpine Linux, +[alpine]_ have been gaining more popularity than ever. Many of them +based on musl, [musl]_ a different libc implementation from glibc, and +therefore cannot use the existing ``manylinux`` platform tags. This +means that Python package projects cannot deploy binary distributions +on PyPI for them. Users of such projects demand build constraints from those projects, putting unnecessary burden on project maintainers. @@ -61,7 +61,7 @@ promise distributions on the same architecture built against musl 1.1 or earlier to work on musl 1.2 or later. This is due to a potentially ABI incompatible change in musl 1.2 that switched ``time_t`` to a 64-bit type, and may cause breakage to libraries built on musl 1.1 and -earlier. [1]_ +earlier. [musl-time64]_ Backwards Compatibility @@ -84,11 +84,11 @@ others" rule both is more inclusive and works well enough in practice. References ========== -.. _`Alpine Linux`: https://alpinelinux.org/ +.. [alpine] https://alpinelinux.org/ -.. _musl: https://musl.libc.org +.. [musl] https://musl.libc.org -.. [1] https://musl.libc.org/time64.html +.. [musl-time64] https://musl.libc.org/time64.html Copyright From 0b6b83dfe1f58afc94bd8b6ba9026c13be129a52 Mon Sep 17 00:00:00 2001 From: Tzu-ping Chung Date: Fri, 12 Feb 2021 01:38:14 +0800 Subject: [PATCH 03/13] Remove the 1.1-1.2 exception --- pep-9999.rst | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index 2b367e68764..84590ebfc63 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -55,14 +55,6 @@ in :pep:`600`, including: 2. The distribution's ``${ARCH}`` matches the return value of ``sysconfig.get_platform()`` on the host system. -One exception is made to the first promise: If a project chooses to -publish a distribution using musl 1.2 or later on, it does not need to -promise distributions on the same architecture built against musl 1.1 -or earlier to work on musl 1.2 or later. This is due to a potentially -ABI incompatible change in musl 1.2 that switched ``time_t`` to a -64-bit type, and may cause breakage to libraries built on musl 1.1 and -earlier. [musl-time64]_ - Backwards Compatibility ======================= @@ -88,8 +80,6 @@ References .. [musl] https://musl.libc.org -.. [musl-time64] https://musl.libc.org/time64.html - Copyright ========= From 48188147133d21ddc26395222d169fa45e7bbeeb Mon Sep 17 00:00:00 2001 From: Tzu-ping Chung Date: Wed, 17 Mar 2021 21:28:02 +0800 Subject: [PATCH 04/13] Add refs on ABI backwards compatibility guarantees --- pep-9999.rst | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/pep-9999.rst b/pep-9999.rst index 84590ebfc63..e9d012ff094 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -35,6 +35,13 @@ those projects, putting unnecessary burden on project maintainers. Rationale ========= +According to the documentation, musl has a stable ABI, and maintains +backwards compatibility, [musl-compatibility]_ [compare-libcs]_ so a +binary compiled against an earlier version of musl is guaranteed to +run against a newer musl runtime. [musl-compat-ml]_ Therefore, we use +a scheme similar to the glibc-version-based manylinux tags, but +against musl versions instead of glibc. + Logic behind the new platform tag largely follows :pep:`600`, and require wheels using this tag make similar promises. Please refer to the PEP for more details on rationale and reasoning behind the design. @@ -80,6 +87,12 @@ References .. [musl] https://musl.libc.org +.. [musl-compatibility] https://wiki.musl-libc.org/compatibility.html + +.. [compare-libcs] https://www.etalabs.net/compare_libcs.html + +.. [musl-compat-ml] https://mail.python.org/archives/list/distutils-sig@python.org/message/VRXSTNXWHPAVUW253ZCWWMP7WDTBAQDL/ + Copyright ========= From 04666349093ea3ea874c892f4c2c50e7601b2838 Mon Sep 17 00:00:00 2001 From: Tzu-ping Chung Date: Wed, 17 Mar 2021 21:46:55 +0800 Subject: [PATCH 05/13] Recommendation to package indexes --- pep-9999.rst | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/pep-9999.rst b/pep-9999.rst index e9d012ff094..1f339ee7a0b 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -62,6 +62,24 @@ in :pep:`600`, including: 2. The distribution's ``${ARCH}`` matches the return value of ``sysconfig.get_platform()`` on the host system. +It is recommended for Python package repositories, including PyPI, to +accept platform tags matching the following regular expression:: + + musllinux_([0-9]+)_([0-9]+)_([^-]+) + +Python package repositories may impose additional requirements to +reject Wheels with known issues, including but not limited to: + +* A ``musl_1_1`` wheel containing symbols only available in musl 1.2. +* Wheel that depends on external libraries not considered generally + available to the intended audience of the package index. +* A platform tag claiming compatibility to a non-existent musl version + (like ``musl_9000_0``). + +Such policies are ultimately up to individual package repositories. +It is not the author's intention to impose restrictions to the +maintainers. + Backwards Compatibility ======================= From 7d1c3ef2d875849f88ab5adb9de2e77d0404f967 Mon Sep 17 00:00:00 2001 From: Tzu-ping Chung Date: Wed, 17 Mar 2021 21:47:37 +0800 Subject: [PATCH 06/13] Set Brett as sponsor --- pep-9999.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-9999.rst b/pep-9999.rst index 1f339ee7a0b..6c99a9b0ef5 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -1,7 +1,7 @@ PEP: 9999 Title: Platform Tag for Linux Distributions Using Musl Author: Tzu-ping Chung -Sponsor: TBD +Sponsor: Brett Cannon PEP-Delegate: TBD Discussions-To: https://discuss.python.org/t/7165 Status: Draft From 98ce5e72f97153355bf026c31dd25faec3cae03d Mon Sep 17 00:00:00 2001 From: Tzu-ping Chung Date: Wed, 17 Mar 2021 21:53:13 +0800 Subject: [PATCH 07/13] Set creation date --- pep-9999.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-9999.rst b/pep-9999.rst index 6c99a9b0ef5..5c587e38da8 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -7,7 +7,7 @@ Discussions-To: https://discuss.python.org/t/7165 Status: Draft Type: Informational Content-Type: text/x-rst -Created: TBA +Created: 2021-03-17 Abstract From d82f83faf8374774635ccd206824574ba12acc87 Mon Sep 17 00:00:00 2001 From: Tzu-ping Chung Date: Wed, 17 Mar 2021 23:25:54 +0800 Subject: [PATCH 08/13] Assign musllinux as PEP 656 --- pep-9999.rst => pep-0656.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename pep-9999.rst => pep-0656.rst (99%) diff --git a/pep-9999.rst b/pep-0656.rst similarity index 99% rename from pep-9999.rst rename to pep-0656.rst index 5c587e38da8..cae78d45376 100644 --- a/pep-9999.rst +++ b/pep-0656.rst @@ -1,4 +1,4 @@ -PEP: 9999 +PEP: 656 Title: Platform Tag for Linux Distributions Using Musl Author: Tzu-ping Chung Sponsor: Brett Cannon From 3f33807bb6186b164b3f2b5482d8eb91b50fee68 Mon Sep 17 00:00:00 2001 From: Tzu-ping Chung Date: Wed, 17 Mar 2021 23:31:24 +0800 Subject: [PATCH 09/13] musllinux tag value examples --- pep-0656.rst | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/pep-0656.rst b/pep-0656.rst index cae78d45376..bfba2ef469e 100644 --- a/pep-0656.rst +++ b/pep-0656.rst @@ -62,10 +62,25 @@ in :pep:`600`, including: 2. The distribution's ``${ARCH}`` matches the return value of ``sysconfig.get_platform()`` on the host system. +Example values:: + + musllinux_1_1_x86_64 # musl 1.1 running on x86-64. + musllinux_1_2_aarch64 # musl 1.2 running on ARM 64-bit. + +The value can be formatted with the following Python code:: + + import sysconfig + + def format_musllinux(musl_version: tuple[int, int]) -> str: + os_name, sep, arch = sysconfig.get_platform().partition("-") + assert os_name == "linux" and sep, "Not a Linux" + arch = arch.replace("-", "_") + return f"musllinux_{musl_version[0]}_{musl_version[1]}_{arch}" + It is recommended for Python package repositories, including PyPI, to accept platform tags matching the following regular expression:: - musllinux_([0-9]+)_([0-9]+)_([^-]+) + musllinux_([0-9]+)_([0-9]+)_([^-]+) Python package repositories may impose additional requirements to reject Wheels with known issues, including but not limited to: From c5449adb9fa24c6d8e0b091dc8975c3031d69728 Mon Sep 17 00:00:00 2001 From: Tzu-ping Chung Date: Thu, 18 Mar 2021 01:03:22 +0800 Subject: [PATCH 10/13] The name is musllinux, not just musl --- pep-0656.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0656.rst b/pep-0656.rst index bfba2ef469e..dfa1af996f9 100644 --- a/pep-0656.rst +++ b/pep-0656.rst @@ -85,11 +85,11 @@ accept platform tags matching the following regular expression:: Python package repositories may impose additional requirements to reject Wheels with known issues, including but not limited to: -* A ``musl_1_1`` wheel containing symbols only available in musl 1.2. +* A ``musllinux_1_1`` wheel containing symbols only available in musl 1.2. * Wheel that depends on external libraries not considered generally available to the intended audience of the package index. * A platform tag claiming compatibility to a non-existent musl version - (like ``musl_9000_0``). + (like ``musllinux_9000_0``). Such policies are ultimately up to individual package repositories. It is not the author's intention to impose restrictions to the From 3efba0f19267ef9a5a0d0e2b7be1e27456eb75c2 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 17 Mar 2021 10:55:00 -0700 Subject: [PATCH 11/13] Update pep-0656.rst --- pep-0656.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0656.rst b/pep-0656.rst index dfa1af996f9..30c82643197 100644 --- a/pep-0656.rst +++ b/pep-0656.rst @@ -54,7 +54,7 @@ Tags using the new scheme will take the form:: musllinux_${MUSLMAJOR}_${MUSLMINOR}_${ARCH} -Distributions using the tag make similar promises to those discribed +Distributions using the tag make similar promises to those described in :pep:`600`, including: 1. The distribution works on any mainstream Linux distributions with From 5c161c60b834480c102f108e2522bdf6653fa431 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 17 Mar 2021 10:55:07 -0700 Subject: [PATCH 12/13] Update pep-0656.rst --- pep-0656.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0656.rst b/pep-0656.rst index 30c82643197..712a78f287e 100644 --- a/pep-0656.rst +++ b/pep-0656.rst @@ -36,9 +36,9 @@ Rationale ========= According to the documentation, musl has a stable ABI, and maintains -backwards compatibility, [musl-compatibility]_ [compare-libcs]_ so a +backwards compatibility [musl-compatibility]_ [compare-libcs]_, so a binary compiled against an earlier version of musl is guaranteed to -run against a newer musl runtime. [musl-compat-ml]_ Therefore, we use +run against a newer musl runtime [musl-compat-ml]_. Therefore, we use a scheme similar to the glibc-version-based manylinux tags, but against musl versions instead of glibc. From 0529d2731259626c2f705b437315f66de2497fa8 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 17 Mar 2021 10:55:11 -0700 Subject: [PATCH 13/13] Update pep-0656.rst --- pep-0656.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/pep-0656.rst b/pep-0656.rst index 712a78f287e..83546ad6037 100644 --- a/pep-0656.rst +++ b/pep-0656.rst @@ -23,9 +23,9 @@ targeting platforms based on musl instead. Motivation ========== -With the wide use of containers, distributions such as Alpine Linux, -[alpine]_ have been gaining more popularity than ever. Many of them -based on musl, [musl]_ a different libc implementation from glibc, and +With the wide use of containers, distributions such as Alpine Linux +[alpine]_, have been gaining more popularity than ever. Many of them +based on musl [musl]_, a different libc implementation from glibc, and therefore cannot use the existing ``manylinux`` platform tags. This means that Python package projects cannot deploy binary distributions on PyPI for them. Users of such projects demand build constraints from