Skip to content

feature: add manylinux_2_28 image #1277

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 30, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@ jobs:
platform: "i686"
- policy: "manylinux2010"
platform: "x86_64"
- policy: "manylinux_2_28"
platform: "x86_64"

env:
POLICY: ${{ matrix.policy }}
Expand Down
6 changes: 6 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,12 @@ jobs:
env: POLICY="manylinux_2_24" PLATFORM="s390x"
- arch: ppc64le
env: POLICY="manylinux_2_24" PLATFORM="ppc64le"
- arch: arm64-graviton2
virt: vm
group: edge
env: POLICY="manylinux_2_28" PLATFORM="aarch64"
- arch: ppc64le
env: POLICY="manylinux_2_28" PLATFORM="ppc64le"
- arch: arm64-graviton2
virt: vm
group: edge
Expand Down
20 changes: 16 additions & 4 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,11 @@ on June 30th, 2024.

PEP 600 has been designed to be "future-proof" and does not enforce specific symbols and a specific distro to build.
It only states that a wheel tagged ``manylinux_x_y`` shall work on any distro based on ``glibc>=x.y``.
The manylinux project supports ``manylinux_2_24`` images for ``x86_64``, ``i686``, ``aarch64``, ``ppc64le`` and ``s390x``.
The manylinux project supports:

- ``manylinux_2_24`` images for ``x86_64``, ``i686``, ``aarch64``, ``ppc64le`` and ``s390x``.

- ``manylinux_2_28`` images for ``x86_64``, ``aarch64`` and ``ppc64le``.


Wheel packages compliant with those tags can be uploaded to
Expand All @@ -55,8 +59,9 @@ pip:
| ``manylinux`` tag | Client-side pip | CPython (sources) version | Distribution default pip compatibility |
| | version required | embedding a compatible pip | |
+===================+==================+============================+===========================================+
| ``manylinux_x_y`` | pip >= 20.3 | 3.8.10+, 3.9.5+, 3.10.0+ | ALT Linux 10+, Debian 11+, Fedora 34+, |
| | | | Mageia 8+, Photon OS 3.0 with updates, |
| ``manylinux_x_y`` | pip >= 20.3 | 3.8.10+, 3.9.5+, 3.10.0+ | ALT Linux 10+, RHEL 9+, Debian 11+, |
| | | | Fedora 34+, Mageia 8+, |
| | | | Photon OS 3.0 with updates, |
| | | | Ubuntu 21.04+ |
+-------------------+------------------+----------------------------+-------------------------------------------+
| ``manylinux2014`` | pip >= 19.3 | 3.7.8+, 3.8.4+, 3.9.0+ | CentOS 7 rh-python38, CentOS 8 python38, |
Expand Down Expand Up @@ -98,6 +103,13 @@ done the work for you. The images are uploaded to `quay.io`_ and are tagged
for repeatable builds.


manylinux_2_28 (AlmaLinux 8 based)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- x86_64 image: ``quay.io/pypa/manylinux_2_24_x86_64``
- aarch64 image: ``quay.io/pypa/manylinux_2_24_aarch64``
- ppc64le image: ``quay.io/pypa/manylinux_2_24_ppc64le``

manylinux_2_24 (Debian 9 based)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Expand Down Expand Up @@ -147,7 +159,7 @@ Image content

All images currently contain:

- CPython 3.6, 3.7, 3.8, 3.9, 3.10, and PyPy 3.7, 3.8, 3.9 installed in
- CPython 3.6, 3.7, 3.8, 3.9, 3.10, 3.11, and PyPy 3.7, 3.8, 3.9 installed in
``/opt/python/<python tag>-<abi tag>``. The directories are named
after the PEP 425 tags for each environment --
e.g. ``/opt/python/cp37-cp37m`` contains a CPython 3.7 build, and
Expand Down
5 changes: 5 additions & 0 deletions build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,11 @@ elif [ "${POLICY}" == "manylinux_2_24" ]; then
DEVTOOLSET_ROOTPATH=
PREPEND_PATH=
LD_LIBRARY_PATH_ARG=
elif [ "${POLICY}" == "manylinux_2_28" ]; then
BASEIMAGE="${MULTIARCH_PREFIX}almalinux:8"
DEVTOOLSET_ROOTPATH="/opt/rh/gcc-toolset-11/root"
PREPEND_PATH="${DEVTOOLSET_ROOTPATH}/usr/bin:"
LD_LIBRARY_PATH_ARG="${DEVTOOLSET_ROOTPATH}/usr/lib64:${DEVTOOLSET_ROOTPATH}/usr/lib:${DEVTOOLSET_ROOTPATH}/usr/lib64/dyninst:${DEVTOOLSET_ROOTPATH}/usr/lib/dyninst"
elif [ "${POLICY}" == "musllinux_1_1" ]; then
BASEIMAGE="${MULTIARCH_PREFIX}alpine:3.12"
DEVTOOLSET_ROOTPATH=
Expand Down
6 changes: 5 additions & 1 deletion docker/build_scripts/build-sqlite3.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,11 @@ strip_ /manylinux-rootfs

# Install
cp -rlf /manylinux-rootfs/* /
ldconfig /
if [ "${BASE_POLICY}" == "musllinux" ]; then
ldconfig /
elif [ "${BASE_POLICY}" == "manylinux" ]; then
ldconfig
fi

# Clean-up for runtime
rm -rf /manylinux-rootfs/usr/local/share
12 changes: 10 additions & 2 deletions docker/build_scripts/install-build-packages.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,12 @@ set -exuo pipefail
# if a devel package is added to COMPILE_DEPS,
# make sure the corresponding library is added to RUNTIME_DEPS if applicable

if [ "${AUDITWHEEL_POLICY}" == "manylinux2010" ] || [ "${AUDITWHEEL_POLICY}" == "manylinux2014" ]; then
PACKAGE_MANAGER=yum
if [ "${AUDITWHEEL_POLICY}" == "manylinux2010" ] || [ "${AUDITWHEEL_POLICY}" == "manylinux2014" ] || [ "${AUDITWHEEL_POLICY}" == "manylinux_2_28" ]; then
if [ "${AUDITWHEEL_POLICY}" == "manylinux_2_28" ]; then
PACKAGE_MANAGER=dnf
else
PACKAGE_MANAGER=yum
fi
COMPILE_DEPS="bzip2-devel ncurses-devel readline-devel tk-devel gdbm-devel libpcap-devel xz-devel openssl openssl-devel keyutils-libs-devel krb5-devel libcom_err-devel libidn-devel curl-devel uuid-devel libffi-devel kernel-headers"
if [ "${AUDITWHEEL_POLICY}" == "manylinux2010" ]; then
COMPILE_DEPS="${COMPILE_DEPS} db4-devel"
Expand Down Expand Up @@ -40,6 +44,10 @@ elif [ "${PACKAGE_MANAGER}" == "apt" ]; then
rm -rf /var/lib/apt/lists/*
elif [ "${PACKAGE_MANAGER}" == "apk" ]; then
apk add --no-cache ${COMPILE_DEPS}
elif [ "${PACKAGE_MANAGER}" == "dnf" ]; then
dnf -y install --allowerasing ${COMPILE_DEPS}
dnf clean all
rm -rf /var/cache/yum
else
echo "Not implemented"
exit 1
Expand Down
3 changes: 3 additions & 0 deletions docker/build_scripts/install-libxcrypt.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ source $MY_DIR/build_utils.sh
if [ "$BASE_POLICY" == "musllinux" ]; then
echo "Skip libxcrypt installation on musllinux"
exit 0
elif [ "${AUDITWHEEL_POLICY}" == "manylinux_2_28" ]; then
echo "Skip libxcrypt installation on manylinux_2_28"
exit 0
fi

# We need perl 5.14+
Expand Down
22 changes: 20 additions & 2 deletions docker/build_scripts/install-runtime-packages.sh
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ source $MY_DIR/build_utils.sh


# MANYLINUX_DEPS: Install development packages (except for libgcc which is provided by gcc install)
if [ "${AUDITWHEEL_POLICY}" == "manylinux2010" ] || [ "${AUDITWHEEL_POLICY}" == "manylinux2014" ]; then
if [ "${AUDITWHEEL_POLICY}" == "manylinux2010" ] || [ "${AUDITWHEEL_POLICY}" == "manylinux2014" ] || [ "${AUDITWHEEL_POLICY}" == "manylinux_2_28" ]; then
MANYLINUX_DEPS="glibc-devel libstdc++-devel glib2-devel libX11-devel libXext-devel libXrender-devel mesa-libGL-devel libICE-devel libSM-devel zlib-devel expat-devel"
elif [ "${AUDITWHEEL_POLICY}" == "manylinux_2_24" ]; then
MANYLINUX_DEPS="libc6-dev libstdc++-6-dev libglib2.0-dev libx11-dev libxext-dev libxrender-dev libgl1-mesa-dev libice-dev libsm-dev libz-dev libexpat1-dev"
Expand All @@ -44,7 +44,7 @@ else
fi

# RUNTIME_DEPS: Runtime dependencies. c.f. install-build-packages.sh
if [ "${AUDITWHEEL_POLICY}" == "manylinux2010" ] || [ "${AUDITWHEEL_POLICY}" == "manylinux2014" ]; then
if [ "${AUDITWHEEL_POLICY}" == "manylinux2010" ] || [ "${AUDITWHEEL_POLICY}" == "manylinux2014" ] || [ "${AUDITWHEEL_POLICY}" == "manylinux_2_28" ]; then
RUNTIME_DEPS="zlib bzip2 expat ncurses readline tk gdbm libpcap xz openssl keyutils-libs libkadm5 libcom_err libidn libcurl uuid libffi"
if [ "${AUDITWHEEL_POLICY}" == "manylinux2010" ]; then
RUNTIME_DEPS="${RUNTIME_DEPS} db4"
Expand Down Expand Up @@ -120,6 +120,22 @@ elif [ "${AUDITWHEEL_POLICY}" == "manylinux_2_24" ]; then
apt-get upgrade -qq -y
apt-get install -qq -y --no-install-recommends ca-certificates gpg curl locales
TOOLCHAIN_DEPS="binutils gcc g++ gfortran"
elif [ "${AUDITWHEEL_POLICY}" == "manylinux_2_28" ]; then
PACKAGE_MANAGER=dnf
BASETOOLS="${BASETOOLS} curl glibc-locale-source glibc-langpack-en hardlink hostname libcurl libxcrypt which"
# See https://unix.stackexchange.com/questions/41784/can-yum-express-a-preference-for-x86-64-over-i386-packages
echo "multilib_policy=best" >> /etc/yum.conf
# Error out if requested packages do not exist
echo "skip_missing_names_on_install=False" >> /etc/yum.conf
# Make sure that locale will not be removed
sed -i '/^override_install_langs=/d' /etc/yum.conf
dnf -y upgrade
dnf -y install dnf-plugins-core
dnf config-manager --set-enabled powertools # for yasm
TOOLCHAIN_DEPS="gcc-toolset-11-binutils gcc-toolset-11-gcc gcc-toolset-11-gcc-c++ gcc-toolset-11-gcc-gfortran"
if [ "${AUDITWHEEL_ARCH}" == "x86_64" ]; then
TOOLCHAIN_DEPS="${TOOLCHAIN_DEPS} yasm"
fi
elif [ "${AUDITWHEEL_POLICY}" == "musllinux_1_1" ]; then
TOOLCHAIN_DEPS="binutils gcc g++ gfortran"
BASETOOLS="${BASETOOLS} curl util-linux"
Expand All @@ -136,6 +152,8 @@ elif [ "${PACKAGE_MANAGER}" == "apt" ]; then
apt-get install -qq -y --no-install-recommends ${BASETOOLS} ${TOOLCHAIN_DEPS} ${MANYLINUX_DEPS} ${RUNTIME_DEPS}
elif [ "${PACKAGE_MANAGER}" == "apk" ]; then
apk add --no-cache ${BASETOOLS} ${TOOLCHAIN_DEPS} ${MANYLINUX_DEPS} ${RUNTIME_DEPS}
elif [ "${PACKAGE_MANAGER}" == "dnf" ]; then
dnf -y install --allowerasing ${BASETOOLS} ${TOOLCHAIN_DEPS} ${MANYLINUX_DEPS} ${RUNTIME_DEPS}
else
echo "Not implemented"
exit 1
Expand Down
11 changes: 10 additions & 1 deletion docker/build_scripts/update-system-packages.sh
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,10 @@ elif [ "${AUDITWHEEL_POLICY}" == "manylinux_2_24" ]; then
find /etc/ssl/certs -name 'DST_Root_CA_X3.pem' -delete
update-ca-certificates
fi
elif [ "${AUDITWHEEL_POLICY}" == "manylinux_2_28" ]; then
dnf -y upgrade
dnf clean all
rm -rf /var/cache/yum
elif [ "${AUDITWHEEL_POLICY}" == "musllinux_1_1" ]; then
apk upgrade --no-cache
else
Expand All @@ -53,7 +57,7 @@ if [ "${BASE_POLICY}" == "manylinux" ]; then
if localedef --list-archive | grep -sq -v -i ^en_US.utf8; then
localedef --list-archive | grep -v -i ^en_US.utf8 | xargs localedef --delete-from-archive
fi
if [ "${AUDITWHEEL_POLICY}" == "manylinux2014" ] || [ "${AUDITWHEEL_POLICY}" == "manylinux2010" ]; then
if [ "${AUDITWHEEL_POLICY}" == "manylinux2010" ] || [ "${AUDITWHEEL_POLICY}" == "manylinux2014" ] || [ "${AUDITWHEEL_POLICY}" == "manylinux_2_28" ]; then
mv -f ${LOCALE_ARCHIVE} ${LOCALE_ARCHIVE}.tmpl
build-locale-archive --install-langs="en_US.utf8"
elif [ "${AUDITWHEEL_POLICY}" == "manylinux_2_24" ]; then
Expand Down Expand Up @@ -99,5 +103,10 @@ fi
if [ -f /usr/local/lib/libcrypt.so.1 ]; then
# Remove libcrypt to only use installed libxcrypt instead
find /lib* /usr/lib* \( -name 'libcrypt.a' -o -name 'libcrypt.so' -o -name 'libcrypt.so.*' -o -name 'libcrypt-2.*.so' \) -delete
fi

if [ "${BASE_POLICY}" == "musllinux" ]; then
ldconfig /
elif [ "${BASE_POLICY}" == "manylinux" ]; then
ldconfig
fi
4 changes: 4 additions & 0 deletions tests/run_tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ elif [ "${AUDITWHEEL_POLICY}" == "manylinux_2_24" ]; then
apt-get update -qq
elif [ "${AUDITWHEEL_POLICY}" == "musllinux_1_1" ]; then
PACKAGE_MANAGER=apk
elif [ "${AUDITWHEEL_POLICY}" == "manylinux_2_28" ]; then
PACKAGE_MANAGER=dnf
else
echo "Unsupported policy: '${AUDITWHEEL_POLICY}'"
exit 1
Expand Down Expand Up @@ -65,6 +67,8 @@ elif [ "${PACKAGE_MANAGER}" == "apt" ]; then
apt-get install -qq -y --no-install-recommends openssh-client
elif [ "${PACKAGE_MANAGER}" == "apk" ]; then
apk add --no-cache openssh-client
elif [ "${PACKAGE_MANAGER}" == "dnf" ]; then
dnf -y install --allowerasing openssh-clients
else
echo "Unsupported package manager: '${PACKAGE_MANAGER}'"
exit 1
Expand Down