Add an option to create arch specific, python version independent pkgs by isuruf · Pull Request #5456 · conda/conda-build

isuruf · 2024-08-12T15:46:50Z

Description

Checklist - did you ...

Add a file to the news directory (using the template) for the next release's release notes?
Add / update necessary tests?
Add / update outdated documentation?

Implementation for ABI3 as described in https://github.com/conda/ceps/blob/main/cep-0020.md
This implementation depends on a specific python-abi3 package (already in conda-forge at https://github.com/conda-forge/python-abi3-feedstock) in host to set the correct runtime dependencies.
By being ABI3 agnostic, we leave room for other standards like ABI4, HPy to be supported.
Added documentation has an example, and a more complicated example for conda-forge is at https://github.com/conda-forge/python-abi3-feedstock/blob/main/recipe/example-meta.yaml

codspeed-hq · 2024-08-12T21:15:54Z

CodSpeed Performance Report

Merging #5456 will not alter performance

_{Comparing isuruf:python-ver-indep (efe1e9c) with main (caa6d2f)}

Summary

✅ 5 untouched benchmarks

beckermr

Is there a test here that asserts that the output package is put into an arch-specific part of the repodata (e.g., linux-64)? It is not obvious to me from the changes that this would happen, but I am still a conda-build n00b.

jaimergp · 2025-01-09T11:18:04Z

Thanks for the docs! We are missing the news/ entry and perhaps a short description in the PR linking to the relevant issues, discussions, CEPs, etc. I'll review once those are in.

isuruf · 2025-01-09T12:22:19Z

Thanks. Added a news entry and updated the PR description.

wolfv · 2025-01-09T12:57:42Z

Awesome work!

jaimergp

I have some suggestions and comments that need to be addressed, but nothing major. Once that's in, I'll be happy to approve!

jaimergp · 2025-01-10T18:16:52Z

-        if target_file.find("site-packages") >= 0:
-            return target_file[target_file.find("site-packages") :]
-        elif target_file.startswith("bin") and (
+        if target_file.startswith("bin") and (


Should this include the slash to prevent matches with paths like binocular.dat?

Suggested change

if target_file.startswith("bin") and (

if target_file.startswith("bin/") and (

(And in that case, update the replace below?)

Let's leave it for a different PR?

jaimergp · 2025-01-10T18:17:14Z

+        if target_file.startswith("bin") and (
            target_file not in entry_point_script_names
        ):
            return target_file.replace("bin", "python-scripts")


Suggested change

return target_file.replace("bin", "python-scripts")

return target_file.replace("bin/", "python-scripts/")

jaimergp · 2025-01-10T18:17:55Z

+            # This is a hack to make conda/mamba/micromamba compile the pure python files
+            # and for micromamba to move the files in site-packages to the correct dir.


Which versions require this hack? All of them?

I added a clarification to the comment here.

jaimergp · 2025-01-10T18:21:50Z

+     python_version_independent: true
+
+A package builder also has to indicate which standard is supported by
+the package, i.e., for ABI3,


Suggested change

the package, i.e., for ABI3,

the package, i.e., for ABI3 in conda-forge

AFAIK, python-abi3 is a conda-forge-specific package, right? Or is it in defaults too?

We should add it to defaults too.

cc @jezdez @chenghlee

@chenghlee do you think it's reasonable to add this package to defaults? Otherwise I'll update this PR to reflect that the package is only in conda-forge.

@chenghlee let me know how you want to proceed here.

We already introduced python_abi in main to support free threading:

$ conda search -c main python_abi Loading channels: done # Name Version Build Channel python_abi 3.13 0_cp313 pkgs/main python_abi 3.13 0_cp313t pkgs/main

So I don't see any need to say that the python_abi package is conda-forge specific.

(What I can't definitely say is when python_abi support will be extended to Python versions before 3.13, but that feels more like an Anaconda, rather than a conda community, problem.)

Thanks @chenghlee

Co-authored-by: jaimergp <jaimergp@users.noreply.github.com>

Co-authored-by: Jannis Leidel <jannis@leidel.info> Co-authored-by: jaimergp <jaimergp@users.noreply.github.com>

Co-authored-by: jaimergp <jaimergp@users.noreply.github.com>

jschueller · 2025-02-25T08:13:36Z

this sounds great, should save us some ci times, looking forward to this!

casperdcl · 2025-10-17T19:30:48Z

Sorry just to clarify, is this the intention?

meta.yml modification	target platform	target python
(none)	`linux-64` etc	`3.9` etc
`noarch: python`	`noarch`	`*`
`python_version_independent: true`	`linux-64` etc	`*`

If so, that would be great.

However, from my testing it seems that python_version_independent: true has no effect (i.e. it creates lib/python3.12/site-packages rather than pyver-independent site-packages).

isuruf · 2025-10-17T19:41:24Z

Yes, that's correct.

The line you linked to doesn't matter. conda-build's relocation logic works after the link check logic.

casperdcl · 2025-10-17T20:00:34Z

I still don't get it... with python_version_independent: true:

$ conda build --output-folder dist ...
$ tar tf dist/*/mypackagename-*.tar.bz2
...
lib/python3.12/site-packages/mypackagename/...
$ mamba create -n testing -c ./dist python=3.13 mypackagename
...
Could not solve for environment specs
The following packages are incompatible
├─ mypackagename * is installable and it requires
│  └─ python_abi 3.12.* *_cp312, which requires
│     └─ python 3.12.* *_cpython, which can be installed;
└─ python 3.13**  is not installable because it conflicts with any installable versions previously reported.

casperdcl · 2025-10-20T20:14:21Z

Should I open an issue?

conda-bot added the cla-signed [bot] added once the contributor has signed the CLA label Aug 12, 2024

isuruf force-pushed the python-ver-indep branch from 65657d8 to 0e05755 Compare August 12, 2024 17:38

isuruf mentioned this pull request Aug 30, 2024

CEP: support for ABI3 packages conda/ceps#86

Merged

jaimergp reviewed Nov 26, 2024

View reviewed changes

Comment thread conda_build/metadata.py Outdated

jaimergp reviewed Nov 26, 2024

View reviewed changes

Comment thread conda_build/metadata.py Outdated

beckermr reviewed Nov 27, 2024

View reviewed changes

isuruf force-pushed the python-ver-indep branch from 90a41d0 to 7f8a9ed Compare January 3, 2025 08:08

isuruf marked this pull request as ready for review January 3, 2025 08:09

isuruf requested a review from a team as a code owner January 3, 2025 08:09

isuruf requested review from beckermr and jaimergp January 6, 2025 12:43

isuruf added 8 commits January 8, 2025 14:21

Add an option to create arch specific, python version independent pkgs

b336f6e

Fix link_json

b80db8c

Fix multi-output recipes

64a26b2

Add a test

f028f78

Add docs

0c1e9c4

update test

5ffc58d

fix test

3e0bb37

fix test again

fa7b1bb

isuruf force-pushed the python-ver-indep branch from aa23fdd to fa7b1bb Compare January 8, 2025 08:51

isuruf added 2 commits January 8, 2025 16:53

fix windows

32eaa9b

fix docs

1f81f8f

isuruf added 2 commits January 9, 2025 17:51

add news

acce288

Merge branch 'main' into python-ver-indep

1729889

wolfv mentioned this pull request Jan 9, 2025

start implementing cep 20 for abi3 packages prefix-dev/rattler-build#1320

Merged

jezdez reviewed Jan 9, 2025

View reviewed changes

Comment thread docs/source/resources/define-metadata.rst Outdated

jaimergp reviewed Jan 10, 2025

View reviewed changes

isuruf and others added 6 commits January 13, 2025 12:47

use walrus operator

aa72898

Co-authored-by: jaimergp <jaimergp@users.noreply.github.com>

doc improvements

fc6fe30

Co-authored-by: Jannis Leidel <jannis@leidel.info> Co-authored-by: jaimergp <jaimergp@users.noreply.github.com>

Update news

834dadd

Co-authored-by: jaimergp <jaimergp@users.noreply.github.com>

clarify noarch: python

3071b17

clarify more

99fe265

noarch_python

efe1e9c

isuruf requested a review from jaimergp January 14, 2025 13:53

jaimergp approved these changes Jan 14, 2025

View reviewed changes

isuruf merged commit 6e6bb13 into conda:main Feb 25, 2025

github-project-automation bot moved this from ✅ Approved to 🏁 Done in 🔎 Review Feb 25, 2025

isuruf deleted the python-ver-indep branch March 6, 2025 20:08

kenodegard mentioned this pull request Mar 17, 2025

Release 25.3.x #5632

Closed

59 tasks

casperdcl mentioned this pull request Oct 17, 2025

conda: fix noarch => python_version_independent TomographicImaging/CCPi-Regularisation-Toolkit#218

Draft

	if target_file.startswith("bin") and (
	if target_file.startswith("bin/") and (

	return target_file.replace("bin", "python-scripts")
	return target_file.replace("bin/", "python-scripts/")

		# This is a hack to make conda/mamba/micromamba compile the pure python files
		# and for micromamba to move the files in site-packages to the correct dir.

	the package, i.e., for ABI3,
	the package, i.e., for ABI3 in conda-forge

Conversation

isuruf commented Aug 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist - did you ...

Uh oh!

codspeed-hq bot commented Aug 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging #5456 will not alter performance

Summary

Uh oh!

Uh oh!

Uh oh!

beckermr left a comment

Choose a reason for hiding this comment

Uh oh!

jaimergp commented Jan 9, 2025

Uh oh!

isuruf commented Jan 9, 2025

Uh oh!

wolfv commented Jan 9, 2025

Uh oh!

Uh oh!

jaimergp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

jschueller commented Feb 25, 2025

Uh oh!

casperdcl commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

isuruf commented Oct 17, 2025

Uh oh!

casperdcl commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

casperdcl commented Oct 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

isuruf commented Aug 12, 2024 •

edited

Loading

codspeed-hq bot commented Aug 12, 2024 •

edited

Loading

casperdcl commented Oct 17, 2025 •

edited

Loading

casperdcl commented Oct 17, 2025 •

edited

Loading