Description
In pypa/packaging-problems#342 (comment), it's been suggested that Fromager can solve the bootstrapping problem because it allows for prepare_build to pull built wheel artifacts. It's unclear to me, however, how Fromager breaks the cyclic dependency problem. Stated simply, the bootstrapping problem is that if a build tool depends on itself or any package that requires that build tool to build, there's no way in general to build entirely from source. There's no order in which all packages can be built purely from source.
A few examples:
- setuptools depends on importlib_metadata, which depends on setuptools to build.
- coherent.build depends on coherent.build to build.
- hatchling depends on several projects, any of which could adopt hatchling.
Currently, none of these scenarios break the build-from-source world because:
- Setuptools doesn't declare its dependencies and is forced to vendor them.
- coherent.build isn't popular enough that any system integrators yet care.
- hatchling hasn't adopted any dependencies that have adopted hatchling.
These assumptions are untenable because they impose undue constraints on build tools and their dependencies.
In https://hackmd.io/@jaraco/SJSQ40tv0, I'm proposing a methodology that system integrators would need to adopt to break the cycle and allow build tools to declare dependencies. The tl;dr is that the integrator needs to provide pre-built artifacts for all supported backends (including their dependencies) and it can't expect to build those from source.
Reading the bootstrapping mode, it seems that fromager only has support for building when there are no cyclic dependencies.
all dependencies are being built in the correct order.
What does Fromager do when a package depends on itself, directly or recursively, at build time? Does Fromager allow the private wheel index to be pre-seeded with "trusted" pre-built artifacts that can break the cycle?
I'm going to try some experiments to verify, but I'm interested in this project's maintainers' insights.