Fix support for pyproject.toml dependencies with multiple selectors#597
Conversation
We want to prioritize toml deps over setup deps. We don't want to remove redundant packages in toml deps because these can have corresponding, different selectors.
We want to prioritize sdist deps over pypi deps. We don't want to remove redundant packages in sdist deps because these can have corresponding, different selectors. The progressbar has become a bit more complicated because we iterate separately over sdist and pypi dependencies.
While `is_arch` gets set to true if there are extras, we also want to do so if selectors have already been defined.
When removing duplicate packages, we want to consider different selectors as unique. Thus, we only remove dependencies if they have the same package name *and* the same selector.
Previously, selectors were getting dropped as tags and comments
These tests are designed to make sure multiple selectors for the same package are preserved correctly at every stage of the process of merging dependencies from different sources. When parsing poetry dependencies, we introduce selectors early in the process, whereas for other approaches, we retain "extras" (separated by semicolons) for much of the merging process. Packages employing each strategy are used in the added tests.
for more information, see https://pre-commit.ci
|
@marcelotrevisani, as always, I would very much appreciate your input and review. @lorepirri, this work builds on yours in #592. I would very much appreciate your feedback as well. I have put several days into learning my way around |
marcelotrevisani
left a comment
marcelotrevisani
left a comment
There was a problem hiding this comment.
LGTM! Thanks again! :)
the modification looks sensible to me, I believe we can integrate it, thanks! Don't worry, I am not taking that as a lack of interest at all, I am very glad to the insights and inputs you are bringing to gray skull! |
|
@marcelotrevisani, wonderful, thanks so much. I'd like to follow up at some point when I have time with some updates to |
2 participants
Description
It is common for the same dependency to have multiple selectors, like the following:
This merge attempts to fully support this capability.
In several places in the code where dependencies are merged, care is now taken to allow multiple versions of the same package as long as they come from the same source. This happens when merging
tomlandsetupdependencies intosdistdependencies, and again when mergingsdistdependencies withpypidependencies. It also takes place when removing redundancies later in the process of building a recipe.In addition, because some strategies (namely parsing of
poetrydependencies) add selectors early in the process, care has been taken not to drop selectors that may mistakenly be identified as tags or comments.Finally, a fix has been added to identify that a package
is_archwhen it has selectors, just as it is identified asis_archwhen "extras" are detected.Four new pypi tests have been added and new testing has been added to the existing
test_run_requirements_sdisttest to ensure that multiple selectors for the same package are handled correctly.