Function `get_required_dists` hangs when trying to find distributions required by "pytest"

[phamour]

Describe the bug
When using distlib.database.get_required_dists to find out pytest's dependencies, the process hangs for a long time without response.

To Reproduce
Steps to reproduce the behavior:

1. Make and activate a whole new isolated venv
2. Run pip install click==7.1.2 distlib==0.3.4 pytest==6.1.2 toml==0.10.2 twine==3.4.1 wheel==0.36.2
3. Run python -c "from distlib.database import DistributionPath, get_required_dists; dists = list(DistributionPath(include_egg=True).get_distributions()); dist_pytest = next(d for d in dists if d.name.lower() == 'pytest'); get_required_dists(dists, dist_pytest)"
4. Process hangs infinitely

Expected behavior
A list of distributions required by "pytest" is returned within acceptable amount of time.

Environment

• MacOS Big Sur 11.6.5
• distlib 0.3.4

Additional information
I've read the code of the function get_required_dists. The main logic here is to dig into the dependency graph starting from a given node (distribution object here) and collect all adjacent distributions.

But it seems that visited nodes are not clearly recorded to avoid re-visiting, which could stuck this login when there are cycles in the graph (here I did not read the code of making the dependency graph and thus am not sure whether it is truly acyclic as expected).

Another weird behavior which might prove the re-visiting thing is that get_required_dists for "requests" returned a list containing 2 times of "certifi" and 2 time of "idna" (same procedure to reproduce as describe above, except for step 3 run python -c "from distlib.database import DistributionPath, get_required_dists; dists = list(DistributionPath(include_egg=True).get_distributions()); dist_pytest = next(d for d in dists if d.name.lower() == 'requests'); print(get_required_dists(dists, dist_pytest))" instead).

I've then tried to make an explicit record of visited nodes and the problem of hanging was solved and the list of distributions required by "pytest" is returned.

I'm glad to make a PR for this if needed.

[vsajip]

I'm glad to make a PR for this if needed.

Yes, please do!