A way to get complete PyPi index in machine-readable form

[AMDmi3]

Hi!

I'm author of repology.org which tracks package versions in 100+ repositories. It supported PyPi as well, which is very useful as it may point to the newerver version of any python package as sonn as it's updated on PyPi, but this has broken because index of all pypi packages is no longer available from https://pypi.python.org/pypi/. https://pypi.python.org/simple is not suitable either, as I absolutely need package versions (and preferably other metadata such as summary and license) in addition to packages names, and it would be to slow to download page for each package (and will also generate extra server load).

I was directed here from #python, so I ask for a simple way to get a machine readable index of all PyPi packages. It doesn't need to be realtime, at least for repology hourly/daily generated file would be quite sufficient. Machine-readable format such as json is also highly preferred, e.g. something like this.

[
    {
        "name": "requests",
        "version": "2.18.1",
        "summary": "Python HTTP for Humans."
    }
    ...
]

[di]

Hi @AMDmi3,

There is already a per-project JSON endpoint, e.g. https://pypi.org/pypi/requests/json.

There is also an RSS feed which gives every new release of any project: https://pypi.org/rss/updates.xml.

It seem to me like these would mostly cover your use case, do you agree?

[AMDmi3]

Unfortunately, no. Info on all packages is needed, and per-project json is too long to fetch and, as mentioned, would generate unneeded server load.

[dstufft]

Hi! Thanks for filing this issue, it is close enough in scope to another issue, that I'm going to close this as a duplicate of #347. Ideally please comment on that issue with the relevant items you would need from such an API.