The benchmarks module is being shipped with the tarball.

[ocefpaf]

If that is by design please close this issue but if not we need to update the exclude list in:

pystac/setup.py

Line 22 in 9940a12

packages=find_packages(exclude=["tests*"]),