Make tables a required dependency

[kandersolar]

• Closes make tables a required dependency #1286
• I am familiar with the contributing guidelines
• Tests added
• Updates entries to docs/sphinx/source/api.rst for API changes.
• Adds description and name entries in the appropriate "what's new" file in docs/sphinx/source/whatsnew for all changes. Includes link to the GitHub Issue with :issue:`num` or this Pull Request with :pull:`num`. Includes contributor name and/or GitHub username (link with :ghuser:`user`).
• New code is fully documented. Includes numpydoc compliant docstrings, examples, and comments where necessary.
• Pull request is nearly complete and ready for detailed review.
• Maintainer: Appropriate GitHub Labels and Milestone are assigned to the Pull Request and linked Issue.
• remove try/except in clearsky.lookup_linke_turbidity

[cwhanse]

didn't expect such a fast response, thank you!

[kandersolar]

Python 3.9 wheels aren't available on pypi for Windows & OSX (PyTables/PyTables#823), though there are some other failures that don't seem related to that. I will take a closer look later today.

[kandersolar]

can we move tables off the optional list

So long as the 3.9 wheels are missing on pypi, I think this would make pip install pvlib fail on python 3.9 for Windows and OS X unless the user runs conda install tables first or is able to compile tables and its dependencies from source. This will also prevent the bare_macOS Python39 tests from working until pypi has 3.9 wheels, as building from source on the CI is surely a can of worms we don't want to open. At least one other project skips tests for py3.9 on Win/OS X for this reason: https://github.com/pylhc/omc3/blob/master/.github/workflows/tests.yml#L22-L26

I propose putting off making tables a required dependency until wheels become available on pypi. At that point I think the only remaining issue will be addressing whatever is wrong with 3.6-min, which I haven't really looked at yet.

[kandersolar]

It occurred to me that we could install tables by default on all combinations of (OS, python) except (OS X, 3.9). So (OSX, 3.9) is no worse off than before, but everyone else is better off. What do people think?

[wholmgren]

I'm -1 on different requirements for different platforms.

I'm confused about the status given the comments in the conda files. I was going to ask if we can simply switch to conda environments for mac OS tests. But the comments suggest that creates a problem for windows. And have the windows runs not included the tests that require tables? I couldn't tell as the output of the CI is nearly impossible to parse right now. Would switching to the windows-2019 azure vm help?

On a related note, azure is complaining that ubuntu-16.04 is deprecated and will be removed in a month.

vm docs: https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops&tabs=yaml

[kandersolar]

I'm -1 on different requirements for different platforms.

Ok. I think in that case our only options are (1) keeping tables as optional for now or (2) having pip install pvlib fail on OS X in a clean python 3.9 environment. @cwhanse seems unenthusiastic about (1), but to me it seems better than (2).

I'm confused about the status given the comments in the conda files.

Me too, I was going to take another look at that if it was decided that different requirements for different platforms was acceptable.

On a related note, azure is complaining that ubuntu-16.04 is deprecated and will be removed in a month.

Maybe after 0.9 I will find some time to migrate to github actions...

[wholmgren]

having pip install pvlib fail on OS X in a clean python 3.9 environment

While I don't claim to represent all mac users, this is fine with me. It's only in the last year or two that you could easily pip install pvlib on mac os anyways.

[cwhanse]

I think in that case our only options are (1) keeping tables as optional for now or (2) having pip install pvlib fail on OS X in a clean python 3.9 environment.

If (1) is needed to get v0.9 released soon, I vote for the release and taking this up for v0.9.1.

[wholmgren]

Better to add new requirements in 0.x rather than 0.x.y releases, but I'm not strictly opposed to it.

[kandersolar]

Ok I assumed the mac os py 3.9 issue would be a dealbreaker. I will try this again.

[kandersolar]

Ready for review. I could not figure out a way to disable the OS X / 3.9 job without just making a duplicate of posix.yml and removing 3.9; maybe someone better at azure than I am knows how to do that.

The new pins for blosc and numexpr have to do with DLL bundling and numpy ABI incompatibility... not sure I can explain much beyond that. For blosc I found this: conda-forge/pytables-feedstock#31 (comment) and for numexpr I looked at what failed, figured out release dates, and pinned a version that matched.

[wholmgren]

I could not figure out a way to disable the OS X / 3.9 job without just making a duplicate of posix.yml and removing 3.9; maybe someone better at azure than I am knows how to do that.

Not worth doing more when we want to switch to github actions!

[wholmgren]

hearing no objections... thanks @kanderso-nrel