Very frequent segfaults with the new `netCDF4=1.6.1`

[valeriupredoi]

Heads up guys, we are seeing some very frequent segfaults in our CI when we have the new, hours-old, netCDF4=1.6.1 in our environment. It's most probably due to it, since HDF5 has been at 1.12.2 for a while now - more than a month, and with netCDF4=1.6.0 all works fine (and other packages staying at the same version and hash point). Apologies if this proves out to be due to a different package, but better safe than sorry in terms of a forewarning. Cheers muchly 🍺

[ocefpaf]

Looks like you are using conda-forge's netcdf4. Maybe open an issue at https://github.com/conda-forge/netcdf4-feedstock instead.

PS: could you also test the wheels just to be sure they are OK?

[valeriupredoi]

@ocefpaf good call, mate! Will do so, cheers 🍺

[neutrinoceros]

We're also having issues on yt with the windows wheels for version 1.6.1. Namely, h5py is raising a warning at import. See yt-project/yt#4128

[trexfeathers]

Same with us - see SciTools/iris#4968. Sometimes manifests as segfaults, sometimes as crashed GHA workers (maybe segfault underneath).

@ocefpaf I've confirmed the same problems appear when installing from PyPI OR from conda-forge.

[valeriupredoi]

@ocefpaf I've confirmed the same problems appear when installing from PyPI OR from conda-forge.

Many thanks, I was about to test the PyPi version - cheers for testing, that saves me some lunch time 😁

[ocefpaf]

@trexfeathers what platforms are failing when you tested the PyPI wheels? I'm particularly interested in the Windows wheels for 1.6.1 b/c those are built in a different way now.

[trexfeathers]

@trexfeathers what platforms are failing when you tested the PyPI wheels? I'm particularly interested in the Windows wheels for 1.6.1 b/c those are built in a different way now.

GHA's ubuntu-latest. Despite several attempts we are yet to get Iris' test suite working on Windows.

[neutrinoceros]

@ocefpaf to clarify, on yt we're testing with PyPI wheels for all three major platforms, and we're only seeing issues on windows.

[valeriupredoi]

ah I realized I've not clarified it myself: segfaults from a conda-forge install on both ubuntu-latest, OSX-latest on GHA, and ubuntu on CircleCI, off conda-forge as well, no Win testing for us since we've not been able to get a working install of our packages there either, snif but not snif 😁

[valeriupredoi]

So here's a pretty interesting case study that may lead to fixing this current issue - and a rather seldomly recurring issue @agstephens and myself have noticed in the past, with older (and stable, bullet-proof versions of netCDF4):

• our tests fail with a multitude of HDF-related SegFaults, complaints about close dataset etc, anyway:
• I tried recreating the netcdf sample data that the tests fail on, but I couldn't recreate it since the same segfaults crept up while trying to create them, so...
• I simply moved them out and back in the location where they should be, and...
• no more Segfaults (yes I ran quite a few iterations of the test so I am 100% sure they don't fail)

My colleague Ag noticed the same behaviour, way back, on very very few occasions - an HDF Segfault on a certain file would automagically disappear if we moved the file out and back in its location, we blamed it on the FS back then, but thinking in restrospect, it could be the same issue here?

[ocefpaf]

and we're only seeing issues on windows.

@jswhit it may be prudent to yank those wheels until we figure out what is going on. They do pass the tests in the repo but are not holding well the "production test" :-/

However, the other platforms are failing in other CIs so this is quite confusing and we'll need the reports here to help us sort this out.

---

Edit: @neutrinoceros your report upstream is about h5py and not netcdf4, right? xref: yt-project/yt#4128