Output filtered data to a netCDF file #14
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
angus-g
force-pushed
the
netcdf-outfile
branch
2 times, most recently
from
23cc698 to
132e6c2
Compare
angus-g
force-pushed
the
netcdf-outfile
branch
2 times, most recently
from
52fd8e8 to
c8d70a7
Compare
angus-g
added
incomplete
|
While I believe I have the core functionality and some unit tests in place, I don't know whether this is quite ready for a merge. However, it does deserve some real-world testing. One issue I suspect will become more apparent is #27 (warn on existing output), since this will wipe existing filtered output with an experiment of the same name! I've tagged this as |
angus-g
added
testing
Merged
Using the source files, and the variable/dimension dictionaries, a netCDF dataset is created to hold the filtered data. The approach I've taken here is pretty ugly, and makes too many assumptions. Ideally, I'd like to use xarray to do most of the work, but incremental writes to a dataset are one of the unsupported operations.
This is really just the most basic case, and doesn't cover any tricky corner cases.
This adds a few more test cases. In particular, the staggered test case caught missing behaviour when variables have their own separate dimensions (as is the case for staggered variables).
I had been testing wildcard filenames before, and in those cases it seems like parcels expanded the wildcard when creating the FieldSet. However, with two wildcards, this wasn't happening. We manually expand wildcards when querying the input datasets for the metadata needed to create the output file. Closes #34.
This adds a flag to filtering and create_out to control whether an existing output file should be clobbered or not. Closes #27.
angus-g
force-pushed
the
netcdf-outfile
branch
from
a229710 to
6795575
Compare
For example, a data variable might only have dims ("time", "lat",
"lon"), but a "depth" dimension was specified in indices.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Using the source files, and the variable/dimension dictionaries, a netCDF dataset is created to hold the filtered data. The approach I've taken here is pretty ugly, and makes too many assumptions. Ideally, I'd like to use xarray to do most of the work, but incremental writes to a dataset are one of the unsupported operations.
See #1.