Skip to content

PV LIB - ECMWF - functionality is not compatible with non-global ERA5 files #817

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
shall-resurety opened this issue Nov 13, 2019 · 4 comments

Comments

@shall-resurety
Copy link

Description/Details
When working with ERA5 netCDF files which are not 'global' (i.e. lat -90 to +90 and lon -180 to +180) the pvlib function get_nearest_indices returns incorrect values due to an incorrect derivation of the grid size intervals. The appropriate grid size inside get_nearest_indices the derived ERA5 grid spacing does not correlate with the actual 0.25 degree grid spacing which is represented in all ERA5 netCDF files (geo-subset or otherwise). Ultimately this results in data being returned for the wrong lat and lon position based on a misaligned file index. This is something which could be resolved by ensuring the standard 0.25 degree value is used for the delta_lat and delta_lon values when instantiating a ECMWF object from a geo-subset netCDF file. Thank you.

*** Perhaps this should be a feature request instead but logging as a bug due to the incorrect values being returned 'silently'.

To Reproduce
*Assume you have a geo-subset (CONUS) ERA5 netCDF file path stored in the variable filePath

  met = pvlib.iotools.read_ecmwf_macc(filename  = filePath,
                                      latitude  = 34,
                                      longitude = -101.25)

Expected behavior
The ECMWF object has a lat, lon delta defined as 1.8 versus the correct 0.25 which results with incorrect row indices being used in the met data return and therefore not representing the requested lat and lon position which is input.

@mikofski
Copy link
Member

Actually it was only meant for the CAMS: MACC Reanalysis (wiki).

Maybe I should update the read_ecmwf_macc documentation to be more clear about that.

@shall-resurety , @cwhanse , @wholmgren do you all think we should make a new iotool for ERA5 or can we adapt this one to be more flexible for different grids?

I've been eager to add a MERRA2 & CAMS reader to provide global data coverage. CAMS is very fast, and it's easy to use, but lacks temperature.

@shall-resurety
Copy link
Author

Thanks for the fast reply. Probably should have been obvious based on the name but it did seem to work on the surface i.e. ran without error so it was assumed to be working as expected despite the mismatch. That being said I think an ERA5 adaptation would be really useful/worth while.

@shall-resurety
Copy link
Author

I am going to mark this 'closed' seeing as it is not a bug but there could be a follow on 'feature request' in a separate ticket.

@cwhanse
Copy link
Member

cwhanse commented Nov 14, 2019

Thanks again @shall-resurety for bringing up the issue. Please feel free to open another issue as a feature request. It doesn't commit you to work on it, but of course any contribution is welcome.

Jacc0027 added a commit to Jacc0027/pvlib-python that referenced this issue May 17, 2023
Trying to solve pvlib#814 and pvlib#817 issues.
kandersolar added a commit that referenced this issue Jun 23, 2023
* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Merge branch 'Spectral-corrections' of https://github.com/Jacc0027/pvlib-python into Spectral-corrections

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update pvlib/atmosphere.py

Co-authored-by: Kevin Anderson <[email protected]>

* relocation of parameter descriptions according to the order of input parameters

* Update api.rst

Adding atmosphere.AM_AOD_PW_spectral correction as requested in #1296

* remove input screening

* move reference values to be optional parameters

* fix implementation issues

* first cut at tests using file from Jacc0027

* CI correction. Line #215

* Update test_atmosphere.py

* Update test_atmosphere.py

* Update test_atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Testing tests

* Update test_atmosphere.py

* Update test_atmosphere.py

* Update test_atmosphere.py

* Update test_atmosphere.py

* Update test_atmosphere.py

* Update atmosphere.py

* Test Review

* Update atmosphere.py

* Update test_atmosphere.py

* Update test_atmosphere.py

* Update test_atmosphere.py

* Update test_atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

* Update test_atmosphere.py

* Update atmosphere.py

* Update test_atmosphere.py

* Update test_atmosphere.py

* Update test_atmosphere.py

* Update test_atmosphere.py

* Update test_atmosphere.py

* Update pvlib/atmosphere.py

Co-authored-by: Cliff Hansen <[email protected]>

* Update atmosphere.py

Added the requested changes

* Update atmosphere.py

Fixed 2 blank line errors.

* Update test_atmosphere.py

Updated test function name to: caballero_spectral_correction

* Update api.rst

Updated the atmosphere function name from AM_AOD_PW_spectral_correction to caballero_spectral_correction

* Update test_atmosphere.py

Updated some functions calls from AM_AOD_PW_spectral_correction to caballero_spectral_correction

* Update atmosphere.py

Updated the logic to show errors.

* Update pvlib/atmosphere.py

Co-authored-by: Cliff Hansen <[email protected]>

* Update pvlib/atmosphere.py

Co-authored-by: Cliff Hansen <[email protected]>

* Update pvlib/atmosphere.py

Co-authored-by: Cliff Hansen <[email protected]>

* Update pvlib/atmosphere.py

Co-authored-by: Cliff Hansen <[email protected]>

* Update pvlib/atmosphere.py

Co-authored-by: Cliff Hansen <[email protected]>

* Update pvlib/atmosphere.py

Co-authored-by: Cliff Hansen <[email protected]>

* Update atmosphere.py

* Update atmosphere.py

* Update atmosphere.py

Trying to solve #814 and #817 issues.

* Update atmosphere.py

* Update api.rst

* Delete api.rst

* Update test_atmosphere.py

* Update atmosphere.py

* Update test_atmosphere.py

* Update atmosphere.py

* Update test_atmosphere.py

* Update test_atmosphere.py

* Update test_atmosphere.py

* Update test_atmosphere.py

* Update test_atmosphere.py

* Update test_atmosphere.py

* update function logic and get tests working

* Update atmosphere.py

Updated both "TODO" issues.

* Update atmosphere.py

* Update v0.9.6.rst

Added caballero spectral correction as a new function.

* Update airmass_atmospheric.rst

Added a new function: atmosphere.caballero_spectral_correction

* Update pvlib/atmosphere.py

Co-authored-by: Anton Driesse <[email protected]>

* rename function and move to pvlib.spectrum

following the conventions set out in #1628

* misc cleanup

- reorder and rename parameters to match #1768
- remove reference parameters
- shorten/simplify docstring
- restructure calculation to be more readable

---------

Co-authored-by: Kevin Anderson <[email protected]>
Co-authored-by: Kevin Anderson <[email protected]>
Co-authored-by: Cliff Hansen <[email protected]>
Co-authored-by: Anton Driesse <[email protected]>
Co-authored-by: Kevin Anderson <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants