Behavior of pvlib.solarposition.get_sun_rise_set_transit

[cwhanse]

Why does this function return a sunset time >24 hours ahead of the specified time?

E.g.,

import pandas as pd
import pvlib

dr = pd.DatetimeIndex(start=datetime(2017, 2, 12, 0, 0, 0, tzinfo=pytz.UTC), end=datetime(2017, 2, 12, 0, 15, 0, tzinfo=pytz.UTC), freq='15T')
tmp = pvlib.solarposition.get_sun_rise_set_transit(dr, 35.05, -115)
tmp['sunset']

returns

2017-02-12 00:00:00+00:00   2017-02-13 01:19:07.319358208+00:00
2017-02-12 00:15:00+00:00   2017-02-13 01:19:07.319358208+00:00
Freq: 15T, Name: sunset, dtype: datetime64[ns, UTC]

The comments in get_sun_rise_set_transit don't describe what the output should be.

[wholmgren]

I'm not very familiar with this function. I think @alorenzo175 contributed it, so maybe he can help you.

[adriesse]

Only the date part of your time stamps is used to identify the day in question at your location. This day spans the period 2017-02-12 07:00:00 UTC to 2017-02-13 07:00:00, and your sunset is within this period.

Probably not the ideal behaviour for this function.

[alorenzo175]

This is less than ideal, but makes sense when you consider the entire tmp dataframe. Like @adriesse said, only the date part of the time stamp is used, so the algorithm calculates the sunrise, then transit, then sunset for that day (2017-02-12). Sunset should always occur after sunrise (2017-02-12 14:30) in the algorithm, hence the > 24 hour sunset time.

This should at least be better documented. The sunrise, sunset, and transit could also be split into separate functions where it would then make sense to always return a result on the input day.

[cwhanse]

I can follow the logic now, but the function doesn't make sense, to me. The function returns the sunrise time based on the date alone, so the returned sunrise which can be before or after the specified time. E.g.,

dr = pd.DatetimeIndex(start=datetime(2017, 2, 12, 0, 0, 0, tzinfo=pytz.UTC), end=datetime(2017, 2, 12, 23, 45, 0, tzinfo=pytz.UTC), freq='15T') tmp = pvlib.solarposition.get_sun_rise_set_transit(dr, 35.05, -115)

returns for sunrise both
2017-02-12 00:00:00+00:00 2017-02-12 14:28:43.514198272+00:00
and
2017-02-12 17:00:00+00:00 2017-02-12 14:28:43.514198272+00:00
For sunset, the function returns the next sunset time after the sunrise time associated with the timestamp, e.g.,
2017-02-12 00:00:00+00:00 2017-02-13 01:19:07.319358208+00:00
and
2017-02-12 17:00:00+00:00 2017-02-13 01:19:07.319358208+00:00

The non-intuitive result is that the timestamp 2017-02-12 00:00:00+00:00 is daylight at the location of interest, so there's a sunset at 2017-02-12 01:19(ish)+00:00 which isn't returned.

What is the intended use? In my application I have a list of times and I need all sunrises and sunsets which occur between the beginning and end of the list. This function doesn't meet that need.

[wholmgren]

The timezone behavior also seems like a potential source of error:

In [3]: get_sun_rise_set_transit(pd.Timestamp('20170212 000000', tz='UTC'), 35.05, -115)
Out[3]:
                                                      sunrise  \
2017-02-12 00:00:00+00:00 2017-02-12 14:28:43.514198272+00:00

                                                       sunset  \
2017-02-12 00:00:00+00:00 2017-02-13 01:19:07.319358208+00:00

                                                      transit
2017-02-12 00:00:00+00:00 2017-02-12 19:54:11.281640960+00:00

In [4]: get_sun_rise_set_transit(pd.Timestamp('20170212 000000', tz='UTC').tz_convert('Etc/GMT+7'), 35.05, -115)
Out[4]:
                                                      sunrise  \
2017-02-11 17:00:00-07:00 2017-02-11 07:29:43.966113536-07:00

                                                       sunset  \
2017-02-11 17:00:00-07:00 2017-02-11 18:18:08.224896768-07:00

                                                      transit
2017-02-11 17:00:00-07:00 2017-02-11 12:54:12.247179776-07:00

Seems fixable by passing the full datetime input, rather than the so-called utcday, to the spa.transit_sunrise_sunset function, and using zenith/azimuth to determine the local date. That's the behavior that we want, right?

pyephem has some functions for this, too.

[mikofski]

FYI: when I call the compiled SPA C-files (as of v0.6.0, I do get the expected behavior:

from pvlib.spa_c_files.spa_py import spa_calc

result = spa_calc(
    year=2018, month=1, day=1, hour=0, minute=0, second=0,
    time_zone=-7, longitude=-105.178, latitude=39.743,
    elevation=1830.14, pressure=820, temperature=11, delta_t=67)

print(result['sunset'], result['sunrise'])  # SPA hours
#16.785243982117017 7.364247922706669

print(timedelta(hours=16.785242) +
    pytz.timezone('Etc/GMT+7').localize(datetime(2018, 1, 1, 0, 0, 0)))
#2018-01-01 16:47:06.871200-07:00

print(timedelta(hours=7.364246) +
    pytz.timezone('Etc/GMT+7').localize(datetime(2018, 1, 1, 0, 0, 0)))
#2018-01-01 07:21:51.285600-07:00

[wholmgren]

@mikofski the pvlib.solarposition.get_sun_rise_set_transit function that Cliff is referring to does not use the NREL SPA c code. It uses the python implementation of it (pvlib.spa) plus a wrapper (pvlib.spa.transit_sunrise_sunset)

[cwhanse]

I think I understand the cause for the odd behavior. This line:
utcday = pd.DatetimeIndex(time.date).tz_localize('UTC')
extracts the date from time to create a new datetime index, then localizes to UTC. As a result the timezone information in time is lost. The timezone matters, because time.date depends on the timezone. For example,

dt = pd.DatetimeIndex(['2018-09-20 14:00:00', '2018-09-20 20:00:00']).tz_localize('MST')
dt2 = dt.tz_convert('UTC')

dt and dt2 are the same times, referenced to different time zones (local and UTC). However,

print(dt.date)
print(dt2.date)

produces different dates:

[datetime.date(2018, 9, 20) datetime.date(2018, 9, 20)]
[datetime.date(2018, 9, 20) datetime.date(2018, 9, 21)]

which, if passed through to the sunrise calculation, returns sunrise times on different dates:

from pvlib.solarposition import get_sun_rise_set_transit

result = get_sun_rise_set_transit(dt, latitude=35, longitude=-105)
result2 = get_sun_rise_set_transit(dt2, latitude=35, longitude=-105)

print(result['sunrise'])
print(result2['sunrise'])

2018-09-20 14:00:00-07:00   2018-09-20 05:46:34.717640704-07:00
2018-09-20 20:00:00-07:00   2018-09-20 05:46:34.717640704-07:00
Name: sunrise, dtype: datetime64[ns, MST]
2018-09-20 21:00:00+00:00   2018-09-20 12:46:34.717640704+00:00
2018-09-21 03:00:00+00:00   2018-09-21 12:47:18.737046272+00:00
Name: sunrise, dtype: datetime64[ns, UTC]

[wholmgren]

Does this work?

unixtime = np.array(times.tz_convert('UTC').floor('1D').astype(np.int64)/10**9)

We'd need to bump the minimum pandas version to use floor but I am guessing it is faster than going to the python date.

[cwhanse]

Does this work?
unixtime = np.array(times.tz_convert('UTC').floor('1D').astype(np.int64)/10**9)

I don't think that helps, it's the same behavior - references the date to midnight UTC, rather than the date at the location/time of interest. At 6pm tonight (9/20/2018) where I am, UTC has tomorrow's date, if that makes sense.

[wholmgren]

Sorry, I transposed the tz_convert and the floor. I thought it would be different than the currently implemented code because it's not dropping tz awareness when calculating the date. Didn't test though so sorry if I'm just spamming.

[cwhanse]

The desired behavior of get_sun_rise_set_transit is date and location in, sunrise/sunset out. The current function does not have this desired behavior because of the timezone issues described above; even when it's time input is localized, the date can be shifted one day ahead or behind when converted to UTC for spa, returning sunrise/sunset times on dates different than expected.

I'll prepare a pull request to modify how get_sun_rise_set_transit converts from its time input to the date that accounts for the conversion to UTC. However, it will require that the time input to get_sun_rise_set_transit be referenced to the timezone at the input location, otherwise the date being requested is ambiguous.

I'm posting this comment to ask: is requiring the time input to be localized to location's timezone too restrictive?

[wholmgren]

is requiring the time input to be localized to location's timezone too restrictive?

Ok with me.

[cwhanse]

Closed by #603