Skip to content

ENH: add timezone awareness for formatting iso8601 datetimes to json #41573

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

jeliashi
Copy link

@jeliashi jeliashi commented May 19, 2021

This PR allows for non-UTC printing of datetime strings from the to_json method when passing in local timestamps

ts = pd.Timestamp('2020-10-5 4:00:05', tz='US/Pacific')
print(pd.io.json.dumps(ts, iso_dates=True))
print(ts.isoformat())

would produce
"2020-10-05T11:00:05.000Z"
and
"2020-10-05T04:00:05.000-07:00"
respectively.

Now they both produce
"2020-10-05T04:00:05.000-07:00"

So what do I mean by not complete?
This still doesn't work for arrays that have datetime values. This only works for single datetime values.
So:
pd.DataFrame({'foo': range(5)}, index=pd.date_range('2020-10-12', freq='H', periods=5, tz='US/Eastern')).to_json(date_format='iso')

will still produce:
'{"foo":{"2020-10-12T04:00:00.000Z":0,"2020-10-12T05:00:00.000Z":1,"2020-10-12T06:00:00.000Z":2,"2020-10-12T07:00:00.000Z":3,"2020-10-12T08:00:00.000Z":4}}'

and not
'{"foo":{"2020-10-12T00:00:00.000-04:00":0,"2020-10-12T01:00:00.000-04:00":1,"2020-10-12T02:00:00.000-04:00":2,"2020-10-12T03:00:00.000-04:00":3,"2020-10-12T04:00:00.000-04:00":4}}'

Working on it currently. Open to recommendations!

@mzeitlin11
Copy link
Member

Thanks for the pr @jeliashi! Looks like build failing (maybe from warnings you see locally that are errors with our CI flags) (https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=60052&view=logs&j=b516280a-8c75-541b-8ba0-ae13561718a7&t=2bc93ab4-1a3f-5fe5-4e3f-0d861e20f7c6&l=721)

Also, this looks like a pretty large change - needs testing! That would also be helpful to make clear what this behavior change is

@mzeitlin11 mzeitlin11 added Enhancement IO JSON read_json, to_json, json_normalize Timezones Timezone data dtype labels May 19, 2021
@pep8speaks
Copy link

pep8speaks commented May 22, 2021

Hello @jeliashi! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-05-26 15:40:11 UTC

@jeliashi
Copy link
Author

Thanks for the pr @jeliashi! Looks like build failing (maybe from warnings you see locally that are errors with our CI flags) (https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=60052&view=logs&j=b516280a-8c75-541b-8ba0-ae13561718a7&t=2bc93ab4-1a3f-5fe5-4e3f-0d861e20f7c6&l=721)

Also, this looks like a pretty large change - needs testing! That would also be helpful to make clear what this behavior change is

Hi @mzeitlin11,

I've actually made it bigger now lol. Sorry about that. I'll add testing in a bit to account for the new features this branch provides. This now should compile everywhere! Excited to be contributing. I use pandas almost every work day so it feels good to fix a bug that has been bothering me for a while.

@jeliashi jeliashi closed this May 25, 2021
@jeliashi jeliashi reopened this May 26, 2021
@jeliashi jeliashi force-pushed the jeliashi-fix-make-iso8601-timezone branch from a50af89 to 4f72070 Compare May 26, 2021 15:41
@jeliashi
Copy link
Author

@mzeitlin11 @simonjayhawkins This is ready for review :)

@github-actions
Copy link
Contributor

This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.

@github-actions github-actions bot added the Stale label Jun 26, 2021
@simonjayhawkins
Copy link
Member

@mzeitlin11 @simonjayhawkins This is ready for review :)

ci is failing. can you fix-up to pass and move release note to 1.4

@mroeschke
Copy link
Member

Thanks for the PR, but it appears this PR has gone stale. Closing due to inactivity but please let us know if you're still interested in working on this and we can reopen.

@mroeschke mroeschke closed this Aug 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement IO JSON read_json, to_json, json_normalize Stale Timezones Timezone data dtype
Projects
None yet
Development

Successfully merging this pull request may close these issues.

to_json converts to UTC when encoding ISO formatted datetimes
5 participants