Skip to content

BUG: Bug fix implement __reduce__/__setstate__ for Period pickle support #10441

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 17 commits into from
Closed

BUG: Bug fix implement __reduce__/__setstate__ for Period pickle support #10441

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
f9c517f
implement __reduce__/__setstate__ for Period pickle support
Jun 25, 2015
f55d3dc
create test for period class with pickle round trip method
Jun 25, 2015
81f76f2
move to tseries tests
Jun 25, 2015
22d9ad5
move def test_round_trip_pickle to tseries.test.test_period.py
Jun 25, 2015
4f7b514
changed to test_round_trip
Jun 26, 2015
c8e8a10
Squashed commit of the following:
Aug 5, 2015
278930e
Merge branch 'master' of https://github.com/scarrucciu/pandas
Aug 5, 2015
c2d0e87
Merge branch 'master' of https://github.com/pydata/pandas
Aug 20, 2015
95f2db4
add _create_period to generate_legacy_storage_files
Aug 20, 2015
235570d
use self.round_trip_pickle to test in test_period
Aug 20, 2015
f64f746
10441 Use self.round_trip_pickle to test in test_period
Aug 20, 2015
a4cc715
10441 add period series to create_data in generate_legacy_data
Aug 20, 2015
2c25c21
Merge branch 'master' of https://github.com/scarrucciu/pandas
Aug 20, 2015
047839c
10441 implement __reduce__/__setstate__ for Period pickle support
Aug 20, 2015
608a28e
Merge branch 'master' of https://github.com/scarrucciu/pandas
Aug 20, 2015
f3aa671
Squashed commit of the following:
Aug 20, 2015
d1739d4
Merge branch 'master' of https://github.com/scarrucciu/pandas
Aug 20, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions pandas/src/period.pyx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -969,6 +969,14 @@ cdef class Period(object):
value = ("%s" % formatted)
return value

def __setstate__(self, state):
self.freq=state[1]
self.ordinal=state[2]

def __reduce__(self):
object_state = None, self.freq, self.ordinal
return (Period, object_state)

def strftime(self, fmt):
"""
Returns the string representation of the :class:`Period`, depending
Expand Down
13 changes: 13 additions & 0 deletions pandas/tseries/tests/test_period.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@

"""

import pickle
import os

from datetime import datetime, date, timedelta

from numpy.ma.testutils import assert_equal
Expand Down Expand Up @@ -2537,6 +2540,16 @@ def test_searchsorted(self):
ValueError, 'Different period frequency: H',
lambda: pidx.searchsorted(pd.Period('2014-01-01', freq='H')))

def test_round_trip(self):

p = Period('2000Q1')
pickle_path = os.path.join(tm.get_data_path(),
'period.pickle')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pls use self.round_trip_pickle to test


with open(pickle_path, 'wb') as f: pickle.dump(p, f)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use test_round_trip

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pls use test_round_trip

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean result = self.round_trip_pickle(..)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also need to add a period in pandas/io/tests/generate_legacy_data.py

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you mean in pandas/io/tests/generate_legacy_pickles?

On Wed, Aug 5, 2015 at 6:00 PM, Jeff Reback [email protected]
wrote:

In pandas/tseries/tests/test_period.py
#10441 (comment):

@@ -2537,6 +2540,16 @@ def test_searchsorted(self):
ValueError, 'Different period frequency: H',
lambda: pidx.searchsorted(pd.Period('2014-01-01', freq='H')))

  • def test_round_trip_pickle(self):
  •    p = Period('2000Q1')

  •    pickle_path = os.path.join(tm.get_data_path(),

  •                               'period.pickle')

  •    with open(pickle_path, 'wb') as f: pickle.dump(p, f)

also need to add a period in pandas/io/tests/generate_legacy_data.py

—
Reply to this email directly or view it on GitHub
https://github.com/pydata/pandas/pull/10441/files#r36359951.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/pydata/pandas/blob/master/pandas/io/tests/generate_legacy_storage_files.py

u may need to rebase on master
this was recently changed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generate_legacy_storage_files needs to be updated with a Period

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry a bit new with this, what is it you need me to do?

Thanks.

On Thu, Aug 20, 2015 at 9:17 AM, Jeff Reback [email protected]
wrote:

In pandas/tseries/tests/test_period.py
#10441 (comment):

@@ -2537,6 +2540,16 @@ def test_searchsorted(self):
ValueError, 'Different period frequency: H',
lambda: pidx.searchsorted(pd.Period('2014-01-01', freq='H')))

  • def test_round_trip_pickle(self):
  •    p = Period('2000Q1')

  •    pickle_path = os.path.join(tm.get_data_path(),

  •                               'period.pickle')

  •    with open(pickle_path, 'wb') as f: pickle.dump(p, f)

generate_legacy_storage_files needs to be updated with a Period

—
Reply to this email directly or view it on GitHub
https://github.com/pydata/pandas/pull/10441/files#r37527106.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 things

  • use self.round_trip_pickle to test the pickling in test_period instead of the ad-hoc code above.
  • fix generate_legacy_storeage_files.py to test Period (and Timestamp) as singular objects. If you are comfortable with this, gr8, if not just do the above and i'll fix the rest.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, used the self.round_trip_pickle and add a period series to the
create_data in generate_legacy. Please let me know if this is alright and
if there is anything I need to do.

On Thu, Aug 20, 2015 at 10:03 AM, Jeff Reback [email protected]
wrote:

In pandas/tseries/tests/test_period.py
#10441 (comment):

@@ -2537,6 +2540,16 @@ def test_searchsorted(self):
ValueError, 'Different period frequency: H',
lambda: pidx.searchsorted(pd.Period('2014-01-01', freq='H')))

  • def test_round_trip_pickle(self):
  •    p = Period('2000Q1')

  •    pickle_path = os.path.join(tm.get_data_path(),

  •                               'period.pickle')

  •    with open(pickle_path, 'wb') as f: pickle.dump(p, f)

2 things

  • use self.round_trip_pickle to test the pickling in test_period
    instead of the ad-hoc code above.
  • fix generate_legacy_storeage_files.py to test Period (and Timestamp)
    as singular objects. If you are comfortable with this, gr8, if not just do
    the above and i'll fix the rest.

—
Reply to this email directly or view it on GitHub
https://github.com/pydata/pandas/pull/10441/files#r37532256.


self.assertEqual(p, pd.read_pickle(pickle_path))

def _permute(obj):
return obj.take(np.random.permutation(len(obj)))

Expand Down