Skip to content

Panel shift revert #6974

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Apr 28, 2014
Merged

Panel shift revert #6974

merged 2 commits into from
Apr 28, 2014

Conversation

dalejung
Copy link
Contributor

Reverts #6605 closes #6959 #6826

vs 13.1

-------------------------------------------------------------------------------
Test name                                    | head[ms] | base[ms] |  ratio   |
-------------------------------------------------------------------------------
panel_shift                                  |   0.0773 |   0.0990 |   0.7809 |
-------------------------------------------------------------------------------
Test name                                    | head[ms] | base[ms] |  ratio   |
-------------------------------------------------------------------------------

Ratio < 1.0 means the target commit is faster then the baseline.
Seed used: 1234

Target [3d0099a] : BUG: Change Panel.shift to use slice_shift #6605 #6826
Base   [d10a658] : RLS: set released to True. v0.13.1

vs master

Note that the `pct_change is slower due to the deferred alignment.

-------------------------------------------------------------------------------
Test name                                    | head[ms] | base[ms] |  ratio   |
-------------------------------------------------------------------------------
panel_shift                                  |   0.1650 | 522.0180 |   0.0003 |
panel_pct_change_major                       | 7615.9450 | 6213.1660 |   1.2258 |
panel_pct_change_minor                       | 7864.9763 | 5877.1353 |   1.3382 |
panel_pct_change_items                       | 8646.0753 | 5884.1163 |   1.4694 |
-------------------------------------------------------------------------------
Test name                                    | head[ms] | base[ms] |  ratio   |
-------------------------------------------------------------------------------

Ratio < 1.0 means the target commit is faster then the baseline.
Seed used: 1234

Target [3ec426f] : BUG: Change Panel.shift to use slice_shift #6605 #6826
Base   [0d2966f] : COMPAT:  32-bit platform compat for Panel.count

@jreback
Copy link
Contributor

jreback commented Apr 27, 2014

IIRC pct_change is doing this sectino by section (filling forward) and that is the issue here.

@TomAugspurger what do you think?

as an aside let's make the vbench shorter so it takes at most 1000ms

@jreback jreback added this to the 0.14.0 milestone Apr 27, 2014
@dalejung
Copy link
Contributor Author

Anyone else get random travis failings on asserts that should pass?

======================================================================
FAIL: test_sum (pandas.tests.test_frame.TestDataFrame)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/virtualenv/python2.6_with_system_site_packages/lib/python2.6/site-packages/pandas/tests/test_frame.py", line 10590, in test_sum
    has_numeric_only=True, check_dtype=False, check_less_precise=True)
  File "/home/travis/virtualenv/python2.6_with_system_site_packages/lib/python2.6/site-packages/pandas/tests/test_frame.py", line 10780, in _check_stat_op
    check_less_precise=check_less_precise)  # HACK: win32
  File "/home/travis/virtualenv/python2.6_with_system_site_packages/lib/python2.6/site-packages/pandas/util/testing.py", line 513, in assert_series_equal
    assert_almost_equal(left.values, right.values, check_less_precise)
  File "testing.pyx", line 58, in pandas._testing.assert_almost_equal (pandas/src/testing.c:2554)
  File "testing.pyx", line 93, in pandas._testing.assert_almost_equal (pandas/src/testing.c:1796)
  File "testing.pyx", line 140, in pandas._testing.assert_almost_equal (pandas/src/testing.c:2387)
AssertionError: expected 0.00144 but got 0.00144

@jreback
Copy link
Contributor

jreback commented Apr 27, 2014

@dalejung I have seen this one a few times....I think its just a precision issue, can iu nvestigate?

@dalejung
Copy link
Contributor Author

Yar. I'll track it down.

Sent from my iPhone

On Apr 27, 2014, at 10:31 AM, jreback [email protected] wrote:

@dalejung I have seen this one a few times....I think its just a precision issue, can iu nvestigate?

—
Reply to this email directly or view it on GitHub.

dalejung added 2 commits April 27, 2014 11:11
@dalejung
ENH: Added NDFrame.slice_shift
5e7bf20 
added slice_shift func docs and removed freq argument
@dalejung
BUG: Change Panel.shift to use slice_shift pandas-dev#6605 pandas-dev…
81675b6 
…#6826

TST: Make sure Panel.shift retains dtypes

DOC: removed previous doc entries for pandas-dev#6605

Re-add note about dropping shifted periods

DOC: added note about bug fix

don't pass on freq
jreback
jreback reviewed Apr 27, 2014
View reviewed changes
pandas/core/generic.py
@@ -3227,6 +3227,42 @@ def shift(self, periods=1, freq=None, axis=0, **kwds):

return self._constructor(new_data).__finalize__(self)

def slice_shift(self, periods=1, axis=0, **kwds):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not 100% happy with this (I like your soln better), but perf is odd. can we do this inside of Block instead? (e.g. just automatically do it if ndim >= 3), a bit cleaner that way and all the code in the same place

jreback added a commit that referenced this pull request Apr 28, 2014
@jreback
Merge pull request #6974 from dalejung/panel_shift_revert
a4870b5 
Panel shift revert
@jreback jreback merged commit a4870b5 into pandas-dev:master Apr 28, 2014
This was referenced Apr 28, 2014
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
Bug Dtype Conversions Unexpected or buggy dtype conversions Performance Memory or execution speed performance
Projects
None yet
Milestone
0.14.0
Development

Successfully merging this pull request may close these issues.

Panel.shift does not respect dtypes 13.1
2 participants
@dalejung @jreback