Change Panel.shift to use NDFrame.shift

[dalejung]

closes #4867

This brings Panel.shift in line with DataFrame.shift. The shifted data retains the same dimensions/indexes and doesn't drop the empty rows. So I have two concerns.

1. This is an API change and could conceivably break existing code.
2. The perf regresses quite a bit.

Here is the vbench.

-------------------------------------------------------------------------------
Test name                                    | head[ms] | base[ms] |  ratio   |
-------------------------------------------------------------------------------
panel_shift                                  | 578.2867 |   0.0773 | 7478.4491 |
-------------------------------------------------------------------------------
Test name                                    | head[ms] | base[ms] |  ratio   |
-------------------------------------------------------------------------------

The previous implementation did not create a copy and thus had a constant time. The general shift scales linearly.

I think the current implementation just defers work done in later alignments, but I can only speak to my own workflow.

[jreback]

hmm. I bet the rolling stuff that makes a 2-d much faster makes this much slower.

I think you mnay have to roll this in a different way somehow.

[dalejung]

Hm. You talking about #6404? I'm going to setup a test battery just to get an idea of what the timings are. I think that the shift isn't that much worse than a straight copy. The regression is going to look disgusting compared to grabbing a view :P

[jreback]

#6404 was merged in, pls rebase and see about the perf tests and what if anything can do about them

[dalejung]

With #6747 the panel shift speed is on par with pre-quick-shift.

http://nbviewer.ipython.org/gist/dalejung/9731798/panel%20shift.ipynb
I monkey patched the old shift code onto the Panel to compare.

[dalejung]

@jreback ping on green.

[jreback]

gr8 can you add a release note (referencing the issue)

and squash downto 1 commit

ping me whengreen

[dalejung]

@jreback green

[jreback]

I thiought this was fixed?

panel_shift                                  | 263.2827 |   0.0746 | 3528.0756 |
-------------------------------------------------------------------------------
Test name                                    | head[ms] | base[ms] |  ratio   |
-------------------------------------------------------------------------------

Ratio < 1.0 means the target commit is faster then the baseline.
Seed used: 1234

Target [e84efe5] : Merge pull request #6825 from cpcloud/replace-dict-vbenches

BENCH: add vbench for issue 6697
Base   [d10a658] : RLS: set released to True. v0.13.1

[jreback]

#6826

[dalejung]

That particularly perf hit won't be fixable if Panel uses the general shift. The 0.13.1 Panel.shift just creates a view, which is near instant and constant time.

The quick shift fixes #6747 fixes speed for shift generally which included the Panel general shift case.

So the timeline:

1. Panel.shift takes views on data. 0ms
2. General shift PR. Perf hit. 500ms
3. quick shift PR (np.roll) Quick Frame Shift Implementation #6404. Panel.shift/F-ordered shift degrade 1.2s
4. perf fixes for quick shift Doc fixes #6746 Panel.shift back down to 500ms

Sorry, I think the fix from 3->4 was conflated with fixing the perf hit from 1->2.

Let me know if I need to back out the PR.

[jreback]

ok

either back it out completely or
incorporate into the same shift logic in core/internals

so it's at least in 1 place (even if special cased)

[dalejung]

moved discussion to #6826