BUG: Kurtosis bug?

[mindryu]

Why pandas kurtosis and scipy kurtosis result is different?
During the test, I also realized that the Pandas kurtosis value was constantly changing according to the starting point.

[mroeschke]

1. pandas statistical methods are unbiased, scipy.stats.kurtosis has bias=True by default
2. It would be easier to describe the problem/difference comparing the numerical values instead visually with these graphs.

[mindryu]

bias = False also different.
Apart from that, isn't it a bug that values constantly changing according to the starting point? Just looking at the screen shot, you can see the one line sticking out alone, even though I just sliding the window. If it's normal, it should all be the same shape, regardless of location.

[mindryu]

It's weird. I'm trying to create an example by random generate time series. It can't be recreated. Several other stock charts work fine, too.
Is this what happens only some case or Did I do something wrong?

from scipy.stats import kurtosis
df = pd.read_csv('test.txt')
test_length=200

for d in range(10):
    test = df[-test_length-d:-d].reset_index(drop=True)
    plt.plot(test.close.rolling(4).kurt())
plt.show()


for d in range(10):
    test = df[-test_length-d:-d].reset_index(drop=True)
    plt.plot(test.close.rolling(4).apply(lambda x: kurtosis(x, bias=False), raw=True))
plt.show()

test.txt

[bashtage]

Is the variance of your data very close to zero? If so, you could be seeing differences in rounding/numerical precision.

[bashtage]

excess kurtosis should always be > -3. Since you are seeing large values then you are experiencing numerical errors and limits of numerical precision. I suspect many of your blocks have 0 or near 0 variance.

[bashtage]

To repro, just share observations 20 - 30 of test.csv.

[mindryu]

@bashtage

I think it's just like you said.
In fact, there's not much change in the 1 minute chart. It changes rapidly in a moment.
It wasn't stable enough to believe until rolling length 40. Until than, value have changed little by little

This problem made my three day trained network obsolete...
Meaning of sharing observations? Should I upload csv with the kurtosis value again?
Thank you.

Rolling 30

[mroeschke]

We're happy to reopen this issue when we can validate the issue with a reproducible example. Looking at small raw data samples should be sufficient instead of plots