computation.tests.test_eval.TestScope.test_global_scope generates errors with numexpr engine and numpy 1.7.0

[neirbowj]

$ nosetests -v pandas.computation.tests.test_eval:TestScope.test_global_scope
pandas.computation.tests.test_eval.TestScope.test_global_scope('_var_s * 2', 'python', 'python') ... ok
pandas.computation.tests.test_eval.TestScope.test_global_scope('_var_s * 2', 'python', 'pandas') ... ok
pandas.computation.tests.test_eval.TestScope.test_global_scope('_var_s * 2', 'numexpr', 'python') ... ERROR
pandas.computation.tests.test_eval.TestScope.test_global_scope('_var_s * 2', 'numexpr', 'pandas') ... ERROR

======================================================================
ERROR: pandas.computation.tests.test_eval.TestScope.test_global_scope('_var_s * 2', 'numexpr', 'python')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/obrienjw/src/github/pandas/pandas/computation/tests/test_eval.py", line 1430, in check_global_scope
    parser=parser))
  File "/home/obrienjw/src/github/pandas/pandas/computation/eval.py", line 208, in eval
    ret = eng_inst.evaluate()
  File "/home/obrienjw/src/github/pandas/pandas/computation/engines.py", line 53, in evaluate
    self.expr.terms.return_type)
  File "/home/obrienjw/src/github/pandas/pandas/computation/align.py", line 250, in _reconstruct_object
    ret = ret_value.item()
IndexError: index 140737488325120 is out of bounds for axis 0 with size 10

======================================================================
ERROR: pandas.computation.tests.test_eval.TestScope.test_global_scope('_var_s * 2', 'numexpr', 'pandas')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/obrienjw/src/github/pandas/pandas/computation/tests/test_eval.py", line 1430, in check_global_scope
    parser=parser))
  File "/home/obrienjw/src/github/pandas/pandas/computation/eval.py", line 208, in eval
    ret = eng_inst.evaluate()
  File "/home/obrienjw/src/github/pandas/pandas/computation/engines.py", line 53, in evaluate
    self.expr.terms.return_type)
  File "/home/obrienjw/src/github/pandas/pandas/computation/align.py", line 250, in _reconstruct_object
    ret = ret_value.item()
IndexError: index 140737488325120 is out of bounds for axis 0 with size 10

----------------------------------------------------------------------
Ran 4 tests in 0.007s

FAILED (errors=2)

Here are the version details of my environment.

$ git describe
v0.12.0-1110-gd250d64
$ ci/print_versions.py

INSTALLED VERSIONS
------------------
Python: 2.7.5.final.0
OS: FreeBSD 9.2-PRERELEASE FreeBSD 9.2-PRERELEASE #0 r254955: Tue Aug 27 11:01:00 EDT 2013     [email protected]:/usr/obj/usr/src/sys/NARB amd64
byteorder: little
LC_ALL: None
LANG: None

pandas: Not installed
Cython: 0.19.1
Numpy: 1.7.0
Scipy: 0.12.1
statsmodels: Not installed
    patsy: Not installed
scikits.timeseries: Not installed
dateutil: 2.1
pytz: 2013.8
bottleneck: Not installed
PyTables: Not Installed
    numexpr: 2.2.2
matplotlib: 1.2.0
openpyxl: Not installed
xlrd: Not installed
xlwt: Not installed
xlsxwriter: Not installed
sqlalchemy: Not installed
lxml: Not installed
bs4: Not installed
html5lib: Not installed
bigquery: Not installed
apiclient: Not installed

Edited title to reflect latest analysis; affected code depends on the version of numpy, not the platform.

[jtratner]

Weird question: does it generate exactly the same error each time?

[neirbowj]

Yes.

$ for i in $(seq 100); do nosetests -v pandas.computation.tests.test_eval:TestScope.test_global_scope 2>&1 | grep IndexError; done | sort -u
IndexError: index 140737488325120 is out of bounds for axis 0 with size 10

[jtratner]

Thanks, didn't mean to make you run that 100 times there.

[neirbowj]

I wouldn't have known the answer without running it a bunch of times. ;-)

[neirbowj]

This appears to be caused by a regression in numpy 1.7.0 that was fixed in 1.8.0 by numpy/numpy@04b89c63, and in 1.7.1 by numpy/numpy@b84555ac.

>>> import numpy as np
>>> np.__version__
'1.7.0'
>>> x = np.ndarray(5)
>>> x.item()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: index 679811744 is out of bounds for axis 0 with size 5