Skip to content

Dec cleanup #18844

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Dec 21, 2017
Merged

Dec cleanup #18844

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
bdc6ace
Replaced skip_if_no_package
WillAyd Dec 19, 2017
a5479d1
Cleaned up unusued funcs in testing.py
WillAyd Dec 19, 2017
404b786
Removed imperative scipy skipif from fixture
WillAyd Dec 19, 2017
e5032b9
Removed pytables min_version
WillAyd Dec 20, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions pandas/conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
from distutils.version import LooseVersion
import numpy
import pandas
import pandas.util.testing as tm
import dateutil


Expand Down Expand Up @@ -51,7 +50,6 @@ def add_imports(doctest_namespace):

@pytest.fixture(params=['bsr', 'coo', 'csc', 'csr', 'dia', 'dok', 'lil'])
def spmatrix(request):
tm._skip_if_no_scipy()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can we just remove this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of calling it imperatively I replaced it with decorators in the test functions that were using this fixture

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm...I think that we should still skip within the resource if it cannot be used. It is more decoupled that way IMO.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So you'd rather remove the decorator and keep it to the fixture to skip tests? Or do both and live with the duplication? I'd argue either one violates TMTOWTDI given the former would make this inconsistent with other SciPy-requiring tests that don't need this fixture (which are the majority). Not to mention it would defeat the purpose of the original issue. The latter violation should be self-evident.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah this is ok

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough. You could just do scipy = pytest.importorskip(...), but if we're okay with leaving it to the developer to be mindful of that dependency, sure.

from scipy import sparse
return getattr(sparse, request.param + '_matrix')

Expand Down
2 changes: 0 additions & 2 deletions pandas/tests/io/test_pytables.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -718,8 +718,6 @@ def test_put_compression(self):

@td.skip_if_windows_python_3
def test_put_compression_blosc(self):
tm.skip_if_no_package('tables', min_version='2.2',
app='blosc support')
df = tm.makeTimeDataFrame()

with ensure_clean_store(self.path) as store:
Expand Down
17 changes: 10 additions & 7 deletions pandas/tests/sparse/test_frame.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
from numpy import nan
import numpy as np
import pandas as pd
from distutils.version import LooseVersion

from pandas import Series, DataFrame, bdate_range, Panel
from pandas.core.dtypes.common import (
Expand All @@ -20,6 +21,7 @@
from pandas.compat import lrange
from pandas import compat
from pandas.core.sparse import frame as spf
import pandas.util._test_decorators as td

from pandas._libs.sparse import BlockIndex, IntIndex
from pandas.core.sparse.api import SparseSeries, SparseDataFrame, SparseArray
Expand Down Expand Up @@ -1169,14 +1171,13 @@ def test_notna(self):
tm.assert_frame_equal(res.to_dense(), exp)


@td.skip_if_no_scipy
@pytest.mark.parametrize('index', [None, list('abc')]) # noqa: F811
@pytest.mark.parametrize('columns', [None, list('def')])
@pytest.mark.parametrize('fill_value', [None, 0, np.nan])
@pytest.mark.parametrize('dtype', [bool, int, float, np.uint16])
def test_from_to_scipy(spmatrix, index, columns, fill_value, dtype):
# GH 4343
tm.skip_if_no_package('scipy')

# Make one ndarray and from it one sparse matrix, both to be used for
# constructing frames and comparing results
arr = np.eye(3, dtype=dtype)
Expand Down Expand Up @@ -1225,13 +1226,17 @@ def test_from_to_scipy(spmatrix, index, columns, fill_value, dtype):
assert sdf.to_coo().dtype == np.object_


@td.skip_if_no_scipy
@pytest.mark.parametrize('fill_value', [None, 0, np.nan]) # noqa: F811
def test_from_to_scipy_object(spmatrix, fill_value):
# GH 4343
dtype = object
columns = list('cd')
index = list('ab')
tm.skip_if_no_package('scipy', max_version='0.19.0')
import scipy
if (spmatrix is scipy.sparse.dok_matrix and LooseVersion(
scipy.__version__) >= LooseVersion('0.19.0')):
pytest.skip("dok_matrix from object does not work in SciPy >= 0.19")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a slightly different take on the existing code, which was the only place I saw the max_version argument being used. Rather than re-build that in the new module, I changed this around to be more explicit about the issue encountered when running this for SciPy v > 0.19.

Note that all of the scenarios in this test issue the warning that"object dtype is not supported by sparse matrices"


# Make one ndarray and from it one sparse matrix, both to be used for
# constructing frames and comparing results
Expand Down Expand Up @@ -1270,10 +1275,9 @@ def test_from_to_scipy_object(spmatrix, fill_value):
assert sdf.to_coo().dtype == res_dtype


@td.skip_if_no_scipy
def test_from_scipy_correct_ordering(spmatrix):
# GH 16179
tm.skip_if_no_package('scipy')

arr = np.arange(1, 5).reshape(2, 2)
try:
spm = spmatrix(arr)
Expand All @@ -1290,10 +1294,9 @@ def test_from_scipy_correct_ordering(spmatrix):
tm.assert_frame_equal(sdf.to_dense(), expected.to_dense())


@td.skip_if_no_scipy
def test_from_scipy_fillna(spmatrix):
# GH 16112
tm.skip_if_no_package('scipy')

arr = np.eye(3)
arr[1:, 0] = np.nan

Expand Down
11 changes: 6 additions & 5 deletions pandas/tests/test_nanops.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
from pandas.core.dtypes.common import is_integer_dtype
import pandas.core.nanops as nanops
import pandas.util.testing as tm
import pandas.util._test_decorators as td

use_bn = nanops._USE_BOTTLENECK

Expand Down Expand Up @@ -381,8 +382,8 @@ def test_nanstd(self):
allow_str=False, allow_date=False,
allow_tdelta=True, allow_obj='convert')

@td.skip_if_no('scipy', min_version='0.17.0')
def test_nansem(self):
tm.skip_if_no_package('scipy', min_version='0.17.0')
from scipy.stats import sem
with np.errstate(invalid='ignore'):
self.check_funs_ddof(nanops.nansem, sem, allow_complex=False,
Expand Down Expand Up @@ -441,17 +442,17 @@ def _skew_kurt_wrap(self, values, axis=None, func=None):
return 0.
return result

@td.skip_if_no('scipy', min_version='0.17.0')
def test_nanskew(self):
tm.skip_if_no_package('scipy', min_version='0.17.0')
from scipy.stats import skew
func = partial(self._skew_kurt_wrap, func=skew)
with np.errstate(invalid='ignore'):
self.check_funs(nanops.nanskew, func, allow_complex=False,
allow_str=False, allow_date=False,
allow_tdelta=False)

@td.skip_if_no('scipy', min_version='0.17.0')
def test_nankurt(self):
tm.skip_if_no_package('scipy', min_version='0.17.0')
from scipy.stats import kurtosis
func1 = partial(kurtosis, fisher=True)
func = partial(self._skew_kurt_wrap, func=func1)
Expand Down Expand Up @@ -549,8 +550,8 @@ def test_nancorr_pearson(self):
self.check_nancorr_nancov_1d(nanops.nancorr, targ0, targ1,
method='pearson')

@td.skip_if_no_scipy
def test_nancorr_kendall(self):
tm.skip_if_no_package('scipy.stats')
from scipy.stats import kendalltau
targ0 = kendalltau(self.arr_float_2d, self.arr_float1_2d)[0]
targ1 = kendalltau(self.arr_float_2d.flat, self.arr_float1_2d.flat)[0]
Expand All @@ -561,8 +562,8 @@ def test_nancorr_kendall(self):
self.check_nancorr_nancov_1d(nanops.nancorr, targ0, targ1,
method='kendall')

@td.skip_if_no_scipy
def test_nancorr_spearman(self):
tm.skip_if_no_package('scipy.stats')
from scipy.stats import spearmanr
targ0 = spearmanr(self.arr_float_2d, self.arr_float1_2d)[0]
targ1 = spearmanr(self.arr_float_2d.flat, self.arr_float1_2d.flat)[0]
Expand Down
86 changes: 0 additions & 86 deletions pandas/util/testing.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
from datetime import datetime
from functools import wraps, partial
from contextlib import contextmanager
from distutils.version import LooseVersion

from numpy.random import randn, rand
import numpy as np
Expand Down Expand Up @@ -317,35 +316,6 @@ def close(fignum=None):
_close(fignum)


def _skip_if_mpl_1_5():
import matplotlib as mpl

v = mpl.__version__
if LooseVersion(v) > LooseVersion('1.4.3') or str(v)[0] == '0':
import pytest
pytest.skip("matplotlib 1.5")
else:
mpl.use("Agg", warn=False)


def _skip_if_no_scipy():
import pytest

pytest.importorskip("scipy.stats")
pytest.importorskip("scipy.sparse")
pytest.importorskip("scipy.interpolate")


def _skip_if_no_mock():
try:
import mock # noqa
except ImportError:
try:
from unittest import mock # noqa
except ImportError:
import pytest
raise pytest.skip("mock is not installed")

# -----------------------------------------------------------------------------
# locale utilities

Expand Down Expand Up @@ -1979,62 +1949,6 @@ def __init__(self, *args, **kwargs):
dict.__init__(self, *args, **kwargs)


# Dependency checker when running tests.
#
# Copied this from nipy/nipype
# Copyright of respective developers, License: BSD-3
def skip_if_no_package(pkg_name, min_version=None, max_version=None,
app='pandas', checker=LooseVersion):
"""Check that the min/max version of the required package is installed.

If the package check fails, the test is automatically skipped.

Parameters
----------
pkg_name : string
Name of the required package.
min_version : string, optional
Minimal version number for required package.
max_version : string, optional
Max version number for required package.
app : string, optional
Application that is performing the check. For instance, the
name of the tutorial being executed that depends on specific
packages.
checker : object, optional
The class that will perform the version checking. Default is
distutils.version.LooseVersion.

Examples
--------
package_check('numpy', '1.3')

"""

import pytest
if app:
msg = '{app} requires {pkg_name}'.format(app=app, pkg_name=pkg_name)
else:
msg = 'module requires {pkg_name}'.format(pkg_name=pkg_name)
if min_version:
msg += ' with version >= {min_version}'.format(min_version=min_version)
if max_version:
msg += ' with version < {max_version}'.format(max_version=max_version)
try:
mod = __import__(pkg_name)
except ImportError:
mod = None
try:
have_version = mod.__version__
except AttributeError:
pytest.skip('Cannot find version for {pkg_name}'
.format(pkg_name=pkg_name))
if min_version and checker(have_version) < checker(min_version):
pytest.skip(msg)
if max_version and checker(have_version) >= checker(max_version):
pytest.skip(msg)


def optional_args(decorator):
"""allows a decorator to take optional positional and keyword arguments.
Assumes that taking a single, callable, positional argument means that
Expand Down