python >= 3.6, add scipy 1.2.0 to min requirements (#1035)

* add scipy==1.2.0 to min requirements

* missed a needs numpy 10

* revert an import change in test_location

* whats new

* add missing requires_tables

* import cleanup. fix stray requires_scipy

* and fix new requires_scipy marks

* remove unneeded pytest.param

* fix depedency specs

* lint

* sdm.py import clean up

* move 3.5 packages to pip. spec versions in all yml files

* require python 3.6 or greater

* remove 3.5 ci, rename 35 min file

* update ci for 36

* remove from readme

* installation instructions

* fix python 3.5 references in contributing

Author: wholmgren

README.md

@@ -105,10 +105,6 @@ Installation
 pvlib-python releases may be installed using the ``pip`` and ``conda`` tools.
 Please see the [Installation page](http://pvlib-python.readthedocs.io/en/stable/installation.html) of the documentation for complete instructions.
 
-pvlib-python is compatible with Python 3.5 and above.
-
-**Python 2.7 support ended on June 1, 2019, with pvlib-python 0.6.3.**
-
 
 Contributing
 ============

ci/azure/conda_linux.yml

@@ -8,10 +8,8 @@ jobs:
     vmImage: ${{ parameters.vmImage }}
   strategy:
     matrix:
-      Python35:
-        python.version: '35'
-      Python35-min:
-        python.version: '35'
+      Python36-min:
+        python.version: '36'
         suffix: '-min'
       Python36:
         python.version: '36'

ci/azure/conda_windows.yml

@@ -8,8 +8,6 @@ jobs:
     vmImage: ${{ parameters.vmImage }}
   strategy:
     matrix:
-      Python35-windows:
-        python.version: '35'
       Python36-windows:
         python.version: '36'
       Python37-windows:

ci/azure/posix.yml

@@ -8,8 +8,6 @@ jobs:
     vmImage: ${{ parameters.vmImage }}
   strategy:
     matrix:
-      Python35:
-        python.version: '3.5'
       Python36:
         python.version: '3.6'
       Python37:

ci/requirements-py35.yml

@@ -9,11 +9,12 @@ dependencies:
     - pytest-cov
     - pytest-mock
     - pytest-timeout
-    - python=3.5
+    - python=3.6
     - pytz
     - requests
     - pip:
         - numpy==1.12.0
         - pandas==0.22.0
+        - scipy==1.2.0
         - pytest-rerunfailures # conda version is >3.6
         - pytest-remotedata # conda package is 0.3.0, needs > 0.3.1

ci/requirements-py35-min.yml renamed to ci/requirements-py36-min.yml

@@ -9,8 +9,8 @@ dependencies:
     - netcdf4
     - nose
     - numba
-    - numpy
-    - pandas
+    - numpy >= 1.12.0
+    - pandas >= 0.22.0
     - pip
     - pytables  # tables when using pip+PyPI
     - pytest
@@ -22,7 +22,7 @@ dependencies:
     - python=3.6
     - pytz
     - requests
-    - scipy
+    - scipy >= 1.2.0
     - shapely  # pvfactors dependency
     - siphon  # conda-forge
     - statsmodels

ci/requirements-py36.yml

@@ -9,8 +9,8 @@ dependencies:
     - netcdf4
     - nose
     - numba
-    - numpy
-    - pandas
+    - numpy >= 1.12.0
+    - pandas >= 0.22.0
     - pip
     - pytables  # tables when using pip+PyPI
     - pytest
@@ -22,7 +22,7 @@ dependencies:
     - python=3.7
     - pytz
     - requests
-    - scipy
+    - scipy >= 1.2.0
     - shapely  # pvfactors dependency
     - siphon  # conda-forge
     - statsmodels

ci/requirements-py37.yml

@@ -9,8 +9,8 @@ dependencies:
     - netcdf4
     - nose
     - numba
-    - numpy
-    - pandas
+    - numpy >= 1.12.0
+    - pandas >= 0.22.0
     - pip
     - pytables  # tables when using pip+PyPI
     - pytest
@@ -22,7 +22,7 @@ dependencies:
     - python=3.8
     - pytz
     - requests
-    - scipy
+    - scipy >= 1.2.0
     - shapely  # pvfactors dependency
     - siphon  # conda-forge
     - statsmodels

ci/requirements-py38.yml

@@ -196,8 +196,6 @@ pvlib python generally follows the `PEP 8 -- Style Guide for Python Code
 <https://www.python.org/dev/peps/pep-0008/>`_. Maximum line length for code
 is 79 characters.
 
-Code must be compatible with Python 3.5 and above.
-
 pvlib python uses a mix of full and abbreviated variable names. See
 :ref:`variables_style_rules`. We could be better about consistency.
 Prefer full names for new contributions. This is especially important
@@ -475,9 +473,10 @@ PVSystem method is called through ``ModelChain.run_model``.
         mc.run_model(times)
 
         # assertion fails if PVSystem.sapm is not called once
-        # if using returned m, prefer this over m.assert_called_once()
-        # for compatibility with python < 3.6
-        assert m.call_count == 1
+        m.assert_called_once()
+
+        # use `assert m.call_count == num` if function should be called
+        # more than once
 
         # ensure that dc attribute now exists and is correct type
         assert isinstance(mc.dc, (pd.Series, pd.DataFrame))

docs/sphinx/source/contributing.rst

@@ -224,9 +224,9 @@ environment) when you start a new shell or terminal.
 Compatibility
 -------------
 
-pvlib-python is compatible with Python 3.5 and above.
+pvlib-python is compatible with Python 3.
 
-pvlib-python requires Pandas and Numpy. The minimum version requirements
+pvlib-python requires Pandas, Numpy, and SciPy. The minimum version requirements
 are specified in
 `setup.py <https://github.com/pvlib/pvlib-python/blob/master/setup.py>`_.
 They are typically releases from several years ago.
@@ -235,8 +235,8 @@ A handful of pvlib-python features require additional packages that must
 be installed separately using pip or conda. These packages/features
 include:
 
-* scipy: single diode model, clear sky detection
 * pytables (tables on PyPI): Linke turbidity look up for clear sky models
+* statsmodels: parameter fitting
 * numba: fastest solar position calculations
 * pyephem: solar positions calculations using an astronomical library
 * siphon: forecasting PV power using the pvlib.forecast module

docs/sphinx/source/installation.rst

@@ -158,7 +158,10 @@ Documentation
 
 Requirements
 ~~~~~~~~~~~~
+* Python 3.6 or greater. (:pull:`1035`)
 * Minimum pandas version increased to v0.22.0, released Dec 31, 2017. (:pull:`1003`)
+* Scipy >= v1.2.0, released Dec 17, 2018, is now a required dependency
+  rather an optional dependency. (:issue:`972`, :pull:`1035`)
 
 Contributors
 ~~~~~~~~~~~~

docs/sphinx/source/whatsnew/v0.8.0.rst

@@ -8,6 +8,10 @@
 
 import numpy as np
 
+import scipy.constants
+from scipy import optimize
+from scipy.special import lambertw
+
 from pvlib.pvsystem import singlediode, v_from_i
 
 from pvlib.ivtools.utility import constants, rectify_iv_curve, _numdiff
@@ -197,12 +201,6 @@ def fit_desoto(v_mp, i_mp, v_oc, i_sc, alpha_sc, beta_voc, cells_in_series,
        2006.
     """
 
-    try:
-        from scipy.optimize import root
-        import scipy.constants
-    except ImportError:
-        raise ImportError("The fit_desoto function requires scipy.")
-
     # Constants
     k = scipy.constants.value('Boltzmann constant in eV/K')
     Tref = temp_ref + 273.15  # [K]
@@ -222,8 +220,8 @@ def fit_desoto(v_mp, i_mp, v_oc, i_sc, alpha_sc, beta_voc, cells_in_series,
              Tref, k)
 
     # computing with system of equations described in [1]
-    optimize_result = root(_system_of_equations_desoto, x0=params_i,
-                           args=(specs,), **root_kwargs)
+    optimize_result = optimize.root(_system_of_equations_desoto, x0=params_i,
+                                    args=(specs,), **root_kwargs)
 
     if optimize_result.success:
         sdm_params = optimize_result.x
@@ -657,8 +655,8 @@ def _fit_desoto_sandia_diode(ee, voc, vth, tc, specs, const):
     try:
         import statsmodels.api as sm
     except ImportError:
-        raise ImportError('Parameter extraction using Sandia method requires',
-                          ' statsmodels')
+        raise ImportError(
+            'Parameter extraction using Sandia method requires statsmodels')
 
     x = specs['cells_in_series'] * vth * np.log(ee / const['E0'])
     y = voc - specs['beta_voc'] * (tc - const['T0'])
@@ -791,11 +789,10 @@ def _extract_sdm_params(ee, tc, iph, io, rs, rsh, n, u, specs, const,
     # Get single diode model parameters from five parameters iph, io, rs, rsh
     # and n vs. effective irradiance and temperature
     try:
-        from scipy import optimize
         import statsmodels.api as sm
     except ImportError:
-        raise ImportError('Parameter extraction using Sandia method requires',
-                          ' scipy and statsmodels')
+        raise ImportError(
+            'Parameter extraction using Sandia method requires statsmodels')
 
     tck = tc + 273.15
     tok = const['T0'] + 273.15  # convert to to K
@@ -1183,12 +1180,6 @@ def _calc_theta_phi_exact(vmp, imp, iph, io, rs, rsh, nnsvth):
        real solar cells using Lambert W-function", Solar Energy Materials and
        Solar Cells, 81 (2004) 269-277.
     """
-
-    try:
-        from scipy.special import lambertw
-    except ImportError:
-        raise ImportError('calc_theta_phi_exact requires scipy')
-
     # handle singleton inputs
     vmp = np.asarray(vmp)
     imp = np.asarray(imp)

pvlib/ivtools/sdm.py

@@ -6,6 +6,9 @@
 import numpy as np
 import pandas as pd
 
+import scipy.optimize
+from scipy.spatial.distance import pdist
+
 
 def wvm(clearsky_index, positions, cloud_speed, dt=None):
     """
@@ -59,12 +62,6 @@ def wvm(clearsky_index, positions, cloud_speed, dt=None):
 
     # Added by Joe Ranalli (@jranalli), Penn State Hazleton, 2019
 
-    try:
-        import scipy.optimize
-        from scipy.spatial.distance import pdist
-    except ImportError:
-        raise ImportError("The WVM function requires scipy.")
-
     pos = np.array(positions)
     dist = pdist(pos, 'euclidean')
     wavelet, tmscales = _compute_wavelet(clearsky_index, dt)

pvlib/scaling.py

@@ -7,23 +7,11 @@
 import numpy as np
 from pvlib.tools import _golden_sect_DataFrame
 
-# Try to import brentq from scipy to use when specified in bishop88_i_from_v,
-# bishop88_v_from_i, and bishop88_mpp methods below. If not imported, raises
-# ImportError when 'brentq' method is specified for those methods.
-try:
-    from scipy.optimize import brentq
-except ImportError:
-    def brentq(*a, **kw):
-        raise ImportError(
-            "brentq couldn't be imported. Is SciPy installed?")
-
-# FIXME: change this to newton when scipy-1.2 is released
-try:
-    from scipy.optimize import _array_newton
-except ImportError:
-    from pvlib.tools import _array_newton
-# rename newton and set keyword arguments
-newton = partial(_array_newton, tol=1e-6, maxiter=100, fprime2=None)
+from scipy.optimize import brentq, newton
+from scipy.special import lambertw
+
+# set keyword arguments for all uses of newton in this module
+newton = partial(newton, tol=1e-6, maxiter=100, fprime2=None)
 
 # intrinsic voltage per cell junction for a:Si, CdTe, Mertens et al.
 VOLTAGE_BUILTIN = 0.9  # [V]
@@ -510,11 +498,6 @@ def _prepare_newton_inputs(i_or_v_tup, args, v0):
 
 def _lambertw_v_from_i(resistance_shunt, resistance_series, nNsVth, current,
                        saturation_current, photocurrent):
-    try:
-        from scipy.special import lambertw
-    except ImportError:
-        raise ImportError('This function requires scipy')
-
     # Record if inputs were all scalar
     output_is_scalar = all(map(np.isscalar,
                                [resistance_shunt, resistance_series, nNsVth,
@@ -592,11 +575,6 @@ def _lambertw_v_from_i(resistance_shunt, resistance_series, nNsVth, current,
 
 def _lambertw_i_from_v(resistance_shunt, resistance_series, nNsVth, voltage,
                        saturation_current, photocurrent):
-    try:
-        from scipy.special import lambertw
-    except ImportError:
-        raise ImportError('This function requires scipy')
-
     # Record if inputs were all scalar
     output_is_scalar = all(map(np.isscalar,
                                [resistance_shunt, resistance_series, nNsVth,

pvlib/singlediode.py

@@ -6,6 +6,8 @@
 import datetime
 import numpy as np
 import pandas as pd
+from scipy.special import erf
+
 from pvlib.tools import cosd
 
 
@@ -62,11 +64,6 @@ def hsu(rainfall, cleaning_threshold, tilt, pm2_5, pm10,
        Change. J. Seinfeld and S. Pandis. Wiley and Sons 2001.
 
     """
-    try:
-        from scipy.special import erf
-    except ImportError:
-        raise ImportError("The pvlib.soiling.hsu function requires scipy.")
-
     # never use mutable input arguments
     if depo_veloc is None:
         depo_veloc = {'2_5': 0.0009, '10': 0.004}

pvlib/soiling.py

@@ -20,6 +20,7 @@
 
 import numpy as np
 import pandas as pd
+import scipy.optimize as so
 import warnings
 
 from pvlib import atmosphere
@@ -911,12 +912,6 @@ def calc_time(lower_bound, upper_bound, latitude, longitude, attribute, value,
         If the given attribute is not an attribute of a
         PyEphem.Sun object.
     """
-
-    try:
-        import scipy.optimize as so
-    except ImportError:
-        raise ImportError('The calc_time function requires scipy')
-
     obs, sun = _ephem_setup(latitude, longitude, altitude,
                             pressure, temperature, horizon)