Model chains (#117)

* Add normalized_filter() function to replace the mannual filter in example

* Initial working system analysis class

* system analysis version that reproduces notebook results

Works for both sensor and clearsky workflows

* Improve underlying analysis

These improvements slightly change the clearsky results relative to the existing version of the notebook.

* Allow poa, temperature, and temperature_coefficient to be None

This leaves room for BYO modeled PV perofmrance in the future

* Add system_analysis to rdtools namespace

* Basic example of system_analysis() use

* Update degradation_and_soiling_example.ipynb

1) Use apparent zenith in irradiance calculations instead of zenith
2) Apply clearsky filter based on poa irradiance rather than insolation
3) Include ground diffuse in irradiance transposition calculations

* Update system_analysis_example notebook

Update notes and change initial poa calculation to include ground diffuse

* Add a plotting module

* Update notebook to use plotting module

* Add plotting methods

* update system_analysis example notebook

* add docstrings

* Delete model chain dev notes

* add matplotlib to setup.py and update requirements

* change matplotlib version

* Move docstring to class

* renormalization bug fix

* Update temperature input interface

(Also fix merge bug in plotting)

* update docstring for new temperature interface

* Remove __init__ docstring

* update class name to CamelCase

* update kwarg explanatations in docstrings

* remove model parameter from calc_clearsky_poa()

It is still addressable through kwargs passed to pvlib

* Update SystemAnalysis_example.ipynb

* Reformat dosctrings to numpy style

* command style docstrings

* Remove duplicate line

* Update plotting docstrings to numpy style

* Format normalized_filter docstring according to numpy style

Other docstrings in this module have been updated in #125

* Add an ad hoc filter example to the notebook

* Fixes system_analysis bug #132 when pv_nameplate is passed into syste… (#133)

* Fixes system_analysis bug #132 when pv_nameplate is passed into system_analysis object.

* typo fix

Co-Authored-By: Kevin Anderson <[email protected]>

* add bins parameter to plotting.degradation_summary_plot from #132

(cherry picked from commit efcbe2e)

# Conflicts:
#	rdtools/plotting.py

* plotting bug fix

* Drop py2.7 and add 3.7 and 3.8 to testing (#135)

* Drop 2.7 and add 3.7 and 3.8 to testing, update docs.

* creating DatetimeIndex directly is deprecated, switch to pd.date_range

* require pandas < 1.0.0

* bump requirements.txt numpy to 1.17.3 for testing on py3.8

* more requirements.txt updates for py3.8 wheel availability

* Update v2.0.0.rst

* add matplotlib to requirements

* matplotlib 3.1

* matplotlib

* merge pytests and jupyter example

* PVLib > 0.7 changes to cell_temp calculation

* Modelchain pytests (#196)

* pytests for system_analysis.  coverage: 91%. 

* rename system_analysis.py to analysis.py. Rename class to RdAnalysis. Update pvwatts_kws to match Master updates

* Allow temp_model_params to either be string or dict with 'a','b', 'deltaT' keys

* update calc_cell_temperature for pvlib > 0.6.3, use energy_normalized in normalize_with_pvwatts

* Warn if temp coefficient not passed into normalize_with_Pvwatts instead of exit with error.

* Add tables=3.6.1 to requirements.txt to allow clearsky analysis

* add tables to setup.py since it appears to be a required pvlib for the clearsky workflow and not installed by default.

* Add RdAnalysis notebook for pvdaq4 system

* add max_timedelta=15T and clearsky poa calculation update to pvdaq4 standard notebook example

* Sphinx release notes for 2.1.0 updated and API update

Co-authored-by: Michael Deceglie <[email protected]>
Co-authored-by: Kevin Anderson <[email protected]>

* Update tests, include tables in setup.py

* suppress import warnings for soiling module.

* Change warning suppression from warnings.resetwarnings to with warnings.catch_warnings().
Re-run notebooks

* Model chains exp energy (#232)

* enable normalize_with_expected_energy. 

* Update RdAnalysis_example.ipynb with manually defined power_expected option.

* Add post-filter error check for < 2 yrs data per kanderson

* Remove error message for no thermal model if power_expected is passed in.

* remove tcell_filter from filter list if power_expected is passed in but not cell_temperature

* pep8 compliance

* more robust frequency check

* shorten long lines

* Model chains set clearsky (#233)

* Move CS inputs to new set_clearsky function: pvlib_location ,pv_tilt, pv_azimuth, clearsky_poa, clearsky temp, albedo

* Explicit error message if set_clearsky not run prior to clearsky_analysis.

* remove Py2.7 check in block 3 of RdAnalysis_example.ipynb

* boost analysis.py test coverage to 94%

* add pv_energy pytest case. Change to ValueError rather than basic Exception when set_clearsky hasn't been run.

Co-authored-by: Kevin Anderson <[email protected]>

* improve index equality conditional

* pep8 cleanup

* change dict key syntax

* Clarify behavior when pv_nameplate is omitted

* Useful errors for plotting methods

* update requreiments for tables

* reduce minimum version of tables

* Delete RdAnalysis_example.ipynb

* Remove max_timedelta parameter use in PVDAQ example

* Move sun position calculation to clearsky section of example

This brings the clearsky results into better allignement with the objected oriented example, now sun position is calculated after interpolation on both

* Update chage log version number

* Allow warning for experimental modules

* pep8 analysis_test.py

* Change module and class names

* Update example with new module/class name

* Fix outdated ref to system_analysis

* Update api.rst with analysis chains module name

* Fix changelog underline length

* update index.rst with TrendAnalysis info

* poa -> poa_global

* cell_temperature -> temperature_cell

* ambient_temperature->temperature_ambient

* temperature_coefficient->gamma_pdc

* temperature_model docstring

* pv_nameplate->power_dc_rated

* clearsky_poa->poa_global_clearsky

* clearsky_temperature_cell->temperature_cell_clearsky

* clearsky_temperature_ambient->temperature_ambient_clearsky

* Align yoy and srr parameters with functional API

* Use `case` in plotting methods

* Change notebook kernel

* weakly privatize some methods

* Update rdtools/analysis_chains.py

Co-authored-by: Kevin Anderson <[email protected]>

* Interpolate windspeed

* Update rdtools/analysis_chains.py

Co-authored-by: Kevin Anderson <[email protected]>

* implement review suggestions

* more TrendAnalysis tests to improve coverage

* cleanup

* Fix copy/paste error in notebook

* change _calc_cell_temeprature parameter order

* More elegant error if filtering results in empty series

* Changelog consolidation

* change log update

Co-authored-by: cdeline <[email protected]>
Co-authored-by: Kevin Anderson <[email protected]>
Co-authored-by: Kevin Anderson <[email protected]>

Author: 3 people

docs/TrendAnalysis_example_pvdaq4.ipynb

@@ -34,7 +34,6 @@ pyzmq==17.1.0
 qtconsole==4.3.1
 Send2Trash==1.5.0
 simplegeneric==0.8.1
-tables==3.6.1
 terminado==0.8.1
 testpath==0.3.1
 tornado==5.1

docs/degradation_and_soiling_example_pvdaq_4.ipynb

@@ -0,0 +1,3 @@
+{
+    "path": "../../TrendAnalysis_example_pvdaq4.ipynb"
+}

docs/notebook_requirements.txt

@@ -14,6 +14,7 @@ analysis workflow.
 .. autosummary::
    :toctree: generated/
 
+   analysis_chains
    degradation
    soiling   
    availability
@@ -23,7 +24,25 @@ analysis workflow.
    clearsky_temperature
    plotting
 
+Analysis Chains
+===============
 
+Object-oriented end-to-end analysis
+
+.. autosummary::
+   :toctree: generated/
+
+   analysis_chains.TrendAnalysis
+   analysis_chains.TrendAnalysis.set_clearsky
+   analysis_chains.TrendAnalysis.sensor_analysis
+   analysis_chains.TrendAnalysis.clearsky_analysis
+   analysis_chains.TrendAnalysis.plot_degradation_summary
+   analysis_chains.TrendAnalysis.plot_soiling_rate_histogram
+   analysis_chains.TrendAnalysis.plot_soiling_interval
+   analysis_chains.TrendAnalysis.plot_soiling_monte_carlo
+   analysis_chains.TrendAnalysis.plot_pv_vs_irradiance
+   
+   
 Degradation
 ===========
 

docs/sphinx/source/TrendAnalysis_example.nblink

@@ -1,8 +1,10 @@
 RdTools Change Log
 ==================
 
+.. include:: changelog/v2.1.0-beta.0.rst
 .. include:: changelog/v2.0.2.rst
 .. include:: changelog/v2.0.1.rst
 .. include:: changelog/v2.0.0.rst
 .. include:: changelog/pre_2.0.0.rst
 
+

docs/sphinx/source/api.rst

@@ -0,0 +1,33 @@
+*********************************
+v2.1.0-beta.0 (November 20, 2020)
+*********************************
+
+
+Enhancements
+------------
+* Add new :py:mod:`~rdtools.analysis_chains` module to focus on objected-oriented analysis workflows
+  combining other RdTools modules. Includes :py:class:`~rdtools.analysis_chains.TrendAnalysis` class
+  for sensor- and clear-sky-based soiling and degradation analyses (:pull:`117`).
+
+
+Requirements
+------------
+* tables added as a requirement (:pull:`196`).
+
+
+Example Updates
+---------------
+* New example notebook based on PVDAQ system #4 for the new
+  :py:class:`~rdtools.analysis_chains.TrendAnalysis` analysis work flow (:pull:`196` and
+  :pull:`117`).
+* Update ``degradation_and_soiling_example_pvdaq_4.ipynb`` example to match best practice, including 
+  ``pvlib.get_total_irradiance()`` in rdtools.interpolate (:pull:`196` and :pull:`117`).
+* Update ``degradation_and_soiling_example_pvdaq_4.ipynb`` example to use a single
+  ``soiling * ac_power`` signal (:pull:`196`).
+  
+
+Contributors
+------------
+* Mike Deceglie (:ghuser:`mdeceglie`)
+* Kevin Anderson (:ghuser:`kanderso-nrel`)
+* Chris Deline (:ghuser:`cdeline`)

docs/sphinx/source/changelog.rst

@@ -28,7 +28,7 @@ Full examples are worked out in the example notebooks in the
 To report issues, contribute code, or suggest improvements to this
 documentation, visit the RdTools development repository on `github`_.
 
-Degradation and Soiling
+Trends
 -----------------------
 
 Both degradation and soiling analyses are based on normalized yield, similar to performance
@@ -98,6 +98,13 @@ identified soiling rates for the dataset.
    :width: 320
    :height: 216
 
+TrendAnalysis
+^^^^^^^^^^^^^
+An object-oriented API for complete soiling and degradation analysis including 
+the normalize, filter, aggregate, analyze steps is available in
+:py:class:`.analysis_chains.TrendAnalysis`. See the `TrendAnalysis example`_ 
+for details. 
+
 Availability
 ------------
 
@@ -285,6 +292,7 @@ Documentation Contents
    :maxdepth: 2
 
    Degradation and Soiling <rd_example>
+   TrendAnalysis <TrendAnalysis_example>
    Inverter Downtime <system_availability_example>
    API Reference <api>
    Change Log <changelog>
@@ -299,7 +307,7 @@ Indices and tables
 
 
 .. links and references
-
+.. _TrendAnalysis example: TrendAnalysis_example.nblink 
 .. _example notebook: rd_example.nblink
 .. _release: https://github.com/NREL/rdtools/releases
 .. _github: https://github.com/NREL/rdtools

docs/sphinx/source/changelog/v2.1.0-beta.0.rst

@@ -14,9 +14,10 @@
 from rdtools.filtering import tcell_filter
 from rdtools.filtering import clip_filter
 from rdtools.filtering import normalized_filter
-# from rdtools.soiling import soiling_srr
-# from rdtools.soiling import monthly_soiling_rates
-# from rdtools.soiling import annual_soiling_ratios
+#from rdtools.soiling import soiling_srr
+#from rdtools.soiling import monthly_soiling_rates
+#from rdtools.soiling import annual_soiling_ratios
+from rdtools.analysis_chains import TrendAnalysis 
 from rdtools.plotting import degradation_summary_plots
 # from rdtools.plotting import soiling_monte_carlo_plot
 # from rdtools.plotting import soiling_interval_plot