Skip to content

chore: upgrade the notebooks, VCS versioning, clean nnet traces #96

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
arogozhnikov merged 7 commits into from
Jul 1, 2025
Merged

chore: upgrade the notebooks, VCS versioning, clean nnet traces #96

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
8927e46
chore: upgrade the notebooks, VCS versioning and remove any traces of…
jonas-eschle Jun 17, 2025
ba709f3
chore: run pre-commit
jonas-eschle Jun 17, 2025
e87bccd
docs: build docs
jonas-eschle Jun 17, 2025
08760e0
docs: upgrade notebooks
jonas-eschle Jun 30, 2025
e6a031d
correct comment
arogozhnikov Jul 1, 2025
1d660b5
remove matplotlib inline
arogozhnikov Jul 1, 2025
4fb19c8
minor
arogozhnikov Jul 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,3 +57,10 @@ docsrc/tmp*

# OS X garbage
.DS_Store

# version file
/hep_ml/_version.py

# data files
/data/*.root
/data/MiniBooNE_PID.txt
3 changes: 0 additions & 3 deletions .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@ ci:
autoupdate_commit_msg: "chore: update pre-commit hooks"
autofix_commit_msg: "style: pre-commit fixes"
autoupdate_schedule: quarterly
# skip:
# - oxipng


repos:
Expand Down Expand Up @@ -32,7 +30,6 @@ repos:
- id: check-docstring-first
- id: debug-statements


- repo: https://github.com/pre-commit/pygrep-hooks
rev: v1.10.0
hooks:
Expand Down
3 changes: 1 addition & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@
* __uBoost__ optimized implementation inside
* __UGradientBoosting__ (with different losses, specially __FlatnessLoss__ is of high interest)
* measures of uniformity (see **hep_ml.metrics**)
* advanced losses for classification, regression and ranking for __UGradientBoosting__ (see **hep_ml.losses**).
* **hep_ml.nnet** - theano-based flexible neural networks
* advanced losses for classification, regression and ranking for __UGradientBoosting__ (see **hep_ml.losses**).
* **hep_ml.reweight** - reweighting multidimensional distributions <br />
(_multi_ here means 2, 3, 5 and more dimensions - see GBReweighter!)
* **hep_ml.splot** - minimalistic sPlot-ting
Expand Down
3 changes: 1 addition & 2 deletions builddocs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@
# script should be sourced from repo root
# documentation is served from docs/ folder, not separate branch

pip install sphinx_rtd_theme
pip install -e .
pip install -e .[docs]
pushd docsrc
# Forcing to ignore caches
make SPHINXOPTS="-E" html
Expand Down
375 changes: 156 additions & 219 deletions docs/_modules/hep_ml/gradientboosting.html
View file
Open in desktop

Large diffs are not rendered by default.

778 changes: 421 additions & 357 deletions docs/_modules/hep_ml/losses.html
View file
Open in desktop

Large diffs are not rendered by default.

369 changes: 133 additions & 236 deletions docs/_modules/hep_ml/metrics.html
View file
Open in desktop

Large diffs are not rendered by default.

984 changes: 0 additions & 984 deletions docs/_modules/hep_ml/nnet.html
View file
Open in desktop

This file was deleted.

248 changes: 75 additions & 173 deletions docs/_modules/hep_ml/preprocessing.html
View file
Open in desktop

Large diffs are not rendered by default.

363 changes: 149 additions & 214 deletions docs/_modules/hep_ml/reweight.html
View file
Open in desktop

Large diffs are not rendered by default.

258 changes: 88 additions & 170 deletions docs/_modules/hep_ml/speedup.html
View file
Open in desktop

Large diffs are not rendered by default.

183 changes: 39 additions & 144 deletions docs/_modules/hep_ml/splot.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,171 +1,79 @@


<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<html class="writer-html5" lang="en" data-content_root="../../">
<head>
<meta charset="utf-8" />

<meta name="viewport" content="width=device-width, initial-scale=1.0" />

<title>hep_ml.splot &mdash; hep_ml 0.7.0 documentation</title>



<link rel="stylesheet" href="../../_static/css/theme.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../_static/css/theme.css" type="text/css" />
<title>hep_ml.splot &mdash; hep_ml 0.7.4.dev5+gba709f3.d20250617 documentation</title>
<link rel="stylesheet" type="text/css" href="../../_static/pygments.css?v=03e43079" />
<link rel="stylesheet" type="text/css" href="../../_static/css/theme.css?v=e59714d7" />










<!--[if lt IE 9]>
<script src="../../_static/js/html5shiv.min.js"></script>
<![endif]-->


<script type="text/javascript" id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
<script data-url_root="../../" id="documentation_options" src="../../_static/documentation_options.js"></script>
<script src="../../_static/jquery.js"></script>
<script src="../../_static/underscore.js"></script>
<script src="../../_static/doctools.js"></script>

<script type="text/javascript" src="../../_static/js/theme.js"></script>


<script src="../../_static/documentation_options.js?v=a931b840"></script>
<script src="../../_static/doctools.js?v=9a2dae69"></script>
<script src="../../_static/sphinx_highlight.js?v=dc90522c"></script>
<script src="../../_static/js/theme.js"></script>
<link rel="index" title="Index" href="../../genindex.html" />
<link rel="search" title="Search" href="../../search.html" />
</head>

<body class="wy-body-for-nav">


<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">

<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" >



<a href="../../index.html" class="icon icon-home"> hep_ml



<a href="../../index.html" class="icon icon-home">
hep_ml
</a>




<div class="version">
0.7.0
</div>




<div role="search">
<form id="rtd-search-form" class="wy-form" action="../../search.html" method="get">
<input type="text" name="q" placeholder="Search docs" />
<input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>


</div>


<div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">






</div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../index.html">hep_ml documentation</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../gb.html">Gradient boosting</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../losses.html">Losses for Gradient Boosting</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../uboost.html">uBoost</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../metrics.html">Metric functions</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../nnet.html">Neural networks</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../preprocessing.html">Preprocessing data</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../reweight.html">Reweighting algorithms</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../speedup.html">Fast predictions</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../splot.html">sPlot</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../notebooks.html">Code Examples</a></li>
</ul>



</div>

</div>
</nav>

<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">


<nav class="wy-nav-top" aria-label="top navigation">

<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="../../index.html">hep_ml</a>

</nav>


<div class="wy-nav-content">

<div class="rst-content">



















<div role="navigation" aria-label="breadcrumbs navigation">

<div role="navigation" aria-label="Page navigation">
<ul class="wy-breadcrumbs">

<li><a href="../../index.html" class="icon icon-home"></a> &raquo;</li>

<li><a href="../index.html">Module code</a> &raquo;</li>

<li>hep_ml.splot</li>


<li><a href="../../index.html" class="icon icon-home" aria-label="Home"></a></li>
<li class="breadcrumb-item"><a href="../index.html">Module code</a></li>
<li class="breadcrumb-item active">hep_ml.splot</li>
<li class="wy-breadcrumbs-aside">

</li>

</ul>


<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">

<h1>Source code for hep_ml.splot</h1><div class="highlight"><pre>
<span></span><span class="sd">&quot;&quot;&quot;</span>
<span class="sd">**sPlot** is reweighting technique frequently used in HEP to reconstruct the distributions of features in mixture.</span>
Expand All @@ -188,16 +96,18 @@ <h1>Source code for hep_ml.splot</h1><div class="highlight"><pre>

<span class="sd">&quot;&quot;&quot;</span>

<span class="kn">from</span> <span class="nn">__future__</span> <span class="kn">import</span> <span class="n">division</span><span class="p">,</span> <span class="n">print_function</span><span class="p">,</span> <span class="n">absolute_import</span>
<span class="kn">import</span> <span class="nn">pandas</span>
<span class="kn">import</span> <span class="nn">numpy</span>
<span class="kn">from</span> <span class="nn">.commonutils</span> <span class="kn">import</span> <span class="n">check_sample_weight</span>
<span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span>
<span class="kn">import</span><span class="w"> </span><span class="nn">pandas</span>

<span class="kn">from</span><span class="w"> </span><span class="nn">.commonutils</span><span class="w"> </span><span class="kn">import</span> <span class="n">check_sample_weight</span>

<span class="n">__author__</span> <span class="o">=</span> <span class="s1">&#39;Alex Rogozhnikov&#39;</span>
<span class="n">__author__</span> <span class="o">=</span> <span class="s2">&quot;Alex Rogozhnikov&quot;</span>


<div class="viewcode-block" id="compute_sweights"><a class="viewcode-back" href="../../splot.html#hep_ml.splot.compute_sweights">[docs]</a><span class="k">def</span> <span class="nf">compute_sweights</span><span class="p">(</span><span class="n">probabilities</span><span class="p">,</span> <span class="n">sample_weight</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;Computes sWeights based on probabilities obtained from distribution fit.</span>
<div class="viewcode-block" id="compute_sweights">
<a class="viewcode-back" href="../../splot.html#hep_ml.splot.compute_sweights">[docs]</a>
<span class="k">def</span><span class="w"> </span><span class="nf">compute_sweights</span><span class="p">(</span><span class="n">probabilities</span><span class="p">,</span> <span class="n">sample_weight</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;Computes sWeights based on probabilities obtained from distribution fit.</span>

<span class="sd"> :param probabilities: pandas.DataFrame with probabilities of shape [n_samples, n_classes].</span>
<span class="sd"> These probabilities are obtained after fit (typically, mass fit).</span>
Expand All @@ -210,11 +120,11 @@ <h1>Source code for hep_ml.splot</h1><div class="highlight"><pre>
<span class="c1"># checking sample_weight</span>
<span class="n">sample_weight</span> <span class="o">=</span> <span class="n">check_sample_weight</span><span class="p">(</span><span class="n">probabilities</span><span class="p">,</span> <span class="n">sample_weight</span><span class="o">=</span><span class="n">sample_weight</span><span class="p">)</span>
<span class="c1"># checking that all weights are positive</span>
<span class="k">assert</span> <span class="n">numpy</span><span class="o">.</span><span class="n">all</span><span class="p">(</span><span class="n">sample_weight</span> <span class="o">&gt;=</span> <span class="mi">0</span><span class="p">),</span> <span class="s1">&#39;sample weight are expected to be non-negative&#39;</span>
<span class="k">assert</span> <span class="n">numpy</span><span class="o">.</span><span class="n">all</span><span class="p">(</span><span class="n">sample_weight</span> <span class="o">&gt;=</span> <span class="mi">0</span><span class="p">),</span> <span class="s2">&quot;sample weight are expected to be non-negative&quot;</span>

<span class="n">p</span> <span class="o">=</span> <span class="n">numpy</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="n">probabilities</span><span class="p">)</span>
<span class="c1"># checking that probabilities sum up to 1.</span>
<span class="k">assert</span> <span class="n">numpy</span><span class="o">.</span><span class="n">allclose</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">),</span> <span class="mi">1</span><span class="p">,</span> <span class="n">atol</span><span class="o">=</span><span class="mf">1e-3</span><span class="p">),</span> <span class="s1">&#39;sum of probabilities is not equal to 1.&#39;</span>
<span class="k">assert</span> <span class="n">numpy</span><span class="o">.</span><span class="n">allclose</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">),</span> <span class="mi">1</span><span class="p">,</span> <span class="n">atol</span><span class="o">=</span><span class="mf">1e-3</span><span class="p">),</span> <span class="s2">&quot;sum of probabilities is not equal to 1.&quot;</span>

<span class="c1"># computations</span>
<span class="n">initial_stats</span> <span class="o">=</span> <span class="p">(</span><span class="n">p</span> <span class="o">*</span> <span class="n">sample_weight</span><span class="p">[:,</span> <span class="n">numpy</span><span class="o">.</span><span class="n">newaxis</span><span class="p">])</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
Expand All @@ -224,49 +134,34 @@ <h1>Source code for hep_ml.splot</h1><div class="highlight"><pre>
<span class="c1"># Final formula</span>
<span class="n">sweights</span> <span class="o">=</span> <span class="n">p</span><span class="o">.</span><span class="n">dot</span><span class="p">(</span><span class="n">V</span><span class="p">)</span> <span class="o">*</span> <span class="n">sample_weight</span><span class="p">[:,</span> <span class="n">numpy</span><span class="o">.</span><span class="n">newaxis</span><span class="p">]</span>
<span class="k">return</span> <span class="n">pandas</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">sweights</span><span class="p">,</span> <span class="n">columns</span><span class="o">=</span><span class="n">probabilities</span><span class="o">.</span><span class="n">keys</span><span class="p">())</span></div>

</pre></div>

</div>

</div>
<footer>

<hr/>

<div role="contentinfo">
<p>
&#169; Copyright 2015-2017, Yandex; Alex Rogozhnikov and contributors.

</p>
<p>&#169; Copyright 2015-2017, Yandex; Alex Rogozhnikov and contributors.</p>
</div>



Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a


Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>

provided by <a href="https://readthedocs.org">Read the Docs</a>.
provided by <a href="https://readthedocs.org">Read the Docs</a>.


</footer>
</div>
</div>

</section>

</div>


<script type="text/javascript">
<script>
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>





</script>

</body>
</html>
</html>
Loading
Loading