You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: myst_nbs/pymc3_howto/lasso_block_update.myst.md
+74-29
Original file line number
Diff line number
Diff line change
@@ -6,74 +6,119 @@ jupytext:
6
6
format_version: 0.13
7
7
jupytext_version: 1.13.7
8
8
kernelspec:
9
-
display_name: Python PyMC (Dev)
9
+
display_name: Python 3 (ipykernel)
10
10
language: python
11
-
name: pymc-dev-py38
11
+
name: python3
12
12
---
13
13
14
+
(lasso_block_update)=
14
15
# Lasso regression with block updating
15
16
16
-
Sometimes, it is very useful to update a set of parameters together. For example, variables that are highly correlated are often good to update together. In PyMC 3 block updating is simple, as example will demonstrate.
17
-
18
-
Here we have a LASSO regression model where the two coefficients are strongly correlated. Normally, we would define the coefficient parameters as a single random variable, but here we define them separately to show how to do block updates.
:author: Chris Fonnesbeck, Raul Maldonado, Michael Osthege, Thomas Wiecki, Lorenzo Toniazzi
21
+
:::
21
22
22
23
```{code-cell} ipython3
23
-
%matplotlib inline
24
+
:tags: []
24
25
26
+
%matplotlib inline
25
27
import arviz as az
28
+
import matplotlib.pyplot as plt
26
29
import numpy as np
27
30
import pymc as pm
28
31
29
-
from matplotlib import pylab
32
+
print(f"Running on PyMC v{pm.__version__}")
33
+
```
34
+
35
+
```{code-cell} ipython3
36
+
RANDOM_SEED = 8927
37
+
rng = np.random.default_rng(RANDOM_SEED)
38
+
az.style.use("arviz-darkgrid")
39
+
```
40
+
41
+
Sometimes, it is very useful to update a set of parameters together. For example, variables that are highly correlated are often good to update together. In PyMC block updating is simple. This will be demonstrated using the parameter `step` of {class}`pymc.sample`.
42
+
43
+
Here we have a [LASSO regression model](https://en.wikipedia.org/wiki/Lasso_(statistics)#Bayesian_interpretation) where the two coefficients are strongly correlated. Normally, we would define the coefficient parameters as a single random variable, but here we define them separately to show how to do block updates.
44
+
45
+
First we generate some fake data.
30
46
31
-
d = np.random.normal(size=(3, 30))
32
-
d1 = d[0] + 4
33
-
d2 = d[1] + 4
34
-
yd = 0.2 * d1 + 0.3 * d2 + d[2]
47
+
```{code-cell} ipython3
48
+
x = rng.standard_normal(size=(3, 30))
49
+
x1 = x[0] + 4
50
+
x2 = x[1] + 4
51
+
noise = x[2]
52
+
y_obs = x1 * 0.2 + x2 * 0.3 + noise
35
53
```
36
54
37
55
Then define the random variables.
38
56
39
57
```{code-cell} ipython3
40
-
lam = 3
58
+
:tags: []
59
+
60
+
lam = 3000
41
61
42
62
with pm.Model() as model:
43
-
s = pm.Exponential("s", 1)
44
-
tau = pm.Uniform("tau", 0, 1000)
63
+
sigma = pm.Exponential("sigma", 1)
64
+
tau = pm.Uniform("tau", 0, 1)
45
65
b = lam * tau
46
-
m1 = pm.Laplace("m1", 0, b)
47
-
m2 = pm.Laplace("m2", 0, b)
66
+
beta1 = pm.Laplace("beta1", 0, b)
67
+
beta2 = pm.Laplace("beta2", 0, b)
48
68
49
-
p = d1 * m1 + d2 * m2
69
+
mu = x1 * beta1 + x2 * beta2
50
70
51
-
y = pm.Normal("y", mu=p, sigma=s, observed=yd)
71
+
y = pm.Normal("y", mu=mu, sigma=sigma, observed=y_obs)
52
72
```
53
73
54
-
For most samplers, including Metropolis and HamiltonianMC, simply pass a list of variables to sample as a block. This works with both scalar and array parameters.
74
+
For most samplers, including {class}`pymc.Metropolis` and {class}`pymc.HamiltonianMC`, simply pass a list of variables to sample as a block. This works with both scalar and array parameters.
* Authored by [Chris Fonnesbeck](https://github.com/fonnesbeck) in Dec, 2020
107
+
* Updated by [Raul Maldonado](https://github.com/CloudChaoszero) in Jan, 2021
108
+
* Updated by Raul Maldonado in Mar, 2021
109
+
* Reexecuted by [Thomas Wiecki](https://github.com/twiecki) and [Michael Osthege](https://github.com/michaelosthege) with PyMC v4 in Jan, 2022 ([pymc-examples#264](https://github.com/pymc-devs/pymc-examples/pull/264))
110
+
* Updated by [Lorenzo Toniazzi](https://github.com/ltoniazzi) in Feb, 2022 ([pymc-examples#279](https://github.com/pymc-devs/pymc-examples/pull/279))
0 commit comments