Skip to content

363 example tensor for cp als needs to have cp structure #392

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

363 example tensor for cp als needs to have cp structure #392

Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
836e6c3
Checkpoint
dmdunla Feb 3, 2025
090f892
Running nbstripout
dmdunla Feb 5, 2025
cf1f99e
Running ruff format
dmdunla Feb 5, 2025
42adca7
Run nbstripout
dmdunla Feb 5, 2025
7517213
Checkpoint
dmdunla Mar 21, 2025
65dc774
Merge branch 'main' into 363-example-tensor-for-cp_als-needs-to-have-…
dmdunla Mar 21, 2025
c1745e6
Ruff formatting
dmdunla Mar 21, 2025
8dfda3f
Running nbstripout
dmdunla Mar 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
182 changes: 110 additions & 72 deletions docs/source/tutorial/algorithm_cp_als.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Generate data"
"## Generate low-rank data tensor"
]
},
{
Expand All @@ -45,18 +45,27 @@
"metadata": {},
"outputs": [],
"source": [
"# Pick the shape and rank\n",
"R = 3\n",
"np.random.seed(0) # Set seed for reproducibility\n",
"X = ttb.tenrand(shape=(6, 8, 10))"
"# Choose the rank and shape\n",
"R = 2\n",
"tensor_shape = (3, 4, 5)\n",
"\n",
"# Set the random seed for reproducibility\n",
"np.random.seed(0)\n",
"\n",
"# Create a low-rank dense tensor from a Kruskal tensor (i.e., ktensor)\n",
"factor_matrices = [np.random.randn(s, R) for s in tensor_shape]\n",
"M_true = ttb.ktensor(factor_matrices) # true solution\n",
"X = M_true.to_tensor()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Basic call to the method, specifying the data tensor and its rank\n",
"This uses a *random* initial guess. At each iteration, it reports the *fit* `f` which is defined as \n",
"## Run `cp_als` using default parameters\n",
"The `cp_als` method has two required inputs: a data tensor (X) and the rank of the CP model (R) to compute.\n",
"\n",
"By default, `cp_als` uses a *random* initial guess. At each iteration, it reports the *fit* `f` which is defined for a data tensor `X` and CP model `M` as \n",
"```\n",
"f = 1 - ( X.norm()**2 + M.norm()**2 - 2*<X,M> ) / X.norm()\n",
"``` \n",
Expand All @@ -70,19 +79,18 @@
"outputs": [],
"source": [
"# Compute a solution with final ktensor stored in M1\n",
"np.random.seed(0) # Set seed for reproducibility\n",
"short_tutorial = 10 # Cut off solve early for demo\n",
"M1 = ttb.cp_als(X, R, maxiters=short_tutorial)"
"np.random.seed(1) # Set the random seed for reproducibility\n",
"M1, M1_init, M1_info = ttb.cp_als(X, R)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Since we set only a single output, `M1` is actually a *tuple* containing:\n",
"1. `M1[0]`: the solution as a `ktensor`. \n",
"2. `M1[1]`: the initial guess as a `ktensor` that was generated at runtime since no initial guess was provided. \n",
"3. `M1[2]`: a dictionary containing runtime information with keys:\n",
"The `cp_als` method returns the following:\n",
"1. `M1`: the solution as a `ktensor`. \n",
"2. `M1_init`: the initial guess as a `ktensor` that was used in computing the solution. \n",
"3. `M1_info`: a `dict` containing runtime information with keys:\n",
" * `params`: parameters used by `cp_als`\n",
" * `iters`: number of iterations performed\n",
" * `normresidual`: the norm of the residual `X.norm()**2 + M.norm()**2 - 2*<X,M>`\n",
Expand All @@ -95,17 +103,23 @@
"metadata": {},
"outputs": [],
"source": [
"print(f\"M1[2]['params']: {M1[2]['params']}\")\n",
"print(f\"M1[2]['iters']: {M1[2]['iters']}\")\n",
"print(f\"M1[2]['normresidual']: {M1[2]['normresidual']}\")\n",
"print(f\"M1[2]['fit']: {M1[2]['fit']}\")"
"print(\"M1_info:\")\n",
"for k, v in M1_info.items():\n",
" print(f\"\\t{k}: {v}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Run `cp_als` using different initial guesses"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Run again with a different initial guess, output the initial guess."
"Different random initial guesses can be generated and used by setting different random seeds (via `numpy`). You can also explicitly set the `init` parameter to `\"random\"` to make it clear that the default random initialization is being used, although this is not necessary as illustrated above."
]
},
{
Expand All @@ -114,16 +128,15 @@
"metadata": {},
"outputs": [],
"source": [
"np.random.seed(1) # Set seed for reproducibility\n",
"M2bad, Minit, _ = ttb.cp_als(X, R, maxiters=short_tutorial)"
"np.random.seed(2) # Set seed for reproducibility\n",
"M2, M2_init, _ = ttb.cp_als(X, R, init=\"random\") # leaving third output unassigned"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Increase the maximum number of iterations\n",
"Note that the previous run kicked out at only 10 iterations, before reaching the specified convegence tolerance. Let's increase the maximum number of iterations and try again, using the same initial guess."
"A specific `ktensor` can also be used as an initial guess. As an example, using the same initial guess (and all other parameters) as the previous run of `cp_als` gives the exact same solution."
]
},
{
Expand All @@ -132,16 +145,14 @@
"metadata": {},
"outputs": [],
"source": [
"less_short_tutorial = 10 * short_tutorial\n",
"M2 = ttb.cp_als(X, R, maxiters=less_short_tutorial, init=Minit)"
"M2_rerun, _, _ = ttb.cp_als(X, R, init=M2_init)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Compare the two solutions\n",
"Use the `ktensor` `score()` member function to compare the two solutions. A score of 1 indicates a perfect match."
"The `\"nvecs\"` initialization option computes the initial guess using the eigenvectors of the outer product of the matricized data tensor. This initialization process will require more computation than the default random initialization, but it can often lead to better solutions in fewer iterations."
]
},
{
Expand All @@ -150,16 +161,21 @@
"metadata": {},
"outputs": [],
"source": [
"M1_ktns = M1[0]\n",
"M2_ktns = M2[0]\n",
"score = M1_ktns.score(M2_ktns)"
"M2_nvecs, _, _ = ttb.cp_als(X, R, init=\"nvecs\")\n",
"# score_M3 = M3.score(M_true)\n",
"# print(f\"Score of M2 and M_true: {score_M2[0]}\")\n",
"# print(f\"Score of M3 and M_true: {score_M3[0]}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here, `score()` returned a tuple `score` with the score as the first element:"
"## Evaluate and compare the outputs of `cp_als`\n",
"Use the `ktensor.score()` method to compare outputs of `cp_als` to the true solution if known. In the examples above, the data tensor was generated from a `ktensor`, `M_true`, which can be used to evaluate solutions computed using `cp_als`. A score of 1 indicates a perfect match.\n",
"\n",
"\n",
"Note that the `ktensor.score()` method returns a tuple with the score as the first element and other information related to the score computation as the remaining elements. See the `ktensor` documentation for more information about the return values."
]
},
{
Expand All @@ -168,22 +184,39 @@
"metadata": {},
"outputs": [],
"source": [
"score[0]"
"score_M1 = M1.score(M_true) # not a good solution\n",
"score_M2 = M2.score(M_true) # a better solution\n",
"score_M2_nvecs = M2_nvecs.score(M_true) # an even better solution\n",
"\n",
"print(f\"Score of M1 and M_true: {score_M1[0]}\")\n",
"print(f\"Score of M2 and M_true: {score_M2[0]}\")\n",
"print(f\"Score of M2_nvecs and M_true: {score_M2_nvecs[0]}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"See the `ktensor` documentation for more information about the return values of `score()`."
"When two solutions match, as is the case with `M2` and `M2_rerun`, the score will be 1 (up to floating point error)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"score_M2_rerun = M2.score(M2_rerun)\n",
"\n",
"print(f\"Score of M2 and M2_rerun: {score_M2_rerun[0]}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Rerun with same initial guess\n",
"Using the same initial guess (and all other parameters) gives the exact same solution."
"## Increase the maximum number of iterations\n",
"Note that the previous run kicked out at only 10 iterations, before reaching the specified convegence tolerance. Let's increase the maximum number of iterations and try again, using the same initial guess."
]
},
{
Expand All @@ -192,10 +225,8 @@
"metadata": {},
"outputs": [],
"source": [
"M2alt = ttb.cp_als(X, R, maxiters=less_short_tutorial, init=Minit)\n",
"M2alt_ktns = M2alt[0]\n",
"score = M2_ktns.score(M2alt_ktns) # Score of 1 indicates the same solution\n",
"print(f\"Score: {score[0]}.\")"
"more_iters = 10 * few_iters\n",
"M2_better, _, _ = ttb.cp_als(X, R, maxiters=more_iters, init=M_true)"
]
},
{
Expand All @@ -212,7 +243,7 @@
"metadata": {},
"outputs": [],
"source": [
"M2alt2 = ttb.cp_als(X, R, maxiters=less_short_tutorial, init=Minit, printitn=20)"
"M = ttb.cp_als(X, R, printitn=5)"
]
},
{
Expand All @@ -229,33 +260,23 @@
"metadata": {},
"outputs": [],
"source": [
"M2alt2 = ttb.cp_als(X, R, printitn=0) # No output"
"M = ttb.cp_als(X, R, printitn=0) # No output"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Use HOSVD initial guess\n",
"Use the `\"nvecs\"` option to use the leading mode-$n$ singular vectors as the initial guess."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"M3 = ttb.cp_als(X, R, init=\"nvecs\", printitn=20)\n",
"s = M2[0].score(M3[0])\n",
"print(f\"score(M2,M3) = {s[0]}\")"
"## Use initial guess based \n",
"Use the `\"nvecs\"` option to initialize `cp_als` with the leading mode-$n$ singular vectors of the input tensor. This initialization process will require more computation than the default `\"random` initialization in general, but it can lead to better solutions."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Change the order of the dimensions in CP"
"## Change the order of the dimensions in CP\n",
"Changing the order of the dimensions in which `cp_als` iterates over the input tensor can lead to a different solution."
]
},
{
Expand All @@ -264,16 +285,19 @@
"metadata": {},
"outputs": [],
"source": [
"M4, _, info = ttb.cp_als(X, 3, dimorder=[1, 2, 0], init=\"nvecs\", printitn=20)\n",
"s = M2[0].score(M4)\n",
"print(f\"score(M2,M4) = {s[0]}\")"
"M4, _, M4_info = ttb.cp_als(\n",
" X, R, maxiters=few_iters, init=\"nvecs\", printitn=0, dimorder=[2, 1, 0]\n",
")\n",
"score_M4 = M4.score(M_true)\n",
"print(f\"Score of M3 and M_true: {score_M3[0]}\")\n",
"print(f\"Score of M4 and M_true: {score_M4[0]}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the last example, we also collected the third output argument `info` which has runtime information in it. The field `info[\"iters\"]` has the total number of iterations. The field `info[\"params\"]` has the information used to run the method. Unless the initialization method is `\"random\"`, passing the parameters back to the method will yield the exact same results."
"In the last example, we also collected the third output argument `M4_info` which has runtime information in it. The field `M4_info[\"iters\"]` has the total number of iterations. The field `M4_info[\"params\"]` has the information used to run the method. Unless the initialization method is `\"random\"`, passing the parameters back to the method will yield the exact same results."
]
},
{
Expand All @@ -282,17 +306,25 @@
"metadata": {},
"outputs": [],
"source": [
"M4alt, _, info = ttb.cp_als(X, 3, **info[\"params\"])\n",
"s = M4alt.score(M4)\n",
"print(f\"score(M4alt,M4) = {s[0]}\")"
"M4_rerun, _, M4_rerun_info = ttb.cp_als(X, R, init=\"nvecs\", **M4_info[\"params\"])\n",
"score_M4_rerun = M4.score(M4_rerun)\n",
"print(f\"Score of M4 and M4_rerun: {score_M4_rerun[0]}\")\n",
"\n",
"print(\"M4_info:\")\n",
"for k, v in M4_info.items():\n",
" print(f\"\\t{k}: {v}\")\n",
"\n",
"print(\"M4_rerun_info:\")\n",
"for k, v in M4_rerun_info.items():\n",
" print(f\"\\t{k}: {v}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Change the tolerance\n",
"It's also possible to loosen or tighten the tolerance on the change in the fit. You may need to increase the number of iterations for it to converge."
"## Change the stopping tolerance\n",
"It's also possible to loosen or tighten the stopping tolerance on the change in the fit. Note that you may need to increase the number of iterations for it to converge."
]
},
{
Expand All @@ -301,7 +333,7 @@
"metadata": {},
"outputs": [],
"source": [
"M5 = ttb.cp_als(X, 3, init=\"nvecs\", stoptol=1e-12, printitn=100)"
"M5 = ttb.cp_als(X, R, init=\"nvecs\", maxiters=1000, stoptol=1e-12, printitn=100)"
]
},
{
Expand All @@ -318,17 +350,23 @@
"metadata": {},
"outputs": [],
"source": [
"X = ttb.ktensor(\n",
"# Create rank-2 tensor\n",
"X2 = ttb.ktensor(\n",
" factor_matrices=[\n",
" np.array([[1.0, 1.0], [1.0, -10.0]]),\n",
" np.array([[1.0, 1.0], [1.0, -10.0]]),\n",
" np.array([[1.0, 1.0], [-1.0, -10.0]]),\n",
" np.array([[1.0, 1.0], [-2.0, -10.0]]),\n",
" ],\n",
" weights=np.array([1.0, 1.0]),\n",
")\n",
"M1 = ttb.cp_als(X, 2, printitn=1, init=ttb.ktensor(X.factor_matrices))\n",
"print(M1[0]) # default behavior, fixsigns called\n",
"M2 = ttb.cp_als(X, 2, printitn=1, init=ttb.ktensor(X.factor_matrices), fixsigns=False)\n",
"print(M2[0]) # fixsigns not called"
"print(f\"X2=\\n{X2}\\n\")\n",
"\n",
"M_fixsigns, _, _ = ttb.cp_als(X2, 2, printitn=0, init=ttb.ktensor(X2.factor_matrices))\n",
"print(f\"M_fixsigns=\\n{M_fixsigns}\\n\") # default behavior, fixsigns called\n",
"\n",
"M_no_fixsigns, _, _ = ttb.cp_als(\n",
" X2, 2, printitn=0, init=ttb.ktensor(X2.factor_matrices), fixsigns=False\n",
")\n",
"print(f\"M_no_fixsigns=\\n{M_no_fixsigns}\\n\") # fixsigns not called"
]
},
{
Expand Down
Loading