diff --git a/docs/source/tutorial/algorithm_cp_als.ipynb b/docs/source/tutorial/algorithm_cp_als.ipynb index a74c943..062f6f0 100644 --- a/docs/source/tutorial/algorithm_cp_als.ipynb +++ b/docs/source/tutorial/algorithm_cp_als.ipynb @@ -36,7 +36,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Generate data" + "## Generate low-rank data tensor" ] }, { @@ -45,18 +45,27 @@ "metadata": {}, "outputs": [], "source": [ - "# Pick the shape and rank\n", - "R = 3\n", - "np.random.seed(0) # Set seed for reproducibility\n", - "X = ttb.tenrand(shape=(6, 8, 10))" + "# Choose the rank and shape\n", + "R = 2\n", + "tensor_shape = (3, 4, 5)\n", + "\n", + "# Set the random seed for reproducibility\n", + "np.random.seed(0)\n", + "\n", + "# Create a low-rank dense tensor from a Kruskal tensor (i.e., ktensor)\n", + "factor_matrices = [np.random.randn(s, R) for s in tensor_shape]\n", + "M_true = ttb.ktensor(factor_matrices) # true solution\n", + "X = M_true.to_tensor()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Basic call to the method, specifying the data tensor and its rank\n", - "This uses a *random* initial guess. At each iteration, it reports the *fit* `f` which is defined as \n", + "## Run `cp_als` using default parameters\n", + "The `cp_als` method has two required inputs: a data tensor (X) and the rank of the CP model (R) to compute.\n", + "\n", + "By default, `cp_als` uses a *random* initial guess. At each iteration, it reports the *fit* `f` which is defined for a data tensor `X` and CP model `M` as \n", "```\n", "f = 1 - ( X.norm()**2 + M.norm()**2 - 2* ) / X.norm()\n", "``` \n", @@ -70,19 +79,18 @@ "outputs": [], "source": [ "# Compute a solution with final ktensor stored in M1\n", - "np.random.seed(0) # Set seed for reproducibility\n", - "short_tutorial = 10 # Cut off solve early for demo\n", - "M1 = ttb.cp_als(X, R, maxiters=short_tutorial)" + "np.random.seed(1) # Set the random seed for reproducibility\n", + "M1, M1_init, M1_info = ttb.cp_als(X, R)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Since we set only a single output, `M1` is actually a *tuple* containing:\n", - "1. `M1[0]`: the solution as a `ktensor`. \n", - "2. `M1[1]`: the initial guess as a `ktensor` that was generated at runtime since no initial guess was provided. \n", - "3. `M1[2]`: a dictionary containing runtime information with keys:\n", + "The `cp_als` method returns the following:\n", + "1. `M1`: the solution as a `ktensor`. \n", + "2. `M1_init`: the initial guess as a `ktensor` that was used in computing the solution. \n", + "3. `M1_info`: a `dict` containing runtime information with keys:\n", " * `params`: parameters used by `cp_als`\n", " * `iters`: number of iterations performed\n", " * `normresidual`: the norm of the residual `X.norm()**2 + M.norm()**2 - 2*`\n", @@ -95,17 +103,23 @@ "metadata": {}, "outputs": [], "source": [ - "print(f\"M1[2]['params']: {M1[2]['params']}\")\n", - "print(f\"M1[2]['iters']: {M1[2]['iters']}\")\n", - "print(f\"M1[2]['normresidual']: {M1[2]['normresidual']}\")\n", - "print(f\"M1[2]['fit']: {M1[2]['fit']}\")" + "print(\"M1_info:\")\n", + "for k, v in M1_info.items():\n", + " print(f\"\\t{k}: {v}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Run `cp_als` using different initial guesses" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Run again with a different initial guess, output the initial guess." + "Different random initial guesses can be generated and used by setting different random seeds (via `numpy`). You can also explicitly set the `init` parameter to `\"random\"` to make it clear that the default random initialization is being used, although this is not necessary as illustrated above." ] }, { @@ -114,16 +128,15 @@ "metadata": {}, "outputs": [], "source": [ - "np.random.seed(1) # Set seed for reproducibility\n", - "M2bad, Minit, _ = ttb.cp_als(X, R, maxiters=short_tutorial)" + "np.random.seed(2) # Set seed for reproducibility\n", + "M2, M2_init, _ = ttb.cp_als(X, R, init=\"random\") # leaving third output unassigned" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Increase the maximum number of iterations\n", - "Note that the previous run kicked out at only 10 iterations, before reaching the specified convegence tolerance. Let's increase the maximum number of iterations and try again, using the same initial guess." + "A specific `ktensor` can also be used as an initial guess. As an example, using the same initial guess (and all other parameters) as the previous run of `cp_als` gives the exact same solution." ] }, { @@ -132,16 +145,14 @@ "metadata": {}, "outputs": [], "source": [ - "less_short_tutorial = 10 * short_tutorial\n", - "M2 = ttb.cp_als(X, R, maxiters=less_short_tutorial, init=Minit)" + "M2_rerun, _, _ = ttb.cp_als(X, R, init=M2_init)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Compare the two solutions\n", - "Use the `ktensor` `score()` member function to compare the two solutions. A score of 1 indicates a perfect match." + "The `\"nvecs\"` initialization option computes the initial guess using the eigenvectors of the outer product of the matricized data tensor. This initialization process will require more computation than the default random initialization, but it can often lead to better solutions in fewer iterations." ] }, { @@ -150,16 +161,21 @@ "metadata": {}, "outputs": [], "source": [ - "M1_ktns = M1[0]\n", - "M2_ktns = M2[0]\n", - "score = M1_ktns.score(M2_ktns)" + "M2_nvecs, _, _ = ttb.cp_als(X, R, init=\"nvecs\")\n", + "# score_M3 = M3.score(M_true)\n", + "# print(f\"Score of M2 and M_true: {score_M2[0]}\")\n", + "# print(f\"Score of M3 and M_true: {score_M3[0]}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Here, `score()` returned a tuple `score` with the score as the first element:" + "## Evaluate and compare the outputs of `cp_als`\n", + "Use the `ktensor.score()` method to compare outputs of `cp_als` to the true solution if known. In the examples above, the data tensor was generated from a `ktensor`, `M_true`, which can be used to evaluate solutions computed using `cp_als`. A score of 1 indicates a perfect match.\n", + "\n", + "\n", + "Note that the `ktensor.score()` method returns a tuple with the score as the first element and other information related to the score computation as the remaining elements. See the `ktensor` documentation for more information about the return values." ] }, { @@ -168,22 +184,39 @@ "metadata": {}, "outputs": [], "source": [ - "score[0]" + "score_M1 = M1.score(M_true) # not a good solution\n", + "score_M2 = M2.score(M_true) # a better solution\n", + "score_M2_nvecs = M2_nvecs.score(M_true) # an even better solution\n", + "\n", + "print(f\"Score of M1 and M_true: {score_M1[0]}\")\n", + "print(f\"Score of M2 and M_true: {score_M2[0]}\")\n", + "print(f\"Score of M2_nvecs and M_true: {score_M2_nvecs[0]}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "See the `ktensor` documentation for more information about the return values of `score()`." + "When two solutions match, as is the case with `M2` and `M2_rerun`, the score will be 1 (up to floating point error)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "score_M2_rerun = M2.score(M2_rerun)\n", + "\n", + "print(f\"Score of M2 and M2_rerun: {score_M2_rerun[0]}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Rerun with same initial guess\n", - "Using the same initial guess (and all other parameters) gives the exact same solution." + "## Increase the maximum number of iterations\n", + "Note that the previous run kicked out at only 10 iterations, before reaching the specified convegence tolerance. Let's increase the maximum number of iterations and try again, using the same initial guess." ] }, { @@ -192,10 +225,8 @@ "metadata": {}, "outputs": [], "source": [ - "M2alt = ttb.cp_als(X, R, maxiters=less_short_tutorial, init=Minit)\n", - "M2alt_ktns = M2alt[0]\n", - "score = M2_ktns.score(M2alt_ktns) # Score of 1 indicates the same solution\n", - "print(f\"Score: {score[0]}.\")" + "more_iters = 10 * few_iters\n", + "M2_better, _, _ = ttb.cp_als(X, R, maxiters=more_iters, init=M_true)" ] }, { @@ -212,7 +243,7 @@ "metadata": {}, "outputs": [], "source": [ - "M2alt2 = ttb.cp_als(X, R, maxiters=less_short_tutorial, init=Minit, printitn=20)" + "M = ttb.cp_als(X, R, printitn=5)" ] }, { @@ -229,33 +260,23 @@ "metadata": {}, "outputs": [], "source": [ - "M2alt2 = ttb.cp_als(X, R, printitn=0) # No output" + "M = ttb.cp_als(X, R, printitn=0) # No output" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Use HOSVD initial guess\n", - "Use the `\"nvecs\"` option to use the leading mode-$n$ singular vectors as the initial guess." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "M3 = ttb.cp_als(X, R, init=\"nvecs\", printitn=20)\n", - "s = M2[0].score(M3[0])\n", - "print(f\"score(M2,M3) = {s[0]}\")" + "## Use initial guess based \n", + "Use the `\"nvecs\"` option to initialize `cp_als` with the leading mode-$n$ singular vectors of the input tensor. This initialization process will require more computation than the default `\"random` initialization in general, but it can lead to better solutions." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Change the order of the dimensions in CP" + "## Change the order of the dimensions in CP\n", + "Changing the order of the dimensions in which `cp_als` iterates over the input tensor can lead to a different solution." ] }, { @@ -264,16 +285,19 @@ "metadata": {}, "outputs": [], "source": [ - "M4, _, info = ttb.cp_als(X, 3, dimorder=[1, 2, 0], init=\"nvecs\", printitn=20)\n", - "s = M2[0].score(M4)\n", - "print(f\"score(M2,M4) = {s[0]}\")" + "M4, _, M4_info = ttb.cp_als(\n", + " X, R, maxiters=few_iters, init=\"nvecs\", printitn=0, dimorder=[2, 1, 0]\n", + ")\n", + "score_M4 = M4.score(M_true)\n", + "print(f\"Score of M3 and M_true: {score_M3[0]}\")\n", + "print(f\"Score of M4 and M_true: {score_M4[0]}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "In the last example, we also collected the third output argument `info` which has runtime information in it. The field `info[\"iters\"]` has the total number of iterations. The field `info[\"params\"]` has the information used to run the method. Unless the initialization method is `\"random\"`, passing the parameters back to the method will yield the exact same results." + "In the last example, we also collected the third output argument `M4_info` which has runtime information in it. The field `M4_info[\"iters\"]` has the total number of iterations. The field `M4_info[\"params\"]` has the information used to run the method. Unless the initialization method is `\"random\"`, passing the parameters back to the method will yield the exact same results." ] }, { @@ -282,17 +306,25 @@ "metadata": {}, "outputs": [], "source": [ - "M4alt, _, info = ttb.cp_als(X, 3, **info[\"params\"])\n", - "s = M4alt.score(M4)\n", - "print(f\"score(M4alt,M4) = {s[0]}\")" + "M4_rerun, _, M4_rerun_info = ttb.cp_als(X, R, init=\"nvecs\", **M4_info[\"params\"])\n", + "score_M4_rerun = M4.score(M4_rerun)\n", + "print(f\"Score of M4 and M4_rerun: {score_M4_rerun[0]}\")\n", + "\n", + "print(\"M4_info:\")\n", + "for k, v in M4_info.items():\n", + " print(f\"\\t{k}: {v}\")\n", + "\n", + "print(\"M4_rerun_info:\")\n", + "for k, v in M4_rerun_info.items():\n", + " print(f\"\\t{k}: {v}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Change the tolerance\n", - "It's also possible to loosen or tighten the tolerance on the change in the fit. You may need to increase the number of iterations for it to converge." + "## Change the stopping tolerance\n", + "It's also possible to loosen or tighten the stopping tolerance on the change in the fit. Note that you may need to increase the number of iterations for it to converge." ] }, { @@ -301,7 +333,7 @@ "metadata": {}, "outputs": [], "source": [ - "M5 = ttb.cp_als(X, 3, init=\"nvecs\", stoptol=1e-12, printitn=100)" + "M5 = ttb.cp_als(X, R, init=\"nvecs\", maxiters=1000, stoptol=1e-12, printitn=100)" ] }, { @@ -318,17 +350,23 @@ "metadata": {}, "outputs": [], "source": [ - "X = ttb.ktensor(\n", + "# Create rank-2 tensor\n", + "X2 = ttb.ktensor(\n", " factor_matrices=[\n", - " np.array([[1.0, 1.0], [1.0, -10.0]]),\n", - " np.array([[1.0, 1.0], [1.0, -10.0]]),\n", + " np.array([[1.0, 1.0], [-1.0, -10.0]]),\n", + " np.array([[1.0, 1.0], [-2.0, -10.0]]),\n", " ],\n", " weights=np.array([1.0, 1.0]),\n", ")\n", - "M1 = ttb.cp_als(X, 2, printitn=1, init=ttb.ktensor(X.factor_matrices))\n", - "print(M1[0]) # default behavior, fixsigns called\n", - "M2 = ttb.cp_als(X, 2, printitn=1, init=ttb.ktensor(X.factor_matrices), fixsigns=False)\n", - "print(M2[0]) # fixsigns not called" + "print(f\"X2=\\n{X2}\\n\")\n", + "\n", + "M_fixsigns, _, _ = ttb.cp_als(X2, 2, printitn=0, init=ttb.ktensor(X2.factor_matrices))\n", + "print(f\"M_fixsigns=\\n{M_fixsigns}\\n\") # default behavior, fixsigns called\n", + "\n", + "M_no_fixsigns, _, _ = ttb.cp_als(\n", + " X2, 2, printitn=0, init=ttb.ktensor(X2.factor_matrices), fixsigns=False\n", + ")\n", + "print(f\"M_no_fixsigns=\\n{M_no_fixsigns}\\n\") # fixsigns not called" ] }, {