Add initial linear algebra function specifications (#20)

kgryte · web-flow · commit b79b93b61721 · 2020-09-08T12:11:57.000+02:00
* Add cross signature * Add det specification * Add diagonal specification * Add inv specification * Add norm specification * Add outer product specification * Add outer specification * Add trace specification * Add transpose * Update index * Fix type annotation * Update norm behavior for multi-dimensional arrays * Support all of NumPy's norms Further support for supporting all of NumPy's norms comes from pending updates to Torch (see pytorch/pytorch#42749). * Switch order * Split into separate tables * Escape markup * Escape markup * Add matrix_transpose interface Interface inspired by TensorFlow (see https://www.tensorflow.org/api_docs/python/tf/linalg/matrix_transpose) and Torch (see https://pytorch.org/docs/stable/generated/torch.transpose.html). Allows transposing a stack of matrices. * Rename parameters * Remove matrix_transpose signature
diff --git a/spec/API_specification/index.rst b/spec/API_specification/index.rst
@@ -14,3 +14,4 @@ API specification
    out_keyword
    elementwise_functions
    statistical_functions
+   linear_algebra_functions
diff --git a/spec/API_specification/linear_algebra_functions.md b/spec/API_specification/linear_algebra_functions.md
@@ -0,0 +1,262 @@
+# Linear Algebra Functions
+
+> Array API specification for linear algebra functions.
+
+A conforming implementation of the array API standard must provide and support the following functions adhering to the following conventions.
+
+-   Positional parameters must be [positional-only](https://www.python.org/dev/peps/pep-0570/) parameters. Positional-only parameters have no externally-usable name. When a function accepting positional-only parameters is called, positional arguments are mapped to these parameters based solely on their order.
+-   Optional parameters must be [keyword-only](https://www.python.org/dev/peps/pep-3102/) arguments.
+-   Broadcasting semantics must follow the semantics defined in :ref:`broadcasting`.
+-   Unless stated otherwise, functions must support the data types defined in :ref:`data-types`.
+-   Unless stated otherwise, functions must adhere to the type promotion rules defined in :ref:`type-promotion`.
+-   Unless stated otherwise, floating-point operations must adhere to IEEE 754-2019.
+
+<!-- NOTE: please keep the functions in alphabetical order -->
+
+### <a name="cross" href="#cross">#</a> cross(x1, x2, /, *, axis=-1)
+
+Returns the cross product of 3-element vectors. If `x1` and `x2` are multi-dimensional arrays (i.e., both have a rank greater than `1`), then the cross-product of each pair of corresponding 3-element vectors is independently computed.
+
+#### Parameters
+
+-   **x1**: _&lt;array&gt;_
+
+    -   first input array.
+
+-   **x2**: _&lt;array&gt;_
+
+    -   second input array. Must have the same shape as `x1`. 
+
+-   **axis**: _int_
+
+    -   the axis (dimension) of `x1` and `x2` containing the vectors for which to compute the cross product. If set to `-1`, the function computes the cross product for vectors defined by the last axis (dimension). Default: `-1`.
+
+#### Returns
+
+-   **out**: _&lt;array&gt;_
+
+    -   an array containing the cross products.
+
+### <a name="det" href="#det">#</a> det(x, /)
+
+Returns the determinant of a square matrix (or stack of square matrices) `x`.
+
+#### Parameters
+
+-   **a**: _&lt;array&gt;_
+
+    -   input array having shape `(..., M, M)` and whose innermost two dimensions form square matrices.
+
+#### Returns
+
+-   **out**: _&lt;array&gt;_
+
+    -   if `x` is a two-dimensional array, a zero-dimensional array containing the determinant; otherwise, a non-zero dimensional array containing the determinant for each square matrix.
+
+### <a name="diagonal" href="#diagonal">#</a> diagonal(x, /, *, axis1=0, axis2=1, offset=0)
+
+Returns the specified diagonals. If `x` has more than two dimensions, then the axes (dimensions) specified by `axis1` and `axis2` are used to determine the two-dimensional sub-arrays from which to return diagonals. 
+
+#### Parameters
+
+-   **x**: _&lt;array&gt;_
+
+    -   input array. Must have at least `2` dimensions.
+
+-   **axis1**: _int_
+
+    -   first axis (dimension) with respect to which to take diagonal. Default: `0`.
+
+-   **axis2**: _int_
+
+    -   second axis (dimension) with respect to which to take diagonal. Default: `1`.
+
+-   **offset**: _int_
+
+    -   offset specifying the off-diagonal relative to the main diagonal.
+
+        -   `offset = 0`: the main diagonal.
+        -   `offset > 0`: off-diagonal above the main diagonal.
+        -   `offset < 0`: off-diagonal below the main diagonal.
+
+        Default: `0`.
+
+#### Returns
+
+-   **out**: _&lt;array&gt;_
+
+    -   if `x` is a two-dimensional array, a one-dimensional array containing the diagonal; otherwise, a multi-dimensional array containing the diagonals and whose shape is determined by removing `axis1` and `axis2` and appending a dimension equal to the size of the resulting diagonals. Must have the same data type as `x`.
+
+### <a name="inv" href="#inv">#</a> inv(x, /)
+
+Computes the multiplicative inverse of a square matrix (or stack of square matrices) `x`.
+
+#### Parameters
+
+-   **x**: _&lt;array&gt;_
+
+    -   input array having shape `(..., M, M)` and whose innermost two dimensions form square matrices.
+
+#### Returns
+
+-   **out**: _&lt;array&gt;_
+
+    -   an array containing the multiplicative inverses. Must have the same data type and shape as `x`.
+
+### <a name="norm" href="#norm">#</a> norm(x, /, *, axis=None, keepdims=False, ord=None)
+
+Computes the matrix or vector norm of `x`.
+
+#### Parameters
+
+-   **x**: _&lt;array&gt;_
+
+    -   input array.
+
+-   **axis**: _Optional\[ Union\[ int, Tuple\[ int, int ] ] ]_
+
+    -   If an integer, `axis` specifies the axis (dimension) along which to compute vector norms.
+    
+        If a 2-tuple, `axis` specifies the axes (dimensions) defining two-dimensional matrices for which to compute matrix norms.
+    
+        If `None`,
+    
+        -   if `x` is one-dimensional, the function computes the vector norm.
+        -   if `x` is two-dimensional, the function computes the matrix norm.
+        -   if `x` has more than two dimensions, the function computes the vector norm over all array values (i.e., equivalent to computing the vector norm of a flattened array).
+
+        Negative indices must be supported. Default: `None`.
+
+-   **keepdims**: _bool_
+
+    -   If `True`, the axes (dimensions) specified by `axis` must be included in the result as singleton dimensions, and, accordingly, the result must be compatible with the input array (see :ref:`broadcasting`). Otherwise, if `False`, the axes (dimensions) specified by `axis` must not be included in the result. Default: `False`.
+
+-   **ord**: _Optional\[ int, float, Literal\[ inf, -inf, 'fro', 'nuc' ] ]_
+
+    -   order of the norm. The following mathematical norms must be supported:
+
+        | ord              | matrix                          | vector                     |
+        | ---------------- | ------------------------------- | -------------------------- |
+        | 'fro'            | 'fro'                           | -                          |
+        | 'nuc'            | 'nuc'                           | -                          |
+        | 1                | max(sum(abs(x), axis=0))        | L1-norm (Manhattan)        |
+        | 2                | largest singular value          | L2-norm (Euclidean)        |
+        | inf              | max(sum(abs(x), axis=1))        | infinity norm              |
+        | (int,float >= 1) | -                               | p-norm                     |
+
+        The following non-mathematical "norms" must be supported:
+
+        | ord              | matrix                          | vector                         |
+        | ---------------- | ------------------------------- | ------------------------------ |
+        | 0                | -                               | sum(a != 0)                    |
+        | -1               | min(sum(abs(x), axis=0))        | 1./sum(1./abs(a))              |
+        | -2               | smallest singular value         | 1./sqrt(sum(1./abs(a)\*\*2))   |
+        | -inf             | min(sum(abs(x), axis=1))        | min(abs(a))                    |
+        | (int,float < 1)  | -                               | sum(abs(a)\*\*ord)\*\*(1./ord) |
+
+        When `ord` is `None`, the following norms must be the default norms:
+
+        | ord              | matrix                          | vector                     |
+        | ---------------- | ------------------------------- | -------------------------- |
+        | None             | 'fro'                           | L2-norm (Euclidean)        |
+        
+        where `fro` corresponds to the **Frobenius norm**, `nuc` corresponds to the **nuclear norm**, and `-` indicates that the norm is **not** supported.
+
+        For matrices,
+
+        -   if `ord=1`, the norm corresponds to the induced matrix norm where `p=1` (i.e., the maximum absolute value column sum).
+        -   if `ord=2`, the norm corresponds to the induced matrix norm where `p=inf` (i.e., the maximum absolute value row sum).
+        -   if `ord=inf`, the norm corresponds to the induced matrix norm where `p=2` (i.e., the largest singular value).
+
+        If `None`,
+
+        -   if matrix (or matrices), the function computes the Frobenius norm.
+        -   if vector (or vectors), the function computes the L2-norm (Euclidean norm).
+        
+        Default: `None`.
+
+#### Returns
+
+-   **out**: _&lt;array&gt;_
+
+    -   an array containing the norms. Must have the same data type as `x`. If `axis` is `None`, the output array is a zero-dimensional array containing a vector norm. If `axis` is a scalar value (`int` or `float`), the output array has a rank which is one less than the rank of `x`. If `axis` is a 2-tuple, the output array has a rank which is two less than the rank of `x`.
+
+### <a name="outer" href="#outer">#</a> outer(x1, x2, /)
+
+Computes the outer product of two vectors `x1` and `x2`.
+
+#### Parameters
+
+-   **x1**: _&lt;array&gt;_
+
+    -   first one-dimensional input array of size `N`.
+
+-   **x2**: _&lt;array&gt;_
+
+    -   second one-dimensional input array of size `M`.
+
+#### Returns
+
+-   **out**: _&lt;array&gt;_
+
+    -   a two-dimensional array containing the outer product and whose shape is `NxM`.
+
+### <a name="trace" href="#trace">#</a> trace(x, /, *, axis1=0, axis2=1, offset=0)
+
+Returns the sum along the specified diagonals. If `x` has more than two dimensions, then the axes (dimensions) specified by `axis1` and `axis2` are used to determine the two-dimensional sub-arrays for which to compute the trace. 
+
+#### Parameters
+
+-   **x**: _&lt;array&gt;_
+
+    -   input array. Must have at least `2` dimensions.
+
+-   **axis1**: _int_
+
+    -   first axis (dimension) with respect to which to compute the trace. Default: `0`.
+
+-   **axis2**: _int_
+
+    -   second axis (dimension) with respect to which to compute the trace. Default: `1`.
+
+-   **offset**: _int_
+
+    -   offset specifying the off-diagonal relative to the main diagonal.
+
+        -   `offset = 0`: the main diagonal.
+        -   `offset > 0`: off-diagonal above the main diagonal.
+        -   `offset < 0`: off-diagonal below the main diagonal.
+
+        Default: `0`.
+
+#### Returns
+
+-   **out**: _&lt;array&gt;_
+
+    -   if `x` is a two-dimensional array, a zero-dimensional array containing the trace; otherwise, a multi-dimensional array containing the traces.
+    
+        The shape of a multi-dimensional output array is determined by removing `axis1` and `axis2` and storing the traces in the last array dimension. For example, if `x` has rank `k` and shape `(I, J, K, ..., L, M, N)` and `axis1=-2` and `axis1=-1`, then a multi-dimensional output array has rank `k-2` and shape `(I, J, K, ..., L)` where
+    
+        ```text
+        out[i, j, k, ..., l] = trace(a[i, j, k, ..., l, :, :])
+        ```
+
+### <a name="transpose" href="#transpose">#</a> transpose(x, /, *, axes=None)
+
+Transposes (or permutes the axes (dimensions)) of an array `x`.
+
+#### Parameters
+
+-   **x**: _&lt;array&gt;_
+
+    -   input array.
+
+-   **axes**: _Optional\[ Tuple\[ int, ... ] ]_
+
+    -   tuple containing a permutation of `(0, 1, ..., N-1)` where `N` is the number of axes (dimensions) of `x`. If `None`, the axes (dimensions) are permuted in reverse order (i.e., equivalent to setting `axes=(N-1, ..., 1, 0)`). Default: `None`.
+
+#### Returns
+
+-   **out**: _&lt;array&gt;_
+
+    -   an array containing the transpose. Must have the same data type as `x`.
diff --git a/spec/purpose_and_scope.md b/spec/purpose_and_scope.md
@@ -50,6 +50,10 @@ For the purposes of this specification, the following terms and definitions appl
 
 a (usually fixed-size) multidimensional container of items of the same type and size.
 
+### axis
+
+an array dimension.
+
 ### broadcast
 
 automatic (implicit) expansion of array dimensions to be of equal sizes without copying array data for the purpose of making arrays with different shapes have compatible shapes for element-wise operations.
@@ -62,6 +66,10 @@ two arrays whose dimensions are compatible (i.e., where the size of each dimensi
 
 an operation performed element-by-element, in which individual array elements are considered in isolation and independently of other elements within the same array.
 
+### matrix
+
+a two-dimensional array.
+
 ### rank
 
 number of array dimensions (not to be confused with the number of linearly independent columns of a matrix).
@@ -74,6 +82,10 @@ a tuple of `N` non-negative integers that specify the sizes of each dimension an
 
 a dimension whose size is one.
 
+### vector
+
+a one-dimensional array.
+
 * * *
 
 ## Normative References
