Proposal: split linalg.norm into dedicated matrix and vector APIs

[kgryte]

linalg.norm was added to the specification in gh-20 and subsequently placed in a linalg extension in gh-182.

Signature

linalg.norm(x, /, *, axis=None, keepdims=False, ord=None)

Background

The current API is derived from the comparison of API signatures across array libraries. The initial intent was to codify existing practice among array libraries to ensure API consistency; less attention was given to whether the norm API was sound/desirable to begin with.

Problem

norm is easily one of the most complex APIs in the specification. Depending on the dimensionality of x and the values of the function's keyword arguments, many different behaviors are possible. For example,

• when axis is

  • None and x is

    • one-dimensional: vector norm.
    • two-dimensional: matrix norm.
    • n-dimensional: vector norm over flattened array.

  • an int, then specifies dimension along which to compute the vector norm.

  • a 2-tuple, then specifies the batch dimensions for computing the matrix norm.

Furthermore, the ord keyword is constrained based on whether one is computing a matrix or vector norm (and thus, the dimensionality of x and the value of axis). For example,

• the ord values fro or nuc are strictly matrix norms and thus invalid when provided vectors.
• when ord is an int value other than +-1 and +-2, matrix norms are not currently supported.

Consequently, from an implementation perspective, disentangling valid combinations can be challenging and portends brittle and hard-to-maintain code.

Lastly, the overloading of behavior does not allow for performing batch vector norm operations. One can only batch matrix operations.

Proposal

Torch has introduced separate APIs for computing vector and matrix norms.

• linalg.vector_norm

  torch.linalg.vector_norm(A, ord=2, dim=None, keepdim=False, *, dtype=None, out=None) → Tensor
  

• linalg.matrix_norm

  torch.linalg.matrix_norm(A, ord='fro', dim=(-2, -1), keepdim=False, *, dtype=None, out=None) → Tensor
  

For vector_norm, ord is restricted to only applicable vector norms and dim is allowed to be an n-tuple to specify dimensions over which to perform batch operations (e.g., to compute the vector norms over the last two dimensions).

For matrix_norm, ord is restricted to only applicable matrix norms and dim is limited to a 2-tuple to specify batch dimensions (by default, the last two dimensions). This API is similar to other linear algebra APIs in the specification which perform batch operations.

This proposal seeks to follow Torch's lead by more clearly delineating behavior and splitting norm operations into separate APIs. Hence, this proposal seeks to

• remove linalg.norm
• add linalg.vector_norm
• add linalg.matrix_norm

Accordingly, using the same keyword argument names as norm, the added functions would have the following signatures:

linalg.vector_norm(x, /, *, axis=None, keepdims=False, ord=2)

with the default ord being the L2-norm (Euclidean norm).

linalg.matrix_norm(x, /, *, axis=(-2,-1), keepdims=False, ord='fro')

with the default ord being the Frobenius norm over the last two dimensions.

Questions

• Is this proposal desirable?
• For vector_norm, should the axis default be None or the last dimension? If None, would follow linalg.norm and compute the vector norm over the flattened array.
• For vector_norm, should axis be allowed to be an n-tuple? The Torch implementation does not seem to disallow this, but not clear the use case for, e.g., specifying three dimensions over which to compute a batched vector norm.