Skip to content

Add initial linear algebra function specifications #20

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 21 commits into from
Sep 8, 2020
Merged

Conversation

kgryte
Copy link
Contributor

@kgryte kgryte commented Aug 24, 2020

This PR:

  • adds specifications for initial (basic) linear algebra functions.
  • is derived from comparing API signatures across array libraries.

Notes

  • This list of linear algebra functions is an initial set of linear algebra functions which can pave the way for additional specs in subsequent pull requests. These functions were identified as the set of functions with the broadest support among array libraries and relatively higher usage among downstream libraries.

Questions

  • What data types should linear algebra functions support? All supported array data types or, e.g., just floating-point?

  • This proposal tries to thread the needle and specify that set of behavior which is most common among array libraries, while also ensuring consistency among the specified functions. For example,

    • NumPy, e.g., deviates from the definition of a mathematical norm, which, e.g., TensorFlow does not support. Torch currently only supports two of the eight matrix norms which NumPy supports. Despite lack of universal support for NumPy's semantics, this proposal embraces NumPy's more liberal definition of a "norm". Where array libraries have gaps in their support, e.g., Torch and TensorFlow, adding support should be straightforward.
    • The spec does not include support for length 2 vectors when computing the cross product (NumPy) or specifying axes independently (NumPy), as this is not supported by Torch or TensorFlow.
    • The spec is currently silent as to whether transpose returns a copy or a view.
    • The spec limits the outer product to two 1-dimensional arrays and does not specify that higher dimensional arrays should be flattened (NumPy).

    Are we satisfied with the design decisions included here?

Copy link
Member

@rgommers rgommers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The spec is currently silent as to whether transpose returns a copy or a view.

It would be good to keep a tracking issue for this, that we add to any time we encounter a function where copy/view behavior is unclear or varies across libraries.

Are we satisfied with the design decisions included here?

All seems reasonable to me.

@kgryte
Copy link
Contributor Author

kgryte commented Aug 25, 2020

Re: copy/view. In this case, for transpose, TensorFlow and MXNet both return copies, while others (often those which support strides) return views.

@rgommers
Copy link
Member

Re: copy/view. In this case, for transpose, TensorFlow and MXNet both return copies, while others (often those which support strides) return views.

Created gh-24 for this topic.

@rgommers
Copy link
Member

rgommers commented Sep 8, 2020

I believe the copy/view discussion won't affect the content of this PR; whatever we end up doing there, there seems to be agreement that it's not possible to spec actual copy/view behaviour, and that therefore it should not matter (except performance-wise) whether a particular method returns a copy or a view.

All other comments were addressed, so this should be good as is. Merging, thanks @kgryte.

@rgommers rgommers merged commit b79b93b into master Sep 8, 2020
@rgommers rgommers deleted the linalg branch September 8, 2020 10:11
@rgommers
Copy link
Member

rgommers commented Sep 8, 2020

I un-ticked the linear algebra box in the tracking issue, because this was just the initial set - the more complex functions like svd, qr and einsum are still TODO.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants