Skip to content

Add NEON implementation of Fused8BitRowwiseQuantizedSBFloatToFloatOrHalf #3707

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

Nicoshev
Copy link
Contributor

Summary:
QuantUtilsNeon.cc has been introduced, Fused8BitRowwiseQuantizedSBFloatToFloatOrHalfNeon implemented as first function

We have observed a ~x12 performance improvement for the downcasting case.
The case where a float32_t is returned maintains the same speed:

Full results:

before:

P1732996851

after:

P1732996401

Differential Revision: D69573878

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D69573878

Copy link

netlify bot commented Feb 18, 2025

Deploy Preview for pytorch-fbgemm-docs ready!

Name Link
🔨 Latest commit 295e157
🔍 Latest deploy log https://app.netlify.com/sites/pytorch-fbgemm-docs/deploys/67be3c84778e5b00082f991e
😎 Deploy Preview https://deploy-preview-3707--pytorch-fbgemm-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Nicoshev added a commit to Nicoshev/FBGEMM that referenced this pull request Feb 18, 2025
…alf (pytorch#3707)

Summary:
X-link: facebookresearch/FBGEMM#789


QuantUtilsNeon.cc has been introduced, Fused8BitRowwiseQuantizedSBFloatToFloatOrHalfNeon implemented as first function

We have observed a ~x12 performance improvement for the downcasting case.
The case where a float32_t is returned maintains the same speed:

Full results:

before:

P1732996851

after:

P1732996401

Differential Revision: D69573878
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D69573878

Nicoshev added a commit to Nicoshev/FBGEMM that referenced this pull request Feb 18, 2025
…alf (pytorch#3707)

Summary:
X-link: facebookresearch/FBGEMM#789


QuantUtilsNeon.cc has been introduced, Fused8BitRowwiseQuantizedSBFloatToFloatOrHalfNeon implemented as first function

We have observed a ~x12 performance improvement for the downcasting case.
The case where a float32_t is returned maintains the same speed:

Full results:

before:

P1732996851

after:

P1732996401

Differential Revision: D69573878
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D69573878

Nicoshev added a commit to Nicoshev/FBGEMM that referenced this pull request Feb 18, 2025
…alf (pytorch#3707)

Summary:
X-link: facebookresearch/FBGEMM#789


QuantUtilsNeon.cc has been introduced, Fused8BitRowwiseQuantizedSBFloatToFloatOrHalfNeon implemented as first function

We have observed a ~x12 performance improvement for the downcasting case.
The case where a float32_t is returned maintains the same speed:

Full results:

before:

P1732996851

after:

P1732996401

Differential Revision: D69573878
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D69573878

Nicoshev added a commit to Nicoshev/FBGEMM that referenced this pull request Feb 18, 2025
…alf (pytorch#3707)

Summary:
X-link: facebookresearch/FBGEMM#789


QuantUtilsNeon.cc has been introduced, Fused8BitRowwiseQuantizedSBFloatToFloatOrHalfNeon implemented as first function

We have observed a ~x12 performance improvement for the downcasting case.
The case where a float32_t is returned maintains the same speed:

Full results:

before:

P1732996851

after:

P1732996401

Differential Revision: D69573878
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D69573878

Nicoshev added a commit to Nicoshev/FBGEMM that referenced this pull request Feb 18, 2025
…alf (pytorch#3707)

Summary:
X-link: facebookresearch/FBGEMM#789


QuantUtilsNeon.cc has been introduced, Fused8BitRowwiseQuantizedSBFloatToFloatOrHalfNeon implemented as first function

We have observed a ~x12 performance improvement for the downcasting case.
The case where a float32_t is returned maintains the same speed:

Full results:

before:

P1732996851

after:

P1732996401

Differential Revision: D69573878
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D69573878

Nicoshev added a commit to Nicoshev/FBGEMM that referenced this pull request Feb 18, 2025
…alf (pytorch#3707)

Summary:
X-link: facebookresearch/FBGEMM#789


QuantUtilsNeon.cc has been introduced, Fused8BitRowwiseQuantizedSBFloatToFloatOrHalfNeon implemented as first function

We have observed a ~x12 performance improvement for the downcasting case.
The case where a float32_t is returned maintains the same speed:

Full results:

before:

P1732996851

after:

P1732996401

Differential Revision: D69573878
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D69573878

Nicoshev added a commit to Nicoshev/FBGEMM that referenced this pull request Feb 19, 2025
…alf (pytorch#3707)

Summary:
X-link: facebookresearch/FBGEMM#789


QuantUtilsNeon.cc has been introduced, Fused8BitRowwiseQuantizedSBFloatToFloatOrHalfNeon implemented as first function

We have observed a ~x12 performance improvement for the downcasting case.
The case where a float32_t is returned maintains the same speed:

Full results:

before:

P1732996851

after:

P1732996401

Differential Revision: D69573878
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D69573878

Nicoshev added a commit to Nicoshev/FBGEMM that referenced this pull request Feb 24, 2025
…alf (pytorch#3707)

Summary:
X-link: facebookresearch/FBGEMM#789


QuantUtilsNeon.cc has been introduced, Fused8BitRowwiseQuantizedSBFloatToFloatOrHalfNeon implemented as first function

We have observed a ~x12 performance improvement for the downcasting case.
The case where a float32_t is returned maintains the same speed:

Full results:

before:

P1732996851

after:

P1732996401

Differential Revision: D69573878
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D69573878

Nicoshev added a commit to Nicoshev/FBGEMM that referenced this pull request Feb 24, 2025
…alf (pytorch#3707)

Summary:
X-link: facebookresearch/FBGEMM#789


QuantUtilsNeon.cc has been introduced, Fused8BitRowwiseQuantizedSBFloatToFloatOrHalfNeon implemented as first function

We have observed a ~x12 performance improvement for the downcasting case.
The case where a float32_t is returned maintains the same speed:

Full results:

before:

P1732996851

after:

P1732996401

Differential Revision: D69573878
Nicoshev added a commit to Nicoshev/FBGEMM that referenced this pull request Feb 24, 2025
…alf (pytorch#3707)

Summary:
X-link: facebookresearch/FBGEMM#789


QuantUtilsNeon.cc has been introduced, Fused8BitRowwiseQuantizedSBFloatToFloatOrHalfNeon implemented as first function

We have observed a ~x12 performance improvement for the downcasting case.
The case where a float32_t is returned maintains the same speed:

Full results:

before:

P1732996851

after:

P1732996401

Differential Revision: D69573878
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D69573878

1 similar comment
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D69573878

Nicoshev added a commit to Nicoshev/FBGEMM that referenced this pull request Feb 24, 2025
…alf (pytorch#3707)

Summary:
X-link: facebookresearch/FBGEMM#789

Pull Request resolved: pytorch#3707

QuantUtilsNeon.cc has been introduced, Fused8BitRowwiseQuantizedSBFloatToFloatOrHalfNeon implemented as first function

We have observed a ~x12 performance improvement for the downcasting case.
The case where a float32_t is returned maintains the same speed:

Full results:

before:

P1732996851

after:

P1732996401

Differential Revision: D69573878
…alf (pytorch#3707)

Summary:
X-link: facebookresearch/FBGEMM#789


QuantUtilsNeon.cc has been introduced, Fused8BitRowwiseQuantizedSBFloatToFloatOrHalfNeon implemented as first function

We have observed a ~x12 performance improvement for the downcasting case.
The case where a float32_t is returned maintains the same speed:

Full results:

before:

P1732996851

after:

P1732996401

Differential Revision: D69573878
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D69573878

@facebook-github-bot
Copy link
Contributor

This pull request has been merged in 3de6774.

q10 pushed a commit to q10/FBGEMM that referenced this pull request Apr 10, 2025
…alf (pytorch#789)

Summary:
Pull Request resolved: facebookresearch/FBGEMM#789

X-link: pytorch#3707

QuantUtilsNeon.cc has been introduced, Fused8BitRowwiseQuantizedSBFloatToFloatOrHalfNeon implemented as first function

We have observed a ~x12 performance improvement for the downcasting case.
The case where a float32_t is returned maintains the same speed:

Full results:

before:

P1732996851

after:

P1732996401

Reviewed By: q10

Differential Revision: D69573878

fbshipit-source-id: b3d5045cf718a13356c068846a7805ff60f61d87
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants