Add decorator for custom op and inductor decomp registration#408
Closed
jerryzh168 wants to merge 1 commit intopytorch:mainfrom
Closed
Add decorator for custom op and inductor decomp registration#408jerryzh168 wants to merge 1 commit intopytorch:mainfrom
jerryzh168 wants to merge 1 commit intopytorch:mainfrom
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/408
Note: Links to docs will display an error until the docs builds have been completed. ❌ 2 New FailuresAs of commit ac2e283 with merge base bc8599f ( NEW FAILURES - The following jobs have failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
jerryzh168
force-pushed
the
executorch-ir
branch
3 times, most recently
from
cf3234c to
2ba121f
Compare
supriyar
reviewed
Jun 21, 2024
| quant_min (Optional[int]): minimum quantized value for output Tensor, if not specified, it will be derived from dtype | ||
| quant_max (Optional[int]): maximum quantized value for output Tensor, if not specified, it will be derived from dtype | ||
| zero_point_domain (ZeroPointDomain): the domain that zero_point is in, should be eitehr integer or float | ||
| zero_point_domain (str): the domain that zero_point is in, should be eitehr "int" for "float" |
Contributor
There was a problem hiding this comment.
why did you change these from Enum to str?
Contributor
Author
There was a problem hiding this comment.
oh it's because custom op API doesn't support enum right now
Contributor
There was a problem hiding this comment.
@zou3519 any plans to add support for this in the API?
Contributor
There was a problem hiding this comment.
No, the set of types supported by operators is intentionally limited so that people don't have a difficult time working with it (for example, writing graph passes on FX graphs with operators). You can wrap the call to the operator in a python function that does support enums
Contributor
Author
There was a problem hiding this comment.
You can wrap the call to the operator in a python function that does support enums
oh, I see, maybe that's what we should be doing here
supriyar
reviewed
Jun 21, 2024
|
|
||
| return quant | ||
|
|
||
| @register_custom_op("quant::dequantize_affine") |
Contributor
There was a problem hiding this comment.
does this make the default behavior to preserve the higher level op when we run export? What about compile?
Contributor
Author
There was a problem hiding this comment.
yes, and in compile we'll register a inductor decomposition to decompose the op
jerryzh168
commented
Jun 21, 2024
| if TORCH_VERSION_AFTER_2_5: | ||
| opdef = torch.library.custom_op(name, mutates_args=())(fn) | ||
| opdef.register_fake(fn) | ||
| register_decomposition([opdef._opoverload])(fn) |
Contributor
Author
There was a problem hiding this comment.
@supriyar here we register a decomp for inductor, so in torch.compile these ops will still be decomposed
Contributor
Author
|
Requires pytorch/pytorch#129179 and pytorch/pytorch#129189 to be landed, and the nightly version got updated until the CI can pass |
Summary: This PR adds a decorator to register custom op and also an inductor dcomposition. The goal is for torch.export path to be able to see high level ops like quantize_affine instead of breaking down the op, this is because some backends like xnnpack wants to work with these higher level ops. Test Plan: regression tests: `python test/quantization/test_quant_api.py` `python test/integration/test_integration.py` also need to check performance with `python tutorials/quantize_vit/run_vit_b_quant.py` Reviewers: Subscribers: Tasks: Tags:
jerryzh168
force-pushed
the
executorch-ir
branch
from
2ba121f to
ac2e283
Compare
jerryzh168
added a commit
to jerryzh168/ao
that referenced
this pull request
Jun 25, 2024
Summary: This PR adds a decorator to register custom op and also an inductor dcomposition. The goal is for torch.export path to be able to see high level ops like quantize_affine instead of breaking down the op, this is because some backends like xnnpack wants to work with these higher level ops. This is a redo for pytorch#408, difference is we can preserve the enums on the python side in this PR Test Plan: regression tests: python test/quantization/test_quant_api.py python test/integration/test_integration.py also need to check performance with python tutorials/quantize_vit/run_vit_b_quant.py Reviewers: Subscribers: Tasks: Tags:
jerryzh168
added a commit
to jerryzh168/ao
that referenced
this pull request
Jun 25, 2024
Summary: This PR adds a decorator to register custom op and also an inductor dcomposition. The goal is for torch.export path to be able to see high level ops like quantize_affine instead of breaking down the op, this is because some backends like xnnpack wants to work with these higher level ops. This is a redo for pytorch#408, difference is we can preserve the enums on the python side in this PR Test Plan: regression tests: python test/quantization/test_quant_api.py python test/integration/test_integration.py also need to check performance with python tutorials/quantize_vit/run_vit_b_quant.py Reviewers: Subscribers: Tasks: Tags:
Contributor
Author
|
recreated the PR in #434 |
jerryzh168
added a commit
that referenced
this pull request
Jun 28, 2024
Summary: This PR adds a decorator to register custom op and also an inductor dcomposition. The goal is for torch.export path to be able to see high level ops like quantize_affine instead of breaking down the op, this is because some backends like xnnpack wants to work with these higher level ops. This is a redo for #408, difference is we can preserve the enums on the python side in this PR Test Plan: regression tests: python test/quantization/test_quant_api.py python test/integration/test_integration.py also need to check performance with python tutorials/quantize_vit/run_vit_b_quant.py Reviewers: Subscribers: Tasks: Tags:
jerryzh168
added a commit
to jerryzh168/ao
that referenced
this pull request
Jul 1, 2024
Summary: This PR adds a decorator to register custom op and also an inductor dcomposition. The goal is for torch.export path to be able to see high level ops like quantize_affine instead of breaking down the op, this is because some backends like xnnpack wants to work with these higher level ops. This is a redo for pytorch#408, difference is we can preserve the enums on the python side in this PR Test Plan: regression tests: python test/quantization/test_quant_api.py python test/integration/test_integration.py also need to check performance with python tutorials/quantize_vit/run_vit_b_quant.py Reviewers: Subscribers: Tasks: Tags:
jerryzh168
added a commit
to jerryzh168/ao
that referenced
this pull request
Jul 1, 2024
Summary: This PR adds a decorator to register custom op and also an inductor dcomposition. The goal is for torch.export path to be able to see high level ops like quantize_affine instead of breaking down the op, this is because some backends like xnnpack wants to work with these higher level ops. This is a redo for pytorch#408, difference is we can preserve the enums on the python side in this PR Test Plan: regression tests: python test/quantization/test_quant_api.py python test/integration/test_integration.py also need to check performance with python tutorials/quantize_vit/run_vit_b_quant.py Reviewers: Subscribers: Tasks: Tags:
jerryzh168
added a commit
to jerryzh168/ao
that referenced
this pull request
Jul 1, 2024
Summary: This PR adds a decorator to register custom op and also an inductor dcomposition. The goal is for torch.export path to be able to see high level ops like quantize_affine instead of breaking down the op, this is because some backends like xnnpack wants to work with these higher level ops. This is a redo for pytorch#408, difference is we can preserve the enums on the python side in this PR Test Plan: regression tests: python test/quantization/test_quant_api.py python test/integration/test_integration.py also need to check performance with python tutorials/quantize_vit/run_vit_b_quant.py Reviewers: Subscribers: Tasks: Tags:
jerryzh168
added a commit
to jerryzh168/ao
that referenced
this pull request
Jul 2, 2024
Summary: This PR adds a decorator to register custom op and also an inductor dcomposition. The goal is for torch.export path to be able to see high level ops like quantize_affine instead of breaking down the op, this is because some backends like xnnpack wants to work with these higher level ops. This is a redo for pytorch#408, difference is we can preserve the enums on the python side in this PR Test Plan: regression tests: python test/quantization/test_quant_api.py python test/integration/test_integration.py also need to check performance with python tutorials/quantize_vit/run_vit_b_quant.py Reviewers: Subscribers: Tasks: Tags:
jerryzh168
added a commit
to jerryzh168/ao
that referenced
this pull request
Jul 2, 2024
Summary: This PR adds a decorator to register custom op and also an inductor dcomposition. The goal is for torch.export path to be able to see high level ops like quantize_affine instead of breaking down the op, this is because some backends like xnnpack wants to work with these higher level ops. This is a redo for pytorch#408, difference is we can preserve the enums on the python side in this PR Test Plan: regression tests: python test/quantization/test_quant_api.py python test/integration/test_integration.py also need to check performance with python tutorials/quantize_vit/run_vit_b_quant.py Reviewers: Subscribers: Tasks: Tags:
jerryzh168
added a commit
to jerryzh168/ao
that referenced
this pull request
Jul 2, 2024
Summary: This PR adds a decorator to register custom op and also an inductor dcomposition. The goal is for torch.export path to be able to see high level ops like quantize_affine instead of breaking down the op, this is because some backends like xnnpack wants to work with these higher level ops. This is a redo for pytorch#408, difference is we can preserve the enums on the python side in this PR Test Plan: regression tests: python test/quantization/test_quant_api.py python test/integration/test_integration.py also need to check performance with python tutorials/quantize_vit/run_vit_b_quant.py Reviewers: Subscribers: Tasks: Tags:
jerryzh168
added a commit
to jerryzh168/ao
that referenced
this pull request
Jul 2, 2024
Summary: This PR adds a decorator to register custom op and also an inductor dcomposition. The goal is for torch.export path to be able to see high level ops like quantize_affine instead of breaking down the op, this is because some backends like xnnpack wants to work with these higher level ops. This is a redo for pytorch#408, difference is we can preserve the enums on the python side in this PR Test Plan: regression tests: python test/quantization/test_quant_api.py python test/integration/test_integration.py also need to check performance with python tutorials/quantize_vit/run_vit_b_quant.py Reviewers: Subscribers: Tasks: Tags:
jerryzh168
added a commit
that referenced
this pull request
Jul 2, 2024
Summary: This PR adds a decorator to register custom op and also an inductor dcomposition. The goal is for torch.export path to be able to see high level ops like quantize_affine instead of breaking down the op, this is because some backends like xnnpack wants to work with these higher level ops. This is a redo for #408, difference is we can preserve the enums on the python side in this PR Test Plan: regression tests: python test/quantization/test_quant_api.py python test/integration/test_integration.py also need to check performance with python tutorials/quantize_vit/run_vit_b_quant.py Reviewers: Subscribers: Tasks: Tags:
yanbing-j
pushed a commit
to yanbing-j/ao
that referenced
this pull request
Dec 9, 2024
* Refactor tokenizer Summary: Prepare to add tiktoken Test Plan: Rely on CI jobs Reviewers: Subscribers: Tasks: Tags: * Fix aoti Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags: * Fix more aoti Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags: * Fix tokenizer read format Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
This PR adds a decorator to register custom op and also an inductor dcomposition.
The goal is for torch.export path to be able to see high level ops like quantize_affine instead of breaking down the op, this is because some backends like xnnpack wants to work with these higher level ops.
Test Plan:
regression tests:
python test/quantization/test_quant_api.pypython test/integration/test_integration.pyalso need to check performance with
python tutorials/quantize_vit/run_vit_b_quant.pyReviewers:
Subscribers:
Tasks:
Tags: