[KT.regroup Ops][6/N] use new op in KTRegroupAsDict module #2210
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D53590566 |
|
This pull request was exported from Phabricator. Differential Revision: D53590566 |
TroyGarden
force-pushed
the
export-D53590566
branch
from
86f1984 to
7db7fb4
Compare
TroyGarden
added a commit
to TroyGarden/torchrec
that referenced
this pull request
Jul 7, 2024
Summary: Pull Request resolved: pytorch#2210 # context * adding PackedTensorAccessor for passing the index tensor to kernel * GPU trace reading slows down from 2.20ms to 2.26ms # traces * previous ~4.90s {F1747994738} * after ~2.00ms {F1747994032} Differential Revision: D53590566
|
This pull request was exported from Phabricator. Differential Revision: D53590566 |
TroyGarden
force-pushed
the
export-D53590566
branch
from
7db7fb4 to
78be7d3
Compare
TroyGarden
added a commit
to TroyGarden/torchrec
that referenced
this pull request
Jul 8, 2024
Summary: Pull Request resolved: pytorch#2210 # context * adding PackedTensorAccessor for passing the index tensor to kernel * GPU trace reading slows down from 2.20ms to 2.26ms # traces * previous ~4.90s {F1747994738} * after ~2.00ms {F1747994032} Differential Revision: D53590566
|
This pull request was exported from Phabricator. Differential Revision: D53590566 |
TroyGarden
force-pushed
the
export-D53590566
branch
from
78be7d3 to
d359594
Compare
TroyGarden
added a commit
to TroyGarden/torchrec
that referenced
this pull request
Jul 9, 2024
Summary: Pull Request resolved: pytorch#2210 # context * adding PackedTensorAccessor for passing the index tensor to kernel * GPU trace reading slows down from 2.20ms to 2.26ms # traces * previous ~4.90s {F1747994738} * after ~2.00ms {F1747994032} Differential Revision: D53590566
|
This pull request was exported from Phabricator. Differential Revision: D53590566 |
TroyGarden
force-pushed
the
export-D53590566
branch
from
d359594 to
fc76364
Compare
TroyGarden
added a commit
to TroyGarden/torchrec
that referenced
this pull request
Jul 11, 2024
Summary: Pull Request resolved: pytorch#2210 # context * adding PackedTensorAccessor for passing the index tensor to kernel * GPU trace reading slows down from 2.20ms to 2.26ms # traces * previous ~4.90s {F1747994738} * after ~2.00ms {F1747994032} Differential Revision: D53590566
TroyGarden
changed the title
|
This pull request was exported from Phabricator. Differential Revision: D53590566 |
TroyGarden
added a commit
to TroyGarden/torchrec
that referenced
this pull request
Jul 26, 2024
Summary: Pull Request resolved: pytorch#2210 # context * the new op `permute_multi_embedding` outperforms the original op `permute_pooled_embs_auto_grad` * this diff makes the move to switch to the new op * benchmark results: D58907223 # benchmark * [traces](https://drive.google.com/drive/folders/1v_kD9n1jOkGUmYyix3-dUYiBDE_C3Hiv?usp=drive_link) * previous prod {F1747994738} * new prod {F1747994032} * metrics |Operator|GPU runtime|GPU memory|notes| |---|---|---|---|---| |**[previous prod] permute_pooled_embs**|4.9 ms|1.5 K|GPU-boudned, does **NOT** allow duplicates, PT2 non-compatible `pin_and_move`| |**[new prod] permute_multi_embedding**|2.0 ms|1.0 K|both CPU and GPU runtime/memory improved, **ALLOW** duplicates, PT2 friendly| Reviewed By: dstaay-fb Differential Revision: D53590566
TroyGarden
force-pushed
the
export-D53590566
branch
from
fc76364 to
926646f
Compare
PaulZhang12
pushed a commit
to PaulZhang12/torchrec
that referenced
this pull request
Jul 29, 2024
Summary: Pull Request resolved: pytorch#2210 # context * he new op `permute_multi_embedding` outperforms the original op `permute_pooled_embs_auto_grad` * this diff makes the move to switch to the new op * benchmark results: D58907223 # benchmark * [traces](https://drive.google.com/drive/folders/1v_kD9n1jOkGUmYyix3-dUYiBDE_C3Hiv?usp=drive_link) * previous prod {F1747994738} * new prod {F1747994032} * metrics |Operator|GPU runtime|GPU memory|notes| |---|---|---|---|---| |**[previous prod] permute_pooled_embs**|4.9 ms|1.5 K|GPU-boudned, does **NOT** allow duplicates, PT2 non-compatible `pin_and_move`| |**[new prod] permute_multi_embedding**|2.0 ms|1.0 K|both CPU and GPU runtime/memory improved, **ALLOW** duplicates, PT2 friendly| Differential Revision: D53590566
TroyGarden
added a commit
to TroyGarden/torchrec
that referenced
this pull request
Aug 6, 2024
Summary: Pull Request resolved: pytorch#2210 # context * the new op `permute_multi_embedding` outperforms the original op `permute_pooled_embs_auto_grad` * this diff makes the move to switch to the new op * benchmark results: D58907223 # benchmark * [traces](https://drive.google.com/drive/folders/1v_kD9n1jOkGUmYyix3-dUYiBDE_C3Hiv?usp=drive_link) * previous prod {F1747994738} * new prod {F1747994032} * metrics |Operator|GPU runtime|GPU memory|notes| |---|---|---|---|---| |**[previous prod] permute_pooled_embs**|4.9 ms|1.5 K|GPU-boudned, does **NOT** allow duplicates, PT2 non-compatible `pin_and_move`| |**[new prod] permute_multi_embedding**|2.0 ms|1.0 K|both CPU and GPU runtime/memory improved, **ALLOW** duplicates, PT2 friendly| Reviewed By: dstaay-fb Differential Revision: D53590566
TroyGarden
force-pushed
the
export-D53590566
branch
from
926646f to
4544b58
Compare
|
This pull request was exported from Phabricator. Differential Revision: D53590566 |
1 similar comment
|
This pull request was exported from Phabricator. Differential Revision: D53590566 |
TroyGarden
added a commit
to TroyGarden/torchrec
that referenced
this pull request
Aug 7, 2024
Summary: Pull Request resolved: pytorch#2210 # context * the new op `permute_multi_embedding` outperforms the original op `permute_pooled_embs_auto_grad` * this diff makes the move to switch to the new op * benchmark results: D58907223 # benchmark * [traces](https://drive.google.com/drive/folders/1v_kD9n1jOkGUmYyix3-dUYiBDE_C3Hiv?usp=drive_link) * previous prod {F1747994738} * new prod {F1747994032} * metrics |Operator|GPU runtime|GPU memory|notes| |---|---|---|---|---| |**[previous prod] permute_pooled_embs**|4.9 ms|1.5 K|GPU-boudned, does **NOT** allow duplicates, PT2 non-compatible `pin_and_move`| |**[new prod] permute_multi_embedding**|2.0 ms|1.0 K|both CPU and GPU runtime/memory improved, **ALLOW** duplicates, PT2 friendly| Reviewed By: dstaay-fb Differential Revision: D53590566
TroyGarden
force-pushed
the
export-D53590566
branch
from
4544b58 to
daae7a0
Compare
|
This pull request was exported from Phabricator. Differential Revision: D53590566 |
TroyGarden
added a commit
to TroyGarden/torchrec
that referenced
this pull request
Aug 7, 2024
Summary: Pull Request resolved: pytorch#2210 # context * the new op `permute_multi_embedding` outperforms the original op `permute_pooled_embs_auto_grad` * this diff makes the move to switch to the new op * benchmark results: D58907223 # benchmark * [traces](https://drive.google.com/drive/folders/1v_kD9n1jOkGUmYyix3-dUYiBDE_C3Hiv?usp=drive_link) * previous prod {F1747994738} * new prod {F1747994032} * metrics |Operator|GPU runtime|GPU memory|notes| |---|---|---|---|---| |**[previous prod] permute_pooled_embs**|4.9 ms|1.5 K|GPU-boudned, does **NOT** allow duplicates, PT2 non-compatible `pin_and_move`| |**[new prod] permute_multi_embedding**|2.0 ms|1.0 K|both CPU and GPU runtime/memory improved, **ALLOW** duplicates, PT2 friendly| Reviewed By: dstaay-fb Differential Revision: D53590566
TroyGarden
force-pushed
the
export-D53590566
branch
from
daae7a0 to
954d652
Compare
Summary: # context * previously `KTRegroupAsDict` can't really supported by torch.export (IR) because this module has an intialization step as running the first batch. Differential Revision: D57578012
TroyGarden
added a commit
to TroyGarden/torchrec
that referenced
this pull request
Aug 10, 2024
Summary: Pull Request resolved: pytorch#2210 # context * he new op `permute_multi_embedding` outperforms the original op `permute_pooled_embs_auto_grad` * this diff makes the move to switch to the new op * benchmark results: D58907223 # benchmark * [traces](https://drive.google.com/drive/folders/1v_kD9n1jOkGUmYyix3-dUYiBDE_C3Hiv?usp=drive_link) * previous prod {F1747994738} * new prod {F1747994032} * metrics |Operator|GPU runtime|GPU memory|notes| |---|---|---|---|---| |**[previous prod] permute_pooled_embs**|4.9 ms|1.5 K|GPU-boudned, does **NOT** allow duplicates, PT2 non-compatible `pin_and_move`| |**[new prod] permute_multi_embedding**|2.0 ms|1.0 K|both CPU and GPU runtime/memory improved, **ALLOW** duplicates, PT2 friendly| Differential Revision: D53590566
|
This pull request was exported from Phabricator. Differential Revision: D53590566 |
TroyGarden
added a commit
to TroyGarden/torchrec
that referenced
this pull request
Aug 10, 2024
Summary: Pull Request resolved: pytorch#2210 # context * the new op `permute_multi_embedding` outperforms the original op `permute_pooled_embs_auto_grad` * this diff makes the move to switch to the new op * benchmark results: D58907223 # benchmark * [traces](https://drive.google.com/drive/folders/1v_kD9n1jOkGUmYyix3-dUYiBDE_C3Hiv?usp=drive_link) * previous prod {F1747994738} * new prod {F1747994032} * metrics |Operator|GPU runtime|GPU memory|notes| |---|---|---|---|---| |**[previous prod] permute_pooled_embs**|4.9 ms|1.5 K|GPU-boudned, does **NOT** allow duplicates, PT2 non-compatible `pin_and_move`| |**[new prod] permute_multi_embedding**|2.0 ms|1.0 K|both CPU and GPU runtime/memory improved, **ALLOW** duplicates, PT2 friendly| Reviewed By: dstaay-fb Differential Revision: D53590566
TroyGarden
force-pushed
the
export-D53590566
branch
from
954d652 to
196d544
Compare
Summary: Pull Request resolved: pytorch#2210 # context * the new op `permute_multi_embedding` outperforms the original op `permute_pooled_embs_auto_grad` * this diff makes the move to switch to the new op * benchmark results: D58907223 # benchmark * [traces](https://drive.google.com/drive/folders/1v_kD9n1jOkGUmYyix3-dUYiBDE_C3Hiv?usp=drive_link) * previous prod {F1747994738} * new prod {F1747994032} * metrics |Operator|GPU runtime|GPU memory|notes| |---|---|---|---|---| |**[previous prod] permute_pooled_embs**|4.9 ms|1.5 K|GPU-boudned, does **NOT** allow duplicates, PT2 non-compatible `pin_and_move`| |**[new prod] permute_multi_embedding**|2.0 ms|1.0 K|both CPU and GPU runtime/memory improved, **ALLOW** duplicates, PT2 friendly| Reviewed By: dstaay-fb Differential Revision: D53590566
|
This pull request was exported from Phabricator. Differential Revision: D53590566 |
TroyGarden
force-pushed
the
export-D53590566
branch
from
196d544 to
b353525
Compare
TroyGarden
added a commit
to TroyGarden/torchrec
that referenced
this pull request
Aug 10, 2024
Summary: Pull Request resolved: pytorch#2210 # context * he new op `permute_multi_embedding` outperforms the original op `permute_pooled_embs_auto_grad` * this diff makes the move to switch to the new op * benchmark results: D58907223 # benchmark * [traces](https://drive.google.com/drive/folders/1v_kD9n1jOkGUmYyix3-dUYiBDE_C3Hiv?usp=drive_link) * previous prod {F1747994738} * new prod {F1747994032} * metrics |Operator|GPU runtime|GPU memory|notes| |---|---|---|---|---| |**[previous prod] permute_pooled_embs**|4.9 ms|1.5 K|GPU-boudned, does **NOT** allow duplicates, PT2 non-compatible `pin_and_move`| |**[new prod] permute_multi_embedding**|2.0 ms|1.0 K|both CPU and GPU runtime/memory improved, **ALLOW** duplicates, PT2 friendly| Differential Revision: D53590566
PaulZhang12
pushed a commit
to PaulZhang12/torchrec
that referenced
this pull request
Aug 12, 2024
Summary: Pull Request resolved: pytorch#2210 # context * he new op `permute_multi_embedding` outperforms the original op `permute_pooled_embs_auto_grad` * this diff makes the move to switch to the new op * benchmark results: D58907223 # benchmark * [traces](https://drive.google.com/drive/folders/1v_kD9n1jOkGUmYyix3-dUYiBDE_C3Hiv?usp=drive_link) * previous prod {F1747994738} * new prod {F1747994032} * metrics |Operator|GPU runtime|GPU memory|notes| |---|---|---|---|---| |**[previous prod] permute_pooled_embs**|4.9 ms|1.5 K|GPU-boudned, does **NOT** allow duplicates, PT2 non-compatible `pin_and_move`| |**[new prod] permute_multi_embedding**|2.0 ms|1.0 K|both CPU and GPU runtime/memory improved, **ALLOW** duplicates, PT2 friendly| Differential Revision: D53590566
PaulZhang12
pushed a commit
that referenced
this pull request
Aug 19, 2024
Summary: Pull Request resolved: #2210 # context * the new op `permute_multi_embedding` outperforms the original op `permute_pooled_embs_auto_grad` * this diff makes the move to switch to the new op * benchmark results: D58907223 # benchmark * [traces](https://drive.google.com/drive/folders/1v_kD9n1jOkGUmYyix3-dUYiBDE_C3Hiv?usp=drive_link) * previous prod {F1747994738} * new prod {F1747994032} * metrics |Operator|GPU runtime|GPU memory|notes| |---|---|---|---|---| |**[previous prod] permute_pooled_embs**|4.9 ms|1.5 K|GPU-boudned, does **NOT** allow duplicates, PT2 non-compatible `pin_and_move`| |**[new prod] permute_multi_embedding**|2.0 ms|1.0 K|both CPU and GPU runtime/memory improved, **ALLOW** duplicates, PT2 friendly| Reviewed By: dstaay-fb Differential Revision: D53590566 fbshipit-source-id: 220878f99111fabc3de8a0ba83d319b36ee519f6
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
context
permute_multi_embeddingoutperforms the original oppermute_pooled_embs_auto_gradbenchmark
{F1747994738}
{F1747994032}
|Operator|GPU runtime|GPU memory|notes|
|---|---|---|---|---|
|[previous prod] permute_pooled_embs|4.9 ms|1.5 K|GPU-boudned, does NOT allow duplicates, PT2 non-compatible
pin_and_move||[new prod] permute_multi_embedding|2.0 ms|1.0 K|both CPU and GPU runtime/memory improved, ALLOW duplicates, PT2 friendly|
Differential Revision: D53590566