Skip to content

[torchlib] Implement aten::_scaled_dot_product_efficient_attention.default #1160

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
BowenBao opened this issue Nov 16, 2023 · 2 comments · Fixed by #1197
Closed

[torchlib] Implement aten::_scaled_dot_product_efficient_attention.default #1160

BowenBao opened this issue Nov 16, 2023 · 2 comments · Fixed by #1197
Assignees
@titaiwangms
Labels
contribution welcome We welcome code contributions for this module: torchlib Related to the torch/aten function lib in development

Comments

@BowenBao
Copy link
Contributor

BowenBao commented Nov 16, 2023 •
edited
Loading

From models. This op is emitted from cuda export only.

    cait_m36_384
    deit_base_distilled_patch16_224
    crossvit_9_240
    mobilevit_s
    pit_b_224
    nanogpt
    timm_vision_transformer
    beit_base_patch16_224
    stable_diffusion_unet

From https://github.com/microsoft/onnx-converters-private/issues/196

cc @justinchuby
@BowenBao BowenBao added the module: torchlib Related to the torch/aten function lib in development label Nov 16, 2023
@justinchuby justinchuby added the contribution welcome We welcome code contributions for this label Nov 16, 2023
@justinchuby justinchuby mentioned this issue Nov 16, 2023
Open
@titaiwangms
Copy link
Contributor

Does it make any difference on torchlib implementation if the op is meant for CUDA? If not, I can take this real quick.

@titaiwangms titaiwangms self-assigned this Nov 29, 2023
@BowenBao
Copy link
Contributor Author

It appears only in cuda export, other than that I don't feel in general there is a difference for torchlib impl.

@titaiwangms titaiwangms mentioned this issue Nov 30, 2023
Merged
titaiwangms added a commit that referenced this issue Dec 1, 2023
@titaiwangms
Add Op(aten::_scaled_dot_product_efficient_attention) | feat(torchlib) (
3c05276 
#1197)

Fix #1160 
Follow up #1043 It's another `scaled_dot_product_XXX_attention`

It only supports CUDA.


https://github.com/pytorch/pytorch/blob/38ae17d166a001ef6837553d1ddffa111624df27/torch/_meta_registrations.py#L5195-L5236

NOTE: This PR also enables CUDA tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@titaiwangms titaiwangms

Labels
contribution welcome We welcome code contributions for this module: torchlib Related to the torch/aten function lib in development
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add Op(aten::_scaled_dot_product_efficient_attention) | feat(torchlib) titaiwangms/onnx-script
3 participants
@BowenBao @justinchuby @titaiwangms