Skip to content

Support BFloat16 dtype in Android Tensor API #9881

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
GregoryComer opened this issue Apr 3, 2025 · 3 comments
Closed

Support BFloat16 dtype in Android Tensor API #9881

GregoryComer opened this issue Apr 3, 2025 · 3 comments
Assignees
Labels
module: android Issues related to Android code, build, and execution triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@GregoryComer
Copy link
Member

GregoryComer commented Apr 3, 2025

🚀 The feature, motivation and pitch

Currently, the ExecuTorch Android Java tensor class does not support BFloat16 dtype tensors. This prevents Java users from passing BFloat16 tensors in or out of the model. BFloat16 is increasingly common for GenAI, so this will be a useful feature to have.

To implement support, we will need to logic to Tensor.java. This task will require the following:

  • Create a Tensor_bfloat16 implementation, similar to the existing Tensor_int8 and Tensor_float32.
  • Update the dtype switches in Tensor.java to support bfloat16.
  • Add corresponding tests to TestTensor.java.

CC #6571

Alternatives

No response

Additional context

No response

RFC (Optional)

No response

cc @kirklandsign @cbilgin

@GregoryComer GregoryComer added module: android Issues related to Android code, build, and execution triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module labels Apr 3, 2025
@phaiting
Copy link
Contributor

phaiting commented Apr 4, 2025

I'd like to work on this task. Can I have it assigned to me? I'll make sure to follow the guidelines and keep the conversation updated."

@kirklandsign
Copy link
Contributor

Per discussion with @larryliu0820 @JacobSzwejbka we should not expose bf16 or fp16 as java input/output type.

Also per discussion with @phaiting seems this is not related to #6571

Let's close this one and fix #6571 instead?

@kirklandsign
Copy link
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module: android Issues related to Android code, build, and execution triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

3 participants