Skip to content

Migrate export_llama to new ao quantize API #8422

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
jackzhxng opened this issue Feb 12, 2025 · 1 comment · Fixed by #8501
Closed

Migrate export_llama to new ao quantize API #8422

jackzhxng opened this issue Feb 12, 2025 · 1 comment · Fixed by #8501
Assignees
@jackzhxng
Labels
module: examples Issues related to demos under examples/ module: llm Issues related to LLM examples and apps, and to the extensions/llm/ code triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@jackzhxng
Copy link
Contributor

jackzhxng commented Feb 12, 2025 •
edited by pytorch-bot bot
Loading

🚀 The feature, motivation and pitch

Int8DynActInt4WeightQuantizer for -qmode 8da4w is no longer being developed by ao and doesn't support bias. Migrate to the new quantize_ api which can take in int8_dynamic_activation_int4_weight.

Alternatives

No response

Additional context

No response

RFC (Optional)

No response

cc @mergennachin @iseeyuan @lucylq @helunwencser @tarun292 @kimishpatel @cccclai
@jackzhxng jackzhxng added module: examples Issues related to demos under examples/ module: llm Issues related to LLM examples and apps, and to the extensions/llm/ code labels Feb 12, 2025
@jackzhxng jackzhxng self-assigned this Feb 12, 2025
@github-project-automation github-project-automation bot moved this to To triage in ExecuTorch Core Feb 12, 2025
@mergennachin
Copy link
Contributor

@jackzhxng

see #5106 and D62223540 for some historical context

@lucylq lucylq moved this from To triage to In progress in ExecuTorch Core Feb 13, 2025
@jackzhxng jackzhxng added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Feb 14, 2025
@jackzhxng jackzhxng mentioned this issue Feb 24, 2025
Merged
@github-project-automation github-project-automation bot moved this from In progress to Done in ExecuTorch Core Feb 25, 2025
@jackzhxng jackzhxng reopened this Mar 3, 2025
@github-project-automation github-project-automation bot moved this from Done to Backlog in ExecuTorch Core Mar 3, 2025
@jackzhxng jackzhxng moved this from Backlog to In progress in ExecuTorch Core Mar 3, 2025
@jackzhxng jackzhxng moved this from In progress to Backlog in ExecuTorch Core Mar 11, 2025
@jackzhxng jackzhxng moved this from Backlog to In progress in ExecuTorch Core Mar 20, 2025
@tarun292 tarun292 moved this from In progress to Done in ExecuTorch Core Mar 27, 2025
@tarun292 tarun292 closed this as completed by moving to Done in ExecuTorch Core Mar 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jackzhxng jackzhxng

Labels
module: examples Issues related to demos under examples/ module: llm Issues related to LLM examples and apps, and to the extensions/llm/ code triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
ExecuTorch Core
Status: Done
etLLM: LLMs via ExecuTorch
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Switch to new ao quant api for 8da4w pytorch/executorch
3 participants
@mergennachin @jackzhxng @tarun292