Repository navigation
Quantization cleanpup - #49548
Merged
Merged
Quantization cleanpup#49548
Conversation
The various agent PRs most probably did not understand the core logic / idea behind the weight converter, this left a lot of method patching here and ther. 1. Deserialize: some method just need a cast, other need more work. The ConversionOps added as the "first" operation, updating the weights conversion (each one of them) are here for that. 2. TP: bloated code to just update a plan. 3. Patch: I have no idea how, but we accepted plans that had explicit overwrite. This is legacy
Saw another PR recently: #48759, had the same bug. this adresses it
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
ArthurZucker
commented
Oct 13, 2026
ArthurZucker
commented
Oct 13, 2026
tarekziade
reviewed
Oct 13, 2026
Collaborator
Author
|
run-slow: fbgemm_fp8, finegrained_fp8, higgs, hqq, metal, mxfp4, nvfp4, sinq |
IlyasMoutawwakil
approved these changes
Oct 14, 2026
IlyasMoutawwakil
left a comment
IlyasMoutawwakil
left a comment
Member
There was a problem hiding this comment.
Overall very nice and clean !
left two nits on the class vs instance attrs changing nature and the two deepgemm/megamoe comments
…m0 float32 scale loading test
…rns end on a name boundary now
Collaborator
Author
|
run-slow: fbgemm_fp8, finegrained_fp8, higgs, hqq, metal, mxfp4, nvfp4, sinq |
Contributor
|
[For maintainers] Suggested jobs to run (before merge) run-slow: fbgemm_fp8, finegrained_fp8, higgs, hqq, metal, mxfp4, nvfp4, sinq |
ArthurZucker
enabled auto-merge
October 16, 2026 09:33
ArthurZucker
disabled auto-merge
October 16, 2026 09:38
Contributor
CI recapDashboard: View test results in Grafana |
This was referenced Oct 20, 2026
Stanley00
pushed a commit
to stanley-fork/hf-transformers
that referenced
this pull request
Oct 30, 2026
…uggingface#50150) * Restore MegaBlocksMoeMLP hub kernel mapping for MXFP4 GPT-OSS MLP huggingface#49548 replaces GptOssMLP with Mxfp4GptOssMLP, which dropped the MegaBlocksMoeMLP hub mapping, so use_kernels=True no longer kernelizes the MoE MLP of MXFP4 GPT-OSS on CPU/XPU. * Test the megablocks kernel on a real MXFP4 GPT-OSS model on CPU/XPU Replaces the decorator-only check with a slow test that loads openai/gpt-oss-20b with use_kernels=True, checks every MLP keeps MXFP4 experts and runs the MegaBlocksMoeMLP kernel, and checks generation. Limited to CPU/XPU: other devices dequantize to bf16 with use_kernels=True and keep GptOssMLP. * Use the modeling flash-attn2 kernel version in benchmark_v2's FA2 check is_fa2_or_kernel_available() pinned kernels-community/flash-attn2 to version=1, which has no XPU build, so flash_attention_2 benchmarks silently fell back to SDPA on XPU. Use FLASH_ATTN_KERNEL_VERSIONS, the same version the model loads. --------- Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Too much shit got past the reviews. Fixes #48757.
I think AI just does not understand it when it's spread across many files. We probably don't have the greatest doc either.
Anyways, there are 3 case:
Those 3 cases each have a different injection point in core model loading:
ConversionOpsto start with deserialization.get_quantize_ops