Repository navigation

DeepGEMM BF16 + mixed FP8/FP4 + MegaMoE + refactor - #45634

Merged

IlyasMoutawwakil merged 164 commits into

mainfrom

deepgemm-isolation

Jun 2, 2026

Merged

DeepGEMM BF16 + mixed FP8/FP4 + MegaMoE + refactor#45634
IlyasMoutawwakil merged 164 commits into
mainfrom
deepgemm-isolation

Commits on Apr 23, 2026
- init
  IlyasMoutawwakil
  committed
  94a2786
- Merge branch 'main' into deepgemm
  IlyasMoutawwakil
  committed
  8fc43d7
- style
  IlyasMoutawwakil
  committed
  357a035
- full support
  IlyasMoutawwakil
  committed
  741b5eb
Commits on Apr 24, 2026
- support EP better using offsets !
  IlyasMoutawwakil
  committed
  9fc3662
- comments
  IlyasMoutawwakil
  committed
  84552ae
- get rid of neutralize_ep_sentinels
  IlyasMoutawwakil
  committed
  1d9f319
- remove deepgemm stuff
  IlyasMoutawwakil
  committed
  9b86043
- Merge branch 'main' into deepgemm
  IlyasMoutawwakil
  committed
  8a3f88b
- fix
  IlyasMoutawwakil
  committed
  996d67d
- prefix
  IlyasMoutawwakil
  committed
  d033a83
- move
  IlyasMoutawwakil
  committed
  e15cfe6
- fix
  IlyasMoutawwakil
  committed
  10b6d90
- remove comment
  IlyasMoutawwakil
  committed
  d4a6b30
- fix unintilized outputs leaking
  IlyasMoutawwakil
  committed
  1d6054f
- revert unnecessary changes
  IlyasMoutawwakil
  committed
  137393c
- more unnecessary changes
  IlyasMoutawwakil
  committed
  774f901
- revert downcast
  IlyasMoutawwakil
  committed
  81230fe
- keep it simple
  IlyasMoutawwakil
  committed
  9f2ff08
- guard deepgemm cuda version
  IlyasMoutawwakil
  committed
  c55b7b7
- fix style
  IlyasMoutawwakil
  committed
  20858db
- update
  IlyasMoutawwakil
  committed
  bfea94f
- add deepgemm testing
  IlyasMoutawwakil
  committed
  eada47e
Commits on Apr 26, 2026
- moe sentinel support
  IlyasMoutawwakil
  committed
  89d2f0b
- Merge branch 'main' into deepgemm
  IlyasMoutawwakil
  authored
  ef72b0e
- fix
  IlyasMoutawwakil
  committed
  60db1ca
Commits on Apr 27, 2026
- Merge branch 'main' into deepgemm
  IlyasMoutawwakil
  authored
  e732af1
- compilable sonicmoe
  IlyasMoutawwakil
  committed
  68b7b0f
Commits on Apr 29, 2026
- mega moe kernel support attempt
  IlyasMoutawwakil
  committed
  faaa7aa
- use package for now
  IlyasMoutawwakil
  committed
  b502cd6
- Merge branch 'deepgemm' into deepgemm-isolation
  IlyasMoutawwakil
  committed
  cc753ca
- skip ep router and experts pre/post processing
  IlyasMoutawwakil
  committed
  1c17452
- simpler
  IlyasMoutawwakil
  committed
  5a8ceae
- fix
  
  IlyasMoutawwakil
  and
  Copilot
  committed
  a05aa39
- Merge branch 'main' into deepgemm-isolation
  IlyasMoutawwakil
  authored
  854f4fb
- fix
  IlyasMoutawwakil
  committed
  053c9df
Commits on May 1, 2026
- fix
  IlyasMoutawwakil
  committed
  80a6fe5
- dtensor support
  IlyasMoutawwakil
  committed
  ad8226c
- more dtensor
  IlyasMoutawwakil
  committed
  a663f4d
- simpler
  IlyasMoutawwakil
  committed
  74c3f2e
- remove comment
  IlyasMoutawwakil
  committed
  d3cae33
Commits on May 2, 2026
- Merge branch 'deepgemm' into deepgemm-isolation
  IlyasMoutawwakil
  committed
  c1bfa0f
Commits on May 4, 2026
- Merge branch 'main' into deepgemm-isolation
  IlyasMoutawwakil
  committed
  6c43611
- revert
  IlyasMoutawwakil
  committed
  0528e0e
- bc order
  IlyasMoutawwakil
  committed
  2bfd029
- revert extra indent
  IlyasMoutawwakil
  committed
  368db00
- revert unnecessary change
  IlyasMoutawwakil
  committed
  28fbb84
- update
  IlyasMoutawwakil
  committed
  6ef27ab
- less defensive
  IlyasMoutawwakil
  committed
  e436b77
Commits on May 5, 2026
- allow all kernels
  IlyasMoutawwakil
  committed
  8e8f0ee
- alow all kernels
  IlyasMoutawwakil
  committed
  c494e35
- hub only
  IlyasMoutawwakil
  committed
  03b0442
- Merge branch 'main' into deepgemm-isolation
  IlyasMoutawwakil
  authored
  527d8b4
- fix
  IlyasMoutawwakil
  committed
  2e51b3c
- fix
  IlyasMoutawwakil
  committed
  82c5fb5
- test
  IlyasMoutawwakil
  committed
  99fdf71
- test
  IlyasMoutawwakil
  committed
  21398a8
Commits on May 6, 2026
- sync
  IlyasMoutawwakil
  committed
  f21db7c
- check nvcc
  IlyasMoutawwakil
  committed
  1fe3768
- probe
  IlyasMoutawwakil
  committed
  bbfda35
- fix
  IlyasMoutawwakil
  committed
  68a373c
- test psum
  IlyasMoutawwakil
  committed
  825f6d4
- test
  IlyasMoutawwakil
  committed
  028a39f
- test
  IlyasMoutawwakil
  committed
  ee173a5
- probe
  IlyasMoutawwakil
  committed
  900e984
- fix
  IlyasMoutawwakil
  committed
  09a711f
- test
  IlyasMoutawwakil
  committed
  a0d4940
- nan issue
  IlyasMoutawwakil
  committed
  653b7b3
- repro
  IlyasMoutawwakil
  committed
  ed8af6b
- repro
  IlyasMoutawwakil
  committed
  fb8d338
- fix
  IlyasMoutawwakil
  committed
  562eb51
- simplifications
  IlyasMoutawwakil
  committed
  481e5e6
- fix
  IlyasMoutawwakil
  committed
  d3dbd32
- fix
  IlyasMoutawwakil
  committed
  9f168ff
- fix
  IlyasMoutawwakil
  committed
  7274c22
- fix
  IlyasMoutawwakil
  committed
  a800b8c
- fix
  IlyasMoutawwakil
  committed
  804c988
- fix
  IlyasMoutawwakil
  committed
  7f36562
- fix
  IlyasMoutawwakil
  committed
  6df9f47
- empty
  IlyasMoutawwakil
  committed
  c3432c9
- simplify
  IlyasMoutawwakil
  committed
  77f09fe
- test deepseek
  IlyasMoutawwakil
  committed
  7dfbedd
- dsv4 only
  IlyasMoutawwakil
  committed
  689cc29
- download dsv4
  IlyasMoutawwakil
  committed
  8de089c
- fix test
  IlyasMoutawwakil
  committed
  cb4d6f9
- Merge branch 'main' into deepgemm-isolation
  IlyasMoutawwakil
  committed
  389eee8
- push
  IlyasMoutawwakil
  committed
  8f29ed6
- test
  IlyasMoutawwakil
  committed
  946e200
- fix
  IlyasMoutawwakil
  committed
  6619647
- fix ep plan
  IlyasMoutawwakil
  committed
  913339a
- fix attempt
  IlyasMoutawwakil
  committed
  7cfc2b2
- debug
  IlyasMoutawwakil
  committed
  4c12f6e
- attempt
  IlyasMoutawwakil
  committed
  92e6337
- debug
  IlyasMoutawwakil
  committed
  a9997f5
Commits on May 11, 2026
- fixes in modeling
  IlyasMoutawwakil
  committed
  197a8f6
- more modeling changes
  IlyasMoutawwakil
  committed
  30d0780
- more modeling changes
  IlyasMoutawwakil
  committed
  3b1c470
- initial megamoe works but wrong output, deepgemm fully functional
  IlyasMoutawwakil
  committed
  51a9425
- mega moe works !
  IlyasMoutawwakil
  committed
  a4130db
- simplification
  IlyasMoutawwakil
  committed
  b140f7c
- comments
  IlyasMoutawwakil
  committed
  9ff656b
Commits on May 12, 2026
- Merge branch 'main' into deepgemm-isolation
  IlyasMoutawwakil
  committed
  b5b741e
- fix merge
  IlyasMoutawwakil
  committed
  1231258
- fix rank divergence
  IlyasMoutawwakil
  committed
  a8cc32a
- use _keep_in_fp32_modules
  IlyasMoutawwakil
  committed
  c6486cb
- simplify quantizer
  IlyasMoutawwakil
  committed
  bd68b6f
- introduce indexerscorer
  IlyasMoutawwakil
  committed
  8630f9d
- remove testing file
  IlyasMoutawwakil
  committed
  f3b933f
- style
  IlyasMoutawwakil
  committed
  029a4c9
- weight rename for scorer
  IlyasMoutawwakil
  committed
  85b5768
- style
  IlyasMoutawwakil
  committed
  7f19516
- address review comments
  IlyasMoutawwakil
  committed
  e395e4c
- style
  IlyasMoutawwakil
  committed
  1b94261
- align with laguna
  IlyasMoutawwakil
  committed
  af51a6e
- Merge branch 'main' into deepgemm-isolation
  IlyasMoutawwakil
  authored
  ca934bf
- stale comments and styling
  IlyasMoutawwakil
  committed
  f18ab28
Commits on May 19, 2026
- use grouped finegrained fp8 for grouped linear
  IlyasMoutawwakil
  committed
  26f3ba7
- Merge branch 'main' into deepgemm-isolation
  IlyasMoutawwakil
  authored
  4039ed6
- comments and docs
  IlyasMoutawwakil
  committed
  9f2c97b
Commits on May 28, 2026
- Revert "[DeepSeek V4] Fix MoE converter substring-matching FP8 scale companions (#45930)"
  ArthurZucker
  committed
  8dc75bd
- Merge remote-tracking branch 'origin/main' into deepgemm-isolation
  IlyasMoutawwakil
  committed
  57081cc
- fix quantizer and use upstream deepgemm
  IlyasMoutawwakil
  committed
  78fcee6
- remove force al kernels
  IlyasMoutawwakil
  committed
  2e4203f
- [deepseek_v4] fix FP8 save-side substring match + fp32 gate through softmax
  ArthurZucker
  committed
  5f9e5cd
- always floats scal inv with triton
  IlyasMoutawwakil
  committed
  afd1b70
- minimal changes i quantizer
  IlyasMoutawwakil
  committed
  619b5e9
- force stable abi build for now
  IlyasMoutawwakil
  committed
  3e76df5
- support transposed
  IlyasMoutawwakil
  committed
  f0e2859
Commits on May 29, 2026
- fix yarn
  IlyasMoutawwakil
  committed
  d0c4a6d
- unnecessary change
  IlyasMoutawwakil
  committed
  c1adcd7
- add flash base test and fix the incorrect and crash issue during enab… (#46076)
  sywangyi
  authored
  143835f
- merge
  IlyasMoutawwakil
  committed
  b47b30f
- better errors across deepgemm
  IlyasMoutawwakil
  committed
  8bd9619
- more better errors for deepgemm
  IlyasMoutawwakil
  committed
  751c8b5
- ep tests
  IlyasMoutawwakil
  committed
  70c7c3b
- Apply suggestions from code review
  ArthurZucker
  authored
  464512e
- scope supports_round_trip to quantization ops only
  
  ArthurZucker
  and
  claude
  committed
  d6fd64d
- Apply suggestions from code review
  ArthurZucker
  authored
  6e17509
- Update src/transformers/core_model_loading.py
  ArthurZucker
  authored
  3259618
- style: ruff format
  
  ArthurZucker
  and
  claude
  committed
  2c06aef
Commits on Jun 1, 2026
- custom ep/tp plan for megamoe
  IlyasMoutawwakil
  committed
  7dfe6bd
- process megamoe after weight loading
  IlyasMoutawwakil
  committed
  1889163
- fix megamoe process
  IlyasMoutawwakil
  committed
  46dc289
- up
  ArthurZucker
  committed
  e9ccaa0
- the actual fix???
  ArthurZucker
  committed
  7d74684
- up
  ArthurZucker
  committed
  87d2ee9
- nit comment for futur
  ArthurZucker
  committed
  7be266b
- style: drop unused exception variable in Fp8Dequantize._dequantize_one
  ArthurZucker
  committed
  5e102f1
- up
  ArthurZucker
  committed
  8631e6f
- style: apply make fix-repo docstring formatting
  ArthurZucker
  committed
  b05bda4
- Merge pr-46265 into deepgemm-isolation, drop redundant scale-merge workaround
  
  IlyasMoutawwakil
  and
  claude
  committed
  7481aa6
- simplify after merge
  IlyasMoutawwakil
  committed
  f769847
- review comments
  IlyasMoutawwakil
  committed
  2b8bba0
- Merge branch 'main' into deepgemm-isolation
  IlyasMoutawwakil
  committed
  beac5f2
Commits on Jun 2, 2026
- revert unncessary
  IlyasMoutawwakil
  committed
  f72bc20
- avoid name contraction
  IlyasMoutawwakil
  committed
  b5f2d45
- fixes
  IlyasMoutawwakil
  committed
  014707d
- dynamic tp/ep plan swap
  IlyasMoutawwakil
  committed
  2a77aa7
- fix deepgemm single error and device assertion docs
  IlyasMoutawwakil
  committed
  6922e3a
- properly support bias in grouped linear
  IlyasMoutawwakil
  committed
  7471109
- simplify
  IlyasMoutawwakil
  committed
  b63ce4f
- fix dist error handling and min cuda compatbility
  IlyasMoutawwakil
  committed
  c12e892
- fix least cuda compat
  IlyasMoutawwakil
  committed
  1ec0147
- get alignment from deepgemm and cache it
  IlyasMoutawwakil
  committed
  4fd793d

Commits

Commits on Apr 23, 2026

Commits on Apr 24, 2026

Commits on Apr 26, 2026

Commits on Apr 27, 2026

Commits on Apr 29, 2026

Commits on May 1, 2026

Commits on May 2, 2026

Commits on May 4, 2026

Commits on May 5, 2026

Commits on May 6, 2026

Commits on May 11, 2026

Commits on May 12, 2026

Commits on May 19, 2026

Commits on May 28, 2026

Commits on May 29, 2026