Repository navigation

Fix Double Application of Softmax for Router Logits in MoE models - #45346

Closed

ionut-anghelina wants to merge 3 commits into

huggingface:mainfrom

ionut-anghelina:dev/ionut/FixDoubleSoftmax

Closed

Fix Double Application of Softmax for Router Logits in MoE models#45346
ionut-anghelina wants to merge 3 commits into
huggingface:mainfrom
ionut-anghelina:dev/ionut/FixDoubleSoftmax

Commits on Mar 30, 2026
- [Bugfix] Fix double softmax in MoE router load-balancing loss
  
  ionut-anghelina
  and
  claude
  committed
  fb4d936
Commits on Apr 9, 2026
- Add regression tests and fix dtype cast to use raw logits dtype
  
  ionut-anghelina
  and
  claude
  committed
  f7d5a44
Commits on Apr 13, 2026
- Merge branch 'main' into dev/ionut/FixDoubleSoftmax
  vasqu
  committed
  3067bc5