Commit 842ae3f
perf: fp16 centroid LUT — decode +6-14% at long context (TheTom#33)
Half-precision centroid table in vec flash attention dequant.
Reduces constant cache pressure at high access volumes.
Decode improvements:
Short: 75.3 → 77.2 (+2.5%)
8K: 59.2 → 67.3 (+13.7%)
48K (Mario PDF): 36.7 → 39.0 (+6.3%)
PPL: unchanged (6.211)
Prefill: no regression
Fixes TheTom#33
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: tturney@psyguard.ai1 parent 613b455 commit 842ae3f
1 file changed
Lines changed: 13 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
688 | 688 | | |
689 | 689 | | |
690 | 690 | | |
| 691 | + | |
| 692 | + | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
| 696 | + | |
691 | 697 | | |
692 | 698 | | |
693 | 699 | | |
694 | | - | |
695 | | - | |
| 700 | + | |
696 | 701 | | |
697 | 702 | | |
698 | 703 | | |
699 | 704 | | |
700 | | - | |
701 | | - | |
702 | | - | |
703 | | - | |
704 | | - | |
705 | | - | |
| 705 | + | |
| 706 | + | |
| 707 | + | |
| 708 | + | |
| 709 | + | |
| 710 | + | |
706 | 711 | | |
707 | 712 | | |
708 | 713 | | |
| |||
0 commit comments