Fused soft cap and SIMD-ified GeLU #9

Merged

ikawrakow merged 8 commits into

mainfrom

Aug 20, 2024

Commits on Aug 19, 2024

Softcap: WIP
Iwan Kawrakow
committed
softcap: CUDA
Iwan Kawrakow
committed
softcap: CUDA
Iwan Kawrakow
committed
softcap: Metal and NEON
Iwan Kawrakow
committed
Simdified gelu
Iwan Kawrakow
committed

Commits on Aug 20, 2024

softcap, tanh: avoid NaNs for large arguments (AVX2, AVX512)
Iwan Kawrakow
committed
llama-bench: add ability to turn off warmup runs
Iwan Kawrakow
committed
softcap, tanh: avoid NaNs for large arguments (NEON)
Iwan Kawrakow
committed