It was implemented in llama.cpp already, we can probably take much inspiration from there.
It was implemented in llama.cpp already, we can probably take much inspiration from there.