Closed
Description
Turns out that most LLM parameters are redundant, see https://aclanthology.org/2020.emnlp-main.398.pdf.
They run the experiment with BERT and XLNet. Code for the pruning is provided.
There's lots of room for improvement apparently, since LLama is very similar to those. If someone's interested, that could be a nice thing to try 😄
Metadata
Metadata
Assignees
Labels
No labels