[etLLM] Explore the way to load a new LLM to llama_transformer

[iseeyuan]

It's related to the first part (eager mode definition of this RFC.

There's llama_transformer that used to be Llama specific. A lot of infra are built around of this model, like source transforms, quantization, export, lowering to each backend, etc.

In open source a lot of models share the same architecture. Can we reuse llama_transformer and all the existing infra, to quickly enable a new model with optimized performance in multiple Edge backends (CPU, NPUs, CoreML, etc.)?

Items to explore:

• Utils to quickly convert different checkpoints to Llama compliant. An example is to leverage the torchtune utils and convert a Hugging Face checkpoint.
• Build the model quickly with proper configs.
• Any interfaces are missing to streamline this flow?
• Hide the technical details (users may not understand operators or a delegate partitioner), and expose necessary APIs for user to config, to meet their KPIs. For example, based on the accuracy and memory, provide supported quantization bit width for users to config.
• Quick return on results (perplexity, performance numbers, etc.)

cc @mergennachin @cccclai @helunwencser @dvorjackz

[iseeyuan]

Chatted with @sxu offline. Some additional points to consider:

• scalar attributes: can be included in the map?
• no submodule class check. Is it the assumption of the subclass type, especially when the model definition is released and fixed?