[Feature] - prepare model support list for vLLM and TGI starting from v1.2 release

[zhiweizhangintc]

Priority

P2-High

OS type

Ubuntu

Hardware type

Gaudi2

Running nodes

Single Node

Description

the goal is to show a list of LLM models that are supported and validated on vLLM and TGI serving backend at Intel Xeon + Gaudi2 platform starting from v1.2 release.

[joshuayao]

Hi @zhiweizhangintc, does the doc look good?

[zhiweizhangintc]

Hi Yi, it looks good. but we should have all these Deepseek distilled models added in the future. Thanks.

DeepSeek-R1-Distill-Qwen-1.5B | Qwen2.5-Math-1.5B | 🤗 HuggingFace
DeepSeek-R1-Distill-Qwen-7B | Qwen2.5-Math-7B | 🤗 HuggingFace
DeepSeek-R1-Distill-Llama-8B | Llama-3.1-8B | 🤗 HuggingFace
DeepSeek-R1-Distill-Qwen-14B | Qwen2.5-14B | 🤗 HuggingFace
DeepSeek-R1-Distill-Qwen-32B | Qwen2.5-32B | 🤗 HuggingFace
DeepSeek-R1-Distill-Llama-70B | Llama-3.3-70B-Instruct | 🤗 HuggingFace

[joshuayao]

Hi Yi, it looks good. but we should have all these Deepseek distilled models added in the future. Thanks.

DeepSeek-R1-Distill-Qwen-1.5B | Qwen2.5-Math-1.5B | 🤗 HuggingFace DeepSeek-R1-Distill-Qwen-7B | Qwen2.5-Math-7B | 🤗 HuggingFace DeepSeek-R1-Distill-Llama-8B | Llama-3.1-8B | 🤗 HuggingFace DeepSeek-R1-Distill-Qwen-14B | Qwen2.5-14B | 🤗 HuggingFace DeepSeek-R1-Distill-Qwen-32B | Qwen2.5-32B | 🤗 HuggingFace DeepSeek-R1-Distill-Llama-70B | Llama-3.3-70B-Instruct | 🤗 HuggingFace

Yes, we will add Deepseek and other new v1.3 models into the README.

[joshuayao]

Hi @XinyaoWa, could you please add DeepSeek v3, 6 DeepSeek distilled models and phi4 into the doc after everything goes well? Thanks.

[yinghu5]

@zhiweizhangintc @joshuayao thank xinyao fix the problem. Please see if the issue can be closed. [remind]

[joshuayao]

@zhiweizhangintc @joshuayao thank xinyao fix the problem. Please see if the issue can be closed. [remind]

Hi @yinghu5 LGTM.
Hi @zhiweizhangintc Does the model list look good?