Popular repositories Loading

1. TensorRT-LLM TensorRT-LLM Public

   Forked from NVIDIA/TensorRT-LLM

   TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

   C++

2. SparTA SparTA Public

   Forked from microsoft/SparTA

   Python

3. nnfusion nnfusion Public

   Forked from microsoft/nnfusion

   A flexible and efficient deep neural network (DNN) compiler that generates high-performance executable from a DNN model description.

   C++

4. PowerInfer PowerInfer Public

   Forked from SJTU-IPADS/PowerInfer

   High-speed Large Language Model Serving for Local Deployment

   C++

5. llama.cpp llama.cpp Public

   Forked from ggml-org/llama.cpp

   LLM inference in C/C++

   C++

6. sparse sparse Public

   Forked from zheng-ningxin/SparTA

   Cuda