Skip to content

@neuralmagic Neural Magic

All

    Repositories list

    116 repositories

    • vllm

      Public
      A high-throughput and memory-efficient inference and serving engine for LLMs
      Python
      •
      Apache License 2.0
      •24k•17•0•38•Updated Oct 30, 2026Oct 30, 2026

    • crucible

      Public
      Crucible: an agentic workflow engine
      kubernetesrustagents
      kubernetesrustagentsautonomous-researchllm
      Rust
      •
      Apache License 2.0
      •3•11•7•19•Updated Oct 29, 2026Oct 29, 2026

    • nm-vllm-omni-ent

      Public
      A framework for efficient model inference with omni-modality models
      Python
      •
      Apache License 2.0
      •2k•4•0•1•Updated Oct 29, 2026Oct 29, 2026

    • evals-refresh

      Public
      Shell
      •4•2•0•27•Updated Oct 29, 2026Oct 29, 2026

    • inspect_evals

      Public
      Collection of evals for Inspect AI
      Python
      •
      MIT License
      •470•0•0•14•Updated Oct 29, 2026Oct 29, 2026

    • vllm-vcr

      Public
      vLLM inference simulator and replay machine
      rustinferencevllm
      rustinferencevllm
      Rust
      •
      Apache License 2.0
      •2•11•15•15•Updated Oct 29, 2026Oct 29, 2026

    • model-validation-configs

      Public
      2•2•1•6•Updated Oct 28, 2026Oct 28, 2026

    • vllm-project.github.io

      Public
      HTML
      •161•0•0•0•Updated Oct 28, 2026Oct 28, 2026

    • speculators-dashboard

      Public
      HTML
      •0•1•34•1•Updated Oct 26, 2026Oct 26, 2026

    • axolotl

      Public
      Go ahead and axolotl questions
      Python
      •
      Apache License 2.0
      •1.5k•0•0•6•Updated Oct 25, 2026Oct 25, 2026

    • vllm-envs

      Public
      Fast disposable vLLM dev environments from any branch/commit, built on content-addressed layer caches
      Python
      •1•3•2•4•Updated Oct 24, 2026Oct 24, 2026

    • inspect_k8s_sandbox

      Public
      A Kubernetes sandbox environment for use with inspect_ai
      Python
      •
      MIT License
      •39•0•0•10•Updated Oct 23, 2026Oct 23, 2026

    • nm-hard-tools

      Public
      MCP services for rigorous inference evaluation and benchmarking
      Python
      •
      Apache License 2.0
      •0•0•0•6•Updated Oct 23, 2026Oct 23, 2026

    • compressed-tensors

      Public
      A safetensors extension to efficiently store sparse quantized tensors on disk
      Python
      •
      Apache License 2.0
      •143•0•0•1•Updated Oct 21, 2026Oct 21, 2026

    • llm-compressor

      Public
      State-of-the-art LLM compression, built for production inference with vLLM
      Python
      •
      Apache License 2.0
      •721•0•0•1•Updated Oct 21, 2026Oct 21, 2026

    • inspect_ai

      Public
      Inspect: A framework for large language model evaluations
      Python
      •
      MIT License
      •879•0•0•0•Updated Oct 21, 2026Oct 21, 2026

    • vllm-openshift-recipes

      Public
      0•0•0•0•Updated Oct 16, 2026Oct 16, 2026

    • DeepGEMM

      Public
      DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling
      Cuda
      •
      MIT License
      •1.5k•0•0•5•Updated Oct 13, 2026Oct 13, 2026

    • vllm-beam-search-plugin

      Public
      Beam search scheduler plugin for vLLM v1 with CoW block table forking
      Python
      •
      Apache License 2.0
      •2•4•1•0•Updated Oct 13, 2026Oct 13, 2026

    • transformers-gdm

      Public
      🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference an…
      Python
      •
      Apache License 2.0
      •35k•0•0•1•Updated Oct 9, 2026Oct 9, 2026

    • eval-hub-contrib

      Public
      Community-contributed evaluation framework adapters for eval-hub
      Python
      •
      Apache License 2.0
      •27•0•0•2•Updated Oct 8, 2026Oct 8, 2026

    • DeepEP

      Public
      DeepEP: an efficient expert-parallel communication library
      Cuda
      •
      MIT License
      •1.5k•1•0•1•Updated Oct 7, 2026Oct 7, 2026

    • eval-hub

      Public
      Go
      •
      Apache License 2.0
      •28•0•0•1•Updated Oct 7, 2026Oct 7, 2026

    • nyann-bench

      Public
      Go
      •
      Apache License 2.0
      •2•3•0•12•Updated Oct 5, 2026Oct 5, 2026

    • llm-manifesto

      Public
      Declarative Kubernetes renderer for llm-d and vLLM deployments
      Python
      •
      Apache License 2.0
      •0•2•0•0•Updated Oct 1, 2026Oct 1, 2026

    • rh-custom-models-plugin

      Public
      Python
      •
      Apache License 2.0
      •0•0•0•0•Updated Oct 1, 2026Oct 1, 2026

    • research

      Public
      Repository to enable research flows
      Python
      •2•4•0•4•Updated Sep 17, 2026Sep 17, 2026

    • gpu-pruner

      Public
      Non-destructive GPU based idle-culler for RHOAI/Kubeflow workloads
      kubernetesrustopenshift
      kubernetesrustopenshiftgpuk8skubernetes-operatorkubeflowrhoaikubeflow-notebooks
      Rust
      •
      MIT License
      •4•6•0•3•Updated Sep 11, 2026Sep 11, 2026

    • fs-offload-experiments

      Public
      Benchmark sweep experiments for FS Offloading
      HTML
      •0•0•0•0•Updated Sep 3, 2026Sep 3, 2026

    • tokenspeed

      Public
      TokenSpeed is a speed-of-light LLM inference engine.
      Python
      •
      MIT License
      •388•0•0•3•Updated Aug 19, 2026Aug 19, 2026
    ProTip! When viewing an organization's repositories, you can use the props. filter to filter by custom property.