Repositories list

• vllm

  Public
  A high-throughput and memory-efficient inference and serving engine for LLMs

  amdcudainference+ 15pytorchtransformerllamagptrocmmodel-servingtpu+ 8

  Python

  •

  Apache License 2.0

  •4.5k•30k•1.8k•396•Updated Nov 10, 2024Nov 10, 2024

• llm-compressor

  Public
  Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

  sparsitycompressionquantization

  Python

  •

  Apache License 2.0

  •54•662•28•30•Updated Nov 9, 2024Nov 9, 2024

• buildkite-ci

  Public
  HCL

  •15•6•0•3•Updated Nov 6, 2024Nov 6, 2024

• vllm-project.github.io

  Public
  HTML

  •

  MIT License

  •5•4•0•0•Updated Oct 30, 2024Oct 30, 2024

• vllm-blog-source

  Public
  SCSS

  •

  MIT License

  •1•1•0•0•Updated Oct 30, 2024Oct 30, 2024

• flash-attention

  Public
  Fast and memory-efficient exact attention

  C++

  •

  BSD 3-Clause "New" or "Revised" License

  •1.3k•28•0•3•Updated Oct 25, 2024Oct 25, 2024

• vllm-nccl

  Public archive
  Manages vllm-nccl dependency

  Python

  •

  Apache License 2.0

  •2•17•2•0•Updated Jun 3, 2024Jun 3, 2024

• dashboard

  Public
  vLLM performance dashboard

  Python

  •

  Apache License 2.0

  •3•18•0•0•Updated Apr 26, 2024Apr 26, 2024