RunLocalAI
server
Open source
free
4.2/5

Text Generation Inference (TGI)

HuggingFace's production inference server. Slightly behind vLLM on raw throughput but tighter integration with the HF ecosystem.

By Fredoline Eruo·Last verified May 6, 2026·9,500 GitHub stars

Overview

HuggingFace's production inference server. Slightly behind vLLM on raw throughput but tighter integration with the HF ecosystem.

Pros

  • Tight HF integration
  • Production-tested at HF scale

Cons

  • Linux only
  • GPU only

Compatibility

Operating systems
Linux
GPU backends
NVIDIA CUDA
AMD ROCm
Intel
LicenseOpen source · free

Get Text Generation Inference (TGI)

GitHub
https://github.com/huggingface/text-generation-inference

Frequently asked

Is Text Generation Inference (TGI) free?

Yes — Text Generation Inference (TGI) is free to download and use and open-source under a permissive license.

What operating systems does Text Generation Inference (TGI) support?

Text Generation Inference (TGI) supports Linux.

Which GPUs work with Text Generation Inference (TGI)?

Text Generation Inference (TGI) supports NVIDIA CUDA, AMD ROCm, Intel. CPU-only inference is also possible but slow.

Reviewed by RunLocalAI Editorial. See our editorial policy for how we evaluate tools.