LiteLLM
Drop-in OpenAI-compatible proxy across 100+ providers. Route to local Ollama or cloud, same code.
Editorial verdict: “Best universal LLM proxy. Foundational layer for multi-provider deployments.”
Compatibility at a glance
Which runtime + OS combos this app works against. Source of truth for "will it run on my setup?"
What it is
LiteLLM is the right choice when you need to swap between local Ollama or llama.cpp and cloud providers like Anthropic or Gemini without rewriting your app code. It exposes a single OpenAI-compatible API shape and routes to over 100 backends, making it ideal for teams A/B testing models or migrating from one provider to another. The built-in fallback, retry, and cost tracking handle operational overhead, but the free tier covers most needs—advanced features require the paid cloud version. This adds a proxy layer to your stack, so you trade simplicity for flexibility. Works on Linux, macOS, and Windows.
✓ Strengths
- +Genuinely universal — 100+ providers
- +OpenAI-shaped API stays put
- +Built-in fallback / retry / cost tracking
△ Caveats
- −Some advanced features paid (LiteLLM Cloud)
- −Extra moving part in your stack
About the SDK / proxy category
Thin SDK / proxy / compatibility layer.
Foundational primitive for Node + browser apps against Ollama. ESM-native, typed.
Foundational primitive for Python scripts against Ollama. Official, maintained, typed.
Cloud-only LLM router. Useful category, novel pricing. The 'unlimited' math has a known failure mode at heavy usage.
Where to go from here
Pre-filled with this app's recommended use case + budget tier. Get the full rig + runtime + model picks.
The full directory — filter by category, runtime, OS, privacy posture, or VRAM.
What this app talks to: Ollama, vLLM, llama.cpp, MLX, LM Studio. The upstream layer.
Did this app work for you on a specific rig? Submit the benchmark — it powers the model + hardware pages.