LiteLLM
Drop-in OpenAI-compatible proxy across 100+ providers. Route to local Ollama or cloud, same code.
Editorial verdict: “Best universal LLM proxy. Foundational layer for multi-provider deployments.”
Compatibility at a glance
Which runtime + OS combos this app works against. Source of truth for "will it run on my setup?"
What it is
LiteLLM is a proxy that exposes OpenAI's API shape but routes to 100+ backends: Anthropic, Gemini, Ollama, Together, Groq, local llama.cpp, etc. Drop it between your app and your LLMs, and your app code stays OpenAI-shaped while you swap providers freely. Excellent for migration and A/B testing.
✓ Strengths
- +Genuinely universal — 100+ providers
- +OpenAI-shaped API stays put
- +Built-in fallback / retry / cost tracking
△ Caveats
- −Some advanced features paid (LiteLLM Cloud)
- −Extra moving part in your stack
About the SDK / proxy category
Thin SDK / proxy / compatibility layer.
Foundational primitive for Python scripts against Ollama. Official, maintained, typed.
Foundational primitive for Node + browser apps against Ollama. ESM-native, typed.
Cloud-only LLM router. Useful category, novel pricing. The 'unlimited' math has a known failure mode at heavy usage.
Where to go from here
Pre-filled with this app's recommended use case + budget tier. Get the full rig + runtime + model picks.
The full directory — filter by category, runtime, OS, privacy posture, or VRAM.
What this app talks to: Ollama, vLLM, llama.cpp, MLX, LM Studio. The upstream layer.
Did this app work for you on a specific rig? Submit the benchmark — it powers the model + hardware pages.