Drop-in OpenAI-compatible proxy across 100+ providers. Route to local Ollama or cloud, same code.
Editorial verdict: “Best universal LLM proxy. Foundational layer for multi-provider deployments.”
Which runtime + OS combos this app works against. Source of truth for "will it run on my setup?"
LiteLLM is the right choice when you need to swap between local Ollama or llama.cpp and cloud providers like Anthropic or Gemini without rewriting your app code. It exposes a single OpenAI-compatible API shape and routes to over 100 backends, making it ideal for teams A/B testing models or migrating from one provider to another. The built-in fallback, retry, and cost tracking handle operational overhead, but the free tier covers most needs—advanced features require the paid cloud version. This adds a proxy layer to your stack, so you trade simplicity for flexibility. Works on Linux, macOS, and Windows.
Thin SDK / proxy / compatibility layer.
Foundational primitive for Node + browser apps against Ollama. ESM-native, typed.
Foundational primitive for Python scripts against Ollama. Official, maintained, typed.
Cloud-only LLM router. Useful category, novel pricing. The 'unlimited' math has a known failure mode at heavy usage.
Pre-filled with this app's recommended use case + budget tier. Get the full rig + runtime + model picks.
The full directory — filter by category, runtime, OS, privacy posture, or VRAM.
What this app talks to: Ollama, vLLM, llama.cpp, MLX, LM Studio. The upstream layer.
Did this app work for you on a specific rig? Submit the benchmark — it powers the model + hardware pages.