Sonar (formerly Aphrodite Engine)

Inference engine based on vLLM, formerly named Aphrodite Engine and now named Sonar. Adds samplers (smoothing factor, dynatemp, mirostat, DRY, XTC) that mainline vLLM doesn't ship. Same continuous-batching architecture.

Install in a Python virtual environment: uv pip install aphrodite-engine --torch-backend=cu130, or use the installer: curl -fsSL https://sonar.dphn.ai/install.sh | bash. The project is now named Sonar; the pip package and CLI keep the aphrodite name. Current source builds and nightly wheels require Python 3.11-3.14. Start: aphrodite serve meta-llama/Llama-3.1-8B-Instruct --port 2242. The server exposes an OpenAI-compatible API at /v1/chat/completions. Verify: curl http://localhost:2242/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"meta-llama/Llama-3.1-8B-Instruct","messages":[{"role":"user","content":"Hello"}]}'. Sonar keeps continuous batching and paged KV-cache management from vLLM. It supports quantized weights and an FP8 KV cache; check the quantization table for AWQ, GPTQ and EXL3 support on your hardware. Kobold-compatible routes are enabled with --launch-kobold-api, and SillyTavern has its own Aphrodite API type.

Best for: self-hosted LLM serving on Linux or macOS with NVIDIA, AMD or Intel GPUs or Apple silicon, roleplay and creative writing scenarios that need samplers such as DRY, XTC and mirostat, SillyTavern and character-chat frontend integration, users who want vLLM's continuous batching and paged KV cache with extra samplers. Not suited for: native Windows (use WSL 2), deployments that cannot accept the AGPL-3.0 license (use vLLM), users who need automatic model management (use Ollama).

Use Sonar (formerly Aphrodite Engine) when you want vLLM-style serving with extra samplers such as DRY, XTC and mirostat, especially for roleplay and creative writing. Switch to vLLM when you need an Apache 2.0 license rather than AGPL-3.0. Use ExLlamaV2 when you want maximum single-stream decode speed on consumer NVIDIA GPUs and can accept EXL2 format conversion. Use Ollama for zero-config desktop LLM with automatic model management — Aphrodite requires explicit model specification and Python environment setup. Use KoboldCPP when you need a bundled chat UI, Windows-native deployment without Python, and the full GGUF ecosystem. Aphrodite adds samplers and model formats on top of vLLM.

Problem: Install fails on native Windows. Fix: Native Windows builds are not supported. Use WSL 2. Problem: A server flag from an older guide is rejected. Fix: Check startup warnings for renamed flags and deprecated values, and compare with the generated server argument reference. Problem: SillyTavern connection fails. Fix: Choose the Aphrodite API type in SillyTavern and point it at the server, which listens on http://127.0.0.1:2242 by default. For KoboldAI clients, start the server with --launch-kobold-api.