RUNLOCALAIv38
→WILL IT RUNBEST GPUCOMPARETROUBLESHOOTSTARTPULSEMODELSHARDWARETOOLSBENCH
RUNLOCALAI

Operator-grade instrument for local-AI hardware intelligence. Hand-written verdicts. Real benchmarks. Reproducible commands.

OP·Fredoline Eruo
DIR
  • Models
  • Hardware
  • Tools
  • Benchmarks
  • Will it run?
GUIDES
  • Best GPU
  • Best laptop
  • Best Mac
  • Best used GPU
  • Best budget GPU
  • Best GPU for Ollama
  • Best GPU for SD
  • AI PC build $2K
  • CUDA vs ROCm
  • 16 vs 24 GB
  • Compare hardware
  • Custom compare
REF
  • Systems
  • Ecosystem maps
  • Pillar guides
  • Methodology
  • Glossary
  • Errors KB
  • Troubleshooting
  • Resources
  • Public API
EDITOR
  • About
  • About the author
  • Changelog
  • Latest
  • Updates
  • Submit benchmark
  • Send feedback
  • Trust
  • Editorial policy
  • How we make money
  • Contact
LEGAL
  • Privacy
  • Terms
  • Sitemap
MAIL · MONTHLY DIGEST
Get monthly local AI changes
Monthly recap. No spam.
DISCLOSURE

Some links on this site are affiliate links (Amazon Associates and other first-class retailers). When you buy through them, we earn a small commission at no extra cost to you. Affiliate links do not influence our verdicts — there are cards we rate highly that we don't have affiliate relationships with, and cards that sell well that we refuse to recommend. Read more →

SYS · ONLINEUPTIME · 100%2026 · operator-owned
RUNLOCALAI · v38
  1. >
  2. Home
  3. /Runtime health
Live runtime status
✓Editorial

Local AI runtime health

Single-glance answer for every major local AI inference engine: is the project active, how much of our benchmark corpus touches it, what's the failure mode if you deploy it. Live counts pulled from the database; cadence labels derived from real timestamps only.

See the runtime-health methodology for how labels are derived, what we measure, and what we don't.

Runtimes tracked
94
Active
94
Stalled
0
Reproduced runs
0
Eval-harness OK
7

Ollama

active · 5d
Setup: low
Latest measurement: 2026-05-10 (fresh)

runner · 5 editorial benchmarks · 0 reproduced community runs

Best workloads
  • · First local-AI deployment
  • · Single-user personal inference
  • · Drop-in OpenAI-compatible API
Avoid if
  • · Custom build flags / experimental kernels needed
  • · Multi-user serving at scale
  • · Reproducibility requires exact runtime version pinning
Common failure modes
  • · Auto-update can ship llama.cpp regressions
  • · WSL backend flakiness on Windows GPU
  • · Daemon restart loses concurrent state
OS support
LinuxmacOSWindows
Hardware
NVIDIAAMD ROCmApple Metal
Compared withOllama vs llama.cppOllama vs LM Studio

MLX-LM

active · 5d
Latest measurement: 2026-05-13 (fresh)

runner · 4 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

llama.cpp

active · 5d
Setup: moderate
Latest measurement: 2026-05-13 (fresh)

runner · 4 editorial benchmarks · 0 reproduced community runs

Best workloads
  • · Cross-platform single-user inference
  • · Mobile / iOS / Android / Pi
  • · Reproducible pinned-commit deployments
Avoid if
  • · Concurrent multi-user serving — sequential by default
  • · Production agent loops with parallel tool calls
Common failure modes
  • · GGUF format drift after major schema changes
  • · Metal kernel issues on macOS major-version transitions
  • · Vulkan support varies wildly by Intel/AMD driver
OS support
LinuxmacOSWindowsiOSAndroid
Hardware
NVIDIA CUDAApple MetalVulkan (any)CPU-only
Compared withOllama vs llama.cppvLLM vs llama.cppMLX vs llama.cpp

vLLM

active · 5d
Setup: high
Latest measurement: 2026-05-06 (fresh)

server · 4 editorial benchmarks · 0 reproduced community runs

Best workloads
  • · Production multi-user serving
  • · Tensor-parallel multi-GPU
  • · OpenAI-compatible API serving
Avoid if
  • · macOS host (unsupported)
  • · Single-user hobby — operator burden too high
  • · Fast-moving experimental architectures (lag at day-zero)
Common failure modes
  • · Flash-attention pinning incompatibilities
  • · OOM on long contexts when KV cache isn't pre-sized
  • · WSL2 GPU passthrough breakage on Windows kernel updates
OS support
LinuxWindows (WSL2)
Hardware
NVIDIAAMD ROCm
Compared withvLLM vs SGLangvLLM vs llama.cppTensorRT-LLM vs vLLM

Text Generation WebUI (oobabooga)

active · 7d

gui · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Jan

active · 7d

gui · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

ExLlamaV2

active · 5d

runner · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Text Generation Inference (TGI)

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Llamafile

active · 7d

runner · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Cursor

active · 5d

ide · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Stable Diffusion WebUI (AUTOMATIC1111)

active · 5d

gui · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

LlamaIndex

active · 5d

orchestrator · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Axolotl

active · 7d

finetuner · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Unsloth

active · 7d

finetuner · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

GPT4All

active · 7d

gui · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Open Interpreter

active · 7d

orchestrator · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Hugging Face Hub CLI

active · 7d

quantizer · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Pinokio

active · 7d

orchestrator · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Codex CLI

active · 5d

agent · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Windsurf (Codeium)

active · 7d

ide · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

JetBrains AI Assistant

active · 7d

ide · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Cline

active · 5d

agent · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Devin

active · 7d

agent · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

OpenCode

active · 7d

agent · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Kilo Code

active · 7d

agent · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

OpenAI Codex

active · 7d

agent · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Droid (Factory)

active · 7d

agent · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Replit Agent 3

active · 7d

agent · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Claude Desktop

active · 7d

agent · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Sourcegraph Cody

active · 7d

agent · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Zed (with AI)

active · 7d

ide · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Roo Code (sunsetting May 15, 2026)

active · 7d

agent · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Msty

active · 7d

gui · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Pi (Inflection AI)

active · 7d

agent · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

OpenHands

active · 7d

agent · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Qdrant

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Model Context Protocol (MCP)

active · 7d

agent · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Letta (memory framework)

active · 5d

agent · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Weaviate

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Neo4j GraphRAG

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Zep (memory platform)

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Chroma

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Milvus

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

SGLang

active · 5d
Setup: high

server · 0 editorial benchmarks · 0 reproduced community runs

Best workloads
  • · Heavy structured-output / function-calling agent loops
  • · Shared-prefix batched workloads (RadixAttention)
  • · Multi-architecture serving
Avoid if
  • · Want largest community / Stack Overflow surface
  • · macOS host
  • · Day-zero new architecture support
Common failure modes
  • · Smaller community = error messages with no Stack Overflow hits
  • · Architecture-specific kernel gaps
  • · Less mature observability — silent failures harder to spot
OS support
Linux
Hardware
NVIDIA
Compared withvLLM vs SGLang

LanceDB

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Redis (vector search)

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Graphiti (Zep)

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

LangSmith

active · 7d

orchestrator · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Open WebUI

active · 7d

gui · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Phoenix (Arize AI)

active · 7d

orchestrator · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Claude Code

active · 5d

agent · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

LocalAI

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

MCP Filesystem Server

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

MCP PostgreSQL Server

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

MCP Brave Search Server

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

IPEX-LLM

active · 7d

runner · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Playwright MCP

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

MCP Fetch Server

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Aider

active · 5d

agent · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

MCP GitHub Server

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

ComfyUI

active · 5d

gui · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

MCP Memory Server

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

MCP Sequential Thinking

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Firecrawl MCP

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

LibreChat

active · 7d

gui · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Ray Serve

active · 7d

orchestrator · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

MCP Git Server

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

CTranslate2

active · 7d

runner · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Intel OpenVINO

active · 7d

runner · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Continue

active · 5d

agent · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

TabbyAPI

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Petals

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

GitHub Copilot

active · 5d

ide · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Goose

active · 5d

agent · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

KoboldCPP

active · 5d

gui · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

LangChain

active · 5d

orchestrator · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

LM Studio

active · 5d
Setup: low

gui · 0 editorial benchmarks · 0 reproduced community runs

Best workloads
  • · Desktop chat interface for non-developers
  • · Browsing HuggingFace model library in-app
  • · Running local AI without a terminal
Avoid if
  • · Headless servers / homelab
  • · Embedded inference in scripts (use Ollama instead)
  • · Reproducibility requirements
Common failure modes
  • · Electron memory bloat on long sessions
  • · GUI updates can silently change inference defaults
  • · Server mode requires the app foregrounded on some OSes
OS support
macOSWindowsLinux
Hardware
NVIDIAApple MetalVulkan
Compared withOllama vs LM Studio

Mem0 (agent memory API)

active · 5d

agent · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Hyperspace (P2P inference network)

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Aphrodite Engine

active · 5d

runner · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

llama-cpp-python

active · 7d

runner · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

DirectML

active · 7d

runner · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

ONNX Runtime Mobile

active · 7d

runner · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

ExecuTorch

active · 7d

runner · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

MLC LLM

active · 5d

runner · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

MLX Swift

active · 7d

runner · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Exo

active · 7d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Qualcomm AI Hub

active · 7d

runner · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

TensorRT-LLM

active · 5d

server · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

OpenClaw

active · 7d

orchestrator · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

SillyTavern

active · 7d

gui · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

ONNX Runtime

active · 7d

runner · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

ROCm

active · 7d

runner · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

AnythingLLM

active · 7d

gui · 0 editorial benchmarks · 0 reproduced community runs

Editorial guidance pending. See the tool detail page for current information.

Next recommended step

See engine head-to-heads
OrLocal AI engine choice matrixBrowse benchmarks