RUNLOCALAIv38
→WILL IT RUNBEST GPUCOMPARETROUBLESHOOTSTARTPULSEMODELSHARDWARETOOLSBENCH
RUNLOCALAI

Operator-grade instrument for local-AI hardware intelligence. Hand-written verdicts. Real benchmarks. Reproducible commands.

OP·Fredoline Eruo
DIR
  • Models
  • Hardware
  • Tools
  • Benchmarks
  • Will it run?
GUIDES
  • Best GPU
  • Best laptop
  • Best Mac
  • Best used GPU
  • Best budget GPU
  • Best GPU for Ollama
  • Best GPU for SD
  • AI PC build $2K
  • CUDA vs ROCm
  • 16 vs 24 GB
  • Compare hardware
  • Custom compare
REF
  • Systems
  • Ecosystem maps
  • Pillar guides
  • Methodology
  • Glossary
  • Errors KB
  • Troubleshooting
  • Resources
  • Public API
EDITOR
  • About
  • About the author
  • Changelog
  • Latest
  • Updates
  • Submit benchmark
  • Send feedback
  • Trust
  • Editorial policy
  • How we make money
  • Contact
LEGAL
  • Privacy
  • Terms
  • Sitemap
MAIL · MONTHLY DIGEST
Get monthly local AI changes
Monthly recap. No spam.
DISCLOSURE

Some links on this site are affiliate links (Amazon Associates and other first-class retailers). When you buy through them, we earn a small commission at no extra cost to you. Affiliate links do not influence our verdicts — there are cards we rate highly that we don't have affiliate relationships with, and cards that sell well that we refuse to recommend. Read more →

SYS · ONLINEUPTIME · 100%2026 · operator-owned
RUNLOCALAI · v38
← Home·/apps·RAG app

PrivateGPT

Fully offline

Air-gappable RAG over your docs. The OG offline-RAG project, now mature and team-friendly.

Editorial verdict: “Best when air-gap compliance is the requirement. Less polished than AnythingLLM, more configurable.”

RAG app
Free
Apache-2.0
★ 4.4 / 5
GitHub ★ 54,000
↗ Homepage↗ GitHub↗ Docs

Compatibility at a glance

Which runtime + OS combos this app works against. Source of truth for "will it run on my setup?"

§ Runtimes supported
ollamallama-cppopenai-compat
§ OS / platform
linuxmacoswindows
§ Hardware + model hint
Minimum VRAM
8 GB
Recommended starter model
Llama 3.1 8B Q4_K_M + bge-small-en embedder
→ Build the rest of the stack with /stack-builder→ Pick a GPU for this app

What it is

PrivateGPT is the OG project for 'chat with your docs, fully offline.' Mature ingestion pipeline, multiple UI options (FastAPI server + Gradio + headless), and a swappable LLM + embedder backend. Choose this when reproducible offline RAG matters more than UI polish.

✓ Strengths

  • +Air-gap-friendly from day one
  • +Swappable LLM + embedder + vector store
  • +Active maintainer, large community

△ Caveats

  • −UI is functional, not pretty
  • −Setup is more involved than AnythingLLM

About the RAG app category

Document retrieval + chat, fully offline-capable.

§ Other rag app apps
Verba

Best for 'don't make me choose chunking strategy' teams. Opinionated stack works.

Khoj

Best 'AI second brain' app. Self-hosted, local-first, works against Obsidian.

Where to go from here

Stack Builder →

Pre-filled with this app's recommended use case + budget tier. Get the full rig + runtime + model picks.

Back to /apps →

The full directory — filter by category, runtime, OS, privacy posture, or VRAM.

Runtimes (/tools) →

What this app talks to: Ollama, vLLM, llama.cpp, MLX, LM Studio. The upstream layer.

Community benchmarks →

Did this app work for you on a specific rig? Submit the benchmark — it powers the model + hardware pages.