GPU
Definition pending
We've cataloged "GPU" but haven't written a full definition yet. Definitions are hand-curated rather than auto-generated, so it takes time to cover every term.
Want this one prioritized? Email us and we'll bump it.
Practical example
The GPU is the engine of local AI. The key number is VRAM capacity, not TFLOPS. A 16 GB RTX 4080 will actually run more models than a 12 GB RTX 4070 Ti, even though the 4070 Ti has slightly higher TFLOPS. For LLM inference: VRAM > memory bandwidth > compute. The model must fit in VRAM or it spills to system RAM at 10× speed penalty.
Workflow example
GPU selection by use case: (1) 8–12 GB VRAM (RTX 4060 Ti, M3 Pro): 7B Q4 models comfortably, 13B Q4 at low context, (2) 16–24 GB VRAM (RTX 4090, M3 Max): 13B–34B Q4, 70B Q4 with CPU offload, (3) 48–80 GB VRAM (A6000, A100): 70B Q4 fully on GPU, 70B FP16, (4) 80+ GB VRAM (H100, multiple GPUs): 70B+ FP16, 405B Q4. Check /hardware/ for specific GPU benchmarks.