Questions

Fine-tuning is NOT dead in 2026 — but the cases where it wins are narrower than 2023-2024. The honest framework: prompt → RAG → distillation → fine-tune ladder.

Fine-tuning vs prompting vs RAG decision framework — when each one wins, when each one wastes money, and the 3-question test that tells you which to pick.

Privacy-first local AI migration — what stays local, what you give up, the realistic hardware path, and the apps that don't phone home.

The 5-layer local LLM inference stack — CUDA / Metal / ROCm at the bottom, llama.cpp / TensorRT-LLM in the middle, Ollama / vLLM as serving runtimes, OpenAI-compat API on top, apps consuming it.

KV cache vs RAG for local LLM apps — when prefix caching beats retrieval-augmentation, the latency math, and the operator decision rule.

Local LLM vs cloud API latency — the honest 200-1000ms tax cloud adds, when it matters, when it doesn't, and the local-first decision rule.

When cloud LLM pricing shifts and you're feeling the squeeze — here's the realistic local migration path by workload (coding, chat, RAG, voice).

4GB VRAM in 2026 — what you can run, what you can't, and the realistic use cases. RX 570 to GTX 1650 owners: here's the honest math.

Local voice-to-voice in 2026 — Whisper for STT + local LLM + Piper for TTS. The realistic latency, hardware requirements, and the wiring guide.

NVFP4 explained — NVIDIA's 4-bit floating-point format, why it claims 75% lossless compression, which hardware supports it, and whether the hype is real.

Hosting (Ollama) vs serving (vLLM) — what each one optimizes for, when each one fails, and the runtime decision rule for solo vs production deployments.

Two used RTX 3090s ($1,200) vs a new RTX 5090 ($1,999) for local AI. VRAM, bandwidth, software complexity, TDP, the actual decision matrix.

Llama 3.3 70B needs ~42GB VRAM at Q4_K_M with 8K context. The realistic GPU options: dual 3090, RTX 6000 PRO Blackwell, M3 Ultra. The honest tradeoffs.

Qwen 3.6 35B-A3B with MTP vs Qwen 3 32B — the throughput uplift, the runtime requirements (vLLM 0.20+ / llama.cpp post-b9148), and when to wait.

MTP support matrix as of May 2026 across vLLM, llama.cpp, Ollama, MLX. Why some runtimes get the throughput gain and others don't.

Distributed inference for local LLMs across a small fleet — what works (vLLM tensor parallel, MLX distributed, exo, Petals), what doesn't, and the network-overhead reality.

Real story: a developer got a $30K AWS bill from a misconfigured agent loop. The five controls that prevent it — rate limits, budgets, local-first defaults, observability, kill switches.

Jetson Orin Nano Super for local AI: 8GB VRAM, 60-100 GB/s memory bandwidth, ARM Linux. What it can run, what it can't, and the realistic use cases.

Ollama vs llama.cpp vs vLLM compared: when each one wins, when each one fails, and the runtime-decision rule that actually fits your stack.

Cline, Aider, Continue, Tabby, Twinny — the 5 coding agents that work against local models. Editorial verdict + when to pick each one.

Coding agents chain 8-12 model calls per edit. Small quant errors compound. The honest Q4 vs Q5 vs Q6 vs Q8 recommendation for agent workloads.

Local LLMs don't have web search by default — here's why, what realistic alternatives exist, and how to wire up offline RAG or air-gapped web tooling that actually works.

Q4_K_M vs Q6_K on Qwen 3 32B: the honest perplexity delta, when it matters, and when it doesn't. Quality curve + VRAM fit visualization included.