Learn local AI that will actually run

Builds a neural net, then a tiny GPT, in plain Python from scratch. The single best free intuition for why a transformer does what it does, and once you’ve watched the weights matter you understand why quantization is lossy at all. Where it lands for us: picking Q4_K_M vs Q6_K on /will-it-run/custom stops being a guess.

The follow-up that trains GPT-2 end to end. The systems detail — mixed precision, kernel fusion, FlashAttention — is what separates “I read the paper” from “I have shipped this.” Where it lands for us: explains why a 24GB card runs a 13B model at Q4 but chokes once context grows — the KV cache math gets concrete.

The visual companion. Watch when you want geometric intuition for attention or backprop’s chain rule without writing a line of code. Pair with Karpathy — don’t substitute. Where it lands for us: the attention visualisation is why our /benchmarks numbers swing so much with context length.

The most comprehensive free written treatment of pre-training, scaling, RLHF, and evaluation we’ve found. Reads like a textbook chapter, not a blog post. Where it lands for us: when a vendor claims their 70B model “beats GPT-4,” CS324 gives you the eval-design vocabulary to tell whether the claim is honest or staged.

The course we’d recommend to someone who’d rather build than watch. Notebooks for tokenization, fine-tuning, evaluation — less theory, more code that runs on a free Colab GPU. Where it lands for us: recreate a tokenizer in the chapter on BPE and finally see why Qwen handles Chinese gracefully and Llama 3.x stumbles.

The top-down approach: ship a state-of-the-art image model in lesson 1, derive what makes it work over the rest. Slightly dated on language models specifically, but the engineering instincts transfer wholesale. Where it lands for us: fast.ai’s “always try the obvious thing first” mindset is why our /benchmarks report real tokens/sec rather than synthetic FLOPs.

Not a course. The README is short; the Discussions tab is where the actual engineering tradeoffs play out — GGUF format changes, quantization formats, kernel tuning per chip. Where it lands for us: when a quantization format changes, this is where it’s argued first, and we cite it on /tools/llama-cpp.

Skip the install section, read the architecture pages. Paged attention is the most consequential inference optimisation of the last three years; understanding it teaches why long contexts are expensive even at small batch sizes. Where it lands for us: maps directly to why /hardware pages surface both memory bandwidth and capacity — both matter, for different reasons.

Dettmers wrote bitsandbytes and did the original LLM.int8() work. The clearest posts on quantization tradeoffs anywhere for free. Start with his 4-bit piece. Where it lands for us: every variants table on a /models page shows Q4_K_M / Q6_K / Q8_0 — Dettmers explains why those specific grades exist and which to pick.

The cleanest from-scratch explanations of RLHF and DPO outside the original papers. Working-engineer level, not researcher. Where it lands for us: explains why DeepSeek R1 explicitly recommends no system prompt — it’s a quirk of its preference-tuning setup, and we surface it as a kit caveat on /models/deepseek-r1.

Two complementary surfaces. Arena is crowdsourced human preference — hardest to game. The Open LLM Leaderboard is benchmark-based — easier to game, useful for capability cuts (math, code, reasoning). Where it lands for us: our /benchmarks page measures inference speed; Arena/Leaderboard cover model quality. Use both. Neither alone is enough.

The paper that started the transformer era. Surprisingly readable. Worth a careful read after Karpathy’s GPT-from-scratch video — the paper makes more sense once you’ve built the thing from the ground up. Where it lands for us: every architecture decision since 2017 references this paper; reading it once means the rest of the field stops feeling like jargon.