Fine-tuning

Fine-tuning is continued training of a pre-trained model on a smaller, task-specific dataset. Pre-training builds general capability from trillions of tokens; fine-tuning specializes the model for a domain, voice, or task using thousands to millions of examples.

Three flavors: full fine-tuning updates every parameter (expensive, best quality, requires 80GB+ VRAM for 7B-class), LoRA / QLoRA updates small adapter matrices (cheap, fits on consumer hardware, slight quality loss), instruction tuning is a form of fine-tuning that teaches the model to follow instructions in a chat-like format.

For local AI: fine-tune when you need consistent voice/format/behavior that prompt engineering can't reliably produce. Don't fine-tune to "add knowledge" — RAG handles that better and is reversible. Tools: Unsloth (fastest, NVIDIA-only), Axolotl (most flexible), HuggingFace TRL (reference implementation).