LoRA (Low-Rank Adaptation)

LoRA is a parameter-efficient fine-tuning technique that adapts a large pre-trained model by training small low-rank matrices added alongside the original weights, leaving the base model frozen.

Instead of updating all 8 billion parameters of a Llama 3.1 8B model (which needs 60+ GB of VRAM and hours of compute), LoRA trains rank-16 or rank-32 adapter matrices — typically 1-2% of original parameter count. You can fine-tune a 7B model on a single 16 GB GPU in an afternoon.

LoRA adapters are tiny (~50-200 MB) and stack: you can load a base model once and swap LoRA adapters at inference time for different tasks. QLoRA combines LoRA with 4-bit quantization of the base model, dropping VRAM requirements another 4× — making 70B fine-tuning possible on a single 24 GB consumer card.

A support team wants a Llama 3.1 8B model that writes in their company's specific tone and cites their internal ticket-resolution patterns, without the cost of full fine-tuning. They collect 2,000 example ticket-response pairs, freeze the base model, and train a rank-16 LoRA adapter targeting the attention projection layers — the job finishes in a few hours on a single RTX 4090. The resulting adapter file is under 100 MB, versus 16 GB for the full model in FP16. They keep three adapters swappable at runtime: one for support tone, one for technical docs, one for sales emails — all loaded against the same cached base model weights, avoiding the VRAM cost of hosting three separate full checkpoints. For a 70B base model on a single 24 GB card, they'd reach for QLoRA instead, since the full-precision base model alone wouldn't fit.