Supervised Learning

Supervised learning is a training paradigm where a model learns to map inputs to outputs using labeled data — each training example pairs an input (e.g., a sentence) with a ground-truth output (e.g., a translation). The model adjusts its weights to minimize the difference between its predictions and the labels. For local AI operators, supervised learning is how most instruction-tuned models (e.g., Llama 3.1 Instruct) are created: a base model is fine-tuned on (prompt, response) pairs so it learns to follow instructions. The quality and diversity of the labeled dataset directly determine how well the model generalizes.

In supervised learning, the training loop repeatedly feeds batches of labeled examples through the model, computes a loss (e.g., cross-entropy for classification, mean squared error for regression), and backpropagates gradients to update weights. For large language models, this is typically done in two stages: pretraining on unlabeled text (self-supervised), then supervised fine-tuning on curated instruction-response pairs. The operator-relevant nuance is that supervised fine-tuning is the step that turns a raw base model into a chatbot or task-specific tool. Datasets like OpenAssistant or Dolly are common sources of labeled pairs. The process requires significant GPU VRAM — a 7B model at full precision needs ~14 GB just for weights, plus optimizer states. Quantized fine-tuning (QLoRA) reduces this to ~6-8 GB, making it feasible on consumer cards. The key operator decision is balancing dataset size, learning rate, and number of epochs to avoid overfitting while achieving desired behavior.

A common operator use case is fine-tuning Llama 3.1 8B on a custom dataset of 1,000 support chat logs. Using QLoRA with bitsandbytes 4-bit quantization, the process fits on an RTX 4090 (24 GB VRAM). The operator prepares a JSONL file with 'instruction' and 'response' fields, then runs a script using Hugging Face Transformers + PEFT. Training takes 2-3 hours at batch size 4. The resulting adapter (200 MB) is merged into the base model for inference.

In practice, an operator using Ollama might download a base model like llama3.1:8b and then apply a supervised fine-tuned adapter via a Modelfile: FROM llama3.1:8b followed by ADAPTER ./lora.safetensors. The runtime loads the base weights and applies the adapter at inference time. Alternatively, using LM Studio, the operator can import a merged model (base + fine-tuned weights) and run it locally. The key workflow step is preparing the labeled dataset — often a CSV or JSONL file — and running a training script from the Unsloth or Axolotl framework.