Instruction Tuning

Instruction tuning is a supervised fine-tuning step where a base language model is trained on (instruction, response) pairs to improve its ability to follow user prompts. Unlike raw pretraining, which predicts the next token on internet text, instruction tuning teaches the model to interpret commands, answer questions, and perform tasks conversationally. This step is what turns a raw base model (e.g., Llama 3.1 base) into a chat model (e.g., Llama 3.1 Instruct). Operators encounter it when downloading chat-optimized variants of models, which have been instruction-tuned on datasets like OpenAssistant or Dolly.

Instruction tuning typically uses a dataset of thousands to millions of (instruction, response) pairs, often generated by a larger model (e.g., GPT-4) or curated by humans. The model is fine-tuned with a standard language modeling loss, but only on the response tokens, not the instruction. This teaches the model to condition its output on the user's request. Variants include multi-turn instruction tuning (for dialogue) and task-specific tuning (e.g., code generation). The process is computationally cheaper than pretraining—often a few hours on a single GPU for a 7B model. After instruction tuning, models are often further aligned with RLHF or DPO to reduce harmful outputs. For operators, the key takeaway: always use the instruct/chat version of a model for interactive use; base models are for research or fine-tuning from scratch.

A base Llama 3.1 8B model, when prompted with "Write a poem about AI," might continue with random internet text. The instruction-tuned version (Llama 3.1 8B Instruct) will actually write a poem. Operators see this distinction when pulling models from Hugging Face or Ollama: "llama3.1:8b" is the instruct version, while "llama3.1:8b-text" is the base. Using the base model for chat yields poor results.

In Ollama, running ollama pull llama3.1:8b downloads the instruction-tuned model. If you instead pull the base model (e.g., llama3.1:8b-text), you'd need to fine-tune it yourself to get chat behavior. In LM Studio, the model card clearly labels "Instruct" or "Chat" versions. When fine-tuning your own model with tools like Unsloth or Axolotl, you prepare an instruction dataset (e.g., in Alpaca format) and run a training script that applies instruction tuning on top of a base model.