Transfer Learning

Transfer learning is a technique where a model trained on one task is reused as the starting point for a second task. In local AI, this means downloading a pre-trained model (e.g., Llama 3.1 8B) and fine-tuning it on your own data instead of training from scratch. This saves VRAM, time, and data because the model already understands language structure. Operators encounter transfer learning when they use Hugging Face Transformers to fine-tune a base model for a custom task, or when they apply LoRA adapters to a quantized model.

Transfer learning works by taking the weights of a model trained on a large, general dataset (e.g., Common Crawl) and then continuing training on a smaller, domain-specific dataset. The earlier layers (which capture general features like syntax and semantics) are often frozen, while later layers are fine-tuned. In local AI, this is critical because training a 7B model from scratch would require thousands of GPU-hours and terabytes of data, while fine-tuning can be done on a single consumer GPU in hours. Variants include full fine-tuning (updates all weights) and parameter-efficient fine-tuning (e.g., LoRA, which updates only a small set of injected weights). Operators typically use libraries like Hugging Face Transformers, Axolotl, or Unsloth to perform transfer learning on their own hardware.

An operator wants a model that answers questions about their company's internal documentation. Instead of training a 7B model from scratch (which would need 56 GB VRAM and weeks of training), they download Llama 3.1 8B (16 GB at FP16) and fine-tune it on 10,000 Q&A pairs using LoRA. With a 24 GB RTX 4090, fine-tuning takes ~4 hours and uses ~12 GB VRAM. The resulting adapter file is ~100 MB, which can be loaded alongside the base model at inference time.

In Hugging Face Transformers, an operator loads a pre-trained model with AutoModelForCausalLM.from_pretrained('meta-llama/Llama-3.1-8B'), then uses the Trainer API with their dataset. They set peft_config to LoraConfig to enable LoRA. After training, they save the adapter with model.save_pretrained('./my-adapter'). At inference, they load the base model and the adapter using PeftModel.from_pretrained(base_model, './my-adapter'). In Ollama, transfer learning isn't directly supported; operators use custom scripts or tools like Unsloth.