Continual Learning

Continual learning (also called lifelong learning) is a machine learning paradigm where a model is trained on a sequence of tasks without forgetting previously learned knowledge. In practice, this means updating a model incrementally as new data arrives, rather than retraining from scratch on the entire dataset. The core challenge is catastrophic forgetting: when a neural network learns new patterns, its weights shift and can overwrite representations for earlier tasks. Operators encounter continual learning when fine-tuning a model on new domains (e.g., adding a new language to a multilingual model) while trying to retain performance on original tasks. Techniques like elastic weight consolidation (EWC) or replay buffers are used to mitigate forgetting, but they add complexity and memory overhead.

Continual learning is distinct from standard fine-tuning because the model must perform well on both old and new tasks after each update. There are three main families of approaches: (1) regularization-based methods (e.g., EWC, SI) that penalize changes to important weights, (2) replay-based methods that store a subset of old data (e.g., in a memory buffer) and interleave it with new data during training, and (3) architectural methods that allocate new parameters for each task (e.g., progressive neural networks). For local AI operators, continual learning is relevant when deploying models that need to adapt to user-specific data over time—for example, a chatbot that learns a user's writing style without forgetting general conversation skills. However, most local inference runtimes (llama.cpp, Ollama) do not natively support training or fine-tuning; continual learning typically requires a separate training framework like Hugging Face Transformers or MLX. The memory and compute cost of storing replay buffers or maintaining task-specific parameters can be significant on consumer hardware.

Consider fine-tuning Llama 3.1 8B to answer questions about a specific codebase. If you train only on new code-related data, the model may forget general chat abilities. Using a replay buffer of 10,000 samples from the original training set (e.g., from the Dolly dataset) and mixing them with new code data (50:50 ratio) during fine-tuning helps retain general knowledge. On an RTX 4090 with 24 GB VRAM, this requires storing the replay buffer in system RAM and loading batches during training, adding ~2 GB memory overhead for the buffer itself.

In Hugging Face Transformers, you would implement continual learning by creating a custom Trainer that samples from both a new dataset and a replay buffer. For example, using the transformers.Trainer with a DataCollator that interleaves batches. In MLX, you can write a training loop that alternates between new data and replay data stored as a memory-mapped array. Neither llama.cpp nor Ollama support training, so continual learning is not directly applicable in those runtimes. However, you could export the fine-tuned model as a GGUF file and then run inference with llama.cpp.