21. Catastrophic Forgetting

Chapter 21 of 24 · 20 min

KEY INSIGHT

Catastrophic forgetting occurs when fine-tuning overwrites pre-trained capabilities. A model that excelled at general reasoning may lose those skills after training on a narrow domain. This represents one of the fundamental challenges in transfer learning. The mechanism: gradient updates during fine-tuning modify weights that were crucial for pre-trained behaviors. Without regularization toward the original function, the model drifts toward the fine-tuning distribution. **Diagnosis**: Evaluate on both pre-training tasks and fine-tuning tasks before and after training. ```python def diagnose_forgetting(before_model, after_model, benchmark_datasets): """ Compare model performance before and after fine-tuning. Returns degradation scores for each capability. """ results = {} for dataset_name, dataset in benchmark_datasets.items(): # Test original model orig_metrics = evaluate_model(before_model, dataset) # Test fine-tuned model new_metrics = evaluate_model(after_model, dataset) degradation = { metric: (orig_metrics[metric] - new_metrics[metric]) / orig_metrics[metric] for metric in orig_metrics } results[dataset_name] = { "before": orig_metrics, "after": new_metrics, "degradation_pct": degradation } return results ``` **Mitigation strategies**: **1. Regularization toward pre-trained weights:** ```python def lora_with_distillation(model, lora_model, teacher_model, alpha=0.5): """ Combine task loss with knowledge distillation from original model. """ def forward_hook(module, input, output): # Compute KL divergence between student and teacher logits student_logits = output with torch.no_grad(): teacher_logits = teacher_model(module.input[0]) kl_loss = F.kl_div( F.log_softmax(student_logits, dim=-1), F.softmax(teacher_logits, dim=-1), reduction="batchmean" ) return kl_loss * alpha return forward_hook ``` **2. Mixed fine-tuning with pre-training data:** ```python def create_mixed_dataset(task_data, pretrain_data, ratio=0.2): """Mix task-specific data with general pre-training data.""" sampled_pretrain = pretrain_data.shuffle().select( range(int(len(task_data) * ratio)) ) return datasets.concatenate_datasets([task_data, sampled_pretrain]) ``` **3. Smaller learning rate with more epochs:** ```python # Conservative fine-tuning hyperparameters hyperparameters = { "learning_rate": 5e-5, # Much lower than default "warmup_ratio": 0.1, # Gradual warmup "weight_decay": 0.1, # Stronger regularization "num_epochs": 3, # More epochs with lower LR } ```

EXERCISE

: Quantify Forgetting in Your Fine-Tune

def measure_forgetting(adapter_path, base_model_path, eval_tasks):
    """
    Measure capability degradation after applying adapter.
    """
    base_model = AutoModelForCausalLM.from_pretrained(base_model_path)
    adapted_model = PeftModel.from_pretrained(base_model, adapter_path)
    
    print("Base model performance:")
    base_results = evaluate_all_tasks(base_model, eval_tasks)
    
    print("Adapted model performance:")
    adapted_results = evaluate_all_tasks(adapted_model, eval_tasks)
    
    # Report degradation
    for task in eval_tasks:
        delta = adapted_results[task]["accuracy"] - base_results[task]["accuracy"]
        print(f"{task}: {delta:+.2%} change")