18. Merging Adapters

Chapter 18 of 24 · 20 min

KEY INSIGHT

Adapter merging consolidates learned weights into the base model, eliminating runtime overhead from the adapter architecture. The merged model behaves identically to running with the adapter active but requires no special loading logic. Merging is essential for deployment scenarios where inference latency matters. An adapter with rank 8 adds 8Ã— reduction matrix multiplications per attention layerâ€”overhead that accumulates at scale. ```python from peft import PeftModel import torch def merge_adapter_to_base(model, adapter_path, output_path): """ Merge a LoRA adapter into its base model. After merging, the model contains the full weights. """ # Load base model base_model = model.base_model.model # Load adapter model = PeftModel.from_pretrained(model, adapter_path) # Merge weights merged_model = model.merge_and_unload() # Save merged model merged_model.save_pretrained(output_path) return merged_model ``` **Merging order matters** with multiple adapters. The mathematical composition depends on whether adapters should blend or replace each other: ```python # Sequential merge: adapter_a + adapter_b = combined def merge_multiple(base_model_path, adapter_paths, output_path): base_model = AutoModelForCausalLM.from_pretrained(base_model_path) model = base_model for adapter_path in adapter_paths: model = PeftModel.from_pretrained(model, adapter_path) model.merge_and_unload() model.save_pretrained(output_path) return model ``` **Weighted merging** combines adapters by contribution: ```python def weighted_merge(model, adapter_weights): """ Merge multiple adapters with weighted averaging. adapter_weights: dict of {adapter_name: weight} """ # Load all adapters model = load_multiple_adapters(model.base_model.model, [ {"path": path, "name": name} for name, path in adapter_weights.items() ]) # Weighted average in LoRA space with torch.no_grad(): for name, param in model.named_parameters(): if "lora_" in name: # Extract adapter weights and combine pass # Implementation depends on weight structure return model.merge_and_unload() ``` **Failure mode**: Merging can cause numerical instability if adapter weights have different scales. Check that merged model outputs are numerically similar to adapter outputs before deployment.

EXERCISE

: Validate Merge Correctness

Compare outputs before and after merging to ensure no degradation:

def validate_merge(original_model, merged_model, test_inputs, tokenizer):
    """Verify merged model produces identical outputs."""
    original_model.eval()
    merged_model.eval()
    
    max_diff = 0
    for test_input in test_inputs:
        inputs = tokenizer(test_input, return_tensors="pt").to("cuda")
        
        with torch.no_grad():
            orig_output = original_model(**inputs).logits[0, -1]
            merged_output = merged_model(**inputs).logits[0, -1]
        
        diff = (orig_output - merged_output).abs().max().item()
        max_diff = max(max_diff, diff)
    
    print(f"Maximum logit difference: {max_diff:.6f}")
    assert max_diff < 1e-3, "Merge introduced significant deviation"