17. Adapter Management

Chapter 17 of 24 · 20 min

KEY INSIGHT

LoRA adapters create modular upgrades to base models. Managing multiple adapters requires systematic organizationâ€”version control, naming conventions, and loading mechanisms that prevent conflicts. Each adapter consists of the learned A and B matrices plus metadata about training configuration, dataset, and purpose. Storing these together enables reproducibility and deployment flexibility. ```python import os import json from pathlib import Path class AdapterRegistry: def __init__(self, registry_dir="./adapters"): self.registry_dir = Path(registry_dir) self.registry_file = self.registry_dir / "registry.json" self.adapters = self.load_registry() def load_registry(self): if self.registry_file.exists(): with open(self.registry_file) as f: return json.load(f) return {"adapters": []} def register(self, adapter_id, adapter_path, metadata): entry = { "id": adapter_id, "path": str(adapter_path), "metadata": metadata } self.adapters["adapters"].append(entry) self.save() def save(self): self.registry_dir.mkdir(parents=True, exist_ok=True) with open(self.registry_file, "w") as f: json.dump(self.adapters, f, indent=2) def list_adapters(self, filter_fn=None): adapters = self.adapters["adapters"] if filter_fn: return [a for a in adapters if filter_fn(a)] return adapters ``` **Loading multiple adapters** requires careful weight management: ```python from peft import PeftModel, PeftConfig def load_adapter(base_model, adapter_path, adapter_name="default"): """Load a single adapter onto the base model.""" model = PeftModel.from_pretrained( base_model, adapter_path, adapter_name=adapter_name ) return model def load_multiple_adapters(base_model, adapter_configs): """Load multiple adapters, each with unique names.""" model = base_model for config in adapter_configs: model = PeftModel.from_pretrained( model, config["path"], adapter_name=config["name"] ) return model ``` **Adapter merging vs. switching**: Running multiple adapters simultaneously requires either merging weights (slower inference, more flexible) or routing (faster inference, requires routing logic). ```python # Switch between adapters without reloading base model def switch_adapter(model, adapter_name): model.set_adapter(adapter_name) return model ```

Local verification checkpoint

Run the smallest example from this chapter in a local workspace and record the package version, runtime, data path, and observed output. If the result depends on model size, vector count, CPU/GPU backend, or available memory, note that constraint beside the exercise so the lesson remains reproducible.

EXERCISE

: Implement Adapter Versioning

Create a system that tracks adapter lineageâ€”learning which adapters influenced which others:

class VersionedAdapter:
    def __init__(self, base_model_id, adapter_id, version, parent_ids=None):
        self.base_model_id = base_model_id
        self.adapter_id = adapter_id
        self.version = version
        self.parent_ids = parent_ids or []
    
    def save(self, path):
        metadata = {
            "base_model_id": self.base_model_id,
            "adapter_id": self.adapter_id,
            "version": self.version,
            "parent_ids": self.parent_ids
        }
        with open(Path(path) / "version_info.json", "w") as f:
            json.dump(metadata, f)
    
    @classmethod
    def load(cls, path):
        with open(Path(path) / "version_info.json") as f:
            metadata = json.load(f)
        return cls(**metadata)