Orchestration (agents)

Orchestration in the context of agents refers to the system that manages the lifecycle, communication, and task delegation among multiple AI agents. It decides which agent handles a request, in what order, and how results are combined. For local AI operators, orchestration matters because running multiple models or agents on a single machine requires careful VRAM and scheduling management to avoid overload and latency spikes.

Orchestration goes beyond simple routing. It involves maintaining state across agent interactions, handling retries, and coordinating parallel or sequential workflows. Common patterns include supervisor agents that delegate to specialized sub-agents, or pipeline orchestration where output from one agent feeds into another. In local setups, orchestration must account for limited VRAM: if agents share the same GPU, the orchestrator may need to load/unload models sequentially or use a single model with multiple capabilities (e.g., function calling). Tools like LangChain, CrewAI, and AutoGen provide orchestration frameworks; locally, operators often use them with Ollama or vLLM backends. The orchestrator also manages context windows, ensuring each agent receives the relevant conversation history without exceeding token limits.

A local rig with an RTX 4090 (24 GB VRAM) runs a two-agent workflow: a summarizer (Llama 3.1 8B Q4, ~5 GB) and a fact-checker (Mistral 7B Q4, ~4.5 GB). The orchestrator loads the summarizer, processes a document, then unloads it and loads the fact-checker to verify claims. If the orchestrator tried to keep both models in VRAM simultaneously, it would exceed capacity and trigger system-RAM offload, dropping tokens/sec from ~50 to ~5.