Multi-Agent System

A multi-agent system (MAS) is a setup where multiple AI agents, each with distinct roles or capabilities, collaborate or compete to solve a task. In local AI, operators run multiple model instances (e.g., a planner agent, a coder agent, a reviewer agent) that communicate via structured messages. Each agent may use the same or different models, and the system coordinates their outputs—often through a supervisor agent or a shared context window. The key operator concern is resource overhead: each agent consumes VRAM and compute, so running 3+ agents on a single GPU may require smaller quantized models or offloading to system RAM.

Multi-agent systems in local AI are typically implemented as a loop where agents take turns generating responses, sometimes with a shared memory or tool-use layer. Common patterns include: (1) Supervisor/worker: one agent delegates subtasks to specialized agents. (2) Debate: agents argue different positions to refine an answer. (3) Role-playing: agents act as different personas (e.g., customer support, technical expert). Each agent is usually a separate inference call, so latency scales linearly with the number of agents. Operators often use frameworks like LangChain, CrewAI, or AutoGen to orchestrate these flows. On consumer hardware, a 2-agent system using Llama 3.1 8B Q4 might fit in 16 GB VRAM with a 4K context, but a 4-agent system would likely require offloading or smaller models (e.g., Qwen 2.5 7B Q4). The main bottleneck is context length: each agent's output is appended to a shared context, which grows quickly and can exceed VRAM limits.

A common multi-agent setup on a single RTX 4090 (24 GB VRAM) uses two agents: a 'planner' (Llama 3.1 8B Q4, ~5 GB) and a 'coder' (CodeLlama 7B Q4, ~4.5 GB). With a 4K context, total VRAM usage is ~12 GB, leaving headroom. If a third 'reviewer' agent is added, VRAM may hit 18 GB, risking OOM. Operators often reduce context to 2K or use a single model for all agents to save memory.

In Ollama, you can simulate a multi-agent system by running multiple model instances in separate terminals: ollama run llama3.1:8b for the planner, then ollama run codellama:7b for the coder, and pipe outputs manually. For automation, use a Python script with the Ollama API: each agent calls ollama.chat() with a system prompt defining its role, and the script manages the conversation history. In LM Studio, you can load multiple models simultaneously (if VRAM allows) and switch between them via the UI, but true multi-agent coordination requires external scripting.