AutoGen

What it is and how it works

AutoGen is Microsoft's framework for building applications out of multiple LLM-backed agents that communicate with each other to solve a task, rather than relying on a single model call or a fixed prompt chain. The core abstraction is the "conversable agent": an object with a system prompt, an optional set of tools/functions, an optional code executor, and rules about who it can talk to. You wire several of these together — a planner, a coder, a critic, a tool-caller, a human proxy — and let them exchange messages in a loop until a termination condition (a specific phrase, a max-turn count, or a human interrupt) is hit. The framework handles message routing, turn-taking, and (in group-chat mode) a "manager" agent that decides which participant speaks next based on the conversation so far.

Architecturally, AutoGen went through a significant split with the v0.4 rewrite. The original v0.2 line was a fairly monolithic conversational-agent library built directly on the OpenAI-style chat completion API. v0.4 restructured the project into layered packages: autogen-core provides an actor-model-style event/message-passing runtime that is intentionally LLM-agnostic, autogen-agentchat builds the familiar high-level conversational-agent and group-chat patterns on top of that core, and autogen-ext holds pluggable extensions (model clients, tools, code executors, memory backends). This means the newer AutoGen is less "a chatbot library" and more "a general async actor framework with a chat-agent layer on top" — a much bigger architectural bet, but one that has fragmented the user base, since a large amount of tutorial content, GitHub issues, and third-party integrations still target the v0.2 API surface.

Because AutoGen talks to models purely through an OpenAI-compatible chat completion interface, it has no hard dependency on any specific model provider. Any local server that speaks that API — Ollama, llama.cpp's server, vLLM, LM Studio, text-generation-webui — can be dropped in as a drop-in model client, typically just by pointing the base_url at the local endpoint. This is what makes it relevant to a local-AI audience: the multi-agent orchestration logic is decoupled from where the actual inference happens.

Deployment patterns

The most common local deployment shape is a single-machine setup: one operator runs a local inference backend (llama.cpp server, Ollama, or vLLM for anything GPU-heavy) and a Python process that instantiates the AutoGen agents against that endpoint. For code-execution-heavy workflows — the canonical "coder writes code, executor runs it, critic reviews the output" pattern — AutoGen's Docker-based code executor is the standard way to sandbox execution so a hallucinated rm -rf or an infinite loop doesn't take down the host. On a homelab or small team server, the typical pattern is a shared vLLM or llama.cpp instance serving one or two strong local models (something in the 30B-70B class quantized, or a strong MoE model if VRAM allows), with multiple AutoGen agent processes hitting it concurrently — this works because the model server, not AutoGen itself, handles request batching and queuing.

AutoGen Studio, the low-code UI shipped alongside the core library, is aimed at people who want to compose agent teams (assign roles, tools, and termination conditions) without writing the underlying Python, then export or run the resulting configuration. It's useful for prototyping a team topology quickly, but production workflows still tend to be written directly against the Python API because Studio's flexibility lags what you can express in code, particularly around custom tool functions and conditional routing.

Because every agent turn is a full LLM call, and multi-agent conversations can run for many turns before termination, local deployments need to budget for this explicitly: a five-agent conversation that runs fifteen rounds is potentially 75+ inference calls before you get an answer. This is a much heavier load profile than a single-agent RAG or tool-use loop, and it's the main reason local AutoGen setups gravitate toward faster, smaller models or aggressive turn/termination limits rather than the largest model available.

How it compares

Against CrewAI, AutoGen is lower-level and more general — CrewAI's role/task/crew abstraction is opinionated and gets a simple sequential or hierarchical agent team running faster, but AutoGen's group-chat and event-driven core give you more control over non-linear conversation patterns (agents interrupting, sub-conversations, dynamic speaker selection) at the cost of more setup code.

Against LangGraph, the comparison is architecture-first vs. conversation-first: LangGraph models workflows as explicit graphs/state machines, which makes complex conditional branching and cycles easier to reason about and debug than an open-ended agent conversation, but requires more upfront design of the graph. AutoGen's conversational model is more natural for the "team of specialists talking it out" pattern but is correspondingly harder to make deterministic or to unit test.

Against OpenAI's Swarm/Agents SDK style frameworks, AutoGen is heavier and more mature, with a much larger ecosystem of examples, integrations, and community-built agent patterns, but that maturity comes with the v0.2/v0.4 split noted above — newer, more minimal frameworks don't carry that legacy-API baggage.

Best use cases and honest limitations

AutoGen is a strong fit for multi-specialist workflows — code generation with review-and-execute loops, research tasks split across a retriever/summarizer/critic team, or any scenario where distinct roles genuinely benefit from separate system prompts and separate conversational context. Its backend-agnosticism makes it one of the more natural agent frameworks to pair with local inference, since nothing about the orchestration logic assumes a hosted API.

It is a poor fit for latency- or cost-sensitive single-turn tasks, where the multi-agent overhead just adds inference calls without adding value. Teams should also go in with eyes open about debuggability: when a five-agent conversation goes off the rails, tracing why is genuinely harder than debugging a single prompt chain, and the framework doesn't yet offer strong built-in observability for this beyond conversation logs. New adopters should deliberately choose v0.4 unless they have a specific reason to use v0.2, since the ecosystem is still catching up and mixing tutorials from both versions is a common source of confusion.