Skip to content

Multi-Agent Systems

What you’ll learn. Why one might use several agents; the main topologies (orchestrator-subagents, pipelines, peer groups, debate); how agents communicate; the evidence on when multi-agent helps and when it hurts; and design rules for handoffs.

A “multi-agent system” is just several LLM loops with distinct contexts (and often distinct prompts, tools or models) that exchange messages. The honest motivations are few:

  1. Context isolation. Each agent has its own window. A research sub-agent can read 50 pages and return 300 words; the parent’s context stays small (Memory).
  2. Parallelism. Independent subtasks run concurrently, cutting wall-clock time.
  3. Specialization. Different system prompts, toolsets and permissions per role (a reviewer with read-only tools; a deployer with deploy access). Least privilege is easier when roles are separate.
  4. Diversity and verification. Independent attempts, or a critic with a different prompt or model, can catch errors (Test-time compute).
  5. Scaling the amount of compute spent on a hard problem beyond one context window.

Anthropic’s report on its multi-agent research system described a lead agent that plans and spawns parallel subagents, and reported large quality gains over a single agent on research tasks together with a multiple-fold increase in token usage, which is the trade-off in one sentence: more tokens for better coverage.

Orchestrator-subagents Pipeline Peer group / swarm Debate
[O] [A] -> [B] -> [C] [A] <-> [B] [Pro] <-> [Con]
/ | \ ^ \ / ^ \ /
[w1][w2][w3] [C] <-> [D] [Judge]
  • Orchestrator-subagents (hierarchical). The most reliable and most common. One agent decomposes, delegates, and synthesizes (pattern lecture). Workers do not talk to each other, which keeps control flow legible.
  • Pipeline / assembly line. Fixed sequence of specialists (planner, coder, tester, reviewer), the structure of systems like MetaGPT and ChatDev, where roles mirror a software team. Effectively a workflow with agent nodes.
  • Peer group. Agents converse freely (AutoGen-style group chat, with a manager choosing the next speaker). Flexible but hard to control and debug; conversations drift and repeat.
  • Handoff / routing. One agent transfers the conversation to a specialist (triage to billing agent). Think of it as routing with persistent conversation state.
  • Debate / society of minds. Several instances argue over an answer for multiple rounds, then a judge or majority decides (Du et al., 2023). Gains were shown on reasoning and factuality tasks, but later work found that, at equal compute, simple self-consistency voting often matches debate.

What passes between agents determines quality more than the topology.

  • Natural-language messages are flexible but lossy and verbose.
  • Structured artifacts (JSON results, files, diffs, a shared task board) are precise and checkable. Passing references to artifacts (file paths) rather than full contents avoids “telephone game” distortion and token cost.
  • Shared state / blackboard: a common store (task list, scratch files) that agents read and write, rather than direct messaging. This decouples agents and eases debugging.
  • Protocols: emerging standards aim to let agents from different vendors interoperate, e.g. Agent2Agent (A2A) for agent-to-agent messaging, alongside MCP for agent-to-tool. Both are young; treat details as moving.

The recurring failure is an under-specified handoff. A subagent starts with nothing except what you give it. A good delegation message states:

  1. Objective: what question to answer or artifact to produce.
  2. Context and constraints: what is already known, what to avoid.
  3. Output format and length: so the parent can use it directly.
  4. Boundaries: tools allowed, effort budget (“use at most 10 tool calls”).
def spawn_subagent(llm_loop, objective, context, output_spec, tools, max_steps=10):
brief = (f"Objective: {objective}\nKnown context: {context}\n"
f"Return: {output_spec}\nBudget: at most {max_steps} tool calls.")
return llm_loop(task=brief, tools=tools, max_steps=max_steps) # fresh context

Scale effort to complexity in the orchestrator’s prompt: simple fact-finding warrants one subagent and a few calls; broad research warrants many. Without such rules, agents over-spawn.

import asyncio
async def research(orchestrator_llm, subagent, question):
plan = await orchestrator_llm(
f"Break into 3-5 independent search questions, JSON list:\n{question}")
subs = json.loads(plan)
findings = await asyncio.gather(*[subagent(q) for q in subs])
return await orchestrator_llm(
f"Question: {question}\nFindings:\n" +
"\n---\n".join(findings) + "\nWrite a cited synthesis; note conflicts.")

Multi-agent systems are oversold. Empirical studies of failures (e.g. the MAST taxonomy by Cemri et al., 2025) found that systems fail from specification and role ambiguity, inter-agent misalignment, and weak verification, not from lack of agents; on many benchmarks well-prompted single agents match multi-agent frameworks at lower cost. Practical consequences:

  • Cost multiplies: each agent re-reads context; token use may be an order of magnitude larger than a single chat.
  • Errors compound through handoffs; a wrong summary is believed by the next agent.
  • Parallelism fits breadth-first tasks (research, search) better than tightly coupled work (one codebase edit), where agents write conflicting changes. Cognition’s widely cited essay argued against multi-agent for coding for this reason: shared context is critical when decisions depend on each other.
  • Debugging is harder: trace the whole tree (Observability).
  • Security: a compromised subagent’s output is untrusted input to the parent (Safety).

A decision procedure:

Can one agent with good tools and context management do it? -> do that.
Is work embarrassingly parallel / breadth-first? -> orchestrator + parallel workers.
Does a noisy subtask pollute context? -> isolate in a subagent.
Do you need independent verification? -> separate reviewer with execution access.
Tightly coupled decisions across the task? -> keep a single thread of context.

Multi-agent ideas also include agent societies for simulation (Generative Agents), market and negotiation settings where agents have conflicting goals, and training setups where agents co-evolve (self-play in games; adversarial red-team and defender agents). The last connects to self-improvement.

  • Multi-agent is justified mainly by context isolation, parallelism, specialization with least privilege, and independent verification.
  • Orchestrator-subagents is the dominant robust topology; free-form peer chat is hard to control.
  • Delegation quality (objective, context, format, budget) drives results; pass artifacts, not paraphrases.
  • Costs multiply and errors compound; many multi-agent systems underperform a good single agent.
  • Use parallel agents for breadth, a single thread for tightly coupled work.
  1. Conceptual. Why might a debate among three copies of the same model add little over majority voting at the same compute?
  2. Compute. A single agent uses 50k tokens. A system with a lead and 4 parallel subagents uses 30k tokens each plus 20k for the lead. What is the token multiple, and under what speedup assumption is it worth it?
  3. Hands-on. Implement the parallel research function with a stub LLM and verify that wall-clock time is the max, not the sum, of worker times (use asyncio.sleep).
  4. Design. Decide whether to use multi-agent for (a) migrating 200 independent files to a new API; (b) designing a database schema. Justify.
Solutions (2, 4)
  1. Total =4×30k+20k=140k=4\times30k+20k=140k, a 2.8x multiple. Worth it if quality or latency gains (e.g. 3-4x faster wall-clock via parallelism, or better coverage) justify 2.8x spend.
  2. (a) Yes: independent, parallel, easily verified per file. (b) Prefer a single agent: decisions are tightly coupled and need shared context.
  • Wu et al., “AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation” (2023).
  • Hong et al., “MetaGPT: Meta Programming for a Multi-Agent Collaborative Framework” (2023).
  • Du et al., “Improving Factuality and Reasoning in Language Models through Multiagent Debate” (2023).
  • Cemri et al., “Why Do Multi-Agent LLM Systems Fail?” (2025).
  • Anthropic, “How we built our multi-agent research system” (2025); Cognition, “Don’t Build Multi-Agents” (2025).