Chapter 7: Orchestration and Multi-Agent Systems — When One Agent Isn’t Enough

Introduction: The Seduction of More Agents

There’s a moment in almost every agent project where a single agent starts to strain. The task is too big for one context window. The agent needs to be a database expert and a frontend specialist and a careful reviewer, and no single system prompt makes it good at all three. Or you look at a problem that obviously splits into parts — research these ten companies, analyze each — and think: why not run ten agents at once?

That instinct is where multi-agent systems come from, and it is both right and dangerous. Right, because some problems genuinely decompose, and a team of specialists coordinated well can do things one generalist can’t. Dangerous, because “just add another agent” is one of the most seductive and most frequently regretted decisions in applied AI. Every agent you add multiplies the token bill, adds a coordination surface where things silently go wrong, and introduces failure modes that don’t exist in a single agent at all.

This chapter is about that trade. We’ll build up the vocabulary of orchestration — the patterns for wiring agents together — and then spend real effort on the question most tutorials skip: does any of this actually help? The honest answer, backed by the best evidence available, is “sometimes, for specific shapes of work, at a large cost, and only if you build it carefully.” A frontier lab reports that multi-agent beat its best single agent by 90% on research tasks — and, in the same breath, that it burned fifteen times the tokens and is a bad fit for anything that looks like coding. A peer-reviewed study finds that on popular benchmarks, multi-agent gains are “often minimal.” The team behind one of the most successful coding agents tells you not to build multi-agent at all for most work.

So this is not a chapter that sells you multi-agent. It’s a chapter that teaches you the patterns, shows you where they pay off, and gives you the sharpest available reasons to reach for them sparingly. By the end you should be able to look at a problem and answer: does this want one agent or several? If several, in what shape? And how do I keep the context, cost, and coordination from eating the benefit?

1. When to Orchestrate

1.1 The Complexity Ladder

Before any pattern, internalize the ladder. Microsoft frames agent design as three rungs of increasing complexity: a direct model call, a single agent with tools, and multi-agent orchestration. Its advice — and this is a vendor that sells the machinery for all three — is to climb only as high as you must: a single agent with tools is “often the right default for enterprise use cases.” Multi-agent is the top rung, not the starting point.

This matters because the field’s marketing pushes the opposite. Diagrams of a dozen collaborating agents look impressive; a single well-built agent looks boring. But complexity you don’t need is complexity that bites you later. The discipline is to reach for the lowest rung that reliably does the job.

1.2 The Triggers

So when do you actually climb? AWS names four honest triggers for going multi-agent:

• Context saturation — the task no longer fits in one context window, and no amount of pruning helps.
• Specialization — genuinely distinct expertise or toolsets that don’t coexist well in one prompt (a single agent starts to degrade somewhere around 10–15 tools).
• Parallelism — independent sub-tasks that can run at the same time to cut wall-clock latency.
• Fault isolation — you want a failure in one part contained rather than corrupting the whole run.

And the inverse, equally worth memorizing: stay single-agent when the task fits one window, when the steps are tightly interdependent, or when you’re still prototyping. AWS adds a warning that recurs throughout this chapter — the moment you introduce a coordinator, it becomes a single point of failure and a reasoning bottleneck.

1.3 Where It Helps, Where It Hurts

The single most useful distinction in this whole area is about the shape of the work. Anthropic, describing the limits of its own multi-agent product, puts it plainly: multi-agent systems “excel at valuable tasks that involve heavy parallelization,” especially “breadth-first queries that involve pursuing multiple independent directions simultaneously.” But domains that “require all agents to share the same context or involve many dependencies between agents are not a good fit.”

Their canonical example of a bad fit is coding: “most coding tasks involve fewer truly parallelizable tasks than research, and LLM agents are not yet great at coordinating and delegating to other agents in real time.” When a company that ships a multi-agent research system tells you it wouldn’t use the same architecture for coding, that’s a signal worth trusting.

What to internalize: the first question is never “which framework?” — it’s “does this task parallelize?” Breadth-first work with independent branches (research, enrichment, gathering) is where multi-agent earns its keep. Depth-first work where every step depends on the last, or where all agents need the same evolving context (coding, debugging, tightly-coupled reasoning), is where a single agent usually wins.

2. The Orchestration Pattern Catalog

Vendors and papers name overlapping sets of patterns, but they rhyme. The cleanest way to hold them is by topology — how control and information flow between agents — and, as you’ll see in §5, the topology you pick is really a coordination-cost choice in disguise.

2.1 Orchestrator-Worker (a.k.a. Supervisor, Hierarchical)

This is the recurring default, and if you build only one multi-agent pattern in your career it will probably be this one. A central supervisor decomposes the request, delegates sub-tasks to specialized workers (serially or in parallel), and consolidates their outputs into a final answer.

It shows up in nearly every production reference architecture. Anthropic’s research system is a lead agent coordinating parallel subagents. AWS Bedrock’s multi-agent collaboration is a supervisor that “breaks down requests, delegates tasks, and consolidates outputs.” LangGraph ships a langgraph-supervisor library whose supervisor “controls all communication flow and task delegation.” The reason it’s the default is simple: the supervisor anchors goal alignment. There’s always one agent whose job is to remember what the user actually asked for.

2.2 Sequential (Pipeline)

An assembly line: agents run one after another, each consuming the previous one’s output. Google’s ADK ships this as a first-class SequentialAgent that “functions like an assembly line.” Pipelines are easy to reason about, but brittle in a specific way: a fixed plan → code → test → review sequence can’t re-plan when reality diverges, and reliability compounds downward — a ten-stage chain where each stage is 95% reliable finishes correctly only about 60% of the time (0.95¹⁰). Every link you add is another chance to break the chain.

2.3 Concurrent (Parallel / Fan-Out, Fan-In)

Multiple agents work simultaneously on independent sub-tasks; a coordinator fans out and later synthesizes. ADK’s ParallelAgent “runs all its sub-agents concurrently.” This is the pattern that captures multi-agent’s real advantage — but only when the sub-tasks are genuinely independent. Fan out work that secretly depends on itself and you get contradictions, not speed.

2.4 Group Chat (Debate / Council)

Several agents converse in a shared thread, often with a moderator controlling turn-taking, to debate or reach consensus. It’s a research favorite. It’s also where a concrete piece of production advice applies: Microsoft recommends limiting group chat to three or fewer agents, because flow control and infinite-loop prevention get harder as the group grows. And as §3 shows, debate is one of the patterns with the weakest independent evidence that it beats a single agent at equal cost.

2.5 Handoff and Swarm

A triage agent routes the conversation to a specialist that becomes the active agent for the rest of the turn. Control transfers — unlike the orchestrator-worker case, where control returns to the supervisor. In the fully decentralized version, called a swarm, there’s no central router at all: routing intelligence lives inside each agent (encoded in its prompt plus a “handoff tool”), and only one agent is active at a time. Three real frameworks implement this — OpenAI’s (now-deprecated) Swarm, where a function returns another agent to transfer control; LangGraph Swarm, which persists the last active agent so multi-turn conversations resume with whoever last held control; and LlamaIndex’s AgentWorkflow (below). The common mechanism: each agent knows which peers it can hand off to, and transfers by naming one.

Swarm vs. group chat — don’t confuse them. Both are decentralized and neither has a supervisor consolidating results, so they look alike on a diagram. But they do different jobs. Group chat is deliberation: several agents share one conversation and all stay active, critiquing and refining toward a better answer — usually with a moderator picking who speaks next. A swarm is routing: only one agent is active at a time, control passes like a baton via handoffs, and there’s no moderator — the routing logic lives in each agent. The shorthand: group chat is a meeting (many voices in one room, often chaired); a swarm is a relay race (one runner at a time, passing the baton, no coach directing traffic).

2.6 Blackboard (Shared Workspace)

Agents collaborate indirectly through a shared workspace they all read from and write to, rather than messaging each other directly. Confluent implements the blackboard as a shared Kafka topic that workers produce to and consume from. The architectural payoff is real: broadcasting to a shared board eliminates the point-to-point connection explosion between agents, and a replayable event log gives you failure recovery for free. A 2026 research result even found a decentralized blackboard beat an orchestrator baseline by 13–57% on a data-discovery task — one data point, but a useful counter to “the supervisor always wins.”

2.7 Magentic (Adaptive Planning)

When you don’t know the task structure up front, you want an orchestrator that plans as it goes. Microsoft’s Magentic pattern (from the Magentic-One system) is an orchestrator-worker that plans, assigns work, checks progress each round, and re-plans when it stalls. It’s the most instructive concrete design in this space, and we’ll take it apart in the framework survey — it’s the clearest example of stall detection and recovery done well.

3. Does Multi-Agent Actually Help?

This is the section most treatments skip, and it’s the most important one for a practitioner deciding where to spend effort. The evidence is unusually candid — including from the vendors selling multi-agent products.

3.1 The Case For (Vendor, Self-Reported)

Anthropic’s multi-agent research system is the strongest public case. It reports a multi-agent setup — Claude Opus as lead, Claude Sonnet subagents — outperforming a single Opus agent by 90.2% on an internal research eval. That’s a large number, and it’s real, but read the fine print: it’s a vendor-internal benchmark with undisclosed methodology, it’s for breadth-first research specifically, and it came at a cost the same post is honest about — multi-agent systems use about 15× the tokens of a chat. They also found that, on one eval, token usage alone explained most of the performance variance. Which is a slightly awkward finding: a big part of why multi-agent did better is simply that it spent far more compute.

3.2 The Case Against (Independent)

Here’s the part that should change how you default. When researchers held the token budget equal between single-agent and multi-agent systems, single-agent consistently matched or beat multi-agent on multi-hop reasoning — while spending fewer tokens. They back it with an information-theoretic argument: passing summaries between agents cannot add information about the answer, so a single agent that keeps the full context has an inherent advantage. Multi-agent only pulls ahead when the single agent’s context gets degraded (long, noisy inputs) or when the multi-agent system is quietly handed extra compute. Their conclusion is blunt — a lot of reported “multi-agent wins” are better explained by unaccounted-for extra computation than by architecture.

A separate ICLR 2025 evaluation of five multi-agent-debate frameworks across nine benchmarks reached a compatible verdict: debate doesn’t reliably beat single-agent strategies even when you give it more compute. Existing debate designs are inefficient at turning extra inference into better answers.

What to internalize: the burden of proof is on multi-agent. At equal cost, a single well-managed agent is a strong default. Reach for multiple agents when the work genuinely parallelizes or the context genuinely overflows — not by reflex, and not because a diagram looked impressive.

3.3 Why It Fails: The MAST Taxonomy

The most useful academic contribution here isn’t a benchmark — it’s a catalog of how these systems break. The MAST paper (“Why Do Multi-Agent LLM Systems Fail?”, UC Berkeley, NeurIPS 2025) analyzed 1,600+ execution traces across seven multi-agent frameworks and organized the failures into 14 modes in three categories:

1. Specification and system-design issues — bad prompts, unclear roles, a broken workflow.
2. Inter-agent misalignment — agents talking past each other, withholding information, contradicting each other’s assumptions.
3. Task verification and termination — no one checks the work, or the system stops too early or never stops.

The sobering result: the researchers tried the obvious fixes — better role prompts, an added verification step — and got only +9.4% and +15.6% improvements that “did not resolve all failure modes.” Their read is that inter-agent misalignment isn’t a prompt bug you patch; it needs structural redesign. In other words, the coordination problems are intrinsic to gluing agents together, not incidental.

3.4 The Contrarian Voice

Cognition, the team behind the Devin coding agent, wrote the sharpest version of the skeptical case in a post titled, simply, “Don’t Build Multi-Agents.” Their core argument is about context: when parallel subagents can’t see each other’s work, their outputs drift into mutual inconsistency. Their memorable example is a Flappy Bird clone where one subagent built a Super Mario–style background while another built a bird that didn’t match — each locally reasonable, globally incoherent. Their two principles: share full agent traces, not just messages, and actions carry implicit decisions that silently conflict when agents work in parallel. Their recommended default is a single-threaded linear agent; a later, more nuanced post concedes multi-agent works when writes stay single-threaded and the extra agents contribute intelligence (analysis, review) rather than parallel actions — for example, a dedicated reviewer with a clean context.

4. Context Engineering Across Agents

Every pattern above is, underneath, a decision about how context flows across process boundaries. This is where multi-agent systems are actually won or lost, so it deserves its own treatment. (Chapter 3 covers memory; here we care specifically about what one agent passes to another.)

4.1 The Central Knob: Full History vs. Isolation

There are two poles, and every serious framework makes the choice configurable:

• Full-history sharing — each agent sees everything that came before. Preserves continuity, but multiplies token cost across every agent and can confuse a simpler worker with a giant, mostly-irrelevant transcript.
• Isolated windows — each agent gets only a scoped task. Cheaper and cleaner, but agents can drift into inconsistent work (Cognition’s failure mode).

The frameworks land on opposite defaults, which tells you the choice is genuinely hard. LangGraph’s supervisor passes the full message history to a worker by default; isolation is opt-in. AWS Bedrock makes history-sharing an explicit toggle, warning that full context “might confuse simpler subagents with complex task histories” and advising you enable it only when you need continuity. Newer designs lean toward isolation: Anthropic’s lead agent, as it nears its context limit, writes its plan to external memory and hands each subagent a self-contained task with no knowledge of the others.

A nice hybrid worth stealing: isolate the working context but share the ground rules. A multi-agent coding setup can give every agent a fresh window for its task while preloading a shared conventions file (project architecture, style) into all of them — shared rules, isolated work.

4.2 The Four Strategies

LangChain’s context-engineering taxonomy maps cleanly onto orchestration. You have exactly four levers: Write (save state outside the window — scratchpads, external memory), Select (pull the right context in — retrieval, memory), Compress (summarize or trim), and Isolate (split context across sub-agents — this is the lever multi-agent structurally pulls). Microsoft’s version of the same advice: in multi-agent orchestration, context grows fast because each agent appends its own reasoning and tool output, so compact between agents and externalize shared state to a durable store.

4.3 The Token Economics

Multi-agent’s cost isn’t linear in the number of agents; it’s worse. A few numbers worth carrying:

• The naive loop grows quadratically. Because model APIs re-bill the full history on every call, a naive agent loop costs roughly N(N+1)/2 in tokens — a 20-step loop can burn over 10× a per-step estimate.
• Multi-agent multiplies every cost surface. Each subagent has its own inference, retrieval, and tool costs — the source of that ~15× figure. This makes per-agent cost attribution non-negotiable: without it, an aggregate token count hides the one runaway subagent doing over-broad retrieval.
• There’s a crossover. For work with high per-step token growth, a single agent can be cheaper than a coordinator-plus-specialists setup until surprisingly late in a task. Don’t assume splitting work saves money; often it doesn’t.

5. Coordination and Failure Propagation

5.1 Topology Is a Coordination-Cost Choice

Here’s the structural reason the supervisor pattern dominates. Communication paths in a peer-to-peer swarm scale quadratically — N(N−1)/2, so ten agents have forty-five possible channels — while a supervisor scales linearly: adding a worker adds no new peer-to-peer paths. The failure points scale the same way. Swarm pipelines that exceed roughly eight to ten sequential handoffs show measurable quality degradation. Centralizing coordination is, in large part, a way to stop the coordination surface from exploding — and a shared event bus or blackboard is the other way to tame that N² growth.

5.2 Errors Compound

In a chain of agents, a single hallucination early on poisons every downstream decision. This is the practical face of MAST’s “inter-agent misalignment”: the code-assistant literature documents concrete cases of two agents editing the same file, or each assuming the other would handle a missed step. The more hops between a mistake and the final output, the more expensive it is to catch.

5.3 The Primitives That Contain Failure

Three mechanisms recur across production systems, and you should plan for all three:

• Stall guardrails. Cap the loops. Magentic-One exposes explicit limits — a max round count, a max stall count, a max reset count — so a stuck orchestrator resets and re-plans instead of spinning forever. At minimum, give any orchestration loop a hard ceiling.
• Single-threaded writes. Cognition’s hardest-won lesson: let multiple agents read, analyze, and advise in parallel, but keep the writes on one thread. Parallel writers make conflicting implicit decisions.
• Trace lineage. A multi-agent run is an execution tree, not a linear log. Give each agent a run ID and a parent ID, record what was delegated and what came back, or you will not be able to replay one agent, attribute cost, or find which agent produced a bad answer. And treat a subagent’s output like any other untrusted input — validate it before it flows into durable state, rather than trusting it because it “came from one of our agents.”

6. The Framework Landscape

Every framework offers roughly the same patterns through different abstractions. The selection driver is usually team familiarity and how it fits your ops stack, not exclusive pattern support. (Deep API and production comparison is Chapter 13’s job; here we care only about the orchestration primitive each one hands you.)

• LangGraph models orchestration as an explicit state graph — nodes are agents, edges are transitions, conditional edges give you branching and cycles. It natively supports supervisor, hierarchical (nested graphs), and swarm styles. Best when you want the control flow to be legible and inspectable. (Note: LangChain now nudges people toward a manual tool-calling supervisor over the older langgraph-supervisor library — verify the current recommendation.)

• Microsoft AutoGen → Agent Framework. AutoGen popularized conversation-based, actor-model agents. Its successor, the Microsoft Agent Framework (built by the AutoGen and Semantic Kernel teams, 1.0 in 2026), keeps a low-level actor layer for message-passing and a higher-level Team abstraction like RoundRobinGroupChat on top. Its standout is Magentic: a dual-ledger orchestrator worth teaching as a design.

  Magentic-One’s orchestrator runs two nested loops. The outer loop maintains a Task Ledger — the facts it knows, what it still needs to find, and the plan — and re-plans when progress stalls. The inner loop maintains a Progress Ledger, rebuilt each round by answering five questions: is the task done? are we looping? are we making progress? who acts next? what do we tell them? A stall counter increments when it detects a loop; past a small threshold, the orchestrator breaks out, reflects, and revises the plan. It’s the clearest concrete answer to “how does an orchestrator know it’s stuck and recover?”

• OpenAI Agents SDK. Frames the whole question as LLM-driven vs. code-driven orchestration — let the model decide the flow, or fix it in code — and says to mix them. Two named primitives: agents-as-tools (a manager calls a specialist via Agent.as_tool() and control returns) and handoffs (control transfers to a specialist). The experimental Swarm framework that pioneered its handoff style is deprecated in favor of this SDK.

• CrewAI. Organizes work around a role/goal/backstory metaphor and splits into two constructs: a Crew (a collaborative team, good for exploratory work) and a Flow (an event-driven, deterministic workflow that orchestrates crews and tasks via @start/@listen/@router decorators). Its default process is sequential; the hierarchical process requires a manager (manager_llm or a custom manager_agent) that delegates to workers by capability. One production-relevant detail: allow_delegation now defaults to False — collaboration is opt-in — and turning it on grants an agent explicit “Delegate Work” and “Ask Question” tools.

• Google ADK. Cleanly separates reasoning agents from orchestration agents (SequentialAgent, ParallelAgent, LoopAgent) that direct flow without doing the work, and enforces a strict one-parent hierarchy. It offers three ways for agents to coordinate: shared session state (blackboard-style), LLM-driven delegation (routing on child descriptions), and agent-as-tool via AgentTool.

• AWS Bedrock Multi-Agent Collaboration. A managed supervisor-worker service with a standard mode and a routing mode (route simple requests straight to a subagent, skipping full orchestration to cut latency). It prescribes minimal, structured handoff payloads and maps shared state onto concrete services (task state to DynamoDB, session context to a cache, domain knowledge to a knowledge base).

• LlamaIndex. Its multi-agent AgentWorkflow is built on an event-driven Workflow primitive — functions marked with @step that emit and consume typed Event objects, with shared state in a Context. Multi-agent orchestration is decentralized handoffs: designate a root agent and give each agent a can_handoff_to list. A third first-party example of the swarm pattern.

Here’s the landscape at a glance:

7. Evaluating Orchestration

A fair question: how do you know if your multi-agent system is actually coordinating well, versus just producing an answer? The tooling here is thinner than for single agents, but not empty.

A survey found only about 32 multi-agent evaluation papers, most measuring miscoordination and leaning heavily on party games (Werewolf, Avalon) rather than production scenarios. But two benchmarks do explicitly grade coordination quality, not just task output: MultiAgentBench uses a milestone-based scoring scheme and systematically tests topologies against each other (notably finding that a decentralized graph protocol did best and group discussion did worst — structure matters), and Meta’s PARTNR measures human-agent collaboration and found state-of-the-art models coordinate notably worse than humans, needing about 1.5× the steps.

The practical takeaway: there is still no standard “is my production orchestration good?” benchmark. You evaluate end-task success, and you instrument coordination failures yourself using the trace-lineage discipline from §5. (Evaluation methods in general are Chapter 6.)

Summary: What to Internalize

• Multi-agent is the top rung of a ladder, not a starting point. Start with a single agent with tools. Climb only when context saturates, expertise genuinely splits, work truly parallelizes, or you need fault isolation.
• Shape of the work decides everything. Breadth-first, parallelizable work (research, enrichment) is where multi-agent helps. Depth-first, shared-context work (coding, tightly-coupled reasoning) is where a single agent usually wins.
• The burden of proof is on multi-agent. At equal token budget, a single agent often matches or beats it. Much of multi-agent’s apparent advantage is just extra compute — and it costs ~15× the tokens.
• Orchestrator-worker is the default pattern. A supervisor that decomposes, delegates, and consolidates. It scales coordination linearly where swarms scale it quadratically.
• Context flow is the real design problem. Full-history sharing vs. isolated windows is the central knob. Isolate the working context, share the ground rules, and attribute cost per agent.
• Coordination failures are structural, not incidental. MAST’s 14 failure modes don’t patch away with better prompts. Keep writes single-threaded, cap your loops, and treat subagent output as untrusted until validated.
• Instrument the tree. A multi-agent run is an execution tree. Without run/parent IDs and trace lineage, you can’t debug it, replay it, or find which agent produced the bad answer.