Multi-agent systems explained: patterns, examples and when to use them

A multi-agent system is a set of agents, each an LLM using tools in a loop, that work together on one task. The common patterns are sequential chains, parallel workers, an orchestrator with workers, handoffs and critic loops. Anthropic reported that its multi-agent systems used about 15 times more tokens than chats, so one agent with tools is usually the right starting point.

What is a multi-agent system?

Anthropic defines a multi-agent system as multiple agents, meaning LLMs autonomously using tools in a loop, working together. Its Research feature is a well-documented published example: a lead agent plans a research process and then creates subagents that search in parallel.

A fixed chain of LLM calls written in code is not the same thing. Anthropic calls that a workflow, because the code path is predefined, and reserves "agent" for systems where the LLM directs its own process and tool use. Many products mix the two, with code deciding the outer flow and agents deciding the inner steps.

Which multi-agent patterns exist?

The table lists six patterns, the names vendors use for them and the situation each fits. Names differ between vendors, so the "also called" column helps when reading documentation.

Microsoft's Azure Architecture Center describes sequential, concurrent, group chat, handoff and magentic orchestration. Anthropic's list adds orchestrator-workers and evaluator-optimizer. The debate form comes from a 2023 paper by Du and colleagues, in which several model instances propose and debate answers over rounds, and which reported better mathematical and strategic reasoning and fewer hallucinations on the tasks studied.

OpenAI's SDK docs separate two of these cleanly. With agents as tools, a manager agent keeps control of the conversation and calls specialists through Agent.as_tool(). With handoffs, a triage agent routes the conversation and the specialist becomes the active agent for the rest of the turn. Handoffs are shown to the model as tools, such as transfer_to_refund_agent.

How does an orchestrator-worker system work?

A lead agent breaks a question into subtasks, workers run them, often at the same time, and the lead combines the results. In Anthropic's Research system the lead saves its plan to memory, spawns subagents with specific search tasks, collects their findings and decides whether more research is needed. A separate citation agent then adds source references.

Anthropic reported that a lead model with subagents beat a single agent by 90.2% on its internal research evaluation, with the biggest gains on breadth-first questions that pursue independent directions. It also found that token usage alone explained 80% of the variance in one browsing benchmark, and concluded that these systems work mainly because they spend enough tokens on the problem.

The sketch below is a hypothetical fan-out. Each worker sleeps in place of a model call and tool use, so the timing shows concurrency, not model speed.

import asyncio
import time


async def worker(topic: str) -> str:
    await asyncio.sleep(0.2)  # stands in for a model call plus tool use
    return f"notes on {topic}"


async def orchestrator(question: str) -> str:
    topics = ["pricing", "reviews", "delivery"]  # a lead agent would plan these
    notes = await asyncio.gather(*(worker(t) for t in topics))
    return f"{question} -> " + "; ".join(notes)


start = time.perf_counter()
print(asyncio.run(orchestrator("Compare two sellers")))
print(f"{time.perf_counter() - start:.1f}s for three workers in parallel")

Output from running it with Python 3.9:

Compare two sellers -> notes on pricing; notes on reviews; notes on delivery
0.2s for three workers in parallel

OpenAI's docs describe the same shape for code-driven orchestration, listing structured-output routing, chaining agents, a loop with an evaluator agent and running agents in parallel with asyncio.gather. They add that orchestrating through code makes speed, cost and performance more predictable than letting an LLM decide.

When is one agent enough?

Most of the time. OpenAI's practical guide says to maximise a single agent's capabilities first, because more agents add complexity and overhead. Microsoft's guidance lays out three levels, a direct model call, a single agent with tools and multiagent orchestration, and says to use the lowest level that reliably meets the requirement.

Vendors name the same triggers for splitting. OpenAI lists complex logic, where prompts hold many conditional branches, and tool overload, where the problem is similar or overlapping tools, not the raw count: some systems manage more than 15 distinct tools while others struggle with fewer than 10 overlapping ones. LangChain lists context management, distributed development across teams and parallelization. Microsoft adds distinct security boundaries.

A practical test follows from this. Build the single-agent version, measure where it fails, and split only the part that fails. The framework comparison shows how four libraries express handoffs and approval steps.

What are the costs and failure modes of multi-agent systems?

Cost comes first. In Anthropic's data, agents typically used about 4 times more tokens than chat interactions and multi-agent systems about 15 times more, so they need tasks valuable enough to pay for the extra performance. Anthropic also says domains where all agents must share context or have many dependencies, such as most coding tasks, are a poor fit today.

Coordination is next. Anthropic's early agents spawned 50 subagents for simple queries, searched endlessly for sources that did not exist and distracted each other with updates. Cognition argues that agents should share full traces and that actions carry implicit decisions. Its example is a game clone split between two subagents that each made stylistically conflicting choices, and it recommends a single-threaded linear agent as the simplest design.

Systematic evidence agrees that failures are common. A 2025 study (Cemri and colleagues) annotated 1600+ traces from 7 popular multi-agent frameworks and grouped failures into 14 modes in 3 categories: system design issues, inter-agent misalignment and task verification. Its abstract notes that performance gains on popular benchmarks are often minimal.

Microsoft adds that multiagent systems inherit distributed-systems problems such as node failures, message loss and cascading errors, and recommends timeouts, retries and validating one agent's output before passing it on. Claude Code's own documentation says its experimental agent teams use significantly more tokens than a single session and suit parallel exploration, not sequential tasks or same-file edits. The AI coding tools comparison covers where subagents appear in those products.

How do frameworks name these patterns?

Each framework packages a subset under its own names. OpenAI's SDK offers agents as tools and handoffs. LangChain lists subagents, handoffs, skills, router and custom workflows. Google's Agent Development Kit has sequential, loop and parallel template workflows that run without asking a model to orchestrate, plus coordinator-led teams in ADK 2.0. Microsoft Agent Framework lists sequential, concurrent, handoff, group chat and Magentic-One. Claude Code and Cursor each document subagents that run in their own context windows.

For a first project, a research agent with parallel workers, compared against a single-agent baseline, shows the pattern and its cost. More ideas are in top 10 agentic AI project ideas for beginners.

Where do multi-agent systems fit in an AI and ML programme?

Certification in Artificial Intelligence and Machine Learning is an online programme from Vishlesan i-Hub, IIT Patna, delivered on Masai's online platform, that covers Python, machine learning, deep learning, generative AI and LLMs, agentic systems and deploying models to production. Multi-agent design belongs after single-agent tool use and evaluation, which suits the order of its module on Generative AI, LLM Engineering and Agentic Systems followed by Production Deployment, MLOps and Capstone.

Frequently asked questions

What is a multi-agent system in AI?

A multi-agent system is a set of AI agents, each an LLM using tools in a loop, that work together on a task. Anthropic gives this definition and builds its Research feature as a lead agent that creates parallel subagents. A fixed chain of LLM calls written in code is a workflow, not a multi-agent system.

When should you use multiple agents instead of one?

Use multiple agents when a single agent demonstrably fails: prompts with many conditional branches, similar or overlapping tools, context that does not fit one window, separate security boundaries or independent subtasks that benefit from parallel work. OpenAI and Microsoft both advise maximising one agent first and adding agents only where measurement shows a gap.

What are the risks of multi-agent systems?

The main risks are cost, coordination and inconsistency. Anthropic's data put multi-agent systems at about 15 times the tokens of chats. Agents without shared context make conflicting decisions, and a 2025 study of 7 frameworks catalogued 14 failure modes in 3 categories. Timeouts, retries and output validation between agents reduce the damage.

What is the supervisor or orchestrator-worker pattern?

In the orchestrator-worker pattern, a lead agent, often called the supervisor or manager, plans the task, delegates subtasks to worker agents and combines their results. Anthropic's Research feature works this way. OpenAI's SDK implements a version as agents as tools, where the manager keeps control of the conversation.

What are agent handoffs?

A handoff transfers a conversation from one agent to a better-suited specialist, which then becomes the active agent. OpenAI's SDK shows a handoff to the model as a tool, such as transfer_to_refund_agent. Microsoft's documentation also calls the pattern routing, triage, transfer or dispatch.

Next step

Build one single-agent assistant, write down where it fails, and split only that part into a second agent. The Certification in Artificial Intelligence and Machine Learning page lists the agentic systems and production deployment modules for readers who want a structured route to that kind of project.