I've now sat through enough architecture review meetings where someone proposes a five-agent swarm to solve a problem that a single well-prompted model could handle in one call. I've also seen the opposite mistake: teams jamming an entire multi-step workflow into one agent's context window because 'agents are agents,' then wondering why it hallucinates halfway through a 40-step task. The truth is that single-agent and multi-agent systems are different infrastructure patterns with different failure modes, different cost curves, and different operational overhead. Picking the wrong one doesn't just cost you elegance — it costs you uptime and money.
What It Is
A single-agent AI system is one model instance, wrapped in a loop, given a set of tools, and pointed at a task until it either finishes or times out. It has one context window, one memory of the conversation, and one decision-making thread. Everything the agent knows about the task has to fit into that single context — the system prompt, the tool definitions, the conversation history, and whatever data it has retrieved along the way.
A multi-agent system splits work across multiple agent instances, each with its own context, its own role, and often its own model configuration. One agent might plan, another might execute code, another might review output, and an orchestrator ties them together. The agents communicate through some shared channel — a message bus, a shared memory store, or direct function calls — rather than all living inside one context window.
Neither of these is inherently 'better AI.' They're both just ways of allocating compute and context to a problem, the same way you'd decide between a monolith and a microservices architecture for a backend system. And like that decision, the right answer depends heavily on the shape of the workload, not on which pattern is currently trending on conference talks.
How It Works
In a single-agent setup, the request lifecycle is straightforward. A request comes in, gets appended to the agent's context along with the system prompt and available tools, the model reasons over it, optionally calls a tool, observes the result, and loops until it produces a final answer. On infrastructure like
sw-infrarunbook-01, this typically looks like a stateless API service backed by a vector store for retrieval and a Redis-backed session cache for conversation history:
POST /v1/agent/invoke
Host: api.solvethenetwork.com
Content-Type: application/json
{
"session_id": "sess_8841",
"input": "Summarize last week's incident tickets and flag SLA breaches",
"tools": ["ticket_search", "sla_lookup"]
}The agent loop here is bounded by context length and by how many tool calls you're willing to let it make before you cut it off. I usually cap this at somewhere between 8 and 15 tool calls for a single-agent loop, because past that point you start seeing context degradation — the model loses track of earlier reasoning steps and starts repeating tool calls it already made.
A multi-agent system replaces that single loop with a graph. You have an orchestrator agent that decomposes the incoming task into subtasks, routes each subtask to a specialized worker agent, and then aggregates the results. Each worker has a narrower context — it only sees what's relevant to its subtask — which means you can run them in parallel.
orchestrator.plan(task_id="tk-2291")
-> spawn(agent="researcher", scope="gather_logs")
-> spawn(agent="analyst", scope="correlate_metrics")
-> spawn(agent="writer", scope="draft_report")
researcher.status: RUNNING (pid 41221, ctx_tokens=6.2k)
analyst.status: RUNNING (pid 41222, ctx_tokens=4.8k)
writer.status: WAITING (blocked on researcher, analyst)This is where infrastructure concerns actually change shape. With a single agent, your scaling problem is basically request throughput — how many concurrent conversations can you serve, and how do you manage context window costs per request. With multi-agent, you now have an orchestration layer that needs its own reliability guarantees: what happens when the analyst agent times out but the researcher already finished? Do you retry just that branch, or roll back the whole task? This is functionally the same problem as coordinating a distributed transaction, except the 'transaction' involves a nondeterministic language model instead of a database write.
Why It Matters
The reason this decision matters at the infrastructure level, and not just the prompt-engineering level, is cost and failure isolation. A single agent handling a complex, multi-domain task tends to accumulate context bloat. Every tool result, every intermediate reasoning step, every correction gets appended to the same context window. By the time you're 20 turns into a complex workflow, you're paying for tokens that are mostly irrelevant to the current step, and worse, the model's attention gets diluted across all of that noise. I've watched single-agent systems degrade noticeably in accuracy once they cross roughly 60-70% of their effective context window, well before they hit the hard token limit.
Multi-agent systems solve the context bloat problem by construction — each agent only holds what it needs — but they introduce a different tax: coordination overhead. Every handoff between agents is a potential point of information loss, because the orchestrator has to summarize or pass forward only what it thinks the next agent needs. If that summary is lossy (and it usually is), you get compounding errors, the same way telephone-game distortion compounds across hops. I've debugged more than one multi-agent pipeline where the final output was subtly wrong not because any single agent reasoned incorrectly, but because the handoff between agent two and agent three dropped a critical constraint that was buried in agent one's original context.
Cost also behaves differently. Single-agent systems have a cost profile that's roughly linear with conversation length. Multi-agent systems can multiply cost fast, because you're often running several agents concurrently, each with their own system prompt and context overhead, plus the orchestrator itself consuming tokens to plan and aggregate. On a busy day, a multi-agent workflow that spawns four workers per task can burn four to six times the tokens of an equivalent single-agent run. That's fine if the task genuinely benefits from parallelism and specialization. It's a very expensive way to answer a question a single well-scoped agent could've handled in one pass.
Real-World Examples
A support ticket triage system is a good single-agent case. The task is bounded: read a ticket, classify severity, check it against SLA rules, and either respond or escalate. There's no real benefit to splitting that across multiple agents — the context is small, the tools are few, and the latency requirement (respond fast) works against the coordination overhead multi-agent introduces. I've deployed exactly this pattern for a helpdesk integration hitting an internal API at
10.20.4.15, and a single agent with three tools handled the full workload with predictable latency under 2 seconds per ticket.
GET /internal/tickets/pending?priority=unset
Host: 10.20.4.15:8080
Authorization: Bearer svc_infrarunbook_admin_***
200 OK
{
"agent_decision": "escalate",
"sla_risk": "breach_in_2h",
"assigned_queue": "network-oncall"
}A multi-agent system earns its complexity in something like an infrastructure incident postmortem generator. You genuinely need separate concerns here: one agent pulling logs and metrics from monitoring systems, one correlating timeline events across services, one cross-referencing past incidents for pattern matches, and one drafting the actual writeup in a consistent format. Each of those subtasks has a different tool surface and a different failure mode, and running them in parallel materially reduces wall-clock time compared to one agent doing all four sequentially. I built something close to this for a client whose incident review process previously took an on-call engineer 45 minutes of manual log correlation — the multi-agent pipeline got a first draft in under four minutes, though it still needed human review before going out.
Another place multi-agent architecture genuinely pays off is code review automation across a large monorepo, where you want one agent focused purely on security patterns, another on style and lint conformance, and another on test coverage gaps, each running against the diff independently and reporting back to an aggregator. Running that as a single agent is possible, but the review quality drops because the model is context-switching between very different evaluation criteria within the same reasoning thread.
Common Misconceptions
The biggest misconception I run into is that multi-agent systems are strictly more capable than single-agent ones, as if adding agents were like adding CPU cores. It's not additive that way. Multi-agent systems are better at parallelizable, decomposable work with clear subtask boundaries. They are not automatically better at reasoning — in fact, for tasks that require holding a lot of context simultaneously and reasoning over all of it at once, splitting that context across agents can hurt accuracy, because no single agent ever sees the whole picture.
Another common mistake is assuming multi-agent systems are more reliable because they have 'redundancy.' In practice they usually have less reliability per unit of work, not more, because you've added more components that can fail independently: the orchestrator can misroute, an individual worker can time out, the aggregation step can drop information. Each of those is a new failure surface that didn't exist in a single-agent design. If you're moving to multi-agent for reliability reasons alone, you probably want better retry logic and circuit breakers in your single-agent system instead, not more agents.
There's also a persistent belief that multi-agent systems are inherently more 'autonomous' or advanced. Autonomy is a property of how much a system can act without human checkpoints, not how many model instances it uses. I've seen single-agent systems running fully autonomous remediation loops against production infrastructure, and I've seen multi-agent systems that require human approval at every single handoff. The agent count tells you nothing about the autonomy level — that's a separate design decision layered on top.
Last one: teams often assume you have to pick one pattern for an entire product. In practice, the systems that work best in production are usually hybrid — a single-agent front door that handles routine requests directly, with an escalation path that spins up a multi-agent pipeline only when the task's complexity crosses some threshold. That threshold is worth defining explicitly rather than letting it emerge by accident, because 'sometimes multi-agent, sometimes not' without clear routing logic is how you end up with unpredictable latency and cost that's impossible to forecast.
My rule of thumb when a team asks me which pattern to use: if you can write the task as a single, unambiguous set of instructions that fits comfortably in one context window with room to spare, use a single agent. If the task naturally splits into independent subtasks with different tool requirements and you can tolerate the coordination overhead, multi-agent is worth the investment. Everything in between is usually a sign you need to simplify the task definition before you pick an architecture at all.