Count Your Handoffs Before You Add Agents

A GenAI feature demos beautifully. A planner agent reads the request, a researcher agent gathers context, a coder agent drafts the change, a reviewer agent checks it. On the screen it looks like a tidy little team, each member a specialist, each handoff a clean pass of the baton. Then it goes near production and the numbers stop holding. The same pipeline that nailed the demo now succeeds maybe two times in three on real inputs, and nobody can say exactly which agent dropped the ball, because the failure moves around.

This is the gap I keep watching teams fall into: the multi-agent design that earned a standing ovation in the demo and then degraded the moment it met production traffic. When reliability slips, the instinct is to add another agent, a checker, a refiner, a second reviewer. That instinct is backwards. Each serial handoff you add is usually another reliability term below one, unless verification, redundancy, or recovery changes the effective rate, and the math of a chain is unforgiving. Before you decide single agent vs multi agent, count the handoffs, because the handoff, not the agent, is the unit of reliability risk.

Quick answer: End-to-end reliability is roughly the product of per-step reliability, so every agent handoff is a multiplier below one. A chain that looks impressive, one agent per role, decays end-to-end faster than teams expect. Most multi-agent designs get added for org-chart legibility, mirroring a human team, not for reliability, and the compounding math punishes that. The operator move is to count handoffs before adding agents and collapse the design back to one agent whenever a step does not earn its multiplier. The strongest and most common reasons to keep an agent are genuine parallelism, real context isolation, and sharply separated tool and domain sets, and any other pattern, such as an independent reviewer or a voting branch, has to earn its multiplier the same way.

Reliability compounds, it does not average

Here is the arithmetic most multi-agent diagrams quietly assume away. If a single step in a pipeline succeeds 95 percent of the time, that feels reliable. Most people round it to "basically always works." But a pipeline is a chain, and a chain succeeds only if every link holds. So the end-to-end success rate is closer to the product of the per-step rates, not the average of them.

Walk it out. One step at 95 percent is 0.95. Two steps is 0.95 times 0.95, about 0.90. Five steps lands near 0.77. Ten steps, the kind of decomposition a four-or-five-agent pipeline with retries and routing easily reaches, is 0.95 raised to the tenth power, which is about 0.60. A ten-step chain where every step is "basically always works" succeeds end-to-end only about three times in five.

I need to be precise about what that 95 percent is and is not. It is a sensitivity assumption, a round number chosen to illustrate the shape of the math, not a benchmark and not a measured rate for any real system. The real per-step number for your pipeline has to be measured from task-level evals on your actual inputs, using a fixed task set, logged step boundaries, predeclared success criteria, retry policy, failure attribution, and confidence intervals, and it will rarely be a clean 95. The point of the calculation is not the specific figure. The point is the curve: small drops in per-step reliability compound into large drops end-to-end, and they compound faster the more steps you add.

I also need to be honest that the simple product is a baseline model, not an iron law. It assumes the steps fail independently and that each step's success is conditional only on the prior step delivering. Real pipelines bend that assumption in both directions. A retry on a flaky step pushes the effective per-step rate back up. A verification step that catches and repairs a bad handoff can raise end-to-end reliability above the naive product. And correlated failures, where one bad upstream interpretation poisons every step after it, can make the real number worse than the product suggests. So treat 0.95 to the tenth as the intuition pump, the thing that tells you which direction the risk moves, and then measure your own chain to find where it actually sits.

There is a second cost the product-of-probabilities model does not even capture, and it is the one that bites in production: context loss at the handoff. A probability multiplier assumes each step either succeeds or fails cleanly. But an agent handoff is rarely a clean pass. It is usually a compression. Agent A finishes its work and hands a summary, a partial result, a chunk of context to Agent B, and something is lost in the translation, unless the system passes complete artifacts, references, or durable state across the boundary rather than a re-summarized digest. Anthropic describes the failure mode precisely: when you decompose a problem so that one agent writes the feature, another writes the tests, and a third reviews, you create a telephone game, where each handoff loses fidelity and the final output drifts from what the first agent actually understood. That telephone-game loss is on top of the probability multiplier, not instead of it.

What "per-step reliability" actually means

Before the math is usable, "step success" has to mean something specific, because a number you cannot define is a number you cannot measure or improve. A step in an agent pipeline succeeds when four things hold at once.

First, the step interprets its task correctly: it understood what it was asked to do, not a plausible-but-wrong neighbor of it. Second, it receives sufficient context transfer from the prior step: the handoff carried enough of the upstream state that this step is not guessing. Third, it produces a valid output: a tool call that actually ran, a structured result that parses, a change that applies. Fourth, the output is accepted downstream: the next step, or the final consumer, can use it without rejecting or silently degrading on it.

Retries sit inside this definition, not outside it, and you have to decide where consistently. If a step fails and a retry succeeds, do you count the step as a success at a latency and cost penalty, or as a failure that got papered over? Both are defensible, but you have to pick one and apply it the same way everywhere, or your per-step numbers stop being comparable and the whole multiplicative model turns to mush. The teams that get reliable measurements are the ones that wrote down what "success" means for each step before they started counting, not after.

This is also where a multi agent systems reliability problem hides in plain sight. When a pipeline degrades and nobody can localize the failure, it is almost always because "success" was never defined per step, so there is no instrument that tells you which handoff lost the context. You cannot fix a multiplier you cannot see. (For more on the failure surfaces that production AI systems systematically underweight, see Three Production Failure Modes Engineers Underweight.)

Teams add agents for the org chart, not for reliability

So if the math punishes long chains, why do multi-agent designs keep growing? Because adding an agent is the most legible way to express a plan. When you sketch a problem on a whiteboard, you naturally decompose it the way you would decompose it for a human team: someone plans, someone researches, someone builds, someone reviews. The planner / researcher / coder / reviewer quartet is not a reliability architecture. It is an org chart drawn in software.

The org chart is appealing for reasons that have nothing to do with whether the system works. It is easy to explain to a stakeholder: "we have a dedicated review agent" sounds like rigor. It is easy to divide labor across a team: each engineer owns one agent. And it matches the consensus you read everywhere, that more agents means more capability, that a sophisticated problem deserves a sophisticated, multi-specialist pipeline. Agent count gets read as a sophistication signal.

That consensus is where I take a different position. More agents does not read as more capability once you put a number on it. More agents reads as more handoffs, and more handoffs reads as more multipliers below one, plus more context-loss surfaces, plus more coordination overhead. The sophistication signal and the reliability signal point in opposite directions. A design that looks more capable on the whiteboard is, by default, less reliable in production, unless each added agent is buying something the math credits.

The cost is not only reliability. Every handoff adds coordination latency, and the dollar cost of a workflow rises as that overhead compounds across handoffs. Anthropic reports that multi-agent systems can consume 3 to 10 times more tokens than a single agent doing the same work. So the org-chart design is not free legibility. It is legibility you pay for in latency, in dollars, and in the reliability the compounding math quietly drains. That is the heart of the single agent vs multi agent decision, and almost nobody frames it as a liability with a number attached.

Put your own numbers in the table

The argument so far rests on one illustrative figure. The honest move is to show the whole curve so you can plug in your own measured per-step rate and read your own end-to-end number. The table below holds three per-step reliability levels against three chain lengths. Every cell is the per-step rate raised to the number of steps, the baseline product-of-probabilities model.

Read the table by row and the lesson is per-step quality. Read it by column and the lesson is chain length. A pipeline at 0.90 per step, which sounds nearly reliable, collapses to about a coin-flip by ten steps and to roughly one in three actually completing the whole chain. To hold a ten-step chain above 0.90 end-to-end you need per-step reliability at 0.99, which is a different engineering regime entirely, the regime of relentless evals and verification rather than of clever decomposition. These numbers are the model, not a measurement. The retries, verification steps, and correlated failures from the earlier caveat all shift the real cell value. But the table tells you where to look: if your chain is long and your measured per-step rate is anywhere below 0.99, the agent count is the first thing to interrogate, not the model choice and not the prompt.

This is also why "just wait for a better model" is a weak answer to a reliability problem. A better model lifts the per-step rate, which helps, but it cannot repeal the exponent. Ten steps at an improved 0.97 per step is still only about 0.74 end-to-end. The exponent is set by your architecture, by how many handoffs you chose. Architecture is the lever you actually control.

When an agent earns its multiplier

None of this means multi-agent designs are wrong. It means each agent has to earn the multiplier it costs. There are real cases where decomposition buys something a single agent cannot get, and Anthropic names three situations where the additional structure pays for itself.

The first is genuine parallelism. When a task breaks into independent subtasks that do not depend on each other's output, running them concurrently across separate agents is a real win, because the subtasks are not a serial chain. Three agents each researching a different vendor in parallel, then a single synthesis step, is not a ten-link chain. It is three short independent chains feeding one join. Parallelism breaks the serial exponent: depth, not branch count, is what compounds, so a fan-out of short branches does not decay the way a long serial chain does. Be precise about the join, though. The multiplicative decay applies along each branch's depth, and if the final answer requires every branch to land (an AND-join, which the vendor-research example is), the branch successes still compose at that join. What parallelism buys is a shorter critical path and independent exploration, not immunity from the math. The reliability win is real where the branches are genuinely independent and the join is tolerant or redundant; where every branch is mandatory, you have traded a long chain for a wide one and still have to clear each link.

The second is genuine context isolation. A single agent has one context window, and on a large task that window fills with the residue of everything it has touched, stale tool output, abandoned reasoning, half-finished sub-problems. When that pollution starts degrading the agent's judgment, splitting the work so each agent carries only its own clean slice is a reliability gain, not a loss. The test is whether the single agent is actually context-poisoned, measurably, not whether you can imagine it might be.

The third is genuinely separate tool or domain sets. When one part of a task needs database access and SQL fluency and another part needs a totally different toolset and a different domain expertise, forcing both into one agent means one agent juggling two distinct skill profiles and two tool permission sets. A clean split there can reduce errors, because each agent operates in a narrower, better-instrumented world. The separation has to be configured, though, not just implied by the split: real permission isolation comes from explicit tool allowlists and scoped tool servers, not from a prompt role alone.

Notice what these three have in common. In every case the decomposition buys a structural advantage that a single agent provably cannot get, parallelism the chain cannot offer, isolation a single window cannot maintain, separation a single tool profile cannot hold. That is the bar. An agent earns its multiplier when it buys a structural advantage no single agent could achieve. (For where the reliability and evaluation gate lives in the broader stack, see Quality Harness Engineering: The Emerging Stack for Reliable AI Systems.)

Count the handoffs, then collapse the ones that do not earn

Here is the operator test, and it is deliberately mechanical. Before you decompose a problem into agents, draw the pipeline and count the handoffs. Then walk each handoff and ask one question: does this step buy parallelism, real context isolation, or genuine tool and domain separation, or is it org-chart mirroring? If it buys one of the three structural advantages, keep it. If it is there because it mirrors a human role, collapse it.

Most of the collapse candidates fall into a single pattern: sequential problem-type decomposition, where you split one continuous reasoning task into stages because a human team would have stages. The planner-then-coder split is the common one. A single capable agent that plans and then implements, holding the full context across both, usually beats two agents passing a plan across a handoff, because the handoff is where the plan's intent gets compressed and partially lost. The decomposition did not buy parallelism, the steps are sequential. It did not buy isolation, the context is shared and wanted shared. It did not buy tool separation, planning and coding draw on the same world. It bought legibility, and it cost a multiplier.

The reviewer is the case that needs care, because here the answer is genuinely conditional. The instinct is to collapse an independent reviewer agent into a verification step inside the building agent, and often that is right: a self-check or a deterministic test gate inside one agent avoids a handoff and its context loss. But not always. Collapse the reviewer into a verification step only when an independent review does not measurably improve defect detection or auditability. If a separate review agent, with its own clean context and its own adversarial framing, catches defects the building agent's self-check misses, or if regulatory and audit needs require an independent record of who checked what, then the independent reviewer is earning its multiplier and you keep it. The rule is not "always collapse the reviewer." The rule is "measure whether independent review pays, and keep it only if it does."

There is a fourth collapse trigger that is easy to miss: orchestration scaffolding built for a weaker model. A lot of multi-agent complexity exists to compensate for what an earlier, less capable model could not do alone, the routing, the retries, the helper agents that babysit a fragile core. When the underlying model improves, that scaffolding does not automatically dissolve. It sits there as dead weight, each helper agent still a handoff, still a multiplier, still a latency and token cost, now compensating for a weakness the model no longer has. Periodically re-asking "does this agent still earn its keep against the current model" is part of keeping the handoff count honest.

The reliability-engineering view sharpens why this matters. Treat LLM agents as what they are: unreliable distributed-system components whose errors propagate through the system topology. In a distributed system, every additional component and every additional message boundary is a place where failures originate and spread. The topology is the risk surface. Adding an agent adds a node and an edge to that topology, and the errors travel along the edges. The discipline that keeps a distributed system reliable is the same discipline that should govern an agent pipeline: minimize the components, instrument the boundaries, and add a node only when it provably reduces total failure rather than relocating it.

The handoff count is an operating-model decision, not a framework decision

The temptation, once the math lands, is to read it as a tooling problem: pick the framework that orchestrates agents most cleanly, add observability, tune the retries. That is the vendor consensus, and it is not where the decision actually lives. Counting handoffs and collapsing the ones that do not earn their multiplier is an operating-model decision, and it touches three things that no framework choice settles for you.

It touches workflows and handoffs. The handoff is the unit you are now designing around, which means someone has to own the question of how many handoffs a given task is allowed to survive before the design is wrong. That is a workflow standard, set deliberately, not an accident of how the pipeline grew.

It touches the review and control standard, the place where the primary, explicit eval gate lives. Reliability is not a property you inspect at the end. It is a property you measure at a defined gate, and for long-running workflows that may mean checkpoint evals along the way rather than a single gate at the finish. The decision of where the eval gate sits, what it measures, and what per-step reliability bar a step must clear before it is allowed into the chain, is a control standard, and it is the load-bearing one.

And it touches decision rights: who owns the reliability target. If "make it reliable" is everyone's job, it is no one's job, and the agent count grows because adding an agent is always the easy local move and nobody owns the global cost. Someone has to own the end-to-end number, with the authority to say "this step does not earn its multiplier, collapse it," even when a stakeholder likes how the org-chart pipeline reads.

That is the shift. The reliable design is not the one with the most specialists. It is the one where the handoff count is a deliberate constraint, the eval gate is a named control, and the reliability target has an owner. Call that organizational reliability discipline a harness if you like, but understand the harness here is operating-model discipline, not software scaffolding around a model. The math is the same whether you call your pipeline a Shift Harness or anything else. A multi-agent system is not an org chart. It is a probability chain, and you get to choose how many links it has.

AI Transparency Notice: This article and its accompanying images were created with the assistance of generative AI. The author directed the content, contributed the underlying ideas and analysis, and reviewed the final publication.

Frequently Asked Questions