Multi-Agent Orchestration Patterns: When You Need a Supervisor (and When You Don't)

Almost every team that adopts multi-agent architecture does it for the wrong reason: the single-agent prompt got long and confusing, so splitting it into "agents" felt like progress. It rarely is. Multi-agent orchestration adds real cost — more latency, more failure surface, more state to reason about — and that cost needs to buy something a single agent with a better tool set could not.

The default should be one agent

A single agent with a well-designed tool set handles the large majority of tasks that look like they need "multiple agents": research a topic, draft a document, answer a question using several data sources. If the subtasks share context and none of them need a fundamentally different capability or failure-handling strategy, splitting them into separate agents adds coordination overhead without adding capability.

When a supervisor pattern actually earns its complexity

• Subtasks need genuinely different tool access or permissions — a data-extraction step that touches a production database and a report-writing step that should never see raw records.
• Subtasks have different failure modes that need different retry or fallback logic — a scraping step that should retry silently versus a validation step that should halt the whole pipeline.
• Steps benefit from different models — a cheap, fast model for classification and routing, a stronger model for the step that actually needs reasoning.
• The task is long-running enough that isolating state per stage (what a supervisor/StateGraph pattern gives you) makes debugging and resumption tractable.

StateGraph vs. a single agent with more tools

A supervisor pattern — a StateGraph with a coordinating node that routes to specialized worker nodes — makes sense when the workers are doing structurally different jobs: an ETL agent, an analysis agent, a report-generation agent, each with its own tools and its own definition of "done." The supervisor's job is routing and aggregation, not creative reasoning, which is exactly why it should stay as deterministic as possible.

def supervisor(state):
    if state["stage"] == "extract":
        return "etl_agent"
    if state["stage"] == "analyze":
        return "analysis_agent"
    if state["stage"] == "report":
        return "report_agent"
    return END

graph.add_conditional_edges("supervisor", supervisor)

The failure mode: multi-agent as a workaround for a vague task

If a team can't clearly say what each agent's tool access and success criteria are, that's a sign the task itself hasn't been decomposed properly — adding agents won't fix an underspecified task, it will just distribute the ambiguity across more moving parts and make it harder to find where things went wrong.

Keep the supervisor deterministic where you can

The temptation is to let an LLM decide routing at every step, because it feels flexible. In practice, a rule-based or lightly-classified router (cheap model, or even plain conditional logic on state) is faster, cheaper, and — critically — debuggable. Save the expensive, non-deterministic reasoning for the worker nodes that actually need it, not for deciding which worker runs next.

The takeaway

Multi-agent orchestration is a tool for structurally different subtasks with different context, tools, or failure modes — not a refactoring pattern for a prompt that got too long. Start with one agent. Split only when you can name, specifically, what capability the split buys you.

Christian Chukwuka

Founder & AI Systems Engineer