Multi-Agent Orchestration: Managers, Workers, and Pipelines
Why one agent isn't enough, and the coordination patterns that keep a team of agents from deadlocking.
In Part 1 you saw how a single ReAct loop turns a model into a thinking agent. But one agent, like one employee, can only handle one focused task at a time. The moment your workload splits into classification, summarization, drafting, and scheduling, you need a team. You also need a plan to keep them from deadlocking.
Multi-agent orchestration is how you assign tasks to specialized agents, coordinate their work, and prevent infinite retries. This article covers sequential pipelines, manager-worker coordination, state-passing strategies, and the feedback loops that keep a team of agents from grinding to a halt. Everything is grounded in Mailmind, an AI assistant that lives in your inbox: it reads email, drafts replies, schedules meetings, tracks orders and refunds, and files receipts.
Here is the surprising part. A manager that simply tells a worker "try again" can deadlock the entire pipeline in under a second. The fix is not better prompts. It is a stagnation detector that notices the worker keeps giving the same wrong answer.
How does a sequential pipeline work, and when should you use it?
A sequential pipeline runs agents in a fixed order. One agent’s output becomes the next agent’s input. This simple pattern shines when each step depends directly on the previous one.
Think of a restaurant kitchen. Before a dish reaches the customer, it passes through prep, cooking, assembly, and an expeditor who checks the plate. Each station builds on the work done before it. You cannot plate a steak before the grill finishes cooking it.
Mailmind uses a three-step pipeline for drafting important replies. First a triage agent reads the thread and assigns a priority: reply-now, reply-later, or no-reply. If the verdict is reply-now, the outcome flows into a drafting agent that composes a reply using the thread context and priority. Finally a review agent checks the draft for tone, completeness, and factual errors. The review agent can reject the draft and loop back to the drafting agent with specific feedback.
State passes directly from one agent to the next. The triage agent’s output is a structured object that the drafting agent consumes. The reviewer receives both the draft and the triage metadata. This linear flow makes debugging straightforward: you can trace a bad reply back through exactly three agents. The tradeoff is rigidity. A pipeline cannot adapt; it runs the same sequence every time.
The next pattern handles tasks that do not depend on each other at all.
How does a manager-worker pattern parallelize independent work?
A manager-worker setup hands independent sub-tasks to multiple workers in parallel. The manager collects the results, evaluates them, and decides whether to accept, retry, or escalate. This pattern turns a batch of unrelated jobs into a single orchestrated run.
Picture a restaurant during the morning prep. The head chef tells one cook to chop vegetables, another to make soup stock, and a third to prepare sauces. All three work at once. The head chef inspects each result and deals with problems as they come.
Mailmind’s morning-brief feature demonstrates the pattern perfectly. A manager agent receives a batch of new emails. It spawns three workers:
- Worker one classifies each email by urgency and category.
- Worker two generates a two-sentence summary of every thread from the last 24 hours.
- Worker three extracts explicit action items: “Who needs a reply, who mentioned a deadline, who asked for a document.”
All three workers run in parallel because their tasks are fully independent. The manager collects the raw outputs, deduplicates overlapping action items, and synthesizes them into a single morning-brief report for the user.
Parallel execution cuts latency dramatically. Instead of three sequential calls, the user waits for only the slowest worker plus the manager’s synthesis step. The risk is that the manager must be smart enough to combine or reject conflicting outputs. A vague manager that simply concatenates raw text creates an unreadable brief.
- Agents run one after another
- Total time: sum of all agent durations
- Simple to implement and debug
- Best for dependent tasks
- Manager dispatches tasks in parallel
- Total time: slowest worker + manager
- Higher coordination overhead
- Best for independent tasks
Next we will look at how agents share information across these patterns.
What are the three patterns for passing state between agents?
Agents must share context. Without a clear state-passing strategy, information gets lost and the pipeline breaks. There are three practical approaches, each with its own tradeoffs.
Direct passing is the simplest: agent A’s output becomes agent B’s input. This works well in short sequential pipelines like Mailmind’s triage-draft-review flow. The chain is easy to audit. As the team grows, direct passing becomes unwieldy. Every new agent needs to know the full schema of the previous one. You end up with brittle, tightly coupled dependencies.
Shared memory store lets all agents read and write to a central store keyed by a thread_id. Mailmind uses this when multiple agents contribute to the same email thread over time. A scheduler writes a proposed meeting time, a drafting agent reads the last message, and a follow-up agent checks if the user sent a reply. The store acts as a blackboard. The downside is that any agent can corrupt the shared state, and recovery from bad writes is difficult.
Context object is a structured record that gets enriched at each stage. Mailmind defines a ThreadContext that starts with the original email. The triage agent adds priority and intent. The drafting agent appends draft_body and confidence. The review agent stamps approved or rejection_reason. Each stage reads the fields it needs and writes new ones without touching anything else. This approach keeps the schema explicit and auditable. It scales better than raw direct passing and avoids the chaos of a single shared blob.
Choosing among these patterns comes down to how many agents you have, how often they run, and how much you need to debug a single run.
Understanding how state moves sets the stage for the biggest danger in orchestration: the loop that never ends.
How do you prevent an agent team from deadlocking?
Deadlocks happen when a manager rejects a worker’s output, the worker retries, produces the same thing, and gets rejected again. This infinite loop wastes time, tokens, and the user’s patience. Three mechanisms stop it.
Max retries cap is the blunt instrument. After N attempts, the manager must make a decision: accept the best output, fall back to a simpler agent, or flag for human review. Mailmind sets a default cap of three retries for any worker. A cap alone is not enough; if every retry produces an identical failure, you burn all attempts with no progress.
Stagnation detection catches the identical-output problem. The system records a hash of the worker’s output. If the same hash appears twice in a row, the retry loop stops immediately. Mailmind applies this in the scheduling agent. When the worker keeps returning “no available time” without checking the user’s calendar, stagnation detection kills the loop after the second identical response instead of waiting for the cap.
Progressive fallback adds a ladder of degraded actions. First, the manager retries with specific feedback. Second, it calls a simpler fallback agent (for example, a rule-based scheduler instead of an LLM (large language model)). Finally, it escalates to the user with the best attempt and a clear explanation. Mailmind’s feedback is precise: “You missed the flight confirmation in the thread. Re-scan messages from airlines.” Generic “try again” instructions cause retries to produce the same result.
These three safeguards transform a fragile pipeline into one that degrades gracefully. They also unlock the next insight: not every task needs to run in order.
When do you run agents in parallel versus sequentially?
Run agents in parallel when their work does not depend on each other. Run them in sequence when one task needs the answer from the previous one. This sounds obvious, but misjudging dependencies is the most common orchestration bug.
In the morning-brief example, classification, summarization, and action-item extraction are independent. They can run in parallel. If you tried to classify before summarizing, you would waste time because the classification does not need the summary. Conversely, Mailmind’s reply pipeline cannot run in parallel. The drafting agent must know the priority from triage before it writes a response.
The rule is simple: draw a dependency graph. Each edge forces sequential execution. Nodes with no shared edges can fan out to parallel workers. Ignoring this graph leaves latency on the table or, worse, creates races where one agent reads stale data from a shared store.
With parallelism decided, the next question is who does what.
Why does specialization outperform a single all-purpose agent?
A single agent that triages, drafts, summarizes, and schedules performs worse than four focused agents. Specialization lets each agent’s prompt, tools, and context window be tuned to one job.
Mailmind’s triage agent only needs access to the email subject, sender, and first few sentences. Its prompt is short and directive: “Classify this email as urgent, reply-needed, or informational.” The drafting agent receives a larger context window with the full thread and a different instruction: “Draft a polite reply that addresses the sender’s questions.” A monolith agent would need a massive prompt covering all tasks. It would confuse priorities, mix personas, and waste tokens on irrelevant context.
Specialization also makes debugging easier. When the drafting agent writes a bad reply, you know exactly which prompt to fix. When the triage agent misclassifies, you adjust its few-shot examples without risking the draft quality. Single-responsibility applies to agents as much as it applies to classes in software design.
Beyond these core patterns, real systems layer additional coordination strategies.
What lies beyond pipelines and managers?
The patterns we have covered solve most practical problems. When the workload grows, a few advanced patterns become valuable.
Hierarchical orchestration introduces managers of managers. Mailmind could have a weekly-digest manager that delegates to a Monday-morning manager and a Friday-summary manager. Each of those runs its own set of workers. The top-level manager synthesizes the day-level reports into one weekly email.
Event-driven orchestration triggers agents based on events rather than a fixed schedule. A new email arriving in Mailmind fires a triage agent. If triage marks it important, that event triggers a drafting agent and a scheduling agent. No central conductor calls each step; agents react to state changes. This model reduces latency because work starts the moment the event arrives.
Blackboard architecture goes further: agents share a workspace and self-select tasks. A new email appears on the blackboard. A triage agent notices it has no priority and claims it. Later, an unaddressed action item sits on the board. Any free scheduling agent can pick it up. This pattern is highly resilient because no single point of failure exists. It is also harder to debug because the sequence is emergent.
Choreography vs orchestration frames the control decision. Orchestration uses a central conductor that knows the full workflow. You can log every step and reason about failures. Choreography has no central brain; agents communicate directly through events. It is more resilient to partial failures but far harder to trace. Most production systems, including Mailmind, use orchestration for critical paths and choreography for background tasks like receipt filing.
Quick Reference
| Pattern / Technique | What it does | Use when |
|---|---|---|
| Sequential pipeline | Agents run in order, output feeds next | Steps depend on previous results |
| Manager-worker | Manager parallelizes independent tasks | Subtasks have no shared dependencies |
| Direct state passing | Output → input, no intermediate store | Short pipelines, tight coupling |
| Shared memory store | All agents read/write keyed store | Multiple agents over a long-lived thread |
| Context object | Structured record enriched at each stage | Need schema control and auditability |
| Max retries cap | Hard limit on retries | Prevent infinite burn |
| Stagnation detection | Stop if output unchanged | Identical-response deadlock |
| Progressive fallback | Step-down from retry to fallback to human | Graceful degradation |
Frequently Asked Questions
Q: When does a manager-worker pattern add more overhead than it saves? A manager introduces an extra LLM call and coordination logic. For a batch of two small tasks, running them sequentially without a manager is usually faster. The point of diminishing returns sits around three genuinely independent tasks. Beyond that, the parallel speedup justifies the overhead.
Q: How do you decide between a context object and a shared memory store? Use a context object when you have a fixed set of fields that get filled at known stages. You can statically type the record and validate it. Use a shared memory store when agents run at unpredictable times and need to read arbitrary keys. The store is more flexible and much harder to keep consistent.
Q: Can a worker be both a specialized agent and a manager itself? Yes. A worker that receives a complex task can spawn its own sub-workers as a local manager. That is hierarchical orchestration. The top-level manager only sees the composite result. You pay the cost of nested LLM calls, so only justify it when the sub-task truly requires its own decomposition.
Q: What is the simplest code pattern for stagnation detection? Hash the worker’s output string with a fast hash like SHA-256. Store the hash after each attempt. If the new hash matches the previous one, break the loop immediately. This catches identical outputs even when the token-level text varies slightly, but you must decide whether to hash the raw text or a normalized version.
Q: Why not always use event-driven orchestration for everything? Event-driven systems are great for latency but horrible for debugging. When a bad reply reaches the user, you have to trace a chain of events with no single execution record. For user-facing flows like drafting an email, Mailmind uses orchestration so every step appears in a single trace. For background jobs like receipt filing, events are fine.
Test yourself
Mailmind’s manager keeps rejecting the scheduling worker’s output. After five retries, the pipeline is stuck, and the user sees no meeting invitation. Your logs show that each retry produces a slightly different but still unacceptable suggestion. The manager’s feedback is “Schedule a time that works for both parties.”
What is the most likely cause, and how would you unblock and prevent this?
Answer: The problem is feedback that lacks specificity. The manager says “that works for both parties,” but the worker has no access to actual calendar availability, so it guesses. Each attempt produces a different wrong time. First, unblock the pipeline by taking the most plausible suggestion, flagging it for the user with a note that scheduling failed, and letting the other workers finish. Then fix the root cause: give the manager a calendar tool to query free slots, and rewrite the feedback to include concrete constraints. Instead of “try again,” the manager should say “The proposed time 3 PM conflicts with the user’s existing meeting. Find the next free 30-minute slot tomorrow morning.” To prevent recurrence, add a stagnation detector that also compares the semantic similarity of outputs, not just exact hashes. If three consecutive outputs score above 0.9 cosine similarity, break and escalate.
If you want this level of detail every week, how real AI systems work under the hood, not fluffy hype, subscribe to Internals Decoded at internalsdecoded.com. The next episode picks up where this left off: we will tear down how tool calling actually works inside an agent loop and why function schemas are more dangerous than you think.
Sources
- ReAct: Synergizing Reasoning and Acting in Language Models, the foundational paper on the ReAct loop that underpins single-agent reasoning and provides the substrate for multi-agent orchestration.
- internalsdecoded.com