Agent Design Patterns
What you’ll learn. The standard vocabulary of agent architectures: prompt chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer, ReAct, plan-and-execute and reflection. For each: the mechanism, a code sketch, where it helps, and how it fails.
Patterns as compositions of one primitive
Section titled “Patterns as compositions of one primitive”Every pattern below is built from a single primitive, the augmented LLM call: a model invocation that may use tools and retrieved memory. Patterns differ in how calls are wired together and who decides the wiring. Keep the workflow vs agent spectrum in mind: the first four patterns are workflows with developer-defined structure; ReAct and plan-and-execute move control to the model.
Throughout, llm(prompt) -> str is a generic text-completion function.
Prompt chaining
Section titled “Prompt chaining”Decompose a task into a fixed sequence; each call consumes the previous output. Optionally insert programmatic gates between steps.
def write_report(llm, topic): outline = llm(f"Write a 5-point outline on: {topic}") if outline.count("\n") < 4: # gate outline = llm(f"Expand to 5 points:\n{outline}") draft = llm(f"Write the report following this outline:\n{outline}") return llm(f"Fix grammar and tighten:\n{draft}")Use when the task decomposes cleanly and you want to trade latency for accuracy: each call does one easy thing. The failure mode is error propagation; add gates and validation.
Routing
Section titled “Routing”Classify the input, then dispatch to a specialized prompt, model or tool.
ROUTES = {"refund": handle_refund, "technical": handle_tech, "other": handle_general}
def route(llm, query): label = llm(f"Classify as refund/technical/other. One word.\n{query}").strip().lower() return ROUTES.get(label, handle_general)(query)Routing is also the main cost lever: send easy queries to a small cheap model and hard ones to a large model (see Production, observability and cost). The router itself must be evaluated; a misroute is silent.
Parallelization
Section titled “Parallelization”Two variants:
- Sectioning: split independent subtasks and run concurrently (e.g. review a document for security, style and correctness in three parallel calls).
- Voting: run the same task several times and aggregate (majority vote, or a judge). This is the simplest form of test-time compute.
import asyncioasync def review(allm, code): aspects = ["security", "performance", "readability"] notes = await asyncio.gather(*[allm(f"Review {code!r} for {a}.") for a in aspects]) return dict(zip(aspects, notes))Latency is the max, not the sum, of branches; cost is the sum.
Orchestrator-workers
Section titled “Orchestrator-workers”A central LLM dynamically decides the subtasks, delegates to worker calls, and synthesizes. Unlike parallelization, the subtasks are not known in advance (e.g. “which files need editing for this feature?”).
def orchestrate(llm, task): plan = json.loads(llm(f"Split into independent subtasks as a JSON list of strings:\n{task}")) results = [llm(f"Do this subtask:\n{s}") for s in plan] # could be parallel return llm(f"Task: {task}\nSubtask results:\n" + "\n".join(results) + "\nSynthesize a final answer.")This is the seed of multi-agent systems. The orchestrator’s context stays small because workers return summaries, not transcripts.
Evaluator-optimizer
Section titled “Evaluator-optimizer”One call generates, another critiques against explicit criteria, and the loop repeats until the evaluator accepts or a budget ends.
def refine(llm, task, criteria, rounds=3): draft = llm(task) for _ in range(rounds): verdict = llm(f"Criteria: {criteria}\nDraft:\n{draft}\nReply PASS or list concrete fixes.") if verdict.strip().startswith("PASS"): break draft = llm(f"{task}\nPrevious draft:\n{draft}\nFixes required:\n{verdict}") return draftIt works when (a) evaluation is easier than generation and (b) you have clear criteria. It fails when the evaluator is the same model with the same blind spots; an external signal (tests, a compiler, a retrieval check) is far stronger. Details in Self-refinement and learning from feedback.
ReAct: interleaving reasoning and acting
Section titled “ReAct: interleaving reasoning and acting”ReAct (Yao et al., 2023) has the model emit a thought, then an action, then receive an observation, repeating.
Thought: I need the population of Lyon. I'll search.Action: search("Lyon population")Observation: Lyon has about 520,000 inhabitants (city proper).Thought: The question asks about the metro area; search again.Action: search("Lyon metropolitan area population")...Action: finish("About 2.3 million")The thoughts serve as scratch space that conditions later steps and provides a readable audit trail. With modern native tool-calling and reasoning models, the “Thought” is often hidden or internal, but the structure is the default agent loop shown in the previous lecture. Failure modes: looping on the same failed action, hallucinating observations when the format is text-parsed, and thought/action drift on long runs.
Plan-and-execute
Section titled “Plan-and-execute”Separate a planner (produces a multi-step plan up front) from an executor (runs steps, possibly with a cheaper model), with optional replanning when a step fails.
Plan: 1. find the failing test 2. read the stack trace 3. locate the function 4. patch 5. rerun testsExecute step 1 ... observe ... if surprise: REPLANBenefits: fewer expensive planning calls, a plan a human can approve, resistance to myopic wandering. Costs: plans made without seeing observations are often wrong; rigid plans are brittle. Practical hybrids keep a living plan (a todo list the agent edits) in context, which both guides and records progress. Coding agents commonly expose exactly this as a todo tool.
Reflection
Section titled “Reflection”Reflection means the agent critiques its own trajectory and carries the lesson forward. Reflexion (Shinn et al., 2023) stores verbal self-critiques of failed attempts in memory and conditions the next attempt on them. Within a single run, reflection can be as small as “before finishing, re-read the task and check each requirement”. It is the evaluator-optimizer loop applied to trajectories, with the same caveat: reflection without external grounding can reinforce errors, and evidence suggests small models often cannot correct themselves without a verifier.
Choosing a pattern
Section titled “Choosing a pattern”Is the structure known in advance? yes -> workflow: sequential, easy steps ........ prompt chaining input categories differ ....... routing independent parts / need votes parallelization no -> dynamic: subtasks unknown, need delegate orchestrator-workers open-ended tool exploration .... ReAct loop (+ todo-list planning)Do you have a checker (tests, schema, critic)? -> add evaluator-optimizerDid a previous attempt fail and you can retry? -> reflection with memoryPrinciples that hold across patterns:
- Start simple. A single call with good retrieval beats a baroque graph you cannot debug.
- Make the plan and state visible. Logged intermediate artifacts are your debugging surface.
- Prefer external verification to self-judgement wherever one exists.
- Bound everything: steps, tokens, wall-clock, retries.
- Frameworks are optional. They help with plumbing but obscure prompts and control flow; understand the raw loop first (Build an agent from scratch).
Pitfalls
Section titled “Pitfalls”- Over-decomposition: each added call multiplies latency, cost and error probability ( again).
- Unclear termination: evaluator loops without a round cap and agents without a stop condition run away.
- Context bloat: keeping every observation in a ReAct transcript degrades later steps; see Memory and context engineering.
- Leaky delegation: workers given too little context redo or contradict each other. Pass the goal, the constraints and the expected output format.
Key takeaways
Section titled “Key takeaways”- All patterns compose augmented LLM calls; they differ in who controls the wiring.
- Chaining, routing, parallelization, orchestrator-workers and evaluator-optimizer are the workflow toolkit; ReAct and plan-and-execute give the model control.
- Reflection and evaluator loops help most when grounded in external checks.
- Pick the simplest pattern that passes your evaluation; every extra step costs reliability.
Exercises
Section titled “Exercises”- Conceptual. Which pattern fits: (a) translating text to five languages; (b) triaging support emails; (c) fixing an unknown number of lint errors across a repo; (d) generating SQL that must execute without error?
- Compute. A chain of 4 calls each succeeds 95% of the time, with a gate after each that catches 70% of failures and triggers one retry (retry succeeds 95%). Estimate the end-to-end success.
- Hands-on. Implement the evaluator-optimizer loop with a Python
exec-based test as the evaluator for “writeis_prime”. Compare to an LLM-only critic using any client. - Design. Sketch an orchestrator-workers system for “summarize 40 research papers”; decide what each worker returns to keep orchestrator context small.
Solutions (1, 2)
- (a) parallelization (sectioning); (b) routing; (c) orchestrator-workers or ReAct; (d) evaluator-optimizer with execution as evaluator.
- Per step success after gate/retry: . Over 4 steps: about versus without gates (assuming the gate has no false positives).
Further reading
Section titled “Further reading”- Anthropic, “Building effective agents” (2024).
- Yao et al., “ReAct” (2023); Shinn et al., “Reflexion” (2023); Madaan et al., “Self-Refine” (2023).
- Wang et al., “Plan-and-Solve Prompting” (2023).
- Wu et al., “AutoGen” (2023) for orchestration conversations.