Agent Architecture Introduction

Choosing how to arrange an agent system: one agent, a pipeline, a router, or more

Animated steps and three labs you can run. Each lab runs six shapes on the same job, prices each one, and marks it against the answer key.

Decision Q&A

Four questions this volume answers

1Why does this matter?

Every article recommends a different arrangement with equal confidence: one agent, a pipeline, a router, a planner, a second look, or a fan-out. Run on three workloads, a different one was cheapest each time, two were never cheapest anywhere, and the arrangement alone tripled the cost and decided whether the answer was right. On a small, clean job the simplest arrangement wins, and you can stop there.

Shown in step 1 · 8 · 11
2How would I design it?

Answer four questions before choosing any arrangement: what proves the work right, what must a person decide, how many kinds of problem there are, and can the simplest arrangement do it (run it once). Only then decide on a router, a plan, a second look or splitting the work. Price an arrangement by what it asks: questions to the model, lines sent to a person, and wrong answers. The arithmetic is free.

Shown in step 3 · 4 · 6
3How would I debug it?

Passing checks doesn't mean right. On the largest workload, five of six arrangements passed every check with one box wrong, because one word (“press”) put a line in the wrong category. Look for what no check can compare, and who knows the right answer some other way. A router only helps if the signal it routes on lines up with the errors, so look at where its careful looks actually went.

Shown in step 2 · 9 · 10
4How would I take it to production?

Treat sending work to a person as a cost to weigh: most of the time it caught nothing, and twice it caught exactly the error that made a winner win. Compare again when your assumptions change, because the price of a wrong answer and of a person's time both changed the winner. And test whether your model loses track on large batches, because that decides whether asking about many lines at once is really free.

Shown in step 7 · 11 · 12

Introduction

You need to choose how to arrange an agent system: one agent with tools, a pipeline, a router, a planner, a checker, or work split across several agents. Every article you’ve read recommends a different one, with equal confidence.

They can all be right. This volume runs one job through six arrangements on three workloads, and gets a different cheapest arrangement on each. No arrangement wins twice. That’s a fact about workloads, not about architectures.

The job every comparison runs

A small shop in the UK files a sales-tax return every three months (in the UK it’s called a VAT return). The form has nine boxes: tax on sales, tax reclaimed on purchases, the totals, and so on. To fill it in, the system:

  • reads the shop’s sales and purchase lines;
  • sorts each line into a category (cold food, hot food, newspapers, rent, and so on);
  • looks up the tax rate for that category;
  • adds the lines up into the nine boxes.

Sorting is the only step that needs a model. Each shape is a different way of arranging that same work. The comparisons run on three different quarters of the shop’s year, each with its own lines.

This is an example, not tax advice. The shop is imaginary, the rules are an example, and nothing here ever files anything. The lessons don’t depend on the country or the tax.

What a shape is

An agent’s shape is how the work is arranged: who asks the model, how many times, in what order, and who decides what happens next.

The six shapes here run the same job on the same lines, against the same 9 checks, with the same model and the same instructions. Only the arrangement differs, and that alone can triple the cost and decide whether the answer is right.

shapeoften calledthe arrangement
one agent with toolsa single agentone agent, asked about each line in turn, with tools to read files and look up rates
fixed stagesa pipelineread, sort, work out the tax, check: always in that order, with one question per batch of lines
sort first, then handlea routerdecide which kind each line is, and send each kind down its own route
plan, then carry outa plannerask for a plan, then carry it out
do, then checka second opiniondo the work, then take a second look at it before handing it in
split and rejoina fan-outsplit the lines into batches, handle each one separately, then put the results back together

The result, up front

quarterlineskinds of problemcheapest shapecost
the quiet quarter222fixed stages tied with split and rejoin$2.10
the messy quarter1406do, then check$23.42
the busy quarter4,0002sort first, then handle$45.19
  • No shape is cheapest twice.
  • Two of the six are never cheapest anywhere.
  • One is the most expensive on one quarter and the cheapest on another.
And on the busy quarter, every check passes and the form is wrong. Five of the six shapes pass all 9 checks, and every one of them gets one box wrong, because of one word in one line. No check could have caught it.

The three things that cost money

Reading files, looking rates up, adding the boxes and running the checks are arithmetic, and arithmetic is free. So a shape is priced by what it asks, not by what it does.

what costs moneypriced at (an assumption you can change)
asking the model a question$0.01 a question, plus $1 for every thousand words sent
sending one line to a person$2.00 a line: somebody stopping to look, find the bill, and decide
getting a box wrong$40.00 a box: a correction, often a penalty, and an afternoon of somebody’s time. Only you know what it really costs you
everything elsefree: reading, looking up, adding, sorting, comparing, and running every check

These prices are a plausible guess, not anyone’s price list. The labs let you change each one and watch the winner move.

Eight questions, and an exit at the fourth

The first four come before you choose any shape. None of them mentions a shape, and you can answer all four in an afternoon. The fourth is an exit: on one of the three quarters, it ends the job.

#the questionwhat answering it takes
1what proves the work is rightwriting the checks
2what must a person decide, whatever you buildnaming those cases
3how many different kinds of problem are in herecounting them
4can the simplest shape do itrunning it once
5do different kinds of line need different handlinga route per kind
6does anything need a plan before it can be donea plan first
7is a wrong answer expensive enough to pay for a second looka second look
8is there enough work to split, and what does putting it back together costbatches

Is this for you?

if…then
you’re choosing between a pipeline and a routerstep 6, but run step 4 first: it may not matter
your input is large, so you’re reaching for a routerstep 9. Many lines and many kinds of problem are different things, and they pick different shapes
all your checks pass and you want to know what that provesstep 10. On one quarter it proves nothing
someone asked you to justify a second-look stepstep 7, then step 11
you’ve already picked a frameworkstep 1. A framework is one way of building one shape

The three quarters

Same shop, same rules, same checks, same six shapes. Only the lines differ, and what decides the winner isn’t size.

quarterlineskinds of problemwhat it teaches
the quiet quarter222nothing beats the simplest shape, so you can stop
the messy quarter1406many kinds of problem, not many lines
the busy quarter4,0002many lines, and a wrong answer nothing can see

The busy quarter has fewer kinds of problem than the messy one, on purpose. One is about many lines, the other about many kinds of problem, and they have different winners for that reason.

What you’ll get

  • Four questions to answer before you choose any shape, and the one that can end the job.
  • A way to price a shape by what it asks, so you can compare them before building anything.
  • Three labs you can run on your own computer. The next section explains them.

The labs: what they are

Each lab is a small Python program you download and run yourself. Once you own the volume, the Download the labs button at the top of this page gives you all three in one zip. Each one runs all six shapes on one quarter:

  • Quiet quarter: 22 lines, hand-written and clean.
  • Messy quarter: 140 lines with 6 kinds of problem in them.
  • Busy quarter: 4,000 lines, only 2 kinds of problem, and one wrong box no check can see.

What you do in the labs

  • Count the kinds of problem in a quarter, and run the simplest shape.
  • Run all six shapes and price each one.
  • Send lines to a person, and see when that pays.
  • Change what a wrong answer costs, and watch the winner move.

What you’ll learn

  • Why the simplest shape is often the answer, and how to find out in one run.
  • Why a router can come last on the very workload it seems built for.
  • Why every check can pass while the answer is wrong.
  • Why the price of a wrong answer, which only you can set, decides the winner.

Two ways to run them

waywho does the sortingwhat it coststhe results
scripteda built-in scripted model that sorts by matching wordsfree, and works offlinethe same every run. These pages quote them.
livea real model: Claude through the API, Claude Code, or an OpenAI modelabout a cent: each quarter has only about twenty different lines, so it’s usually one questiona real model’s own sorting, marked against the same answer key

All three labs can run either way.

Before a live run

  • Run the setup script in the labs folder. It installs everything a live run needs.
  • Pick a model and put its key in the settings file, labs/.env. For Claude Code you need no key. The labs’ README shows every line.
  • Check the connection with one tiny call. It costs well under a cent.
  • Start the run. Before sending anything, it shows how many questions it will ask and what they should cost, and waits for you to type yes. A spending cap stops the run if costs go over, and every answer is saved, so you never pay twice for the same question.
Your live results may differ from these pages. A real model sorts by understanding the words, not by matching them, so it may get right what the scripted model gets wrong, or the other way round. The same answer key and the same checks mark it.

How to read the labels

Each step has a label that tells you where its content comes from.

labelmeans
conceptexplained from first principles; no lab numbers
measureda lab compares the shapes and reports the difference
laba guided lab you run yourself, step by step
your own texthow to use the method on your own work
These labs aren’t a benchmark. Three quarters of one shop, one scripted model, and a cost model in which asking about many lines at once gives the same answers as asking about one. That last assumption is why one of the six shapes can never win, and step 12 says so. Every finding also depends on there being a way to check the work at all.