Choosing how to arrange an agent system: one agent, a pipeline, a router, or more
Animated steps and three labs you can run. Each lab runs six shapes on the same job, prices each one, and marks it against the answer key.
- All 12 steps, read online
- Runnable labs to download (plain Python, runs offline)
- Free updates whenever it's revised
- 14-day refund (conditions)
Four questions this volume answers
Every article recommends a different arrangement with equal confidence: one agent, a pipeline, a router, a planner, a second look, or a fan-out. Run on three workloads, a different one was cheapest each time, two were never cheapest anywhere, and the arrangement alone tripled the cost and decided whether the answer was right. On a small, clean job the simplest arrangement wins, and you can stop there.
Answer four questions before choosing any arrangement: what proves the work right, what must a person decide, how many kinds of problem there are, and can the simplest arrangement do it (run it once). Only then decide on a router, a plan, a second look or splitting the work. Price an arrangement by what it asks: questions to the model, lines sent to a person, and wrong answers. The arithmetic is free.
Passing checks doesn't mean right. On the largest workload, five of six arrangements passed every check with one box wrong, because one word (“press”) put a line in the wrong category. Look for what no check can compare, and who knows the right answer some other way. A router only helps if the signal it routes on lines up with the errors, so look at where its careful looks actually went.
Treat sending work to a person as a cost to weigh: most of the time it caught nothing, and twice it caught exactly the error that made a winner win. Compare again when your assumptions change, because the price of a wrong answer and of a person's time both changed the winner. And test whether your model loses track on large batches, because that decides whether asking about many lines at once is really free.
Introduction
You need to choose how to arrange an agent system: one agent with tools, a pipeline, a router, a planner, a checker, or work split across several agents. Every article you’ve read recommends a different one, with equal confidence.
They can all be right. This volume runs one job through six arrangements on three workloads, and gets a different cheapest arrangement on each. No arrangement wins twice. That’s a fact about workloads, not about architectures.
The job every comparison runs
A small shop in the UK files a sales-tax return every three months (in the UK it’s called a VAT return). The form has nine boxes: tax on sales, tax reclaimed on purchases, the totals, and so on. To fill it in, the system:
- reads the shop’s sales and purchase lines;
- sorts each line into a category (cold food, hot food, newspapers, rent, and so on);
- looks up the tax rate for that category;
- adds the lines up into the nine boxes.
Sorting is the only step that needs a model. Each shape is a different way of arranging that same work. The comparisons run on three different quarters of the shop’s year, each with its own lines.
What a shape is
An agent’s shape is how the work is arranged: who asks the model, how many times, in what order, and who decides what happens next.
The six shapes here run the same job on the same lines, against the same 9 checks, with the same model and the same instructions. Only the arrangement differs, and that alone can triple the cost and decide whether the answer is right.
| shape | often called | the arrangement |
|---|---|---|
| one agent with tools | a single agent | one agent, asked about each line in turn, with tools to read files and look up rates |
| fixed stages | a pipeline | read, sort, work out the tax, check: always in that order, with one question per batch of lines |
| sort first, then handle | a router | decide which kind each line is, and send each kind down its own route |
| plan, then carry out | a planner | ask for a plan, then carry it out |
| do, then check | a second opinion | do the work, then take a second look at it before handing it in |
| split and rejoin | a fan-out | split the lines into batches, handle each one separately, then put the results back together |
The result, up front
| quarter | lines | kinds of problem | cheapest shape | cost |
|---|---|---|---|---|
| the quiet quarter | 22 | 2 | fixed stages tied with split and rejoin | $2.10 |
| the messy quarter | 140 | 6 | do, then check | $23.42 |
| the busy quarter | 4,000 | 2 | sort first, then handle | $45.19 |
- No shape is cheapest twice.
- Two of the six are never cheapest anywhere.
- One is the most expensive on one quarter and the cheapest on another.
The three things that cost money
Reading files, looking rates up, adding the boxes and running the checks are arithmetic, and arithmetic is free. So a shape is priced by what it asks, not by what it does.
| what costs money | priced at (an assumption you can change) |
|---|---|
| asking the model a question | $0.01 a question, plus $1 for every thousand words sent |
| sending one line to a person | $2.00 a line: somebody stopping to look, find the bill, and decide |
| getting a box wrong | $40.00 a box: a correction, often a penalty, and an afternoon of somebody’s time. Only you know what it really costs you |
| everything else | free: reading, looking up, adding, sorting, comparing, and running every check |
These prices are a plausible guess, not anyone’s price list. The labs let you change each one and watch the winner move.
Eight questions, and an exit at the fourth
The first four come before you choose any shape. None of them mentions a shape, and you can answer all four in an afternoon. The fourth is an exit: on one of the three quarters, it ends the job.
| # | the question | what answering it takes |
|---|---|---|
| 1 | what proves the work is right | writing the checks |
| 2 | what must a person decide, whatever you build | naming those cases |
| 3 | how many different kinds of problem are in here | counting them |
| 4 | can the simplest shape do it | running it once |
| 5 | do different kinds of line need different handling | a route per kind |
| 6 | does anything need a plan before it can be done | a plan first |
| 7 | is a wrong answer expensive enough to pay for a second look | a second look |
| 8 | is there enough work to split, and what does putting it back together cost | batches |
Is this for you?
| if… | then |
|---|---|
| you’re choosing between a pipeline and a router | step 6, but run step 4 first: it may not matter |
| your input is large, so you’re reaching for a router | step 9. Many lines and many kinds of problem are different things, and they pick different shapes |
| all your checks pass and you want to know what that proves | step 10. On one quarter it proves nothing |
| someone asked you to justify a second-look step | step 7, then step 11 |
| you’ve already picked a framework | step 1. A framework is one way of building one shape |
The three quarters
Same shop, same rules, same checks, same six shapes. Only the lines differ, and what decides the winner isn’t size.
| quarter | lines | kinds of problem | what it teaches |
|---|---|---|---|
| the quiet quarter | 22 | 2 | nothing beats the simplest shape, so you can stop |
| the messy quarter | 140 | 6 | many kinds of problem, not many lines |
| the busy quarter | 4,000 | 2 | many lines, and a wrong answer nothing can see |
The busy quarter has fewer kinds of problem than the messy one, on purpose. One is about many lines, the other about many kinds of problem, and they have different winners for that reason.
What you’ll get
- Four questions to answer before you choose any shape, and the one that can end the job.
- A way to price a shape by what it asks, so you can compare them before building anything.
- Three labs you can run on your own computer. The next section explains them.
The labs: what they are
Each lab is a small Python program you download and run yourself. Once you own the volume, the Download the labs button at the top of this page gives you all three in one zip. Each one runs all six shapes on one quarter:
- Quiet quarter: 22 lines, hand-written and clean.
- Messy quarter: 140 lines with 6 kinds of problem in them.
- Busy quarter: 4,000 lines, only 2 kinds of problem, and one wrong box no check can see.
What you do in the labs
- Count the kinds of problem in a quarter, and run the simplest shape.
- Run all six shapes and price each one.
- Send lines to a person, and see when that pays.
- Change what a wrong answer costs, and watch the winner move.
What you’ll learn
- Why the simplest shape is often the answer, and how to find out in one run.
- Why a router can come last on the very workload it seems built for.
- Why every check can pass while the answer is wrong.
- Why the price of a wrong answer, which only you can set, decides the winner.
Two ways to run them
| way | who does the sorting | what it costs | the results |
|---|---|---|---|
| scripted | a built-in scripted model that sorts by matching words | free, and works offline | the same every run. These pages quote them. |
| live | a real model: Claude through the API, Claude Code, or an OpenAI model | about a cent: each quarter has only about twenty different lines, so it’s usually one question | a real model’s own sorting, marked against the same answer key |
All three labs can run either way.
Before a live run
- Run the setup script in the labs folder. It installs everything a live run needs.
- Pick a model and put its key in the settings file,
labs/.env. For Claude Code you need no key. The labs’ README shows every line. - Check the connection with one tiny call. It costs well under a cent.
- Start the run. Before sending anything, it shows how many questions it will ask and what they should cost, and waits for you to type yes. A spending cap stops the run if costs go over, and every answer is saved, so you never pay twice for the same question.
How to read the labels
Each step has a label that tells you where its content comes from.
| label | means |
|---|---|
| concept | explained from first principles; no lab numbers |
| measured | a lab compares the shapes and reports the difference |
| lab | a guided lab you run yourself, step by step |
| your own text | how to use the method on your own work |