Loop Engineering Introduction

When to give an agent another attempt, and when to add another agent

Animated steps and three labs you can run. Each lab runs a loop of agents with eight layers switched on and off, and scores every run with the check and the answer key.

Decision Q&A

Four questions this volume answers

1Why does this matter?

On hard cases agents go in circles, give up, or say they're done when they're wrong. More turns help a loop that knows it failed, and do nothing for one whose own check is fooled by the wrong answer. And unless you count every model call across every agent, four agents finishing a job look the same as one.

Shown in step 1 · 2 · 9
2How would I design it?

Decide at the whiteboard: take a plausible wrong answer and ask what your check says about it. If it goes red and names the case, a second attempt is enough. If it gives full marks, you need a second agent and someone with the authority to settle a disagreement. If you split the work, hand over a structured record rather than a summary, and add an arbitrator; either alone leaves you short.

Shown in step 1 · 6 · 7 · 11
3How would I debug it?

The dangerous state is “done” and wrong: the loop stops, reports success, and nothing raises an alert. Compare the check the loop runs with an answer key it never sees. When a split answer is off, look for the fact lost in a summary: one dropped date produced a wrong total that the check still scored 3 of 4. And don't judge layers by adding them in order: on two of three jobs the score fell for four stages before the last layer brought it back.

Shown in step 2 · 8 · 10
4How would I take it to production?

A turn limit and stall detection did nothing in the labs, because the agents never went in circles. Keep them anyway: once your agent can loop, the limit is the difference between a bill and an incident. Report progress per turn next to how much got done, since every layer buys a more complete answer with more turns. Splitting costs turns straight away and pays only when agents disagree in ways one agent couldn't notice.

Shown in step 4 · 5 · 12

Introduction

Your agent handles the easy cases. On the rest it does one of three things: it goes in circles, it gives up, or it says it’s done when it’s wrong. You’ve read that the fix is more agents, and you’re about to build a team of four, because that’s what the examples show.

Two failures that look the same, and aren’t

Everything here rests on telling these two apart. Almost every technique in the subject treats only the first.

the failurewhat the loop seeswhat more turns do
it knows it failedits check says not donehelp. Step 6 measures how much
it thinks it succeededits check says done, and it’s wrongnothing, ever. Step 9

More turns help a loop that knows it has failed. They do nothing for a loop whose own check is fooled by the wrong answer. No amount of retrying, turn limits or stall detection changes that, because none of them looks at whether the answer is right.

What loop engineering is

Loop engineering is deciding how many attempts an agent gets, who makes them, and what has to be true before it’s allowed to stop.

This volume splits it into eight layers you can switch on and off, in four phases. It measures what each one buys by keeping the agents the same (scripted, so they make the same choices every run) and switching the layers around them. Any change in the result can only come from the loop.

phasethe question it answersthe two layers
more timehow long before you stop?a turn limit and stall detection
another gowhat if one attempt wasn't enough?a second attempt and a changed second attempt
more agentswhat if one agent isn't enough?splitting across agents and a reviewer
keeping them honestwhat does splitting break?structured handoffs and an arbitrator

The jobs

Each lab runs one job, made of four parts, so a run can be partly right. The jobs differ on the one thing step 11 turns into a rule: what the loop’s own check says about a wrong answer.

jobthe taska wrong answer looks likecan a second attempt see what went wrong?
fix-a-testfix four bugs in one small programthe test suite goes red, and names the caseyes
triagefind which framework caused one failing testa plausible wrong cause, given full marksno
answeranswer a lodging question from four documentsa wrong total that the check barely noticesyes

The main result, up front

The most efficient loop measured here is one agent, alone, with nothing switched on. It finishes three of four parts at 0.75 parts per turn, and the complete loop with all eight layers never beats 0.36. Every layer here buys a more complete answer with more turns. None makes a loop faster, and on the fix-a-test job, adding layers in order makes it worse for four stages in a row before it gets better.

Is this for you?

if…then
your agent stops early and says it’s donestep 9, and more turns aren’t the answer
you’re about to split one task across several agentsstep 7, which counts what that costs before what it buys
you have a team of four and they disagreestep 7’s second half: someone has to be allowed to settle it
you’re adding layers in order and it’s getting worsestep 11. That’s measured here, and it recovers
your loop finishes, correctly, and you can’t say whystep 4. Nothing else works until a turn is defined

How the measuring works

One scripted set of agents, one fixed turn order, and every figure reproduces exactly. There are four ways of measuring, because the first one misleads:

way of measuringwhat it doeswhat it misses
adding layers one at a timeswitch the layers on in order, one more each timethat adding the next layer can make things worse
taking each layer outremove one layer at a time from the complete loopany layer whose job is to repair another layer’s damage
starting from a split loopadd each repair to a loop that has only been splitnothing: this is where the repairs show up
each layer on and offswitch one layer off in the complete loop, then onthe same blind spot as taking each layer out

The third way exists because of a result in step 7: a layer that repairs damage done by another layer looks worthless wherever the damage is already repaired. That’s a fact about the measuring, not about the layer.

What you’ll get

  • One question about your own check that tells you whether you need a second attempt or a second agent, before you build anything.
  • A way to count what a loop costs, so you can compare one agent with four.
  • Three labs you can run on your own computer. The next section explains them.

The labs: what they are

Each lab is a small Python program you download and run yourself. Once you own the volume, the Download the labs button at the top of this page gives you all three in one zip. Each one runs the loop on one job:

  • Fix-a-test: fix four bugs so a test suite passes. The suite names what failed.
  • Triage: find which framework caused a failing test. The check gives full marks to a plausible wrong cause.
  • Answer: answer a lodging question from four documents, where one fact is a date that a summary can lose.

What you do in the labs

  • Run the bare loop, and count its turns.
  • Add the layers one at a time, then take each one out of the complete loop.
  • Start from a split loop and add the repairs.
  • Run any loop with every turn shown, and compare the check with the answer key.

What you’ll learn

  • Why the most efficient loop is often the one with nothing switched on.
  • Why a second attempt and a second agent are alternatives, not steps.
  • What splitting work loses, and the two layers that repair it.
  • How to tell, from your check alone, which cure you need.

Two ways to run them

waywho writes the final answerwhat it coststhe results
scriptedbuilt-in scripted agents, the same every runfree, and works offlinethe same every run. These pages quote them.
livea real model: Claude through the API, Claude Code, or an OpenAI modela cent or two: two or three short questions per laba real model’s own answer, marked by the same check and answer key

All three labs can run either way.

Before a live run

  • Run the setup script in the labs folder. It installs everything a live run needs.
  • Pick a model and put its key in the settings file, labs/.env. For Claude Code you need no key. The labs’ README shows every line.
  • Check the connection with one tiny call. It costs well under a cent.
  • Start the run. Before sending anything, it shows how many questions it will ask and what they should cost, and waits for you to type yes. A spending cap stops the run if costs go over, and every answer is saved, so you never pay twice for the same question.
Your live results may differ from these pages. A real model writes the final answer in its own words, and may do better or worse than the scripted agent with the same facts. The answer keys accept equivalent wording, so a right answer in different words still scores.

How to read the labels

Each step has a label that tells you where its content comes from.

labelmeans
conceptexplained from first principles; no lab numbers
measureda lab compares loops and reports the difference
laba guided lab you run yourself, step by step
your own texthow to use the method on your own work
These labs aren’t a benchmark. Three jobs, four parts each, scripted agents, and the agents never run at the same time: they take turns in a fixed order so every figure reproduces. Step 12 lists every limit, starting with that one.