Agent Harness Engineering Concepts › What an agent harness is
01 · part 1, conceptsconcept

What an agent harness is

An agent harness is everything around the model call that you write yourself: the tools the model may use, the messages it gets back, the checks on what it claims, the room it works in, and the record of what it did.

The model is the part you didn’t write

A working agent is a model inside a program. Almost none of that program is the model, and almost all of it is yours.

The model is the part you did not write tools errors checker compaction stuck detection record model A model B a vendor wrote this swap the model… …and every ring stays That is why the harness is the thing you can measure: hold the model fixed, change one ring, and the difference can only have come from the ring. Six rings. Every one of them is code you own.
The model in the middle, the harness around it. Read it from the center out.
Swap the model and every ring stays where it was. That’s what lets this volume measure each ring on its own.
  • The model sits in the middle. Someone else built it.
  • Each ring around it is one layer of the harness, and you build all of them.
  • Swap in a different model and the rings stay the same.

Why the harness is where to work

Improving the prompt is the obvious fix, and it runs out. A better prompt can’t:

  • create a tool that doesn’t exist;
  • tell the model which test failed when the error message didn’t say;
  • stop a model that has already announced success and is wrong.

The harness can do all three.

The experiment this makes possible

Because the harness is code, you can keep the model the same and change the harness. That’s the only fair comparison here. If you change the prompt and the model together, you can’t tell which one made the difference.

Every figure in this volume comes from that experiment: one scripted model that always behaves the same way, six layers you can switch on and off, and the same task run again with one layer changed.

The scripted model is what keeps the comparison fair, and also what limits it. Nothing here predicts exactly how a real model behaves with these layers. The labs can run a real model too, and step 12 lists this limit first.

The six layers

layerwhat it decidesstep
tool designwhat actions the model has, and how hard they are to get wrong3
clear errorswhat the model learns from a failure4
a checkerwhether you believe the model when it says it’s done5
compactionwhat survives when the room runs out6
stuck detectionwhat happens when the same thing keeps happening7
a full recordwhether you can see what happened afterward7

Part 2 takes them one at a time. What each one is worth is measured, not argued, and the labs in Part 3 compare them across three tasks.

Try it. In the labs folder, run:
python3 lab.py fix-a-test

Part 1 runs the bare loop with no layers at all. Part 2 adds the six layers one at a time, so you can see which ones change the result.

What to remember

  • The harness is the part of an agent you wrote, and it’s where you can still make gains once prompting runs out.
  • Keep the model the same and change the harness. It’s the only fair comparison.
  • Six layers, and none of them is about wording the prompt.