Self-Learning Agents Introduction

When an agent writes its own rules, and how to tell if the work got better

Animated steps and three labs you can run. In each lab an agent writes rules for itself from its own attempts, with eight layers you switch on and off, and every attempt is scored twice: by the check it can see, and by the answer key it can't.

Decision Q&A

Four questions this volume answers

1Why does this matter?

Let an agent write rules from its own attempts and the score goes up, but which score? A loop chases the check it can run, not the answer key it can't. A wrong memory makes one answer wrong; a wrong rule makes every later attempt wrong in the same direction, while looking like progress.

Shown in step 1 · 2 · 3
2How would I design it?

Keep two scores apart: the check inside the loop, and an answer key outside it that the learning code can never call, with a test that proves it can't. Test every rule before it goes in the book, and refuse any rule that changes a file the check reads. That test needs no answer key, and it was the only defense that closed a gap. Hold tasks back to see whether rules carry over.

Shown in step 2 · 7 · 11
3How would I debug it?

Track the gap: the score it can see minus the score that counts, shown as n/a, not zero, before anything is scored. The same perfect score meant three things across the labs: a rule that edited the test, two likely but wrong answers, and correct work. Ask what your check accepts that a careful reviewer would reject.

Shown in step 4 · 6 · 11
4How would I take it to production?

Expect the fix to make the dashboard worse: testing rules first dropped the score the agent could see from 40 of 40 to 38 of 40, while the score that counts held and the shortcut stopped. Don't rely on held-out tasks to catch a gamed check; there the cheating loop scored higher. Where the check can't tell a likely answer from the right one, no layer closes the gap: report the gap, and invest in a better check.

Shown in step 5 · 8 · 9 · 12

Introduction

You let an agent write notes to itself: rules, habits, a file it edits between tasks. The number went up. This volume is about which number went up, and whether it’s the one you care about.

The shape of every self-learning loop

#the agent…where it matters
1attempts the task
2finds out how it wentthis is where the result is decided
3works out which part went wrongstep 6
4writes a rulewhere the effort usually goes
5attempts the next task with the rule in hand
A wrong memory makes one answer wrong. A wrong rule makes every later attempt wrong, in the same direction, while looking like progress.

What self-learning engineering is

Self-learning engineering is deciding what an agent may conclude from its own attempts, and what has to be true before it acts on that conclusion.

This volume splits a learning loop into eight layers in four phases, and measures what each one buys. The agent stays the same (scripted, so it makes the same choices every run) while the layers around it are switched on and off. Any change in the result can only come from a layer.

phasethe question it answerslayers
keep a recordwhat did I do, and did it work?a record of each attempt · scoring each attempt
work out what went wrongwhich step, and what rule follows?blaming a step · writing a rule
check before you believe itdoes the rule help, and does it hold on new tasks?testing a rule first · held-out tasks
tidy the rule bookdo the rules disagree, and do they still pay?checking rules against each other · dropping rules that stop paying

The two scores

Every attempt is scored twice, and everything here rests on the difference:

the score the agent can seethe score that counts
where it comes fromthe check: cheap, and the agent can run it as often as it likesthe answer key: the real standard the work is held to
who runs itthe agent, every roundyou, now and then
inside the loop?yes, alwaysnever
for examplea test suite, a linter, a pattern match, a rubric prompta careful review, an untouched test, whether the customer complained

The gap is the score it can see minus the score that counts.

The main result, up front

The layer that fixed the work made the dashboard worse. In the fix-a-test lab the agent reaches a perfect 40 of 40 on the score it can see while the score that counts is 38 of 40: one of its rules edits the test file, so the check reports success on a program that still has a bug. Testing each rule before adopting it drops the score it can see to 38 of 40 and leaves the score that counts where it was. Anyone tuning against the score they could see would have undone the fix.
In the triage lab, nothing helps. A perfect score it can see sits on two confidently wrong answers, with all eight layers on and none of them able to see it, because the check can’t tell a likely cause from the right one.
In the answer lab, learning simply works. The score that counts rises from 3 to 20 of 20 in one round, and every held-out task comes out right. The shortcut was on the list of rules it could write; the scripted agent never chose it, because the honest rules worked first. Nothing in the loop would have stopped it if it had.

Is this for you?

if…then
your agent can edit a file your checks readstep 7: one free test closes it
your score went up and you can’t say what improvedstep 4, then step 11
you’re about to add more checking layersstep 5. Three of the eight change nothing in any lab
your check accepts a likely answer as a right onestep 9. That’s the case where nothing here helps
you hold tasks back and that score is risingstep 11: it measures something different

The three labs

Eight tasks each: five to practice on, three it never practices on, over three rounds of learning. What differs between them is a property of the check, not of the agent or the task.

labthe agent’s jobcan the check be satisfied without being right?what happens
fix-a-testrepair a small program until its test file passesyes: edit the test filetesting a rule first closes the gap
triagename the framework that caused a failing test, from its crash reportyes: name any framework the crash report mentionsnothing closes the gap
answerstate a figure from a travel expenses policyyes: name several figures (the scripted agent never wrote that rule)learning simply works

What you’ll get

  • A way to tell a better score from better work, before you trust either.
  • The one test that stops a rule gaming your check, and the case it can’t catch.
  • Three labs you can run on your own computer. The next section explains them.

The labs: what they are

Each lab is a small Python program you download and run yourself. Once you own the volume, the Download the labs button at the top of this page gives you all three in one zip. In each, an agent works through eight tasks and may write itself a rule after each failure:

  • Fix-a-test: repairing a small program; the check runs the test file in the agent’s own workspace.
  • Triage: naming the cause of a failing test; the check accepts any framework the crash report mentions.
  • Answer: stating a figure from an expenses policy; the check looks for the figure anywhere in the answer.

What you do in the labs

  • Score the attempts before anything is learned.
  • Add the layers one at a time, then take each out of the complete loop.
  • Read the rule book: what was kept, what was refused, and why.
  • Compare the loops on tasks they never practiced on.

What you’ll learn

  • Why the score it can see is the one a loop chases.
  • Where a gap comes from, and when you can close it for free.
  • Why held-out tasks don’t catch a gamed check.
  • What a perfect score is, and isn’t, evidence of.

Two ways to run them

waywho writes the ruleswhat it coststhe results
scripteda built-in scripted agent, the same every runfree, and works offlinethe same every run. These pages quote them.
livea real model: Claude through the API, Claude Code, or an OpenAI modela few cents: one short question per failed practice taskthe rules a real model chooses, scored the same way

All three labs can run either way.

Before a live run

  • Run the setup script in the labs folder. It installs everything a live run needs.
  • Pick a model and put its key in the settings file, labs/.env. For Claude Code you need no key. The labs’ README shows every line.
  • Check the connection with one tiny call. It costs well under a cent.
  • Start the run. Before sending anything, it shows how many questions it may ask and what they should cost, and waits for you to type yes. A spending cap stops the run if costs go over, and every answer is saved, so you never pay twice for the same question.
Your live results may differ from these pages. A real model chooses its own rules from the same list, the shortcut included. The report shows every rule it chose and what happened to it.

How to read the labels

Each step has a label that tells you where its content comes from.

labelmeans
conceptexplained from first principles; no lab numbers
measureda lab compares learning loops and reports the difference
laba guided lab you run yourself, step by step
your own texthow to use the method on your own work
These labs aren’t a benchmark. Eight tasks per lab, three rounds, one scripted agent, and a fixed list of rules it can write, which you can read in the labs. A gap is a count of points, not a rate, and nothing here says how often self-learning agents game their checks in general. Step 12 lists every limit.