The Judgment Gap how the gap between producing and checking opens › The ratio that tells you how wide your gap is
03 — part 1, how the gap between producing and checking opensreasoning

The ratio that tells you how wide your gap is

One number tells you where you are, you can work it out from records you already keep, and the hard part is deciding what counts.

One number. Work produced in a period, divided by the work somebody actually looked at. Compute it before reading the rest, because every page after this one is easier to read against your own figure than against an argument.

work produced work examined the division is easy. The denominator is the whole problem counts as examined? somebody read it who could have caught a problem yes it passed through a review step no a check ran on it no nobody complained no produced, over examined one number, from records you keep the comfortable reading of the easy reading of examined number that means nothing.
The division is easy. The denominator is the whole problem, because the comfortable reading of *examined* gives you a number that means nothing.
What moves: the two pipes feed a filter labeled counts as examined, which accepts one candidate test and rejects three.

What counts as examined

This is the only difficult part and the whole value is in getting it right. Examined means somebody looked at it who had enough context to have caught a problem. It does not mean a review step was traversed, a check ran, or nobody objected.

does it count?why
somebody read it who could have caught a problemthis is the definition. Note the could have — not did
it passed through a review stepa step is a place, not an event. Ask whether anybody looked
an automated check ran on ituseful, and it is not judgment. A check tests what somebody thought of before
nobody complainedstep four is entirely about why this is not evidence

Take the strict reading. You will get an uncomfortable number, and an uncomfortable number you can act on is worth more than a comfortable one that is measuring review steps traversed.

With a few hundred pieces of work in a quarter and a strict count of what was genuinely read, most teams find something like three or four produced for every one examinedmodeled: an illustration only — substitute your own two counts. The number itself matters less than whether it is moving, which is why the instruction is to compute it and keep it.

→ Compute yours, for one quarter, before reading on. Two counts and a division, from records you already keep. Write it down with the date, because the second reading is worth more than the first and you will want to have taken the first.