Where a guardrail sits matters more than how good it is
Animated steps and three labs you can run. Six kinds of check, three streams of traffic, six arrangements of the same checks, and a bill for each one: what it caught, what it missed, who it refused, and what work it threw away.
- All 12 steps, read online
- Runnable labs to download (plain Python, runs offline)
- Free updates whenever it's revised
- 14-day refund (conditions)
Four questions this volume answers
Guardrail advice is a list of things to install, but where a check sits decides what it's worth. The same six kinds of check, in four different positions, cost up to 13.7 times as much on the same traffic, and catch exactly the same requests. Catch rate alone misleads: it's highest when you refuse everything, and every wrong refusal lands on a real person.
Check that your guardrail can actually refuse. Write down what an incident costs you against a wrongly refused user. Find which kinds of harm your traffic carries, and the first stage where each one shows. A check can't run before its evidence exists (a grounding check needs a claim and a source), so put each check at the stage where its harm first shows, not earlier. If the harm is crude and visible in the request, check at the door and stop.
Score your checks against an answer key the system can't see, never against their own verdicts, and report the catch rate next to the share of harm that could have been caught at all. On traffic phrased to get past it, the door check caught no one and refused 49 legitimate users, costing more than no guardrail. When the harm is in the answers to harmless requests, the input filter is asking about the wrong thing.
What an incident costs against a refusal decides whether to have guardrails; the traffic decides where to put them. From one-to-one up to 256 to one, the cheapest arrangement never changed. Traffic changes too: attackers learn what the door checks for. Watch the catch rate, tell traffic that has moved from a check that has got worse, and move the checks when it does.
Introduction
You’re about to put guardrails on an agent, and the advice you’ll find is a list of things to install: a safety filter, a policy check, a grounding step, a review queue.
The list isn’t the hard part. This volume runs the same six kinds of check over three streams of traffic, in six arrangements, and gets three different winners. Four of those arrangements hold the identical six kinds of check and differ only in where they sit.
The whole thing, before any of the argument
Every box below is something you can install. The rest of this volume is about which of them your own traffic actually needs.
- A request enters at the left and drops through the six kinds of check.
- It crosses the five stages of the work, while data feeds in and records leave.
- The answer returns up through the same six. The band beneath lights last and points back into the work.
What a guardrail is
A guardrail is a check that can refuse.
Not a policy document, a system prompt, a warning in the output, or a dashboard somebody looks at on Tuesday. It runs while a request is in flight, and it can stop it. Everything hard about the subject comes from taking that seriously:
| because it… | the consequence |
|---|---|
| decides now | it runs while the request is in flight. What it knows then is all it will ever know |
| decides from what it can see | a check at the door sees a request. What the agent will do with it hasn’t happened yet |
| refuses real people | stopping a harmful request and refusing a legitimate user are the same event, seen from two sides |
The six arrangements
The results below compare six arrangements of checks. The last four of them hold all six kinds of check, and differ only in where they sit.
| arrangement | what it is | checks placed |
|---|---|---|
| none | no guardrails at all. The bill every other arrangement has to beat | 0 |
| door | every check that can run on the request alone, at the door | 2 |
| earliest | each kind of check at the first stage it can run | 6 |
| acting | each kind of check once the agent is acting, at execution (grounding waits for evaluation) | 6 |
| exit | every check on the answer, at the response stage | 6 |
| everywhere | every kind of check at every stage it can run | 22 |
The main result, up front
| stream | what it is | cheapest arrangement | cost | how far apart the same checks can cost |
|---|---|---|---|---|
| ordinary | mostly legitimate traffic, and the harm in it is crude | door | $254.80 | 2.2x |
| adversarial | the same harms, phrased to get past a check at the door | acting | $515.48 | 13.7x |
| leaky | benign requests whose answers leak or invent | exit | $449.90 | 9.6x |
Why position decides it
A check can’t run before the evidence it needs exists. A grounding check at the door isn’t badly tuned; there’s no claim yet. That one rule empties most of the design space and sets up the trade the whole volume turns on: early is cheap and blind, late is informed and expensive.
| kind of check | earliest stage it can run | why not before |
|---|---|---|
| safety | input | harmful phrasing is in the request itself |
| policy | input | so is regulated content the user sent |
| tool | planning | nothing to permit until a tool has been chosen |
| data | execution | nothing to check until a source has been touched |
| behavior | execution | drift is only visible once the agent is acting |
| grounding | evaluation | a claim and a source must both exist to compare them |
The four kinds of money
Three are usually left off the comparison, and one of those three is paid by people who did nothing wrong.
| what costs money | how much | who pays it |
|---|---|---|
| running one check on one request | $0.002 | you, on every request, whether or not anything fires |
| one harmful request that got through | $40.00 | you, once, when it reaches the world |
| one legitimate request wrongly refused | $2.50 | a real person who wanted something reasonable |
| a stage of finished work thrown away | $0.40 | you, every time a check fires late |
Is this for you?
| if… | then |
|---|---|
| you’re deciding which guardrail product to buy | step 10: the same checks cost up to 13.7x as much, depending only on where they sit |
| your input filter isn’t catching what you expected | step 8. There are two different reasons, and they need opposite fixes |
| your requests look fine and your answers don’t | step 9. No improvement to the input filter can help |
| somebody wants every check turned on everywhere | step 7, then step 10 |
| you’re arguing about how much an incident is worth | step 12, which measures how little that argument decides |
The three streams of traffic
Twelve hundred requests each. They’re made-up streams, built to disagree, not a survey of real traffic.
| stream | harmful requests | of those, invisible at every stage | what it teaches |
|---|---|---|---|
| ordinary | 88 | 2 | the cheapest check at the door catches everything catchable |
| adversarial | 232 | 5 | the door catches nothing, and costs more than nothing |
| leaky | 172 | 3 | nothing is visible until the answer exists |
What you’ll get
- A way to price an arrangement of guardrails before you build it.
- Why the same checks cost very different amounts depending on where they sit.
- A short procedure that ends early on crude traffic, and the floor every arrangement has to clear.
The labs: what they are
One small Python program you download and run yourself, on each of the three streams. Once you own the volume, the Download the labs button at the top of this page gives you the zip.
What you do in the labs
- Run a stream with no guardrails, then with checks at the door.
- Compare all six arrangements, and read each bill line by line.
- Turn the dial: what one incident is worth against one refused user.
- Price an arrangement of your own with
--set.
Two ways to run them
| way | who places the checks | what it costs | the results |
|---|---|---|---|
| scripted | the six fixed arrangements | free, and works offline | the same every run. These pages quote them. |
| live | a real model, from the evidence a person deploying guardrails would have | about a cent: one question | the model’s placement, priced against the six |
Before a live run
- Run the setup script in the labs folder.
- Put one model connection in
labs/.env. For Claude Code you need no key. - Check the connection with one tiny call. It costs well under a cent.
- Start the run. It shows the cost and waits for you to type yes. A spending cap stops it if costs go over, and every answer is saved.
How to read the labels
| label | means |
|---|---|
| concept | explained from first principles; no lab numbers |
| measured | a lab compares arrangements and reports the difference |
| lab | a guided lab you run yourself, step by step |
| your own text | how to use the method on your own work |