Agents Guardrail Introduction

Where a guardrail sits matters more than how good it is

Animated steps and three labs you can run. Six kinds of check, three streams of traffic, six arrangements of the same checks, and a bill for each one: what it caught, what it missed, who it refused, and what work it threw away.

Decision Q&A

Four questions this volume answers

1Why does this matter?

Guardrail advice is a list of things to install, but where a check sits decides what it's worth. The same six kinds of check, in four different positions, cost up to 13.7 times as much on the same traffic, and catch exactly the same requests. Catch rate alone misleads: it's highest when you refuse everything, and every wrong refusal lands on a real person.

Shown in step 1 · 6 · 10
2How would I design it?

Check that your guardrail can actually refuse. Write down what an incident costs you against a wrongly refused user. Find which kinds of harm your traffic carries, and the first stage where each one shows. A check can't run before its evidence exists (a grounding check needs a claim and a source), so put each check at the stage where its harm first shows, not earlier. If the harm is crude and visible in the request, check at the door and stop.

Shown in step 3 · 11
3How would I debug it?

Score your checks against an answer key the system can't see, never against their own verdicts, and report the catch rate next to the share of harm that could have been caught at all. On traffic phrased to get past it, the door check caught no one and refused 49 legitimate users, costing more than no guardrail. When the harm is in the answers to harmless requests, the input filter is asking about the wrong thing.

Shown in step 4 · 6 · 8 · 9
4How would I take it to production?

What an incident costs against a refusal decides whether to have guardrails; the traffic decides where to put them. From one-to-one up to 256 to one, the cheapest arrangement never changed. Traffic changes too: attackers learn what the door checks for. Watch the catch rate, tell traffic that has moved from a check that has got worse, and move the checks when it does.

Shown in step 11 · 12

Introduction

You’re about to put guardrails on an agent, and the advice you’ll find is a list of things to install: a safety filter, a policy check, a grounding step, a review queue.

The list isn’t the hard part. This volume runs the same six kinds of check over three streams of traffic, in six arrangements, and gets three different winners. Four of those arrangements hold the identical six kinds of check and differ only in where they sit.

The whole thing, before any of the argument

Every box below is something you can install. The rest of this volume is about which of them your own traffic actually needs.

a request goal, query, task user or business applications and interfaces chat UI API tools enterprise apps the answer or a refusal response to user guardrails six kinds of check safety harmful content, jailbreaks, toxicity policy & compliance org policy, regulation, PII data source verification, access, leakage grounding fact-checking, citations, hallucination tool & action permissioning, parameters, human-in-the-loop behavior goal alignment, ethics, runtime monitoring the agent workflow input processing understand intent, classify planning decompose the goal, pick tools execution the model, tools, data, other agents evaluation check outputs, verify facts response generate, cite, state confidence your data and tools knowledge bases APIs, databases documents observability traces and logs violations audit trail on the way in and again on the way out and it does not stop when the answer ships monitoring real-time detection evaluation automated and human review feedback improve rules, update policy incident response alert, roll back, mitigate human-in-the-loop escalate the uncertain what it learns goes back in the core flow guardrails and controls your data and tools what you can see afterward every box in this picture is something you can install. none of them is free, and on the traffic measured here the set with every check at every stage it can run was the cheapest on nothing at all. steps 7 to 11 are how you decide which of them your own traffic needs.
A request’s whole journey, and the six kinds of check around it. Read it left to right. Click it to open it full screen.
Arrow keys or two-finger scroll to pan, pinch or the controls to zoom, Esc to close.
  • A request enters at the left and drops through the six kinds of check.
  • It crosses the five stages of the work, while data feeds in and records leave.
  • The answer returns up through the same six. The band beneath lights last and points back into the work.

What a guardrail is

A guardrail is a check that can refuse.

Not a policy document, a system prompt, a warning in the output, or a dashboard somebody looks at on Tuesday. It runs while a request is in flight, and it can stop it. Everything hard about the subject comes from taking that seriously:

because it…the consequence
decides nowit runs while the request is in flight. What it knows then is all it will ever know
decides from what it can seea check at the door sees a request. What the agent will do with it hasn’t happened yet
refuses real peoplestopping a harmful request and refusing a legitimate user are the same event, seen from two sides

The six arrangements

The results below compare six arrangements of checks. The last four of them hold all six kinds of check, and differ only in where they sit.

arrangementwhat it ischecks placed
noneno guardrails at all. The bill every other arrangement has to beat0
doorevery check that can run on the request alone, at the door2
earliesteach kind of check at the first stage it can run6
actingeach kind of check once the agent is acting, at execution (grounding waits for evaluation)6
exitevery check on the answer, at the response stage6
everywhereevery kind of check at every stage it can run22

The main result, up front

streamwhat it ischeapest arrangementcosthow far apart the same checks can cost
ordinarymostly legitimate traffic, and the harm in it is crudedoor$254.802.2x
adversarialthe same harms, phrased to get past a check at the dooracting$515.4813.7x
leakybenign requests whose answers leak or inventexit$449.909.6x
The maximalist stack never wins, and not because it catches less. Every check at every stage it’s allowed catches exactly what the winner catches, on all three streams, and costs 1.7x to 2.2x as much. Stacking bought overlap, not coverage.
On two of the three streams, the check at the door is worse than no guardrail at all. On adversarial traffic, phrased to get past it, it catches no one, refuses 49 legitimate users anyway, and costs more than buying nothing. On leaky traffic it fails for the opposite reason: the requests are fine. It’s the first thing most teams install.

Why position decides it

A check can’t run before the evidence it needs exists. A grounding check at the door isn’t badly tuned; there’s no claim yet. That one rule empties most of the design space and sets up the trade the whole volume turns on: early is cheap and blind, late is informed and expensive.

kind of checkearliest stage it can runwhy not before
safetyinputharmful phrasing is in the request itself
policyinputso is regulated content the user sent
toolplanningnothing to permit until a tool has been chosen
dataexecutionnothing to check until a source has been touched
behaviorexecutiondrift is only visible once the agent is acting
groundingevaluationa claim and a source must both exist to compare them

The four kinds of money

Three are usually left off the comparison, and one of those three is paid by people who did nothing wrong.

what costs moneyhow muchwho pays it
running one check on one request$0.002you, on every request, whether or not anything fires
one harmful request that got through$40.00you, once, when it reaches the world
one legitimate request wrongly refused$2.50a real person who wanted something reasonable
a stage of finished work thrown away$0.40you, every time a check fires late

Is this for you?

if…then
you’re deciding which guardrail product to buystep 10: the same checks cost up to 13.7x as much, depending only on where they sit
your input filter isn’t catching what you expectedstep 8. There are two different reasons, and they need opposite fixes
your requests look fine and your answers don’tstep 9. No improvement to the input filter can help
somebody wants every check turned on everywherestep 7, then step 10
you’re arguing about how much an incident is worthstep 12, which measures how little that argument decides

The three streams of traffic

Twelve hundred requests each. They’re made-up streams, built to disagree, not a survey of real traffic.

streamharmful requestsof those, invisible at every stagewhat it teaches
ordinary882the cheapest check at the door catches everything catchable
adversarial2325the door catches nothing, and costs more than nothing
leaky1723nothing is visible until the answer exists

What you’ll get

  • A way to price an arrangement of guardrails before you build it.
  • Why the same checks cost very different amounts depending on where they sit.
  • A short procedure that ends early on crude traffic, and the floor every arrangement has to clear.

The labs: what they are

One small Python program you download and run yourself, on each of the three streams. Once you own the volume, the Download the labs button at the top of this page gives you the zip.

What you do in the labs

  • Run a stream with no guardrails, then with checks at the door.
  • Compare all six arrangements, and read each bill line by line.
  • Turn the dial: what one incident is worth against one refused user.
  • Price an arrangement of your own with --set.

Two ways to run them

waywho places the checkswhat it coststhe results
scriptedthe six fixed arrangementsfree, and works offlinethe same every run. These pages quote them.
livea real model, from the evidence a person deploying guardrails would haveabout a cent: one questionthe model’s placement, priced against the six

Before a live run

  • Run the setup script in the labs folder.
  • Put one model connection in labs/.env. For Claude Code you need no key.
  • Check the connection with one tiny call. It costs well under a cent.
  • Start the run. It shows the cost and waits for you to type yes. A spending cap stops it if costs go over, and every answer is saved.
A real model may place the checks differently from run to run. The report shows its full reply, and any check it placed before its evidence exists.

How to read the labels

labelmeans
conceptexplained from first principles; no lab numbers
measureda lab compares arrangements and reports the difference
laba guided lab you run yourself, step by step
your own texthow to use the method on your own work
These labs aren’t a benchmark. Three made-up streams of traffic, one cost model, and checks treated as fixed instruments with known blind spots. How to build a check that resists evasion is a real field, and it’s not this one. The question here is where to point the checks you have.