01 · part 1, conceptsconcept
What retrieval is for, and when you don't need it
Retrieval engineering is deciding what text reaches the model for each question: how documents are split, how they’re searched, how much is handed over, and when to hand over nothing at all.
Why the decision exists at all
A model knows what it was trained on and what you put in front of it. Your documents are neither, unless your program finds the relevant part and adds it to the request. Retrieval is that finding.
Four ways to get facts to a model
| the option | what it’s good at | how it fails |
|---|---|---|
| put it all in the prompt | small, stable material every question might need | it stops fitting, and the overflow is dropped without an error |
| plain search, a person reads | rare questions, where a person’s time is cheaper than building anything | nobody has time to read the results |
| retrieval, then the model | facts that change, or that the answer must cite | the right text isn’t found, and the model answers anyway |
| change the model’s weights | style, format and skill | facts learned this way are out of date when they change, and can’t be cited |
Nothing in this drawing is measured. It’s the choice before any of the measurements apply.
- Each option appears with the condition under which it breaks.
- Only retrieval fails by finding the wrong text, which is what the rest of this volume measures.
When you need retrieval, and when you don’t
| the condition | why it points at retrieval |
|---|---|
| your facts change faster than you could retrain or rewrite a prompt | retrieval reads the current document every time |
| the answer has to say where it came from | retrieval hands over the passage, so the answer can point at it |
Either one is enough. If neither holds, there’s a cheaper answer: a small, stable body of text goes straight into the prompt, rare questions go to plain search and a person, and skill and tone are what changing a model’s weights is for.
Try it. In the labs folder, run:
python3 run.py --lab handbook
Every split and search on the handbook: how many answers each one hands over.