Retrieval Engineering Concepts › What retrieval is for, and when you don't need it
01 · part 1, conceptsconcept

What retrieval is for, and when you don't need it

Retrieval engineering is deciding what text reaches the model for each question: how documents are split, how they’re searched, how much is handed over, and when to hand over nothing at all.

Why the decision exists at all

A model knows what it was trained on and what you put in front of it. Your documents are neither, unless your program finds the relevant part and adds it to the request. Retrieval is that finding.

Four ways to get facts to a model

the optionwhat it’s good athow it fails
put it all in the promptsmall, stable material every question might needit stops fitting, and the overflow is dropped without an error
plain search, a person readsrare questions, where a person’s time is cheaper than building anythingnobody has time to read the results
retrieval, then the modelfacts that change, or that the answer must citethe right text isn’t found, and the model answers anyway
change the model’s weightsstyle, format and skillfacts learned this way are out of date when they change, and can’t be cited
put it all in the prompt works until it does not fit breaks when the facts stop fitting plain search, a person reads works when a person has time breaks when nobody has time to read retrieval, then the model works when the facts change or must be cited breaks when the right text is not found change the model's weights works for style and skill, not for facts breaks when the facts change next week your facts Retrieval is the only one of the four that is about finding text. So it is the only one that fails by finding the wrong text, which is the subject of this volume.
Four ways to get your facts to a model, and when each breaks. Read it left to right.
Nothing in this drawing is measured. It’s the choice before any of the measurements apply.
  • Each option appears with the condition under which it breaks.
  • Only retrieval fails by finding the wrong text, which is what the rest of this volume measures.

When you need retrieval, and when you don’t

the conditionwhy it points at retrieval
your facts change faster than you could retrain or rewrite a promptretrieval reads the current document every time
the answer has to say where it came fromretrieval hands over the passage, so the answer can point at it

Either one is enough. If neither holds, there’s a cheaper answer: a small, stable body of text goes straight into the prompt, rare questions go to plain search and a person, and skill and tone are what changing a model’s weights is for.

Try it. In the labs folder, run:
python3 run.py --lab handbook

Every split and search on the handbook: how many answers each one hands over.