Should This Be AI? Concepts › The question before the model
01 · part 1, conceptsconcept

The question before the model

“Can a model do this?” is almost always yes, and it is the wrong question. The question before the model is what each request needs, and what it costs when the answer is wrong.

where each request should go “where is order 48213?” the answer is stored: look it up “reset my password” a short rule is sure: apply it “the thing I bought arrived kind of weird?” open-ended, low stakes: a model “can I carry leave over if I was on sick leave in March?” open-ended, costly if wrong, unsure: a person Only one of the four is a model's job.
Four requests, four destinations. A model is the right tool for one of them.
Nothing in this scene is measured.

Four questions to ask first

askif yes
1is the answer already stored somewhere: a database, a ledger, a calendar?look it up; a model can only add wrong answers
2would a short rule get most requests right, and know when it does not?use the rule, and send the rest on
3is the input open-ended, so that a person would otherwise have to read it?a model may earn its place
4does a wrong answer cost much more than handing the request to a person?the model must be allowed to hand over

None of the four needs a model to answer. They need someone who knows the requests and the business, and an afternoon.

What a model is good at, and what it is not

a model is good ata model is poor at
reading requests written in any wordsreturning a fact it was never given
handling a long tail no rule could listbeing exactly right every time on a closed question
drafting something a person then checksknowing for sure when it is wrong
The answer is rarely all or nothing. Most real features are a mix: some requests are lookups, some fit a rule, some are open-ended. Step 5 measures what happens when they are combined, and which combinations help.
Try it. In the labs folder, run:
python3 run.py

The value a month of each design on each feature, and the decision, in one table.