Cost and Choices

Choosing a model

Match the model to the job

Scroll

Match the model to the job

Most providers offer a family of models in several sizes. The larger ones handle harder reasoning, longer documents and subtler instructions. The smaller ones answer faster and cost a fraction as much per token.

Choose by the task in front of you. Sorting tickets, pulling fields out of a form or rewriting a sentence are jobs a small model usually does well. Multi-step reasoning, careful writing and code that has to be right tend to need a larger one. To find out, try the smallest plausible model on twenty real examples from your own work and check whether its answers are good enough. The golden dataset page explains how to build that set.

Speed is part of the choice too. A smaller model replies sooner, and for anything a person waits on, that can matter as much as quality. Many teams end up using two models: a small one for the bulk of routine requests and a large one for the hard cases. The page on model routing covers how to decide which request goes where.

Cost
THE CHEAPEST MODEL THAT CLEARS THE BAR quality ↑ cost and wait per request → small medium large sort tickets small is enough summarize a contract only large clears it Set the bar with real examples, then pick the cheapest model above it. Many apps use two: a small model for routine requests and a large one for hard ones.
Three model sizes and two tasks. Each task has its own bar, and a different model clears it.