How AI Works

Training vs inference

Expensive once, cheap per request

Scroll

Expensive once, cheap per request

A model's life has two phases. Training is when its weights are set: the model is shown vast amounts of data and adjusted until its predictions are good. Inference is everything after: the frozen model being used to answer requests.

The costs are lopsided. Training a frontier model occupies large clusters of accelerators for weeks and happens once per model version. Inference costs a tiny fraction of that per request — but it happens on every request, from every user, for as long as the model is in service, so it is where the ongoing spend goes.

The split explains a common misunderstanding. A conversation does not train the model: your messages are input to inference, and the weights do not change. When an assistant seems to remember you, something outside the model stored a note and put it back into the input. And a model knows nothing after its training cutoff unless someone puts it in the prompt — the problem retrieval solves.

Foundations
EXPENSIVE ONCE, CHEAP EVERY TIME training · once text corpus web, books, code compute weeks, many GPUs very large cost, paid once per model weights, now frozen inference · every request prompt model reply prompt model reply prompt model reply the weights do not change here Training sets the weights once, at great cost. Inference reuses those frozen weights on every request. Your conversation is input, not training.
One large training run produces frozen weights; every request after that reuses them.