Giving a Model Your Own Data

Vector Index

Close enough, much faster

Scroll

Close enough, much faster

Once your text is a million points, finding the nearest one by comparing all of them is too slow. A vector index builds a navigable structure — usually a layered graph — and walks it instead.

Search drops from checking everything to a handful of hops, which is the difference between a demo on a laptop and a service.

The word people skim past is approximate. The index can miss a true nearest neighbor, and how often it does is a dial you set against speed and memory. If your retrieval quality is mysteriously capped, this parameter is worth looking at before the embedding model.

Retrieval
CLOSE ENOUGH, MUCH FASTER L0 · every vector L1 · a sample L2 · few, long hops query 2 4 8 12 14 16 comparisons exact scan: 24 A million vectors is too many to compare one at a time, so the index builds a graph of shortcuts and walks it instead. The word approximate is doing real work: it can miss a true nearest neighbor. Recall is a dial you set against speed and memory, not a given.
A search hopping down three stacked layers of a graph to the answer, against an exact scan of every point.