Giving a Model Your Own Data
Find the right passages, then answer from them
Scroll
Find the right passages, then answer from them
Retrieval-augmented generation, or RAG, is the name for chunking, indexing and search working together with a model. It has two halves, and they run at different times.
Ahead of time, you split your documents into chunks, turn each chunk into an embedding, and store the embeddings in an index, with a keyword index alongside. At question time, the system searches both with the question, picks the best few chunks, and puts them in the prompt next to the question with an instruction: answer from these passages, and say which one you used.
Each step can fail on its own. Bad chunking cuts the answer in half, weak search never finds it, too many chunks bury it, and a model with no instruction to stay within the passages will fill gaps from memory. So when a RAG system gets something wrong, look at what it retrieved before blaming the model. Often the failure happened before the model saw anything.
Retrieval