Retrieval at Depth
When the answer is split across documents
Scroll
When the answer is split across documents
Some questions cannot be answered from any single passage. Ask who manages the payments service, and the documents may say only that payments is owned by the Atlas team and, somewhere else, that Atlas reports to Priya. The second passage looks nothing like the question, so a similarity search on the question alone rarely finds it.
There are two common fixes. Iterative retrieval treats it as a loop: retrieve, read, notice the missing piece, write a follow-up query such as “who manages Atlas”, and retrieve again. It needs nothing built in advance, but every hop adds a model call and another chance to go wrong. A knowledge graph does the joining before any question arrives. A model reads each document once and records each fact it states as a labeled link between two named things: payments → owned by → Atlas from one document, Atlas → reports to → Priya from another. When the question comes in, the system starts at payments, takes its “owned by” link to Atlas, then takes Atlas’s “reports to” link to Priya. The second document is reached through the link, so it no longer has to match the wording of the question. Graphs can also answer broad questions, such as the main themes across a whole set of documents, by grouping closely linked entities and summarizing each group.
A graph has costs of its own. Extraction takes a model call per chunk, its mistakes become wrong edges that look authoritative, and the graph goes stale as documents change. Before building one, check how often your failed questions really need two hops; many turn out to be ordinary recall problems that better chunking or hybrid search would fix. If they do need hops, start with iterative retrieval and a hop limit, and add a graph once the same relations keep being looked up.
Retrieval