Retrieval at Depth

Multi-hop and graph retrieval

When the answer is split across documents

Scroll

When the answer is split across documents

Some questions cannot be answered from any single passage. Ask who manages the payments service, and the documents may say only that payments is owned by the Atlas team and, somewhere else, that Atlas reports to Priya. The second passage looks nothing like the question, so a similarity search on the question alone rarely finds it.

There are two common fixes. Iterative retrieval treats it as a loop: retrieve, read, notice the missing piece, write a follow-up query such as “who manages Atlas”, and retrieve again. It needs nothing built in advance, but every hop adds a model call and another chance to go wrong. A knowledge graph does the joining before any question arrives. A model reads each document once and records each fact it states as a labeled link between two named things: payments → owned by → Atlas from one document, Atlas → reports to → Priya from another. When the question comes in, the system starts at payments, takes its “owned by” link to Atlas, then takes Atlas’s “reports to” link to Priya. The second document is reached through the link, so it no longer has to match the wording of the question. Graphs can also answer broad questions, such as the main themes across a whole set of documents, by grouping closely linked entities and summarizing each group.

A graph has costs of its own. Extraction takes a model call per chunk, its mistakes become wrong edges that look authoritative, and the graph goes stale as documents change. Before building one, check how often your failed questions really need two hops; many turn out to be ordinary recall problems that better chunking or hybrid search would fix. If they do need hops, start with iterative retrieval and a hop limit, and add a graph once the same relations keep being looked up.

Retrieval
TWO HOPS TO THE ANSWER Who manages the payments service? owned by doc 12 · hop 1 reports to doc 87 · hop 2 payments Atlas Priya billing Orion Sam A search on the question finds hop 1. Hop 2 never mentions payments, so it takes a follow-up query, or an edge stored ahead of time.
A question that needs two documents, joined through the entity they share.