Giving a Model Your Own Data

Semantic Chunking

Split documents by meaning

Scroll

Split documents by meaning

Before anything can be retrieved it has to be cut into pieces. The easy way is every N tokens, which is fast and pays no attention to what it is cutting through.

Semantic chunking cuts where the meaning changes instead. Installation stays with installation, billing with billing, and a section is not sliced down the middle.

This is the least glamorous decision in a retrieval system and one of the most consequential. A fixed cut can leave one chunk holding half a billing rule and half a security warning, so a billing question retrieves a fragment of both and answers neither well. Retrieval quality is largely decided here, before anyone runs a search.

Retrieval
CUT WHERE THE MEANING CHANGES one long document Install requirements setup first run Billing plans invoices refunds Security access encryption warnings A fixed 500-token cut lands mid-sentence and mid-topic, so one chunk holds half a billing rule and half a security warning. Retrieval quality is decided here, before anyone searches: the boundaries determine what can be found later.
One long document split by topic rather than by length, so each chunk holds one idea.