Split documents for AI without cutting the answer away from its conditions
Keep headings, clauses and table context together, then test whether a retrieved chunk can answer the question on its own.
In brief
Document chunking divides text into retrievable units. The goal is not merely equal lengths: a useful chunk keeps a rule with its conditions and the context needed to interpret it.
This matters when a search finds “allowed” but omits the nearby sentence describing who is eligible.
Preserve meaningful boundaries
Keep a short definition or numbered procedure together when possible. For tables, include the relevant column headings with the rows. A row containing only “30 days” is ambiguous without knowing which product and event it refers to.
Add context deliberately
Carry the document title and section heading as metadata or clearly identified context. Keep edition and page references so results from different versions are distinguishable. Avoid repeating the entire document introduction in every chunk if it overwhelms the useful text.
Tune from observed misses
If a question repeatedly retrieves the rule without its exception, change boundaries or retrieval context and retest that question. If results are mostly duplicates, reduce unnecessary overlap. Compare on your own representative questions instead of copying a token count from an unrelated corpus.
Before you act
There is no universally best chunk size or overlap. Excessive overlap adds duplicate material and cost, while oversized chunks can dilute the relevant passage.
What to do next
- Inspect the document’s headings, paragraph boundaries and tables before splitting.
- Keep source, section and page/version metadata with each unit.
- Test real questions and inspect returned chunks for missing conditions or duplicated noise.
