Chunking Strategies
How documents are split into retrievable units — the most impactful parameter for RAG quality.
Chunking is the process of splitting documents into smaller units that can be individually embedded and retrieved. It is the single most impactful parameter for RAG quality: chunk too large and the context window fills with irrelevant text; chunk too small and each chunk loses the context needed to be meaningful.
Semantic Chunking
The Knowledge Hub uses semantic chunking — it respects document structure (headings, paragraphs, lists, tables) and splits at natural boundaries. Each chunk is self-contained: it carries enough context to be meaningful on its own. A paragraph about 'fire safety' is not split mid-sentence; a table is kept intact rather than split by rows.
Chunk Size Trade-offs
- Small chunks (100-200 tokens) — precise retrieval, but each chunk may lack context. The model sees the right passage but doesn't understand its surrounding context.
- Medium chunks (300-500 tokens) — balanced. Most paragraphs and short sections fit in this range. This is the default for most use cases.
- Large chunks (800+ tokens) — rich context, but the context window fills quickly and retrieval precision drops (the chunk contains too many topics).
Chunk size is not a universal constant. The optimal size depends on the document type (dense technical docs vs sparse web pages), the query type (precise lookups vs exploratory questions) and the model's context window. The Knowledge Hub allows per-document-type chunking configuration.
FAQ
Frequently Asked Questions
Yes. Overlapping chunks (e.g., 50 tokens of overlap between adjacent chunks) ensures that information at chunk boundaries is not lost. The Knowledge Hub supports configurable overlap as part of the chunking strategy.
Tables are kept intact as single chunks when possible, or split by rows with header context preserved. Code blocks are kept intact. The chunking strategy respects the semantic structure of the document, not just token count.
Related Topics