How do you choose chunk size?medium
Answer
Too small: lacks context, generation suffers, retrieval misses multi-sentence concepts.
Explanation
Too small: lacks context, generation suffers, retrieval misses multi-sentence concepts. Too large: retrieval is imprecise (chunk covers many topics), more noise in LLM context. Typical range: 256-512 tokens with 10-15% overlap. Tune empirically using context precision/recall metrics. For structured documents: align chunk boundaries to logical units (clauses, sections) rather than token count.
Follow-upCan you give a production example?