What is semantic chunking? How does it differ from fixed-size?medium
Answer
Fixed-size splits at a set token count — simple but can cut mid-concept.
Explanation
Fixed-size splits at a set token count — simple but can cut mid-concept. Semantic chunking splits at natural boundaries: sentences, paragraphs, or detected topic shifts (embedding similarity drops). Each chunk is a coherent unit. Used in a document extraction pipeline because contract clauses are variable-length and splitting mid-clause destroys extraction context.
Follow-upWhen would you choose one approach over the other?