How do chat applications handle conversations that grow past the model's context window?medium
Recent messages are kept in high detail close to raw, older messages get semantically summarized rather than dropped outright, and stable facts or preferences may be stored as long-term memory retrieved across sessions. The final context sent to the model is assembled from these pieces, not the full conversation word-for-word.
This is not exact compression like ZIP, where the original text can be perfectly restored — it's closer to semantic summarization. The system tries to preserve important meaning, decisions, user preferences, project context, and repeated facts, while dropping low-value details like typos, casual comments, and outdated intermediate decisions. For example, a long conversation about building a site called Nextforay on GitHub Pages versus Cloudflare Pages, with a private repo and specific branding preferences, might compress down to a two-sentence summary that keeps the decisions and preferences but drops the back-and-forth that led to them. That means it's lossy: exact wording, code, YAML, DNS records, or precise numbers can get lost or simplified during summarization. For technical or high-precision tasks, it's safer to re-paste the latest exact version than to rely on the model's memory of it.