Optimize retrieval, prompt size, model choice, caching, and streaming.

Latency comes from network calls, retrieval, reranking, model generation, and tool use. Smaller models and cached context can help.