Track token usage, model choice, caching, batching, and routing — cost mainly depends on model size, input tokens, output tokens, request volume, tool calls, and concurrency, so most cost levers are really token and request levers.

Cost-aware systems use cheaper models for simple tasks (classification, extraction, routing, formatting) and reserve stronger models for hard cases via a router. Beyond model choice: reduce prompt size by dropping unnecessary instructions, summarizing history, retrieving only relevant chunks, and avoiding duplicate context; limit output length with a max-token cap and a request for conciseness; cache frequent answers, embeddings, retrieval results, system prompts, and tool outputs so repeated work isn't repaid every time; batch requests where possible for higher throughput per dollar; optimize the RAG layer itself with better chunking, reranking, and metadata filters so less irrelevant context gets sent to the model at all; and cap agent loops with a max-steps limit, since an agent that calls tools repeatedly without a bound can quietly rack up cost far faster than a single generation call.