How do you handle token budget management across a long multi-agent conversation?hard
Answer
Track token usage cumulatively. When approaching limit: (1) Summarize older turns, replace with summary (rolling context compression).
Explanation
Track token usage cumulatively. When approaching limit: (1) Summarize older turns, replace with summary (rolling context compression). (2) Evict least-relevant messages by importance scoring. (3) Move completed context to external memory (DynamoDB), fetch back when needed. (4) Use Bedrock prompt caching to avoid re-processing stable system prompt on every turn. Hard limits per agent via RunConfig.
Follow-upCan you give a production example?