How do you test a multi-agent pipeline end-to-end?hard
Answer
(1) Unit tests: each agent in isolation with mocked tool responses and deterministic LLM outputs.
Explanation
(1) Unit tests: each agent in isolation with mocked tool responses and deterministic LLM outputs. (2) Integration tests: full pipeline with real LLM calls on golden dataset. (3) Contract tests: verify each agent's input/output schema stays stable (Pydantic). (4) Load tests: N parallel executions, verify Step Functions handles concurrency. (5) Chaos tests: inject failures (tool timeout, LLM error), verify retry/fallback logic. Gate deploys on all passing.
Follow-upCan you give a production example?