Architecture Comparison: Workflow vs Single-Agent vs Multi-Agent¶

Date: 2026-03-26 Dataset: Same 30 test cases from baseline evaluation Models: gpt-4o (temperature 0.0) for all three architectures Rubric: Default (correctness 0.4, grounded 0.3, completeness 0.3), threshold 0.7

Summary¶

The Tradeoff¶

Multi-agent improves pass rate by only 3.4 percentage points over single-agent, but costs 2.4x more and takes 2.2x longer. The workflow is cheapest and fastest but misses nuanced questions. For this task -- document question-answering with citation requirements -- single-agent is the sweet spot. It captures the major accuracy gains from being able to refine queries and re-retrieve, without the cost overhead of routing queries through a verifier that mostly confirms what the primary agent already got right.

The data makes this clear: multi-agent's accuracy advantage comes entirely from the comparison and design_reasoning categories. On every other category, it matches single-agent at 2.4x the cost. Unless your query distribution is dominated by cross-document synthesis questions, multi-agent is not worth the overhead.

Where Each Architecture Wins¶

Per-Category Breakdown¶

Cost Breakdown¶

Workflow (1 model call per query)¶

Single Agent (avg 2.8 model calls per query)¶

Multi-Agent (avg 4.6 model calls per query)¶

Latency Distribution¶

The multi-agent P95 is 7.2x the workflow P95. For a user-facing application with a 3-second SLA, multi-agent is not viable without caching or pre-computation. Single-agent fits within a 4-second SLA. Workflow fits comfortably within any reasonable SLA.

Verdict¶

For the Document Intelligence Agent task:

• Use a workflow for simple, single-source questions (60% of real queries). These are lookup queries with clear vocabulary overlap. The workflow handles them at 1/3 the latency and 1/3 the cost of the single agent, with no accuracy penalty.

• Use a single agent for multi-hop or refinement-needed queries (30%). These are technical detail and error handling queries where the first retrieval might miss. The agent's ability to refine its query and re-retrieve justifies the 2.6x cost increase over the workflow.

• Use multi-agent only for high-stakes queries where verification justifies the 2.4x cost premium over single-agent (10%). Comparison and design reasoning queries benefit measurably from a verifier. Everything else does not.

• The hybrid approach (workflow default, agent escalation) outperforms any single architecture. Route simple queries through the workflow. Escalate to the single agent when the workflow's confidence is low. Escalate to multi-agent only for explicitly flagged high-value queries. This hybrid routing reduces average cost by 40% compared to running every query through the single agent, with no reduction in pass rate.

What This Comparison Does Not Show¶

This comparison holds the model constant (gpt-4o for all architectures). In practice, the workflow could use a cheaper model (gpt-4o-mini) for simple queries, reducing the cost gap further. The single agent could route its refinement calls through a cheaper model. These model-routing optimizations are covered in Chapter 6's cost management section but are not reflected in these numbers.

The comparison also holds the dataset constant. In production, the query distribution matters enormously. If 90% of your queries are simple lookups, the workflow is the clear winner. If 50% of your queries require cross-document synthesis, multi-agent starts to justify its cost. Know your query distribution before choosing an architecture.