Five Single AI Agents Is Not Microservices. It Is a Distributed Monolith

Does splitting into multiple agents solve a real problem — or does it just look more sophisticated? Every AI architecture review I attend these days has the same diagram on the whiteboard. Five boxes. Arrows pointing between them. Each box represents a specialized agent: the Orchestrator , the Data Agent , the Decision Agent , the Validation Agent , and the Response Agent . The presenter walks through the execution flow with supreme confidence. The diagram looks impressive. The room nods. Then I ask a simple question: “Why five agents?” The answer is almost always: “Well, the system is complex, and a multi-agent architecture handles complexity better.” That is not a reason to split your system into multiple agents. That is a reason to think harder about your domain boundaries. Today, we are repeating a classic architectural mistake with Large Language Models (LLMs). The industry has collectively decided that if one AI agent is good, a “multi-agent system” must be five times better. We are eagerly carving simple, sequential workloads into networks of specialized agents — the Planner , the Researcher , the Writer , and the Critic — and calling it a “microservices architecture for AI.” But let’s call this what it actually is: Architectural Theater. In our rush to build complex systems, we aren’t building microservices. We are building the AI equivalent of a Distributed Monolith — an architecture that combines the massive operational complexity of distributed systems with the tight coupling of a monolith, all while introducing non-deterministic failure modes that make traditional debugging near-impossible. Multi-agent architecture is not a default solution to complexity. It is a highly specialized tool for specific problems that a single agent cannot solve alone. Using it as your default design pattern adds immense engineering overhead without delivering equivalent value — and it introduces structural pain points across performance, cost, governance, security, and compliance. The Illusion: Why Agents Look Like Microservices It is easy to see why engineers fall into this trap. On paper, the structural analogy between AI agents and traditional microservices is incredibly seductive: You give each agent a distinct system prompt (its “domain”), arm it with specific tools (its “APIs”), and let them communicate. It feels clean. It looks like modular, decoupled software engineering. But this analogy breaks down the moment you look under the hood. What Splitting Into Multiple Agents Actually Costs When you split a single agent into five agents, you aren’t just splitting the workload. You are multiplying every inherent point of failure in agentic AI. Performance: Every single agent-to-agent call adds network latency and generation overhead. Three sequential LLM calls where one would have done the job means your end-user is sitting and waiting for no functional reason. Cost: Every LLM call costs tokens. Five agents processing the same high-level task sequentially will cost up to five times the tokens of a single consolidated agent. At enterprise scale — say, ten thousand decisions per day — unnecessary agents become a massive cost center with zero return on investment. Governance: When a decision goes wrong, tracing the accountability chain across five autonomous agents requires forensic-level investigation. In a single-agent system, the reasoning process is in one place and the audit trail is clean. Security: Every agent-to-agent interface is a potential attack surface. Risks like “agent hijacking” — where compromising an upstream agent allows a malicious actor to manipulate decisions downstream — only exist in multi-agent networks. Compliance: Explaining a decision across five separate, semi-autonomous reasoning processes is a massive compliance liability in highly regulated industries like banking or healthcare. From a Recent Retail Project: We originally built a customer service system using four distinct agents: routing , knowledge retrieval , response generation , and validation . Six weeks after launch, we scrapped it and rebuilt it as one single agent with four tools . The results? Average response times dropped from $4.2 to $1.1. Token consumption went down by $62. The governance trail became instantly clean. Same job, zero unnecessary overhead. When a Single Agent with Tools is the Right Answer Most use cases currently designed as multi-agent systems should actually be single agents armed with multiple tools. A tool is a deterministic function that an agent calls — an API, a database query, or a specific calculator. The agent controls the tools; the tools do not control each other. A multi-agent system , on the other hand, distributes business logic across separate reasoning engines, each with its own context. You should only use this pattern when those responsibilities are genuinely separate — not when they merely sound separate. Sounds like multiple agents: A retail customer service system that understands queries, retrieves database information, generates responses, and validates answers. Is actually: One agent with four tools (Query Parser, Azure AI Search tool, Response Generator, and Confidence Scorer). It is clean, fast, and highly governable. Sounds like multiple agents: A compliance checking system that extracts data, checks regulations, assesses risk, and produces a final verdict. Is actually: One agent running a single, comprehensive context window equipped with a regulatory database tool, a risk-scoring API, and an audit logging function. There is zero orchestration overhead. If your responsibilities can comfortably share a single context window, they belong in one agent. Only split them when they physically cannot share context, or when they must operate with absolute infrastructure-level independence. When Multi-Agent is Genuinely the Right Answer This is not to say that multi-agent architectures are entirely useless. Just as microservices have a legitimate, necessary place in enterprise software, multi-agent systems are highly valuable under the right operational conditions. Multi-agent architecture earns its engineering overhead when it solves a problem that a single agent physically cannot handle. There are four legitimate reasons to split: True Parallelism: If an inventory check, a price calculation, and a customer eligibility check must run simultaneously to meet tight service-level agreements (SLAs), three parallel agents are highly justified. In a retail pricing system handling 10 million decisions per day at a $150\text{ ms}$ SLA, this concurrent execution matters immensely. Scaling Beyond Context Limits: A supply chain analysis processing 500 supplier records simultaneously — each containing detailed history — will quickly overwhelm a single model’s context window or degrade its reasoning capability. Distributing the cognitive load across multiple specialized agents, each analyzing a subset, is the correct architectural choice. Strict Security & Access Boundaries: If Agent A needs direct access to sensitive HR databases containing personally identifiable information (PII), but Agent B interacts directly with public untrusted users, they must be separate. Separating them allows you to enforce access controls at the infrastructure level (e.g., separate Azure Managed Identities) rather than hoping your system prompt prevents data leakage. Independent Update Lifecycles (Conway’s Law): If your pricing logic changes every week under the ownership of the finance team, but your inventory logic changes quarterly under the logistics team, separate them. This allows you to redeploy the pricing agent without re-testing or risking the stability of the inventory agent. SOLID and Microservices: What We Forgot The intense debate about single-agent versus multi-agent is not new. We had this exact same debate about monoliths versus microservices a decade ago — and we are making the same mistakes in the exact same order. The monolithic architectures of the past did everything in one place: simple to build, but incredibly hard to scale. When microservices arrived, the industry over-corrected. Teams began splitting every single database table into its own microservice. They ended up building distributed monoliths — inheriting all the operational complexity of distributed systems with none of the benefits of true separation. The lesson that took software engineering ten years to fully absorb is this: The right way to decompose a system is not by function or task. It is by domain. The Single-Agent Problem: Violating SRP When one agent handles customer queries, pricing decisions, inventory checks, fraud detection, and compliance validation, you have created a God Agent . This is the exact equivalent of a God Class in object-oriented design, violating the Single Responsibility Principle (SRP) . The agent has too many reasons to change. When your pricing logic updates, you have to redeploy the entire agent and retest every unrelated capability. If fraud detection breaks, your basic customer queries break with it. The Five-Agent Problem: The Distributed Monolith The opposite mistake is equally damaging. Building five separate agents for five sequential tasks in a fixed, hard-coded pipeline is a distributed monolith. In traditional software, the worst microservice pattern is splitting at the function level (e.g., one service per API call). The services are tightly coupled through sequential calls; the distributed nature adds latency and deployment friction without adding any true runtime independence. Five agents executing five sequential steps in a rigid, fixed order is exactly this. Nothing runs in parallel. No agent is truly independent. It is a slow, expensive pipeline masquerading as an “autonomous agent swarm.” The Right Answer: Bounded Contexts Domain-Driven Design (DDD) solved this issue for microservices through the concept of a Bounded Context : each service owns a clear domain boundary, operates independently within it, and communicates only via well-defined, stable interfaces. The exact same principle applies to AI. Each agent should own a business domain — not a task, and not a function. This is not a distributed monolith; it is a properly bounded multi-agent system. It aligns perfectly with Conway’s Law, scales horizontally, and maintains strict security boundaries. 5 Principles for Choosing Your Architecture Before you commit to an architecture on your next project, run your design through these five engineering principles: Default to a Single Agent; Justify Every Split: Start with one well-structured agent and a clean set of tools. The burden of proof is always on the split. If you cannot name the exact technical problem a new agent solves that a simple tool cannot, keep it unified. Split by Domain Boundary, Not by Task Count: The number of agents should match the number of independent business domains — not the number of sequential steps in your workflow. Tasks become tools; domains become agents. Validate Against the Four Hard Boundaries: Only split your agent if your use case requires parallel execution, infrastructure-level security isolation, containment of context window limits, or independent deployment pipelines. If It’s Sequential and Fixed, It’s a Tool, Not an Agent: If Agent A always calls Agent B, which always calls Agent C in a rigid, non-overlapping sequence, you have built a pipeline, not an autonomous agentic network. Refactor those steps as tools within a single agent. If You Can’t Write the Domain Boundary in One Sentence, Don’t Split: Before drawing a new agent box on your whiteboard, write a single sentence describing what business domain that agent owns exclusively. If the sentence is vague or uses the word “and” to glue unrelated concepts together, your boundary is wrong. Architectural Decision Matrix This quick reference guide helps engineering and architecture teams evaluate whether to implement a Single-Agent or a Multi-Agent architecture for a given workload. Quick Rule of Thumb Default to Single-Agent if your solution fits within a single domain, shares security contexts, runs tasks sequentially, and comfortably fits inside your LLM’s context window. Pivot to Multi-Agent if you hit a hard boundary regarding parallel processing , distinct security/governance zones , cognitive context limits , or independent domain release lifecycles . Keep it Simple: The Single-Agent Core The test I run on every enterprise project is simple: I draw a line on the whiteboard for every proposed agent, and I write what business domain it owns in a single sentence. If two agents share a domain, I merge them. If one sentence contains the word “and,” I check if both halves can truly deploy independently. Ten minutes on a whiteboard saves weeks of painful refactoring in production. Before you build your next agent swarm, remember the golden rule of distributed systems engineering: Don’t distribute your system unless you absolutely have to. Start with a single, well-structured agent. Write clean, deterministic code to handle your business logic, loops, and state management. Treat the LLM as a powerful engine for reasoning and unstructured translation — not as a replacement for software architecture. Let’s stop building architectural theater. The best system is not the one with the most boxes on the whiteboard; it’s the simplest one that successfully delivers value to your users. Five Single AI Agents Is Not Microservices. It Is a Distributed Monolith was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.