Building Enterprise Multi-Agent Systems on the JVM - A Layered Architecture with Koog, MCP, and A2A

JetBrains' Koog framework, combined with the Model Context Protocol (MCP) and Agent-to-Agent protocol (A2A), enables a three-layer architecture for building enterprise multi-agent systems on the JVM. The approach separates agent construction, tool integration, and agent coordination into independently scalable layers, addressing the complexity of real-world deployments with multiple specialized agents.

Most AI agent tutorials show a single agent calling a few tools. That works for demos. It falls apart the moment a real enterprise system needs ten specialized agents, each with its own tools, coordinating to handle a complex request, all running inside infrastructure that was built years before anyone said the word “agent.” This article lays out a clean three-layer architecture for building exactly that kind of system on the JVM, using three pieces that fit together remarkably well. Koog for building the agents, the Model Context Protocol MCP for connecting agents to tools, and the Agent-to-Agent protocol A2A for letting agents coordinate. Each layer solves one problem, and the separation keeps the whole system maintainable as it grows. The architecture separates three distinct concerns: Layer 1, Agent construction Koog . Each agent is built using Koog, a Kotlin-native agent framework from JetBrains. Koog handles the agent’s internal workflow as a graph, along with retry logic, state persistence, and conversation history management. Layer 2, Tool integration MCP . Each agent connects to the tools it needs through MCP servers. Tools are decoupled from agent logic, so they can be added, swapped, or updated without touching the agents. Layer 3, Agent coordination A2A . When a request needs more than one agent, A2A handles the communication between them. A coordinator agent routes work to specialists, and specialists can delegate further to sub-specialists. This mirrors how microservices decompose a monolith. Each layer is independently developed, tested, and scaled. The sections below walk through each one with code. Koog defines agent behavior as a directed graph of nodes. Each node is a step such as an LLM call, a tool execution, a decision branch. The edges define how the agent moves between steps based on what happened. val agentStrategy = strategy<String, String "Specialist Agent" { val nodeSendInput by nodeLLMRequest val nodeExecuteTool by nodeExecuteTools val nodeSendToolResult by nodeLLMSendToolResults edge nodeStart forwardTo nodeSendInput edge nodeSendInput forwardTo nodeFinish onTextMessage { true } edge nodeSendInput forwardTo nodeExecuteTool onToolCalls { true } edge nodeExecuteTool forwardTo nodeSendToolResult edge nodeSendToolResult forwardTo nodeFinish onTextMessage { true } edge nodeSendToolResult forwardTo nodeExecuteTool onToolCalls { true } } Read the edges as a state machine. When the LLM returns plain text, the agent finishes. When the LLM asks for a tool call, the agent runs the tool, feeds the result back to the LLM, and loops until it has a final answer. The advantage of defining this as an explicit graph is that every path is traceable and the whole thing is type-checked at compile time. The edge conditions, the node inputs, the node outputs all get validated before the code ever runs. Tools are defined as strongly typed Kotlin functions: @LLMDescription "Tools for inventory operations" class InventoryTools : ToolSet { @Tool @LLMDescription "Retrieves current stock level for a product" fun getStockLevel sku: String, locationId: Int : StockResult { return inventoryService.queryStock sku, locationId }} Because locationId is typed as Int, not a loosely typed dictionary value, a mismatch with the upstream API is caught by the compiler rather than surfacing as a runtime error deep inside a multi-step workflow. When agents talk to dozens of internal services, this eliminates an entire category of production bugs. The Model Context Protocol standardizes how agents access tools. Instead of writing bespoke integration code for every API, an agent connects to an MCP server that exposes tools through a consistent interface. val mcpClient = McpClient { transport = McpTransport.sse "http://mcp-server:3000/sse" }val mcpToolRegistry = mcpClient.toToolRegistry val agent = AIAgent promptExecutor = simpleAnthropicAIExecutor , llmModel = AnthropicModels.Claude.SONNET, strategy = agentStrategy, toolRegistry = mcpToolRegistry In a multi-agent system, a shared MCP server can host all the tools, with each agent configured to access only the subset it needs. The inventory agent reaches inventory tool, the scheduling agent reaches scheduling tools. Neither can touch the other’s tools. This is the principle of least privilege applied to agents. Retrieval-Augmented Generation fits naturally here too. Rather than building RAG as a separate pipeline, you expose it as an MCP tool. The agent decides when it needs to look something up, calls the retrieval tool like any other tool, and folds the results into its reasoning. Knowledge retrieval becomes just another tool call, which keeps the architecture consistent. A single agent can only do so much. Real systems need specialists: one agent that knows inventory, another that knows scheduling, another that knows policy. The Agent-to-Agent protocol lets them find each other and delegate work. A specialist agent advertises itself with an agent card: val inventoryAgent = AIAgent promptExecutor = simpleAnthropicAIExecutor , llmModel = AnthropicModels.Claude.SONNET, strategy = agentStrategy, toolRegistry = inventoryToolRegistry { install A2AAgentServer { agentCard { name = "Inventory Agent" description = "Handles stock queries and product lookups" capabilities = listOf "inventory-lookup", "stock-check" } }}inventoryAgent.startA2AServer port = 8081 A coordinator agent connects to the specialists and routes requests to them: val a2aClient = A2AClient "http://localhost:8081" val coordinatorAgent = AIAgent promptExecutor = simpleAnthropicAIExecutor , llmModel = AnthropicModels.Claude.SONNET, strategy = coordinatorStrategy, toolRegistry = localToolRegistry { install A2AAgentClient { addClient a2aClient }} The coordinator routes a request to the right specialist. Specialists can delegate further like a troubleshooting agent might hand off to a diagnostics sub-agent for a specific class of equipment problem. The hierarchy is extensible. Adding a new specialist does not require touching the coordinator or any existing agent. The new agent registers its capabilities through A2A and becomes discoverable. Once you have ten agents coordinating through A2A and calling tools through MCP, you need to see what is happening when something goes wrong. Koog ships with OpenTelemetry support built in: val agent = AIAgent promptExecutor = simpleAnthropicAIExecutor , llmModel = AnthropicModels.Claude.SONNET, strategy = agentStrategy, toolRegistry = toolRegistry { install OpenTelemetry { exporter = LangfuseExporter publicKey = System.getenv "LANGFUSE PUBLIC KEY" , secretKey = System.getenv "LANGFUSE SECRET KEY" , host = "https://cloud.langfuse.com" }} Every LLM call, tool invocation, graph transition, and inter-agent delegation produces a trace. Because every agent runs on the same JVM infrastructure, all those traces land in one observability platform instead of being scattered across separate Python and JVM monitoring stacks. When a user reports that an agent gave a wrong answer, you can follow the entire path, from coordinator to specialist to tool call to LLM response, in a single trace. Putting the three layers together produces a system with some useful properties: Independent evolution. Add a tool without touching agents. Add an agent without touching the coordinator. Each layer changes on its own schedule. Type safety end to end. Tool contracts, agent workflows, and inter-agent messages are all checked at compile time. Errors surface during the build, not in production. Unified operations. One runtime, one deployment pipeline, one observability stack. The agents live inside the existing JVM services rather than alongside them as a separate system. Clear scaling story. Each specialist agent can scale independently based on its own load, the same way microservices do. Not every system needs all three layers. If you have a single agent calling a handful of tools, Koog plus MCP is enough and you do not need A2A until you have multiple agents that must coordinate. Start simple. Add the coordination layer when the number of specialists and the complexity of routing between them justifies it. The same goes for the protocols themselves. MCP is for the agent-to-tool relationship. A2A is for the agent-to-agent relationship. They are complementary, not competing, and a healthy system often uses both. each agent reaches its tools through MCP, and the agents reach each other through A2A. The AI agent space moves fast, and the tooling is still young. Koog reached its 1.0 release recently, MCP arrived in late 2024, and A2A in early 2025. But the architectural shape here, separating agent construction from tool integration from agent coordination, is durable. The specific frameworks will evolve, but the layering will hold. For teams already invested in the JVM, this approach means AI agents do not require abandoning the infrastructure you have spent years building. The agents become a natural extension of the backend, type-safe and observable, rather than a foreign system bolted on the side. That, more than any single feature, is what makes enterprise AI agents practical at scale. Building Enterprise Multi-Agent Systems on the JVM - A Layered Architecture with Koog, MCP, and A2A https://pub.towardsai.net/building-enterprise-multi-agent-systems-on-the-jvm-a-layered-architecture-with-koog-mcp-and-a2a-0a315d2fd0ee was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.