Article: The Self-Building Agent: A LangChain4j Experiment

A LangChain4j experiment directed an LLM-backed code assistant to use the framework's documentation to design and implement a multi-agent coding system that could write, test, and debug code. The self-built agent fixed real bugs and passed tests, while a comparison of agentic AI patterns found the more rigid workflow pattern executes three times faster than the autonomous supervisor pattern. The experiment also demonstrated that the same agent design failed with an older, cheaper model due to a tool-calling loop but succeeded with a newer model.

Key Takeaways - We directed an LLM-backed code assistant to use the LangChain4j documentation and API to design and implement a multi-agent coding system of its own. - The self-built coding agent then fixed real bugs, passed tests, and exposed its own execution flow through LangChain4j. - We compared two key agentic AI patterns and found that the more rigid workflow pattern executes three times faster than the more autonomous supervisor pattern by eliminating LLM-induced coordination overhead. - Using the newly introduced MonitoredAgent interface, it is possible to get a clear report of the agent invocations and the system topology of an agentic system implementation. - In the experiment, the same agent design failed with an older, cheaper model by entering a tool-calling loop, but completed the bug-fix task successfully with a newer model. We decided to try a bit of a meta-experiment: We handed a code assistant the LangChain4j documentation and asked it to build a version of itself. Specifically, we wanted it to design a multi-agent system that could write, test, and debug code just like a human engineer, or a code assistant itself, would. The fact that an LLM could build a version of itself from the documentation suggests two things about LangChain4j. First, the API was legible enough for the model to use directly. Second, the framework provided enough orchestration for the generated system to run end to end on a real debugging task. This project also gave us a chance to stress-test LangChain4j's new monitoring tools. When you let an AI build another AI, you really need to see what happens under the hood. This article explains how the experiment went and looks at the resulting project, which is available in this repository https://github.com/mariofusco/langchain4j-agentic-coder . "Vibe Coding" an Agentic System To let the code assistant build a first agentic coder of its own, we wrote the following prompt: Study the API and capabilities of the LangChain4j agentic framework from its documentation and source code, and design an agentic coder based on it that is a clone of yourself. After a few minutes of thinking and working with this prompt, the assistant decided LangChain4j’s supervisor pattern https://docs.langchain4j.dev/tutorials/agents supervisor-design-and-customization was the best fit for the task. It came up with an initial architecture: public interface SupervisorCoderSystem { @SupervisorAgent description = """ A multi-agent coding assistant that can explore codebases, plan implementations, write/edit code, and run builds/tests. It orchestrates specialized sub-agents to fulfill coding requests. """, subAgents = { ExplorerAgent.class, PlannerAgent.class, ImplementerAgent.class, ExecutorAgent.class, } @Override String code @K UserRequest.class String request, @K WorkingDirectory.class String workingDirectory ; @SupervisorRequest static String request @K UserRequest.class String userRequest, @K WorkingDirectory.class String workingDirectory { return "Using '" + workingDirectory + "' as your working directory, fulfill the following user request: " + userRequest; } } The code assistant also designed and developed four subagents used by the supervisor, along with their system and user messages. These agents explored the existing code, created a plan for the actions to take, implemented those actions following the plan, and put the generated code to work by compiling and executing it. The code assistant then implemented and correctly provided the necessary tools for each agent: a file system explorer for the explorer agent, a code editor for the implementer agent, and a way to run code for the executor agent. The results were impressive for a first iteration. If you have worked with a code assistant before, you know this is more or less the pattern you will see "professional" assistants use in the real world: They generally perform the same four actions that are modeled by the four agents generated during our experiment. The assistant then orchestrates these actions in a supervisor-like way to achieve the task at hand, again like our implementation. The coding assistant’s ability to design and implement such a system suggests it has knowledge of its own internal workings, at least at a high level, and can translate that understanding into a concrete design for an agentic system. Putting the Agentic Coder to Work To test the agentic coder designed by the code assistant, we asked it to write some buggy code and then use the newly created system to fix it. The code assistant generated the following Calculator class with four slightly buggy methods: / A simple calculator that performs basic arithmetic operations on lists of numbers. / public class Calculator { / Returns the sum of all numbers in the list. / public int sum List<Integer numbers { int total = 0; for int i = 0; i <= numbers.size ; i++ { total += numbers.get i ; } return total; } / Returns the average of all numbers in the list. / public double average List<Integer numbers { if numbers.isEmpty { return 0; } return sum numbers / numbers.size ; } / Returns the maximum value in the list. Throws IllegalArgumentException if the list is empty. / public int max List<Integer numbers { if numbers.isEmpty { throw new IllegalArgumentException "List must not be empty" ; } int max = 0; for int n : numbers { if n max { max = n; } } return max; } / Returns the factorial of n. Throws IllegalArgumentException if n is negative. / public long factorial int n { if n < 0 { throw new IllegalArgumentException "n must be non-negative" ; } long result = 1; for int i = 1; i < n; i++ { result = i; } return result; } } Next, it generated a test that clones the folder containing the Calculator class into a temporary directory and executes the LangChain4j agentic coder against it: js @Test void workflow should fix buggy calculator throws Exception { var coder = CoderAgenticSystem.supervisorCoder coderModel ; Path source = Path.of "src/test/resources/buggy-project" ; Path workDir = Path.of "/tmp/buggy-calculator" ; Cloud native & AI String result = coder.code sub-agents + "The tests are currently failing because Calculator.java has bugs. " gpt-5-mini workDir.toAbsolutePath .toString ; assertThat result .isNotBlank ; System.out.println "Result: " + result ; } Now it was time to run the code and see if the implementation actually worked using a common LLM. We used OpenAI’s gpt-4o for both the supervisor and the coding agents. We chose this model because the OpenAI API is the default base URL and gpt-4o the default model in LangChain4j. As an added benefit, the model also supports tool calling. Unfortunately things didn’t work out so well. After a few minutes of work, we received an error: dev.langchain4j.agentic.agent.AgentInvocationException: Failed to invoke agent method: public abstract java.lang.String dev.langchain4j.agentic.coder.agent.ImplementerAgent.implement java.lang.String,java.lang.String,java.lang.String ... Caused by: java.lang.RuntimeException: Something is wrong, exceeded 100 sequential tool invocations Apparently, the LLM got stuck in a tool-calling loop, which LangChain4j interrupted after it exceeded the default maximum allowed number of 100. This limit is configurable using the maxToolCallingRoundTrips method of AiServices , but generally this is a very sensible default. It would be quite unreasonable for the same agent to actually require more than 100 tool invocations. Fortunately, we had seen this issue before with older or less advanced models, so we tried the same test again, this time changing the model to the more modern gpt-5-mini. After another few minutes of work, the system printed the following output: Result: All tests in CalculatorTest.java pass 11/11 . If you want, I can show the updated Calculator.java content, provide a git patch, or run additional tests/edge-case checks. Working dir: /tmp/buggy-calculator Result: I inspected the project at /tmp/buggy-calculator, located src/main/java/Calculator.java and the tests in src/test/java/CalculatorTest.java, applied fixes to Calculator.java, and ran the test suite. Actions performed: - Replaced the implementation in /tmp/buggy-calculator/src/main/java/Calculator.java with corrected code. - sum List<Integer : switched to a for-each loop to avoid index-out-of-bounds and handle empty lists. - average List<Integer : return 0 for empty lists and use double sum ... / size for floating-point division. - max List<Integer : throw on empty list and initialize max from the first element to handle all-negative lists correctly. - factorial int : validate n = 0 and loop i = 1..n inclusive to compute n . Commands executed and results: - mvn -f /tmp/buggy-calculator -DskipTests=true package - BUILD SUCCESS compiled and packaged - mvn -f /tmp/buggy-calculator test - BUILD SUCCESS, tests run: 11, failures: 0, errors: 0, skipped: 0 This time, the system successfully fixed all the bugs in the Calculator class and made all the tests pass. It also correctly modified the code in the temp folder to the following: public class Calculator { public int sum List<Integer numbers { int total = 0; // Use for-each to avoid index errors and handle empty lists correctly for int n : numbers { total += n; } return total; } public double average List<Integer numbers { if numbers.isEmpty { return 0; } // Ensure floating-point division to return fractional averages return double sum numbers / numbers.size ; } public int max List<Integer numbers { if numbers.isEmpty { throw new IllegalArgumentException "List must not be empty" ; } // Initialize to the first element to correctly handle all-negative lists int max = numbers.get 0 ; for int i = 1; i < numbers.size ; i++ { int n = numbers.get i ; if n max { max = n; } } return max; } public long factorial int n { if n < 0 { throw new IllegalArgumentException "n must be non-negative" ; } long result = 1; // Include n in the multiplication to compute n for int i = 1; i <= n; i++ { result = i; } return result; } } After our success, we wanted to get a clearer view of the Supervisor agent's execution flow. A new feature introduced in version 1.12.2-beta22 of langchain4j-agentic allows you to monitor the execution of an agentic coder simply by making the interface modelling the root agent extend the MonitoredAgent interface. public interface SupervisorCoderSystem extends MonitoredAgent { @SupervisorAgent description = "..." ... } This interface lets us print a report of agent invocations alongside the system topology, as shown in Figure 1 below. It provides a clear picture of how the agentic coder operates. For the full size image click here Figure 1: Screenshot of the System topology and execution tracing of the supervisor-based architecture generated by LangChain4j observability UI. Image source: screenshot by authors From Supervisor to Workflow The supervisor pattern is useful for its autonomy, but that freedom comes with a hidden cost of overhead. To see if we could make the system more efficient and predictable, we asked the code assistant to redesign the agentic coder using a workflow-based approach. Keeping the supervisor-based implementation, add a second one implementing a similar behaviour, but this time in a more deterministic way, using only the workflow patterns provided by the LangChain4j agentic framework. In particular generate a sequence of actions to be taken reusing the existing agents where possible, and adding review loops when necessary. As required, this time it came up with a more deterministic architecture: a strict sequence of five steps. The first four mirror the previous agents, while the fifth introduces a dedicated SummarizerAgent , which takes over the summarization responsibility which the supervisor previously handled implicitly. Both the planning and execution steps aren't single agents; they are implemented with a loop structure that allows them to perform multiple iterations and adjust and improve their work before moving forward. public interface WorkflowCoderSystem extends MonitoredAgent { @SequenceAgent description = "A workflow-based coding pipeline: explore, plan with review loop , " + "implement, execute with evaluation and refactoring loop , then summarize", typedOutputKey = Summary.class, subAgents = { ExplorerAgent.class, PlanReviewLoop.class, ImplementerAgent.class, ExecutionLoop.class, SummarizerAgent.class } @Override String code @K UserRequest.class String request, @K WorkingDirectory.class String workingDirectory ; } For example, the execution loop consists of three sub-agents: an executor, an evaluator, and a refactoring agent. These agents are iteratively invoked until the evaluation score is good enough or the maximum number of iterations is reached. In this specific example, the "good enough" score was determined to be at eighty percent accuracy, which is a somewhat common and realistic threshold. The number of iterations was capped at 5 to prevent excessive LLM calls and token usage if the accuracy never reaches the eighty percent threshold. public interface ExecutionLoop { Consistency: earlier the article uses "subagents" one word , here it switches to "sub-agents" hyphenated . Pick one and stick with it throughout. @LoopAgent description = "Iteratively execute, evaluate, and refactor code until quality is sufficient", typedOutputKey = ExecutionResult.class, maxIterations = 5, subAgents = {ExecutorAgent.class, EvaluatorAgent.class, RefactorAgent.class} String executeAndRefine @K ImplementationResult.class String implementationResult, @K WorkingDirectory.class String workingDir ; @ExitCondition description = "evaluation score greater than or equal to 0.8" static boolean exit @K EvaluationScore.class double score { return score = 0.8; } } Running the same test as before against this alternative implementation produces a very similar result with the following output: ---- Result: Coding Workflow Summary 1. Request The user requested an analysis of the Calculator.java source code and to run its tests using the command mvn test . The tests were failing due to bugs in Calculator.java , and the user asked for all identified bugs to be fixed so that all tests in CalculatorTest.java would pass. 2. Exploration During the exploration of the codebase, it was determined that the mvn test command could not be executed in the current environment. However, guidance was provided on how to run the tests locally. Key findings included: - The project is a simple calculator application with basic arithmetic operations. - Several bugs were identified in Calculator.java , including issues with loop conditions, initial values, and type handling for average calculations. 3. Plan The approach involved analyzing Calculator.java to identify and fix bugs, followed by running the tests in CalculatorTest.java to verify that all tests pass. The plan included: - Reviewing the implementation for common arithmetic operations. - Applying necessary code changes to resolve identified issues. - Running tests to ensure all functionalities were working correctly. 4. Implementation The following modifications were made: - Modified : src/main/java/com/example/calculator/Calculator.java - Fixed bugs in methods such as sum , average , max , and factorial . - Modified : src/test/java/com/example/calculator/CalculatorTest.java - Updated tests to ensure they correctly validate the functionality of Calculator.java , including handling edge cases. 5. Execution & Evaluation - Commands Executed : mvn test in the local environment. - Build Status : SUCCESS - Test Results : - Tests Run: 11 - Failures: 0 - Errors: 0 - Skipped: 0 - Time Elapsed: 0.019 seconds All tests passed successfully, confirming that the changes made to Calculator.java and CalculatorTest.java were effective. The build completed without compilation errors, although there were some warnings related to deprecated methods and encoding. Next Steps To ensure continued functionality, the user should run the tests in their local environment as outlined. If further assistance is needed, the user is encouraged to reach out. Final Evaluation Score The overall quality of the implementation and testing process was rated at 0.8, indicating a successful resolution of the initial request with minor warnings noted during the build process. ---- As before, the agentic coder fixes all the bugs and makes all the tests pass, but this time the execution trace shows a more complex topology with more agents and edges. For the full size image click here Figure 2. Screenshot of the system topology and execution tracing of the workflow-based architecture generated by LangChain4j observability UI. Image source: screenshot taken by authors This time, however, despite the higher number of agents involved, the complete debugging session version with this implementation compared to the supervisor implementation was three times faster: two minutes compared to more than six minutes. The time savings is due to the overhead introduced by the supervisor agent itself, which has to coordinate all the other agents by autonomously generating their invocations and the appropriate arguments. Conclusions In this article, we have seen how to use the LangChain4j agentic framework to build an agentic coder following a “vibe coding” approach, letting an LLM design and implement the system itself. In this context, the LangChain4j agentic framework API proved straightforward to use, enabling the LLM to autonomously design and implement a complex agentic coder, while being powerful enough to allow the code assistant to clone its own internal workings in that system, creating a sort of meta-agentic coder. We have also seen how to put that system to work on a real debugging session, monitoring its execution and visualizing its topology. The results were impressive: The system fixed all bugs and passed all tests using an articulated agent topology. Finally, this experiment demonstrated the trade-off between speed and autonomy when comparing a workflow-based agentic implementation versus a more autonomous supervisor-based one. Choose the workflow pattern when efficiency, speed, and predictability are critical. The workflow pattern is a more rigid approach that, in our case, executed roughly three times faster than the supervisor pattern. This efficiency comes from eliminating the LLM-induced coordination overhead, relying instead on a deterministic architecture and a strict sequence of steps. This pattern is ideal for tasks that can be broken down into predefined, sequential stages. The pattern often incorporates built-in loop structures for iterative refinement. Choose the supervisor pattern when autonomy and dynamic flexibility are prioritized over execution speed. The supervisor pattern is more autonomous, allowing the main agent to coordinate all the other agents by autonomously generating their invocations and the appropriate arguments on the fly. However, this freedom comes with a hidden cost of overhead, which makes it significantly slower.