{"slug": "divide-and-let-the-llm-do-the-rest-from-personal-experience-to-architecture", "title": "Divide and Let the LLM Do the Rest: From Personal Experience to Architecture", "summary": "A developer spent nearly a year exploring how far LLMs can be pushed on complex software projects, finding that both instruction-heavy and RAG-based approaches fail as projects grow beyond a few thousand lines of code. The breakthrough came from using artifacts to make decisions explicit and building a custom IDE, leading to an architecture that prioritizes user understanding and avoids over-engineering.", "body_md": "A few years ago, I started thinking about a simple question:\n\nHow far can we actually push an LLM on a complex software project?\n\nThat's already well established. I mean something harder:\n\nCan an LLM take a complex problem, break it into smaller pieces, solve those pieces, test the results, and eventually assemble everything into a working product?\n\nAt the time, the conversation seemed to be split between two extremes:\n\nAI is basically useless, although admittedly pretty cool.\n\nand:\n\nAI is the future. Give it a few years and programmers will be gone.\n\nI never really bought either position.\n\nI've always thought that when you throw a genuinely difficult problem at a system, getting 100% right is unlikely. But getting 0% right is unlikely too.\n\nAnd that's actually interesting. Because if we can reliably measure what worked and what didn't, we can start improving the system instead of arguing about whether AI \"works.\"\n\nSo I spent almost a year trying to find that boundary. Almost a Year of Trial and Error\n\nI started with instructions. A lot of instructions.\n\nThe idea seemed straightforward: give the model enough rules and guidance, and let it dynamically select the instructions relevant to the current task.\n\nIt didn't work particularly well. The more instructions I added, the harder they became to manage. Eventually, I started getting confused by the instructions — and then the AI started getting confused by them too. So I needed a better way to organize knowledge.\n\nThe obvious next step was RAG.\n\nWhat could go wrong?\n\nInstead of having one giant set of instructions, we could break everything into small, atomic rules and let the AI retrieve the rules it needed.\n\nOn paper, this looked great. In practice, it opened up an entirely new set of problems:\n\nAnd that was just the beginning. The same fundamental problem kept coming back.\n\nOnce the project grew large enough — around 5,000 lines of code — things started getting difficult again.\n\nThe problem wasn't simply retrieving information. The problem was understanding the system as a whole. Maybe the Answer Was Refactoring\n\nMy next thought was that we needed some kind of automated refactoring mechanism. But eventually I realized that wasn't quite the right problem either. What I really wanted was a way for the LLM to answer a much more fundamental question:\n\n\"How well do I actually understand this project?\"\n\nThat's when I noticed another threshold. At around 3,000 lines, at least with the models I was using at the time — including DeepSeek V4 Flash — the context would fill up extremely quickly.\n\nOnce that happened, every new task effectively became a fresh start. The model could no longer reliably maintain a mental model of the entire project. And that led me to an uncomfortable conclusion. Maybe we shouldn't try to make the software structure equally convenient for humans and LLMs. At least, I didn't know how to do that.\n\nSomething had to change.\n\nThe breakthrough wasn't a better prompt.\n\nIt wasn't a better RAG system.\n\nIt wasn't a bigger context window.\n\nIt was artifacts.\n\nI started using artifacts as a way to make decisions explicit and track what had actually happened during development.\n\nAt roughly the same time, I stopped using OpenCode and started building my own IDE. That turned out to be a much bigger decision than I expected.\n\nFrom that point on, I started designing around a few simple principles.\n\n**For the User**\n\nBut they should always be able to answer:\n\n**For Software Development**\n\nThe system should prioritize:\n\nAnd there was another principle I didn't want to lose:\n\n**Don't accidentally build SQL.**\n\nThis might sound strange, but I've seen the pattern before.\n\nSomething starts as:\n\n\"Let's manage a database using natural language.\"\n\nAnd eventually turns into a massive infrastructure project with layers upon layers of machinery. I didn't want to build an enormous system just to make LLMs slightly easier to use.\n\nThe system itself had to remain understandable.\n\nConsider a clinic manager.\n\nThey probably know exactly how the clinic should operate.\n\nThey know:\n\nThey may understand the business process extremely well. But turning that knowledge into software normally requires a whole team of specialists.\n\nSo here's the question I'm interested in:\n\nHow much of that work can we safely move onto the shoulders of AI?\n\nNotice the word **safely**.\n\nI'm not arguing that we should \"** give everything to AI.**\"\n\nI'm asking:\n\nWhich parts can we delegate?\n\nAnd I suspect the answer depends much less on the raw intelligence of the model than we currently think.\n\nIt may depend on the harness around the model. And on choosing the right artifacts.\n\nWhere Could This Lead?\n\nThere are several possible outcomes.\n\n**Maybe AI Can't Do It**\n\nWe might eventually discover that reliable task decomposition simply doesn't generalize.\n\nEvery real-world problem may be too different.\n\nMaybe there is no universal architecture that makes AI + LLM + harness work reliably.\n\nAnd maybe, in some cases, hiring an expert will simply be cheaper.\n\nThat's a perfectly valid outcome.\n\n**Maybe We Get 10–15%**\n\nAI might reliably handle only a portion of the work.\n\nSuppose we save 10–15% of someone's time.\n\nThat may not sound revolutionary.\n\nBut 12.5% is roughly one employee out of eight.\n\nFor a large organization, a 10% reduction in labor costs is already a very serious business case.\n\n**Maybe AI Becomes an Operator**\n\nPerhaps we can delegate a significant portion of operational processes to AI.\n\nHumans supervise the system and step in when necessary.\n\nThink about a power plant operator.\n\nThey don't manually control every parameter every second.\n\nThey sit at the control panel, monitor the system, and intervene when something requires human judgment.\n\nMaybe software systems will work the same way.\n\n**Maybe AI Runs Almost Everything**\n\nTake aviation as another analogy.\n\nThere are situations where modern aircraft can handle an enormous amount of the actual flight work.\n\nThe pilots remain responsible for critical decisions and exceptional situations.\n\nIn a future version of this model, humans might spend most of their time monitoring, diagnosing, and handling edge cases rather than continuously operating the system.\n\n**Or Maybe We Go All the Way**\n\nAnd there is an even more extreme possibility.\n\nAI handles almost everything. Human experts don't disappear. Instead, they become highly specialized service providers.\n\nA company might only need an expert every couple of weeks rather than having one embedded in the organization full-time.\n\nI don't know which of these futures is correct.\n\nThat's exactly what I want to find out.\n\nThis is where I want to start a discussion.\n\nI realize the idea is controversial.\n\nThat's fine.\n\nMaybe I'm wrong.\n\nMaybe someone will take these ideas, throw half of them away, and build something much better.\n\nI'd be perfectly happy with that outcome.\n\nBut there is one shift in perspective that I think is important.\n\n*Code Isn't the Product!*\n\nI'd like to move one level higher in abstraction.\n\nWe tend to think of code as the final product.\n\nI increasingly think that's the wrong abstraction.\n\nCode is a means of solving a problem.\n\nThe actual product is the solution to the client's problem.\n\nIf a clinic comes to you because patients are waiting too long, the deliverable isn't \"50,000 lines of Python.\"\n\nThe deliverable is fewer problems with patient flow.\n\nThe code is just one of the tools used to get there.\n\nThat distinction becomes extremely important when we start delegating parts of software engineering to AI.\n\nThis leads to another idea I've been exploring:\n\n**a library of proven solutions.**\n\nImagine that instead of asking an AI to reinvent everything from scratch, we give it access to a collection of immutable, reusable implementations.\n\nThe AI can:\n\nAnd each solution could accumulate real-world statistics.\n\nBlockchain might potentially play a role here by providing an immutable record of provenance and history.\n\nI'm not claiming blockchain is automatically the answer.\n\nThe important idea is the immutable, verifiable history of a solution, not the technology used to implement it.\n\nExperts in different domains could exchange knowledge without having to exchange huge theoretical documents. Instead, they could exchange deterministic agents and executable artifacts that demonstrate how their ideas actually work.\n\nImagine an expert in healthcare building a workflow that captures how a particular clinical process should operate.\n\nAnother organization could potentially import that knowledge, inspect it, test it, and adapt it.\n\nNot:\n\n\"Here is a 200-page document explaining our methodology.\"\n\nBut:\n\n\"Here is a deterministic artifact that demonstrates the methodology. Run it. Inspect it. Test it. Decide whether it is useful.\"\n\nThat feels much closer to engineering knowledge than traditional documentation.\n\n**And perhaps that's the bigger opportunity.**\n\nMaybe the goal isn't to build an AI that knows everything.\n\nMaybe the goal is to build an environment where AI can reliably use what humans already know.", "url": "https://wpnews.pro/news/divide-and-let-the-llm-do-the-rest-from-personal-experience-to-architecture", "canonical_source": "https://dev.to/evgeniy_kormin/divide-and-let-the-llm-do-the-rest-from-personal-experience-to-architecture-53hc", "published_at": "2026-09-01 14:20:23+00:00", "updated_at": "2026-09-01 14:55:24.284795+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "developer-tools"], "entities": ["DeepSeek V4 Flash", "OpenCode"], "alternates": {"html": "https://wpnews.pro/news/divide-and-let-the-llm-do-the-rest-from-personal-experience-to-architecture", "markdown": "https://wpnews.pro/news/divide-and-let-the-llm-do-the-rest-from-personal-experience-to-architecture.md", "text": "https://wpnews.pro/news/divide-and-let-the-llm-do-the-rest-from-personal-experience-to-architecture.txt", "jsonld": "https://wpnews.pro/news/divide-and-let-the-llm-do-the-rest-from-personal-experience-to-architecture.jsonld"}}