{"slug": "llms-have-learned-all-open-code-on-github-but-missed-the-most-important-part", "title": "LLMs Have Learned All Open Code on GitHub - But Missed the Most Important Part.", "summary": "A developer argues that large language models trained on GitHub's final code are imitators, not real engineers, because they lack the thought process behind code evolution. The developer proposes training models on commit history to teach them the reasoning and decision-making behind code changes, potentially creating AI that understands engineering trade-offs.", "body_md": "Or why AI coders are just imitators, not real engineers.\n\nWe are used to thinking that modern large language models (LLMs) know everything about programming. They are trained on billions of lines of code from open GitHub repositories. They can write even a big app from scratch.\n\nBut there is one problem.\n\nWe gave them millions of ready-made answers. But we never gave them the main thing - the thought process that led to those answers.\n\nImagine you are teaching an architect to build houses by showing him 10 million photos of finished buildings. He will learn that houses have walls, roofs, and windows. He will become a great imitator. But he will never know why the foundation was laid exactly here, why red brick was chosen instead of white, or what mistakes were made and how they were fixed.\n\nThis is the problem with all modern LLM coders. They learn on the final, polished versions of code. They see the ideal result, but they do not see the path to it.\n\nThey are great at mimicking good style, because their memory holds hundreds of millions of lines of clean code. But they have a rather superficial understanding of code and zero vision of the project as a whole - they often only implement a local task, and they need a good architect to stand over their soul and make architectural decisions.\n\nWhat if we gave models not just \"photos\" of code, but the full history of its creation?\n\nEvery project on GitHub contains the history of the project from the very beginning. Thousands of commits. Each commit is a small decision. Each commit message is a piece of the developer's thought.\n\nImagine seeing the history of the Rust language from 1 file to 10k+ files (number pulled out of thin air).\n\nWhat would the model actually learn from this history?\n\nNot just syntax. Not just patterns.\n\nIt would learn why certain decisions were made. It would see that unwrap() was used first, then replaced with Result and the ? operator - not because it's \"better style\", but because unwrap() crashed in production. It would see how error handling evolved from \"just crash\" to \"handle properly with context\".\n\nThis is the difference between mimicking code and understanding it.\n\nA model trained on such a history would be a mega-expert in Rust.\n\nWhat are the other benefits:\n\nSome might say this is too much data. And they'd be right. Most of it is noise - typos, half-finished experiments, abandoned branches. But we don't have to use all of it. We can choose the great ones. Linux. PostgreSQL. Rust. The Go compiler. Redis. These aren't just codebases - they're textbooks written by the best engineers in the world. That's where the real lessons are.\n\nRight now, the whole LLM world looks at GitHub and sees only the final code. But we need to look deeper - at the history of that code.\n\nIf we train models on commit history, we will get not just code generators. We will get experts who understand the evolution, the reasons, and the consequences of every decision.\n\nAnd then AI coders will stop being just imitators. They will become real engineers (though that scares me).\n\nAnd here is what makes programming truly unique. It is the only field where we have a complete, recorded history of a complex system's evolution - from a single file to millions of lines. Every commit is a footprint. Every message is a thought. We have the full DNA of every major project.\n\n*This whole article is just my thoughts on this topic. Please don't be too hard on me.*", "url": "https://wpnews.pro/news/llms-have-learned-all-open-code-on-github-but-missed-the-most-important-part", "canonical_source": "https://dev.to/kostya_m_17bb892f188e6872/llms-have-learned-all-open-code-on-github-but-missed-the-most-important-part-1odg", "published_at": "2026-08-05 13:33:14+00:00", "updated_at": "2026-08-05 14:03:32.854859+00:00", "lang": "en", "topics": ["large-language-models", "artificial-intelligence", "developer-tools"], "entities": ["GitHub", "Rust", "Linux", "PostgreSQL", "Redis"], "alternates": {"html": "https://wpnews.pro/news/llms-have-learned-all-open-code-on-github-but-missed-the-most-important-part", "markdown": "https://wpnews.pro/news/llms-have-learned-all-open-code-on-github-but-missed-the-most-important-part.md", "text": "https://wpnews.pro/news/llms-have-learned-all-open-code-on-github-but-missed-the-most-important-part.txt", "jsonld": "https://wpnews.pro/news/llms-have-learned-all-open-code-on-github-but-missed-the-most-important-part.jsonld"}}