# LLMs Have Learned All Open Code on GitHub - But Missed the Most Important Part. > Source: > Published: 2026-08-05 13:33:14+00:00 Or why AI coders are just imitators, not real engineers. We are used to thinking that modern large language models (LLMs) know everything about programming. They are trained on billions of lines of code from open GitHub repositories. They can write even a big app from scratch. But there is one problem. We gave them millions of ready-made answers. But we never gave them the main thing - the thought process that led to those answers. Imagine you are teaching an architect to build houses by showing him 10 million photos of finished buildings. He will learn that houses have walls, roofs, and windows. He will become a great imitator. But he will never know why the foundation was laid exactly here, why red brick was chosen instead of white, or what mistakes were made and how they were fixed. This is the problem with all modern LLM coders. They learn on the final, polished versions of code. They see the ideal result, but they do not see the path to it. They are great at mimicking good style, because their memory holds hundreds of millions of lines of clean code. But they have a rather superficial understanding of code and zero vision of the project as a whole - they often only implement a local task, and they need a good architect to stand over their soul and make architectural decisions. What if we gave models not just "photos" of code, but the full history of its creation? Every project on GitHub contains the history of the project from the very beginning. Thousands of commits. Each commit is a small decision. Each commit message is a piece of the developer's thought. Imagine seeing the history of the Rust language from 1 file to 10k+ files (number pulled out of thin air). What would the model actually learn from this history? Not just syntax. Not just patterns. It would learn why certain decisions were made. It would see that unwrap() was used first, then replaced with Result and the ? operator - not because it's "better style", but because unwrap() crashed in production. It would see how error handling evolved from "just crash" to "handle properly with context". This is the difference between mimicking code and understanding it. A model trained on such a history would be a mega-expert in Rust. What are the other benefits: Some might say this is too much data. And they'd be right. Most of it is noise - typos, half-finished experiments, abandoned branches. But we don't have to use all of it. We can choose the great ones. Linux. PostgreSQL. Rust. The Go compiler. Redis. These aren't just codebases - they're textbooks written by the best engineers in the world. That's where the real lessons are. Right now, the whole LLM world looks at GitHub and sees only the final code. But we need to look deeper - at the history of that code. If we train models on commit history, we will get not just code generators. We will get experts who understand the evolution, the reasons, and the consequences of every decision. And then AI coders will stop being just imitators. They will become real engineers (though that scares me). And here is what makes programming truly unique. It is the only field where we have a complete, recorded history of a complex system's evolution - from a single file to millions of lines. Every commit is a footprint. Every message is a thought. We have the full DNA of every major project. *This whole article is just my thoughts on this topic. Please don't be too hard on me.*