# Nobody Can Explain Why This AI-Designed Chip Works. They Built It Anyway > Source: > Published: 2026-07-27 19:01:01+00:00 Look at a modern radio-frequency chip designed by an AI, and you won’t see a circuit. You’ll see something that looks like a QR code — a chaotic scatter of metal pixels with no visible logic. It has no symmetry, no repeating blocks, nothing a trained engineer’s eye can follow. And it works. Tested, verified, shipped. The signal loss is lower than anything a human team has designed. The engineers who commissioned the chip can prove, mathematically, that it performs. What they can’t do is point to a section of the layout and explain *why* that particular arrangement of metal makes it better. The gap between knowing it works and understanding why it works is not merely a curiosity. It’s the same gap opening up across almost every frontier of AI right now: in physical hardware, in structural engineering, in how AI agents talk to each other. This piece is about that gap, and about the quiet shift in what “trusting a machine” now means. To see why this happens, it helps to understand, loosely, how these chips are designed in the first place. Traditionally, chip design is an apprenticeship. Engineers use templates, rules of thumb, and years of trial and error, and they build in straight lines and symmetry because that’s what a human mind can hold, trace, and debug. But none of that is a law of physics. Symmetry is a limit of *our* working memory, not a requirement of electromagnetics. So researchers tried something different: strip away every human template and let an algorithm design the chip from scratch, using reinforcement learning. The setup is close to training a dog with treats, just run millions of times in a simulation instead of a living room. The AI places tiny pieces of metal on a grid. A physics simulation scores the result. Good placement, reward. Bad placement, penalty. Repeat that a few million times in a matter of hours, and the AI converges on layouts that no human would have tried — because no human would have thought to try them. What comes out the other end isn’t neat. It’s closer to visual noise. But it’s noise that outperforms decades of human-refined design, because the AI was never constrained by what a person could follow on a schematic in the first place. It’s worth pausing on *why* this happens at all — because it’s not a failure of effort. It’s structural, and it comes from two things stacked on top of each other: the size of the space these systems search, and the shape of the function they end up computing. **The space is too big to see.** A human being can intuit two or three spatial dimensions comfortably — that’s the world we evolved to navigate. The design space an RF chip algorithm explores, or the space a generative-design algorithm searches when it carves a bracket, has thousands of independent variables, each one a “direction” the system can move in at the same time. That’s not complicated the way a hard math problem is complicated. It’s that the space has more directions in it than a human mind has slots for, full stop. This is exactly why the tools doing the searching are deep reinforcement learning and Monte Carlo-style physics simulations: run millions of trial placements, score each one, keep what scores well. The system doesn’t need to *see* the whole space to search it efficiently. We do, and we structurally can’t. **The function is too deep to write down.** Separately, there’s the shape of what a trained neural network actually *is*: one enormous nested mathematical function, built by stacking dozens or hundreds of simple layers — each just a multiplication, an addition, a small nonlinear twist — on top of each other. No individual layer is mysterious. Multiply, add, squash, repeat. But compose that operation a few hundred times, and the resulting function has no closed form left — no equation you could write on a whiteboard and simplify down. It isn’t encrypted, and it isn’t hiding anything on purpose. It’s just structurally too deep to compress into something a person can hold in their head at once. Billions, sometimes trillions, of parameters get tuned to fit that shape, each one playing a role closer to a synapse than a line of code. There’s a well-known trade-off in machine learning between a model’s interpretability and its capability: decision trees and linear regressions stay fully readable precisely *because* they’re too simple to do the job. The models powerful enough to design a chip, or spot a tumor better than a radiologist, buy that power at the cost of legibility. There’s a deeper mismatch underneath the trade-off. Human reasoning, law, and science are all built around *causal* explanation — we want to know *why*, so we can assign blame, build trust, or learn something transferable. A neural network doesn’t reason causally at all. It operates purely on high-dimensional correlation: patterns folded through a space too large to name, that happen to produce correct outputs on the other side. We’re not bad at reading it. There’s decreasingly less of a “why” in there to read. Once you see this pattern in silicon, it starts showing up everywhere. **In physical structures**, Engineers now provide AI with a block of raw material and a set of constraints — where it needs to bolt down and what forces it must withstand — allowing it to remove everything that isn’t structurally essential.**The results look like bone (** **In communication between AI agents**, something even stranger is happening: they’re starting to skip language entirely. When one AI reasons through a problem, its internal state — the “hidden state,” in machine learning terms — is enormously rich: many competing possibilities held at once, in a form far denser than any sentence could carry. Normally, that has to get compressed down into words, one token at a time, mostly for our benefit. A framework called Interlat lets one AI agent skip that compression step and hand its raw hidden state directly to another agent — no text, no translation. The result is communication compressed by roughly 24x, and, more strikingly, agents that retain multiple possible ideas at once instead of being forced to commit to a single linear thought the moment they “speak.” Three completely different domains. Same underlying move: remove the requirement that a human be able to follow along, and performance jumps. Here’s the part that matters more than any single example. For most of engineering history, trust was built on tracing the logic. If a bridge held weight, an engineer could show you the load calculations for every truss. If a chip worked, someone could walk you through the schematic, gate by gate. Understanding *how* something worked was the price of admission for trusting *that* it worked. That’s the assumption breaking down. Nobody involved in building the QR-code chip, the bone-shaped bracket, or the telepathic agent pair can walk you through the logic the way an engineer once could. So the entire discipline has quietly pivoted from checking the reasoning to checking only the result. Run the simulation. Run the clinical trial. Run the stress test. If the output holds up, ship it — regardless of whether anyone can explain the path that got there. We’ve gone from being the architects of these systems to being their auditors. We don’t design the logic anymore. We grade the homework after the fact. A model simple enough for a person to trace end-to-end is, by definition, too simple to search a space with thousands of dimensions. The readable version isn’t a rough draft of the powerful one; *Instead,* *it’s just a weaker model.* Illegibility isn’t a side effect anyone is trying to patch out, either — in every case here, it’s the actual mechanism producing the gain. The chip got better *because* it wasn’t constrained to be readable. So we’re not on our way back to understanding these systems. We’re on our way to getting better at grading them — sharper tests, tighter simulations, harder audits. Which leaves the question none of this actually answers: *once verifying the result is all we have left, how much verification is enough before we’re willing to call something trustworthy? (Because, you don’t know how it works)* [Nobody Can Explain Why This AI-Designed Chip Works. They Built It Anyway](https://pub.towardsai.net/nobody-can-explain-why-this-ai-designed-chip-works-they-built-it-anyway-b7a712954e35) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.