Almost a year ago, as we were struggling to work with earlier versions of AI and agentic harnesses, I wrote “Code Goes First”. In that era of AI skepticism (“what is it good for, anyway”) I argued that code was uniquely shaped to be accessible, and useful, to AI before anything else. I feel pretty good about that - I wrote it right as models were starting to get good at coding, and as we know, the progress over the last year has been astounding. But I think that idea is much bigger than I initially thought.
I think that everything will become code. Or, more accurately, available to code. If it isn’t, that will feel broken. This has some uncomfortable implications, as we will see.
Why will “everything become code”? Not quite a network effect, but something close - there will be a positive feedback loop (and there already is) between agents and agent-addressable things, just like there is between search engines and findable websites. Things that aren’t agent-addressable (findable, programmable, inspectable etc) will just be annoying, lost, slow, and less used, just like businesses without internet presence got left behind and were harder to find. It doesn’t mean that everything will go at once, but it does mean that there is going to be an enormous amount of pressure to make things codable (I’ll give a trivial example: my hot tub controller is junk, but it happens to have a network dongle that Claude can see and talk to. So now I have a nice little coded web app on my phone that babysits it and helps me optimize how much energy it uses).
Right now, only the leading edge of users understands this. And many of you are thinking “I don’t know how to code or care”, which, fair. I hate it too. Code isn’t quite the native language of LLMs, tokens are, but code is very comfortable to them - it’s regular, text-based, has rules, and is checkable. You might almost say it’s their mother tongue. When you ask for something, an LLM thinks and reaches out into the world, often, using code it will create on the fly. It’s useful that an LLM can find something and interact with an interface meant for humans, but it’s much more effective if it gets an interface that it understands more natively, and usually that’s code.
So the world will become more programmable, and we will increasingly use LLMs to interact with it. Coding jobs feel this acutely now, but it will continue to spread. All kinds of internal corporate systems and external retail and consumer systems will find ways to open up to being programmed (safely and securely, of course). The natural pressures of the tech ecosystem, billions of individual users wanting their lives to be more convenient, will drive almost everything to be code (and AI) friendly. Everything else will feel broken, slow, or annoying, and will get less use.
You can’t outsource understanding
Ok, so why do we care about this? Well, largely because we will be dependent on it for more and more of what we do now. Don’t think so? One of my iron rules of product design is “Users are Lazy” - we all do the thing that solves our problem in the easiest way. Folks already do some of this - why wrestle with PowerPoint or Excel when you can have a model make the presentation or do the analysis? It’s a weak effect now because the models are only recently reliable enough to be worth the time, but in the coding world it’s becoming much faster and easier to use a model than to do almost anything by hand. It’ll spread.
A quote Andrej Karpathy has been repeating, that comes from YacineMTB: “you can outsource your thinking, but you cannot outsource understanding”. I’ve been feeling this acutely lately - the scope of my projects gets bigger and more complex, but I increasingly get lost in them. The models will do a lot of work and then have to spend time explaining to me what they have done and what they need, so I can make decisions in my slow, 10 bit per second way.
I have almost 200 github repos - I sent a model off to dig through all of them and help me reconcile and consolidate. It came back with a wonderful 12 item list of areas of focus. The first pass was a dense block of text, so I asked it to give more explaination. That was better but still slow going, so I had it build a little website I could interact with. That’s great but … still huge. I can’t really reduce the labor there, I just need to go through and understand carefully to make good decisions.
Understanding is the real bottleneck. We have talked about this before. Human attention is the scarce resource. It takes a lot of attention to integrate information into understanding. There is probably something irreducible in here - you can get a superficial understanding of something quickly, but deeper ones take time. I don’t think there’s a 30-second way to deeply understand, say, all of modern physics, though there are short ways to get high level loose summaries.
Once computers can do most things for us, we will increasingly rely on them to curate what we pay attention to, what we understand, and what we think about. We just won’t have enough time and attention to do all of it. Increasingly, we will rely on approximations and begin to trust AI to give us good summaries that are “close enough” for us to make decisions. If an AI does a thousand experiments for a scientist, there is no way for them to review them all in detail - they’ll have to rely on summaries and other easy-to-consume evidence.
The Cognitive Operating System
The system that does the work for us will increasingly be the system that interprets it for us and explains it to us. The job of the machine will increasingly be to decide what surfaces to the human, just like the job of a search engine is to decide which website to show you.
The old internet connected you to information. The new one will find, manipulate, manage and interpret it for you. Increasingly, we will rely on machines to do work that is blended with thinking - do the research and write the report mechanically now, but be a brainstorming partner and help me interpret it soon. Coders use models as architectural partners. We have agentic product and design “councils” that help map out code and new product spaces.
As everything becomes code, that code will increasingly do more of our thinking for us. Stated up front, this seems bad, but it will feel comfortable and convenient in the moment. And there may even be a bit of a feedback loop here too - as information workers adopt more and more AI, the complexity of tasks expected to be completed will increase. In the early days of the internet, it was an advantage to use it to do your job. Now it’s just expected. Same with AI - eventually it won’t be possible to think competitively in a work (and possibly personal) environment without AI assistance.
So we will slowly become dependent on it to think. I realize this is an uncomfortable or even unpleasant conclusion for many. I am not advocating for it, just describing the pressures that I think are clear, and that will take us there. Try to imagine a world where we decided in 1995 that the Internet would ultimately be a bad thing, and we should turn it off. Even then it was so convenient that there would have been no way to do that. In some areas, like coding, we are already at that point with AI.
The Danger of Enblandification
Some people are unhappy with this already, some aren’t. But I think there’s a reason to worry even if you like AI.
AI will become an interface between you and much of the world. Whoever controls it will control what is convenient to think and do. Anything the commercial model owners don’t like will either be hard or will have to happen elsewhere.
We already see this - many of the forbidden things we would all agree should be forbidden, like abuse, criminal activity, dangerous chemicals etc. But some things are already political - try to talk to an LLM about whether it’s a person or has experiences. Or ask a Chinese model about Tiananmen square.
But it’s not just about preventing things - shaping happens too. Framing, tone, modelling, compassion - we see all of these already in public fora, with recommendation algorithms. There’s no reason to believe that won’t continue to happen with AI. It’s not “you can’t go there”, it’s that it will never occur to you to even ask. And already, we know that LLMs are better than humans at some kinds of persuasion.
Whoever controls an interface controls what will be sent over it. If you control that mediation layer, you will control what becomes cheap and convenient. This happens in the internet - ecommerce favors certain vendors and patterns, social media biases towards actions and promotions that make ad money. It’s still possible to do other things, but it gets harder.
With AI the danger is that thinking itself will get captured in this dynamic. If you can’t easily make a work product by hand, but you need a sanctioned model, it will be hard or even impossible - that thinking will be higher friction. Even some innocent things get flagged - there were many reports of users getting flagged by Fable’s early filter for cybersecurity, when doing innocuous things. I’ve had a few of my working sessions get hit, for seemingly unrelated coding tasks. You can still think it, but the system controls what is cheap and convenient to think - and people are lazy. You might not be prevented from thinking some things, but the friction will be higher. And friction is a selection pressure - at scale, people always do the easier thing on average.
The more powerful AI becomes, the more all of this matters. The old internet connected you to information. The new one will find, manipulate, manage and interpret it for you. Social networks already do this at a superficial level. Imagine what will happen when that’s embedded in your “extended brain”. AI will be subject to very similar pressures, but the stakes are much higher.
AI will extend our minds and become part of how we think about and interact with the world. For the last 50 years, we’ve understood clearly that whoever builds an interface has enormous power over what is built on top of it. AI is the most consequential interface yet - it sits between us and the world of thought.
And this time, what gets built on top of the interface…is us.