What Silicon Valley gets wrong about AI

A writer argues that Silicon Valley's faith in AI as a panacea for real-world problems like medicine and housing is misplaced, citing Eroom's Law and the high cost of clinical trials as evidence that intelligence alone does not unblock progress. The piece challenges the assumption that AI will cure disease, noting that drug development costs have risen for decades despite advances in scientific tools.

At a recent dinner, someone from an AI lab asked me, quite bluntly, why I do what I do — why I write about the regulatory bottlenecks to medical progress, why I spend my time on policies related to clinical trials. He looked at me with something best summarized as pity. I told him what I believe: that in the age of AI, medicine will be bottlenecked more than ever by regulation and the grind of clinical trials, and that billions poured into faster preclinical research won’t touch that problem, unsexy as it is. I brought up housing: We’ve had the technology to build better housing for decades, yet it’s more expensive than ever, because housing is a question of political will, not pure capability. He looked at me incredulously. Surely, a smart person like me should know that AI, or, better said, AGI, will be hyperpersuasive soon — it already exceeds professional debaters at persuasion, on a bunch of benchmarks. I said, “Hmm.” He said, “Yes, yes,” with the undertone of a man who knows things deeply, things that mere mortals like me, not being AI lab employees, are simply not well placed to grasp. That night I lay awake, twisting and turning, wondering whether every decision I’d ever made had been, in some way, a mistake. What could I have done better? But as the sun came up, I found my way back to the same thought: No matter how “intelligent” AI becomes, intelligence is often not the main bottleneck to things changing in the real world. It is hard to hold onto that conviction when people smarter than you, with access to privileged information, insist otherwise. But, I shall nonetheless stick to my beliefs. Because the people at these labs made fortunes betting on ideas that once looked insane, the world now takes nearly everything they say on faith. That, I think, is a mistake. It is not a given that AI will solve the problems most people actually care about, including medicine, unless we think about those problems clearly. And at the moment, I don’t believe we are — or at least, not to the extent that we could. My case is simple, and it comes from two directions. One is what I observe in the social dynamics of San Francisco and in how people there talk about all this. The other is what I notice in a field I happen to know something about: medicine. The limits of AI in medicine One of the promises most often invoked to justify AI’s risks is that it will “cure disease.” Every major AI lab CEO says it opens in new tab , and investors seem to agree: Any biotech startup with an AI story attached commands an impressive valuation, even as more conventional biotechs struggle for funding and die. But this whole enterprise, as I have long argued, is bottlenecked by many things that have little to do with “intelligence” as such, and the degree to which that often goes unacknowledged is strange to watch. The evidence is everywhere, if you treat scientific advancement as a rough proxy for intelligence and ask whether it alone unblocks progress. Take Eroom’s Law opens in new tab : The number of new drugs approved per dollar of R&D has fallen for decades, even as our scientific tools have grown vastly more powerful — the exact opposite of what the existence of more raw capability would predict. Or take a company like Adaptimmune, which has brought to market in rare cancers and is two transformative therapies opens in new tab https://www.adaptimmune.com/investors-and-media/news-center/press-releases/detail/271/adaptimmune-receives-u-s-fda-accelerated-approval-of , due to the cost of developing them. nonetheless fighting to stay alive opens in new tab https://markets.financialcontent.com/stocks/article/marketminute-2025-10-20-adaptimmune-plunges-after-announcing-nasdaq-delisting-amidst-compliance-failures-and-cost-cutting-measures In biopharma specifically, one great bottleneck is clinical trials opens in new tab , which today consume something like seven years and cost more than $1 billion per drug. And trials are not a mere formality. They generate precisely the kind of data that would train better models in the first place: . human data, which is ultimately irreplaceable opens in new tab https://www.asimov.press/p/clinic-loop We have extensive evidence that faster clinical trials are important for biomedical innovation, both directly and indirectly, through second-order effects like increasing the risk appetite of those working in the industry and helping better align incentive through fast feedback loops . This ranges from robust economic papers showing that shorter trials massively boost investment in a disease area, all else equal, to the natural experiment of China. China is threatening to race ahead of the U.S. in biotech, with Chinese biotechs encompassing more than a half of big Western pharma opens in new tab licensing deals. A mere decade ago, this percentage was zero. This transformation happened while China remains worse in terms of basic science, mostly due to that allow faster iterative learning using in-human data. regulatory reforms opens in new tab https://worksinprogress.co/issue/the-blood-cancer-that-became-solvable/ Trials can be made shorter, more informative and better, including with AI. One area I am particularly bullish on is surrogate endpoints and biomarkers opens in new tab , where AI could turn discrete, coarse readouts of whether a drug is working into fast, continuous ones. This would in turn optimize trials immensely. For some indications, the gain could be as high as if the right surrogates are found. And this is not even considering the benefits of simply optimizing the trials without necessarily shortening them: Imagine having the right biomarker that tells you whether a therapy is working early enough. 10 times faster and cheaper trials opens in new tab https://ifp.org/proxy-praxis-why-validating-an-endpoint-took-twelve-years/ AI Is Eating SF https://sfstandard.com/obsession/artificial-intelligence/ But even here the binding constraint is not entirely intelligence. Quite often, it is governance. I keep talking to companies trying to build exactly these biomarkers, and what they run into, again and again, is how hard it is to access the underlying data. Some have been waiting a year for the National Institutes of Health to release imaging datasets they can use to produce better biomarkers. If one needs to interact with the Food and Drug Administration to get their endpoint validated, it is even worse: I have previously written about how the validation of bone mineral density for use as a surrogate endpoint in osteoporosis trials took 12 years opens in new tab , despite the fact that the data to support it already existed in full and the analyses done were basically regressions. When people in the AI sphere do engage with regulation and governance in medicine, it often seems to happen in a rather superficial way. I have seen many times the argument that because most drugs fail for lack of efficacy, regulation can’t explain much of the slow-down in medical progress compared to basic science that we have witnessed in the last decades: After all, approving more inefficacious drugs won’t help. This analysis sounds superficially true, but it’s not. When people talk about the importance of regulation in slowing down biomedical progress, they rarely mean the approval decision at the end. It is about the entire process upstream: how easily human data can be collected, and then how easily, once collected, it can actually be used. I know most about clinical trials, but this is not the only area where governance and policy slows progress. Another example is related to how the structure of our patent system shapes how innovative our drugs are. In biomedicine we have a problem called target herding opens in new tab : Most companies chase the same biological targets because they are de-risked, which means we explore far less of the biological space than we could. The cleanest way to see why is through risk. Two kinds matter in drug discovery. Target risk captures the biological uncertainty: Is this protein causally involved in the disease, and will hitting it help a patient without unacceptable toxicity? Molecule design and optimization risk is the downstream problem, conditional on the target: designing a novel molecule against a known target and improving potency, selectivity, safety, and a myriad of other features. That is mostly in the realm of chemistry and is intrinsically far more tractable and predictable; given a validated target, good teams usually reach a decent molecule. 1 readers have shared their thoughts on this story Figure 4. The logic behind target herding The patent system then further disincentivizes companies from taking target risk, because it rewards novel chemical matter, not novel biology. Composition of matter is what most patents protect. That refers to a specific molecule, which means they don’t protect the insight that a target is worth drugging at all. Whoever validates a novel target and implicitly, discovers new biology bears the enormous target risk but can’t capture the reward due to the way the patent system is designed: Once a first-in-class drug posts convincing clinical data, the target is validated, and that validation is essentially public. Competitors then design distinct molecules against the same de-risked target, and each fast-follower earns its own strong patent. So firms end up piling onto proven targets instead of validating new ones, so we get wave after wave of drugs aimed at the same type of biology. The irony is that the AI-for-biology companies drawing the largest rounds are aimed mostly at the same well-served problem of molecule optimization. For example, Chai Discovery — valued at $3.8 billion opens in new tab , enormous by early-stage biotech standards — and Isomorphic Labs are both molecule-optimization engines: They design compounds faster and with more accuracy. This is great, but as I and others have argued before, expanding the frontier of validated targets would be worth more overall to society. Yet this is precisely the part that the patent system does not incentivize. It turns out that even AI-driven companies ultimately respond to economic and legal incentives The same principle applies, I suspect, well beyond medicine. Diffusion is hard, and the rate at which a technology spreads through an economy is not entirely or even mainly predicted by the raw pace of its underlying progress. A capability existing is not the same as a capability being absorbed into institutions, and, perhaps most important, translated into outcomes people care about. And there is expansive evidence this is the case. Walking around the world today, one might notice that it is weirdly unchanged. LLMs can spawn agents and write complex applications, yet something as mundane as customer service still hasn’t been automated well. In fact, often it’s the automated chatbots that are the worst part of the experience. GDP is up, which is good, but it is not dramatically up. And entry-level jobs, despite repeated warnings of an apocalypse, seem largely intact opens in new tab , if not increasing. To many, this is surprising. Just the other day, I was at a conference where someone remarked that if he could have seen today’s AI capabilities a few years ago, he would have been astonished — and would have assumed the world by now would look far more transformed, with much higher GDP growth. It was offered as a fresh observation. But in fact, it has been made before, and this whole trajectory was quite predictable to those thinking about real world barriers to diffusion. I remember a Progress Conference in 2023 where the economist Tyler Cowen made exactly this argument to a room full of AGI-pilled attendees, many of whom were frankly incredulous. The capabilities, he suggested, would arrive faster than the changes they were meant to produce; GDP growth would disappoint relative to the hype, and there would be no massive job displacement. So far, the data seem to bear him out. The people who saw this coming simply have a better feel for friction for how slowly and unevenly a technology actually diffuses into the world. Strangely, it seems that the core San Francisco tech scene, full of exceptionally intelligent people, does not. How can so many smart people share this one blind spot? The making of a monoculture My view is that certain communities in San Francisco which I shall henceforth call “SF people,” even though I do not mean literally everyone or even most people in SF have become something close to a monoculture, and this monoculture has two reinforcing features. The first is that it has become genuinely low-status to suggest AGI might not be wholly transformative, at least not on its own. The people who were early and right about AI’s potential made a great deal of money, and being right about the big thing has hardened into an assumption that they are right about everything. To question the most extreme claims is to be marked as dim, low-IQ, insufficiently “AGI-pilled.” The belief persists that anyone who works at a lab, especially a CEO, holds secrets about the future that no outsider can access. The result is self-censorship: a lot of smart people I know in San Francisco see the bottlenecks to AI’s diffusion and say nothing to seem more “AGI-pilled.” I think the winners of the AI race might be miscalibrated in a way that might have helped in their core endeavour — developing AI — but maladaptive when it comes to judging things beyond that. Delusion is adaptive in startups: An unreasonable conviction is exactly what lets a small group attempt the impossible and occasionally pull it off. Byrne Hobart and Tobias Huber make a version of this case in “ Boom opens in new tab ,” arguing that financial bubbles, usually maligned as destructive, have in fact been an engine of breakthrough progress: What looks like collective delusion is often what funds and forces genuine advancement. But that cuts both ways. Being productively delusional and winning does not make one right about everything else. In fact, the very trait that produced the success might inflate confidence in unproductive ways in other areas. The second feature of this monoculture is a kind of counterfeit contrarianism. Because most of the outside world has long been skeptical of AI’s potential, people in San Francisco feel like brave dissenters. This is despite the fact that, within their own circles, holding those views is the safe and status-conferring position. Believing yourself to be a misunderstood contrarian is intoxicating, and it makes the underlying conviction harder to dislodge. Back when I was an academic, I often noticed that academics were, on the whole, deeply progressive yet thought of themselves as contrarians. This was at least in part because most had come from places more conservative than the university. They’d gather, compare themselves to the outside world rather than to one another, and conclude that their progressivism was somehow brave, despite it being the plain consensus of the room. In my view, SF has done the same thing with AI. In elite tech circles, it is simply not contrarian to take AGI seriously; if anything, the reverse. But because the comparison class is always someone else — for example, a policy person in D.C. — the feeling of embattled dissent survives contact with reality. The question, I suppose, is whether any of this matters. Why would it be bad? For all I know, the AGI god frenzy should continue in the tech bubble. But it’s seeping into general public discourse in a way that might realign priorities for the worse. At the most basic level, I think being closer to the empirical truth is simply better than being farther from it. But there is a more concrete cost: Mistaken beliefs can lead to mistaken priorities. Given that I do expect AI to be disruptive albeit not necessarily in the same ways the SF crowd thinks , this might be a particularly bad time to have bad priorities. Again, I shall bring the example of medicine: We might be aiming our effort at the wrong bottlenecks. In medicine, as I’ve argued, regulatory reform gets a tiny fraction of the attention lavished on AI-enabled biology, and even within AI-enabled biology, the interventions that would actually move the needle are the ones that often seem less funded. And this goes well beyond medicine. We fixate on AGI gods, while more mundane problems pile up unattended. The social fabric is fraying in plain sight. More and more young adults cannot afford a house or sustain a relationship. People pour their youth into gambling apps, some of them now made more powerful by the very AI we keep discussing. People are de-skilling as they hand more thinking to machines. I sometimes say I worry less that AI will take my job than that people will grow too incurious to read at all. Every one of these problems needs people working on it. Being in the thrall of digital deities that will magically dissolve all our problems pulls attention away from the simpler, more human troubles that are already here. Ruxandra Teslo is a writer at Works in Progress magazine and cofounder of Clinical Trial Abundance project. A version of this article previously appeared on Ruxandra Teslo’s Substack opens in new tab https://substack.com/@ruxandrabio .