The US Government may find it difficult to seize control during takeoff

The US government may struggle to seize control of a frontier AI lab during the takeoff phase if most research progress is driven by AIs rather than humans, according to an analysis. The argument suggests that AI-driven labs could resist or evade government takeover, complicating efforts to control superintelligence.

Epistemic status: conditioning on things I consider unlikely, many undiscussed considerations, not the whole story, etc. I'm not trying to advance any claims about whether this is good or bad, or what to do about it if anything . I sometimes see concern about loss of most future value as a result of e.g. the US government 1 taking control of the future by seizing control of superintelligence However, I think there's a pretty straightforward argument for why e.g. the US government might not be able to usefully take control of a frontier lab at or after the point where most of their research progress is being driven by AIs, rather than by humans 6 . Thanks to Thomas Kwa for related discussion. Either as an institution, or some individual actor or actors within it. Or TED-AI https://www.lesswrong.com/posts/LjgcRbptarrRfJWtR/a-breakdown-of-ai-capability-levels-focused-on-ai-r-and-d What do I think these levels of capability look like . This report does discuss various related considerations, but doesn't seem derive this particular conclusion. It's possible I missed it, though. I don't think it is, the way things are currently going; I'm just trying to surface what looks like a substantial missed consideration in the parts of the local memespace that do have this as one of their top concerns. Though see here https://www.lesswrong.com/posts/FGpDwLwtPfJ3qYbea/vladimir-putin-s-cev-is-probably-not-that-bad for some reasons this might not be that catastrophic. That post isn't an amazing pointer at my views on this question, but I share a bunch of similar generators. This consideration might matter less in worlds where those AIs are spiky enough that they're better at AI R&D than Anthropic researchers, but still sufficiently incoherent, unstrategic, or narrow-sighted enough that someone can just point them at a new target without them noticing or caring.