When I talk to people about what might be done about AI threatening approximately everything that everyone cares about, I notice a common oddity in their resistance to a variety of ideas. They seem to take for granted that certain entities—especially Trump and China—would be acting against their own interests, were they to cooperate or take proactive action to avert the building of dangerous AI.
The speaker often thinks there is a fairly substantial risk of the AI thus produced killing or disempowering everyone, including Trump and China. And I imagine in a situation where a certain course of action were going to produce a 20% chance of Trump being shot in the head or China being heavily nuked, that these parties would actually be considered to be ‘following incentives’ to avoid it. Yet they talk as though the idea of Trump or China responding to such risks is akin to the idea of these parties suddenly becoming zealous proponents of universal selfless love randomly.
It’s like while believing in the risk, they also kind of believe that it’s a totally uncompelling story that nobody in real geopolitics would ever be touched by.
Or that these parties actual utility functions are ‘be the bad guy’, which, you know, sometimes involves extreme selflessness as long as it puts everyone else at risk too.
Any other hypotheses for what’s going on?