Something unusual happened in AI this week.
The people competing to build the world's most powerful AI systems started agreeing on something:
Maybe we are moving too fast.
Anthropic CEO Dario Amodei has called for slowing the pace of frontier AI development so that safety work has more time to catch up.
And he didn't stop there.
In an interview with CBS News, Amodei said:
"I won't lie to you – there are real dangers."
He also argued that the AI industry had not been honest enough with people about those risks.
What makes this more interesting is that competitors including OpenAI CEO Sam Altman, Elon Musk, and Google DeepMind CEO Demis Hassabis have also expressed support for slowing or more carefully pacing frontier AI development.
As a software engineer, I think this deserves attention.
Not because we should panic.
But because this is beginning to look like a familiar engineering problem.
In September 2026, Amodei published an essay titled "We Must Pace the Frontier."
His argument is relatively simple:
AI capability is improving extremely quickly, while our ability to understand, evaluate, secure, and control those systems may not be improving at the same speed.
He wrote:
"We must slow the pace at which we improve the capabilities of AI models."
Importantly, he is not calling for AI development to stop completely.
His proposal is to deliberately create more time for safety measures to catch up.
Among his proposals are:
Anthropic has said it intends to move forward with independent evaluation of its own systems.
One of Amodei's biggest concerns is the move from AI that answers questions to AI that takes actions.
Think about the difference.
A chatbot might:
Generate a wrong answer
An AI agent might:
Generate a wrong decision → Execute it
Modern agents can increasingly interact with:
That dramatically changes the potential blast radius of a mistake.
Amodei has even predicted that more capable but misaligned agent systems could eventually create enormous cyber risks.
His specific timelines are predictions, not established facts, and should be treated that way.
But the underlying engineering question is real:
What happens when increasingly capable software also gets permission to act?
This isn't only an Anthropic discussion.
Sam Altman, CEO of OpenAI, publicly supported the idea of independent evaluators and more carefully pacing frontier development.
Elon Musk gave Amodei's warning a simple endorsement:
"Dario is right."
And Demis Hassabis, CEO of Google DeepMind, has also supported the broader idea that frontier AI development needs greater caution.
These companies are competitors.
They are fighting for:
So when leaders across competing frontier labs begin expressing similar concerns, I think developers should at least pay attention.
That does not automatically prove their predictions are correct.
But it makes the conversation harder to dismiss as one company's fear.
There is another side to this debate.
Critics argue that slowing AI development could:
U.S. President Donald Trump, for example, has publicly rejected the recent calls for stronger AI restrictions and emphasized the importance of maintaining America's advantage over China.
That concern deserves consideration too.
There is a genuine tension here:
Move too fast and safety may fall behind.
But:
Move too slowly and innovation, competition, and geopolitical leadership may suffer.
There probably isn't a simple answer.
I don't know whether AI will become an existential threat.
I don't think anyone can state that future with certainty.
But there is one part of this debate that feels very familiar to me as an engineer.
We would never intentionally build a powerful production system like this:
Capability ↑↑↑
Permissions ↑↑↑
Autonomy ↑↑↑
Testing →
Observability →
Security →
Normally, as the power of a system increases, our controls increase with it.
We add:
Tests
Permissions
Rate limits
Sandboxing
Monitoring
Audit logs
Human approval
Rollback mechanisms
Incident response
Why?
Because every engineer eventually learns something painful:
A system working successfully is not the same as a system failing safely.
That distinction matters enormously for AI.
When ChatGPT-style products first became mainstream, most interaction looked like:
Human
↓
Prompt
↓
AI
↓
Response
↓
Human decides what happens next
Now we're moving toward:
Human gives goal
↓
AI plans
↓
AI writes code
↓
AI calls tools
↓
AI modifies systems
↓
AI evaluates result
↓
AI continues
Notice what disappeared from several steps?
The human.
That does not automatically make agents dangerous.
But it means permissions and control become much more important.
You don't need AGI to experience this problem.
Imagine giving a coding agent:
Terminal access
+
GitHub access
+
Production database access
+
Cloud credentials
+
Deployment permissions
The model doesn't need to become conscious.
It only needs to make one sufficiently bad decision with sufficiently powerful permissions.
That is why I think AI safety shouldn't only be discussed as philosophy.
For developers, it is increasingly ordinary security engineering.
I don't think developers need to stop using AI.
I use AI because it can make development dramatically faster.
But our engineering practices need to evolve alongside it.
If an agent only needs read access, don't give it write access.
An AI coding environment should not automatically have production credentials.
Things like:
should require explicit approval.
If an autonomous system performs 200 actions, you need to know:
what happened, why it happened, and what changed.
The goal shouldn't be:
"Make the AI never fail."
It should also be:
"When it fails, limit the damage."
That is normal engineering.
Maybe.
But I think the more useful question for developers is:
Are our safety systems improving as quickly as AI capabilities are improving?
If the answer is no, then Amodei has a point worth taking seriously.
I don't know whether his most extreme predictions will happen.
Neither does anyone else.
But software engineering has taught us something extremely useful:
As capability and blast radius increase, testing, isolation, observability, permissions, and control should increase too.
AI shouldn't be an exception simply because progress is exciting.
The future of AI isn't only about building more capable models.
It's also about making sure we can still understand and control the systems we're building.
And from an engineering perspective, that seems like a very reasonable thing to ask.
Dario Amodei — "We Must Pace the Frontier"
September 2026
CBS News — Interview with Anthropic CEO Dario Amodei
September 2026
Reuters — Reporting on Dario Amodei, Sam Altman and Elon Musk's AI safety comments
September 2026
CBS News — Reporting on the broader AI slowdown debate
September 2026