Full Blog on : https://www.iamleopard.com/blog/don-t-look-up-what-happens-when-the-ai-race-becomes-more-important-than-the-warning
The most unsettling part of the current AI debate isn’t that someone
thinks AI could become dangerous.
It’s that some of the people building it are now asking whether we
should slow down — while governments and markets have powerful reasons
to keep accelerating.
There is a scene in Don’t Look Up that has stayed with me.
A scientist discovers that a comet is heading toward Earth.
The mathematics are straightforward.
The danger is enormous.
The solution is technically possible.
And yet, somehow, the hardest part isn’t understanding the comet.
It’s convincing everyone else that the comet matters.
The scientists try.
Politicians calculate.
Businesses see opportunities.
The media turns catastrophe into entertainment.
The public scrolls past it.
Eventually, the question stops being:
Is the comet dangerous?
It becomes:
What happens to everyone’s incentives if we admit that it is?
That is why the movie feels strangely relevant to the current AI debate.
Because in September 2026, the people building frontier AI are openly
debating whether the technology is moving faster than our ability to
evaluate and control it.
And the political response is pointing in the opposite direction:
We cannot afford to lose the AI race.
The AI industry has reached its “Don’t Look Up” moment
Anthropic CEO Dario Amodei recently published an essay arguing that the
development of frontier AI should be deliberately paced so that safety
and evaluation can keep up with capability development.
The important distinction is that this isn’t necessarily a call to stop
AI.
It is an argument about pace.
The basic idea is simple:
If capability improves faster than our ability to understand and control the system, the gap itself becomes a risk.
Amodei’s proposal includes stronger evaluation, independent testing,
coordination among frontier AI companies, and eventually broader
international cooperation.
That would already be a significant position coming from an AI safety
researcher.
But it is more significant when the conversation is happening inside the
companies building the frontier.
Other technology leaders have also expressed support for slowing or
pacing parts of frontier development.
Then came the political counterargument.
“Whoever wins AI wins”
Donald Trump’s position is fundamentally different.
His argument is not primarily about whether AI risks exist.
It is about what happens if the United States voluntarily slows down
while another country continues accelerating.
From that perspective, AI is not merely a software technology.
It is strategic infrastructure.
It affects:
economic productivity
military capability
scientific research
industrial competitiveness
semiconductor demand
energy infrastructure
national security
geopolitical influence
And China is the obvious competitor in that calculation.
So the argument becomes:
If AI leadership matters this much, can the United States afford to put itself at a competitive disadvantage by slowing down?
This is a much harder question than “Is AI safe?”
Because both sides can be right about different things.
The paradox of the AI race
Imagine two countries standing at a starting line.
Country A develops a more capable AI system.
Country B sees it.
Country B accelerates.
Country A responds.
Companies compete for researchers.
Investors pour money into infrastructure.
Governments provide incentives.
Data centers expand.
Models become more capable.
Agents become more autonomous.
And eventually, slowing down becomes politically difficult.
Why?
Because slowing down only works if the other players also slow down.
This creates a classic coordination problem.
If everyone agrees to move carefully, everyone might benefit. But if you believe your competitor will continue accelerating, slowing
down can feel less like safety and more like surrender.
That is the geopolitical version of the AI race.
AI doesn’t need to be evil
This is where Don’t Look Up offers an important lesson.
The comet doesn’t hate humanity.
It doesn’t have political beliefs.
It doesn’t want to destroy civilization.
It simply follows physics.
The danger comes from the interaction between the object and the system
around it.
AI could present a similar systems problem.
We don’t necessarily need an evil machine for things to go badly.
We need only:
increasingly capable systems
poorly understood behavior
large-scale deployment
insufficient testing
competitive pressure
economic incentives
rushed decision-making
humans assuming everything will probably be fine
That combination can create serious risk without anyone explicitly
intending harm.
The part engineers should pay attention to
For developers, discussions about AI safety can sometimes sound abstract.
“Alignment.”
“Frontier risk.”
“AI governance.”
“Existential risk.”
These phrases can feel like something that belongs in policy conferences
rather than GitHub repositories.
But the underlying problem is familiar to engineers.
We already know what happens when software is deployed faster than it
can be tested.
You get:
bugs.
security vulnerabilities.
unexpected interactions.
production incidents.
Now increase the complexity.
Give the system tools.
Give it access to APIs.
Give it memory.
Allow it to execute code.
Connect it to other agents.
Let it operate continuously. Then tell the team:
“We’ll monitor it.”
That starts sounding much less reassuring.
Production AI is a systems problem
This is one of the biggest lessons from actually building AI-powered
products.
A production AI system isn’t just a model.
It is:
model + prompts + tools + data + permissions + APIs + users + business
logic + monitoring + failure modes.
Consider an AI agent that can read a CRM.
That’s one level of risk.
Now give it permission to modify customer records.
Different risk.
Now allow it to send emails.
Different risk again.
Now allow it to create accounts, spend money, modify production
infrastructure, and publish content without approval.
The model may be exactly the same.
But the system is radically different.
That’s why AI safety cannot be reduced to asking whether a model is
“smart” or “aligned.”
The surrounding architecture matters.
Least privilege applies to AI too
Security engineers have known this principle for decades:
Give a system only the permissions it needs.
AI agents shouldn’t be treated differently just because they communicate
through natural language.
If an agent only needs to read customer information, don’t give it write access.
If it needs to send an email, don’t give it access to your entire communication platform.
If an action can cause financial, legal, or irreversible consequences, introduce an explicit confirmation layer.
The more powerful the action, the stronger the control should be.
This is not anti-AI.
It’s good engineering.
Humans still matter
There is a temptation to interpret “AI automation” as:
Remove humans from the loop.
That isn’t always the right goal.
For low-risk tasks, removing manual approval can be fantastic. Generating a draft?
Automate it.
Summarizing documents?
Classifying routine data?
But for high-impact actions, the question should be:
What happens if the model is wrong?
If the answer is “nothing important,” automate aggressively.
If the answer is “we could lose money, expose private data, damage a
customer relationship, or take an irreversible action,” then human
oversight becomes much more valuable.
The right architecture isn’t “human everywhere.”
It is human oversight proportional to the consequences of failure.
Regulation isn’t a magic solution either
It’s tempting to turn this into a simple argument:
AI is dangerous, therefore regulate it.
But regulation can create problems of its own.
Poorly designed regulation can:
make compliance unaffordable for startups
favor incumbent companies
slow useful research
freeze outdated assumptions into law
create conflicting requirements across countries
push development into less transparent environments
And there is another difficult problem.
If one country slows down while another doesn’t, the competitive pressure returns.
That’s why frontier AI governance is not just a national policy problem.
It is also a coordination problem.
Maybe “slow down” is the wrong phrase
When people hear:
“Slow down AI.”
they often hear:
“Stop innovation.”
But those aren’t necessarily the same thing.
A better interpretation might be:
Don’t let capability growth consistently outrun our ability to
evaluate the consequences.
Think about aviation.
We didn’t stop building faster aircraft because aircraft can crash.
Instead, society built systems around aviation:
testing
certification
inspections
redundancy
air-traffic control
pilot training
incident reporting
emergency procedures
We didn’t eliminate progress.
We built infrastructure around progress.
AI may need a similar approach.
The race may not really be USA vs China
This is perhaps the most interesting way to frame the whole debate.
We keep talking about:
USA vs China.
But another race is happening underneath it:
capability vs understanding.
Can our ability to build increasingly capable systems advance faster
than our ability to understand what those systems are doing?
That’s an engineering question.
Imagine building a massive application without reading most of the code.
Then giving it access to your production database.
Then giving it administrator privileges.
Then allowing it to modify its own behavior.
Then deploying it globally.
Then saying:
Any experienced engineer would immediately ask:
Where are the boundaries?
Where are the tests?
What happens when it fails?
Who can stop it?
Can we roll it back?
Can we explain why it did that?
Those questions become more important, not less, as AI becomes more
autonomous.
This is where the AI engineering mindset matters
At Leopard, we spend a lot of time thinking about AI not as a magical
chatbot, but as a component inside real software systems.
Our [AI & Machine Learning work] focuses on practical applications of
AI: structured context, retrieval, automation, agentic workflows, and
systems that connect models to actual business processes.
Our [Tryneth case study] is a useful example of that approach.
The important part isn’t simply that an AI model exists.
The important part is everything around it:
orchestration
APIs
data
usage tracking
authentication
permissions
business logic
monitoring
reliability
human interaction
That is where AI becomes software engineering.
And that is also where AI safety becomes practical.
What developers can do today
We don’t control national AI policy.
We don’t control frontier model roadmaps.
We don’t control geopolitical competition.
But developers do control the systems they build.
Here are a few principles worth adopting.
A model can be impressive and still be wrong.
Validate important outputs.
Use schemas. Check assumptions.
Don’t let generated text automatically become trusted application state.
Use least privilege. Separate read and write capabilities.
Restrict tools by task.
Require explicit confirmation for dangerous actions. Log:
model calls
tool calls
important decisions
failures
retries
latency
token usage
cost
user approvals
If you can’t reconstruct what happened, debugging an autonomous system becomes extremely difficult.
A model benchmark can tell you something about the model.
It cannot tell you everything about your application.
Test the complete workflow.
Test adversarial inputs.
Test tool misuse.
Test permission boundaries.
Test failure recovery.
Test what happens when an external service is unavailable.
A good AI system should have a safe state.
If the model fails, the application shouldn’t necessarily fail catastrophically.
Fallbacks matter.
Timeouts matter.
Rate limits matter.
Human escalation matters.
Kill switches matter.
The irony of the current moment
For years, people warned: AI might become too powerful.
Then AI became more powerful.
People said:
We need AI safety research.
Safety research expanded.
Then companies began building increasingly autonomous systems.
Now some people inside the frontier AI industry are saying:
Maybe we should pace development.
And the political counterargument is:
What if someone else gets there first?
That’s the paradox.
The more strategically valuable AI becomes, the harder it becomes to
slow down.
The harder it becomes to slow down, the more important safety becomes.
And the more important safety becomes, the more expensive it can feel to
prioritize it.
That’s the loop.
We shouldn’t look away
I don’t think AI is literally the comet from Don’t Look Up.
That analogy would be too simplistic.
AI can create enormous benefits.
It can accelerate scientific discovery.
It can improve software development.
It can automate repetitive work.
It can make sophisticated tools accessible to smaller teams.
It can help researchers and engineers solve problems that were
previously too expensive or time-consuming.
The answer isn’t to panic.
The answer is to engineer responsibly.
Testing.
Monitoring.
Red-teaming.
Independent evaluation.
Security.
Governance.
Transparency.
Human oversight.
International coordination.
And, where necessary, pacing.
The question isn’t whether we should build AI
We should.
The real question is:
Can we build increasingly powerful systems without becoming less
capable of controlling them?
That’s the conversation worth having.
Not:
“AI will destroy humanity.”
“AI will solve everything.”
But:
How do we make powerful AI useful, reliable, observable, secure, and
controllable?
That’s an engineering problem.
And unlike the comet in Don’t Look Up, we still have the opportunity
to do something about it.
So let’s not look away.
Let’s look up.
Related Leopard projects
If you want to explore the engineering side of this topic: AI & Machine
Learning
— our approach to building practical AI-powered systems.
Tryneth — an example of
AI-agent orchestration inside a production SaaS environment.
Work & Case Studies — more examples
of software, AI, and product engineering.
Leopard — learn more about our work.
Discussion
What do you think?
Should frontier AI development be deliberately paced so that safety
and evaluation can catch up?
Or does slowing down create an unacceptable strategic disadvantage in
the global AI race?
I’d especially like to hear from developers building AI agents and
AI-powered products.