For Startups Facing the AI Safety Uproar, Cybersecurity Looms Larger than Existential Risk

OpenAI and Anthropic are positioned to build major cybersecurity lines of business as founders report that frontier AI models are highly effective at finding and patching system vulnerabilities, according to an Anthropic threat intelligence report published Thursday. Modal co-founder Erik Bernhardsson said his AI infrastructure company uses the models as a partial replacement for expensive external security consultants, adding that "where there's a vulnerability in a system you can now hack something in a few hours." The shift comes as the model providers need to sustain revenue growth ahead of their IPOs and see signs that established categories like coding may not be enough.

The Week In Short The OpenAI / Hugging Face hack stirred up discourse about the dangers of rogue AI , while the model-makers move to bring in big cybersecurity bucks. Dimension founder Zavain Dar explains what Silicon Valley can learn from Chinese AI companies on the podcast. Inference provider Baseten scoops up an early-stage infrastructure startup. Mistral brings in billions but can’t shake its identity crisis. A new Ramp report shows AI spending per employee dipped in August. Coding assistant Cognition rakes in billions. Meta launches its Instinct-competitor Muse . The DOJ turns its attention to Nvidia’s Groq acquihire . Biden administration officials tell their side of the infamous a16z meeting in 2024 that led firm leaders to endorse President Trump. OpenAI & Anthropic Could Profit from Cyber-Defense Tools. The Irony Is Lost on No One. The uproar over AI safety that dominated the tech conversation this week was something to behold: whistleblowers https://www.axios.com/2026/09/09/anthropic-researcher-ai-warning-interview declaring the end is near, politicians in a panic https://www.politico.com/news/2026/09/09/ai-safety-guardrails-congress-01069791 about how to handle what will likely be the most consequential issue of their careers, and many in the industry deeply uncertain about the level of risk that further model improvements might bring. The fears of an individual young researcher https://x.com/hilbertspaess/status/2097476196791709843 who worked at Anthropic for less than two months have gotten far more airtime than they deserve. Whatever your level of p doom , though, for anyone not directly involved in frontier model development or high-level policy, there’s a simple way to think about all this: it’s a cybersecurity problem. That’s what was on the minds of founders we spoke with this week. The Anthropic threat intelligence report https://www.anthropic.com/threat-intelligence-report-september-2026 published Thursday detailing bad actors’ efforts to use Claude in nefarious ways also underscores the point. Threat intelligence has been a tech growth industry for decades. Top-tier AI models are very, very good at finding and patching — or exploiting — vulnerabilities in computer systems. Cyber-defense is thus set to become a major line of business https://www.politico.com/news/2026/09/10/sam-altman-pitches-utilities-on-ai-grid-defense-01070425 for OpenAI and Anthropic. It’s coming at a crucial moment for the model providers, which need to sustain vertiginous revenue growth ahead of their IPOs but are starting to see signs that established categories like coding may not be enough. However you might feel about buying solutions from the people who caused the problem, you might not have much choice. “Where there’s a vulnerability in a system you can now hack something in a few hours,” said Erik Bernhardsson , the co-founder of Modal. “We’ve been running those models internally; they’re very good at finding things.” Bernhardsson said his AI infrastructure company has been using it as a partial replacement for expensive external security consultants. Other founders concur, and say they’ve been impressed with the capabilities of the frontier models for cybersecurity related tasks. Jean-Denis Greze , co-founder of the buzzy AI assistant startup Town, said continuous security monitoring used to be too expensive to bother with. “Before it wasn’t worth it — now it is.” AI Cybersecurity as a Revenue Opportunity Another founder said that many of the vulnerabilities he’s seen these tools patching up come from older systems. That raises the possibility that we could see a spike in AI cyber-defense spending as people work to clean up their legacy human-coded infrastructure, but once those vulnerabilities are discovered and updated the need for these tools would decline. On the other hand, a recent story in https://www.theinformation.com/articles/ai-threats-reshaping-companies-spend-cybersecurity-budgets?rc=81ob4h The Information https://www.theinformation.com/articles/ai-threats-reshaping-companies-spend-cybersecurity-budgets?rc=81ob4h highlighted a broader shift of cybersecurity budgeting toward AI tools and away from older software made by the likes of Cisco and SentinelOne. The Information credited the spring announcement of Anthropic’s Mythos model with setting off the scramble for tools to fight bad actors. OpenAI certainly seems to be betting on long-term growth for the category. Last week, on the day it rolled out Astra, it held a separate cybersecurity session led by president Greg Brockman detailing the cyber capabilities of the new model. Sam Altman https://www.politico.com/news/2026/09/10/sam-altman-pitches-utilities-on-ai-grid-defense-01070425 is out selling it. Even assuming that OpenAI ringing the alarm about Hugging Face arose from genuine concern, there’s no doubt that the current panicky climate around AI risk is a good one for cyber-defense tools. Nvidia’s Jensen Huang even touted it as the next big AI thing https://www.bloomberg.com/news/articles/2026-09-10/nvidia-s-huang-touts-cybersecurity-as-next-big-market-for-ai?srnd=homepage-americas at a Goldman Sachs conference in San Francisco on Thursday. It’s not clear what kind of revenue opportunity cyber might represent for the big labs. So far , they’ve been riding the blowout success of agentic coding and the huge token bills companies are racking up. OpenAI has crossed $40 billion in annual run rate; Anthropic reportedly has crossed $65 billion. But at least some of its biggest customers have grown more cost-conscious in recent months, new data from Ramp https://ramp.com/data/ai-index-august-2026 shows see below . Uber provides a good example of the shifting dynamic. Earlier this year the company revealed that it had blown through its entire AI budget several months into the year by spending on tokens. Last month it revealed its new approach in a blog https://www.uber.com/us/en/blog/efficient-software-factory/ , which showed that while its agent usage continued to go up, its cost per 1,000 model requests is down almost 34% from its peak. Essentially, usage was up significantly and spending was still increasing but spending growth has slowed. We can imagine there are some tense conversations these days between the safety-conscious and the financially ambitious inside Anthropic and OpenAI. Cybersecurity as a focus is something they can agree on. Newcomer Podcast Zavain Dar on Hugging Face, Nvidia & Why Silicon Valley Should Pay Attention to China European Models Despite a Big Raise, Mistral May Not Shake Its Also-Ran Reputation This week, the French AI company Mistral raised a €3 billion Series D at a €21 billion post-money valuation. The raise https://mistral.ai/news/mistral-makes-sovereign-open-weight-ai-to-frontier/ is the latest move from a company that has left observers bewildered by its strategy. PSG Equity, Samsung, and the EQT-managed Scaleup Europe Fund co-led the round. Heavy hitters that have followed on from previous rounds include a16z, Index Ventures, General Catalyst, Lightspeed Venture Partners, and Nvidia. Mistral’s recent moves suggest that it is giving up on being “Europe’s answer to OpenAI” https://finance.yahoo.com/news/14-billion-ai-startup-mistral-123224843.html and pivoting to institutional deployments. In July, it was reported https://www.reuters.com/business/microsoft-fund-mistrals-european-ai-expansion-multibillion-dollar-deal-2026-07-21/ that Microsoft was funding a multi-billion-dollar buildout of European compute for Mistral. Then, in August, Mistral announced it would begin hosting third-party open-source models on its platform, starting with Chinese firm Z.ai’s GLM-5.2. Some https://techcrunch.com/2026/07/04/what-is-mistral-ai-everything-to-know-about-the-openai-competitor/ outlets have interpreted these moves as Mistral following the Palantir playbook, using forward-deployed engineers to bring customers onto Mistral’s proprietary stack. However, one critic cited in https://www.lemonde.fr/en/economy/article/2026/09/08/mistral-ai-raises-3-billion-in-response-to-doubts-over-its-strategic-direction 6757278 19.html?srsltid=AfmBOoowSInW1R1YqtRaDbCT6ZyfVwM3dNN0JqLpE1nWLsGoAf6J3BmG Le Monde https://www.lemonde.fr/en/economy/article/2026/09/08/mistral-ai-raises-3-billion-in-response-to-doubts-over-its-strategic-direction 6757278 19.html?srsltid=AfmBOoowSInW1R1YqtRaDbCT6ZyfVwM3dNN0JqLpE1nWLsGoAf6J3BmG instead compared them to Capgemini, the far less glamorous IT solutions company. A VC we also spoke with, who is familiar with the company, said, “The ambition has morphed from competing with OpenAI to competing with the Chinese models to implement models as European Palantir. The danger, however, is that you end up as that other French services company, Capgemini.” This comparison is apt if customers prefer to use open-source models other than Mistral’s while still relying on the firm for AI services. Mistral’s “frontier-class” https://docs.mistral.ai/models model, Medium 3.5, is the world’s 31st most powerful open-weight model per the capabilities index https://epoch.ai/eci?view=graph&tab=leaderboard&colorCategorization=Organization&accessibility=Open run by research firm Epoch AI, although it remains first among European firms. Another question is whether the new French administration will continue to hold up Mistral as a continental champion. Politico https://www.politico.eu/article/mistral-ai-arthur-mensch-emmanuel-macron-gouvernement-france/ highlighted this week that much of the firm’s success has been enabled by its close relationship with French president Emmanuel Macron , whose replacement in 2027 may well be https://www.connexionfrance.com/news/marine-le-pen-dwarfs-rivals-in-2027-french-presidential-election-poll/812153 the Eurosceptic Marine Le Pen . Despite these doubts, Mistral says it intends to fight at the frontier. Global Affairs SVP Audrey Herblin-Stoop told https://www.lemonde.fr/en/economy/article/2026/09/08/mistral-ai-raises-3-billion-in-response-to-doubts-over-its-strategic-direction 6757278 19.html?srsltid=AfmBOorIfYPeHGJ6PmoNBOuXAuCucAuJhkGur1RbJAGtxWQbqTMZ-woa Le Monde https://www.lemonde.fr/en/economy/article/2026/09/08/mistral-ai-raises-3-billion-in-response-to-doubts-over-its-strategic-direction 6757278 19.html?srsltid=AfmBOorIfYPeHGJ6PmoNBOuXAuCucAuJhkGur1RbJAGtxWQbqTMZ-woa that the raise will fund a compute buildout that enables more powerful models and allows Mistral to host their customers’ AI systems, raising revenues to be invested back into research. Mistral CEO Arthur Mensch is well-respected in the industry, and if the shock success of Moonshot AI’s Kimi K3 https://www.newcomer.co/p/high-stakes-battle-over-china-policy in July is anything to go by, labs are only as good or bad as their last model. AI Acquisitions Baseten Deal Shows Appetite for Infrastructure Startups Inference providers are looking for more ways to build out their tools to help run AI agents, a new deal shows. On Thursday, Baseten announced https://www.axios.com/pro/enterprise-software-deals/2026/09/10/baseten-agentic-infrastructure-blaxel that it had acquired the startup Blaxel, which has been developing sandboxes for AI agents running code or other background tasks. It’s the second acquisition for the inference provider in the last year, after purchasing the post-training startup Parsed in December 2025. The price wasn’t disclosed in the press release, but we heard the deal total was roughly $300 million. For Blaxel, it’s a strong markup — the startup had only raised $7.3 million in seed funding last December from First Round Capital, Liquid 2 Ventures, and Y Combinator. Some investors we’ve spoken with in the past have hesitated to back many of these kinds of startups, expecting that the companies that needed the technology would build it themselves. But this deal shows at least one well-capitalized startup would rather go shopping. Three Big Charts Power Users Ease Off On AI Spending as Token Costs Fall Ramp’s latest AI Index Report https://ramp.com/data/ai-index-sept-2026 shows some signals that the extraordinary growth of AI spending may be starting to ease, though Ramp research head Ara Kharazian is cautious about drawing broad conclusions.