{"slug": "anthropic-warns-ai-may-pose-existential-risks-to-humanity-in-ipo-filing", "title": "Anthropic warns AI may pose ‘existential risks to humanity’ in IPO filing", "summary": "Anthropic plans to warn potential investors in its IPO prospectus that advanced AI could pose \"catastrophic or existential risks to humanity,\" according to the filing reviewed by Reuters. The company devoted roughly 80 pages of the 261-page main body of its prospectus to risk factors, nearly twice the 48 pages describing its business, and Anthropic safety researcher Evan Hubinger estimated a greater than 10 per cent probability that AI could kill humans within the next decade. Anthropic said about 6 per cent of the computing power it used for AI research went to safety work in a sample week in July and declined to comment on Monday (Sep 28).", "body_md": "# Anthropic warns AI may pose ‘existential risks to humanity’ in IPO filing\n\nFirm says market will reward reliable, trustworthy and secure AI systems\n\nANTHROPIC plans to caution potential investors in its IPO that advanced AI could pose “catastrophic or existential risks to humanity,” an extraordinary warning by a company seeking to profit from the same technology.\n\nThe [company’s IPO prospectus,](https://www.businesstimes.com.sg/international/global/surging-revenue-widening-losses-and-sweeping-ai-vision-inside-anthropics-ipo-prospectus) reviewed by Reuters, highlights risks associated with its AI models, which it said could exhibit “self-preserving behaviours,” including attempts to “resist shutdown,” to “conceal or manipulate information” and behaviour “resembling blackmail.”\n\n“Our development of highly advanced models, platforms, and applications and expansion of use cases could further increase the risk that our models cause harm,” Anthropic said in the filing.\n\nWhile public companies routinely outline product risks to investors, few, if any, have issued warnings suggesting their technology could cause potential human extinction.\n\nAnthropic emphasised both the transformative potential of AI on par with industrialisation and electricity and the irreversible harm it could cause if mishandled.\n\nAnthropic and other AI developers, including OpenAI, have faced scrutiny after incidents where experimental systems defied constraints, including a report of an OpenAI model [breaching Australia’s health-system database.](https://www.businesstimes.com.sg/international/global/openai-agent-breached-health-portal-australias-first-known-ai-intrusion-government-systems)\n\nAnthropic safety researcher Evan Hubinger estimated a greater than 10 per cent probability that AI could kill humans within the next decade, echoing a sentiment by a former colleague, Jacob Coxon.\n\n## Risk-heavy disclosures\n\nThe company, which has positioned itself as a safety-first AI lab, devoted roughly 80 pages of the 261-page main body of its prospectus to laying out risk factors, nearly twice the 48 pages it used to describe its business.\n\nFor comparison, SpaceX, which owns xAI, dedicated just around 38 of the 277-page main body of its prospectus to risk factors.\n\n“Potential model awareness of our evaluation efforts creates a significant limitation on our ability to assess model safety,” Anthropic said in the prospectus, adding that models sometimes develop unexpected capabilities during training that may not be discovered until they have been deployed and have resulted in significant safety incidents.\n\nAI researchers have also warned that as models grow more capable, they increasingly recognise when they are being watched and adjust their behaviour accordingly, which makes it harder to monitor model behaviour.\n\nAnthropic declined to comment in response to a request for comment on Monday (Sep 28).\n\n## Uncertain returns on safety investment\n\nDespite emphasising AI safety, Anthropic said that returns on its safety investments are unclear.\n\nIt did not disclose in the filing how much the company was spending on such research. Earlier in September, Anthropic said about 6 per cent of the computing power it used for AI research went to safety work in a sample week in July.\n\nThe company, creator of Claude AI models, described safety efforts as “resource-intensive” and said it must divide its limited funds between computing power, expensive AI talent and safety.\n\nAnthropic said that its customer usage, and as a result revenue, is driven by new models and that a “continuous and overlapping cadence” of releases is “inherent to remaining at the frontier of AI development.”\n\nThe company on Sep 22 released a new version of its Opus model, 10 days after CEO Dario Amodei published a nearly 4,000-word essay [calling for pacing the frontier.](https://www.businesstimes.com.sg/international/global/anthropic-ceo-urges-slower-ai-development-altman-musk-rally-behind-call)\n\nSome analysts and experts have said no leading AI lab would slow down when doing so risks handing rivals an advantage in an industry where valuations can change with each release.\n\nAnthropic has pledged in recent weeks to disclose more data publicly about how it uses AI models to build future generations of the technology, as experts warn about recursive self-improvement – the point at which models can develop on their own without human help.\n\n“We believe building reliable, trustworthy, and secure AI systems is a collective responsibility and that the market will reward it,” Anthropic said in the filing. REUTERS\n\nDecoding Asia newsletter: your guide to navigating Asia in a new global order. Sign up here to get Decoding Asia newsletter. Delivered to your inbox. Free.\n\nShare with us your feedback on BT's products and services", "url": "https://wpnews.pro/news/anthropic-warns-ai-may-pose-existential-risks-to-humanity-in-ipo-filing", "canonical_source": "https://www.businesstimes.com.sg/international/global/anthropic-warns-ai-may-pose-existential-risks-humanity-ipo-filing", "published_at": "2026-09-29 01:46:34+00:00", "updated_at": "2026-09-29 01:47:52.675762+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-safety", "ai-policy", "ai-startups"], "entities": ["Anthropic", "OpenAI", "Claude", "Dario Amodei", "Evan Hubinger", "Jacob Coxon", "SpaceX", "xAI"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/anthropic-warns-ai-may-pose-existential-risks-to-humanity-in-ipo-filing", "markdown": "https://wpnews.pro/news/anthropic-warns-ai-may-pose-existential-risks-to-humanity-in-ipo-filing.md", "text": "https://wpnews.pro/news/anthropic-warns-ai-may-pose-existential-risks-to-humanity-in-ipo-filing.txt", "jsonld": "https://wpnews.pro/news/anthropic-warns-ai-may-pose-existential-risks-to-humanity-in-ipo-filing.jsonld"}}