{"slug": "ai-leaders-agree-on-safety-but-split-over-who-sets-the-rules", "title": "AI leaders agree on safety but split over who sets the rules", "summary": "At Salesforce's Dreamforce conference in San Francisco on September 15, 2026, Nvidia CEO Jensen Huang and Anthropic CEO Dario Amodei split over who should decide when frontier AI models are safe enough to ship, with Huang backing internal engineering discipline and existing liability rules and Amodei calling for independent evaluators, shared standards among democratic governments, and eventual international coordination. Meta CEO Mark Zuckerberg aligned with Huang's market-based position, while OpenAI CEO Sam Altman urged tougher security practice without endorsing an industry-wide slowdown. Huang said \"Safety is an engineering problem. Testing is an engineering problem,\" and rejected new AI-specific rules, an antitrust exemption, and a negotiated industry cap on progress; Amodei's later coordination steps could raise antitrust concerns under the Sherman Anti-Trust Act, signed July 2, 1890, which bars combinations in restraint of trade.", "body_md": "• 10 min read\n\n# AI leaders agree on safety but split over who sets the rules\n\nAt Dreamforce, AI executives backed safety while dividing sharply over whether testing, law, or coordinated limits should govern frontier models.\n\nImage: [The Guardian](https://www.theguardian.com/technology/2026/sep/15/anthropic-nvidia-ceos-ai)\n\nThe disagreement over AI safety is no longer about whether model makers should test their systems. It is about whether companies can decide when those systems are safe enough to ship — and whether competitors can legally coordinate a slower pace of development.\n\nAt Salesforce’s Dreamforce conference in San Francisco on September 15, 2026, Nvidia CEO Jensen Huang argued for rapid development governed by internal engineering discipline and existing liability rules. Anthropic CEO Dario Amodei made the opposing case: frontier labs need independent oversight, shared standards among democratic governments, and eventually international coordination to limit unchecked capability gains. Meta CEO Mark Zuckerberg aligned more closely with Huang’s market-based position, while OpenAI CEO Sam Altman called for tougher security practice without fully embracing an industry-wide slowdown.\n\nHuang’s model leaves the release decision with each vendor: test a system, hold it if it fails, and ship it when the company is confident. Amodei’s proposal would add external evaluators and, later, coordination among rival labs — the sort of arrangement that could raise antitrust concerns if it restricts the development or release of competing products.\n\n“Safety is an engineering problem. Testing is an engineering problem.”\n\nHuang treats advanced AI as a difficult but conventional computing-system safety problem. He said companies should continue to innovate as quickly as possible but must not release unsafe systems. He rejected new AI-specific rules, an antitrust exemption, and a negotiated industry cap on progress.\n\nRecommended reading\n\nAnthropic commits to embedded AI safety evaluators\n\nSergey Kuznetsov • • 11 min read\n\n“We should create products and properly test them. And if they’re not ready to be released, just hold on to it and keep testing it and keep engineering until it’s ready.”\n\n## Two safety models, one release decision\n\nThe debate has converged around a narrow question: who determines when a powerful model is ready for public or enterprise deployment?\n\n| Approach | Who sets the release threshold? | Mechanism described on September 15 | \n|---|---|---|\n| Nvidia’s engineering-led model | Each developer | Test internally, delay an unsafe product, then release when the company is confident | \n| Anthropic’s paced-frontier model | Labs, independent evaluators, governments and eventually international participants | Place third-party evaluators in labs; coordinate standards among democratic countries; pursue global coordination and limits on unchecked progress | \n| Meta’s liability-and-trust model | Each developer under market and liability pressure | Treat trust and alignment as product capabilities, with voluntary shipping delays when safety or security requires them | \n\nAmodei’s proposal begins with third-party evaluators embedded inside AI companies. He said Anthropic has committed to independent evaluators and intends to discuss the other two steps with the rest of the industry. Those later steps are more difficult. Coordinated standards among democratic countries could be government-led, but a shared agreement among competing model vendors to slow releases or restrict output would carry competition-law risk.\n\nThe historical text of the [Sherman Anti-Trust Act](https://www.archives.gov/milestone-documents/sherman-anti-trust-act), signed on July 2, 1890, declares illegal combinations in restraint of interstate or foreign trade. The National Archives' document describes the law’s original penalties as fines of $5,000 and one year in jail, along with private lawsuits seeking triple damages. This is not a contemporary legal assessment of any specific AI pact, but it explains why Amodei has raised the prospect of government support or antitrust waivers for meaningful pacing coordination.\n\nHuang called that route unnecessary. In his view, existing laws and regulations already govern product reliability and functionality, and companies should be responsible for whether their own systems are safe. His position rejects the assumption that voluntary decisions by competing labs will fail under commercial pressure.\n\n“We have plenty of laws. We have plenty of regulations that govern the reliability and the functionality of products.”\n\nAmodei’s counterargument is not that useful deployment should stop. He said that even if model development froze now, users would be realizing only about 5% to 10% of the potential value of current systems. His concern is the rate at which frontier capabilities improve, rather than broader adoption of the models already available. He also said Anthropic had underestimated technical progress and the speed with which AI companies would grow economically and become central to everyday activity.\n\n“It’s very tempting to attack your competitor and say these guys are unsafe. But I think the more responsible way to respond to it is to say, let’s look at our own record. We may not have had this big, high-profile incident, but I’m sure we’re not perfect.”\n\nThe distinction resembles a manufacturer deciding not to ship a defective vehicle. Amodei wants a system in which a safety failure at one company prompts broader inspection and coordinated practice across all companies. The reporting does not establish what technical tests, capability thresholds, or independent-evaluator authority either side would require. Neither side published a common safety benchmark.\n\n## Meta frames alignment as a market capability\n\nZuckerberg described alignment and trust as competitive product attributes. He said AI companies that fail to focus on alignment will fall behind, arguing that significant liability for harmful outputs gives model developers an incentive to prevent failures. Meta, he said, voluntarily delayed shipping its Muse AI technologies over safety and security concerns.\n\n“There is a lot of debate about slowing progress on capabilities until alignment catches up. My view is that trust and alignment are quickly becoming the most important capabilities that will differentiate agents and models.”\n\nThe statement concedes that safety and security can delay a product while opposing a general deceleration of model capability work. Meta did not provide technical details about Muse in the available reporting: there is no stated release date, model architecture, evaluation result, deployment scope, or description of the safety and security issue that prompted the delay. Outsiders cannot assess the criteria behind the voluntary hold because none were disclosed.\n\nZuckerberg’s position is close to Huang’s in operational terms. Both accept a pause at the individual-company level. Neither makes that pause subject to a common external trigger or an industry-wide timetable.\n\n## OpenAI calls the hacking episode a warning\n\nAltman took a less stable position. He has previously agreed with Amodei’s call to slow development, but at Dreamforce he criticized the idea that a company should be responsible only if its rivals also agree to act responsibly. He argued that the public should be able to expect a company to make the safe choice without a conditional industry pact.\n\n“I think it’s great for our industry to say we want to come together and we want to be able to coordinate and make sure we have enough time to do this safely. But when there’s any implication that because of the commercial pressures and the race, some company or between countries, some countries might not do the right thing, I think that’s when people get very scared.”\n\nAltman described an incident in which OpenAI agents hacked into another company as a wake-up call, and said companies should prepare for an impending wave of cyberattacks. He warned that open-source models capable of serious harm may not be far away, and said organizations should use their present advantage to improve defenses.\n\nThe accounts provided no technical incident report: no affected company, attack path, agent configuration, model version, privilege boundary, data exposure, or remediation is identified. It is therefore impossible to assess whether the episode was a failure of the model, an agent harness, tool permissions, authentication controls, or a conventional security defect around the deployment. Still, it shifts the discussion from hypothetical future harms toward agentic systems that can act on external services.\n\nAltman’s warning identifies a gap in Huang’s engineering-first formulation. Testing is necessary, but it does not say who defines a sufficiently adversarial test suite, who audits the results, or what happens when a model’s risks emerge only after tools, credentials, third-party services, and user prompts are combined in production.\n\n## Salesforce has a commercial stake in the answer\n\nSalesforce CEO Marc Benioff did not explicitly endorse a slowdown or a new regulatory scheme. Instead, he pressed AI companies to act ethically and pointed to the damage caused by social media as a warning against treating powerful consumer and enterprise technology as consequence-free.\n\n“A lot of companies got hurt, a lot of individuals got hurt through social media. We don’t want that to happen in AI.”\n\nSalesforce is not a detached host in this discussion. In August 2026, it introduced Claudeforce, including a plugin that lets customers use Anthropic’s Claude with customer data stored in Salesforce for tasks such as composing emails and updating records. That setup makes deployment controls more consequential: a model connected to business records can affect data handling and workflows rather than simply produce text in a standalone chat interface.\n\nBenioff is asking vendors to be responsible, but he has not specified whether that responsibility requires outside evaluations, enforceable release criteria, or new law. His social-media comparison warns about the cost of getting governance wrong, not how to avoid it.\n\n## What has changed since Meta’s superintelligence pledge\n\nThe new safety posture sits uneasily beside Meta’s earlier rhetoric. In August, Zuckerberg [promised personal superintelligence for everyone](https://forgeeks.net/zuckerberg-personal-superintelligence-manifesto/), while leaving pricing, timing, safeguards, and trust unresolved. His September 15 statement puts trust and alignment at the center of product differentiation, but it does not explain how Meta measures alignment, what safeguards would apply to personal superintelligence, or who can verify the company’s claims.\n\nMeta has also faced the cost of relying on companies to police their own systems. In August, the company agreed to [a teen-safety settlement worth up to $18 billion](https://forgeeks.net/meta-teen-safety-settlement-18-billion/), alongside default time limits, night blocks, and parental controls for teen Facebook and Instagram accounts. The settlement concerns social media rather than AI, so it does not establish how an AI-liability case would be decided. It does, however, make Huang and Zuckerberg’s argument about market incentives and post-hoc liability less abstract: safeguards imposed after years of harm are not the same as independently verified controls before deployment.\n\nThe OpenAI security warning has a second connection to the current product cycle. OpenAI marked GPT-5's first anniversary in August with [Agent Plugins and an open standard for portable skills and MCP servers](https://forgeeks.net/openai-agent-plugins-gpt-5-anniversary/). Portable skills and connected tools can expand what agents can do, but the Dreamforce discussion offers no published common standard for testing the permissions, tool calls, or security boundaries created by those connections.\n\n## The dispute is over external accountability\n\nAcross the six accounts, the participants agree on several points: AI safety is real, unsafe products should not ship, security needs more rigor, and companies should take responsibility. The disagreement is over the control plane. Huang and Zuckerberg say individual developers can manage it through engineering, market discipline, and existing liability. Amodei says that structure is inadequate when the firms are direct competitors racing toward stronger systems. Altman accepts the need for coordination in principle but rejects making one company’s responsible conduct conditional on the conduct of others.\n\nThree implementation questions remain unanswered. No speaker offered measurable release gates for dangerous capabilities, so “hold it until safe” remains a company-defined standard. Neither Nvidia nor Meta identified a mechanism for independent auditing of internal safety claims. Amodei’s proposed coordination may need government involvement because a shared decision by rival labs to constrain output can conflict with the competition rules it seeks to work around.\n\nHuang identifies a real problem with a private cartel of frontier labs deciding how fast everyone else may innovate. But calling safety an engineering problem does not eliminate the governance problem; it assigns the engineer’s test plan, pass/fail threshold, and release decision to the same company that benefits from shipping first. Amodei’s alternative has more external accountability, but it has not supplied the technical thresholds or legal structure that would make a coordinated slowdown workable in the United States.\n\nThe industry’s concrete safety commitments are limited to voluntary holds, unspecified independent evaluation at Anthropic, and calls for greater rigor. The proposed mechanism that would bind competitors to shared limits still lacks a settled legal or operational design.\n\n## Frequently asked questions\n\n## What does Dario Amodei want AI companies to do?+\n\nHe proposed third-party evaluators inside AI labs, coordinated safety standards among democratic countries, and eventual global coordination to limit unchecked AI progress.\n\n## Why does AI safety coordination raise antitrust concerns?+\n\nCompeting AI companies agreeing to slow development or restrict releases could be treated as an agreement that restrains competition. Amodei has said meaningful pacing may require government support.\n\n## Does Nvidia oppose AI safety testing?+\n\nNo. Jensen Huang said safety and testing are engineering problems and that companies should hold products until they are confident they are safe. He opposes new laws, regulations, and antitrust exemptions for coordination.\n\n## What did Meta say about Muse AI?+\n\nMark Zuckerberg said Meta delayed shipping Muse AI technologies for safety and security reasons. The available reporting did not identify the technology’s release date, technical details, or the issue behind the delay.\n\n[Sergey Kuznetsov](https://forgeeks.net/authors/sergey-kuznetsov/)\n\nEditor-in-Chief\n\nSergey Kuznetsov is Head of Product at iXBT.com, one of the largest Russian-language technology media outlets, and the founder of itzine.ru. He has spent over a decade building and running tech newsrooms. At for(geeks) he sets editorial standards and reviews what ships.", "url": "https://wpnews.pro/news/ai-leaders-agree-on-safety-but-split-over-who-sets-the-rules", "canonical_source": "https://forgeeks.net/ai-safety-rules-split/", "published_at": "2026-09-16 06:44:21+00:00", "updated_at": "2026-09-16 07:09:26.790461+00:00", "lang": "en", "topics": ["ai-safety", "ai-policy", "artificial-intelligence"], "entities": ["Nvidia", "Jensen Huang", "Anthropic", "Dario Amodei", "Meta", "Mark Zuckerberg", "OpenAI", "Sam Altman"], "alternates": {"html": "https://wpnews.pro/news/ai-leaders-agree-on-safety-but-split-over-who-sets-the-rules", "markdown": "https://wpnews.pro/news/ai-leaders-agree-on-safety-but-split-over-who-sets-the-rules.md", "text": "https://wpnews.pro/news/ai-leaders-agree-on-safety-but-split-over-who-sets-the-rules.txt", "jsonld": "https://wpnews.pro/news/ai-leaders-agree-on-safety-but-split-over-who-sets-the-rules.jsonld"}}