New York City Unveils AI Policy for Children, Anthropic Wins Court Battle Against Pentagon

New York City Mayor Zohran Mamdani announced a formal policy governing AI systems that interact with children under 18, requiring age-appropriate design, transparency, and human review, while Anthropic won a federal court ruling that blocks the Pentagon from using its Claude models for military operations without explicit permission. The policy, which applies to public agencies and mandates child impact assessments, and the court decision, which enforces usage restrictions through contract law, signal tightening AI governance at municipal and judicial levels.

September 3, 2026, Inside AI — New York City has released a formal policy to govern how artificial intelligence systems interact with children, while Anthropic secured a legal victory that blocks the Pentagon from using its AI models for military purposes. Both developments signal a tightening relationship between AI governance and public institutions. The New York policy, announced by Mayor Zohran Mamdani, sets boundaries for AI tools used in schools, child services, and public spaces. It requires age-appropriate design, transparency about automated decisions, and prohibits AI systems from collecting unnecessary data on minors. The policy also mandates human review before any AI-driven decision affects a child's education or welfare. Anthropic's court win came in a federal case where the company argued that its terms of service prohibit military use of its models. The judge agreed, ruling that the Pentagon cannot deploy Anthropic's Claude models for defense operations without explicit permission. This decision sets a precedent for how AI companies can enforce usage restrictions through contract law rather than relying solely on export controls or voluntary commitments. These two events highlight a growing tension in the AI industry. On one side, local governments are moving quickly to protect vulnerable populations. On the other, AI developers are using legal tools to control how their technology is deployed by powerful institutions. Why New York's Child AI Policy Breaks New Ground New York's policy is among the first municipal AI rules focused specifically on children. It borrows from the UK's Age Appropriate Design Code and California's Age-Appropriate Design Code Act, but goes further by covering public agencies, not just private companies. The policy applies to any AI system that interacts with children under 18, including chatbots used in schools, predictive analytics in child welfare, and facial recognition in public spaces. The policy requires AI vendors to complete a child impact assessment before deployment. This assessment must document how the system protects privacy, prevents bias, and ensures explainability. Agencies must also publish summaries of these assessments, creating a public record of how AI affects children. Critics argue the policy lacks enforcement teeth. It relies on agency compliance rather than fines or penalties. But supporters say the transparency requirements alone will pressure agencies to avoid risky AI deployments. Anthropic's Legal Strategy Could Reshape AI Governance Anthropic's court win is significant because it uses standard contract language to restrict military use. The company's terms of service already prohibited using its models for weapons development or military operations. The Pentagon argued that national security needs outweighed these terms. The court disagreed. Legal experts say this ruling could encourage other AI companies to add similar restrictions to their terms. It also creates a roadmap for enforcing those restrictions through civil litigation rather than waiting for government regulation. The Pentagon has not announced whether it will appeal. If the ruling stands, it could limit the military's access to leading AI models, forcing it to rely on in-house development or vendors willing to allow military use. These developments come as the AI industry faces increasing scrutiny from regulators worldwide. The European Union's AI Act is now being implemented, and several U.S. states have passed their own AI laws. New York's child policy and Anthropic's court win show that governance is happening at multiple levels, from city halls to federal courts. For AI developers, the message is clear: terms of service are becoming enforceable legal instruments, not just boilerplate. For governments, the challenge is balancing innovation with protection for the most vulnerable.