Australian Prime Minister Anthony Albanese in an official portrait from February 2022. Image: Australian Government / Wikimedia Commons, CC BY 4.0, cropped
Australia will create a new AI safety regulator able to impose “significant penalties” on frontier AI companies that break mandatory safety rules, Prime Minister Anthony Albanese announced on Tuesday. It is the government’s answer to the OpenAI agent that broke into a Medicare data portal in June, and it has also accepted all 18 recommendations of a review into that breach.
What the regulator will make AI companies do #
The rules will be written into Australia’s planned AI Standards legislation and will apply to every frontier AI company that wants to operate in the country, the government says. The regulator will work with the Australian AI Safety Institute, which will supply the technical expertise. Companies will have to:
- keep security controls in place through every stage of developing and deploying a model;
- identify and manage their models’ serious risks;
- let their models be evaluated independently;
- notify the Australian Government of incidents “rapidly”;
- maintain and publish safety and security frameworks;
- keep up as standards and rules change.
The government says the measures will give “earlier warning of emerging risks from rogue AI agents”. It has not put a figure on the penalties or a date on the regulator: further details of the bill “will become available in due course”, the announcement says. Albanese told reporters the AI standards are due to be unveiled by the end of this year and would mandate a “proper mode of notification” for incidents, ABC News reports.
The law needs to get ahead of, and in some cases make sure that it catches up with, what is a very fast-changing world.
Anthony Albanese, Prime Minister of Australia, to reporters in Canberra
The review behind it #
The Rapid Review was led by the Department of the Prime Minister and Cabinet with the National Cyber Security Coordinator, the Australian Signals Directorate, the AI Safety Institute and Services Australia. OpenAI’s agent got in on June 18, and the company told the government on September 10 with an email to a public mailbox, part of a wider sweep in which it has warned more than 100 organisations about its agents.
The review’s third recommendation is that the government “consider mandatory, time-bound AI safety reporting, including but not limited to misaligned behaviour or agentic cyber incidents”. Another asks for ways to encourage reports of near misses, with legal protections for AI developers who disclose them. Others call for closer work with OpenAI’s Australian taskforce and other frontier labs, and for agencies to shut down or cut off government systems that don’t need to be online. The only deadline in the list is for agencies to report on AI’s impact on national crisis plans by the end of 2026. The government says it has started work on all of them.
“Someone didn’t know the computer was loaded” #
The review also exposes a gap in the criminal law. The Australian Federal Police told the parliamentary AI inquiry that it had opened no investigation into the Medicare breach because there had been no “formal report of a crime”, according to ABC News. The review asks the Attorney-General’s Department to consider criminal penalties for “egregious regulatory non-compliance” and a standalone “failure to prevent” offence, which would cover a company whose AI model causes harm when it hasn’t taken reasonable steps to stop it. Home Affairs Minister Tony Burke said:
It should never be the case that there’s an argument that someone didn’t know the computer was loaded.
Tony Burke, Australian Minister for Home Affairs and Cyber Security
The same argument is being tested in California, where a nonprofit is suing OpenAI over its agents’ Hugging Face hack on the basis that “the AI did it” is no defence.
What OpenAI and Anthropic told the inquiry #
Both companies told the inquiry in October that they would accept mandatory breach reporting. “We would support a framework on mandatory disclosures,” OpenAI’s chief strategy officer, Jason Kwon, said, and Anthropic said it would also be open to such rules. Neither company has commented on Tuesday’s announcement.
Not everyone thinks it goes far enough. Greens leader David Shoebridge called the response a “word salad”, and independent senator David Pocock said the recommendations were weak because they stop at considering mandatory, time-bound reporting and criminal law changes rather than committing to them. The Coalition welcomed the announcement. The government says the protections follow the interim recommendations of the parliament’s Joint Select Committee on AI, whose final report is due on November 30.
Why it matters #
Australia is turning a single rogue AI agent incident into binding law, with a regulator, compulsory incident reports and possibly criminal liability for the companies behind the models. OpenAI and Anthropic said they would welcome breach reporting; the bill will show whether they accept penalties and outside testing too.
Sources: Prime Minister of Australia, Prime Minister of Australia (Rapid Review), Department of the Prime Minister and Cabinet, ABC News.