Anthropic said it has chosen Accenture as its first embedded evaluator to help slow the pace of AI development as proposed last week by CEO Dario Amodei.
Amodei published a three-step proposal on Sept. 12 to slow the development of AI and allow safeguards to be put in place amid warnings of a potential for catastrophic harm in the wake of swift, unchecked development.
OpenAI CEO Sam Altman and SpaceX CEO Elon Musk responded positively to Amodei’s proposal, although Nvidia CEO Jensen Huang did not, arguing that such regulation was not necessary.
Still, Amodei wrote in his proposal that “AI has been advancing drastically faster, driven primarily by AI’s growing ability to build the next generation of AI. This dynamic is called recursive self-improvement,” and “left unchecked, it could outrun our ability to understand and control these systems.”
Related: Anthropic chief urges slowdown in AI development to safer pace
The first step in Amodei’s proposal is having independent evaluators with employee-like access, to which Anthropic had already unilaterally committed, Amodei wrote.
Anthropic announced a move toward fulfilling this commitment on Friday by partnering with Accenture and its AI business Faculty in “evaluating and red-teaming models, conducting alignment assessments and testing model safeguards.”
Details of how this will happen are still being worked out, as embedded evaluation is new, the company said.
Anthropic and Accenture each expect to invest at least $1 billion in the project over the next five years, according to the announcement.
Anthropic will fund work directly amid urgency #
Anthropic noted that there is also no existent system for funding independent evaluation, so long-term funding should come from pooled or government sources, though given the urgency of the work, Anthropic will fund Accenture’s work directly.
The partnership is non-exclusive, and Anthropic expects to announce other evaluators in forthcoming weeks with which it will also work.
Accenture’s Faculty is expert in testing and evaluating models for some of the world’s leading AI labs and building complex AI systems that are safe and ethical by design, the company said on Friday.
“Embedded evaluation is an emerging area, and we look forward to partnering with Anthropic to help accelerate the development of embedded evaluators, which we see as an important part of the safety landscape going forward,” said Julie Sweet, chair and CEO of Accenture.
Magazine: Why are AI’s biggest companies suddenly asking to slow down?