Building AI Research Teams for Autonomous Discovery of Linear Attention Architectures

A research team proposes organizing autonomous AI agents into collaborative "AI research teams" that explore orthogonally to escape local minima, aiming to accelerate scientific discovery. The team applies the approach to the autonomous discovery of linear attention architectures, with agents proposing ideas, modifying code, and running experiments. The work targets a known failure mode in agent-driven discovery, where search converges on local optima.

Autonomous agents can increasingly propose ideas, modify code, and run experiments, creating an opportunity to accelerate scientific discovery using teams of collaborative AI researchers. To escape local minima in the discovery, we propose to encourage orthogonal exploration by constructing the team