arXiv:2609.27051v1 Announce Type: new Abstract: Language-model agents now run the whole of quantitative factor research: they propose investment factors, backtest them, select the survivors and retire them. We ask which of those jobs an agent should keep. Our answer is governed self-evolution: the agent may propose, and a frozen statistical referee that the agent cannot touch must judge. The referee scores each candidate only on market outcomes revealed after submission, by betting, so its false-discovery guarantee holds at every stopping time for any proposal policy. We cross three proposers (a script, a bandit and a language model) with this referee and with three deliberately leaky ones, in a synthetic world with planted truth, a probe-authoring environment and a ten-year walk-forward on the CSI 500. Who judges sets the number of false admissions: the frozen referee admits 5-11 times fewer sub-threshold factors than the leaky referees under a scripted proposer, and no proposer closes that gap. Who proposes sets the yield: the language model beats the script, matches the bandit, and adds the one capability a bandit lacks, writing its own diagnostic probes. The certificate's price is time: an admitted true factor waits about 500 trading days, and the certified portfolio's Sharpe ratio therefore trails an ungated one. Judging belongs to the procedure; proposing and instrument-making belong to the agent.
Propose, Don't Judge: An Anytime-Valid Referee for LLM Agents That Mine Investment Factors
A frozen statistical referee that scores LLM-proposed investment factors only on post-submission market outcomes admitted 5 to 11 times fewer sub-threshold factors than three deliberately leaky referees under a scripted proposer, according to an arXiv paper (2609.27051v1) that cross-tested a script, a bandit and a language model as proposers. The paper reports no proposer closed that false-admission gap, while the language model beat the script and matched the bandit on yield and uniquely wrote its own diagnostic probes. The certificate's cost is time: an admitted true factor waits about 500 trading days, and the certified portfolio's Sharpe ratio trails an ungated one in a ten-year walk-forward on the CSI 500.
Run your AI side-project on zahid.host
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.