{"slug": "set-a-metric-walk-away-let-the-agent-optimize-overnight", "title": "Set a metric. Walk away. Let the agent optimize overnight.", "summary": "Andrej Karpathy's 'autoresearch' technique lets AI agents optimize any metric overnight by running hundreds of experiments autonomously. One engineer applied the method to file compression, beating off-the-shelf tools for $40. The approach works by setting a metric, bounding it with constraints, and letting an agent iterate while the human sleeps.", "body_md": "# Set a metric. Walk away. Let the agent optimize overnight.\n\n### Karpathy runs 100 experiments while he sleeps. One engineer beat off-the-shelf compression tools for $40. Here is the exact playbook to run this on any metric you care about.\n\nThere is a new way to work, and it fits in one sentence:\n\n### Pick a number, bound it with constraints, and let an [agent](https://www.the-ai-corner.com/t/ai-agents?r=1krivi) push it while you sleep.\n\nKarpathy calls it autoresearch. His repo gives an agent one file, one metric, and a fixed 5-minute budget per experiment. The agent edits, trains, keeps what improves, reverts what fails, and loops. Roughly **12 experiments an hour, about 100 overnight**. Shopify’s CEO woke up to a model that beat his hand-tuned baseline. Karpathy’s own agent caught a bug he had missed for months.\n\nHis line is the one to keep:\n\n“Any metric you care about that is reasonably efficient to evaluate can be autoresearched by an agent swarm. It’s worth thinking about whether your problem falls into this bucket too.”\n\nAnd this pattern extends past machine learning. One engineer pointed it at file compression with [Claude Code](https://www.the-ai-corner.com/t/claude-and-anthropic?r=1krivi): 10 unsupervised iterations at about $4 each, and the home-built algorithm beat common tools on audio and video. Zero ML involved. Just a metric, two constraints, and a loop.\n\nThat is the whole trick, and it remains a rare skill. The gap between reading about loops and shipping one is a handful of decisions most people get wrong on the first try: which metric, which constraints, which loop mechanism, and how to stop the agent from gaming you.\n\nBehind the paywall, the full system:\n\n▫️\n\nthat make or break a loop, and the defaults that workThe 5 design decisions▫️\n\nincluding the silent trap that bent the compression experimentThe metric-picker framework,▫️\n\nthe harness builder, the iteration prompt, and the metric auditorThe 3 copy-paste prompts:▫️\n\nautoresearch vs Ralph loops vs /goal, /loop, and /batch, and when each winsThe tooling menu,▫️\n\nthat stop reward hacking before it startsThe constraint patterns▫️\n\nwhat a loop costs per iteration and when the ROI turns positiveThe cost math,▫️\n\nrunning this on conversion, latency, and content metrics with slow feedbackThe business translation,▫️\n\nthe problems where loops burn money and a human winsThe skip list,\n\n### One subscription unlocks every playbook\n\nThis is one system in a growing library. Premium opens all of them:\n\n▫️ [Loop engineering for coding agents](https://www.the-ai-corner.com/p/loop-engineering-coding-agents-2026?r=1krivi)\n\n▫️ [The Claude managed agents guide](https://www.the-ai-corner.com/p/claude-managed-agents-guide-2026?r=1krivi)\n\n▫️ [The AI agent reliability playbook](https://www.the-ai-corner.com/p/ai-agent-reliability-playbook?r=1krivi)\n\nPlus a fresh build every week. One overnight loop that lands a win pays the subscription back on the first run.\n\n# 🔁 The Autoresearch Playbook\n\nThe 5 design decisions, the metric framework, the 3 prompts, the tooling menu, the anti-gaming constraints, and the cost math, in one system you can launch tonight.\n\n**Try premium free for 7 days. Or get 50% off this week only.**\n\n### Get **The Autoresearch Playbook** below 👇\n\n## Keep reading with a 7-day free trial\n\nSubscribe to The AI Corner to keep reading this post and get 7 days of free access to the full post archives.", "url": "https://wpnews.pro/news/set-a-metric-walk-away-let-the-agent-optimize-overnight", "canonical_source": "https://www.the-ai-corner.com/p/autoresearch-playbook-agent-optimization-loops-2026", "published_at": "2026-07-03 16:54:59+00:00", "updated_at": "2026-07-03 21:07:36.772858+00:00", "lang": "en", "topics": ["ai-agents", "machine-learning", "ai-research", "ai-tools"], "entities": ["Andrej Karpathy", "Shopify", "Claude Code", "The AI Corner"], "alternates": {"html": "https://wpnews.pro/news/set-a-metric-walk-away-let-the-agent-optimize-overnight", "markdown": "https://wpnews.pro/news/set-a-metric-walk-away-let-the-agent-optimize-overnight.md", "text": "https://wpnews.pro/news/set-a-metric-walk-away-let-the-agent-optimize-overnight.txt", "jsonld": "https://wpnews.pro/news/set-a-metric-walk-away-let-the-agent-optimize-overnight.jsonld"}}