{"slug": "can-llms-engineer-their-own-agent-harness-bytedance-seeds-harnessdev-says-only", "title": "Can LLMs Engineer Their Own Agent Harness? ByteDance Seed’s HarnessDev Says Only 34 of 64 Changes Generalize", "summary": "ByteDance Seed, SUTD, Georgia Tech, M-A-P, and TokenWave.AI released HarnessDev, a benchmark that scores the runnable harness a model builds rather than the answer it returns, with 6 creator LLMs constructing harnesses across 5 benchmarks and 2,207 tasks from a seed that scores 0. Self-built harnesses matched human references on writing and ML experimentation but trailed on code and search, and only 34 of 64 evolution changes moved the same direction on held-out tasks.", "body_md": "ByteDance Seed, SUTD, Georgia Tech, M-A-P, and TokenWave.AI introduce HarnessDev, a benchmark that scores the runnable harness a model builds rather than the answer it returns. Starting from a seed that scores 0, 6 creator LLMs construct harnesses across 5 benchmarks and 2,207 tasks, then evolve them from execution feedback. Self-built harnesses match human references on writing and ML experimentation but trail on code and search, and only 34 of 64 evolution changes move the same direction on held-out tasks.\n\nThe post [Can LLMs Engineer Their Own Agent Harness? ByteDance Seed’s HarnessDev Says Only 34 of 64 Changes Generalize](https://www.marktechpost.com/2026/09/11/can-llms-engineer-their-own-agent-harness-bytedance-seeds-harnessdev-says-only-34-of-64-changes-generalize/) appeared first on [MarkTechPost](https://www.marktechpost.com).", "url": "https://wpnews.pro/news/can-llms-engineer-their-own-agent-harness-bytedance-seeds-harnessdev-says-only", "canonical_source": "https://www.marktechpost.com/2026/09/11/can-llms-engineer-their-own-agent-harness-bytedance-seeds-harnessdev-says-only-34-of-64-changes-generalize/", "published_at": "2026-09-11 22:01:08+00:00", "updated_at": "2026-09-11 22:23:34.766190+00:00", "lang": "en", "topics": ["ai-agents", "large-language-models", "ai-research", "ai-safety"], "entities": ["ByteDance Seed", "SUTD", "Georgia Tech", "M-A-P", "TokenWave.AI", "HarnessDev"], "alternates": {"html": "https://wpnews.pro/news/can-llms-engineer-their-own-agent-harness-bytedance-seeds-harnessdev-says-only", "markdown": "https://wpnews.pro/news/can-llms-engineer-their-own-agent-harness-bytedance-seeds-harnessdev-says-only.md", "text": "https://wpnews.pro/news/can-llms-engineer-their-own-agent-harness-bytedance-seeds-harnessdev-says-only.txt", "jsonld": "https://wpnews.pro/news/can-llms-engineer-their-own-agent-harness-bytedance-seeds-harnessdev-says-only.jsonld"}}