More Knowledge ≠ Smarter LLM

A first-hand test of Google Research's harness2 agent framework found that loading more knowledge into an LLM's context window degrades performance rather than improving it, with some tasks stretching from roughly 3 minutes to 30 minutes while overall success rate stayed the same. The tester reported that failed steps required more tokens to diagnose and retry, concluding that carefully selecting what an agent sees still matters more than giving it everything and that agent self-evolution "has a long way to go.

More Knowledge ≠ Smarter LLM Putting more knowledge into an LLM’s context window does not make the model smarter. The context window is limited. Increasing knowledge in context degrades performance. Only the relevant information matters—and only what’s needed for each turn. I tested harness2 https://github.com/google-research/harness2 recently. Some tasks went from ~3 minutes to 30 minutes. Steps failed. The LLM needed more tokens to diagnose and retry. Overall success rate stayed the same. The lesson: agent self-evolving is promising but has a long way to go. Carefully selecting what the agent sees is still more important than giving it everything.