Kolab: Benchmarking Human-Agent Collaboration in Long-Horizon Workflows

A benchmark called Kolab evaluates human-agent collaboration in long-horizon workflows, finding that humans place greater importance on ongoing interaction with AI agents, such as revising plans, than on an agent's ability to complete a task in a single attempt. The work focuses on what happens to agents' contributions when people collaborate with them on long-horizon tasks.

When humans collaborate with AI agents on long-horizon tasks, what happens to the agents' contributions? For solving long-horizon tasks, humans place greater importance on ongoing interaction with AI agents e.g., revising plans than on an agent's ability to complete the task in a single attempt. H