arXiv:2611.21647v1 Announce Type: new Abstract: Generative AI agents can help journalists explore large document collections. But exploratory workflows can require substantial token use and time, especially when the task is broad and the relevant subquestions are not yet known. This paper introduces FieldGuide, an agent-facing corpus map that organizes a document collection into a topic hierarchy and exposes tools for corpus orientation, scoped search, and bounded reading. I evaluate FieldGuide on five journalistic document-exploration tasks using 38 Chicago Police Department public-records documents, comparing it with raw-file access and retrieval-augmented generation. Across the five tasks, FieldGuide used 27% fewer tokens and 41% less elapsed time than raw-file access; relative to RAG, it used 38% fewer tokens and 21% less elapsed time. An accuracy audit identified four minor issues in FieldGuide's responses, comparable to the other conditions. In blind ratings of generated reporting leads, FieldGuide's mean ratings were higher than RAG's and lower than the raw-file condition's. These descriptive results suggest that corpus orientation may reduce runtime token use and elapsed time without severe accuracy failures. They also motivate further study of whether selective access limits broad, cross-document discovery.
FieldGuide: A Corpus Map for Token-Efficient Agentic Document Exploration
A new arXiv paper (2611.21647v1) introduces FieldGuide, an agent-facing corpus map that organizes a document collection into a topic hierarchy with tools for corpus orientation, scoped search and bounded reading. Tested on five journalistic document-exploration tasks using 38 Chicago Police Department public-records documents, FieldGuide used 27% fewer tokens and 41% less elapsed time than raw-file access, and 38% fewer tokens and 21% less elapsed time than retrieval-augmented generation, with an accuracy audit finding four minor issues in FieldGuide's responses, comparable to the other conditions. Blind ratings of generated reporting leads placed FieldGuide above RAG and below the raw-file condition, suggesting corpus orientation can cut runtime token use and elapsed time without severe accuracy failures.
Give document-exploration agents a topic-hierarchy corpus map with scoped search and bounded reading instead of raw files or plain RAG; the paper measured 27-38% fewer tokens and 21-41% less elapsed time.
more in playbook: Retrieval and RAGRun your AI side-project on zahid.host
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.