BatchDAG reduces LLM calls by 47x with entity-aware batching

A new architecture called BatchDAG reduces LLM API calls by up to 47x through entity-aware batching, enabling queries over 50,000+ meetings in under 60 seconds at $0.02–$0.24 each, according to a preprint on arXiv. The system compiles natural language requests into deterministic, parallelized execution graphs of SQL, vector search, and code, achieving a 98.8% valid plan execution rate and a 77% evidence rate.

arXiv https://arxiv.org/abs/2607.18241 BatchDAG reduces LLM calls by 47x with entity-aware batching Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated. Instead of a ReAct-style agent looping sequential tool calls, an LLM here plans a typed DAG once and hands it to a deterministic engine, with entity-aware batching cutting LLM calls up to 47x and enabling queries over 50,000+ meetings in under 60 seconds at $0.02–$0.24 each. The practical takeaway: for exhaustive cross-entity analysis, plan-then-execute beats agentic reasoning loops on cost, latency, and provenance 77% evidence rate , and structured JSON intermediates instead of prose summaries measurably cut hallucination. If you're running RAG or analytical agents at scale, this is the architecture to steal—separate one-shot planning from parallel deterministic execution. An entity-aware batching architecture that groups data before LLM fan-out reduces compounding API calls by up to 47x, cutting query execution costs over 50,000 documents to under twenty-four cents. Compiling natural language requests into deterministic, parallelized execution graphs of SQL, vector search, and code achieves a 98.8 percent valid plan execution rate in under 60 seconds. This layout enables you to completely replace multiple hand-engineered, pipeline-specific workflows with a single, general-purpose orchestration layer that eliminates sequential agent latency.