BatchDAG: LLM-Planned Execution Graphs for Scalable Ad-Hoc Analysis Over Enterprise Data

Brevian.ai's BatchDAG system, which uses an LLM to generate typed directed acyclic graphs of operations for ad-hoc analysis over enterprise data, achieves quality comparable to expert-designed pipelines while reducing LLM calls by up to 47x through entity-aware batching. In production, BatchDAG processes queries over 50,000+ meetings in under 60 seconds at per-query costs of $0.02-$0.24 using GPT-5.1 pricing.

arXiv:2607.18241v1 Announce Type: new Abstract: Large language models LLMs excel at analyzing individual documents but break down on exhaustive, cross-entity analytical questions over enterprise-scale datasets due to context overflow, loss of per-entity attribution, and linear latency from sequential tool calls. We present BatchDAG, a system in which an LLM generates a typed directed acyclic graph DAG of operations -- SQL queries, semantic searches, in-memory transforms, parallel fan-outs, and single-shot analyses -- which a deterministic engine evaluates with topological-wave parallelism and structured JSON data flow. A key optimization, entity-aware batching, groups rows by logical entity before fan-out, reducing LLM calls by up to 47x. BatchDAG is not primarily an accuracy improvement over hand-optimized pipelines; rather, it is a general-purpose orchestration layer that replaces multiple hand-engineered workflows with a single system that generates the appropriate execution strategy from natural language. In controlled experiments on 12 transcript-heavy queries, BatchDAG 3.74/5 achieves quality comparable to an expert-designed pipeline 3.25/5 and significantly outperforms a ReAct agent 3.09/5, p<0.01 , with superior provenance 77% transcript evidence rate vs. 46-60% for baselines . A controlled ablation shows structured JSON intermediates reduce hallucinations by 27% versus prose summaries paired t-test, p=0.107, n=12 . The planner achieves 98.8% valid-DAG rate across 300 planning calls. In production at Brevian.ai, BatchDAG processes queries over 50,000+ meetings in under 60 seconds, with measured per-query costs of $0.02-$0.24 at published GPT-5.1 pricing.