{"slug": "buysideflow-a-point-in-time-benchmark-for-llm-agents-in-investment-research", "title": "BuySideFlow: A Point-in-Time Benchmark for LLM Agents in Investment Research", "summary": "A new arXiv paper (2611.12966v1) introduces BuySideFlow, a point-in-time benchmark of 408 practitioner-derived Chinese-language investment-research tasks spanning equities, funds, fixed income and macroeconomics, drawn from a fixed snapshot of a licensed production financial database. Across nine agent configurations, result accuracy ranged from 16.2% to 36.8%, and an expert audit of 900 runs on 100 shared tasks found point-in-time violations in 193 of the 380 runs that passed result checks (50.8%). The authors conclude that output agreement alone does not establish temporal validity for LLM agents generating SQL and SQL-Python workflows in investment research.", "body_md": "arXiv:2611.12966v1 Announce Type: new \nAbstract: Large language models (LLMs) can generate SQL and SQL--Python workflows for investment research, but ensuring point-in-time (PIT) validity remains challenging. Agents must enforce disclosure cutoffs and select the appropriate historical states and record versions throughout data retrieval and financial analysis. We introduce \\textsc{BuySideFlow}, a benchmark of 408 practitioner-derived Chinese-language tasks spanning equities, funds, fixed income, and macroeconomics over a fixed snapshot from a licensed production financial database. We provide executable references and a taxonomy of temporal errors, pairing result checks with expert audits of executed workflows. We evaluate nine agent configurations on all tasks, with result accuracy ranging from 16.2\\% to 36.8\\%. An audit of 900 runs on 100 shared tasks confirms PIT violations in 193 of the 380 runs that pass result checks (50.8\\%), showing that output agreement alone does not establish temporal validity.", "url": "https://wpnews.pro/news/buysideflow-a-point-in-time-benchmark-for-llm-agents-in-investment-research", "canonical_source": "https://arxiv.org/abs/2611.12966", "published_at": "2026-11-11 05:00:00+00:00", "updated_at": "2026-11-11 05:23:10.556212+00:00", "lang": "en", "topics": ["large-language-models", "ai-agents", "ai-research", "machine-learning"], "entities": ["BuySideFlow", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/buysideflow-a-point-in-time-benchmark-for-llm-agents-in-investment-research", "markdown": "https://wpnews.pro/news/buysideflow-a-point-in-time-benchmark-for-llm-agents-in-investment-research.md", "text": "https://wpnews.pro/news/buysideflow-a-point-in-time-benchmark-for-llm-agents-in-investment-research.txt", "jsonld": "https://wpnews.pro/news/buysideflow-a-point-in-time-benchmark-for-llm-agents-in-investment-research.jsonld"}}