A developer built a 1.5B parameter text-to-SQL model that outperforms a 7B model on the Spider benchmark, achieving 71.5% accuracy through self-consistency voting. The project, starting with Qwen2.5-0…
A new dependency-free audit script from developer Ashish Sinha found that six human-authored Hugging Face benchmarks—BIRD-CRITIC 1.0, Spider, GSM8K, and HumanEval—are clean, with only 18 genuinely dup…
AutoThinkSQL, a framework integrating an auto-thinking mechanism into Supervised Fine-Tuning and Direct Preference Optimization for Text-to-SQL, enables dynamic bypassing of reasoning for simple queri…
GRID (Grammar-Railed Decoding), a grammar-constrained decoding engine for enterprise SQL generation, achieves a median per-token mask latency of 3.6-6.7 microseconds using Rust kernels, outperforming …
Turing Award winner and Postgres creator Mike Stonebraker revealed that large language models score 0% accuracy on real-world data warehouse queries, contradicting the 80-85% reported on popular bench…
A study on text-to-SQL systems found that verification-based signals, such as LLM judges, outperform black-box signals like self-consistency in predicting query correctness, with a two-provider ensemb…
Researchers introduced Outcome Reward Models (ORMs) for Text-to-SQL verification, using semantic scoring to outperform traditional heuristics. Their GradeSQL framework automates candidate generation a…
Researchers introduced GradeSQL, a framework for training Outcome Reward Models (ORMs) to improve test-time verification in Text-to-SQL tasks. ORM-based selection outperformed execution-based Best-of-…