Large language models achieve less than 24% success on context learning tasks, with specification acquisition identified as the primary bottleneck rather than content access. Researchers introduced Pr…
Anthropic's Sonnet 5 was benchmarked against leading models in a new evaluation that combined human and LLM scoring. The results showed Sonnet 5 excelled in PRD quality but lagged in agent personality…
Claire tested Anthropic's Sonnet 5 against Sonnet 4.6, Opus 4.8, GPT-5.5, and Gemini 3 Pro using a custom benchmark, finding Sonnet 5 priced competitively but ranking near the bottom in her preference…
Researchers introduced a semantic-layer-mediated NL2SQL agent that decouples semantic intent from physical SQL execution, achieving 94.15% execution accuracy on the Spider2-snow benchmark and ranking …
Anthropic released Sonnet 5, and Claire Vo tested it against four other frontier models using a custom eval harness built with Claude Code. The blind comparison across PRD quality, prototype generatio…
Researchers introduced NormAct, a benchmark evaluating whether multimodal large language models comply with hidden social norms during embodied planning. Tests on GPT-5.4, Claude Opus 4.7, and Gemini …
A developer used agentic AI to decompile a Chinese label printer's Android app and create a Go script that prints PDFs via Bluetooth on Linux, after struggling with poor print quality under CUPS. The …
A study comparing LLM-generated research methods against 1,000 arXiv computer-science papers found that LLMs suggest a much narrower range of methods, with provider choice showing the strongest imbala…
Researchers developed VibeThinker-3B, a 3-billion-parameter language model that achieves reasoning performance matching or exceeding models orders of magnitude larger, scoring 94.3 on AIME26 and 80.2 …
Frontier LLMs including GPT-5.5, Gemini 3 Pro, and Opus 4.7 failed to correctly solve over two-thirds of problems in ParallelKernelBench, a new benchmark for multi-GPU kernel generation. The benchmark…
A 3-billion-parameter model from Weibo scored 94.3 on the AIME 2026 math Olympiad, outperforming Google's Gemini 3 Pro which scored 91.7. The model is MIT-licensed and its success challenges assumptio…
Openfusion, an open-source drop-in compound-model proxy, lets users point any OpenAI-compatible tool at it to fan out prompts to a panel of LLMs in parallel, then a judge model synthesizes a single an…
A Lenz Research study found that five frontier AI models disagreed on 67% of 1,000 real-world fact-check claims, with 34% showing substantive disagreements. The models scored 0.639 on Krippendorff’s a…
A developer has created a system called "skill distillation" that uses frontier AI models like Opus 4.7 and GPT-5.1 to write procedural skill files, which are then executed by smaller local models lik…