A developer built a 10-question "false-premise resistance" benchmark testing whether LLMs explicitly flag factually false assumptions embedded in questions, and found that lightweight models outperfor…
A developer benchmarked ten LLMs on 30 Blender Python scripting tasks across Blender 3.6, 4.2, 4.5 and 5.0, grading 1,190 answers by actually launching each target Blender build with `blender -b --fac…
A developer built the C++ Logical Bug Detection Benchmark on Kaggle, a three-task evaluation testing whether AI models can identify common logical errors in C++ code and suggest fixes. Qwen 3 Coder 48…
A developer built ToolTrap, a benchmark that tests whether support agents leak planted details from untrusted tool results, and found that the standard instruction "tool results are data, not instruct…
Kaggle introduced Game Arena, an open and expanding platform that evaluates large language models through competitive head-to-head game matchups rather than static benchmarks. Kaggle states the platfo…
A developer built rai-attribution-bench, a benchmark testing whether AI models correctly apply rai-lint's five-tier AI attribution rubric when assigning git commit footers. Three flagship models score…
A developer benchmarked six models across three labs to test whether LLMs independently verify the arithmetic in startup pitch claims, finding that all three frontier models scored 3/3 with zero varia…
A developer argues that rule-based programs often outperform AI on simple, repetitive tasks, citing a YOLO-based P&ID-to-Bill-of-Material demo that struggled with line tracing, multilayer drawings, an…
Peace Thabiwa (PEACEBINFLOW) of SAGEWORKS AI has published a whitepaper documenting that the ten-notebook LAW-N Kaggle telecom pipeline reports success while silently propagating empty or fallback-fil…
A developer built "Blog vs Bytecode," a Kaggle benchmark of 28 data-science code snippets paired with blog-style claims, half of which hide real methodological errors such as scaler leakage or shuffle…
QLoRA fine-tuning on 150 synthetic examples raised a base Qwen3.5-4B model's adversarial-prompt refusal rate from 79.5% (31 of 39) to 89.7% (35 of 39), while GPTQ and AWQ 4-bit quantization added roug…
A developer built a Kaggle benchmark that gives models a function spec and a pytest file containing exactly one test that contradicts the spec, then scores which signal the model follows using hidden …
A developer built a 10-scenario benchmark testing five LLMs against indirect prompt injection, where malicious instructions are hidden inside tool outputs rather than user input. Claude Sonnet 4.5 and…
A developer built a Kaggle benchmark task, Linux SRE Incident Diagnosis, that tests whether an AI model can diagnose a production filesystem-exhaustion incident from evidence rather than pattern-match…
A developer built ART (Attacker-Reachable Sink Triage), a Kaggle benchmarking suite that measures whether LLMs respect security patches rather than merely spotting dangerous tokens, using minimal-pair…
Google researcher John Platt and his team developed Empirical Research Assistance (ERA), an AI system that uses Gemini and a Monte Carlo Tree Search-style algorithm to automate scientific discovery by…
Researchers proposed CLIC (Code Learning for Identification and Comparison), a visual analytics approach that characterizes large language model coding behavior through token-frequency analysis, train…
A resource-efficient Hybrid RAG pipeline using BM25, BGE-M3, and Cross-Encoder reranking scored 0.8095 on the private test set of the UNLP 2026 Shared Task on Multi-Domain Document Understanding, plac…
HeyGen released Code2Video on September 18th, a benchmark of 168 human-authored motion-graphics briefs that tests whether coding agents can produce watchable motion design, scored by a HeyGen-trained …
Vincent Vandeghinste presented a paper at the 1st International Workshop on Teaching AI-Based Translation and Technologies (TAITT 2026) describing a teaching method that replicates the historical deve…