A new method called COPA reduces attack success rates by 6.3× versus the best static baseline and 4.4× on average across lifelong attack streams, while retaining 92% defense on month-old attacks compa…
Jie Tang's team at Z.ai argues that LLM scaling should optimize for fixed inference budgets, favoring smaller, deeper architectures trained longer over wider models. Their ablation shows a 7B model tr…
IBM Research has introduced BenchDrift, an open-source tool that quantifies how much large language model benchmark scores shift when test prompts are rephrased without changing meaning, revealing tha…
A new method called Forward-Pass-Only MLP training (FPO) adapts large language models without a backward pass, achieving 2.7–3.2x the throughput of standard fine-tuning at ~40% less peak training memo…
A developer's guide outlines the three pillars for building production-ready AI agents: benchmarking, cost optimization, and tooling. It emphasizes task-level evaluation over static benchmarks, using …
A new study introducing BenchDrift shows that rephrasing benchmark problems while keeping meaning and answer fixed flips LLM correctness in both directions across eight models and three benchmarks (GS…
A new arXiv study (2608.09942v1) finds that chain-of-thought (CoT) prompting is not a universal enhancer of large language model reasoning, but rather a bandwidth bypass that helps only when serial co…
Researchers introduced LLM Agents Factory, a retrieval-based framework that constructs domain-specific LLM agents on demand from a base of over 20,000 predetermined agent profiles, achieving accuracy …
Computer Anthology, a new benchmark family from the ACL Anthology repository, evaluates AI models on reasoning over newly published NLP papers, with monthly updates to prevent data contamination. The …
Researchers introduced Gaokerena, a family of compact Persian medical language models, with Gaokerena-V improving a translated medical MMLU benchmark from 46.28% to 49.31% and Gaokerena-R achieving 52…
Researchers introduced TokenSwap, a method that replaces textual concepts with semantically aligned images to create image-interleaved inputs, and found that across 42 multimodal large language models…
A new arXiv preprint (2607.28801v1) introduces a dataset-centric meta-evaluation framework that audits LLM benchmark datasets at the sample level across five dimensions, revealing internal heterogenei…
A study submitted to arXiv on 30 Jul 2026 found that gently-compressed large language models (LLMs) can pass standard data-free quality guards—including perplexity, MMLU accuracy, and output-fidelity …
Redwood Research finds that hint-based chain-of-thought faithfulness evaluations still work on Claude models, contradicting Anthropic system card claims that recent models no longer use hints. The rep…
A study at 120B scale finds that inserting constitutional content during midtraining produces durable alignment gains, with constitutionally midtrained models outperforming a control on alignment gene…
A new open-source benchmarking framework, Inspect India Evals, reveals that large language models (LLMs) tested on Indian linguistic and cultural contexts show significant performance gaps, with Sarva…
Researchers present EaaS, a cloud-native reference architecture that operationalizes AI evaluation methods as six stateless Kubernetes microservices, achieving sub-2ms p99 latency for conformal predic…
Three prompting techniques that worked in 2023 now actively cost accuracy in 2026, according to recent research. Expert personas reduced factual accuracy to 68.0% versus 71.6% without on the MMLU know…
A new permutation diagnostic reveals that position bias in multiple-choice LLM benchmarks is only detectable within a 60-95% base-accuracy Goldilocks zone, with frontier-tier models operating above th…
A new study from arXiv introduces an uncertainty-aware trust estimation method for multi-LLM systems, adapting Cooke-style log weighting from structured expert judgment to aggregate predictions from h…