Recent AI research introduces RL-driven agentic optimization using dense token-level supervision and progress advantage signals to stabilize training. PhysiFormer injects 3D geometric reasoning into d…
Researchers proposed Dynamic-dLLM, a training-free framework to accelerate Diffusion Large Language Models (dLLMs) by dynamically allocating cache budgets and calibrating decoding thresholds. The meth…
A developer released ishvacerto, an open-source Python tool that verifies AI-generated code by running it against doctests, user tests, or a reference implementation, returning a counterexample if the…
Vibe coding—prompting an AI agent and shipping unread output—introduces a measurable defect tax that makes it unsuitable for mission-critical systems, with 45% of AI-generated code containing security…
A new guide explains how to build a personal AI model leaderboard by running blind comparisons and tracking results over time, arguing that public benchmarks are insufficient for task-specific perform…
LLM benchmarks like MMLU and HumanEval are irrelevant for most businesses building AI products, as they measure generic performance rather than specific system tasks. Teams should instead build custom…
Anthropic released Claude 3.5 Sonnet, a new AI model that outperforms the previous top-tier Claude 3 Opus in intelligence, speed, and cost. The model achieves a 64% solve rate on internal agentic codi…
Kog AI achieved over 3,000 output tokens per second per request for an FP16 2B model on a single 8× MI300X node using a monokernel that eliminates per-token kernel launches. The technique collapses th…
A cloud architect evaluated four Chinese LLM families—DeepSeek, Qwen, Kimi, and GLM—in a multi-region production pipeline serving thousands of requests per second via Global API's unified endpoint. De…
Mendral, an AI DevOps agent developer, cannot provide a single accuracy metric for its CI failure diagnosis agent despite processing 36,564 investigations across 5.7 million CI jobs and 14.4 billion l…
Researchers have developed Fast-dLLM++, a training-free extension to diffusion large language models that accelerates inference by selecting parallel token commit sets based on the full sorted confide…
Anthropic's Claude Opus 4.8 autonomously built a full economic simulation benchmark from scratch, designing test scenarios, generating ground-truth outcomes, and writing evaluation rubrics without hum…
A new benchmark comparing agent runtime performance across C++, Python, TypeScript, and Rust found that C++ achieved a peak memory footprint of approximately 93 MiB while running 100 concurrent coding…
Researchers have introduced DeepSuite, a new benchmark for evaluating AI coding agents that tests them using short, natural-language prompts requiring complex solutions across entire code repositories…
Claude Opus 4.7 outperformed GPT 5.5 on repository understanding and multi-file generation tasks in the DeepSuite benchmark, a new coding evaluation designed to mirror real-world developer workflows. …
A developer argues that current LLM benchmarks like MMLU, HumanEval, and SWE-bench measure only knowledge recall and one-shot task completion, not the behavioral traits—such as debugging, adaptation, …
The Gemma 4 26B model was the first local AI to achieve a perfect score on the HumanEval benchmark, including solving the notoriously difficult problem 145. This problem requires sorting integers by t…