Large language model (LLM) news — GPT-4, Claude, Gemini, Llama, Mistral and the latest research on training, fine-tuning, RLHF, and deployment of LLMs.
A new study from Japan's AI Security Institute finds that LLM-as-judge safety evaluations are not reproducible even at temperature 0, with per-item disagreement up to 50% across runs. The researchers tested 690 API calls…
A developer building tgo, a TypeScript-to-Go compiler, reports progress on supporting Node libraries and compiling Lodash, with 122 of 316 entry points still failing. The project's complexity forced a complete overhaul o…
Researchers introduced Know2Guess, a contamination-aware multi-zone benchmark with 1,200 items across five domains to evaluate large language models' ability to distinguish answerable knowledge from abstention-expected u…
A new study on Llama 3.1 8B finds that helpfulness post-training (SFT and GRPO) significantly degrades animal compassion values compared to coding-domain post-training, with a 35.7% vs. 65.2% gap on the Animal Harm Bench…
Researchers released ProfileFoundry, a synthetic dataset of 100,000 person objects with 709,228 events, 40,338 households, and 52,491 employers, designed for evaluating LLM agents on memory, privacy, and tool-use tasks. …
Researchers found that applying Nonviolent Communication constraints to large language models reduces conversational escalation in conflict-prone dialogues, improving trustworthiness without requiring extensive retrainin…
Researchers developed a methodology to transform Hindi WordNet into 1.25 million instruction-response pairs, fine-tuning a 12B-parameter language model to create a specialized conversational AI for low-resource languages…
Researchers present the first systematic survey of unified vision-language perception in Multimodal Large Language Models (MLLMs), tracing the paradigm evolution through a five-stage taxonomy and identifying open challen…
Researchers propose SOLAR, an auxiliary objective for supervised fine-tuning that aligns soft-token representations across languages using English as a pivot, improving cross-lingual reasoning in multilingual large langu…
Researchers propose DocArena, a fully automated pipeline that transforms raw document collections into controllable training environments for document search agents, using multimodal large language models to structure do…
Researchers propose VIGIL, a reinforcement-learning post-training framework that reduces hallucinations in multimodal large language models by maximizing mutual information between visual input and generated responses. T…
Researchers introduced Hankel Reduced order Model (HRM) adapters, a state space model-based fine-tuning method that outperforms LoRA on long-context tasks like QuALITY and QMSum by leveraging FFT-based parallel scan for …
Researchers introduced an LLM-powered pipeline to compare governance structures of AI agent protocols ERC-8004 (permissionless) and Google A2A (corporate-led), analyzing 4,323 governance records. They found that while bo…
Researchers developed a provenance-aware, knowledge-graph-based multi-agent framework that integrates 466,525 Reddit posts, 60,782 WebMD reviews, and 20 years of FDA adverse event data for nine antidepressants. The syste…
Researchers at arXiv introduced a new benchmark evaluating tool-augmented LLM agents on 243 real-world energy market analytics tasks, revealing that current models struggle with live data retrieval and multi-step reasoni…
Researchers at arXiv identified a failure mode in prompt-composed agentic systems called compositional behavioral leakage (CBL), where editing one prompt module silently shifts the behavior of others due to transformer s…
A new paper argues that Ray Kurzweil's theory of accelerating returns applies primarily to executional and infrastructural capability, not to the qualitative reasoning essential for scientific discovery. The authors poin…
Researchers introduced narration-of-thought (NoT), a system prompt that structures chain-of-thought reasoning into five sections to improve ethical reasoning in large language models. On 100 DailyDilemmas scenarios, NoT …
A developer building an AI-powered Due Diligence and Compliance Reporting platform on Amazon Bedrock with Claude found that system prompts matter more than user prompts for production reliability. The team discovered tha…
A developer built otp-gateway, a zero-dependency Python library that reads OTP codes from email inboxes via IMAP, eliminating the need for external APIs or services. The library supports generating email aliases and wait…