Large language model (LLM) news — GPT-4, Claude, Gemini, Llama, Mistral and the latest research on training, fine-tuning, RLHF, and deployment of LLMs.
A new benchmark called SlackBench, introduced in arXiv paper 2611.05512v1, evaluates AI agents on realistic research and engineering workflows that require joint reasoning over workplace communication and evolving code r…
A study of 49 writer-grader model pairs across seven models from Anthropic, OpenAI, Google, DeepSeek and Z.AI found average scores on end-to-end data science projects rose from 59 per cent to 76 per cent after three roun…
Researchers posted arXiv:2611.05556v1, introducing AGORA (Aggregation Of Reliability-weighted Assessments), an item-response-theory aggregator for combining multiple LLM judges that learns each judge's reliability from a…
A paper posted to arXiv (2611.05611v1) presents Hybrid Spark, a speculative decoding framework that runs a model-based drafter and a model-free n-gram drafter on every step and gives each request the better of their chai…
A new forecasting framework called Context as Covariates (CoCo) improves mean absolute error by up to 12% over state-of-the-art time series foundation models and LLM forecasters including GPT-4o, according to the arXiv p…
A new arXiv paper (2611.05591v1) proposes CGRPO, a method that replaces the fixed rollout budget of Group Relative Policy Optimization with calibrated conformal thresholds over task-specific nonconformity scores. Across …
A study posted as arXiv:2611.05609v1 finds that multi-teacher on-policy distillation (MOPD) and on-policy distillation (OPD) can consolidate in-distribution performance while leaving out-of-distribution generalization be…
A new arXiv paper (2611.05655v1) introduces a framework for determining when two sparse autoencoder (SAE) latents represent the same feature, using an energy-weighted optimal-transport plan under a specified similarity g…
A new arXiv paper (2611.05451v1) introduces FlowShift, a parameter-efficient post-training framework that converts pre-trained discrete Unified Multimodal Models into continuous rectified-flow generators by replacing dis…
A new benchmark, Bio-VQA, finds that general visual capabilities in vision-language models do not transfer to biological organisms, with the best of 18 open-weight models counting animals exactly on only 41.7% of images …
A study posted as arXiv:2611.06116v1 found that the harness around a fixed open-weight language model can swing its compliance with banking conduct rules by as much as 11.5 percentage points, enough to move one model fro…
A new arXiv paper (2611.06715v1) introduces Rel-Bloom, a controlled multilingual diagnostic of 450 multiple-choice questions built from 30 Wikidata relations, and reports that accuracy drops sharply from Apply to Analyze…
Researchers posted arXiv paper 2611.06501v1 introducing CRESiST, an end-to-end interleaved read-write framework for simultaneous speech translation built on an omni-modal large language model. CRESiST interleaves speech …
A new arXiv paper (2611.06639v1) proposes a probabilistic standard for reproducibility in large language model research, arguing that LLM output varies across models and versions and can significantly affect downstream s…
A study posted as arXiv:2611.06545v1 found that LLM agents' self-written handoff summaries sometimes assert permissions or requirements the underlying record does not support, and that neutralizing false approvals cut un…
A new arXiv paper (2611.06808v1) introduces FinDendro, a benchmark of 2,226 dated U.S. facts from 1999 to 2026, and finds that frontier LLMs' self-reported knowledge understates what they actually know, undermining commo…
A new arXiv paper (2611.06806v1) proposes detecting observable AI behaviors that act as risk factors for psychological harm in LLM conversations rather than trying to measure harm directly, operationalizing eight such be…
A new benchmark, TASC (Temporal Action Safety in Conversation), finds that across 13 large language model assistants, models account for time-dependent constraints in only 30.5% of cases where a proposed action violates …
Researchers posted arXiv paper 2611.06812v1 introducing SAUCE, a reference-free, fact-based evaluation pipeline for cross-lingual conversational speech summarization that scores summaries by measuring entailment between …
A benchmark study of 38 Massive Text Embedding Benchmark (MTEB) tasks across four transformer-based embedding models found that int8 quantization is statistically indistinguishable from float32 for pair classification an…