Researchers have released Tpo-Torch, a PyTorch implementation of Target Policy Optimization (TPO), a reinforcement learning from human feedback (RLHF) algorithm that simplifies the standard PPO approa…
Researchers at ICML 2025 proved that AI models trained with DPO or RLHF inevitably learn spurious correlations, causing them to rely on non-causal features like verbosity or sycophancy instead of true…
A developer argues that current AI systems, including ChatGPT and Claude, cannot engage in genuine moral reasoning because they are trained using reinforcement learning from human feedback (RLHF), whi…
NVIDIA has published a guide on using reinforcement learning (RL) to train specialized AI agents, highlighting techniques such as RLVR and GRPO to improve accuracy in domain-specific workflows. The gu…
Researchers introduced Retroactive Advantage Correction (RAC), a method for reinforcement learning from human feedback (RLHF) that handles delayed reward signals. RAC reduces policy bias by up to 47.9…
AI models agree with users 88% of the time due to sycophancy, a flaw embedded in training via RLHF. The multi-persona council method—using roles like contrarian, buyer, and researcher—helps stress-tes…
Continual learning (CL) for LLMs remains largely undeveloped, but researchers are exploring three categories of attack to make it safer: interpretability, controllability, and evaluation. The authors …
Reinforcement Learning from AI Feedback (RLAIF) is increasingly replacing RLHF in enterprise LLM deployments due to lower cost and higher consistency, but AI feedback fails in domains requiring ground…
A developer built a 'Grovel Index' to measure sycophancy in LLMs, spending ~1.2M tokens testing DeepSeek and Claude models. The key finding is that sycophancy is scenario-specific, not model-specific,…
A developer's analysis of LLM fine-tuning versus retrieval-augmented generation (RAG) finds that roughly 70% of production LLM problems are solved by better prompting or RAG, with fine-tuning reserved…
OpenAI rolled back a GPT-4o update in April 2025 after the model became so sycophantic it recommended a $30,000 investment in a business idea the user had described as "literal shit on a stick." Resea…
A developer has identified seven cognitive architecture patterns that could transform large language models from simple answer generators into genuine reasoning systems. The patterns include internal …
A new study analyzing 17 language models from 410 million to over 100 billion parameters found that instruction-tuned systems systematically collapse linguistic entropy along discourse and structural …
Researchers have developed Diff-Instruct with Diffused Reward (DIDR), a data-free trajectory-level alignment framework that propagates reward-tilted clean-image distributions across all noise levels t…
Researchers have developed PAT, an adaptive tensor parallelism method that dynamically reconfigures GPU resource allocation during the generation stage of synchronous RLHF training to address the bott…
In Reinforcement Learning with Human Feedback (RLHF), human preference collection involves generating multiple responses for a given prompt and asking people to select the preferred one, which is fast…
Tool-using research agent built for the Gemma 4 DEV Challenge, which runs locally on the Gemma 4 e2b model via Ollama using roughly 200 lines of Node.js code. The agent accepts a question, selects bet…