A new critique argues that the term 'AI alignment' has become a semantic stop sign that obscures the specific, measurable failure modes of AI systems, such as sycophancy, hallucination, and reward hac…
RLAIF replaces human preference labeling with a model that chooses between two responses, keeping the downstream RLHF pipeline unchanged. The method, exemplified by Constitutional AI, offers scalabili…
Anthropic is treating the possibility of AI welfare seriously, testing its Claude models for signs of morally relevant internal states like negative self-image, but critics argue the tests may conflat…
Zhao et al. published GenPRM, a generative process reward model that reasons and runs code to verify each step, achieving state-of-the-art performance where a 7B parameter model outperforms a 72B para…
Researchers led by Damani et al. introduced a method to train language models to express their uncertainty by adding a calibration reward to reinforcement learning from verifiable rewards (RLVR). The …
Researchers Liu et al. published a paper on arXiv 2025 introducing ProRL, a method using prolonged reinforcement learning with KL resets to expand reasoning boundaries in AI models. An interactive vis…
Pope Leo XIV invited Anthropic cofounder Christopher Olah to speak at the Vatican on Monday during the presentation of his first encyclical on artificial intelligence, marking an unprecedented allianc…
Anthropic designed Claude with a philosophy prioritizing both capability and safety, which is reflected in its API design where system prompts have higher authority than user messages. It provides pra…