{"slug": "neyman-orthogonal-activation-steering", "title": "Neyman Orthogonal Activation Steering", "summary": "A new arXiv paper (2610.33217v1) proposes Neyman orthogonal activation steering, an estimator that separates the steering direction LLMs target from the estimator used to recover it, addressing a conflation the authors say is common in the literature. The authors frame existing activation-steering methods as plug-in estimators of the average gradient of an outcome regression with a binary contrastive label, extend them to continuous outcomes such as log-probabilities or oracle ratings, and add a complementary Stein-score-based estimator drawn from debiased machine learning. Combining the two routes yields an orthogonalized estimator with classical double-robustness properties, which the authors illustrate on activation steering across several benchmark datasets.", "body_md": "arXiv:2610.33217v1 Announce Type: new \nAbstract: Activation steering is a popular approach for shifting LLM output toward a desired behavior, and increasingly for generating realistic synthetic data. The standard recipe estimates a steering direction by contrasting LLM activations on prompts with and without a target concept, such as truthfulness. In this paper, we separate the estimand (the steering direction these methods target) from the estimator used to recover it, two notions the literature often conflates. We frame nearly all existing methods as plug-in estimators of the average gradient of an outcome regression, with the binary contrastive label fixed as the outcome. Using this framing, we can immediately incorporate flexible modeling approaches and extend existing methods to allow for continuous outcomes such as log-probabilities or oracle ratings. Flexible ML models that fit the outcome well, however, might not estimate the gradient well, which motivates estimators that target the gradient directly. Drawing on the debiased machine learning literature, we develop a complementary estimator built on the Stein score, the Riesz representer of the gradient functional. Combining the two routes gives an orthogonalized estimator with classical double-robustness properties. We illustrate the framework on activation steering applied to a number of benchmark datasets.", "url": "https://wpnews.pro/news/neyman-orthogonal-activation-steering", "canonical_source": "https://arxiv.org/abs/2610.33217", "published_at": "2026-10-28 04:00:00+00:00", "updated_at": "2026-10-28 04:13:55.453028+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-research"], "entities": ["arXiv", "Neyman orthogonal activation steering", "Stein score", "Riesz representer", "debiased machine learning"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/neyman-orthogonal-activation-steering", "markdown": "https://wpnews.pro/news/neyman-orthogonal-activation-steering.md", "text": "https://wpnews.pro/news/neyman-orthogonal-activation-steering.txt", "jsonld": "https://wpnews.pro/news/neyman-orthogonal-activation-steering.jsonld"}}