{"slug": "a-coin-flip-per-token-bernoulli-sparse-steering-of-large-language-models", "title": "A Coin Flip Per Token: Bernoulli Sparse Steering of Large Language Models", "summary": "Researchers introduce Stochastic Token Steering (STS) and Stochastic Block Steering (SBS), which apply sparse autoencoder-based activation steering to only a fraction of tokens, reducing fluency degradation while maintaining behavioral control. Steering 50% of tokens recovers most of the dense-steering effect, and 30% surpasses prompt-based control, showing that SAE-mediated control depends on cumulative signal dosage.", "body_md": "arXiv:2607.05615v1 Announce Type: new\nAbstract: Activation steering via sparse autoencoders (SAEs) enables behavioral control of large language models without task-specific fine-tuning, but standard methods apply the steering signal at every generated token, incurring constant per-token perturbation that risks degrading fluency. We ask: is dense intervention necessary? We introduce Stochastic Token Steering (STS), which gates each token independently with probability $p$, and Stochastic Block Steering (SBS), which gates a leading window once per sequence; neither requires a reward model or learned gating policy. Across two model families and two behavioral tasks, steering only 50% of the tokens recovers most of the dense-steering effect while preserving fluency, and steering as few as 30% surpasses prompt-based control. The optimal steering magnitude scales inversely with the intervention ratio, revealing that SAE-mediated control is rate-limited: the behavioral outcome depends on cumulative signal dosage across a sequence.", "url": "https://wpnews.pro/news/a-coin-flip-per-token-bernoulli-sparse-steering-of-large-language-models", "canonical_source": "https://arxiv.org/abs/2607.05615", "published_at": "2026-07-08 04:00:00+00:00", "updated_at": "2026-07-08 04:17:07.715955+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "natural-language-processing"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/a-coin-flip-per-token-bernoulli-sparse-steering-of-large-language-models", "markdown": "https://wpnews.pro/news/a-coin-flip-per-token-bernoulli-sparse-steering-of-large-language-models.md", "text": "https://wpnews.pro/news/a-coin-flip-per-token-bernoulli-sparse-steering-of-large-language-models.txt", "jsonld": "https://wpnews.pro/news/a-coin-flip-per-token-bernoulli-sparse-steering-of-large-language-models.jsonld"}}