{"slug": "sentence-level-contextual-entrainment-in-large-language-models", "title": "Sentence-Level Contextual Entrainment in Large Language Models", "summary": "Researchers discovered sentence-level contextual entrainment in large language models, where sentences in prompts increase their probability during inference even if counterfactual. The phenomenon decreases with model size and is controlled by 2-4% of attention heads, which can be turned off to mitigate it without performance loss.", "body_md": "arXiv:2606.24077v1 Announce Type: new\nAbstract: Contextual entrainment, which is a newly discovered phenomenon in large language models (LLMs), refers to the tendency of a model to assign higher probabilities to tokens that appear in its context. In this work, we extend this phenomenon from the token level to the sentence level by examining the per-token mean log-probability of a sentence instead of the probabilities of individual tokens. We investigate sentence-level contextual entrainment across 26 LLMs from seven families and two datasets, which cover both subjective and objective tasks. We find that sentence-level contextual entrainment exists. This means that the sentences in the prompt (even if they are counterfactual statements) can significantly increase their probability during model inference time. As the model size increases, contextual entrainment gradually decreases. We also find that contextual entrainment is controlled by 2% to 4% of the attention heads. Turning off these attention heads can effectively mitigate contextual entrainment without hurting the model's performance.", "url": "https://wpnews.pro/news/sentence-level-contextual-entrainment-in-large-language-models", "canonical_source": "https://arxiv.org/abs/2606.24077", "published_at": "2026-06-24 04:00:00+00:00", "updated_at": "2026-06-24 04:16:12.135812+00:00", "lang": "en", "topics": ["large-language-models", "artificial-intelligence", "natural-language-processing"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/sentence-level-contextual-entrainment-in-large-language-models", "markdown": "https://wpnews.pro/news/sentence-level-contextual-entrainment-in-large-language-models.md", "text": "https://wpnews.pro/news/sentence-level-contextual-entrainment-in-large-language-models.txt", "jsonld": "https://wpnews.pro/news/sentence-level-contextual-entrainment-in-large-language-models.jsonld"}}