{"slug": "issue-with-llamasdpaattention-not-being-utilized", "title": "Issue with LlamaSdpaAttention Not Being Utilized", "summary": "In Transformers 4.48.3, a model printing LlamaAttention in print(model) does not by itself indicate that SDPA is disabled, because LlamaAttention.forward selects its attention function internally via self.config._attn_implementation, according to a Hugging Face forum clarification. The post advises checking print(model.config._attn_implementation) as a first step, noting that this reports the configured implementation rather than proof of the GPU kernel actually executed, and that requesting output_attentions=True with SDPA triggers an eager-attention fallback in that version. The clarification states the unchanged class name is consistent with SDPA being selected and does not establish the cause of the separate memory issue reported.", "body_md": "A version-specific clarification for [@jacobvinje](https://discuss.huggingface.co/u/jacobvinje) and future readers: in **Transformers 4.48.3**, seeing `LlamaAttention` in `print(model)` does **not** by itself mean SDPA is disabled.\n\nIn that version, `LlamaAttention.forward` selects its attention function internally using `self.config._attn_implementation`. The implementation is visible in the [v4.48.3 source](https://github.com/huggingface/transformers/blob/v4.48.3/src/transformers/models/llama/modeling_llama.py).\n\nFor an existing loaded model, this is a useful first check:\n\n```\nprint(model.config._attn_implementation)\n```\n\nThat reports the configured implementation, not proof of the GPU kernel actually executed. In this version, requesting `output_attentions=True` with SDPA triggers an eager-attention fallback. A profiler is needed to verify the executed operations and any memory implications.\n\nSo the unchanged class name is consistent with SDPA being selected; it is not, on its own, a reproducible backend failure. This clarification does not establish the cause of the separate memory issue you reported.\n\nFor a fuller explanation, I also maintain a free [Transformer guide (English and Spanish)](https://karlesmarin.github.io/transformers-guide/). The [KV-cache chapter](https://karlesmarin.github.io/transformers-guide/en/chapters/20-kv-cache.html) explains the memory budget and why fitting in memory, retaining attention mass and preserving task quality are separate questions.", "url": "https://wpnews.pro/news/issue-with-llamasdpaattention-not-being-utilized", "canonical_source": "https://discuss.huggingface.co/t/issue-with-llamasdpaattention-not-being-utilized/140678#post_3", "published_at": "2026-10-25 13:31:56+00:00", "updated_at": "2026-10-25 13:43:24.285502+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-infrastructure"], "entities": ["Hugging Face", "Transformers 4.48.3", "LlamaAttention", "LlamaSdpaAttention", "jacobvinje"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/issue-with-llamasdpaattention-not-being-utilized", "markdown": "https://wpnews.pro/news/issue-with-llamasdpaattention-not-being-utilized.md", "text": "https://wpnews.pro/news/issue-with-llamasdpaattention-not-being-utilized.txt", "jsonld": "https://wpnews.pro/news/issue-with-llamasdpaattention-not-being-utilized.jsonld"}}