{"slug": "lora-for-gender-inclusive-rewriting-and-activation-steering-for-counter", "title": "LoRA for Gender-Inclusive Rewriting and Activation Steering for Counter-Narrative Generation", "summary": "The IHLC system achieved an official score of 80.00% for gender-inclusive rewriting using parameter-efficient Low-Rank Adaptation (LoRA) fine-tuning, and 78.12% for counter-narrative generation via a compute-efficient inference-time representation engineering approach that injects a principal steering direction into Gemma-3-4B-it's intermediate representations. The system's manual analysis identified key failure modes including semantic drift, residual bias leakage, layer sensitivity, over-steering, and text degeneration, highlighting both the potential and limitations of activation steering for socially aligned language generation.", "body_md": "arXiv:2607.23083v1 Announce Type: new\nAbstract: Gender-inclusive language generation seeks to transform biased text into inclusive alternatives while preserving semantic meaning and contextual coherence. This paper presents the IHLC system for the LT-EDI 2026 Shared Task, addressing both gender-inclusive rewriting and counter-narrative generation. For gender-inclusive rewriting, we employ parameter-efficient Low-Rank Adaptation (LoRA) fine-tuning, achieving an official score of 80.00%. Our primary contribution is a compute-efficient inference-time representation engineering approach for counter-narrative generation. We derive a principal steering direction from contrastive hidden-state activations using principal component analysis (PCA) and inject it into the intermediate representations of Gemma-3-4B-it during inference, enabling behavioral steering toward inclusive responses without modifying model weights. Combined with constrained prompting, this approach produces polite and contextually appropriate counter-narratives, achieving an official score of 78.12%. We further present a manual analysis of steering behavior, identifying key failure modes including semantic drift, residual bias leakage, layer sensitivity, over-steering, and text degeneration. Our findings highlight both the practical potential and current limitations of activation steering as a lightweight alternative to parameter updates for controllable and socially aligned language generation.", "url": "https://wpnews.pro/news/lora-for-gender-inclusive-rewriting-and-activation-steering-for-counter", "canonical_source": "https://arxiv.org/abs/2607.23083", "published_at": "2026-07-28 04:00:00+00:00", "updated_at": "2026-07-28 04:26:06.138407+00:00", "lang": "en", "topics": ["artificial-intelligence", "natural-language-processing", "ai-ethics", "ai-research"], "entities": ["IHLC", "LT-EDI 2026 Shared Task", "Gemma-3-4B-it"], "alternates": {"html": "https://wpnews.pro/news/lora-for-gender-inclusive-rewriting-and-activation-steering-for-counter", "markdown": "https://wpnews.pro/news/lora-for-gender-inclusive-rewriting-and-activation-steering-for-counter.md", "text": "https://wpnews.pro/news/lora-for-gender-inclusive-rewriting-and-activation-steering-for-counter.txt", "jsonld": "https://wpnews.pro/news/lora-for-gender-inclusive-rewriting-and-activation-steering-for-counter.jsonld"}}