{"slug": "scalable-and-culturally-specific-stereotype-dataset-construction-via-human-llm", "title": "Scalable and Culturally Specific Stereotype Dataset Construction via Human-LLM Collaboration", "summary": "Researchers introduced a human-LLM collaborative annotation framework to create EspanStereo, a Spanish-language stereotype dataset covering multiple Spanish-speaking countries. The dataset captures both well-documented and culturally specific biases, and evaluation of Spanish-supporting LLMs revealed significant variation in stereotypical behavior across countries. This work broadens stereotype analysis in LLMs and provides a scalable path toward multilingual bias benchmarks.", "body_md": "arXiv:2607.07895v1 Announce Type: new\nAbstract: Research on stereotypes in large language models (LLMs) has largely focused on English-speaking contexts, due to the lack of datasets in other languages and the high cost of manual annotation in underrepresented cultures. To address this gap, we introduce a cost-efficient human-LLM collaborative annotation framework and apply it to construct EspanStereo, a Spanish-language stereotype dataset spanning multiple Spanish-speaking countries across Europe and Latin America. EspanStereo captures both well-documented stereotypes from prior literature and culturally specific biases absent from English-centric resources. Using LLMs to generate candidate stereotypes and in-culture annotators to validate them, we demonstrate the framework's effectiveness in identifying nuanced, region-specific biases. Our evaluation of Spanish-supporting LLMs using EspanStereo reveals significant variation in stereotypical behavior across countries, highlighting the need for more culturally grounded assessments. Beyond Spanish, our framework is adaptable to other languages and regions, offering a scalable path toward multilingual stereotype benchmarks. This work broadens the scope of stereotype analysis in LLMs and lays the groundwork for comprehensive cross-cultural bias evaluation.", "url": "https://wpnews.pro/news/scalable-and-culturally-specific-stereotype-dataset-construction-via-human-llm", "canonical_source": "https://arxiv.org/abs/2607.07895", "published_at": "2026-07-10 04:00:00+00:00", "updated_at": "2026-07-10 04:18:29.845250+00:00", "lang": "en", "topics": ["large-language-models", "ai-ethics", "natural-language-processing"], "entities": ["EspanStereo"], "alternates": {"html": "https://wpnews.pro/news/scalable-and-culturally-specific-stereotype-dataset-construction-via-human-llm", "markdown": "https://wpnews.pro/news/scalable-and-culturally-specific-stereotype-dataset-construction-via-human-llm.md", "text": "https://wpnews.pro/news/scalable-and-culturally-specific-stereotype-dataset-construction-via-human-llm.txt", "jsonld": "https://wpnews.pro/news/scalable-and-culturally-specific-stereotype-dataset-construction-via-human-llm.jsonld"}}