WikiOriginQA: Automated Construction of a QA Dataset with Knowledge Origins for Analyzing Cultural Bias

A new arXiv paper (2610.30677v1) introduces WikiOriginQA, a question-answering dataset built automatically by using the creation dates of different Wikipedia language editions to represent the origins of knowledge. Experiments with the dataset found that large language model performance tends to be biased toward Anglophone cultures, and the results point to the importance of knowledge propagation into the Anglosphere. The authors say the automated method enables cultural bias evaluations with greater scalability across diverse languages without extensive manual annotation.

arXiv:2610.30677v1 Announce Type: new Abstract: Existing datasets on cultural knowledge have been typically created by human annotators familiar with each culture. However, the process requires considerable human effort, presenting a significant challenge to scalability. To address this challenge, we introduce WikiOriginQA, a QA dataset automatically constructed by objectively representing the origins of knowledge through the creation dates of different Wikipedia language editions. Experimental results with the created dataset confirm that LLM performance tends to be biased toward Anglophone cultures. Furthermore, the results suggest the importance of knowledge propagation into the Anglosphere. These findings demonstrate the validity of our automated approach, enabling cultural bias evaluations with superior scalability across diverse languages without extensive manual annotation.