Toxicity in Twitch Chats: An LLM-Based Analysis Across Gaming Communities

A new analysis of approximately 20 million chat messages from 4,452 Twitch streams across seven game genres found that 2.4% of all messages were toxic, with MOBA games showing the highest rate at 3.2% and sports games the lowest at 2%. Researchers used a pre-trained Large Language Model to classify messages according to Twitch's toxicity taxonomy, achieving a 94.5% F1 score. The findings reveal that individual games within the same genre have significantly different toxicity distributions, indicating that game-specific community norms and mechanics shape toxic behavior beyond genre-level effects.

arXiv:2605.24000v1 Announce Type: new Abstract: Toxicity in online gaming communities remains a persistent challenge, manifesting across genres, platforms, and player interactions. While much research is focused on in-game toxicity, less is known about how toxic behavior varies between gaming communities on streaming platforms. To address this shortcoming, we analyze approximately 20 million chat messages from 4,452 streams, spanning seven game genres on Twitch. We categorize messages according to Twitch's toxicity taxonomy with a pre-trained Large Language Model using zero-shot classification. The taxonomy comprises four categories and eight subclasses, including harassment, discrimination, sexual content, and profanity. Our approach achieves an F1 score of 94.5% on the TextDetox dataset and demonstrates human-model agreement comparable to inter-human agreement. Our analysis reveals that 2.4% of all messages are classified as toxic, with notable differences across genres: streams of MOBA games exhibit the highest relative rate of toxicity 3.2% , and sports games show the lowest rate 2% . Furthermore, results indicate that individual games differ significantly in their toxicity distributions, even within genres, suggesting the existence of game-specific community norms and mechanics that shape toxic behavior beyond genre-level effects. These findings offer empirical insights into genre- and game-specific toxicity patterns on Twitch and can inform more targeted moderation strategies for gaming communities.