Such clips are circumventing the social media platform’s moderation policies that prohibit extreme violent material, GNET said
- Bookmark
The Global Network on Extremism and Technology (GNET) said such clips bypass the social media platform’s moderation policies designed to strictly ban extreme violent content.
Disguised by a "cute" and "Pixar-like" visual style, some of these videos convey messages from extremist organisations including extreme right "accelerationism", "nihilistic violence extremism", and the so-called Islamic State (IS) group, GNET detailed in a Monday report.
These violent scenes form part of a broader trend of artificial intelligence clips showing humanlike food acting out melodramatic romance, betrayal and scandal storylines.
“More concerning, however, is the escalation from absurdist AI-generated drama into explicitly graphic content.
“The radicalising potential of this content does not necessarily lie in ideological messaging alone, but in the normalisation of cruelty, emotional numbness and algorithmically amplified shock culture.
“In this sense, the transition from harmless surreal humour to graphic AI gore reflects a broader dynamic of online radicalisation: the gradual erosion of emotional boundaries through constant exposure to increasingly violent and dehumanising imagery.”
Animated foods use guns and axes to kill their enemies in some videos, while other videos are linked to specific criminal groups.
One example appears to replicate content from official IS videos with a carrot that makes a potato “explode”, the study found.
Another features a “Carrot Khalifah” telling his army “the Carrot State will rise again”, which experts said evokes an infamous slogan used by IS.
Two IS posts included hashtags often used by the terrorist group’s supporters to disseminate propaganda.
The report also highlighted an account that shares violent fruit and vegetable AI content and also references neo-Nazi terrorist organisation Atomwaffen Division, as well as the nihilistic violence subculture No Lives Matter.
Comments on the content include statements of extreme right-wing ideology, researchers said, with the message “potato supremacy” referring to white supremacy.
One post included in the report shows an AI image of a tomato that says: “No vegetables matter.”
The researchers said: “It is clear that AI-generative platforms are a crucial tool for creating viral content online, including violent and extremist content, as in the aforementioned cases.
“Food violence-based material empowered by AI is easily accessible, putting minors and youth at risk of inadvertently consuming graphic or disturbing content.
“Scenes with AI fruit and vegetables are simple and highly repetitive. They require no cognitive effort, provide constant rewards through unexpected scenes, and are perfect for endless scrolling.
“They are specifically designed for the algorithm, extremely quick to produce, and optimised for engagement. Hence, they act as attention traps.”
The team raised fears that users may “stumble” on the terror material and approach membership into the online ecosystems.
The comments could function as “communication hubs” where users provide extra details, GNET added.
“This type of content can lay the basis for the consumption of further violence-based audiovisual material, making young users fall into the trap of online radicalisation and audiovisual violence consumption.”
The authors called on TikTok to develop “countermeasures” to effectively limit such AI-generated content.
TikTok has been approached for comment.