Google researchers published a new paper detailing a new way to catch spammers who are using generative AI to flood Google’s platform with spam and overwhelm its quality filters. While the research is focused on identifying video content spam, the techniques described could give an idea of methods that Google could use for web content spam. In fact, the research paper discusses a text-based generative AI identification system.
The new system is said to be a “highly accurate defense” against coordinated generative AI spam, which means that something like this could conceivably be in use. The new system is called Scalable Cluster Termination System (S-CTS) and the research paper, Scalable Detection of Adversarial Synthetic Slop and Coordinated Media Abuse: A LoRA-Enabled Multimodal Defense System.
Can This System Be Used For AI-Generated Text Spam? #
The system succeeds because it looks for the organizational structure of an attack, which is the mass reuse of a specific semantic narrative template instead of evaluating isolated videos one by one.
The research paper also describes the use of text embeddings, salient terms, and templated narratives as a part of their content classifier. If a high percentage of accounts in an infrastructure cluster are identified as using the same AI-generated text/media templates, the entire cluster is terminated.
Quickly Adapting To New Kinds Of AI Spam #
The paper says that when attackers adopt new generative models, Google can adapt its synthetic spam detection system faster by using Low-Rank Adaptation (LoRA) and Automatic Prompt Optimization (APO) instead of retraining a massive AI model.
They write:
“The Stage 2 Classifier is specialized for synthetic trend detection using Parameter-Efficient Fine-Tuning (PEFT) techniques, specifically Low-Rank Adaptation (LoRA) and Automatic Prompt Optimization (APO).
…This approach allows for the efficient adaptation of the large proprietary LLM (e.g., Gemini 2.0 Flash) without the prohibitive computational cost of full fine-tuning. Specifically, LoRA significantly reduces the number of trainable parameters and substantially decreases the memory footprint, allowing for rapid, cost-effective execution and parallelized inference on scalable TPU infrastructure.
…APO allows us to engineer prompts that adapt to new “Slop” trends faster than retraining a dense model. We can retrain a LoRA adapter rapidly when a new GenAI model (like Sora or Kling) is released by attackers.”
Sentence-BERT (S-BERT) For Identifying AI-Generated Text #
What will probably be of most interest is that the researchers acknowledge the use of Sentence-BERT (SBERT) as a way to identify semantically similar sentences.
They cite Sentence-BERT to validate a core assumption of their paper: that automated, AI-generated text leaves a distinct mathematical footprint (“text embeddings”) that can be detected.
They then pivot from S-BERT to highlight why their system (S-CTS) is an advancement: because it doesn’t stop at text embedding matching. It scales up to a multimodal, two-stage LLM architecture that evaluates these text patterns alongside infrastructure-level bot-net data.
The researchers write:
“For text-based content, methods like text embeddings generated by models like Sentence-BERT are used to detect scripted AI narratives. For multimedia, traditional techniques include perceptual hashing. However, generative AI introduces unique challenges; our system employs proprietary algorithms that analyze both textual and multimedia content to identify “Generative Artifacts” —subtle markers of synthetic production shared across channels.”
There is another research paper about Sentence-BERT (PDF) and here is how they explain the benefits of it: “In this publication, we present Sentence-BERT (SBERT), a modification of the pretrained BERT network that use Siamese and triplet network structures to derive semantically meaningful sentence embeddings that can be compared using cosine-similarity. This reduces the effort for finding the most similar pair from 65 hours with BERT / RoBERTa to about 5 seconds with SBERT, while maintaining the accuracy from BERT.
We evaluate SBERT and SRoBERTa on common STS tasks and transfer learning tasks, where it outperforms other state-of-the-art sentence embeddings methods.”
For SEO, the mention of S-BERT for identifying generative AI text spam is super interesting because it’s not something the SEO industry really knows about. This expands our knowledge of the kinds of algorithms that are used to identify text-based generative AI spam. Now here’s the interesting part: S-BERT has been around for seven years, and the SEO industry hasn’t really known about it as something that can be used to identify text-based spam. It doesn’t mean that Google has been using it for seven years. Given that generative AI has only been widely available for a few years, it could be that Sentence-BERT has only recently been used by search engines like Google for catching AI-generated text spam.
Problem Being Solved #
The researchers identify three reasons why generative AI spam is out of control and overwhelming current methods for detecting low quality content.
- The problem of low quality AI generated content has become an “exponential challenge” for detecting and catching.
- The paper admits to limitations of current mitigation strategies.
- Focusing on detecting AI-generated spam at the content level increasingly fails because of the scale designed to “overwhelm quality filters.”
The researchers explain:
“Online video platforms face an exponential challenge in detecting and mitigating the flood of AI-generated “slop” and synthetic spam perpetuated by coordinated malicious actors.
This content is increasingly designed to exploit the limitations of traditional media forensics, often utilizing generative AI to produce unique, localized variations of harmful or low-quality material at scale.
Traditional content-centric moderation fails against this coordinated, adversarial generation strategy.”
That phrase, “localized variations,” is interesting because it refers to creating “unique fingerprints for functionally identical content.”
The research paper uses phrases like:
- “unique, localized variations”
- “functionally identical content”
- “infinite, unique variations of functionally identical spam”
This is more than just making little tweaks to the content here and there. They’re talking about spammers deploying infinitely unique content that is “functionally identical” as a way of getting around traditional content analysis and mitigation strategies. This is precisely why they’re zooming out to look at clusters of accounts to identify the actual fingerprints of the spammers or their automation.
The research paper is focused on identifying AI-generated video spam, but it begs the question: Can something like this be used to identify AI-generated text-based spam? It’s certainly something to consider.
How AI-Slop Can Beat Quality Filters #
An interesting fact that the researchers share is that AI slop that’s generated at massive scale can overwhelm quality filters. The researchers also point out that spammers use “adversarial adaptation” to get around the quality filters. Adversarial adaptation means continuously updating their spam to identify patterns that enable it to slide in under a platform’s “violation threshold.”
The Solution #
The researchers propose a system that zooms out from identifying individual incidents of spam in order to focus on detecting clusters of spam that signal a common origin.
The researchers write:
“This paper presents a novel, scalable defense system designed for online video platforms (OVP) to identify and terminate clusters of coordinated accounts exhibiting a prevalence of adversarial synthetic content.”
And the way they do this is by looking at it from two points of view:
The Content Pattern Component
This is a machine learning component that scans for “repetitive, templated narratives common in AI-generated ‘slop’ and “AI-generated scripts” (meaning text/dialogue). They specifically look at the scale by identifying “non-human, high-frequency publishing behaviors characteristic of automated scripts.”The Infrastructure Component
This uses Google’s algorithms to analyze “proprietary infrastructure signals” to identify clusters of accounts that are statistically likely to be originating from the same organization or automation software script.
Details Of Scalable Cluster Termination System (S-CTS) #
Instead of looking at a single suspicious video in isolation, the system uses a two-pronged machine learning approach to spot entire networks of automated accounts (“bot-nets”) that are flooding the platform with low-quality, AI-generated spam. Thus, the goal changes from identifying individual cases of spam to identifying multiple separate accounts that belong to the same spammers or automated software scripts.
The system looks at “infrastructure-level signals and inorganic behavioral patterns” to group related accounts into “Generation Clusters.” Generation Clusters are groups of accounts that are likely to be using the same API or script.
The paper explains:
“The approach leverages a multifaceted architecture incorporating two core machine learning components:
a robust Coordinated Bot-Net Detector (via Account Relatedness)
and a Synthetic Pattern Classifier.
Crucially, we introduce an advanced AI enhancement layer utilizing Large Language Models (LLMs), specialized via Low-Rank Adaptation (LoRA) and Automatic Prompt Optimization (APO), to achieve rapid, high-precision semantic understanding of emerging synthetic spam trends.”
Does S-CTS Work? #
Yes, their test data shows that the system results in “significant impact” in catching “clusters” of spam with a high level of accuracy (precision).
They write:
“Test data demonstrates the system’s significant impact, resulting in the successful termination of clusters at a high precision comprising channels of synthetic spam generators.
Furthermore, the LLM-driven automation significantly improves operational efficiency, resulting in significant human review efficiency gains. This work details a critical system design that provides essential scalability and adversarial resilience against sophisticated generative attacks.”
Takeaways #
Some of the interesting facts in this research paper are:
-
Quality filters can be overwhelmed with a flood of spam.
-
Sentence-BERT is cited as being used for catching AI-generated spam.
-
Scalable Cluster Termination System is a unique approach to identifying spam at the cluster level.
-
Google can quickly adapt to AI-generated spam with Low-Rank Adaptation (LoRA) and Automatic Prompt Optimization (APO). This research, Scalable Detection of Adversarial Synthetic Slop and Coordinated Media Abuse: A LoRA-Enabled Multimodal Defense System, (PDF) shows the variety of techniques Google describes for identifying AI-generated spam, including text and video spam.
Featured Image by Shutterstock/Shutterstock AI