{"slug": "memetag-keyword-driven-meme-classification-through-tag-embedding-reconstruction", "title": "MemeTAG: Keyword-Driven Meme Classification through Tag Embedding Reconstruction", "summary": "Researchers introduced MemeTAG, a dual-objective framework that classifies harmful internet memes by generating descriptive keywords with a pretrained Vision-Language Model and distilling them into a single semantic embedding via the Aggregated Tag Inference Network (ATIN). The embedding serves as the target for an auxiliary reconstruction loss, and combined with a three-stage training strategy, MemeTAG sets a new state-of-the-art on the HarMeme, Hateful Memes Challenge (HMC), and PrideMM datasets, outperforming existing methods.", "body_md": "arXiv:2609.20962v1 Announce Type: new \nAbstract: The proliferation of harmful internet memes poses a significant societal threat, yet their automated classification remains a formidable algorithmic challenge due to the nuanced, multimodal nature of their content. To address this, we introduce MemeTAG, a novel dual-objective framework that pioneers a keyword-aware approach to meme classification. Our core innovation is a two-part semantic guidance mechanism: first, we leverage a pretrained Vision-Language Model to generate a set of descriptive keywords, that capture the high-level semantics. Second, we introduce the Aggregated Tag Inference Network (ATIN), an attention-based module that distills these keywords into a single, rich semantic embedding. This embedding serves as a target for a novel auxiliary reconstruction loss, which compels the model to learn deeply aligned visual and textual features. This approach, combined with an efficient three-stage training strategy, establishes a new state-of-the-art on the HarMeme, Hateful Memes Challenge (HMC), and PrideMM datasets, decisively outperforming existing state-of-the-art methods.", "url": "https://wpnews.pro/news/memetag-keyword-driven-meme-classification-through-tag-embedding-reconstruction", "canonical_source": "https://arxiv.org/abs/2609.20962", "published_at": "2026-09-21 04:00:00+00:00", "updated_at": "2026-09-21 04:26:46.938913+00:00", "lang": "en", "topics": ["machine-learning", "computer-vision", "natural-language-processing", "ai-research", "ai-safety"], "entities": ["MemeTAG", "Aggregated Tag Inference Network", "ATIN", "HarMeme", "Hateful Memes Challenge", "HMC", "PrideMM"], "alternates": {"html": "https://wpnews.pro/news/memetag-keyword-driven-meme-classification-through-tag-embedding-reconstruction", "markdown": "https://wpnews.pro/news/memetag-keyword-driven-meme-classification-through-tag-embedding-reconstruction.md", "text": "https://wpnews.pro/news/memetag-keyword-driven-meme-classification-through-tag-embedding-reconstruction.txt", "jsonld": "https://wpnews.pro/news/memetag-keyword-driven-meme-classification-through-tag-embedding-reconstruction.jsonld"}}