{"slug": "long-live-fine-tuning-task-specific-transformers-outperform-zero-shot-llms-for", "title": "Long Live Fine-Tuning: Task-Specific Transformers Outperform Zero-Shot LLMs for Misinformation Response Classification on Reddit", "summary": "Fine-tuned RoBERTa achieved a 0.62 macro-F1 score for classifying misinformation responses on Reddit, outperforming the best zero-shot large language model (Claude Haiku 4.5) at 0.50 while costing a fraction per query. The supervised advantage concentrated on detecting belief comments—the implicit category every zero-shot model under-detected—and scaling model size did not improve zero-shot performance, with Claude Sonnet 4.6 collapsing belief detection to 0.17 due to safety-alignment artifacts. The findings demonstrate that task-specific fine-tuning remains more reliable than zero-shot LLMs for misinformation response classification, particularly when missing belief comments is the costlier error.", "body_md": "arXiv:2606.04274v1 Announce Type: new\nAbstract: As large language models (LLMs) become default tools for online information verification, an implicit assumption follows them: that scale and general capability are sufficient for nuanced classification of misinformation discourse. We test this assumption directly on 900 Reddit comments spanning three PolitiFact-verified misinformation claims (environment, health, immigration), labelled as belief (propagates the claim), fact-check (corrects it), or other. We compare nine models across three paradigms -- BART-MNLI, three Llama variants, three commercial frontier LLMs (Claude Haiku 4.5, Gemini Flash Lite 2.5, Claude Sonnet 4.6), and fine-tuned DistilBERT and RoBERTa -- under universal and topic-specific label schemas.\nThe assumption does not hold. Fine-tuned RoBERTa reaches 0.62 macro-$F_1$ against a best zero-shot result of 0.50 (Claude Haiku 4.5), at a fraction of the per-query cost; the supervised advantage is concentrated on the belief class, the implicit, affective category every zero-shot model under-detects. Scaling does not help: Llama-3-8B matches Llama-3-70B, and Claude Sonnet 4.6 underperforms the smaller Haiku under generic labels, collapsing belief detection to 0.17 and refusing outright on a subset of comments flagged as sensitive. This is a safety-alignment artefact, not a capacity limit. Label schema and topic jointly shape zero-shot performance, with the same model varying by more than 0.13 macro-$F_1$ across topics under matched labels. In a verification context, where missing belief is the costlier error, task-specific fine-tuning remains the more reliable choice despite the proliferation of large generative models.", "url": "https://wpnews.pro/news/long-live-fine-tuning-task-specific-transformers-outperform-zero-shot-llms-for", "canonical_source": "https://arxiv.org/abs/2606.04274", "published_at": "2026-06-04 04:00:00+00:00", "updated_at": "2026-06-04 04:22:02.432985+00:00", "lang": "en", "topics": ["large-language-models", "natural-language-processing", "machine-learning", "ai-research", "ai-safety"], "entities": ["BART-MNLI", "Llama", "Claude Haiku 4.5", "Gemini Flash Lite 2.5", "Claude Sonnet 4.6", "DistilBERT", "RoBERTa", "PolitiFact"], "alternates": {"html": "https://wpnews.pro/news/long-live-fine-tuning-task-specific-transformers-outperform-zero-shot-llms-for", "markdown": "https://wpnews.pro/news/long-live-fine-tuning-task-specific-transformers-outperform-zero-shot-llms-for.md", "text": "https://wpnews.pro/news/long-live-fine-tuning-task-specific-transformers-outperform-zero-shot-llms-for.txt", "jsonld": "https://wpnews.pro/news/long-live-fine-tuning-task-specific-transformers-outperform-zero-shot-llms-for.jsonld"}}