{"slug": "synthave-scalable-synthetic-labeling-for-e-commerce-with-llm-arena-validation", "title": "SynthAVE: Scalable Synthetic Labeling for E-Commerce with LLM-Arena Validation", "summary": "Researchers introduced SynthAVE, a large-scale benchmark for synthetic labeling of e-commerce attribute extraction, covering 12,726 products across 229 product types, 792 attributes, and four languages. They validated the synthetic labels using a multi-LLM arena framework with 21 judge configurations, achieving 95.2% agreement with human experts (Cohen's κ=0.92). The approach enables cost-effective, scalable labeling while maintaining quality comparable to human review.", "body_md": "arXiv:2607.07469v1 Announce Type: new\nAbstract: Fine-tuning large language models (LLMs) for e-commerce attribute extraction requires labeled data representative across thousands of product types, attributes, and multiple languages. This combinatorial scale translates to millions of annotations, rendering human labeling prohibitively costly. While recent work has demonstrated synthetic label generation using LLMs, deploying such approaches at industrial scale requires integrated quality control mechanisms. We present SynthAVE, a large-scale human-validated benchmark for attribute value extraction spanning 12,726 products across 229 product types, 792 attributes, and 4 languages (Spanish, French, Italian, German). To validate synthetic labels at scale, we introduce a multi-LLM arena framework where samples are independently evaluated by 21 judge configurations (7 model families $\\times$ 3 prompts), with final labels determined via majority voting. The majority vote ensemble agrees with human experts at Cohen's $\\kappa = 0.92$ (95.2% agreement), while individual judges show substantial inter-model agreement (Fleiss' $\\kappa = 0.76$). This demonstrates that diverse models with varying individual judgments aggregate into highly reliable predictions, enabling cost-effective validation at scale while maintaining quality parity with human review.", "url": "https://wpnews.pro/news/synthave-scalable-synthetic-labeling-for-e-commerce-with-llm-arena-validation", "canonical_source": "https://arxiv.org/abs/2607.07469", "published_at": "2026-07-09 04:00:00+00:00", "updated_at": "2026-07-09 04:16:16.043377+00:00", "lang": "en", "topics": ["large-language-models", "artificial-intelligence", "natural-language-processing", "ai-research"], "entities": ["SynthAVE"], "alternates": {"html": "https://wpnews.pro/news/synthave-scalable-synthetic-labeling-for-e-commerce-with-llm-arena-validation", "markdown": "https://wpnews.pro/news/synthave-scalable-synthetic-labeling-for-e-commerce-with-llm-arena-validation.md", "text": "https://wpnews.pro/news/synthave-scalable-synthetic-labeling-for-e-commerce-with-llm-arena-validation.txt", "jsonld": "https://wpnews.pro/news/synthave-scalable-synthetic-labeling-for-e-commerce-with-llm-arena-validation.jsonld"}}