{"slug": "every-time-i-hire-a-linguist-inference-costs-go-down-on-linguistic-rules-as", "title": "Every Time I Hire a Linguist, Inference Costs Go Down: On Linguistic Rules as Effective Prompt Compressors", "summary": "A new study from arXiv:2607.25335 shows that linguistic rules alone can serve as effective prompt compressors for large language models, reducing inference costs without requiring LM-based scoring at compression time. The evolved compressors, which use only CPU-side processing, achieve performance similar to advanced prompt-compression strategies across short passages, multi-document reasoning, and dialogue-memory QA datasets, with strongest results under light-to-moderate compression.", "body_md": "arXiv:2607.25335v1 Announce Type: new\nAbstract: Prompt compression shortens LLM input to reduce inference cost, yet existing methods score token importance through LM forward passes. It remains questionable whether such nuanced, costly token selection is necessary. Compression requires identifying informative content, a problem that linguistic research has long addressed through cues that can be operationalized as deterministic rules. We therefore ask: can \\textbf{linguistic rules alone} serve as effective prompt compressors, without LM-based scoring at compression time?\nTo address this, we conduct offline evolutionary search over lexical, syntactic, semantic, and discourse seeds to find competitive rule combinations. The resulting linguistic compressor requires no LM forward pass at deployment and uses only CPU-side processing for compression. We evaluate it with a dual-path protocol to balance compression quality and reconstruction fidelity.\nAcross short passages, multi-document reasoning, and dialogue-memory QA datasets, evolved compressors achieve performance similar to that of recent advanced prompt-compression strategies. Performance is strongest under light-to-moderate compression and degrades as compression becomes more aggressive, while the Direct and Reconstruction paths exhibit distinct patterns. Evolutionary analysis reveals that effective compression fuses signals across linguistic levels and, as the compression ratio increases, rules shift from token pruning to sentence extraction.", "url": "https://wpnews.pro/news/every-time-i-hire-a-linguist-inference-costs-go-down-on-linguistic-rules-as", "canonical_source": "https://arxiv.org/abs/2607.25335", "published_at": "2026-07-29 04:00:00+00:00", "updated_at": "2026-07-29 04:27:31.355695+00:00", "lang": "en", "topics": ["large-language-models", "natural-language-processing", "ai-research"], "entities": ["arXiv"], "alternates": {"html": "https://wpnews.pro/news/every-time-i-hire-a-linguist-inference-costs-go-down-on-linguistic-rules-as", "markdown": "https://wpnews.pro/news/every-time-i-hire-a-linguist-inference-costs-go-down-on-linguistic-rules-as.md", "text": "https://wpnews.pro/news/every-time-i-hire-a-linguist-inference-costs-go-down-on-linguistic-rules-as.txt", "jsonld": "https://wpnews.pro/news/every-time-i-hire-a-linguist-inference-costs-go-down-on-linguistic-rules-as.jsonld"}}