{"slug": "simplifying-the-modeling-of-arbitrary-conditionals-in-natural-language", "title": "Simplifying the Modeling of Arbitrary Conditionals in Natural Language", "summary": "Researchers propose AC-GPT, a modification to causal Transformers that enables evaluating and sampling from arbitrary conditionals in a single forward pass without degrading standard left-to-right performance. The method preserves the next-token prediction objective, allowing existing LLMs to be fine-tuned for arbitrary conditioning, and outperforms baselines on modeling such conditionals.", "body_md": "arXiv:2606.14943v1 Announce Type: new\nAbstract: Causal Transformers model sequences through an autoregressive factorization of the joint distribution, which enables efficient left-to-right decoding and conditional likelihood computation. However, they cannot tractably sample from or evaluate arbitrary conditionals -- e.g., a block of text conditioned on past and future tokens. Recent work aims to solve this problem through novel architectures, but they often lead to sub-optimal modeling of such conditionals and degraded generations. We propose Arbitrary Conditionals GPT (AC-GPT) which introduces a simple modification to standard causal Transformers to enable evaluating and sampling from arbitrary conditionals -- including past, future, and mixed contexts -- within a single forward pass. Unlike prior approaches, our method preserves the standard left-to-right ordering and next-token prediction objective essential for both strong performance and efficient training on natural language. Crucially, this compatibility allows existing LLMs to be fine-tuned for arbitrary conditioning. Our empirical results indicate that our method outperforms baselines on modeling arbitrary conditionals, without degrading standard left-to-right performance.", "url": "https://wpnews.pro/news/simplifying-the-modeling-of-arbitrary-conditionals-in-natural-language", "canonical_source": "https://arxiv.org/abs/2606.14943", "published_at": "2026-06-16 04:00:00+00:00", "updated_at": "2026-06-16 04:22:40.760530+00:00", "lang": "en", "topics": ["large-language-models", "natural-language-processing", "generative-ai"], "entities": ["AC-GPT", "Causal Transformers"], "alternates": {"html": "https://wpnews.pro/news/simplifying-the-modeling-of-arbitrary-conditionals-in-natural-language", "markdown": "https://wpnews.pro/news/simplifying-the-modeling-of-arbitrary-conditionals-in-natural-language.md", "text": "https://wpnews.pro/news/simplifying-the-modeling-of-arbitrary-conditionals-in-natural-language.txt", "jsonld": "https://wpnews.pro/news/simplifying-the-modeling-of-arbitrary-conditionals-in-natural-language.jsonld"}}