{"slug": "gauge-dependence-and-structured-output-corruption-in-sign-branched-repetition", "title": "Gauge dependence and structured-output corruption in sign-branched repetition penalties: measurements across models, inference stacks, and alternative repetition controls", "summary": "Researchers discovered that the multiplicative repetition penalty used in LLM inference engines (HuggingFace, vLLM, llama.cpp) branches on the sign of raw logits, which is arbitrary because the softmax is invariant to constant shifts. This causes the penalty to be ill-defined and corrupts structured output, dropping JSON schema conformance from 97% to 23% at theta=1.3. The authors propose applying the penalty to normalized log-probabilities instead, which removes both effects.", "body_md": "arXiv:2607.09791v1 Announce Type: new\nAbstract: The multiplicative repetition penalty shipped across the LLM inference ecosystem (HuggingFace, vLLM, llama.cpp, and a dozen further engines) branches on the sign of each raw logit (divide positives by theta, multiply negatives). But the softmax is unchanged by adding a constant to every logit, so a model's logit zero-point is arbitrary, and the sign-branch reads that arbitrary point. The sign-branch is itself the accepted fix for an earlier bug, so the accepted fix branches on a quantity the training objective leaves unconstrained. Two measurable consequences follow. (1) The penalty is not well-defined: re-centring a model's logits by a constant is a provable no-op at theta=1, yet at a routine theta=1.3 it changes 58-96% of greedy tokens, where subtractive and normalized penalties change none; real checkpoints sit at widely different zero-points, so a fixed repetition_penalty is a different operation on every model. (2) It corrupts structured output: on 200 real-world JSON schemas, theta=1.3 drops the rate of valid, schema-conformant output from 97% to 23%. In our measurements, applying the penalty to normalized log-probabilities instead of raw logits removes both effects. HuggingFace already ships that operator (LogitNormalization); today it is off by default and applied after the penalty. This note gives the mechanism, the measurements (five models up to 7B, base and RLHF, on WikiText-103 prefixes; two code models on HumanEval and JSONSchemaBench; both effects replicated inside vLLM and llama.cpp through their own samplers on the same inputs), and the normalized variant.", "url": "https://wpnews.pro/news/gauge-dependence-and-structured-output-corruption-in-sign-branched-repetition", "canonical_source": "https://arxiv.org/abs/2607.09791", "published_at": "2026-07-14 04:00:00+00:00", "updated_at": "2026-07-14 04:22:54.916428+00:00", "lang": "en", "topics": ["large-language-models", "ai-safety", "ai-research"], "entities": ["HuggingFace", "vLLM", "llama.cpp", "WikiText-103", "HumanEval", "JSONSchemaBench"], "alternates": {"html": "https://wpnews.pro/news/gauge-dependence-and-structured-output-corruption-in-sign-branched-repetition", "markdown": "https://wpnews.pro/news/gauge-dependence-and-structured-output-corruption-in-sign-branched-repetition.md", "text": "https://wpnews.pro/news/gauge-dependence-and-structured-output-corruption-in-sign-branched-repetition.txt", "jsonld": "https://wpnews.pro/news/gauge-dependence-and-structured-output-corruption-in-sign-branched-repetition.jsonld"}}