cd /news/large-language-models/3-7-14-17-42-instruction-tuning-inst… · home › topics › large-language-models › article
[ARTICLE · art-159247] src=arxiv.org ↗ pub= topic=large-language-models verified=true sentiment=· neutral

3, 7, 14, 17, 42: Instruction Tuning Installs and Sharpens Number-Choice Heuristics in Language Models

Instruction tuning makes language models less uniform in the integers they assign probability to when asked for a random number, according to an arXiv paper (2610.36178v1) that computed exact probability vectors for 18 open-weight model lines at five ranges up to N = 100. Across the 15 model pairs with matched base and instruction-tuned checkpoints, tuning installed round-number avoidance and sharpened endpoint avoidance: base checkpoints returned 1 and N at 0.30 to 0.60 of the interior rate, while tuned checkpoints almost never returned them and avoided multiples of 10 almost completely. No single favorite integer was shared across models above N = 10 by more than 42 percent of tuned conditions, and P(17) rose in 10 of 15 pairs and fell in 5.

by read2 min views3 publishedOct 28, 2026

arXiv:2610.36178v1 Announce Type: new Abstract: We measure, without sampling error, the probability a language model assigns to each integer from 1 to N when asked for a random number. For a fixed prompt the model defines one probability vector, which we compute by summing over each integer's scored spellings rather than estimate by sampling, removing the noise that limits earlier studies [1, 4]. We evaluate 18 open-weight model lines at five ranges up to N = 100, under 29 decoding configurations, on two inference stacks; 15 of them have matched base and instruction-tuned checkpoints that both give usable answers. Instruction tuning makes the distribution less uniform in all 15 pairs. It installs one heuristic, round-number avoidance, and sharpens another, endpoint avoidance. Base checkpoints do not avoid round numbers once the endpoints are removed from those sets; tuned checkpoints avoid them, multiples of 10 almost completely. Base checkpoints return the endpoints 1 and N at 0.30 to 0.60 of the interior rate; tuned checkpoints almost never return them. A preference for odd numbers is consistent across tuned checkpoints only at N = 100, where base checkpoints also show it weakly. Endpoint avoidance and the odd preference fit the account Kubovy and Psotka [6] proposed for human number choice; round-number avoidance is not part of that account, and the midpoint avoidance it predicts is essentially absent, with one slight exception at N = 100. Tuning does not install a new favourite integer shared across models: above N = 10 no modal integer is shared by more than 42 percent of tuned conditions, and P (17) rises in 10 of 15 pairs and falls in 5. The concentration is present in the model's own distribution: at temperature 1.0, nucleus truncation at top-p 0.95 adds a median of 0.035 to the total variation distance.

── more in #large-language-models 4 stories · sorted by recency
── more on @arxiv 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/3-7-14-17-42-instruc…] indexed:0 read:2min 2026-10-28 · —