{"slug": "bits-or-permutations-spending-a-fixed-inference-budget-on-weight-precision-or-on", "title": "Bits or permutations? Spending a fixed inference budget on weight precision or on option-order averaging in small local language models", "summary": "A study comparing Qwen3-0.6B and Qwen3-1.7B local language models asks whether storing weights in fewer bits worsens sensitivity to answer-option order and which approach is cheaper under a fixed inference budget. The work weighs spending that budget on lower weight precision versus averaging a few option reorderings.", "body_md": "Local language models can save memory by storing weights in fewer bits, and they can reduce sensitivity to the order of answer options by averaging a few reorderings. We ask whether the first makes the second worse, and which is the cheaper way to spend a fixed budget. On Qwen3-0.6B and Qwen3-1.7B w", "url": "https://wpnews.pro/news/bits-or-permutations-spending-a-fixed-inference-budget-on-weight-precision-or-on", "canonical_source": "https://aiflash.com/news/150224/", "published_at": "2026-11-12 09:00:03+00:00", "updated_at": "2026-11-12 09:22:17.388100+00:00", "lang": "en", "topics": ["large-language-models", "machine-learning", "ai-research", "ai-infrastructure"], "entities": ["Qwen3-0.6B", "Qwen3-1.7B"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/bits-or-permutations-spending-a-fixed-inference-budget-on-weight-precision-or-on", "markdown": "https://wpnews.pro/news/bits-or-permutations-spending-a-fixed-inference-budget-on-weight-precision-or-on.md", "text": "https://wpnews.pro/news/bits-or-permutations-spending-a-fixed-inference-budget-on-weight-precision-or-on.txt", "jsonld": "https://wpnews.pro/news/bits-or-permutations-spending-a-fixed-inference-budget-on-weight-precision-or-on.jsonld"}}