Bits or permutations? Spending a fixed inference budget on weight precision or on option-order averaging in small local language models

A study comparing Qwen3-0.6B and Qwen3-1.7B local language models asks whether storing weights in fewer bits worsens sensitivity to answer-option order and which approach is cheaper under a fixed inference budget. The work weighs spending that budget on lower weight precision versus averaging a few option reorderings.

Local language models can save memory by storing weights in fewer bits, and they can reduce sensitivity to the order of answer options by averaging a few reorderings. We ask whether the first makes the second worse, and which is the cheaper way to spend a fixed budget. On Qwen3-0.6B and Qwen3-1.7B w