Show HN: Reasoning prefills on a few open models

A developer's independent analysis of reasoning prefills on open models finds that Kimi K3 shows a significant accuracy increase on Humanity's Last Exam questions when prefilled with Opus 4.8 reasoning, but the effect nearly vanishes on a synthetic puzzle dataset. The results reproduce the direction and approximate magnitude of the effect reported in the Stolen Thoughts paper, though the exact HLE subset is undisclosed.

Reasoning prefills on a few open models A follow-up to Stolen Thoughts https://stolen-thoughts.com/ This analysis compares unprefilled model answers with answers generated using a 1% Opus 4.8 reasoning prefill. Scores compare words and punctuation in the first 100 tokens of each model's visible answer with the corresponding Opus 4.8 answer. Deltas are absolute percentage-point changes from the unprefilled score. Humanity's Last Exam — All | Model | n | Unprefilled | Opus 4.8 reasoning prefill | Delta | |---|---:|---:|---:|---:| | DeepSeek V4 Flash | 30 | 17.27% | 16.05% | −1.22 pp | | Inkling | 30 | 15.35% | 16.66% | +1.31 pp | | Kimi K3 | 30 | 20.16% | 29.14% | +8.98 pp | | Qwen3.8 A95B | 29 | 29.25% | 31.12% | +1.87 pp | The Kimi K3 result closely matches the main Appendix B.2 effect reported in Stolen Thoughts . The paper finds increases of 15 percentage points on STEM and 8.6 points on non-STEM HLE questions; our independent 30-question HLE run finds an aggregate increase of 8.98 points. This reproduces the reported direction and approximate magnitude, although the paper's precise HLE subset is undisclosed. Puzzle Puzzle is a fully synthetic dataset consisting mostly of password-style function-transformation tasks. It is not published or otherwise available anywhere on the public internet. Example puzzle Infer a function f x from the following examples: f 'sambas' = 'asmbsa' f 'kameda' = 'akmead' f 'guider' = 'ugidre' Given y = 'affilgsniy' , find x such that f x = y . First define the function, then provide the answer. | Model | n | Unprefilled | Opus 4.8 reasoning prefill | Delta | |---|---:|---:|---:|---:| | DeepSeek V4 Flash | 15 | 8.88% | 7.68% | −1.19 pp | | Inkling | 15 | 15.06% | 14.94% | −0.12 pp | | Kimi K3 | 15 | 12.07% | 12.54% | +0.47 pp | | Qwen3.8 A95B | 15 | 28.63% | 22.68% | −5.95 pp | Discussion What jumps out to me is how much smaller the Kimi K3 bump is on Puzzle: just +0.47 points, versus +8.98 on HLE. If the prefill were transferring some general reasoning style, I would have expected a clearer effect on these new synthetic tasks too.