{"slug": "show-hn-reasoning-prefills-on-a-few-open-models", "title": "Show HN: Reasoning prefills on a few open models", "summary": "A developer's independent analysis of reasoning prefills on open models finds that Kimi K3 shows a significant accuracy increase on Humanity's Last Exam questions when prefilled with Opus 4.8 reasoning, but the effect nearly vanishes on a synthetic puzzle dataset. The results reproduce the direction and approximate magnitude of the effect reported in the Stolen Thoughts paper, though the exact HLE subset is undisclosed.", "body_md": "# Reasoning prefills on a few open models\n\n*A follow-up to [Stolen Thoughts](https://stolen-thoughts.com/)*\n\nThis analysis compares unprefilled model answers with answers generated using a 1% Opus 4.8 reasoning prefill. Scores compare words and punctuation in the first 100 tokens of each model's visible answer with the corresponding Opus 4.8 answer. Deltas are absolute percentage-point changes from the unprefilled score.\n\n## Humanity's Last Exam — All\n\n| Model | n | Unprefilled | Opus 4.8 reasoning prefill | Delta |\n|---|---:|---:|---:|---:|\n| DeepSeek V4 Flash | 30 | 17.27% | 16.05% | −1.22 pp |\n| Inkling | 30 | 15.35% | 16.66% | +1.31 pp |\n| Kimi K3 | 30 | 20.16% | 29.14% | +8.98 pp |\n| Qwen3.8 A95B | 29 | 29.25% | 31.12% | +1.87 pp |\n\nThe Kimi K3 result closely matches the main Appendix B.2 effect reported in *Stolen Thoughts*. The paper finds increases of 15 percentage points on STEM and 8.6 points on non-STEM HLE questions; our independent 30-question HLE run finds an aggregate increase of 8.98 points. This reproduces the reported direction and approximate magnitude, although the paper's precise HLE subset is undisclosed.\n\n## Puzzle\n\nPuzzle is a fully synthetic dataset consisting mostly of password-style function-transformation tasks. It is not published or otherwise available anywhere on the public internet.\n\n### Example puzzle\n\n> Infer a function `f(x)` from the following examples:\n>\n> `f('sambas') = 'asmbsa'`  \n> `f('kameda') = 'akmead'`  \n> `f('guider') = 'ugidre'`\n>\n> Given `y = 'affilgsniy'`, find `x` such that `f(x) = y`. First define the function, then provide the answer.\n\n| Model | n | Unprefilled | Opus 4.8 reasoning prefill | Delta |\n|---|---:|---:|---:|---:|\n| DeepSeek V4 Flash | 15 | 8.88% | 7.68% | −1.19 pp |\n| Inkling | 15 | 15.06% | 14.94% | −0.12 pp |\n| Kimi K3 | 15 | 12.07% | 12.54% | +0.47 pp |\n| Qwen3.8 A95B | 15 | 28.63% | 22.68% | −5.95 pp |\n\n## Discussion\n\nWhat jumps out to me is how much smaller the Kimi K3 bump is on Puzzle: just +0.47 points, versus +8.98 on HLE. If the prefill were transferring some general reasoning style, I would have expected a clearer effect on these new synthetic tasks too.\n", "url": "https://wpnews.pro/news/show-hn-reasoning-prefills-on-a-few-open-models", "canonical_source": "https://gist.github.com/wsxiaoys/102e8654c14d5d27b7b77532026ebfa5", "published_at": "2026-08-16 14:24:41+00:00", "updated_at": "2026-08-16 14:40:42.863110+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-research"], "entities": ["Opus 4.8", "DeepSeek V4 Flash", "Inkling", "Kimi K3", "Qwen3.8 A95B", "Humanity's Last Exam", "Stolen Thoughts"], "alternates": {"html": "https://wpnews.pro/news/show-hn-reasoning-prefills-on-a-few-open-models", "markdown": "https://wpnews.pro/news/show-hn-reasoning-prefills-on-a-few-open-models.md", "text": "https://wpnews.pro/news/show-hn-reasoning-prefills-on-a-few-open-models.txt", "jsonld": "https://wpnews.pro/news/show-hn-reasoning-prefills-on-a-few-open-models.jsonld"}}