Benchmarks for document tasks for local AI models DocBlocks released docbench, a 100-point weighted benchmark that scored 24 local models from 0.8B to 35B parameters on six document-task families (review and feedback, writing, diagrams and illustrations, slides, study aids, and narration cleanup) across 102 test cases and 142 outputs. Qwen 3.8 27B Q4 led the local field at 81 of 100, with Gemma 4 12B statistically level at 79 and the best models at 4B parameters and under reaching about 71, while cloud models GPT 6.1 Sol, Claude Opus 5.5 and Claude Sonnet 5.5 scored between 87 and 91 via their CLI command lines. DocBlocks said review separates models most, because large models find real problems while 1–2B models pad their reviews with trivial or wrong suggestions. Docbench: how small local models handle document work YouTube version: DocBlocks recently added support for using local models to provide AI features for document tasks. You can select a model, it downloads, and you get basic features like document review and content editing. Pretty quickly, we ran into an obvious usability problem: how are users supposed to know which models to use, especially amongst a choice of a lot of models? Though we have some good measurements from Gezel's overall scorecard, we set out to measure and create a focused synthetic benchmark of model usability for documents. Separately, it was pretty obvious that models varied wildly on usefulness, correlated to model size. We needed a way to quantify that. Enter "docbench". For docbench, we focus on really small models in the under-12-billion-parameter class. This is because we expect that the typical user will not be running on a workstation class machine with powerful video cards; rather, they'll run with modest GPUs or "just" a CPU by itself. Models that are in the 1-4 billion parameter range are typically runnable on conventional modern computers with at least 16gb of RAM, or a modern higher-end phone. So towards that end, we scored 24 local models, from 0.8B to 35B parameters, on the work DocBlocks asks of them: reviewing, writing, diagrams, slides, study aids and transcript cleanup. From this, we created a 100 point weighted scale and included elements of static analysis and AI-based output judging. For comparison, we also compared against GPT 6.1 Sol, Claude Opus 5.5, and Claude Sonnet 5.5 via their CLI command lines. Putting the headlines before the details: - Qwen 3.8 27B Q4 leads at 81 of 100, Gemma 4 12B is statistically level with it at 79, and the best models at 4B and under reach about 71. - Review separates models most: large models find real problems, while 1–2B models pad their reviews with trivial or wrong suggestions. - For comparison, modern cloud models score between 87 and 91, at least 6 points higher than the highest local model tested. Even historical cloud models e.g., Claude Sonnet 4.6 have performed better. - Although this rubric seemed to do a pretty good job of separating out lower-end models on real world tasks, I suspect if we had even harder editing and writing challenges, you'd see cloud models pull ahead farther. In other words, I believe cloud models probably topped out the difficulty of these challenges, but would be more differentiated on tougher challenges reviewing a scientific paper, for example. What docbench measures Docbench covers every AI feature DocBlocks ships, grouped into six families and weighted by how much people rely on them. Each model answers the same 102 test cases 142 outputs, since tasks with variable output run three times , and every answer is scored 0–100. | Family | Weight | What the model is asked to do | Cases | Score mix | |---|---|---|---|---| | Review and feedback | 25% | Eight review styles: general, resume, facts and sources, conciseness, plain language, The Elements of Style , the UN editorial manual, creative ideas | 28 | 15% checks, 85% judge | | Writing | 25% | Compose from scratch, append to a document, insert into one, rewrite a selection | 19 | 30% checks, 70% judge | | Diagrams and illustrations | 20% | Pick what to illustrate, build diagrams of 13 kinds, draw SVG illustrations, write briefs for an image model | 30 | Checks only for picture ideas; 30/70 for the rest | | Slides | 10% | Turn a document into slide suggestions for a deck | 8 | 30% checks, 70% judge | | Study aids | 15% | Summaries, quizzes and flashcards that stay true to the source | 12 | 25% checks, 75% judge | | Narration cleanup | 5% | Clean up a dictated transcript without changing what was said | 5 | 40% checks, 60% judge | A family's score is the mean of its tasks, and docbench is the weighted mean of the six families. The test documents are deliberately moderate in quality, so a good reviewer has real problems to find. See Appendix A for more on how each result is calculated. The results Across a few runs and two devices, we found that four models share the top band, a group of 4–9B models follows within 10 points, and scores fall quickly below 3B. Family scores are 0–100; the order is by docbench score. Note that download size loosely correlates to how much memory is occupied, though some models, like Nanbeige 4.2 are outliers in that they take up far more memory than their size implies. | Model | Download | Docbench 95% range | Review | Writing | Diagrams | Slides | Study | Narration | |---|---|---|---|---|---|---|---|---| | Claude Opus 5.5 via Claude CLI