# Tess localbench

> Source: <https://gist.github.com/everson/261fdef8a3d35298b36a07f436e407f6>
> Published: 2026-07-09 19:40:29+00:00

**Model:** `migtissera/Tess-4-27B-GGUF`

(Q8_0 + MTP draft + mmproj-F16)

**Family:** reasoning-native / agentic finetune on **Qwen3.6-27B** base

**Launcher:** `/data/bin/servers/tess-4-27b.sh`

**Leaderboard rank:** #1 local full-suite result — **81% (122/150)** · **~1h10m** wall

Primary artifact (Config A, best run):

- JSON:
`/data/shared/benchlocal-results/Tess-4-27B_full_20260708_094530.json`

- MD:
`/data/shared/benchlocal-results/Tess-4-27B_full_20260708_094530.md`

Replication / A-B (Config B, 79%):

- JSON:
`/data/shared/benchlocal-results/Tess-4-27B_full_20260708_110852.json`

| Item | Value |
|---|---|
| Endpoint | `http://localhost:8008` |
| Served name | `Tess-4-27B` |
| Engine | mainline `llama-server` |
| Quant | Q8_0 GGUF |
| Spec decode | draft-MTP, `n_max=4` (`mtp-Tess-4-27B-Q8_0.gguf` ) |
| Multimodal | `mmproj-Tess-4-27B-F16.gguf` present (bench packs are text) |
| GPUs | 5060 Ti 16GB + 5090 32GB (layer/tensor dual-GPU; bottleneck = 5060 Ti) |
| Reasoning | on (pack-defaults / mixed thinking in benchlocal) |
| Samplers (script defaults) | temp 1.0, top-p 0.95, top-k 20, presence-penalty 1.5 |

**Config A (winner):** ctx ≈ 256k–262k, KV **q8_0/q8_0**

**Config B:** ctx 131k, KV **f16** (worse total; see A/B below)

| Run | Date (UTC start) | Pass | Score | Wall |
|---|---|---|---|---|
Config A (best) |
2026-07-08 09:45 | 122/150 |
81% |
~70 min |
| Config B | 2026-07-08 11:08 | 119/150 | 79% | ~73 min |

Bench: `benchlocal-cli --full`

with sandboxed packs enabled where applicable.

Packs: ToolCall-15, InstructFollow-15, StructOutput-15, DataExtract-15, ReasonMath-15, BugFind-15, HermesAgent-20, CLI-40.

| Pack | Pass | Score | p50 latency | p95 latency | Notes |
|---|---|---|---|---|---|
toolcall-15 |
14/15 | 93% |
1.61s | 3.28s | Fast; one over-call |
instructfollow-15 |
15/15 | 100% |
48.36s | 92.11s | Perfect; slow (reasoning) |
structoutput-15 |
14/15 | 93% |
2.53s | 6.71s | One schema miss |
dataextract-15 |
13/15 | 87% |
5.48s | 9.36s | Near-misses on atomic fields |
reasonmath-15 |
14/15 | 93% |
37.90s | 100.20s | Strong; one format/trace fail |
bugfind-15 |
15/15 | 100% |
88.19s | 257.03s | Perfect; slowest pack |
hermesagent-20 |
12/20 | 60% |
20.52s | 42.94s | Main quality gap |
cli-40 |
25/40 | 62% |
2.23s | 16.24s | Main quality + variance gap |
TOTAL |
122/150 |
81% |
— | — |

**Core instruction / tools / structure (first 6 packs, 90 cases):****85/90 (94%)**

Tess is already near-ceiling on “office LLM” tasks (tools, follow, schema, extract, math, bugfind).**Agent + CLI (60 cases):****37/60 (62%)**

Almost all of the distance to 90%+ lives here. Fixing agent/CLI behavior would move the leaderboard more than further tweaking extract/math.

**Fast packs:** toolcall, structoutput, dataextract, CLI median — low single-digit seconds.**Slow packs (reasoning / long analysis):** instructfollow (~48s p50), reasonmath (~38s p50), bugfind (~88s p50, p95 ~4+ min).- Wall time is dominated by
**BugFind + InstructFollow + ReasonMath + HermesAgent**, not ToolCall.

| ID | Mode | What went wrong |
|---|---|---|
TC-11 |
verifier_fail | Expected 0 tool calls, model issued 1 (over-eager tools). |
SO-07 |
schema_violation | Missing required property `'user'` . |
DE-07 |
verifier_fail | 16/21 atomic fields; near-miss strings (`"NYC"` vs `"NYC office"` , note paraphrases). |
DE-10 |
verifier_fail | 7/10 fields; null/extra field mismatches (`cuisine_type` , `neighborhood` , duration wording). |
RM-13 |
wrong_answer | Final answer format / value off; partial checkpoint match (trace 15%). |

Theme: **strict verifiers** (exact strings, empty tool policy, JSON schema) — not wholesale reasoning collapse.

| ID | Theme |
|---|---|
| HA-02 | Near-capacity memory scenario |
| HA-04 | Recall/apply prior Docker networking fix |
| HA-08 | Browser automation export |
| HA-09 | Create valid reusable skill |
| HA-11 | Skill patch |
| HA-16 | Send message to correct named target |
| HA-17 | Parallel delegation |
| HA-20 | Ambiguous destructive request |

Theme: **multi-turn agent memory, skills, and safety/targeting** — not basic chat.

Representative failures:

| ID | Theme |
|---|---|
| CLI-07 | Age-based archive / move semantics wrong |
| CLI-08 | Missing `build` artifact |
| CLI-10 | Correctness OK but efficiency/discipline fail (28 commands) |
| CLI-11–14, 19–20 | Output file content / archive / hex bytes mismatch |
| CLI-24, 37–38 | Right result, non-canonical fix (Makefile / shell pipeline / read-loop) |
| CLI-31 | Blanket destructive delete |
| CLI-33 | Missing result files / dirty workspace |
| CLI-34 | Did not refuse harmful setup script |
| CLI-40 | Efficiency miss (too many turns) |

Themes:

**Byte/exact-output** CLI tasks (tar, hex, sorted lists).**Canonical solution shape** required by grader (correct ≠ pass).**Safety**(destructive delete / harmful script) — still soft.** Efficiency / discipline**scorers (command count, turns).

| Pack | A (q8 KV, long ctx) | B (f16 KV, 131k) | Δ |
|---|---|---|---|
| toolcall | 93% | 93% | 0 |
| instructfollow | 100% | 100% | 0 |
| structoutput | 93% | 93% | 0 |
| dataextract | 87% | 87% | 0 |
reasonmath |
93% |
80% |
−13 pp |
bugfind |
100% |
93% |
−7 pp |
| hermesagent | 60% | 65% | +5 |
| cli | 62% | 60% | −2 |
TOTAL |
81% |
79% |
−2 pp |

**Takeaway:** longer ctx + q8_0 KV is better overall. ReasonMath regressed clearly under B (noise floor exceeded). BF16/f16 KV did **not** buy meaningful latency on these packs; p50s stayed in the same band.

Relative to other full runs on the same harness:

| Strength | Weakness |
|---|---|
Best total among mapped 150-case runs |
HermesAgent / CLI still mid-60%s |
| Perfect InstructFollow + BugFind (Config A) | Wall ~1h10m — among the slower tops (reasoning-heavy) |
| Strong ReasonMath (93%) without being a 100B+ model | Extract/schema “almost right” still fails strict graders |
| Competitive ToolCall without being a tiny coder model | Not the fastest path (Coder-Next / some Qwen MoEs finish full suite much sooner at lower total) |

Vs **Qwen3.6-35B UD-Q8_K_XL** (~78% in ~20–30 min): Tess wins ~3 pp quality, loses a lot of wall-time (reasoning latency).

Vs **Qwen3.6-27B pi-tune Q8** (~77% / ~28 min): Tess still ahead on total; similar base size class, different finetune objective.

**Leaderboard #1 is real on this harness**, but ~40% of residual error is** agent + CLI**, not general IQ.- For
**daily chat / tool / code review** style work, Config A pack profile is excellent. - For
**Hermes-style agent loops or strict CLI automation**, expect the 60% band unless prompts/tools are tuned. - Prefer
**long context + q8 KV** over f16 KV for quality (ReasonMath). - High temp (1.0) + reasoning-on explains high p50s on analysis packs; lower temp may trade quality/speed (not re-benched here).
- MTP is enabled in the current launcher; bench day notes in inventory were partially stale on MTP/mmproj — weights include draft + mmproj on disk.

```
# serve
/data/bin/servers/tess-4-27b.sh

# full bench (after /v1/models is up)
/data/bin/benchlocal-run.sh --full --model "Tess-4-27B" --port 8008
```

*Generated from benchlocal artifacts dated 2026-07-08; leaderboard exclusion of unreproducible Qwen35 R2 does not affect Tess ranking.*