{"slug": "typesafe-jev-played-chess-and-landed-next-to-reasoning-models", "title": "TypeSafe Jev Played Chess — And Landed Next to Reasoning Models", "summary": "A developer integrated TypeSafe's Jev model into the LLM Chess benchmark harness, adapting the classifier-style API to return UCI chess moves and landing the model at roughly #59 on the leaderboard with an Elo near 243, alongside qwen3.6-27b and o4-mini-medium. Jev achieved a flat 50% draw rate against Dragon at levels L1 through L3 while costing about $0.0015 per game and 119 tokens, far below chat-model competitors. The developer noted that Jev's fixed-label design suits selecting from a list of legal moves better than free-form text generation, which produced near-words like 'bue' and 'helhoh' in a spelling test.", "body_md": "TypeSafe's [Jev](https://docs.typesafe.ai/introduction) is an odd one. It isn't a chat model - you send a **state** plus typed **questions** (Choice / Score / Noul) and get labels with probabilities — like a classifier, except the input can be free text and you choose the labels each time.\n\nI've been running models through [LLM Chess](https://maxim-saplin.github.io/llm_chess/) for a while. LLMs play in an agentic setting with a multi-turn protocol: `get_current_board`, `get_legal_moves`, `make_move <uci>`. Jev doesn't do that. Can you still put it on the same leaderboard?\n\nApparently yes — if you talk to it the way it expects.\n\nTypeSafe is openly [anti-benchmaxxing](https://typesafe.ai/blog/antibenchmaxxing). In the [System One / Jev launch post](https://typesafe.ai/blog/introducing-system-one-models-and-jev) they say existing LLM benchmarks don't really apply here, and they skip the usual public-eval scoreboard putting it's model next to other models. They prefer dated internal snapshots instead.\n\nI put Jev on [LLM Chess](https://maxim-saplin.github.io/llm_chess/) anyway — same harness, same opponents as the chat models.\n\nIt reminded me of talking to Stephen Hawking: smart, but no free speech. Answers do not come out in a free from\n\nOutside chess I tried a silly next-character loop — Choice over `a–z` plus `<eos>`, one API call per letter. Asked to spell *blue* or finish *hel…*, it spat out almost-words (`bue`, `helhoh`). Started from `pari…` and it finished `paris`. Generating from scratch was weak; finishing a partial worked better. A fixed list of legal chess moves fits this model much better than spelling English letter by letter.\n\nEach ply:\n\n`{ fen, side_to_move }`\nJev returns a UCI string. It turns into `make_move e7e5` — same action chat models send to the eval runner, on the outside seems almost like normal eval interaction.\n\nPricing: **$0.042 / 1M input tokens**, output free.\n\nOn the current leaderboard `jev-latest` is around **#59**, Elo **~243**, sitting next to `qwen3.6-27b` and `o4-mini-medium`:\n\n| # | Player | Elo | Game Duration | Tokens | Cost/Game | Time/Game | Games | \n|---|---|---|---|---|---|---|---|\n| 58 | qwen3.6-27b@q4_k_s | 270.0±148.6 | 100% | 5956 | $1.46 | 7.54h | 27 | \n| **59** | **jev-latest** | **242.9±117.5** | **100%** | **119** | **$0.0015** | **35.6s** | **80** | \n| 60 | o4-mini-medium | 240.3±64.3 | 100% | 2156 | $1.11 | 41.5m | 199 | \n\nDragon (white) vs Jev (black), 10 games per level:\n\n| Dragon | Dragon wins | Draws | Jev wins | \n|---|---|---|---|\n| L1 | 5 | 5 | 0 | \n| L2 | 5 | 5 | 0 | \n| L3 | 5 | 5 | 0 | \n\n**50% draws** at L1, L2, and L3. Flat. With most chat models, stronger Dragon usually means fewer draws — the engine finishes more games. Here the draw rate doesn't drop across three levels. That helps the Elo.\n\nTypeSafe calls these System One models: fast decisions, no deep thinking.\n\nChess still needs real choices — what to capture, when to trade, when to aim for a draw. Jev has no chat, costs fractions of a cent per game, and still lands near mid-pack reasoning models on Elo. So the skill isn't \"writing about chess.\" It's picking a move from a list.\n\nTypeSafe doesn't want a public leaderboard as their proof. This is one outside eval.\n\n`bue` / `helhoh`).\nLeaderboard: [LLM Chess](https://maxim-saplin.github.io/llm_chess/)\n\nHarness: [maxim-saplin/llm_chess](https://github.com/maxim-saplin/llm_chess) (`feat/typesafe-jev-player` / TypeSafe Jev player)", "url": "https://wpnews.pro/news/typesafe-jev-played-chess-and-landed-next-to-reasoning-models", "canonical_source": "https://dev.to/maximsaplin/typesafe-jev-played-chess-and-landed-next-to-reasoning-models-28ga", "published_at": "2026-09-17 17:13:09+00:00", "updated_at": "2026-09-17 17:23:08.263838+00:00", "lang": "en", "topics": ["ai-products", "ai-tools", "large-language-models", "ai-research"], "entities": ["TypeSafe", "Jev", "LLM Chess", "Dragon", "qwen3.6-27b", "o4-mini-medium", "System One"], "alternates": {"html": "https://wpnews.pro/news/typesafe-jev-played-chess-and-landed-next-to-reasoning-models", "markdown": "https://wpnews.pro/news/typesafe-jev-played-chess-and-landed-next-to-reasoning-models.md", "text": "https://wpnews.pro/news/typesafe-jev-played-chess-and-landed-next-to-reasoning-models.txt", "jsonld": "https://wpnews.pro/news/typesafe-jev-played-chess-and-landed-next-to-reasoning-models.jsonld"}}