{"slug": "i-turned-jev-into-a-lousy-chatbot", "title": "I turned Jev into a (lousy) chatbot", "summary": "A developer built jevchat, an experimental chatbot that turns the Jev API into a text generator by repeatedly asking it which symbol comes next and sampling from the returned probability distribution. The tool, described as a \"Claude accelerated experiment\" where the author described the sampling algorithms and Claude implemented them, offers swappable strategies (choice, bisect, buckets, refine) and alphabets, with the author calling the cost \"somewhat impractical\" and the results \"hilarious.", "body_md": "We know [Jev](https://docs.typesafe.ai/api).\n\n`jevchat` turns that into a chat model. At every step it asks Jev one question:\n\nGiven the user's question and the reply written so far, which symbol comes next?\n\nThe options are an alphabet plus an option to stop emitting. Jev returns a probability for each one, and the sampler draws the next symbol from that normalised distribution. Append, repeat, and stop when STOP is drawn.\n\nThere are several alphabets and sampling strategies available.\n\nThe idea is for fun, the cost is somewhat impractical, and the results are hilarious.\n\nThis was a Claude accelerated experiment. I described the sampling algorithms, strategies, and so on, and it implemented them.\n\n```\npoetry install\n```\n\nPut your Jev key in `.env` next to `pyproject.toml` (git-ignored):\n\n```\napi_key=\"...\"\n```\n\n`JEV_API_KEY` and `TYPESAFE_API_KEY` are also accepted. Values in `.env` win over\nexported ones, so editing the file is enough to switch keys.\n\n```\npoetry run jevchat                          # interactive chat\npoetry run jevchat ask \"do people need water?\"\npoetry run jevchat alphabets                # what you can sample from\npoetry run jevchat bench                    # compare every mode (table below)\n```\n\nThe reply appears as it is sampled, in a panel with a live readout of the generation rate — symbols/s, characters/s, milliseconds per API call, elapsed time — and the top few symbols Jev scored at the last step, so you can watch the distribution the sampler is drawing from.\n\n**Ctrl-C cancels.** The first press stops generation once the in-flight request\nreturns and keeps the partial reply; a second press aborts immediately. In chat,\nthe partial reply stays in the conversation history. `ask` exits 130 when cancelled.\n\nChat commands: `/help`, `/alphabet [name]`, `/temp <v>`, `/stop-bias <v>`,\n`/reset`, `/stats`, `/exit`.\n\nTwo things are swappable: **how** the distribution over the next symbol is\nobtained (`-s/--strategy`), and **what** it is over (`-a/--alphabet`). Every\ncombination below is a runnable command.\n\n`choice` asks one question over the whole alphabet. `bisect` sorts the alphabet\nand asks earlier/later yes-no questions until the group is small, then asks one\nchoice question inside it.\n\n```\n# choice — one question over the whole alphabet (the default)\npoetry run jevchat -s choice ask \"how many eyes do people have?\"\n\n# ...without the re-ordering that cancels Jev's position bias (worst mode)\npoetry run jevchat -s choice --no-shuffle-criteria ask \"how many eyes do people have?\"\n\n# ...averaging 4 re-orderings, sent as 4 parallel questions in one request\npoetry run jevchat -s choice --ensemble 4 ask \"how many eyes do people have?\"\n\n# bisect — earlier/later down to groups of 20, each split asked both ways\npoetry run jevchat -s bisect ask \"how many eyes do people have?\"\n\n# ...cheaper: bigger groups, each split asked once\npoetry run jevchat -s bisect --bisect-cutoff 32 --no-bisect-swap ask \"how many eyes do people have?\"\n\n# buckets — the alphabet split across many questions, each with an OTHER escape.\n# The only strategy that can hold more than 255 symbols.\npoetry run jevchat -a words1k -s buckets ask \"what colour is snow?\"\npoetry run jevchat -a bpe5k -s buckets --bucket-size 127 ask \"what is the capital of france?\"\n\n# refine — buckets, then a question over the winners, then a rescored nucleus.\n# Twice the probability on the right symbol and ~19x the vocabulary resolved.\npoetry run jevchat -a words1k -s refine ask \"where do fish live?\"\npoetry run jevchat -a words1k -s refine --refine-nucleus 6 --refine-rounds 2 ask \"…\"\n# hypothesis — options are the resulting texts (the default)\npoetry run jevchat -p hypothesis --window 40 ask \"what colour is snow?\"\n\n# symbol — options are the bare symbols, as the first version of this did\npoetry run jevchat -p symbol ask \"what colour is snow?\"\n# keep 3 candidate replies alive instead of committing symbol by symbol\npoetry run jevchat -b 3 ask \"what is the opposite of hot?\"\n```\n\nCosts one score per live beam per step. Above width 1, `temperature`, `top_p` and\n`top_k` stop applying — beams are ranked by probability, not drawn from.\n\n```\npoetry run jevchat -a lower26 -t 0 ask \"what is 2+2?\"   # a-z and space only\npoetry run jevchat -a ascii   -t 0 ask \"what is 2+2?\"   # spells anything\npoetry run jevchat -a tokens  -t 0 ask \"do people need water?\"   # whole words\n\n# these three exceed 255 options, so they need --strategy buckets\npoetry run jevchat -a words1k -s buckets -t 0 ask \"what colour is grass?\"\npoetry run jevchat -a bpe2k   -s buckets -t 0 ask \"where do fish live?\"\npoetry run jevchat -a bpe5k   -s buckets -t 0 ask \"what do bees make?\"\npoetry run jevchat -a tokens -s bisect --bisect-cutoff 20 ask \"do people need water?\"\npoetry run jevchat -a ascii -s choice --ensemble 12 -t 0.2 --repetition-penalty 1.0 \\\n    ask \"what colour is grass?\"\n```\n\nThere are two ways to ask Jev the same question. Under `--presentation symbol` the\noptions are the symbols themselves — `'a'`, `'i'`, `' the'` — and Jev has to append\nthe option to the reply in its head before judging it. The instructions used to say\nexactly that: *\"judge grammar and spelling on the concatenation, not on the option\non its own.\"*\n\nUnder `--presentation hypothesis` the options are the **resulting texts**:\n\n```\nanswer_so_far = \"The capital of France is Par\"\n\nsymbol      options:  'a'  'i'  's'  …  STOP\nhypothesis  options:  '…he capital of France is Para'\n                      '…he capital of France is Pari'\n                      '…he capital of France is Pars'\n                      '…he capital of France is Par'     <- unchanged: this is STOP\n```\n\nThe append is already done, so Jev only ranks finished strings — which is what a decision model is built for. It is the single largest improvement in the project: on character alphabets it roughly triples top-1 and doubles the probability mass landing on the right symbol, for fewer input tokens than symbol options with their per-option descriptions.\n\n```\npoetry run pytest\n```\n\n158 tests, all offline — a scripted fake client for the generation loop and an\n`httpx.MockTransport` for the HTTP layer. No API key and no network needed.\n`jevchat bench` is the part that does hit the API.", "url": "https://wpnews.pro/news/i-turned-jev-into-a-lousy-chatbot", "canonical_source": "https://github.com/kyle-pena-nlp/jevchat/", "published_at": "2026-09-20 17:51:27+00:00", "updated_at": "2026-09-20 18:22:28.745316+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "generative-ai"], "entities": ["Jev", "jevchat", "Claude", "Typesafe"], "alternates": {"html": "https://wpnews.pro/news/i-turned-jev-into-a-lousy-chatbot", "markdown": "https://wpnews.pro/news/i-turned-jev-into-a-lousy-chatbot.md", "text": "https://wpnews.pro/news/i-turned-jev-into-a-lousy-chatbot.txt", "jsonld": "https://wpnews.pro/news/i-turned-jev-into-a-lousy-chatbot.jsonld"}}