{"slug": "reverse-engineered-jev-like-model", "title": "Reverse-engineered Jev-like model", "summary": "An independent developer released a reverse-engineered, open starter model called Jev-like that scores a changing list of N text options in a single pass, matching the input and output shape of TypeSafe's unpublished commercial Jev model. The repository includes Doom and chess examples sharing one visual scorer from jevlike.vision, with the joint checkpoint averaging 0.60 kills and -97.50 reward over ten recorded episodes and the chess-only checkpoint scoring 4 wins, 46 draws and 0 losses in 50 games against a random mover but 0 wins, 2 draws and 48 losses against Stockfish level 0. The release ships single-game checkpoints, a shared 12-option checkpoint, and training, evaluation and prediction commands using JSONL rows with a zero-based label index.", "body_md": "Train a small model that chooses among a changing list of text options.\n\nA Jev-like model takes a piece of text and a list of `N` text options. It returns one probability for each option. It does this in one pass instead of writing an answer word by word. [Jev](https://typesafe.ai/blog/introducing-system-one-models-and-jev) is TypeSafe's commercial model for this kind of task. TypeSafe has not published its design. This repository is an independent starter model with the same input and output shape.\n\nThe same option-attention head can score controller buttons from image patches. [This ten-second film](https://github.com/vinnylarouge/jevlike/blob/main/docs/jevre-demo-10s-bgm.mp4) joins two selected five-second windows: live `deadly_corridor` combat on the seven Doom buttons, then a chess controller walking to and playing moves with five keys. The diagram shows the tensors used for each decision. The Doom window came from the supplied joint checkpoint, which averaged 0.60 kills and -97.50 reward across its ten recorded episodes. The chess window came from the stronger chess-only checkpoint, which scored 4 wins, 46 draws and 0 losses in 50 sampled games against a random mover, but 0 wins, 2 draws and 48 losses against Stockfish level 0. The windows were selected for activity and are not typical-play or competence claims.\n\nInstall the game extras and record a fresh 640 by 480 Doom trace from the released joint checkpoint:\n\n```\nuv pip install -e '.[games]'\npython examples/doom/play.py examples/checkpoints/joint-imitation.pt --episodes 10 --game-seconds 35.3 --device cpu --capture-resolution 640x480 --output runs/doom.mp4 --trace runs/doom-trace.json\n```\n\nRender the trace in the same visual layout. This writes a silent film because the author-owned soundtrack source is not part of the repository.\n\n```\n(cd examples/film && npm install && npx playwright install chromium)\nexamples/film/make-film.sh runs/doom-trace.json runs/doom-film.mp4 10\n```\n\nThe release includes the [Doom example](https://github.com/vinnylarouge/jevlike/blob/main/examples/doom/README.md), the [chess example](https://github.com/vinnylarouge/jevlike/blob/main/examples/chess/README.md), the single-game checkpoints and the shared 12-option checkpoint. Both games import the visual scorer from `jevlike.vision`; there is no second model copy in either example.\n\nEach option becomes a query vector, which is a short list of numbers representing its text. The query assigns attention weights to the context tokens. Those weights make one context vector for that option. A shared dot product turns each option and context pair into one score. A softmax, which converts scores into probabilities that sum to one, runs across the options.\n\nThe default encoder learns byte embeddings from scratch. An encoder is the part that turns text into vectors. The optional Hugging Face path uses a frozen pretrained encoder, whose existing weights stay fixed while the small scorer learns.\n\nUse one JSON object per line:\n\n```\n{\"context\":\"The customer needs a refund.\",\"options\":[\"refund\",\"sales\",\"technical support\"],\"label\":0}\n```\n\n`label` is the zero-based index of the correct option. Each row may have a different number of options, with a minimum of two.\n\nRun these commands from the repository root. They create local synthetic data, train on it, evaluate the saved model and score one new menu.\n\n```\nuv venv\nsource .venv/bin/activate\nuv pip install -e '.[dev]'\n\njevlike-data synthetic --output data/synthetic\njevlike-train data/synthetic/train.jsonl \\\n  --validation data/synthetic/validation.jsonl \\\n  --output runs/synthetic.pt\njevlike-eval runs/synthetic.pt data/synthetic/test.jsonl\njevlike-predict runs/synthetic.pt \\\n  --context \"Choose the exact badge amber badger. Badge: amber badger.\" \\\n  --option \"azure crane\" \\\n  --option \"amber badger\" \\\n  --option \"gold heron\"\n```\n\nThe evaluation prints top-1 accuracy, which is the fraction of correct first choices. Top-3 accuracy is the fraction with the right answer among the three highest scores. Expected calibration error compares confidence with observed accuracy. The command also prints a shuffled-context control, which pairs each menu with the wrong context. A useful model should beat that control.\n\n1. Export train, validation and test JSONL files in the format above.\n2. Keep all options that the model will see at prediction time in each row.\n3. Split related records together. For example, keep all records for one customer or one target page in one split. This prevents near-duplicates from leaking into the test set.\n4. Run `jevlike-train` with your train and validation files.\n5. Run `jevlike-eval` once on the held-out test file. Held-out means the file was never used for training or model selection.\n\nThe default byte encoder truncates context to 192 bytes and each option to 32 bytes. Raise `--context-tokens` or `--option-tokens` when your text needs more room. Training supports CPU, Apple MPS for a Mac GPU, and CUDA for an NVIDIA GPU through `--device`.\n\nInstall the optional dependency and name any compatible encoder from Hugging Face:\n\n```\nuv pip install -e '.[transformers]'\njevlike-train data/synthetic/train.jsonl \\\n  --validation data/synthetic/validation.jsonl \\\n  --output runs/qwen-head.pt \\\n  --encoder hf \\\n  --hf-model Qwen/Qwen2.5-0.5B \\\n  --rank 256 \\\n  --batch-size 8\n```\n\nThe checkpoint stores the trained scorer head and the encoder name. It does not copy the frozen encoder weights. Loading the checkpoint therefore needs access to the same Hugging Face model.\n\n`--rank` sets the width of the small scorer head. A wider head has more trainable weights and uses more memory.\n\n[`scripts/get_wikispeedia.sh`](https://github.com/vinnylarouge/jevlike/blob/main/scripts/get_wikispeedia.sh) downloads the public SNAP archives and builds next-click JSONL files. The data stay outside this repository.\n\n```\nscripts/get_wikispeedia.sh\njevlike-train data/wikispeedia/jsonl/train.jsonl \\\n  --validation data/wikispeedia/jsonl/validation.jsonl \\\n  --output runs/wikispeedia.pt\n```\n\nCite Robert West and Jure Leskovec, *Human Wayfinding in Information Networks*, WWW 2012. Review the source data terms on the [SNAP dataset page](https://snap.stanford.edu/data/wikispeedia.html).\n\nIn the experiments that led to this starter, the one-pass scorer reached about 98% accuracy on synthetic menus. On target-disjoint Wikispeedia next-click data, a frozen Qwen2.5-0.5B encoder plus the scorer reached 26%, against about 8% for shuffled and random-encoder controls. A small model trained from scratch on 40,000 clicks reached 29%. At eight options, one pass was about 100 times faster than a small decoder forced to write 400 tokens.\n\nThese numbers describe local experiments, not this quickstart run. We did not show equal quality with Jev or reproduce TypeSafe's private training method.\n\n- This is a research starter, not a copy of Jev.\n- Accuracy depends on data quality, split quality and the encoder.\n- The byte encoder is cheap but weak on language meaning.\n- The pretrained path may download a large model and needs more memory.\n- One-pass scoring requires the complete option list before prediction.\n- The speed comparison used a small local decoder rather than a large commercial model.\n\nCode is released under the [MIT License](https://github.com/vinnylarouge/jevlike/blob/main/LICENSE). Downloaded datasets and pretrained models keep their own terms.", "url": "https://wpnews.pro/news/reverse-engineered-jev-like-model", "canonical_source": "https://github.com/vinnylarouge/jevlike", "published_at": "2026-09-16 18:49:57+00:00", "updated_at": "2026-09-16 19:14:24.640012+00:00", "lang": "en", "topics": ["ai-research", "machine-learning", "ai-products", "developer-tools"], "entities": ["Jev-like", "TypeSafe", "Jev", "Doom", "Stockfish", "Hugging Face", "jevlike.vision", "Playwright"], "alternates": {"html": "https://wpnews.pro/news/reverse-engineered-jev-like-model", "markdown": "https://wpnews.pro/news/reverse-engineered-jev-like-model.md", "text": "https://wpnews.pro/news/reverse-engineered-jev-like-model.txt", "jsonld": "https://wpnews.pro/news/reverse-engineered-jev-like-model.jsonld"}}