{"slug": "local-jev-fine-tuned-in-30-mins-on-mac-air", "title": "Local JEV, fine tuned in < 30 mins on Mac Air", "summary": "An independent experiment fine-tuned a 135M-parameter model on the 10,003-message Banking77 dataset in under 30 minutes on a 16 GB Apple MacBook Air, lifting exact intent accuracy from 1.62% to 83.25% and operational decision accuracy from 47.99% to 92.01% on the 3,080-message test split. At a 70% confidence threshold the model auto-routes 67.05% of messages with 98.45% decision accuracy, while the uncertain 32.95% go to human review, and warm single-message inference runs at a 35.16 ms p50 and 26.26 requests per second. The author credits Duarte O.Carmo's \"Jev in 25 lines of Python\" post as the inspiration and states the model is not TypeSafe's Jev and does not reproduce its architecture.", "body_md": "## Models that do not write. They decide.\n\nThis tiny local model turns a banking message into a typed decision in about 35 milliseconds. No generated paragraph. No data-center GPU. I trained the whole thing in under 30 minutes on an Apple MacBook Air with 16 GB of RAM.\n\nEveryone is talking about models that take text in and return typed choices with probabilities. It sounds like a new category of AI. So I built one.\n\n**Credit where it is due.** This experiment was directly inspired by Duarte O.Carmo’s NobodyWho article, [“Jev in 25 lines of Python.”](https://www.nobodywho.ai/posts/jev-in-25-lines/) That post showed how a local open model’s next-token logits can become a closed set of probabilities. I took that idea further by fine-tuning a small model on public banking data, measuring it on a held-out test set, and adding an explicit human-review threshold. Jev itself is TypeSafe AI’s product; this is an independent experiment, not a reproduction of its architecture.\n\nTo be precise, I built my own tiny, local, Jev-shaped decision model. It is not TypeSafe’s Jev and does not claim to reproduce its architecture. It takes the useful product idea seriously: stop asking a language model to compose an answer when the product needs a bounded decision.\n\nI fine-tuned a 135M-parameter model on [Banking77](https://github.com/PolyAI-LDN/task-specific-datasets/tree/master/banking_data): 10,003 customer-support messages spanning 77 intents. The model sees a message such as:\n\nWhy was my card payment declined?\n\nIt does not generate a reply. It returns a probability distribution:\n\n```\n{\n  \"declined_card_payment\": 0.9625,\n  \"reverted_card_payment\": 0.0204,\n  \"declined_cash_withdrawal\": 0.0092\n}\n```\n\nThen deliberately boring policy code turns intent into action:\n\n```\nif confidence < 0.70:\n    action = \"human_review\"\nelif intent in security_intents:\n    action = \"secure_account_now\"\nelif intent in investigation_intents:\n    action = \"investigate_transaction\"\nelif intent in assisted_support_intents:\n    action = \"assisted_support\"\nelse:\n    action = \"self_service\"\n```\n\nThat separation matters. The model estimates what the customer needs. Ordinary, inspectable code decides what the product should do about it.\n\n## Fine. But does my fake Jev actually work?\n\nWe evaluated on Banking77’s 3,080-message test split. The threshold curve below is descriptive test-set analysis; a production system must choose and lock its threshold on separate validation data.\n\n| Result | Before training | After training | \n|---|---|---|\n| Exact intent accuracy | 1.62% | 83.25% | \n| Operational decision accuracy | 47.99% | 92.01% | \n| Calibration error | 27.39% | 6.78% | \n\nThe pre-training intent score is close to the 1-in-77 random baseline. The high pre-training action score is less impressive than it looks: many intents share the same broad action.\n\n## Probability lets the model decline to decide\n\nWith a 70% acceptance threshold, the model automatically routes 67.05% of messages. On that accepted subset, exact intents are 95.88% accurate and operational decisions are 98.45% accurate. The uncertain 32.95% go to a person.\n\n| Threshold | Automated | Decision accuracy | \n|---|---|---|\n| 50% | 83.38% | 96.77% | \n| 70% | 67.05% | 98.45% | \n| 90% | 41.56% | 99.69% | \n\nMore automation is available if we accept more errors. Less automation buys greater precision. The threshold makes that tradeoff explicit instead of hiding it behind a generated paragraph.\n\n## Is it fast?\n\nVery. On the same 16 GB Apple MacBook Air used for training, across 1,000 warm single-message requests:\n\n```\np50        35.16 ms\np95        47.43 ms\np99        57.88 ms\nthroughput 26.26 requests/second\n```\n\nCold model loading took 1.73 seconds. The warm figures include tokenization, inference, probability normalization, thresholding, and action routing.\n\nThe full one-epoch fine-tune on all 10,003 training examples took less than 30 minutes on that laptop. That combination—a sub-30-minute training run and answers in tens of milliseconds—is what made the experiment feel less like a demo and more like a practical local component.\n\n## Run it yourself\n\nThis repository contains the exact [implementation](banking77.py), the [trained LoRA adapter](adapter-banking77/), and the [evaluation evidence](evidence/) behind every number above. Clone it, install the pinned dependencies, and make a prediction:\n\n```\ngit clone https://github.com/shmcsensei/easy-jev-fine-tune.git\ncd easy-jev-fine-tune\npython3.12 -m venv .venv\nsource .venv/bin/activate\npython -m pip install -r requirements.txt\n\npython banking77.py predict \\\n  \"Why was my card payment declined?\"\n```\n\nTo recreate the adapter from the official training split and evaluate it once on the test split:\n\n```\npython banking77.py train \\\n  --examples-per-class 999999 --epochs 1 \\\n  --output adapter-banking77-reproduced\n\npython banking77.py evaluate \\\n  --adapter adapter-banking77-reproduced \\\n  --threshold 0.70\n```\n\n**The code is part of the post.** The README explains the fast smoke-test path, full reproduction commands, evaluation, benchmarking, expected downloads, and sources of result variation.\n\n## There. I built my own “Jev.”\n\nWell, Jev-shaped.\n\nIt is not the actual Jev, and it is not a production banking system. The probabilities are better calibrated after training, but they are not guarantees. Banking77 labels support intents, not fraud outcomes. The action mapping is prototype business policy and would need operational, risk, and compliance review.\n\nThe controversial bit is not that this replaces Jev. It does not. It is that a surprisingly large slice of the product experience is reproducible with a tiny open model, one classification head, some LoRA weights, and an honest threshold.\n\nThe useful part is real: a small local model can turn unstructured customer language into fast probabilistic decisions, expose uncertainty, and hand the hard cases to humans.", "url": "https://wpnews.pro/news/local-jev-fine-tuned-in-30-mins-on-mac-air", "canonical_source": "https://shmcsensei.github.io/easy-jev-fine-tune/", "published_at": "2026-09-24 15:19:00+00:00", "updated_at": "2026-09-24 15:30:17.567556+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "natural-language-processing", "ai-tools"], "entities": ["Banking77", "Apple MacBook Air", "Duarte O.Carmo", "NobodyWho", "TypeSafe", "Jev"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/local-jev-fine-tuned-in-30-mins-on-mac-air", "markdown": "https://wpnews.pro/news/local-jev-fine-tuned-in-30-mins-on-mac-air.md", "text": "https://wpnews.pro/news/local-jev-fine-tuned-in-30-mins-on-mac-air.txt", "jsonld": "https://wpnews.pro/news/local-jev-fine-tuned-in-30-mins-on-mac-air.jsonld"}}