{"slug": "what-do-you-think-about-jev", "title": "What do you think about Jev?", "summary": "Independent measurement of Jev, a non-autoregressive classification model, shows its calibration error runs 2–2.5× above each study's own noise floor, according to a public claims-audit repo cited in a Hugging Face forum thread. The thread's commenter nootxlm said Jev's speed and cost come from its genuine non-autoregressive architecture rather than marketing, but that a single-parameter fix cut most of the calibration gap, indicating the mechanism works while calibration remains unfinished. A 4B fine-tuned model reportedly reached ~93% on BANKING77 in about 30 minutes versus ~80% for Jev alone.", "body_md": "[mrs83](https://discuss.huggingface.co/u/mrs83)\n1\n \nI was in the middle of a seven-day vacation, trying to disconnect, when my phone buzzes. Multiple contacts message me the exact same question: “What do you think about Jev?”\n\nAway from my workstation, my first instinct was to check with Gemini on my mobile phone. The frontier model’s verdict? It politely told me I was probably hallucinating.\n\nFair enough. I opened LinkedIn to see what was actually going on, and sure enough, my feed was already flooded with people hyping it as the next revolutionary breakthrough in AI/ML. I then decided to take a look.\n\nLots of massive efficiency claims, but very little evaluation data. Plenty of words framing it as a \"new paradigm” but thin technical detail on what is actually happening under the hood.\n\nThe concept is practical, but doesn’t look much different from what we already have.\n\nWhich makes me wonder: did someone actually build a genuine architectural breakthrough, or did they just repackage classic classification? Or it is just an LLM wrapper with clever positioning and marketing?\n\nWhat do you think about Jev?\n\nScreenshot from Linkedin Post: [Jev does not remotely assign probabilities correctly even for the most simple examples. However, I like that they introduce that notion and future models will probably perform better on this. Or do I… | Florian Hönicke | 84 comments](https://lnkd.in/p/eQpdc7En)\n\n \n \n[nootxlm](https://discuss.huggingface.co/u/nootxlm)\n2\n \nGood question — and there’s enough independent measurement out there now to separate the two halves of the claim.\n\nThe structural part is real: Jev is genuinely non-autoregressive — one parallel pass, no text generation — so the speed and cost numbers come from the architecture, not the positioning. It’s not a wrapper. The accuracy-and-calibration part is where the measurement matters more than the marketing.\n\nThe sharpest check on any probability model is its calibration curve: the stated probabilities should match observed frequencies. On that, the independent checks disagree with the “calibrated” claim — one public claims-audit repo recomputed the calibration error against each study’s own noise floor and found it running 2–2.5× too high, with the probabilities compressed toward the middle (that’s the same symptom as the Florian Hönicke screenshot you linked). Interestingly, a single-parameter fix cut most of that away, which suggests the mechanism is fine and the calibration just isn’t finished.\n\nSo I’d read it as: genuinely fast classification with genuinely unfinished calibration. The best next step is the boring one — take your own task, score Jev against a plain fine-tuned small model (someone got ~93% on BANKING77 with a 4B model in about 30 minutes vs ~80% for Jev alone), and set your decision threshold off the measured calibration, not the brochure. If the numbers hold on your data, the speed is the actual product.\n\n \n \n[mrs83](https://discuss.huggingface.co/u/mrs83)\n3\n \nThanks! This confirms my read on the calibration trade offs.\n\nHonestly, welcome back machine learning? I love this pivot back to narrow and structured tasks!!\n\nI am tired by the endless parade of all-purpose multimodal instruction-tuned LLMs … anything resembling mid-2010s applied ML brings me  joy \n\nI’m heads-down on a few things through the end of the year, but I’d love to explore this direction further with folks in the EU timezone around January", "url": "https://wpnews.pro/news/what-do-you-think-about-jev", "canonical_source": "https://discuss.huggingface.co/t/what-do-you-think-about-jev/180747#post_3", "published_at": "2026-09-27 21:55:04+00:00", "updated_at": "2026-09-27 21:59:59.893664+00:00", "lang": "en", "topics": ["machine-learning", "large-language-models", "ai-research"], "entities": ["Jev", "Hugging Face", "Gemini", "LinkedIn", "Florian Hönicke", "BANKING77"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/what-do-you-think-about-jev", "markdown": "https://wpnews.pro/news/what-do-you-think-about-jev.md", "text": "https://wpnews.pro/news/what-do-you-think-about-jev.txt", "jsonld": "https://wpnews.pro/news/what-do-you-think-about-jev.jsonld"}}