cd /news/artificial-intelligence/ultrabert-jev-encoder-models-for-sys… · home › topics › artificial-intelligence › article
[ARTICLE · art-170357] src=machinebrief.com ↗ pub= topic=artificial-intelligence verified=true sentiment=↑ positive

UltraBERT-JEV: Encoder models for System One Applications

A technical report on arXiv (2611.17867v1) finds that a fine-tuned UltraBERT-380M encoder scored 69.7% to 72.1% test accuracy on single-pass option selection, versus 67.6% for Qwen3-0.6B trained on the same data with the LLM-as-Jev recipe. Scoring each option by "yes" minus "no" rather than "yes" alone produced the larger gain, lifting accuracy from 69.7% to 71.3% on the test set and from 57.8% to 59.6% on external benchmarks, where the decoder reached 56.1%; sigmoid normalisation added 0.8 test points while a square-root softplus added none. On a single NVIDIA A100 processing one request at a time, the encoder took 29 ms per decision against 68 ms for the decoder but used more peak memory, 2.8 GiB versus 2.2 GiB, and an auxiliary head trained on human-marked evidence and labels selected with a reward adapted from SelfCite predicts in the same pass which sentences support the answer.

by read1 min views3 publishedNov 16, 2026

arXiv:2611.17867v1 Announce Type: new Abstract: We study encoder models that pick one of several options in a single forward pass, without generating text. This technical report compares recipes for fine-tuning UltraBERT-380M with one mask per option. The four recipes differ in how an option is scored and how the scores become probabilities; one follows the expert gate of DeepSeek-V3. Trained on the same data as Qwen3-0.6B with the LLM-as-Jev recipe (67.6% test accuracy), the encoder reaches 69.7% to 72.1%. Scoring an option by "yes" minus "no" instead of "yes" alone gives the larger gain, from 69.7% to 71.3% on the test set and from 57.8% to 59.6% on external benchmarks (56.1% for the decoder). Sigmoid normalisation adds 0.8 test points; a square-root softplus adds none. One request at a time on an NVIDIA A100, the encoder takes 29 ms per decision against 68 ms, but uses more peak memory (2.8 against 2.2 GiB). An auxiliary head, trained on human-marked evidence and on labels selected with a reward adapted from SelfCite, predicts in the same pass the sentences annotated as support for the answer.

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @ultrabert-380m 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/ultrabert-jev-encode…] indexed:0 read:1min 2026-11-16 · —