{"slug": "zet-an-open-source-layer-on-laya-that-knows-when-to-ask-a-human", "title": "Zet: an open-source layer on Laya that knows when to ask a human", "summary": "A developer released Zet, an open-source Apache-2.0 layer built on top of the Laya agent framework that uses conformal prediction sets and a Learn-then-Test threshold, calibrated per language, to mark each answer as sure or unsure so automated responses stay within a user-chosen error budget. On the MASSIVE benchmark with a 5% budget, Zet automated 84% of English and 80% of Swedish requests across a 6-scenario task with zero errors in 75 sure English answers, and automated nothing on a harder 18-scenario task while its prediction sets still contained the correct answer 97-99% of the time. The tool runs locally on ONNX Runtime without PyTorch and includes a web interface for task creation, uploading labeled examples, calibration, and review of unsure answers.", "body_md": "Confidence scores aren't error rates. Zet sits on top of Laya and marks every answer sure or unsure, using conformal prediction sets and a Learn-then-Test threshold calibrated per language, so the answers it automates stay within an error budget you choose (say 5%). On MASSIVE (human-labeled, 300 examples per language, 5% budget), it automated 84% of English and 80% of Swedish requests on a 6-scenario task, with 0 errors in 75 sure English answers. When the task got harder (18 scenarios), it automated nothing, while its prediction sets still contained the right answer 97-99% of the time. That refusal is the feature working. Runs locally on ONNX Runtime with no PyTorch, with a web interface for creating tasks, uploading labeled examples, calibrating, and reviewing unsure answers. Apache-2.0, built on Laya (not endorsed by its authors).", "url": "https://wpnews.pro/news/zet-an-open-source-layer-on-laya-that-knows-when-to-ask-a-human", "canonical_source": "https://dev.to/yoosseph/zet-an-open-source-layer-on-laya-that-knows-when-to-ask-a-human-1o6i", "published_at": "2026-09-30 03:10:19+00:00", "updated_at": "2026-09-30 03:16:58.971782+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "machine-learning", "natural-language-processing", "ai-safety"], "entities": ["Zet", "Laya", "ONNX Runtime", "PyTorch", "MASSIVE"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/zet-an-open-source-layer-on-laya-that-knows-when-to-ask-a-human", "markdown": "https://wpnews.pro/news/zet-an-open-source-layer-on-laya-that-knows-when-to-ask-a-human.md", "text": "https://wpnews.pro/news/zet-an-open-source-layer-on-laya-that-knows-when-to-ask-a-human.txt", "jsonld": "https://wpnews.pro/news/zet-an-open-source-layer-on-laya-that-knows-when-to-ask-a-human.jsonld"}}