Confidence scores aren't error rates. Zet sits on top of Laya and marks every answer sure or unsure, using conformal prediction sets and a Learn-then-Test threshold calibrated per language, so the answers it automates stay within an error budget you choose (say 5%). On MASSIVE (human-labeled, 300 examples per language, 5% budget), it automated 84% of English and 80% of Swedish requests on a 6-scenario task, with 0 errors in 75 sure English answers. When the task got harder (18 scenarios), it automated nothing, while its prediction sets still contained the right answer 97-99% of the time. That refusal is the feature working. Runs locally on ONNX Runtime with no PyTorch, with a web interface for creating tasks, up labeled examples, calibrating, and reviewing unsure answers. Apache-2.0, built on Laya (not endorsed by its authors).
Zet: an open-source layer on Laya that knows when to ask a human
A developer released Zet, an open-source Apache-2.0 layer built on top of the Laya agent framework that uses conformal prediction sets and a Learn-then-Test threshold, calibrated per language, to mark each answer as sure or unsure so automated responses stay within a user-chosen error budget. On the MASSIVE benchmark with a 5% budget, Zet automated 84% of English and 80% of Swedish requests across a 6-scenario task with zero errors in 75 sure English answers, and automated nothing on a harder 18-scenario task while its prediction sets still contained the correct answer 97-99% of the time. The tool runs locally on ONNX Runtime without PyTorch and includes a web interface for task creation, uploading labeled examples, calibration, and review of unsure answers.
Run your AI side-project on zahid.host
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.