I've built a local decision engine drop-in replacement for the TypeSafe System One api because i wanted to test locally some sensitive use cases and at the time jev subscriptions was closed.
Snap supports 4 class of questions that let you classify, rank and decide about an input in sub-second on your local machine. It doesn't generate text, it reads logits in a a single forward pass.
If you have a mac you can run brew install emnlmn/snap/snap and then snap serve. It will pull the snap1-2b model from HF and start a local web server and web playground ui. Even if the benchmarks show a 0.643 zero-shot accuracy score on zero-shot typed-decisions, snap1-2b it's 2b model, you can't expect high intelligence from it.
I'd like to share it with the community and would love to get feedback about the runtime and the training i've done with snap1. It's my first big work in the machine learning world and i hope it can be useful to someone that want to test out decision model locally.
Open weight model, MIT license.
Comments URL: [https://news.ycombinator.com/item?id=49921892](https://news.ycombinator.com/item?id=49921892)
Points: 1