Hi HN, we built TOD, a model and API for jevlike LLM pipelines.
we have people using it for, routing a ticket, classifying intent, deciding whether something is urgent, or rating severity.
Architecture - 12B Gemma fine-tuned to score options. since it is gemma based it has multimodality built in - When there are >120 labels, we have a 150M retriever that narrows them to a shortlist (recall@16 is 0.9–1.0 on our evals), and the 12B model scores that shortlist. - Context goes up to 49k tokens, with up to 8 images per request.
we have open sourced it,
model: [https://huggingface.co/parsecai/tod](https://huggingface.co/parsecai/tod)
API: [https://parseclab.ai/tod](https://parseclab.ai/tod)
We are working on getting the RL environments to finetune for custom use cases.
would love to hear your opinions on this.
Comments URL: [https://news.ycombinator.com/item?id=49981275](https://news.ycombinator.com/item?id=49981275)
Points: 1
source & further reading
parseclab.ai — original article