Announcing decision models

← Back to blog

We are happy to announce that NobodyWho now supports decision models!

Decision models were recently popularized by the release of Jev. They are general-purpose classifiers. Instead of writing a long prompt or using structured outputs, you define a set of criteria and let the model score them.

Our implementation sits on top of the great work from the llama.cpp team. We currently support Laya decision models and three types of questions:

• choice picks one option and gives the probability of each one.
• noul gives the probability that the answer is yes.
• score rates something on a scale you define.

Here's a short example in Python:

from nobodywho import Decision, Model

model = Model(model_path="hf://ggml-org/Laya-GGUF/Laya-Q8_0.gguf")
decisions = Decision(model=model, n_ctx=1024)

result = decisions.decide(
    state="Customer message: I was charged twice for my order last week and nobody has replied.",
    questions={
        "route": {
            "type": "choice",
            "instructions": "Choose the best team for this customer message.",
            "criteria": {
                "billing": "payments, charges, refunds, invoices",
                "shipping": "delivery, tracking, lost or late parcels",
                "technical": "bugs, errors, login problems",
            },
        },
        "angry": {"type": "noul", "instructions": "Is the customer angry?"},
        "urgency": {
            "type": "score",
            "instructions": "How urgent is this?",
            "criteria": ["can wait", "this week", "today", "right now"],
        },
    },
)

route = result["answers"]["route"]
print(f"Team: {route['choice']}")
for team, probability in route["probabilities"].items():
    print(f"  Probability of {team}: {probability:.1%}")

angry = result["answers"]["angry"]
print(f"Probability the customer is angry: {angry['noul']:.1%}")

urgency = result["answers"]["urgency"]
print(f"Urgency score: {urgency['score']:.2f} (0 = can wait, 3 = right now)")
for level, probability in urgency["probabilities"].items():
    print(f"  Probability of {urgency['legend'][level]}: {probability:.1%}")

Output:

Team: billing
  Probability of billing: 97.4%
  Probability of shipping: 1.5%
  Probability of technical: 1.1%
Probability the customer is angry: 78.6%
Urgency score: 1.90 (0 = can wait, 3 = right now)
  Probability of can wait: 4.0%
  Probability of this week: 46.7%
  Probability of today: 4.9%
  Probability of right now: 44.4%

To learn more, visit the docs for the language you prefer:

Music in the video: "Beauty Flow" by Kevin MacLeod (incompetech.com), licensed under CC BY 4.0.

Everything NobodyWho does is open-source, please leave a star on GitHub to support us ❤️

Get new posts in your feed reader: subscribe via RSS.