cd /news/ai-tools/i-let-an-llm-judge-inside-python-s-i… · home topics ai-tools article
[ARTICLE · art-134892] src=dev.to ↗ pub= topic=ai-tools verified=true sentiment=· neutral

I let an LLM judge inside Python's if statements, then ran Ansible's own tests on it

A developer built fuzzyif, a Python library that lets plain-language questions stand in for `if` conditions by calling TypeSafe AI's Jev model, which returns probabilities and label choices instead of generated text. To test it beyond toy examples, the developer patched Ansible's distribution detection, deleting 418 lines and 84 `if`/`elif` branches across thirteen parsers plus the OS_FAMILY_MAP table, and ran Ansible's own test fixtures against the result. The conclusion: the judgement portion of the conditional pile was replaceable, but the extraction portion was not.

by read5 min views2 publishedSep 20, 2026

I wrote a small library that lets you put a question, in plain language, where a Python if condition goes.

from fuzzyif import fuzzy

if fuzzy("Is this message urgent?", msg):
    notify_oncall(msg)

It is called fuzzyif. pip install fuzzyif. The code is here:

https://github.com/Tdual/fuzzyif

This post is about two things: what the library actually does, and whether it can replace a real pile of if/ elif in a project everyone knows. I patched Ansible's distribution detection and ran Ansible's own test fixtures against the result.

The short version: the judgement part of the pile was replaceable. The extraction part was not. Finding exactly where that line falls was the most useful outcome.

If you have ever routed support emails into "bug report", "how-to question" and "billing", you have written this:

if "error" in msg or "crash" in msg or "doesn't work" in msg:
    kind = "bug"
elif "how do I" in msg or "how to" in msg or "usage" in msg:
    kind = "howto"
elif "invoice" in msg or "charge" in msg or "refund" in msg:
    kind = "billing"

This code loses the moment you write it. "The screen goes blank after login" is a bug report with none of the bug keywords. "What does this error mean?" is a how-to question that contains "error". Every keyword you add fixes one case and breaks another, and the ladder never stops growing.

What you wanted to write was the question itself: is this a bug report? fuzzyif lets you write that.

Behind fuzzyif is Jev, a model TypeSafe AI released in September 2026. They call it a "System One" model: it does not generate text. You give it a text and a question, and it returns a probability, a choice among labels, or a position on a scale. Values a program can use directly.

fuzzy("Is this urgent?", msg) sends the question and the text to Jev, gets back a probability (say 0.93), and applies a 0.5 threshold. Because nothing is generated, a call on a warm connection takes about 0.25 s and produces about 20 output tokens. Identical question and text pairs are cached, and the HTTPS connection is reused.

fuzzy(question, text) returns a bool for a yes/no question.fuzzy_match(text, {label: description}) picks one label. Use this when you want exactly one of several. Stacking fuzzy() calls in if / elif is not exclusive: if two questions both cross the threshold, the first branch wins even when the second was more likely.fuzzy_batch(text, [q1, q2]) answers several yes/no questions in one request.fuzzy_score(text, question, [level0, level1, ...]) returns a position on an ordered scale, for things like "how angry is the writer". The support-email router becomes one call:

kind = fuzzy_match(msg, {
    "bug":     "a bug report",
    "howto":   "a how-to question",
    "billing": "a question about invoices or charges",
})

Toy examples prove nothing, so I set three conditions: a library everyone knows, an official test suite, and a target whose purpose anyone can understand.

I picked Ansible. The first thing Ansible does on a host is work out ansible_distribution (Ubuntu? RHEL?) and ansible_os_family (Debian-like? RedHat-like?). That decision lives in distribution.py, 786 lines, structured like this:

/etc/os-release, /etc/redhat-release, /etc/lsb-release, /etc/SuSE-release, and so on parse_distribution_file_* methods, each a ladder of if/ elif over the file textOS_FAMILY_MAP, a hand-maintained table of about 70 entries Everything I deleted, in one picture. 418 lines, 84 if/ elif. It is unreadable at this size on purpose: this is what a pile of if looks like.

One of the thirteen at readable size. This is the SUSE parser, 67 lines.

I rewrote process_dist_files and deleted the thirteen parsers and OS_FAMILY_MAP. This is what is left.

It concatenates whatever release files exist into one block of evidence and asks fuzzy_match "which distribution is this". A second fuzzy_match asks "which family". The label sets are the distribution names Ansible already documents, with a one-line description each.

The file went from 786 lines to 450.

Ansible ships 90 recorded fixtures: real /etc/*-release contents captured from machines, paired with the facts the collector must report. They cover 52 distributions. I ran that test unchanged against the patched code.

65 of 90 fixtures matched on every key. 25 differed on at least one key.

Per key, the picture is much sharper:

distribution: 90 of 90 os_family: 87 of 88 distribution_release: 68 of 88, and this is where it fell apart I went through all 25. Almost none are wrong judgements. They are Ansible's house conventions for cutting substrings out of files:

release: VERSION="15-SP6" becomes 6 15.1 becomes 1 clear-linux-os Stream minor_version keyMarch 2022 from a custom file None of these is a question of what something is. They are questions of which slice of the string to take. My patch left version and codename to the distro library that Ansible already uses as a baseline, so it does not reproduce those conventions.

The one real judgement miss: Ansible has two labels for the same UnionTech OS, Uos (Debian family) and UnionTech (RedHat family), chosen by which release files happen to exist. The judge picked the other one.

Look closely at a pile of if and you find two different jobs mixed together.

One is judgement. "What distribution do these files describe?" "Is this database error a disconnect?" "Is this ticket a bug report?" These are about meaning. Written as keyword matches they grow without bound. They suit fuzzy.

The other is extraction. "Take the value of VERSION_ID." "Keep only the service-pack number." "Pull the codename out of the parentheses." These are about position, not meaning. A regex does them in one line, deterministically. There is no reason to hand them to a model.

fuzzyif replaces the judgement. Keep the extraction. Once you can see which lines are which, you know which part of the pile is safe to delete.

http.client from the standard library, one keep-alive connection per thread. First call about 0.6 s, later calls about 0.25 s.Retry-After. mock() for tests: answer from a mapping without touching the API, so code that uses fuzzy stays unit-testable.str bodies as latin-1, so any non-Latin text failed. Bodies are now sent as UTF-8 bytes. Found while judging Japanese text; glad it made the first release.

pip install fuzzyif

Put a TypeSafe API key in TYPESAFE_API_KEY or ~/.config/typesafe/api_key. The Ansible patch script and everything needed to reproduce the numbers above are in examples/ansible_distribution/ in the repository linked at the top.

Next I want to build the tool that reads an existing pile of if, separates judgement from extraction, and proposes the fuzzy rewrite for just the judgement part.

── more in #ai-tools 4 stories · sorted by recency
── more on @fuzzyif 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
Live at https://your-agent.zahid.host
Get free account → Pricing
from €0/mo · no card required
LIVE [news/i-let-an-llm-judge-i…] indexed:0 read:5min 2026-09-20 ·