Show HN: Jev-CLI – CLI wrapper for JEV typesafe AI model

A developer released Jev-CLI, an open-source command-line wrapper for TypeSafe AI's Jev decision model that returns typed, calibrated answers instead of prose when analyzing JSON, NDJSON, and JSONC system artifacts. The tool anchors every answer to the source lines it came from and attaches a numeric certainty value, evaluating all typed questions from multiple packs in a single parallel API call — the example output shows 8 questions carried in 1 request using 1290 input and 36 output tokens. Jev-CLI installs via uv without sudo or system Python changes, supports noul, choice, and score question types, and offers window modes of 25 records (default), one record, or the whole input, capped by --max-state-bytes.

Analyze JSON, NDJSON, and JSONC system artifacts with TypeSafe AI https://docs.typesafe.ai/introduction 's Jev decision model, and get typed, calibrated answers back instead of prose. bash $ uv run jcli analyze app.ndjson --pack logs.triage --window 2 --mark-uncertain 0.6 -o table where question type answer conf app.ndjson:1-2 is incident noul 0.11 no 0.78 app.ndjson:1-2 severity score 0.2 Informational 0.71 app.ndjson:1-2 category choice application 0.42 app.ndjson:3-4 is incident noul 0.93 yes 0.86 app.ndjson:3-4 severity score 1.87 Outage 0.71 app.ndjson:3-4 category choice dependency 0.88 noul answers report derived certainty |p-0.5| 2, not an API confidence score. 3 call s 1290 in / 36 out tokens Every answer is anchored to the lines it came from, and every answer carries a number saying how sure the model is — so you can gate on it. ./scripts/install.sh puts jcli on your PATH via uv export TYPESAFE API KEY=... or --api-key-file, or ~/.config/jcli/config.toml jcli --help | Flag | | |---|---| | none | install from this checkout | | --dev | editable — source edits take effect without reinstalling | | --dry-run | print what would happen, change nothing | | --uninstall | remove it | It needs uv https://docs.astral.sh/uv/getting-started/installation/ and prints the install command if uv is missing. Nothing goes into the system Python; nothing needs sudo. After installing it verifies the binary runs, reports its version, and checks that the built-in packs and transforms actually shipped — a wheel that builds but omits its YAML data is the failure worth catching at install time, not on first use. To work in the repo without installing, uv sync and prefix commands with uv run . Jev takes a state plus a map of typed questions and evaluates all of them in one call, in parallel . So asking eight questions costs about what asking one costs. jcli leans on that: selecting two packs does not make two requests, it makes one request carrying both packs' questions. bash $ jcli analyze auth.ndjson -p logs.triage -p security.audit --dry-run | jq '. | length' 1 one window - one call $ jcli analyze auth.ndjson -p logs.triage -p security.audit --dry-run \ | jq '. 0 .request.questions | keys | length' 8 ...carrying all eight questions Three question types, straight from the API: | Type | Asks | Returns | |---|---|---| | noul | Is this statement true? | probability 0–1 | | choice | Which of these labels? | label + full distribution + confidence | | score | Where on this rubric? | position may fall between levels + confidence | jcli analyze session.log text log; transform auto-detected jcli analyze app.ndjson NDJSON, streamed line by line jcli analyze cluster.json JSON object, or array of records jcli analyze config.jsonc comments and trailing commas welcome cat events.ndjson | jcli analyze - stdin jcli analyze a.ndjson b.ndjson several sources; windows never straddle files The dialect is auto-detected; --format json|ndjson|jsonc overrides. One flag, --window : | Value | Behaviour | |---|---| | --window 25 default | N records per call. Neighbouring lines give each other context. | | --window record | One call per record. Per-item triage you intend to filter on. Identical to --window 1 . | | --window whole | The entire input as one state. One verdict over the corpus. | Every mode is additionally capped by --max-state-bytes . The API's true ceiling is not published; jcli probe measures it against the live service so you can set the flag from measurement rather than superstition. jcli analyze session.log --filter agent=abc123 just one agent's records jcli analyze audit.ndjson --filter 'level=error' --filter 'status =500' ANDed jcli analyze session.log --filter 'tool =Bash' --filter 'desc~cargo' jcli analyze k8s-events.json -s reason -s message project down to what matters jcli analyze audit.ndjson --redact-preset secrets scrub before anything leaves the process --filter FIELD OP VALUE uses the same operators as --fail-on : = = ~ contains, case-insensitive ~ = <= < . Dotted paths reach nested fields user.id=u1 . Repeating the flag ANDs the terms; for an "or", run the two separately. It is record-aware, which grep is not: on a log whose fields contain embedded newlines, grep agent=X silently truncates every multi-line record. Filtering also runs before --select and --redact , so you can filter on an identifier and then scrub it. A filter that matches nothing is an error listing the values the field actually holds — an empty table is a poor way to discover a typo. Redaction runs strictly before the first byte reaches the SDK. Matched values become redacted rather than disappearing, so "did this request carry credentials" is still answerable. Most system artifacts are not JSON. A transform declares the shape of a text log once, in YAML, and everything downstream — chunking, redaction, packs, confidence gating — works unchanged. jcli transforms list jcli transforms show syslog --sample /var/log/auth.log see what it makes of a real file jcli analyze /var/log/auth.log -p security.audit transform auto-detected jcli analyze weird.log -t ./my-transform.yaml or point at your own -t is an override, not a requirement. When the input is not JSON, every transform is scored by parse rate over a sample; a single clear winner is used and announced on stderr. If nothing matches, or two match equally, the run stops and lists the candidates with their scores — a wrong transform yields plausible records with the wrong fields, so guessing is worse than asking. name: bracketed-kv description: A bracketed-timestamp key=value log whose last field is quoted free text. A record begins here. Lines that do not match belong to the record above, which is what lets a field contain embedded newlines. record start: '^\ \d{4}-\d{2}-\d{2}T' pattern: |- ?s ^\ ?P<ts ^\ \ \s+ agent= ?P<agent \S+ \s+ tool= ?P<tool \S+ \s+ desc=" ?P<desc . "\s $ flags: verbose null values: "-" a bare dash becomes null coerce: { ts: timestamp } int | float | bool | timestamp | string syslog is the only transform that ships. Everything else is a file you write and pass with -t ./my-transform.yaml , or drop into ~/.config/jcli/transforms/ to have it discovered and auto-detected like a built-in. A worked example lives in tests/fixtures/bracketed-kv.yaml . Named groups become JSON keys. Two details matter in practice, both learned from real emitter output rather than assumed: - Records are not lines. record start frames them; continuation lines are joined. - Quoted fields are not shell-quoted. Emitters write command lines into a desc="..." field without escaping the quotes inside, so logfmt, csv, and shlex parsers all truncate at the first inner quote. A greedy group running to the record's final quote is the fix. Nothing is dropped silently. A record the pattern rejects is kept verbatim under unparsed and counted in a warning — a log whose format drifted mid-file is itself a finding, and an empty result set is a bad way to discover it. Transforms layer like packs: built-ins, then ~/.config/jcli/transforms/ .yaml , then -t ./my-transform.yaml . Four ship built in: | Pack | Questions | |---|---| | logs.triage | is incident , severity , category , actionable | | logs.anomaly | anomalous , novelty , recurring pattern | | security.audit | suspicious , threat class , privilege escalation , blast radius | | security.finding | exploitable , false positive likelihood , remediation urgency | | agent.session | activity , progress , read before write , repetition | jcli packs list jcli packs show security.audit --resolved exactly what would be sent jcli packs init mine --from logs.triage scaffold your own jcli packs validate ./mine.yaml --ask / -a defines a question on the command line, no file needed. Two forms: shorthand: a noul. The '=' comes first, so a colon in the question is fine. jcli analyze session.log -a 'stuck=Is the agent repeating itself?' full form: the same YAML a pack file uses, so any type works jcli analyze session.log -a 'focus: {type: score, instructions: "How concentrated?", criteria: scattered, clustered, focused }' It needs no --pack , merges with one when given, and can reword a built-in question by naming its id. Quote any value containing : , , or ? — YAML flow mappings require it. Packs layer, lowest to highest: 1. built-ins 2. ~/.config/jcli/questions/ .yaml — a file whose name: matches a built-in replaces it 3. --questions FILE — deep-merged overlays, repeatable 4. --set QID.path=value — repeatable, JSON-valued when parseable 5. --drop QID jcli analyze app.ndjson -p logs.triage \ --set 'category.criteria.application.what=A bug in our own service code' \ --drop actionable A --set only replaces the leaf it names; sibling keys survive. Overrides are re-validated afterwards, because a --set can break a rubric as easily as fix it. instructions and criteria accept nested structure, not just strings Advanced: structure https://docs.typesafe.ai/primitives/advanced : name: logs.triage version: 1 questions: severity: type: score instructions: How severe is the worst condition in these records? criteria: ordered, lowest first - summary: Informational signals: "debug/info only", "no error fields" - summary: Degraded signals: "elevated latency", "partial retries" - summary: Outage signals: "sustained 5xx", "dependency down", "data loss" category: type: choice instructions: Which subsystem is the most likely origin? criteria: at least two options infra: what: The host, network, disk, or scheduler beneath the service. examples: "OOMKilled", "disk full", "node NotReady" security: what: Authentication, authorization, or tampering signal. not for: Ordinary 401s from routinely expired tokens. is incident: type: noul instructions: Is this a live production incident? criteria: "true": User-visible failure or SLO breach in progress. "false": Routine errors, retries, or known-benign noise. choice and score answers carry a confidence the API derives from the shape of the probability distribution. noul answers do not — a noul is a single probability and nothing else. jcli does not invent one. It reports a separate certainty for nouls, computed as |p − 0.5| × 2 and shown under its own column with a . A noul of 0.5 is maximally unsure; a noul of 0.05 is a confident no . Conflating that with the API's calibrated confidence would make --confident mean two different things in one table. Nouls do carry a probabilities map — {"true": p, "false": 1 - p} — so every answer type has the same shape and one consumer handles all three: {"question": "is incident", "type": "noul", "value": 0.93, "certainty": 0.86, "probabilities": {"true": 0.93, "false": 0.07}} That is a different claim from a confidence. For a two-outcome question the returned number fixes the complement exactly, so writing it out adds no information and invents none. In the table, a noul renders as 0.93 yes / 0.11 no — the same value meaning shape score has always used, so a confident no is never misread as a weak yes . The label is mechanical which side of 0.5 , inventing no threshold; how strongly it leans is the adjacent certainty column. --probabilities / -P adds a distribution column to table and summary : bash $ jcli analyze app.ndjson -p logs.triage -o table --probabilities where question type answer conf distribution app.ndjson:1-2 is incident noul 0.11 no 0.78 false 89% · true 11% app.ndjson:1-2 severity score 0.2 Informational 0.71 Outage 85% · Degraded 10% · ... app.ndjson:1-2 category choice application 0.42 dependency 55% · infra 24% · ... It earns its keep on choice and score , where the runner-up is real information — an even four-way split is a different situation from a clear winner at the same confidence. On a noul the two halves restate one number; they are shown anyway so every row reads the same way. JSON, NDJSON, and CSV output are unaffected by the flag. jcli analyze app.ndjson -p logs.triage --mark-uncertain 0.7 keep weak answers, flagged jcli analyze app.ndjson -p logs.triage --confident 0.7 drop them The threshold is part of the flag, so there is no way to ask for dropping without saying how confident is confident enough. TypeSafe's guidance https://docs.typesafe.ai/confidence : act automatically above ~0.9 for high-stakes decisions, never act below 0.5. --output json|ndjson|table|csv|summary . Defaults to table on a TTY, json in a pipe. { "source": "auth.ndjson", "span": {"source": "auth.ndjson", "indices": 0, 4 , "count": 5, "lines": 1, 5 }, "pack": "security.audit", "question": "threat class", "type": "choice", "value": "privilege escalation", "confidence": 0.93, "probabilities": {"privilege escalation": 0.93, "recon": 0.05, "benign": 0.02}, "below threshold": false } jcli analyze audit.ndjson -p security.audit \ --fail-on 'suspicious =0.8 and blast radius =1.5' echo $? 6 Grammar is QID OP VALUE joined by and / or . A term matches if any window satisfies it. | Exit | Meaning | |---|---| | 0 | success | | 1 | unexpected error | | 2 | usage or pack validation error | | 3 | auth failure 401/403 | | 4 | rate limited past the retry budget | | 5 | partial failure — some windows failed, results still emitted | | 6 | --fail-on matched | Exit 5 is deliberate: a 500 on window 37 of 200 does not throw away 199 good answers. A 401 does abort immediately, because it would fail identically on all 200. Resolved in order: --api-key → --api-key-file → $TYPESAFE API KEY → ~/.config/jcli/config.toml → keyring with the keyring extra . The token is wrapped so it cannot be printed by accident — repr , f-strings, and dataclass reprs all render . A config or key file that is group- or world-readable is refused , not warned about. jcli config show names which source supplied the key without revealing it. bash $ jcli config show config file ~/.config/jcli/config.toml base url https://api.typesafe.ai model jev-latest api key found via $TYPESAFE API KEY Repeating the same pack and transform on every run gets old. ~/.config/jcli/config.toml absorbs it: api key = "..." defaults pack = "agent.session" transform = "syslog" explicit -t still wins; auto-detect is the fallback window = 25 output = "table" An explicit flag always beats a config default, which always beats the built-in default. jcli config prints what resolved, and names any key it ignored. uv run pytest 119 tests, no network uv run pytest -m live smoke tests against the real API needs TYPESAFE API KEY, costs money uv run ruff check . uv run mypy --dry-run prints the exact request bodies without a key and without touching the network — the fastest way to see what a flag combination actually does. jcli analyze audit.ndjson -p security.audit --dry-run | jq '. 0 .request.state'