Show HN: CLI for scoring OpenAPI for LLM legibility

Jentic released a command-line tool that scores OpenAPI documents for AI agent legibility, grading them across six dimensions including compliance, security, and discoverability. The Jentic API Scorecard runs locally in Docker and produces a single weighted grade to help developers identify where their API specifications need improvement for both human and AI consumption.

An OpenAPI document that passes validation isn't necessarily one an AI agent can use. Grammar is one thing; semantic clarity, safety, and discoverability are another. The Jentic API Scorecard scores your OpenAPI document against the Jentic API AI Readiness Framework JAIRF https://github.com/jentic/api-ai-readiness-framework across six dimensions and returns a single grade — so you know exactly where to improve. Each OpenAPI document is evaluated across six lenses — small, targeted improvements in any of them tend to produce outsized gains for both human developers and AI agents: Foundational Compliance FC — structural validity and conformance to OpenAPI itself. Developer Experience & Jentic Compatibility DXJ — documentation quality and how well the OpenAPI document plays with downstream tooling. AI-Readiness & Agent Experience ARAX — semantic clarity and the context an LLM needs to reason about each operation. Agent Usability AU — predictable, safe multi-step orchestration. Security SEC — declared auth schemes and trust boundaries. AI Discoverability AID — how easily an AI system can find and parse the OpenAPI document. Scoring runs locally inside a Docker container in two phases. Analysis runs a battery of validators and structural checks against the OpenAPI document to produce a set of diagnostics and observations. Scoring maps those into ~35 signals across the six JAIRF dimensions, aggregates them into per-dimension scores, and rolls those up into a single weighted score and grade. Node.js 20 LTS or newer = 20.19.0 with npm/npx. See Node.js downloads https://nodejs.org/ . Docker installed and running. See Docker installation https://docs.docker.com/get-docker/ . The CLI pulls the scoring image automatically on first run.- Network access to to pull the image and to whatever URL hosts the OpenAPI document you're scoring the engine fetches it from inside the container . ghcr.io npm install -g @jentic/api-scorecard-cli This installs the CLI globally. The scoring engine Docker image is downloaded automatically the first time you run score — allow a minute or two on a typical connection. For local files or non-OAK URLs you'll also need a JENTIC API KEY — see Anonymous vs keyed access anonymous-vs-keyed-access . Verify the install: jentic-api-scorecard --version Prefer zero-install?You can skip the global install and use npx — every example in this README works with npx @jentic/api-scorecard-cli in place of jentic-api-scorecard . Pin to a specific release with npx @jentic/api-scorecard-cli@<version e.g. @1.0.0 ; the unpinned form resolves to whatever the latest dist-tag points at on each invocation, while npm install -g pins you to the installed version until you explicitly update. OpenAPI documents from Jentic Public APIs OAK https://github.com/jentic/jentic-public-apis score without any key, uncapped — no signup, no config: npx @jentic/api-scorecard-cli@latest score \ https://raw.githubusercontent.com/jentic/jentic-public-apis/refs/heads/main/apis/openapi/swagger-api/petstore/1.0.27/openapi.json For URLs outside OAK or local files, set the API key: JENTIC API KEY=<your-key npx @jentic/api-scorecard-cli@latest score \ https://petstore3.swagger.io/api/v3/openapi.json JENTIC API KEY=<your-key npx @jentic/api-scorecard-cli@latest score ./openapi.yaml Important Free keys come with 100 scorings per month resets at the start of each calendar month . See Anonymous vs keyed access anonymous-vs-keyed-access for signup and quota details. That's it. The CLI pulls the scoring engine automatically on first run. The --detail flag lets you zoom in: Just the headline score and grade npx @jentic/api-scorecard-cli@latest score --detail summary ./openapi.yaml Per-dimension breakdown default npx @jentic/api-scorecard-cli@latest score --detail dimensions ./openapi.yaml Individual signals within each dimension npx @jentic/api-scorecard-cli@latest score --detail signals ./openapi.yaml Full diagnostics with top 5 findings per severity npx @jentic/api-scorecard-cli@latest score --detail diagnostics ./openapi.yaml Add --format json to emit engine-verbatim JSON on stdout filtered by whatever --detail level you pick . Pretty stays the unconditional default; --format json is the canonical way to get a stable machine-readable channel for CI gating, archival, or LLM-assisted review. Gate on the headline score in CI npx @jentic/api-scorecard-cli@latest score ./openapi.yaml --format json | jq .summary.score Capture the full evidence bundle to a file npx @jentic/api-scorecard-cli@latest score ./openapi.yaml \ --format json --detail diagnostics --output report.json --output <file -o writes the report to a path instead of stdout; the spinner stays on stderr. --quiet -q suppresses the stderr spinner even in interactive terminals the spinner already auto-suppresses when stderr isn't a TTY . Engine warnings still pass through stderr. Add --with-llm to unlock LLM-backed signals — deeper semantic reasoning about whether your API descriptions are actionable for agents, whether error responses support autonomous recovery, and more. Requires an LLM provider: cloud OpenAI / Anthropic / Gemini / AWS Bedrock or a local OpenAI-compatible endpoint Ollama, LM Studio, vLLM, … . export OPENAI API KEY=sk-... export LLM PROVIDER=OPENAI export LIGHT LLM PROVIDER=OPENAI export LLM LIGHT MODEL=gpt-4o-mini JENTIC API KEY=<your-key npx @jentic/api-scorecard-cli@latest score ./openapi.yaml --with-llm Token cost is low — the engine uses a lightweight model e.g. Claude Haiku, GPT-4o-mini , processes operations in small batches, and caps at 7 batches regardless of spec size. Local models Ollama cost nothing per call. See LLM Signals guide for all provider recipes including local Ollama , the full environment variable reference, and troubleshooting. OpenAPI documents from Jentic Public APIs OAK https://github.com/jentic/jentic-public-apis score without any key and stay on the free tier — those URLs bypass key validation entirely. For everything else local files, URLs outside OAK , get a key at jentic.com/signup https://jentic.com/signup — once signed in, click Score → CLI & Keys to issue your key. Then set it: export JENTIC API KEY=<your-key Real keys are validated live by the container against api.jentic.com . The same call doubles as the per-key usage / rate-limit accounting hit. Each free key gets 100 scorings per month , resetting at the start of each calendar month. Once that quota is exhausted the CLI exits with code 7 and prints the Retry-After value along with a link to upgrade your plan. jentic-api-scorecard -V | --version -h | --help jentic-api-scorecard <command options | Command | Description | |---|---| score <input | Score an OpenAPI document by URL or local file path. jentic-api-scorecard score <input options | Name | Description | |---|---| <input | https:// URL or local file path to an OpenAPI document. Required. | | Flag | Default | Choices | Description | |---|---|---|---| --with-llm | off | — | Enable LLM-backed analysis. Requires an LLM provider see | --bundle JENTIC API KEY . No-op for local files. -d, --detail <level dimensions summary , dimensions , signals , diagnostics Control output depth control-output-depth . -f, --format <fmt pretty pretty , json Machine-readable output machine-readable-output . -o, --output <file <file . The spinner stays on stderr. -q, --quiet -h, --help score .| Variable | When | Purpose | |---|---|---| JENTIC API KEY | URLs outside OAK and local files | Real key issued at api.jentic.com see Free quota: 100 scorings per calendar month. | --with-llm OPENAI API KEY , ANTHROPIC API KEY , GEMINI API KEY , AWS keys and routing LLM PROVIDER , LIGHT LLM PROVIDER , LLM MODEL , LLM LIGHT MODEL , API URL , LLM MAX TOKENS and forwards them to the container; loopback URLs are rewritten so a host-side Ollama is reachable. Full reference: LLM Signals guide https://github.com/jentic/jentic-api-scorecard/blob/main/docs/llm-signals.md .| Code | Meaning | |---|---| | 0 | Scoring completed regardless of the score itself . | | 1 | Generic error bad input, bundling failure, unexpected container failure . | | 2 | Auth: JENTIC API KEY is set to a value the Jentic backend does not recognize, or a local file / stdin input was used without the key set. | | 3 | Anonymous gate rejected: URL outside the OAK allowlist and no key set. | | 4 | Docker not installed or daemon unreachable. | | 5 | Spec fetch or parse failure. | | 6 | Engine invocation failure. | | 7 | Rate limit reached: the key is valid but the user is over quota. Message includes the server-provided detail and the Retry-After header when present. | jentic.com/scorecard https://jentic.com/scorecard offers the same scoring in a web UI — paste a URL or drop a file, no Docker or Node required. For teams that need to know exactly what's running, verify exactly what was shipped, and run without a runtime dependency on Jentic. Every component in the scoring stack — runner, CLI, release pipeline, and engine — is Apache 2.0 licensed and source-readable. No proprietary blobs, no closed-source shims. Read the code that's about to grade your specs before you adopt it; audit any line, redistribute under the license terms, fork if you ever need to. Every npm tarball and every GHCR image is signed by Sigstore https://www.sigstore.dev/ with SLSA provenance and an SPDX SBOM. Signing happens inside an OIDC-driven GitHub Actions workflow with no long-lived publishing secrets — there is no NPM TOKEN , no PAT, and no human keyholder in the release chain. One command verifies an artifact end-to-end before you install it: — npm provenance, SPDX SBOM, trusted publishing, and the npm package supply chain → https://github.com/jentic/jentic-api-scorecard/blob/main/docs/supply-chain-npm.md gh attestation verify recipes.— per-platform SBOMs, dual-store attestations BuildKit OCI referrers + Sigstore , and verification via either Docker image supply chain → https://github.com/jentic/jentic-api-scorecard/blob/main/docs/supply-chain-docker.md docker buildx imagetools inspect or gh attestation verify . The image is a closed system at scoring time: every Python wheel, Node.js binary, and validator tarball it needs is baked in at build time. Scoring does not call PyPI or npmjs and pulls no runtime packages. The only outbound call to Jentic is a small key-check round-trip against api.jentic.com that authenticates your key and increments the per-key usage counter; OAK URLs jentic-public-apis skip even that. URL inputs additionally reach the network to fetch the OpenAPI document and resolve any external $ref s it points at. --with-llm optionally sends spec context to an LLM provider of your choice; a local endpoint Ollama keeps everything on-machine. Multi-arch images linux/amd64 + linux/arm64 ship from the same release, so the same guarantees hold on Apple Silicon dev machines, ARM CI runners, and x86 servers alike. CLI version, image tag, and engine version are locked one-to-one. Pinning @jentic/api-scorecard-cli@<version resolves to a specific image tag, which in turn pins an exact engine release and exact validator versions. Last month's score is reproducible from last month's pin. The CLI ships stable under the latest npm dist-tag — release cadence is driven by Conventional Commits https://www.conventionalcommits.org/ . Track in-flight work in specs/roadmap.md https://github.com/jentic/jentic-api-scorecard/blob/main/specs/roadmap.md . The :unstable Docker image is rebuilt on every push to main for direct docker run users. Versioned images are published alongside each CLI release. To see which Jentic API AI Readiness Framework signals are active in the current release, check out the scoring engine implementation status https://docs.jentic.com/reference/api-readiness-framework/scoring-engine-status/ . Jentic API Scorecard is licensed under the Apache 2.0 https://github.com/jentic/jentic-api-scorecard/blob/main/LICENSE license. Jentic API Scorecard comes with an explicit NOTICE https://github.com/jentic/jentic-api-scorecard/blob/main/NOTICE file containing additional legal notices and information.