# Show HN: CLI for scoring OpenAPI for LLM legibility > Source: > Published: 2026-06-05 08:35:34+00:00 An OpenAPI document that passes validation isn't necessarily one an AI agent can use. Grammar is one thing; semantic clarity, safety, and discoverability are another. The **Jentic API Scorecard** scores your OpenAPI document against the [Jentic API AI Readiness Framework (JAIRF)](https://github.com/jentic/api-ai-readiness-framework) across six dimensions and returns a single grade — so you know exactly where to improve. Each OpenAPI document is evaluated across six lenses — small, targeted improvements in any of them tend to produce outsized gains for both human developers and AI agents: **Foundational Compliance (FC)**— structural validity and conformance to OpenAPI itself.** Developer Experience & Jentic Compatibility (DXJ)**— documentation quality and how well the OpenAPI document plays with downstream tooling.** AI-Readiness & Agent Experience (ARAX)**— semantic clarity and the context an LLM needs to reason about each operation.** Agent Usability (AU)**— predictable, safe multi-step orchestration.** Security (SEC)**— declared auth schemes and trust boundaries.** AI Discoverability (AID)**— how easily an AI system can find and parse the OpenAPI document. Scoring runs locally inside a Docker container in two phases. **Analysis** runs a battery of validators and structural checks against the OpenAPI document to produce a set of diagnostics and observations. **Scoring** maps those into ~35 signals across the six JAIRF dimensions, aggregates them into per-dimension scores, and rolls those up into a single weighted score and grade. **Node.js** 20 LTS or newer (`>= 20.19.0` ) with npm/npx. See[Node.js downloads](https://nodejs.org/).**Docker** installed and running. See[Docker installation](https://docs.docker.com/get-docker/). The CLI pulls the scoring image automatically on first run.- Network access to (to pull the image) and to whatever URL hosts the OpenAPI document you're scoring (the engine fetches it from inside the container).`ghcr.io` ``` npm install -g @jentic/api-scorecard-cli ``` This installs the CLI globally. The scoring engine (Docker image) is downloaded automatically the first time you run `score` — allow a minute or two on a typical connection. For local files or non-OAK URLs you'll also need a `JENTIC_API_KEY` — see [Anonymous vs keyed access](#anonymous-vs-keyed-access). Verify the install: ``` jentic-api-scorecard --version ``` Prefer zero-install?You can skip the global install and use`npx` — every example in this README works with`npx @jentic/api-scorecard-cli` in place of`jentic-api-scorecard` . Pin to a specific release with`npx @jentic/api-scorecard-cli@` (e.g.`@1.0.0` ); the unpinned form resolves to whatever the`latest` dist-tag points at on each invocation, while`npm install -g` pins you to the installed version until you explicitly update. OpenAPI documents from [Jentic Public APIs (OAK)](https://github.com/jentic/jentic-public-apis) score without any key, uncapped — no signup, no config: ``` npx @jentic/api-scorecard-cli@latest score \ https://raw.githubusercontent.com/jentic/jentic-public-apis/refs/heads/main/apis/openapi/swagger-api/petstore/1.0.27/openapi.json ``` For URLs outside OAK or local files, set the API key: ``` JENTIC_API_KEY= npx @jentic/api-scorecard-cli@latest score \ https://petstore3.swagger.io/api/v3/openapi.json JENTIC_API_KEY= npx @jentic/api-scorecard-cli@latest score ./openapi.yaml ``` Important Free keys come with **100 scorings per month** (resets at the start of each calendar month). See [Anonymous vs keyed access](#anonymous-vs-keyed-access) for signup and quota details. That's it. The CLI pulls the scoring engine automatically on first run. The `--detail` flag lets you zoom in: ``` # Just the headline score and grade npx @jentic/api-scorecard-cli@latest score --detail summary ./openapi.yaml # Per-dimension breakdown (default) npx @jentic/api-scorecard-cli@latest score --detail dimensions ./openapi.yaml # Individual signals within each dimension npx @jentic/api-scorecard-cli@latest score --detail signals ./openapi.yaml # Full diagnostics with top 5 findings per severity npx @jentic/api-scorecard-cli@latest score --detail diagnostics ./openapi.yaml ``` Add `--format json` to emit engine-verbatim JSON on stdout (filtered by whatever `--detail` level you pick). Pretty stays the unconditional default; `--format json` is the canonical way to get a stable machine-readable channel for CI gating, archival, or LLM-assisted review. ``` # Gate on the headline score in CI npx @jentic/api-scorecard-cli@latest score ./openapi.yaml --format json | jq .summary.score # Capture the full evidence bundle to a file npx @jentic/api-scorecard-cli@latest score ./openapi.yaml \ --format json --detail diagnostics --output report.json ``` `--output ` (`-o` ) writes the report to a path instead of stdout; the spinner stays on stderr. `--quiet` (`-q` ) suppresses the stderr spinner even in interactive terminals (the spinner already auto-suppresses when stderr isn't a TTY). Engine warnings still pass through stderr. Add `--with-llm` to unlock LLM-backed signals — deeper semantic reasoning about whether your API descriptions are actionable for agents, whether error responses support autonomous recovery, and more. Requires an LLM provider: cloud (OpenAI / Anthropic / Gemini / AWS Bedrock) or a local OpenAI-compatible endpoint (Ollama, LM Studio, vLLM, …). ``` export OPENAI_API_KEY=sk-... export LLM_PROVIDER=OPENAI export LIGHT_LLM_PROVIDER=OPENAI export LLM_LIGHT_MODEL=gpt-4o-mini JENTIC_API_KEY= npx @jentic/api-scorecard-cli@latest score ./openapi.yaml --with-llm ``` Token cost is low — the engine uses a lightweight model (e.g. Claude Haiku, GPT-4o-mini), processes operations in small batches, and caps at 7 batches regardless of spec size. Local models (Ollama) cost nothing per call. See ** LLM Signals guide** for all provider recipes (including local Ollama), the full environment variable reference, and troubleshooting. OpenAPI documents from [Jentic Public APIs (OAK)](https://github.com/jentic/jentic-public-apis) score without any key and stay on the free tier — those URLs bypass key validation entirely. For everything else (local files, URLs outside OAK), get a key at [jentic.com/signup](https://jentic.com/signup) — once signed in, click **Score → CLI & Keys** to issue your key. Then set it: ``` export JENTIC_API_KEY= ``` Real keys are validated live by the container against `api.jentic.com` . The same call doubles as the per-key usage / rate-limit accounting hit. **Each free key gets 100 scorings per month**, resetting at the start of each calendar month. Once that quota is exhausted the CLI exits with code `7` and prints the `Retry-After` value along with a link to upgrade your plan. ``` jentic-api-scorecard [-V | --version] [-h | --help] jentic-api-scorecard [options] ``` | Command | Description | |---|---| `score ` | Score an OpenAPI document by URL or local file path. ``` jentic-api-scorecard score [options] ``` | Name | Description | |---|---| `` | `https://` URL or local file path to an OpenAPI document. Required. | | Flag | Default | Choices | Description | |---|---|---|---| `--with-llm` | off | — | Enable LLM-backed analysis. Requires an LLM provider (see | `--bundle` `JENTIC_API_KEY` . No-op for local files.`-d, --detail ` `dimensions` `summary` , `dimensions` , `signals` , `diagnostics` [Control output depth](#control-output-depth)).`-f, --format ` `pretty` `pretty` , `json` [Machine-readable output](#machine-readable-output)).`-o, --output ` `` . The spinner stays on stderr.`-q, --quiet` `-h, --help` `score` .| Variable | When | Purpose | |---|---|---| `JENTIC_API_KEY` | URLs outside OAK and local files | Real key issued at `api.jentic.com` (see Free quota: 100 scorings per calendar month. | `--with-llm` `OPENAI_API_KEY` , `ANTHROPIC_API_KEY` , `GEMINI_API_KEY` , AWS keys) and routing (`LLM_PROVIDER` , `LIGHT_LLM_PROVIDER` , `LLM_MODEL` , `LLM_LIGHT_MODEL` , `*_API_URL` , `LLM_MAX_TOKENS` ) and forwards them to the container; loopback URLs are rewritten so a host-side Ollama is reachable. Full reference: [LLM Signals guide](https://github.com/jentic/jentic-api-scorecard/blob/main/docs/llm-signals.md).| Code | Meaning | |---|---| | 0 | Scoring completed (regardless of the score itself). | | 1 | Generic error (bad input, bundling failure, unexpected container failure). | | 2 | Auth: `JENTIC_API_KEY` is set to a value the Jentic backend does not recognize, or a local file / stdin input was used without the key set. | | 3 | Anonymous gate rejected: URL outside the OAK allowlist and no key set. | | 4 | Docker not installed or daemon unreachable. | | 5 | Spec fetch or parse failure. | | 6 | Engine invocation failure. | | 7 | Rate limit reached: the key is valid but the user is over quota. Message includes the server-provided `detail` and the `Retry-After` header when present. | [ jentic.com/scorecard](https://jentic.com/scorecard) offers the same scoring in a web UI — paste a URL or drop a file, no Docker or Node required. For teams that need to know exactly what's running, verify exactly what was shipped, and run without a runtime dependency on Jentic. Every component in the scoring stack — runner, CLI, release pipeline, and engine — is Apache 2.0 licensed and source-readable. No proprietary blobs, no closed-source shims. Read the code that's about to grade your specs before you adopt it; audit any line, redistribute under the license terms, fork if you ever need to. Every npm tarball and every GHCR image is signed by [Sigstore](https://www.sigstore.dev/) with SLSA provenance and an SPDX SBOM. Signing happens inside an OIDC-driven GitHub Actions workflow with no long-lived publishing secrets — there is no `NPM_TOKEN` , no PAT, and no human keyholder in the release chain. One command verifies an artifact end-to-end before you install it: — npm provenance, SPDX SBOM, trusted publishing, and the[npm package supply chain →](https://github.com/jentic/jentic-api-scorecard/blob/main/docs/supply-chain-npm.md)`gh attestation verify` recipes.— per-platform SBOMs, dual-store attestations (BuildKit OCI referrers + Sigstore), and verification via either[Docker image supply chain →](https://github.com/jentic/jentic-api-scorecard/blob/main/docs/supply-chain-docker.md)`docker buildx imagetools inspect` or`gh attestation verify` . The image is a closed system at scoring time: every Python wheel, Node.js binary, and validator tarball it needs is baked in at build time. Scoring does not call PyPI or npmjs and pulls no runtime packages. The **only** outbound call to Jentic is a small key-check round-trip against `api.jentic.com` that authenticates your key and increments the per-key usage counter; OAK URLs (jentic-public-apis) skip even that. URL inputs additionally reach the network to fetch the OpenAPI document and resolve any external `$ref` s it points at. `--with-llm` optionally sends spec context to an LLM provider of your choice; a local endpoint (Ollama) keeps everything on-machine. Multi-arch images (linux/amd64 + linux/arm64) ship from the same release, so the same guarantees hold on Apple Silicon dev machines, ARM CI runners, and x86 servers alike. CLI version, image tag, and engine version are locked one-to-one. Pinning `@jentic/api-scorecard-cli@` resolves to a specific image tag, which in turn pins an exact engine release and exact validator versions. Last month's score is reproducible from last month's pin. The CLI ships **stable** under the `latest` npm dist-tag — release cadence is driven by [Conventional Commits](https://www.conventionalcommits.org/). Track in-flight work in [ specs/roadmap.md](https://github.com/jentic/jentic-api-scorecard/blob/main/specs/roadmap.md). The `:unstable` Docker image is rebuilt on every push to `main` for direct `docker run` users. Versioned images are published alongside each CLI release. To see which Jentic API AI Readiness Framework signals are active in the current release, check out the [scoring engine implementation status](https://docs.jentic.com/reference/api-readiness-framework/scoring-engine-status/). Jentic API Scorecard is licensed under the [Apache 2.0](https://github.com/jentic/jentic-api-scorecard/blob/main/LICENSE) license. Jentic API Scorecard comes with an explicit [NOTICE](https://github.com/jentic/jentic-api-scorecard/blob/main/NOTICE) file containing additional legal notices and information.