{"slug": "show-hn-cli-for-scoring-openapi-for-llm-legibility", "title": "Show HN: CLI for scoring OpenAPI for LLM legibility", "summary": "Jentic released a command-line tool that scores OpenAPI documents for AI agent legibility, grading them across six dimensions including compliance, security, and discoverability. The Jentic API Scorecard runs locally in Docker and produces a single weighted grade to help developers identify where their API specifications need improvement for both human and AI consumption.", "body_md": "An OpenAPI document that passes validation isn't necessarily one an AI agent can use. Grammar is\none thing; semantic clarity, safety, and discoverability are another. The **Jentic API Scorecard**\nscores your OpenAPI document against the\n[Jentic API AI Readiness Framework (JAIRF)](https://github.com/jentic/api-ai-readiness-framework)\nacross six dimensions and returns a single grade — so you know exactly where to improve.\n\nEach OpenAPI document is evaluated across six lenses — small, targeted improvements in any of them tend to produce outsized gains for both human developers and AI agents:\n\n**Foundational Compliance (FC)**— structural validity and conformance to OpenAPI itself.** Developer Experience & Jentic Compatibility (DXJ)**— documentation quality and how well the OpenAPI document plays with downstream tooling.** AI-Readiness & Agent Experience (ARAX)**— semantic clarity and the context an LLM needs to reason about each operation.** Agent Usability (AU)**— predictable, safe multi-step orchestration.** Security (SEC)**— declared auth schemes and trust boundaries.** AI Discoverability (AID)**— how easily an AI system can find and parse the OpenAPI document.\n\nScoring runs locally inside a Docker container in two phases. **Analysis** runs a battery of\nvalidators and structural checks against the OpenAPI document to produce a set of diagnostics and\nobservations.\n**Scoring** maps those into ~35 signals across the six JAIRF dimensions, aggregates them into\nper-dimension scores, and rolls those up into a single weighted score and grade.\n\n**Node.js** 20 LTS or newer (`>= 20.19.0`\n\n) with npm/npx. See[Node.js downloads](https://nodejs.org/).**Docker** installed and running. See[Docker installation](https://docs.docker.com/get-docker/). The CLI pulls the scoring image automatically on first run.- Network access to\n(to pull the image) and to whatever URL hosts the OpenAPI document you're scoring (the engine fetches it from inside the container).`ghcr.io`\n\n```\nnpm install -g @jentic/api-scorecard-cli\n```\n\nThis installs the CLI globally. The scoring engine (Docker image) is downloaded automatically\nthe first time you run `score`\n\n— allow a minute or two on a typical connection.\n\nFor local files or non-OAK URLs you'll also need a `JENTIC_API_KEY`\n\n— see\n[Anonymous vs keyed access](#anonymous-vs-keyed-access).\n\nVerify the install:\n\n```\njentic-api-scorecard --version\n```\n\nPrefer zero-install?You can skip the global install and use`npx`\n\n— every example in this README works with`npx @jentic/api-scorecard-cli`\n\nin place of`jentic-api-scorecard`\n\n. Pin to a specific release with`npx @jentic/api-scorecard-cli@<version>`\n\n(e.g.`@1.0.0`\n\n); the unpinned form resolves to whatever the`latest`\n\ndist-tag points at on each invocation, while`npm install -g`\n\npins you to the installed version until you explicitly update.\n\nOpenAPI documents from [Jentic Public APIs (OAK)](https://github.com/jentic/jentic-public-apis)\nscore without any key, uncapped — no signup, no config:\n\n```\nnpx @jentic/api-scorecard-cli@latest score \\\n  https://raw.githubusercontent.com/jentic/jentic-public-apis/refs/heads/main/apis/openapi/swagger-api/petstore/1.0.27/openapi.json\n```\n\nFor URLs outside OAK or local files, set the API key:\n\n```\nJENTIC_API_KEY=<your-key> npx @jentic/api-scorecard-cli@latest score \\\n  https://petstore3.swagger.io/api/v3/openapi.json\nJENTIC_API_KEY=<your-key> npx @jentic/api-scorecard-cli@latest score ./openapi.yaml\n```\n\nImportant\n\nFree keys come with **100 scorings per month** (resets at the start of each calendar month). See [Anonymous vs keyed access](#anonymous-vs-keyed-access) for signup and quota details.\n\nThat's it. The CLI pulls the scoring engine automatically on first run.\n\nThe `--detail`\n\nflag lets you zoom in:\n\n```\n# Just the headline score and grade\nnpx @jentic/api-scorecard-cli@latest score --detail summary ./openapi.yaml\n\n# Per-dimension breakdown (default)\nnpx @jentic/api-scorecard-cli@latest score --detail dimensions ./openapi.yaml\n\n# Individual signals within each dimension\nnpx @jentic/api-scorecard-cli@latest score --detail signals ./openapi.yaml\n\n# Full diagnostics with top 5 findings per severity\nnpx @jentic/api-scorecard-cli@latest score --detail diagnostics ./openapi.yaml\n```\n\nAdd `--format json`\n\nto emit engine-verbatim JSON on stdout (filtered by whatever\n`--detail`\n\nlevel you pick). Pretty stays the unconditional default; `--format json`\n\nis the canonical way to get a stable machine-readable channel for CI gating, archival,\nor LLM-assisted review.\n\n```\n# Gate on the headline score in CI\nnpx @jentic/api-scorecard-cli@latest score ./openapi.yaml --format json | jq .summary.score\n\n# Capture the full evidence bundle to a file\nnpx @jentic/api-scorecard-cli@latest score ./openapi.yaml \\\n  --format json --detail diagnostics --output report.json\n```\n\n`--output <file>`\n\n(`-o`\n\n) writes the report to a path instead of stdout; the spinner stays on stderr.\n\n`--quiet`\n\n(`-q`\n\n) suppresses the stderr spinner even in interactive terminals (the spinner already\nauto-suppresses when stderr isn't a TTY). Engine warnings still pass through stderr.\n\nAdd `--with-llm`\n\nto unlock LLM-backed signals — deeper semantic reasoning about whether your API\ndescriptions are actionable for agents, whether error responses support autonomous recovery, and\nmore. Requires an LLM provider: cloud (OpenAI / Anthropic / Gemini / AWS Bedrock) or a local\nOpenAI-compatible endpoint (Ollama, LM Studio, vLLM, …).\n\n```\nexport OPENAI_API_KEY=sk-...\nexport LLM_PROVIDER=OPENAI\nexport LIGHT_LLM_PROVIDER=OPENAI\nexport LLM_LIGHT_MODEL=gpt-4o-mini\n\nJENTIC_API_KEY=<your-key> npx @jentic/api-scorecard-cli@latest score ./openapi.yaml --with-llm\n```\n\nToken cost is low — the engine uses a lightweight model (e.g. Claude Haiku, GPT-4o-mini), processes operations in small batches, and caps at 7 batches regardless of spec size. Local models (Ollama) cost nothing per call.\n\nSee ** LLM Signals guide**\nfor all provider recipes (including local Ollama), the full environment variable reference, and\ntroubleshooting.\n\nOpenAPI documents from [Jentic Public APIs (OAK)](https://github.com/jentic/jentic-public-apis)\nscore without any key and stay on the free tier — those URLs bypass key validation entirely.\nFor everything else (local files, URLs outside OAK), get a key at [jentic.com/signup](https://jentic.com/signup) — once signed in, click **Score → CLI & Keys** to issue your key. Then set it:\n\n```\nexport JENTIC_API_KEY=<your-key>\n```\n\nReal keys are validated live by the container against `api.jentic.com`\n\n. The same call doubles\nas the per-key usage / rate-limit accounting hit. **Each free key gets 100 scorings per month**,\nresetting at the start of each calendar month. Once that quota is exhausted the CLI exits with\ncode `7`\n\nand prints the `Retry-After`\n\nvalue along with a link to upgrade your plan.\n\n```\njentic-api-scorecard [-V | --version] [-h | --help]\njentic-api-scorecard <command> [options]\n```\n\n| Command | Description |\n|---|---|\n`score <input>` |\n\nScore an OpenAPI document by URL or local file path.\n\n```\njentic-api-scorecard score <input> [options]\n```\n\n| Name | Description |\n|---|---|\n`<input>` |\n`https://` URL or local file path to an OpenAPI document. Required. |\n\n| Flag | Default | Choices | Description |\n|---|---|---|---|\n`--with-llm` |\noff | — | Enable LLM-backed analysis. Requires an LLM provider (see\n|\n\n`--bundle`\n\n`JENTIC_API_KEY`\n\n. No-op for local files.`-d, --detail <level>`\n\n`dimensions`\n\n`summary`\n\n, `dimensions`\n\n, `signals`\n\n, `diagnostics`\n\n[Control output depth](#control-output-depth)).`-f, --format <fmt>`\n\n`pretty`\n\n`pretty`\n\n, `json`\n\n[Machine-readable output](#machine-readable-output)).`-o, --output <file>`\n\n`<file>`\n\n. The spinner stays on stderr.`-q, --quiet`\n\n`-h, --help`\n\n`score`\n\n.| Variable | When | Purpose |\n|---|---|---|\n`JENTIC_API_KEY` |\nURLs outside OAK and local files | Real key issued at\n`api.jentic.com` (see\nFree quota: 100 scorings per calendar month. |\n\n`--with-llm`\n\n`OPENAI_API_KEY`\n\n, `ANTHROPIC_API_KEY`\n\n, `GEMINI_API_KEY`\n\n, AWS keys) and routing (`LLM_PROVIDER`\n\n, `LIGHT_LLM_PROVIDER`\n\n, `LLM_MODEL`\n\n, `LLM_LIGHT_MODEL`\n\n, `*_API_URL`\n\n, `LLM_MAX_TOKENS`\n\n) and forwards them to the container; loopback URLs are rewritten so a host-side Ollama is reachable. Full reference: [LLM Signals guide](https://github.com/jentic/jentic-api-scorecard/blob/main/docs/llm-signals.md).| Code | Meaning |\n|---|---|\n| 0 | Scoring completed (regardless of the score itself). |\n| 1 | Generic error (bad input, bundling failure, unexpected container failure). |\n| 2 | Auth: `JENTIC_API_KEY` is set to a value the Jentic backend does not recognize, or a local file / stdin input was used without the key set. |\n| 3 | Anonymous gate rejected: URL outside the OAK allowlist and no key set. |\n| 4 | Docker not installed or daemon unreachable. |\n| 5 | Spec fetch or parse failure. |\n| 6 | Engine invocation failure. |\n| 7 | Rate limit reached: the key is valid but the user is over quota. Message includes the server-provided `detail` and the `Retry-After` header when present. |\n\n[ jentic.com/scorecard](https://jentic.com/scorecard) offers the same scoring in a web UI —\npaste a URL or drop a file, no Docker or Node required.\n\nFor teams that need to know exactly what's running, verify exactly what was shipped, and run without a runtime dependency on Jentic.\n\nEvery component in the scoring stack — runner, CLI, release pipeline, and engine — is Apache 2.0 licensed and source-readable. No proprietary blobs, no closed-source shims. Read the code that's about to grade your specs before you adopt it; audit any line, redistribute under the license terms, fork if you ever need to.\n\nEvery npm tarball and every GHCR image is signed by [Sigstore](https://www.sigstore.dev/)\nwith SLSA provenance and an SPDX SBOM. Signing happens inside an OIDC-driven\nGitHub Actions workflow with no long-lived publishing secrets — there is no\n`NPM_TOKEN`\n\n, no PAT, and no human keyholder in the release chain. One command\nverifies an artifact end-to-end before you install it:\n\n— npm provenance, SPDX SBOM, trusted publishing, and the[npm package supply chain →](https://github.com/jentic/jentic-api-scorecard/blob/main/docs/supply-chain-npm.md)`gh attestation verify`\n\nrecipes.— per-platform SBOMs, dual-store attestations (BuildKit OCI referrers + Sigstore), and verification via either[Docker image supply chain →](https://github.com/jentic/jentic-api-scorecard/blob/main/docs/supply-chain-docker.md)`docker buildx imagetools inspect`\n\nor`gh attestation verify`\n\n.\n\nThe image is a closed system at scoring time: every Python wheel, Node.js\nbinary, and validator tarball it needs is baked in at build time. Scoring does\nnot call PyPI or npmjs and pulls no runtime packages. The **only** outbound\ncall to Jentic is a small key-check round-trip against `api.jentic.com`\n\nthat\nauthenticates your key and increments the per-key usage counter; OAK URLs\n(jentic-public-apis) skip even that. URL inputs additionally reach the network to fetch the OpenAPI\ndocument and resolve any external `$ref`\n\ns it points at. `--with-llm`\n\noptionally sends spec context to an LLM provider of your choice; a local\nendpoint (Ollama) keeps everything on-machine. Multi-arch images\n(linux/amd64 + linux/arm64) ship from the same release, so the same guarantees\nhold on Apple Silicon dev machines, ARM CI runners, and x86 servers alike.\n\nCLI version, image tag, and engine version are locked one-to-one. Pinning\n`@jentic/api-scorecard-cli@<version>`\n\nresolves to a specific image tag, which\nin turn pins an exact engine release and exact validator versions. Last\nmonth's score is reproducible from last month's pin.\n\nThe CLI ships **stable** under the `latest`\n\nnpm dist-tag — release cadence is driven by\n[Conventional Commits](https://www.conventionalcommits.org/). Track in-flight work in\n[ specs/roadmap.md](https://github.com/jentic/jentic-api-scorecard/blob/main/specs/roadmap.md).\n\nThe `:unstable`\n\nDocker image is rebuilt on every push to `main`\n\nfor direct `docker run`\n\nusers.\nVersioned images are published alongside each CLI release.\n\nTo see which Jentic API AI Readiness Framework signals are active in the current release, check out the\n[scoring engine implementation status](https://docs.jentic.com/reference/api-readiness-framework/scoring-engine-status/).\n\nJentic API Scorecard is licensed under the\n[Apache 2.0](https://github.com/jentic/jentic-api-scorecard/blob/main/LICENSE) license.\nJentic API Scorecard comes with an explicit\n[NOTICE](https://github.com/jentic/jentic-api-scorecard/blob/main/NOTICE) file containing\nadditional legal notices and information.", "url": "https://wpnews.pro/news/show-hn-cli-for-scoring-openapi-for-llm-legibility", "canonical_source": "https://github.com/jentic/jentic-api-scorecard", "published_at": "2026-06-05 08:35:34+00:00", "updated_at": "2026-06-05 08:47:55.484068+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "ai-infrastructure", "large-language-models", "ai-products"], "entities": ["Jentic API Scorecard", "JAIRF", "Jentic", "OpenAPI"], "alternates": {"html": "https://wpnews.pro/news/show-hn-cli-for-scoring-openapi-for-llm-legibility", "markdown": "https://wpnews.pro/news/show-hn-cli-for-scoring-openapi-for-llm-legibility.md", "text": "https://wpnews.pro/news/show-hn-cli-for-scoring-openapi-for-llm-legibility.txt", "jsonld": "https://wpnews.pro/news/show-hn-cli-for-scoring-openapi-for-llm-legibility.jsonld"}}