An OpenAPI document that passes validation isn't necessarily one an AI agent can use. Grammar is one thing; semantic clarity, safety, and discoverability are another. The Jentic API Scorecard scores your OpenAPI document against the Jentic API AI Readiness Framework (JAIRF) across six dimensions and returns a single grade — so you know exactly where to improve.
Each OpenAPI document is evaluated across six lenses — small, targeted improvements in any of them tend to produce outsized gains for both human developers and AI agents:
Foundational Compliance (FC)— structural validity and conformance to OpenAPI itself.** Developer Experience & Jentic Compatibility (DXJ)— documentation quality and how well the OpenAPI document plays with downstream tooling. AI-Readiness & Agent Experience (ARAX)— semantic clarity and the context an LLM needs to reason about each operation. Agent Usability (AU)— predictable, safe multi-step orchestration. Security (SEC)— declared auth schemes and trust boundaries. AI Discoverability (AID)**— how easily an AI system can find and parse the OpenAPI document.
Scoring runs locally inside a Docker container in two phases. Analysis runs a battery of validators and structural checks against the OpenAPI document to produce a set of diagnostics and observations. Scoring maps those into ~35 signals across the six JAIRF dimensions, aggregates them into per-dimension scores, and rolls those up into a single weighted score and grade.
Node.js 20 LTS or newer (>= 20.19.0
) with npm/npx. SeeNode.js downloads.Docker installed and running. SeeDocker installation. The CLI pulls the scoring image automatically on first run.- Network access to
(to pull the image) and to whatever URL hosts the OpenAPI document you're scoring (the engine fetches it from inside the container).ghcr.io
npm install -g @jentic/api-scorecard-cli
This installs the CLI globally. The scoring engine (Docker image) is downloaded automatically
the first time you run score
— allow a minute or two on a typical connection.
For local files or non-OAK URLs you'll also need a JENTIC_API_KEY
— see Anonymous vs keyed access.
Verify the install:
jentic-api-scorecard --version
Prefer zero-install?You can skip the global install and usenpx
— every example in this README works withnpx @jentic/api-scorecard-cli
in place ofjentic-api-scorecard
. Pin to a specific release withnpx @jentic/api-scorecard-cli@<version>
(e.g.@1.0.0
); the unpinned form resolves to whatever thelatest
dist-tag points at on each invocation, whilenpm install -g
pins you to the installed version until you explicitly update.
OpenAPI documents from Jentic Public APIs (OAK) score without any key, uncapped — no signup, no config:
npx @jentic/api-scorecard-cli@latest score \
https://raw.githubusercontent.com/jentic/jentic-public-apis/refs/heads/main/apis/openapi/swagger-api/petstore/1.0.27/openapi.json
For URLs outside OAK or local files, set the API key:
JENTIC_API_KEY=<your-key> npx @jentic/api-scorecard-cli@latest score \
https://petstore3.swagger.io/api/v3/openapi.json
JENTIC_API_KEY=<your-key> npx @jentic/api-scorecard-cli@latest score ./openapi.yaml
Important
Free keys come with 100 scorings per month (resets at the start of each calendar month). See Anonymous vs keyed access for signup and quota details.
That's it. The CLI pulls the scoring engine automatically on first run.
The --detail
flag lets you zoom in:
npx @jentic/api-scorecard-cli@latest score --detail summary ./openapi.yaml
npx @jentic/api-scorecard-cli@latest score --detail dimensions ./openapi.yaml
npx @jentic/api-scorecard-cli@latest score --detail signals ./openapi.yaml
npx @jentic/api-scorecard-cli@latest score --detail diagnostics ./openapi.yaml
Add --format json
to emit engine-verbatim JSON on stdout (filtered by whatever
--detail
level you pick). Pretty stays the unconditional default; --format json
is the canonical way to get a stable machine-readable channel for CI gating, archival, or LLM-assisted review.
npx @jentic/api-scorecard-cli@latest score ./openapi.yaml --format json | jq .summary.score
npx @jentic/api-scorecard-cli@latest score ./openapi.yaml \
--format json --detail diagnostics --output report.json
--output <file>
(-o
) writes the report to a path instead of stdout; the spinner stays on stderr.
--quiet
(-q
) suppresses the stderr spinner even in interactive terminals (the spinner already auto-suppresses when stderr isn't a TTY). Engine warnings still pass through stderr.
Add --with-llm
to unlock LLM-backed signals — deeper semantic reasoning about whether your API descriptions are actionable for agents, whether error responses support autonomous recovery, and more. Requires an LLM provider: cloud (OpenAI / Anthropic / Gemini / AWS Bedrock) or a local OpenAI-compatible endpoint (Ollama, LM Studio, vLLM, …).
export OPENAI_API_KEY=sk-...
export LLM_PROVIDER=OPENAI
export LIGHT_LLM_PROVIDER=OPENAI
export LLM_LIGHT_MODEL=gpt-4o-mini
JENTIC_API_KEY=<your-key> npx @jentic/api-scorecard-cli@latest score ./openapi.yaml --with-llm
Token cost is low — the engine uses a lightweight model (e.g. Claude Haiku, GPT-4o-mini), processes operations in small batches, and caps at 7 batches regardless of spec size. Local models (Ollama) cost nothing per call.
See ** LLM Signals guide** for all provider recipes (including local Ollama), the full environment variable reference, and troubleshooting.
OpenAPI documents from Jentic Public APIs (OAK) score without any key and stay on the free tier — those URLs bypass key validation entirely. For everything else (local files, URLs outside OAK), get a key at jentic.com/signup — once signed in, click Score → CLI & Keys to issue your key. Then set it:
export JENTIC_API_KEY=<your-key>
Real keys are validated live by the container against api.jentic.com
. The same call doubles
as the per-key usage / rate-limit accounting hit. Each free key gets 100 scorings per month,
resetting at the start of each calendar month. Once that quota is exhausted the CLI exits with
code 7
and prints the Retry-After
value along with a link to upgrade your plan.
jentic-api-scorecard [-V | --version] [-h | --help]
jentic-api-scorecard <command> [options]
| Command | Description |
|---|---|
score <input> |
Score an OpenAPI document by URL or local file path.
jentic-api-scorecard score <input> [options]
| Name | Description |
|---|---|
<input> |
|
https:// URL or local file path to an OpenAPI document. Required. |
| Flag | Default | Choices | Description |
|---|---|---|---|
--with-llm |
|||
| off | — | Enable LLM-backed analysis. Requires an LLM provider (see | |
--bundle
JENTIC_API_KEY
. No-op for local files.-d, --detail <level>
dimensions
summary
, dimensions
, signals
, diagnostics
Control output depth).-f, --format <fmt>
pretty
pretty
, json
Machine-readable output).-o, --output <file>
<file>
. The spinner stays on stderr.-q, --quiet
-h, --help
score
.| Variable | When | Purpose |
|---|---|---|
JENTIC_API_KEY |
URLs outside OAK and local files | Real key issued at
api.jentic.com (see
Free quota: 100 scorings per calendar month. |
--with-llm
OPENAI_API_KEY
, ANTHROPIC_API_KEY
, GEMINI_API_KEY
, AWS keys) and routing (LLM_PROVIDER
, LIGHT_LLM_PROVIDER
, LLM_MODEL
, LLM_LIGHT_MODEL
, *_API_URL
, LLM_MAX_TOKENS
) and forwards them to the container; loopback URLs are rewritten so a host-side Ollama is reachable. Full reference: LLM Signals guide.| Code | Meaning |
|---|---|
| 0 | Scoring completed (regardless of the score itself). |
| 1 | Generic error (bad input, bundling failure, unexpected container failure). |
| 2 | Auth: JENTIC_API_KEY is set to a value the Jentic backend does not recognize, or a local file / stdin input was used without the key set. |
| 3 | Anonymous gate rejected: URL outside the OAK allowlist and no key set. |
| 4 | Docker not installed or daemon unreachable. |
| 5 | Spec fetch or parse failure. |
| 6 | Engine invocation failure. |
| 7 | Rate limit reached: the key is valid but the user is over quota. Message includes the server-provided detail and the Retry-After header when present. |
jentic.com/scorecard offers the same scoring in a web UI — paste a URL or drop a file, no Docker or Node required.
For teams that need to know exactly what's running, verify exactly what was shipped, and run without a runtime dependency on Jentic.
Every component in the scoring stack — runner, CLI, release pipeline, and engine — is Apache 2.0 licensed and source-readable. No proprietary blobs, no closed-source shims. Read the code that's about to grade your specs before you adopt it; audit any line, redistribute under the license terms, fork if you ever need to.
Every npm tarball and every GHCR image is signed by Sigstore
with SLSA provenance and an SPDX SBOM. Signing happens inside an OIDC-driven
GitHub Actions workflow with no long-lived publishing secrets — there is no
NPM_TOKEN
, no PAT, and no human keyholder in the release chain. One command verifies an artifact end-to-end before you install it:
— npm provenance, SPDX SBOM, trusted publishing, and thenpm package supply chain →gh attestation verify
recipes.— per-platform SBOMs, dual-store attestations (BuildKit OCI referrers + Sigstore), and verification via eitherDocker image supply chain →docker buildx imagetools inspect
orgh attestation verify
.
The image is a closed system at scoring time: every Python wheel, Node.js
binary, and validator tarball it needs is baked in at build time. Scoring does
not call PyPI or npmjs and pulls no runtime packages. The only outbound
call to Jentic is a small key-check round-trip against api.jentic.com
that
authenticates your key and increments the per-key usage counter; OAK URLs
(jentic-public-apis) skip even that. URL inputs additionally reach the network to fetch the OpenAPI
document and resolve any external $ref
s it points at. --with-llm
optionally sends spec context to an LLM provider of your choice; a local endpoint (Ollama) keeps everything on-machine. Multi-arch images (linux/amd64 + linux/arm64) ship from the same release, so the same guarantees hold on Apple Silicon dev machines, ARM CI runners, and x86 servers alike.
CLI version, image tag, and engine version are locked one-to-one. Pinning
@jentic/api-scorecard-cli@<version>
resolves to a specific image tag, which in turn pins an exact engine release and exact validator versions. Last month's score is reproducible from last month's pin.
The CLI ships stable under the latest
npm dist-tag — release cadence is driven by Conventional Commits. Track in-flight work in specs/roadmap.md.
The :unstable
Docker image is rebuilt on every push to main
for direct docker run
users. Versioned images are published alongside each CLI release.
To see which Jentic API AI Readiness Framework signals are active in the current release, check out the scoring engine implementation status.
Jentic API Scorecard is licensed under the Apache 2.0 license. Jentic API Scorecard comes with an explicit NOTICE file containing additional legal notices and information.