An OpenAPI document that passes validation isn't necessarily one an AI agent can use. Grammar is one thing; semantic clarity, safety, and discoverability are another. The Jentic API Scorecard scores your OpenAPI document against the Jentic API AI Readiness Framework (JAIRF) across six dimensions and returns a single grade — so you know exactly where to improve. Pair it with the companion api-improve agent skill to apply those improvements automatically: non-breaking edits, a reusable OpenAPI Overlay, and a before/after changelog.

• Quick start
• What it scores
• How it works
• Requirements
• Install
• Try it now
• Control output depth
• Machine-readable output
• HTML report
• LLM analysis
• Anonymous vs keyed access
• Skills
  • jentic-api-scorecard
  • jentic-api-improve
• CLI reference
  • Commands
  • score
• GitHub Action
• Prefer a browser?
• Enterprise-ready by default
  • Your OpenAPI document never leaves your environment
  • Auditable end to end
  • Signed for regulated environments
  • Runs anywhere
  • Pinned for reproducibility
• Status
  • Scoring engine signal status
• License

1. Get a free key at the Jentic Scorecard API Keys page.

2. Score an OpenAPI document:

   JENTIC_API_KEY=<your-key> npx @jentic/api-scorecard-cli score ./openapi.yaml

Each OpenAPI document is evaluated across six lenses — small, targeted improvements in any of them tend to produce outsized gains for both human developers and AI agents:

• Foundational Compliance (FC) — structural validity and conformance to OpenAPI itself.
• Developer Experience & Jentic Compatibility (DXJ) — documentation quality and how well the OpenAPI document plays with downstream tooling.
• AI-Readiness & Agent Experience (ARAX) — semantic clarity and the context an LLM needs to reason about each operation.
• Agent Usability (AU) — predictable, safe multi-step orchestration.
• Security (SEC) — declared auth schemes and trust boundaries.
• AI Discoverability (AID) — how easily an AI system can find and parse the OpenAPI document.

Scoring runs locally inside a Docker container in two phases. Analysis runs a battery of validators and structural checks against the OpenAPI document to produce a set of diagnostics and observations. Scoring maps those into ~35 signals across the six JAIRF dimensions, aggregates them into per-dimension scores, and rolls those up into a single weighted score and grade.

• Node.js 20 LTS or newer (>= 20.19.0) with npm/npx. See Node.js downloads.
• Docker installed and running. See Docker installation. The CLI pulls the scoring image automatically on first run.
• Network access to ghcr.io (to pull the image) and to whatever URL hosts the OpenAPI document you're scoring (the engine fetches it from inside the container).

npm install -g @jentic/api-scorecard-cli

This installs the CLI globally. The scoring engine (Docker image) is downloaded automatically the first time you run score — allow a minute or two on a typical connection.

For local files or non-OAK URLs you'll also need a JENTIC_API_KEY — see Anonymous vs keyed access.

Verify the install:

jentic-api-scorecard --version

Prefer zero-install? You can skip the global install and use npx — every example in this README works with npx @jentic/api-scorecard-cli in place of jentic-api-scorecard. Pin to a specific release with npx @jentic/api-scorecard-cli@<version> (e.g. @1.0.0); the unpinned form resolves to whatever the latest dist-tag points at on each invocation, while npm install -g pins you to the installed version until you explicitly update.

OpenAPI documents from Jentic Public APIs (OAK) score without any key, uncapped — no signup, no config:

npx @jentic/api-scorecard-cli@latest score \
  https://raw.githubusercontent.com/jentic/jentic-public-apis/refs/heads/main/apis/openapi/swagger-api/petstore/1.0.27/openapi.json

For URLs outside OAK or local files, set the API key:

JENTIC_API_KEY=<your-key> npx @jentic/api-scorecard-cli@latest score \
  https://petstore3.swagger.io/api/v3/openapi.json

JENTIC_API_KEY=<your-key> npx @jentic/api-scorecard-cli@latest score ./openapi.yaml

That's it. The CLI pulls the scoring engine automatically on first run.

The --detail flag lets you zoom in:

# Just the headline score and grade
npx @jentic/api-scorecard-cli@latest score --detail summary ./openapi.yaml

# Per-dimension breakdown (default)
npx @jentic/api-scorecard-cli@latest score --detail dimensions ./openapi.yaml

# Individual signals within each dimension
npx @jentic/api-scorecard-cli@latest score --detail signals ./openapi.yaml

# Full diagnostics with top 5 findings per severity
npx @jentic/api-scorecard-cli@latest score --detail diagnostics ./openapi.yaml

Add --format json to emit engine-verbatim JSON on stdout (filtered by whatever --detail level you pick). Pretty stays the unconditional default; --format json is the canonical way to get a stable machine-readable channel for CI gating, archival, or LLM-assisted review.

# Gate on the headline score in CI
npx @jentic/api-scorecard-cli@latest score ./openapi.yaml --format json | jq .summary.score

# Capture the full evidence bundle to a file
npx @jentic/api-scorecard-cli@latest score ./openapi.yaml \
  --format json --detail diagnostics --output report.json

--output <file> (-o) writes the report to a path instead of stdout; the spinner stays on stderr.

--quiet (-q) suppresses the stderr spinner even in interactive terminals (the spinner already auto-suppresses when stderr isn't a TTY). Engine warnings still pass through stderr.

Add --format html to render an interactive, self-contained HTML scorecard — a single file with all JS and CSS inlined (no external CDN, works offline), suitable for CI artifacts and dashboards. It honours --detail, so --detail diagnostics embeds the full evidence bundle.

Because the output is a full HTML document, the CLI refuses to print it straight into an interactive terminal — redirect it or use -o:

# Redirect to a file
npx @jentic/api-scorecard-cli@latest score ./openapi.yaml --format html > scorecard.html

# Or write it with -o, at full detail
npx @jentic/api-scorecard-cli@latest score ./openapi.yaml \
  --format html --detail diagnostics -o scorecard.html

Add --with-llm to unlock LLM-backed signals — deeper semantic reasoning about whether your API descriptions are actionable for agents, whether error responses support autonomous recovery, and more. Requires an LLM provider: cloud (OpenAI / Anthropic / Gemini / AWS Bedrock) or a local OpenAI-compatible endpoint (Ollama, LM Studio, vLLM, …).

Without --with-llm, the LLM-backed signals are not evaluated — they hold an assumed-perfect baseline score until --with-llm actually assesses them. A default (no---with-llm) scorecard therefore reflects only the deterministic signals, with the LLM-backed ones sitting at that perfect baseline; turn on --with-llm to have them genuinely assessed (which can lower your score).

export OPENAI_API_KEY=sk-...
export LLM_PROVIDER=OPENAI
export LIGHT_LLM_PROVIDER=OPENAI
export LLM_LIGHT_MODEL=gpt-4o-mini

JENTIC_API_KEY=<your-key> npx @jentic/api-scorecard-cli@latest score ./openapi.yaml --with-llm

Token cost is low — the engine uses a lightweight model (e.g. Claude Haiku, GPT-4o-mini), processes operations in small batches, and caps at 7 batches regardless of spec size. Local models (Ollama) cost nothing per call.

If the LLM calls fail (bad credentials, an inaccessible model, an unreachable endpoint), the affected LLM-backed signals get scored as perfect — which would inflate their dimension(s) and the overall score. Rather than print a misleading scorecard, the CLI suppresses the report, names the affected signals and the provider error on stderr, and exits 8 — so a CI job running --with-llm fails loudly instead of passing on an inflated score. Fix the provider error and retry, or re-run without --with-llm for a valid score from the non-LLM signals.

See LLM Signals guide for all provider recipes (including local Ollama), the full environment variable reference, and troubleshooting.

OpenAPI documents from Jentic Public APIs (OAK) score without any key and stay on the free tier — those URLs bypass key validation entirely. For everything else (local files, URLs outside OAK), get a key from the Jentic Scorecard API Keys page. Then set it:

export JENTIC_API_KEY=<your-key>

Real keys are validated live by the container against api.jentic.com. The same call doubles as the per-key usage / rate-limit accounting hit. Each free key gets 100 scorings per month, resetting at the start of each calendar month. Once that quota is exhausted the CLI exits with code 7 and prints the Retry-After value along with a link to upgrade your plan.

This repository ships two complementary agent skills — install either or both through whichever path fits your agent.

A versioned agent skill that teaches AI coding agents how to use the CLI correctly — installing it, scoring files and URLs, producing JSON/HTML, wiring it into CI, enabling LLM analysis, and interpreting exit codes.

Claude Code users install it as a plugin — this repository doubles as a plugin marketplace:

/plugin marketplace add jentic/jentic-api-scorecard
/plugin install api-scorecard@jentic-api-scorecard

Once installed, the skill loads automatically when you ask Claude to score an OpenAPI document — no explicit invocation needed:

> Score ./openapi.yaml for AI-readiness
> How AI-ready is https://petstore3.swagger.io/api/v3/openapi.json?

To force it into context regardless of phrasing, invoke it explicitly with /api-scorecard:jentic-api-scorecard.

Install it straight from this repository with the skills CLI:

npx skills add jentic/jentic-api-scorecard --skill jentic-api-scorecard

It's also listed in the skills.sh directory.

The @jentic/api-scorecard-cli npm package also ships this skill inside its published tarball, so it's discoverable by TanStack Intent for projects that already depend on the CLI and want version-aligned agent guidance.

Scoring tells you what to fix; the jentic-api-improve skill closes the loop and fixes it. Point an AI coding agent at an OpenAPI document and it runs a baseline score, identifies the weak dimensions and the semantic diagnostics, applies non-breaking improvements (adding summary/description/example/tags — never changing existing paths, parameters, or response shapes), and produces three artifacts: an improved spec, an OpenAPI Overlay (the reusable, non-breaking delta), and a changelog with before/after scores.

This skill orchestrates the scorecard CLI plus a few extra command-line tools, so it needs more than the CLI alone. In addition to the scorecard CLI's Requirements (Node.js ≥ 20.19, a running Docker daemon, and a JENTIC_API_KEY for local-file scoring), install:

pipx install jentic-openapi-tools     # validation
pipx install jentic-apitools-cli      # overlay verification (command: jentic-apitools)
pipx install check-jsonschema         # overlay schema validation
# plus python3 and jq, which most systems already have

Install the skill through whichever path fits your agent — the same three channels as the scoring skill:

The improve skill is a separate plugin in this repository's marketplace:

/plugin marketplace add jentic/jentic-api-scorecard
/plugin install api-improve@jentic-api-scorecard

Installing the plugin also registers the companion jentic-api-improve subagent (used for multi-iteration improvement loops). Once installed, ask Claude to improve a spec:

> Improve ./openapi.yaml for AI-readiness
> Raise the JAIRF score of my API and give me an overlay

npx skills add jentic/jentic-api-scorecard --skill jentic-api-improve

The @jentic/api-scorecard-cli npm package ships this skill (and the companion agent definition) inside its published tarball, so it's discoverable by TanStack Intent for projects that already depend on the CLI. The agent file lands at agents/jentic-api-improve.md; copy it into your .claude/agents/ to enable the declarative subagent (optional — the skill falls back to spawning a subagent from an inline brief).

jentic-api-scorecard [-V | --version] [-h | --help]
jentic-api-scorecard <command> [options]

Score an OpenAPI document by URL or local file path.

jentic-api-scorecard score <input> [options]

A composite GitHub Action wraps the CLI for CI: it scores an OpenAPI document, gates the build on the score, uploads SARIF findings to the Security tab, attaches the HTML scorecard as a downloadable artifact, and renders a Markdown summary on the run page.

name: API readiness
on: [pull_request]

permissions:
  contents: read
  security-events: write # required so SARIF uploads to the Security tab

jobs:
  scorecard:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: jentic/jentic-api-scorecard@v1 # floating major tag (or pin a full version / commit SHA)
        with:
          input: ./openapi.yaml
          api-key: ${{ secrets.JENTIC_API_KEY }}
          min-score: '70'

To enable LLM-backed analysis, set with-llm: true and provide the provider credentials and routing as job-level env: — the action forwards them to the engine but does not turn raw secrets into those variables for you, so the run fails fast if none are present:

jobs:
  scorecard:
    runs-on: ubuntu-latest
    env:
      OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
      LLM_PROVIDER: OPENAI
      LIGHT_LLM_PROVIDER: OPENAI
      LLM_LIGHT_MODEL: gpt-4o-mini
    steps:
      - uses: actions/checkout@v4
      - uses: jentic/jentic-api-scorecard@v1
        with:
          input: ./openapi.yaml
          api-key: ${{ secrets.JENTIC_API_KEY }}
          with-llm: 'true'

See the LLM Signals guide for the full provider matrix (cloud and local Ollama) and the variable reference.

Choosing your gates. Set min-score to fail PRs below a readiness bar, and/or max-errors / max-warnings to fail on too many findings. Each finding carries one of three levels — error, warning, note — which is how it's labelled on its Security-tab alert. (If you also read --format json, those map from its 1–4 severity: 1 is error, 2 is warning, 3 and 4 are both note.) The severity input only trims which findings reach the Security tab; it never changes your gate, so quieting the Security tab to errors-only won't let a warning-heavy spec slip through.

Outputs land even on a failing build. When a gate fails, you still get the SARIF findings, the HTML artifact, and the Markdown summary — a failing PR is exactly when you want to see them.

Choose your pin deliberately. The @v1 major tag (used in the examples above) floats to the newest v1.x.x on every release, so you auto-receive fixes without a workflow edit and stay within one major — no breaking changes. The trade-off: a release can ship a new scoring engine, and the same document can score differently across engine versions, so a green PR can turn red when @v1 moves under it. If you gate on a hard threshold and need that build reproducible, pin a full version (@v1.8.3) or a commit SHA instead, and bump it deliberately when you're ready to adopt a new engine.

On fork PRs, GitHub gives the workflow a read-only token, so the Security-tab upload can't run; the action skips it with a notice and still publishes the HTML artifact and Markdown summary. (Uploading fork findings anyway needs a writable token from a base-context workflow, supplied via github-token.) In the Security tab, findings link to the document but not yet to a specific line.

jentic.com/scorecard offers the same scoring in a web UI — paste a URL or drop a file, no Docker or Node required.

For teams that need to know exactly what's running, verify exactly what was shipped, and run without a runtime dependency on Jentic.

Scoring runs entirely in a container on your own machine. Local files are piped in over stdin; URLs are fetched on your side — by the container's engine, or host-side by the CLI when you pass --bundle — never by Jentic. Either way, your spec is never uploaded.

The only call to Jentic is a key-check round-trip against api.jentic.com — it carries your key, never any part of your spec, and OAK URLs (jentic-public-apis) skip even that. The one exception is --with-llm, which sends spec context to the LLM provider you choose (point it at a local Ollama to keep that on-machine too).

Every component in the scoring stack — runner, CLI, release pipeline, and engine — is Apache 2.0 licensed and source-readable. No proprietary blobs, no closed-source shims. Read the code that's about to grade your specs before you adopt it; audit any line, redistribute under the license terms, fork if you ever need to.

Every npm tarball and every GHCR image is signed by Sigstore with SLSA provenance and an SPDX SBOM. Signing happens inside an OIDC-driven GitHub Actions workflow with no long-lived publishing secrets — there is no NPM_TOKEN, no PAT, and no human keyholder in the release chain. One command verifies an artifact end-to-end before you install it:

• npm package supply chain → — npm provenance, SPDX SBOM, trusted publishing, and the gh attestation verify recipes.
• Docker image supply chain → — per-platform SBOMs, dual-store attestations (BuildKit OCI referrers + Sigstore), and verification via either docker buildx imagetools inspect or gh attestation verify.

The image is a closed system at scoring time: every Python wheel, Node.js binary, and validator tarball it needs is baked in at build time, so scoring pulls no runtime packages from PyPI or npmjs. Multi-arch images (linux/amd64 + linux/arm64) ship from the same release, so the same guarantees hold on Apple Silicon dev machines, ARM CI runners, and x86 servers alike.

CLI version, image tag, and engine version are locked one-to-one. Pinning @jentic/api-scorecard-cli@<version> resolves to a specific image tag, which in turn pins an exact engine release and exact validator versions. Last month's score is reproducible from last month's pin.

The CLI ships stable under the latest npm dist-tag — release cadence is driven by Conventional Commits. Track in-flight work in specs/roadmap.md.

The :unstable Docker image is rebuilt on every push to main for direct docker run users. Versioned images are published alongside each CLI release.

To see which Jentic API AI Readiness Framework signals are active in the current release, check out the scoring engine implementation status.

Jentic API Scorecard is licensed under the Apache 2.0 license. Jentic API Scorecard comes with an explicit NOTICE file containing additional legal notices and information.