{"slug": "built-a-small-pr-guardrail-for-token-bloat-worth-maintaining", "title": "Built a small PR guardrail for token bloat, worth maintaining?", "summary": "ContextLevy, a new open-source tool, now automatically comments on GitHub pull requests to flag files that could bloat context windows and increase costs for AI coding agents like Cursor and Claude Code. The tool scans diffs for generated code, lockfile churn, and build artifacts, estimating the token weight and classifying risk to prevent \"repo-level context debt\" from degrading agent performance. Available as a GitHub Action and npm CLI, ContextLevy operates without LLM calls or telemetry, using only pull request metadata to provide advisory comments before merge.", "body_md": "**Bundle-size checks, but for AI agent context cost.**\n\nContextLevy comments on pull requests when a diff is likely to make coding agents slower, more expensive, or noisier to use.\n\n| Before ContextLevy | After ContextLevy |\n|---|---|\n| A PR silently adds ~90k tokens of coverage, generated clients, and build output | Reviewers see exactly which files caused the bloat and what to remove |\n| Lockfile churn dominates diffs with no agent-cost signal | ContextLevy flags lockfiles, estimates token weight, and suggests review focus |\n| Agent instruction files change behavior without visibility | High-signal agent config changes appear in the PR thread |\n\n| Use ContextLevy if… | Maybe skip it if… |\n|---|---|\n| Your team uses Cursor, Codex, or Claude Code heavily | Your repo rarely uses AI agents |\n| PRs often include generated output or coverage artifacts | You already have strict artifact hygiene and pre-commit gates |\n| You want advisory PR comments before merge | You need exact tokenizer-accurate billing from your provider |\n| You care about repo-level context debt, not just session tuning | You only need per-session context packs (see\n|\n\nSee [docs/EXAMPLES.md](/unloopedmido/contextlevy/blob/main/docs/EXAMPLES.md) for benchmark tables, monorepo recipes, and output usage.\n\nAI coding agents are powerful, but they are also extremely sensitive to noisy repository context.\n\nA single pull request can accidentally add:\n\n- generated clients\n- coverage reports\n- build output\n- lockfile churn\n- snapshots\n- huge logs\n- vendored files\n- agent instruction dumps\n- compiled bundles\n\nThat may not break your app, but it can absolutely bloat every future AI-assisted coding session.\n\n**ContextLevy catches that before it becomes repo debt.**\n\nIt scans pull request diffs, estimates added context weight, classifies risky files, and leaves a focused PR comment explaining what changed and what to clean up.\n\nSee [docs/COMPARISON.md](/unloopedmido/contextlevy/blob/main/docs/COMPARISON.md) for how ContextLevy compares to bundle tools, [ctx](https://github.com/forjd/ctx), and agent session tools.\n\n| Risk | Examples | Why it matters |\n|---|---|---|\n| Generated code | `generated/client.ts` , `schema.graphql` , SDK output |\nOften huge, repetitive, and better regenerated locally |\n| Coverage output | `coverage/lcov.info` , `htmlcov/` |\nHigh token cost with almost zero agent value |\n| Build artifacts | `dist/` , `build/` , `.next/` , compiled bundles |\nFrequently duplicated from source |\n| Logs and dumps | `*.log` , traces, debug output |\nNoisy context that agents over-read |\n| Lockfile churn | `package-lock.json` , `pnpm-lock.yaml` , `yarn.lock` |\nCan dominate diffs in dependency PRs |\n| Snapshots | `__snapshots__/` , large fixture files |\nUseful sometimes, expensive always |\n| Agent files | `.agents/` , `AGENTS.md` , instruction packs |\nCan silently steer future agent behavior |\n\nContextLevy is intentionally boring:\n\n**No LLM calls****No code upload****No external analysis service****No telemetry required**\n\nIt only uses GitHub pull request metadata and diff patches available inside the workflow.\n\nToken and cost numbers are estimates, not billing-grade accounting.\n\nContextLevy is available as a **GitHub Action** and an **npm CLI**. Choose one setup path:\n\nBest comment attribution and permissions. No repository secrets required.\n\nInstall the [ContextLevy GitHub App](https://github.com/apps/contextlevy) on your repository.\n\nGrant these repository permissions when prompted:\n\n| Permission | Access |\n|---|---|\n| Contents | Read |\n| Pull requests | Read & write |\n| Issues | Read & write |\n\nThe published app posts PR comments with its own identity. You do **not** need to add app credentials as repository secrets or variables.\n\nAfter changing app permissions, accept the updated installation request on the repository.\n\nCreate `.github/workflows/contextlevy.yml`\n\n:\n\n```\nname: ContextLevy\n\non:\n  pull_request:\n    types: [opened, synchronize, reopened]\n\npermissions:\n  contents: read\n  pull-requests: write\n  issues: write\n\njobs:\n  contextlevy:\n    name: Check AI context cost\n    runs-on: ubuntu-latest\n\n    steps:\n      - uses: actions/checkout@v4\n\n      - uses: unloopedmido/contextlevy@v2\n        with:\n          github-token: ${{ github.token }}\n```\n\nThat is the full setup. ContextLevy reads your PR diff, estimates context weight, and comments when thresholds are exceeded.\n\nWorks for many internal PRs without installing the app. Fork PRs may be read-only — see [Fork pull requests](#fork-pull-requests).\n\n```\npermissions:\n  contents: read\n  pull-requests: write\n  issues: write\n\nsteps:\n  - uses: actions/checkout@v4\n  - uses: unloopedmido/contextlevy@v2\n    with:\n      github-token: ${{ github.token }}\n```\n\nMaintainers and contributors only:To test with a self-hosted GitHub App in a private fork, see[CONTRIBUTING.md — Self-hosted GitHub App]for`CONTEXTLEVY_APP_ID`\n\nand`CONTEXTLEVY_APP_PRIVATE_KEY`\n\nsetup. End users should use the published app linked above.\n\nInstall from npm or build from source:\n\n```\nnpm install -g contextlevy\ncontextlevy diff --base main\ncontextlevy diff --base origin/main --format json --fail-on-config\n```\n\nFrom a clone:\n\n```\nnpm install && npm run build:cli\ncontextlevy diff --base main\n```\n\nSee [docs/CLI.md](/unloopedmido/contextlevy/blob/main/docs/CLI.md) for flags, exit codes, and pre-push hook recipes.\n\nTeach coding agents how to set up and use ContextLevy:\n\n```\nnpx skills add unloopedmido/contextlevy --skill contextlevy\n```\n\nSkill source: [.agents/skills/contextlevy/SKILL.md](/unloopedmido/contextlevy/blob/main/.agents/skills/contextlevy/SKILL.md)\n\nContextLevy reads all analysis and comment options from a config file in the repository. Add a config file once — workflow YAML stays minimal.\n\nOn pull requests, ContextLevy reads configuration from the base branch version of the repository. A PR cannot silence the check by changing `.contextlevy.yml`\n\nin the same diff.\n\nSupported config paths, in priority order:\n\n`.contextlevy.yml`\n\n`.contextlevy.yaml`\n\n`.contextlevy.json`\n\n`.github/contextlevy.yml`\n\n`.github/contextlevy.yaml`\n\n`.github/contextlevy.json`\n\n`contextlevy.yml`\n\n`contextlevy.yaml`\n\n`contextlevy.json`\n\nIf no config file is found, ContextLevy uses built-in defaults.\n\nEnable editor autocomplete with the published JSON Schema:\n\n``` php\n# yaml-language-server: $schema=./docs/schema/contextlevy.schema.json\ntoken-threshold: 1000\n```\n\nSchema file: [docs/schema/contextlevy.schema.json](/unloopedmido/contextlevy/blob/main/docs/schema/contextlevy.schema.json)\n\nExample `.contextlevy.yml`\n\n:\n\n```\ntoken-threshold: 1000\nlarge-file-token-threshold: 5000\nmax-high-impact-items: 5\nshow-cost-table: true\ncomment-format: default\n\nignore-paths:\n  - vendor/**\n  - \"**/*.map\"\n\nfail-on-severity: high\n\ncustom-rules:\n  - name: generated-supabase-types\n    paths:\n      - \"supabase/types.ts\"\n      - \"src/database/generated/**\"\n    category: generated\n    label: Generated Supabase types are usually low-value agent context.\n    suggestion: Regenerate locally unless this repo intentionally tracks generated DB types.\n\nestimation-mode: simple\n\nseverity-thresholds:\n  medium-tokens: 5000\n  high-tokens: 20000\n  critical-tokens: 100000\n\npricing-profiles:\n  - name: GPT-5.5\n    inputCostPerMillion: 5.0\n  - name: Opus 4.7\n    inputCostPerMillion: 5.0\n  - name: Team Gateway\n    inputCostPerMillion: 1.75\n```\n\nKeys support both kebab-case and camelCase:\n\n```\ntoken-threshold: 1000\ntokenThreshold: 1000\n```\n\n| Key | Default | Description |\n|---|---|---|\n`token-threshold` |\n`1000` |\nSkip commenting below this estimated token total |\n`large-file-token-threshold` |\n`5000` |\nMark individual files as large context risks |\n`max-high-impact-items` |\n`5` |\nMax files shown in the high-impact table |\n`show-cost-table` |\n`true` |\nInclude estimated model input costs |\n`comment-format` |\n`default` |\n`default` or `compact` |\n`ignore-paths` |\n`[]` |\nGlob patterns excluded from analysis entirely |\n`allow-paths` |\n`[]` |\nGlob patterns counted but not flagged as high-impact |\n`fail-on-severity` |\nunset | Fail workflow at `low` / `medium` / `high` / `critical` or above |\n`fail-above-tokens` |\nunset | Fail workflow when estimated tokens exceed this value |\n`estimation-mode` |\n`simple` |\n`simple` (`ceil(chars / 4)` ) or `tokenizer` (local BPE, no network) |\n`custom-rules` |\n`[]` |\nProject-specific path rules (see example above) |\n`severity-thresholds` |\nbuilt-in defaults | Override token/high-impact counts for Low/Medium/High/Critical |\n`pricing-profiles` |\nbuilt-in defaults | Array of `{ name, inputCostPerMillion }` objects |\n\nWhen `fail-on-severity`\n\nor `fail-above-tokens`\n\nis set, ContextLevy fails the workflow if thresholds are exceeded. **Fail mode runs even when the PR comment is skipped** — for example, when estimated tokens are below `token-threshold`\n\n. Analysis and fail checks always run; `token-threshold`\n\nonly controls whether a comment is posted.\n\nThe action accepts **authentication inputs only**. All behavior tuning belongs in the config file.\n\n| Input | Default | Description |\n|---|---|---|\n`github-token` |\n`GITHUB_TOKEN` env |\nFallback token for reading PR files and writing comments |\n`app-client-id` |\n`CONTEXTLEVY_APP_ID` / `CONTEXTLEVY_APP_CLIENT_ID` env |\nNumeric GitHub App ID |\n`app-private-key` |\n`CONTEXTLEVY_APP_PRIVATE_KEY` env |\nGitHub App private key PEM |\n`app-installation-id` |\n`CONTEXTLEVY_APP_INSTALLATION_ID` env |\nOptional GitHub App installation ID override |\n\nAuth credentials should stay in GitHub secrets or variables. Do not put private keys in `.contextlevy.yml`\n\n.\n\nUse these in downstream workflow steps:\n\n| Output | Type | Example | Description |\n|---|---|---|---|\n`total-estimated-tokens` |\ninteger string | `\"37891\"` |\nTotal estimated net-new context tokens |\n`analyzed-file-count` |\ninteger string | `\"12\"` |\nChanged files included in the estimate |\n`token-source` |\nstring | `\"app\"` |\nAuth source: `app` , `github-token` , or `GITHUB_TOKEN` |\n`estimation-mode` |\nstring | `\"simple\"` |\nEstimation mode used: `simple` or `tokenizer` |\n\n```\n- id: contextlevy\n  uses: unloopedmido/contextlevy@v2\n\n- if: ${{ steps.contextlevy.outputs.total-estimated-tokens > 50000 }}\n  run: echo \"Context cost too high\"\n```\n\nContextLevy also writes a **job summary** with risk level and top findings for every run.\n\nBest for most repositories.\n\nIncludes:\n\n- severity\n- estimated token delta\n- high-impact files\n- file classifications\n- optional cost table\n- cleanup suggestions\n\n```\ncomment-format: default\n```\n\nBest for busy repos that want a smaller PR footprint.\n\nUsually 3–4 lines:\n\n```\ncomment-format: compact\n```\n\nExample:\n\n```\n🤖 ContextLevy · ⚠️ High · ~42.1k tokens\n+31.4k coverage/lcov.info · +8.2k dist/index.js · +2.5k generated/client.ts\n~$0.02–$0.12/session est. input · Add coverage/ and dist/ to .gitignore\n```\n\nDefault pricing profiles are **illustrative** and may drift as model prices change. For accurate internal estimates, configure your own `pricing-profiles`\n\n.\n\nWhen `pricing-profiles`\n\nis omitted, ContextLevy estimates worst-case input cost using:\n\n| Profile | Input cost / 1M tokens |\n|---|---|\n| GPT-5.5 | `$5.00` |\n| Opus 4.7 | `$5.00` |\n| Gemini 3.1 Pro | `$2.00` |\n| Kimi K2.6 | `$0.95` |\n\nHide the cost table in your config file:\n\n```\nshow-cost-table: false\n```\n\nOverride pricing profiles:\n\n```\npricing-profiles:\n  - name: Local 70B\n    inputCostPerMillion: 0.2\n  - name: Team Gateway\n    inputCostPerMillion: 1.75\n```\n\nContextLevy supports two local estimation modes (no LLM calls, no network):\n\n| Mode | Method | Best for |\n|---|---|---|\n`simple` (default) |\n`ceil(chars / 4)` on added diff lines |\nFast warnings, CI everywhere |\n`tokenizer` |\n`cl100k_base` BPE token count on added diff text |\nCloser to GPT-family token counts |\n\nProcess:\n\n- List files changed in the pull request.\n- Read added diff lines from each patch.\n- Estimate tokens using the configured mode.\n- If no patch is available, fall back to\n`additions × 10`\n\n. - Classify risky paths with built-in rules plus optional\n`custom-rules`\n\n.\n\nThis is intentionally approximate.\n\nDifferent models tokenize differently, agents may not read every changed file, and cached-token pricing varies by provider. Cost tables show ±50% ranges. Treat the output as a practical warning signal, not an invoice.\n\n| Severity | Meaning |\n|---|---|\n`Low` |\nSmall context increase, usually safe |\n`Medium` |\nWorth reviewing, especially in agent-heavy repos |\n`High` |\nLikely to affect AI coding sessions |\n`Critical` |\nVery large diff or obvious repo-noise artifact |\n\nOverride thresholds in config:\n\n```\nseverity-thresholds:\n  medium-tokens: 5000\n  high-tokens: 20000\n  critical-tokens: 100000\n  medium-high-impact-count: 1\n  high-high-impact-count: 3\n  critical-high-impact-count: 8\ntoken-threshold: 5000\nmax-high-impact-items: 3\ncomment-format: compact\nestimation-mode: tokenizer\ncustom-rules:\n  - paths:\n      - \"packages/api/src/generated/**\"\n    category: generated\n    label: Generated API clients add repetitive agent context.\n    suggestion: Regenerate locally during development.\nshow-cost-table: false\npricing-profiles:\n  - name: Internal Gateway\n    inputCostPerMillion: 1.25\n  - name: Local Inference\n    inputCostPerMillion: 0.05\n```\n\nContextLevy is most useful when paired with normal repository hygiene.\n\nCommon `.gitignore`\n\nadditions:\n\n```\ncoverage/\nhtmlcov/\ndist/\nbuild/\n.next/\n.cache/\n*.log\n```\n\nGenerated files may still belong in version control depending on your language, package manager, or deployment setup. ContextLevy does not block PRs by default; it gives reviewers a focused warning.\n\nYour workflow token or GitHub App probably does not have enough permissions to create or update PR comments.\n\nCheck:\n\n```\npermissions:\n  contents: read\n  pull-requests: write\n  issues: write\n```\n\nIf you use the GitHub App, confirm the installation has:\n\n- Contents: read\n- Pull requests: read & write\n- Issues: read & write\n\nFor pull requests from forks, GitHub may still provide a read-only workflow token. In that case ContextLevy logs a warning, keeps the action successful, still exposes analysis outputs, and writes a job summary — but may not post a PR comment.\n\nInstall the GitHub App when your organization policy allows it for more reliable fork PR comments.\n\nSee [SECURITY.md — Fork pull requests](/unloopedmido/contextlevy/blob/main/SECURITY.md#fork-pull-requests) for permission details.\n\nMake sure the secret contains the GitHub App private key PEM.\n\nIt should look like this:\n\n```\n-----BEGIN RSA PRIVATE KEY-----\n...\n-----END RSA PRIVATE KEY-----\n```\n\nDo not use the app Client Secret.\n\nContextLevy skips comments below `token-threshold`\n\n. Fail mode (`fail-on-severity`\n\n, `fail-above-tokens`\n\n) still runs in that case — a skipped comment does not mean the check was skipped.\n\nLower the threshold while testing:\n\n```\ntoken-threshold: 0\n```\n\nThat usually means the PR added large generated files, coverage output, build artifacts, or lockfile churn.\n\nIf the files are intentional, either ignore the warning or raise your thresholds.\n\nInstall dependencies:\n\n```\nnpm install\n```\n\nRun tests:\n\n```\nnpm test\n```\n\nBuild the action bundle and CLI:\n\n```\nnpm run build          # both\nnpm run build:action   # GitHub Action only → dist/index.js\nnpm run build:cli      # local CLI only → lib/\n```\n\nCommit `dist/index.js`\n\nafter building the action so workflow consumers do not need to install runtime dependencies. The CLI (`lib/`\n\n) is built automatically on `npm publish`\n\nvia `prepack`\n\n.\n\nVerify the npm tarball before publishing:\n\n```\nnpm run pack:check\n```\n\nReleases are automated when a version bump lands on `main`\n\n. The [release workflow](/unloopedmido/contextlevy/blob/main/.github/workflows/release.yml) detects a `package.json`\n\nversion change, runs tests, verifies `dist/`\n\n, creates a GitHub Release, pushes the semver tag, publishes the CLI to npm via [trusted publishing](https://docs.npmjs.com/trusted-publishers) (OIDC), and updates the major tag.\n\n**Do not push semver tags manually.** Bump the version in `package.json`\n\n, `package-lock.json`\n\n, and `CHANGELOG.md`\n\n, push to `main`\n\n, and CI handles the tag, GitHub Release, and npm publish.\n\nOn [npmjs.com](https://www.npmjs.com/package/contextlevy) → **Package settings** → **Trusted publishing**, configure **GitHub Actions** with repository `unloopedmido/contextlevy`\n\nand workflow filename `release.yml`\n\n. No `NPM_TOKEN`\n\nsecret is required.\n\nIf npm publish fails after a version bump, re-run the **Release** workflow from the Actions tab (`workflow_dispatch`\n\n) once the package is missing on npm — it will retry without another version bump.\n\nExample release sequence:\n\n```\n# After updating package.json, package-lock.json, and CHANGELOG.md\ngit push origin main\n```\n\nThe workflow updates the major-version tag (`v2`\n\n) automatically.\n\nBefore trusted publishing is configured, publish the CLI once from a clean checkout:\n\n```\nnpm ci\nnpm run pack:check\nnpm publish --access public\n```\n\nThen add the trusted publisher on npmjs.com as described above. Later version bumps on `main`\n\npublish automatically via OIDC.\n\nConsumers should usually pin:\n\n```\n- uses: unloopedmido/contextlevy@v2\n```\n\nFor maximum supply-chain safety, consumers can pin a full commit SHA.\n\nContextLevy is a pull request analysis tool. It does not execute changed code and does not send repository contents to an LLM or third-party API.\n\nPlease report security issues privately through GitHub Security Advisories instead of opening a public issue.\n\nMIT", "url": "https://wpnews.pro/news/built-a-small-pr-guardrail-for-token-bloat-worth-maintaining", "canonical_source": "https://github.com/unloopedmido/contextlevy", "published_at": "2026-05-24 23:31:25+00:00", "updated_at": "2026-05-25 00:27:25.068336+00:00", "lang": "en", "topics": ["ai-tools", "ai-agents", "ai-infrastructure", "mlops", "large-language-models"], "entities": ["ContextLevy", "Cursor", "Codex", "Claude Code"], "alternates": {"html": "https://wpnews.pro/news/built-a-small-pr-guardrail-for-token-bloat-worth-maintaining", "markdown": "https://wpnews.pro/news/built-a-small-pr-guardrail-for-token-bloat-worth-maintaining.md", "text": "https://wpnews.pro/news/built-a-small-pr-guardrail-for-token-bloat-worth-maintaining.txt", "jsonld": "https://wpnews.pro/news/built-a-small-pr-guardrail-for-token-bloat-worth-maintaining.jsonld"}}