{"slug": "your-ai-coding-agent-is-only-as-good-as-its-harness-what-changed-in-harness-1-5", "title": "Your AI Coding Agent Is Only as Good as Its Harness - What Changed in Harness Score 1.5", "summary": "Harness Score, an open-source scanner that measures how well a code repository is set up for AI coding agents, has released version 1.5 with customizable scoring, multilingual documentation, and a GitHub Marketplace Action. The tool deterministically checks for agent instructions, tests, linters, and guardrails across repositories, returning a maturity level from L0 to L4. Creator Fellipe Paladini emphasizes that the harness—the repository infrastructure around the model—often determines agent success more than the model itself.", "body_md": "Two teams can use the same model, write the same prompt, and get completely different results.\n\nThe difference is often not the model. It is the repository around it.\n\nOne repository gives the agent durable instructions, scoped rules, tests, linters, CI feedback, and hooks that stop dangerous actions. The other gives the agent a README and hopes for the best.\n\nThat surrounding control system is the **harness**.\n\n[Harness Score](https://github.com/paladini/harness-score) is a free, open-source, deterministic scanner that measures how much of that system a repository actually has. It checks filesystem evidence across Cursor, Claude Code, Windsurf, Cline, Continue, Codex, Copilot, and other tools. It then returns a maturity level from **L0 to L4**, a score across six dimensions, and a ranked list of what to improve next.\n\nNo LLM calls. No telemetry. No network access during the scan. The same repository at the same commit produces the same result.\n\nVersion 1.0 made that model stable. The **1.5 release series** turns it into a broader ecosystem with customizable scoring, five documentation languages, a GitHub Marketplace Action, a public showcase, and a reproducible research repository that tests the scanner against real open-source projects.\n\nSince 1.0, Harness Score has added:\n\n`.harness-score.json`\n\nTry the current release:\n\n```\nnpx harness-score@1.5.1\n```\n\nAI coding tools are increasingly good at producing code. But a capable model does not know your architecture, release process, dangerous commands, or definition of \"done\" unless you encode that knowledge.\n\nTeams usually add those pieces gradually:\n\n`AGENTS.md`\n\n, `CLAUDE.md`\n\n, or equivalent contextThe problem is visibility. Without a measurement, teams do not know which parts exist, which are missing, or whether a pull request quietly removed a critical guardrail.\n\nHarness Score turns those files into a deterministic diagnosis:\n\n```\nMaturity: L3 · Sensing\nScore: 86/108\nTo reach L4: add a gate hook and feedback hook\n```\n\nIt does not claim to judge whether your code is good or whether every rule is correct. It measures whether the infrastructure that guides and verifies an agent exists. That narrow scope makes the result reproducible enough for CI.\n\nVersion 1.1 shipped the complete guide in English, Brazilian Portuguese, and Latin American Spanish.\n\nVersion 1.2 added Simplified Chinese and Hindi.\n\nThis was not a landing-page-only translation. The guide includes the maturity model, check catalog, multi-harness support, sensors, guardrails, and remediation recipes in all five locales.\n\nHarness engineering is a team practice. A failed check is only useful if the people responsible for fixing it can understand the reasoning behind it.\n\n`.harness-score.json`\n\nbecame the repository contract\nVersion 1.3 introduced configuration for a question that a repository-only score could not answer honestly:\n\nWhat does the team commit, and what does the agent on this machine actually see?\n\nThe scanner now reports two views:\n\n| Score | Includes | Best use |\n|---|---|---|\nmaturity |\nRepository files only | CI, badges, and shared team maturity |\neffective |\nRepository plus configured user, system, or extra scopes | Local diagnosis |\n\nA repository can enable those scopes without changing the conservative CI default:\n\n```\n{\n  \"scopes\": {\n    \"user\": true,\n    \"system\": false\n  },\n  \"gate\": \"maturity\"\n}\n```\n\nVersion 1.3.1 expanded user-scope discovery across the documented global paths used by Cursor, Claude Code, Windsurf, Cline, Continue, Codex, OpenCode, Zed, and other supported tools.\n\nNot every check applies to every organization.\n\nFor example, a company policy may prohibit repository-local hooks. Previously, a team had to accept a permanent penalty or fork the maturity model. In 1.5, the repository can explain that decision explicitly:\n\n```\n{\n  \"extends\": [\"no-hooks\"],\n  \"rules\": {\n    \"HYG-05\": \"off\",\n    \"CI-01\": \"error\"\n  }\n}\n```\n\nThe important part is how customization behaves:\n\n`HYG-03`\n\n, `HYG-04`\n\n, and `HYG-06`\n\n- can never be disabled.Customization is transparent, reviewable repository policy, not a hidden way to inflate a score.\n\nRead the [v1.5.0 release notes](https://github.com/paladini/harness-score/releases/tag/v1.5.0).\n\nThe current patch, **v1.5.1**, marks the Action's initial publication on the [GitHub Marketplace](https://github.com/marketplace/actions/harness-score).\n\nA minimal workflow looks like this:\n\n```\nname: Harness maturity\n\non:\n  pull_request:\n  push:\n    branches: [main]\n\njobs:\n  harness:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v4\n      - uses: paladini/harness-score@v1\n        with:\n          min-level: \"3\"\n          badge: \"harness-badge.svg\"\n```\n\nThe Action can gate a minimum maturity level, write a GitHub job summary, generate Markdown and SVG badge output, and optionally maintain a sticky pull request comment with score changes.\n\nFor maximum supply-chain stability, pin the Action to a full commit SHA.\n\nA scanner becomes more useful when people can inspect what it produces outside its own repository.\n\nThe [Harness Maturity Showcase](https://paladini.io/harness-maturity-showcase/) is an open, evidence-backed leaderboard. It currently combines:\n\n`harness-score@1.5.0`\n\nNumeric rankings only include full reports. Badge-only entries display a maturity level but do not invent a numeric score.\n\nEvery ranked entry links to its evidence and measured commit. The goal is not to shame low-scoring projects. A score describes the harness committed in a specific repository, not the competence of the people or company behind it.\n\nThe [Harness Maturity Analysis](https://github.com/paladini/harness-maturity-analysis) is the research layer behind the showcase.\n\nIt pins each repository to an exact commit, pins the scanner version, stores the raw reports, and generates the leaderboard and findings from those artifacts. It also compares automated results with repository-level evidence to identify false positives, false negatives, and categories the maturity model does not represent well.\n\nPhase 1 is complete. All 21 repositories have been rescanned with `harness-score@1.5.0`\n\n, and the first evidence-backed findings are public.\n\nThe blind human-rating phase is still planned, so the project does not yet claim that every automated level agrees with human judgment.\n\nThat distinction matters. Harness Score is not treating its own output as ground truth. The analysis repository exists to find where the model needs to change.\n\nThe path from 1.0 to 1.5 also included less glamorous but important fixes:\n\n`$CLAUDE_PROJECT_DIR/.claude/hooks/setup.sh`\n\nno longer fail HKS-05 when the referenced script is committed.`node_modules/.bin/`\n\nare recognized as installed dependencies instead of missing repository files.Several of these fixes came directly from scanning real projects. That is the feedback loop I want for the project: scan, inspect evidence, find the mismatch, fix the deterministic rule, and run the corpus again.\n\nThe Harness Score repository dogfoods the maturity model and currently reaches **L4 · Self-correcting**.\n\nI generated the screenshot output by running this command from the analysis repository:\n\n```\nnpx harness-score@1.5.1 D:\\code\\harness-score\n```\n\nThe result:\n\n```\nharness-score v1.5.1  D:\\code\\harness-score\n\nMaturity: L4 · Self-correcting   Score: 108/108 (100%)   scopes: repo\nDetected: Antigravity, Claude Code, Cline, Codex, Cursor, Windsurf\n\nContext & Guides     ████████████████████ 100%  20/20 pts\nSkills & Commands    ████████████████████ 100%  17/17 pts\nHooks & Guardrails   ████████████████████ 100%  14/14 pts\nSensors & Feedback   ████████████████████ 100%  20/20 pts\nCI Feedback          ████████████████████ 100%  14/14 pts\nHygiene & Safety     ████████████████████ 100%  23/23 pts\n\nAll checks passed · this repository is fully harnessed.\n```\n\nI am planning to launch Harness Score on **Product Hunt next week**.\n\nThe Product Hunt launch is not the finish line. It is a way to put the tool in front of more teams, collect more real repository evidence, and discover where the maturity model is still wrong or incomplete.\n\nIf you test it before the launch, the most valuable feedback is concrete: share the repository, check ID, file path, and why the result is a false positive or false negative.\n\n```\nnpx harness-score@1.5.1\n```\n\nThen explore the ecosystem:\n\nWhat level did your repository get, and which failed check surprised you most? Drop both in the comments.", "url": "https://wpnews.pro/news/your-ai-coding-agent-is-only-as-good-as-its-harness-what-changed-in-harness-1-5", "canonical_source": "https://dev.to/paladini/your-ai-coding-agent-is-only-as-good-as-its-harness-what-changed-in-harness-score-15-38mi", "published_at": "2026-07-26 11:16:18+00:00", "updated_at": "2026-07-26 11:29:25.570400+00:00", "lang": "en", "topics": ["developer-tools", "ai-agents", "ai-tools", "ai-infrastructure"], "entities": ["Harness Score", "Fellipe Paladini", "Cursor", "Claude Code", "Windsurf", "Cline", "Continue", "Codex"], "alternates": {"html": "https://wpnews.pro/news/your-ai-coding-agent-is-only-as-good-as-its-harness-what-changed-in-harness-1-5", "markdown": "https://wpnews.pro/news/your-ai-coding-agent-is-only-as-good-as-its-harness-what-changed-in-harness-1-5.md", "text": "https://wpnews.pro/news/your-ai-coding-agent-is-only-as-good-as-its-harness-what-changed-in-harness-1-5.txt", "jsonld": "https://wpnews.pro/news/your-ai-coding-agent-is-only-as-good-as-its-harness-what-changed-in-harness-1-5.jsonld"}}