{"slug": "show-hn-jscpd-start-support-type2-type3-clones", "title": "Show HN: Jscpd start support type2/type3 clones", "summary": "Jscpd v5, a copy/paste detector for programming source code maintained by kucherenko, now detects Type-2 renamed clones and Type-3 near-miss clones across 224 supported formats. The Rust-based tool runs on a Rabin-Karp algorithm over language-aware token streams and ships as a self-contained binary installable via npm, PyPI, Cargo, Homebrew, Nix, and Docker, with an MCP server and token-efficient reporter for AI workflows. The release also provides a GitHub Action that uploads SARIF results to GitHub Code Scanning by default.", "body_md": "Copy/paste detector for programming source code. 220+ formats, language-aware tokenization, exact, renamed and near-miss clones, Rust engine, self-contained binary, AI-ready with MCP server and token-efficient reporter.\n\n**Documentation:** [https://jscpd.dev](https://jscpd.dev)\n\njscpd reads code the way its language defines it, not as plain text. Each of the 224 formats is tokenized with its own comment and string syntax, so `#` in Python, `--` in SQL or `'` in Visual Basic opens a comment only where the language says so. JavaScript, TypeScript, JSX and TSX go through the [oxc](https://oxc.rs) parser, which handles template literals, regular expressions, JSX and decorators, and can erase TypeScript-only syntax so a `.ts` file matches its `.js` twin. Vue, Svelte, Astro, Markdown and Razor files are split into their embedded languages first, and each block is tokenized as the language it contains. Identifiers, keywords and literals are classified, which is what lets the renamed-clone pass replace names while keeping keywords in place.\n\nOn that token stream jscpd runs the [Rabin-Karp](https://en.wikipedia.org/wiki/Rabin%E2%80%93Karp_algorithm) algorithm to find duplicated blocks across files. Opt-in passes extend it to blocks that differ only in names or values (Type-2) and to copies with a few edited lines or the same function structure (Type-3), each reported with its kind and a similarity score. See [How detection works](/kucherenko/jscpd/blob/master/docs/rust.md#how-detection-works).\n\n```\n# macOS / Linux\ncurl -fsSL https://jscpd.dev/install.sh | bash\n\n# Windows (PowerShell)\nirm https://jscpd.dev/install.ps1 | iex\n\n# No install — run once with npx (Node.js)\nnpx jscpd .\n```\n\nThen scan a project:\n\n```\njscpd /path/to/code\n```\n\n| Method | Command | Notes | \n|---|---|---|\n| npm | `npm install -g jscpd` | Installs the `jscpd` command; prebuilt binary, no Node.js at runtime | \n| npm ( `cpd` command) | `npm install -g cpd` | Same binary, exposed as `cpd` | \n| PyPI | `pip install jscpd` | Platform wheels with both commands; also `pipx install jscpd` ,`uv tool install jscpd` , or`uvx jscpd .` to run without installing | \n| Cargo | `cargo install jscpd` | Builds from crates.io; installs both `jscpd` and`cpd` | \n| Homebrew | `brew install jscpd` | macOS / Linux | \n| Nix | `nix run github:kucherenko/jscpd -- /path/to/code` | Or `nix profile install github:kucherenko/jscpd` | \n| Docker | `docker run --rm -v \"$PWD:/src\" ghcr.io/kucherenko/jscpd .` | Multi-arch image built from the release binaries | \n\n```\n- uses: kucherenko/jscpd@v5\n  with:\n    threshold: 5\n```\n\nUploads SARIF results to GitHub Code Scanning by default. See [CI & Pre-Commit Hooks](/kucherenko/jscpd/blob/master/docs/ci-and-hooks.md) for all inputs and outputs.\n\n| Document | Description | \n|---|---|\n| [Rust engine](/kucherenko/jscpd/blob/master/docs/rust.md) | Installation, CLI reference, reporters, baseline, summary, blame, config file | \n| [AI-Ready](/kucherenko/jscpd/blob/master/docs/ai-ready.md) | AI reporter, agent skills, MCP server | \n| [Programming API](/kucherenko/jscpd/blob/master/docs/api.md) | Rust API ( `cpd-finder` crate) | \n| [CI & Pre-Commit Hooks](/kucherenko/jscpd/blob/master/docs/ci-and-hooks.md) | GitHub Action, Docker image, pre-commit hooks | \n| [Packages](/kucherenko/jscpd/blob/master/docs/packages.md) | npm packages and crates that make up a release | \n| [Supported formats](/kucherenko/jscpd/blob/master/FORMATS.md) | All 224 formats with their file extensions | \n| [Runnable demos](/kucherenko/jscpd/blob/master/fixtures) | One `fixtures/<feature>-demo/` directory per feature, each README lists the commands with their expected output | \n\njscpd v5 is a Rust engine that ships as a self-contained binary — no runtime required — under two npm names ([`jscpd`](https://www.npmjs.com/package/jscpd) installs the `jscpd` command, [`cpd`](https://www.npmjs.com/package/cpd) installs `cpd`), on [PyPI](https://pypi.org/project/jscpd/), [crates.io](https://crates.io/crates/jscpd), Homebrew, Nix, Docker, and as a GitHub Action.\n\n- **Language-aware tokenization** — per-format comment and string syntax for all 224 formats, the oxc parser for JavaScript/TypeScript/JSX/TSX, embedded-language extraction for Vue, Svelte, Astro, Markdown and Razor, and keyword/identifier/literal classification, so a clone is a repeated sequence of*language tokens* , never a repeated run of text (see[How detection works](/kucherenko/jscpd/blob/master/docs/rust.md#how-detection-works) )\n- **224 language formats** with cross-format detection (Vue SFC, Svelte, Astro, Markdown) and`--cross-formats` groups to match clones across JavaScript and TypeScript\n- **Prebuilt for 8 platforms** — macOS arm64/x64, Linux arm64/x64 (glibc and musl), Windows arm64/x64\n- **Type-2 clones** —`--ignore-identifiers` ,`--ignore-literals` and`--ignore-annotations` find blocks that differ only in names, literal values or annotations, reported as`renamed` (see[docs](/kucherenko/jscpd/blob/master/docs/rust.md#type-2-clones-renamed-identifiers-literals-and-annotations) )\n- **Type-3 near-miss clones** —`--max-gap-lines N` merges a copy with a few inserted or changed lines into one`similar` clone with a similarity score;`--similarity 0.85` compares whole JavaScript/TypeScript functions by syntax-tree structure, so renames and scattered edits are still caught (see[docs](/kucherenko/jscpd/blob/master/docs/rust.md#type-3-clones-near-miss-merging-with---max-gap-lines) )\n- **Clone kinds in every reporter** —`exact` ,`renamed` or`similar` in the console, JSON (`kind` ,`similarity` ,`method` ), XML, HTML, Xcode, SARIF (`jscpd/duplicate-code` ,`jscpd/renamed-code` ,`jscpd/similar-code` ) and Code Climate output; default runs report only`exact` clones and are unchanged\n- **15 reporters** :`console` ,`console-full` ,`json` ,`xml` ,`csv` ,`html` ,`markdown` ,`badge` ,`sarif` ,`codeclimate` ,`openmetrics` ,`ai` ,`xcode` ,`threshold` ,`silent`\n- **Clone baseline** — gate CI on*new* duplication only.`--baseline .jscpd-baseline.json` with`--fail-on-new-clones[=N]` tolerates legacy clones and fails the build on regressions;`--baseline-from-ref origin/main` does the same without a committed file (see[docs](/kucherenko/jscpd/blob/master/docs/rust.md#baseline) )\n- **Exit codes you can gate on** — an unknown`--format` , a missing scan path and a reporter that cannot write its file exit 1 instead of passing with an empty report;`--fail-on-empty` fails a scan that analyzed no files (see[Exit codes](/kucherenko/jscpd/blob/master/docs/rust.md#exit-codes) )\n- **GitLab-ready reporters** —`codeclimate` (`gl-code-quality-report.json` ) and`openmetrics` (`jscpd-metrics.txt` ) plug into`artifacts:reports`\n- **Git blame** with side-by-side author comparison (`--blame --reporters console-full` )\n- **`--summary`** — codebase summary: top files and folders by tokens, lines, size, and a complexity estimate — refactoring hotspots straight from the scan (see[docs](/kucherenko/jscpd/blob/master/docs/rust.md#summary) )\n- **`--mcp`** — built-in MCP server over stdio with fully described tools: point your AI assistant at the binary and it can check snippets for duplication against your codebase, or find structurally similar functions with a`similarity` argument (see[docs](/kucherenko/jscpd/blob/master/docs/ai-ready.md#stdio-transport-rust-v5) )\n- **AI reporter** — token-efficient output for LLM pipelines (~79% fewer tokens than console)\n- **`--skip-local`** — report only clones that cross the scan roots: with` jscpd packages/api packages/web --skip-local` , pairs inside one of the two trees are dropped and only api-to-web duplication remains\n- **`--skip-isolated`** — ignore duplication between monorepo folders owned by different teams (`--skip-isolated \"packages/team-a|packages/team-b\"` )\n- **`--workers`** — control parallelism for file tokenization and detection (default: all CPU cores)\n- **Config discovery** —`.jscpd.json` ,`.config/jscpd.json` , or the`jscpd` key in`package.json`\n- **Quiet in pipelines** — tips and sponsor lines print only on an interactive terminal;`--no-tips` ,`CI` or`JSCPD_NO_TIPS` switch them off everywhere\n\nSee the [Rust docs](/kucherenko/jscpd/blob/master/docs/rust.md) for the full CLI reference and [`rust/CHANGELOG.md`](/kucherenko/jscpd/blob/master/rust/CHANGELOG.md) for release notes.\n\njscpd v4 (TypeScript engine, Node.js API, LevelDB/Redis stores) is maintained on the [`master-v4`](https://github.com/kucherenko/jscpd/tree/master-v4) branch and published as `jscpd@4` / the `latest-4` dist-tag. [README-v4.md](/kucherenko/jscpd/blob/master/README-v4.md) describes it in one page (install, CLI, API, packages, maintenance policy); the same content is at [https://jscpd.dev/getting-started/v4](https://jscpd.dev/getting-started/v4).\n\n| Package | Registry | Description | \n|---|---|---|\n| [jscpd](/kucherenko/jscpd/blob/master/rust/jscpd) | [npm](https://www.npmjs.com/package/jscpd) | Installs the `jscpd` command (prebuilt binary via platform packages) | \n| [cpd](/kucherenko/jscpd/blob/master/rust) | [npm](https://www.npmjs.com/package/cpd) | Installs the `cpd` command (same binary) | \n| [jscpd-<platform>](/kucherenko/jscpd/blob/master/rust/npm) | npm | Platform binary packages pulled in as optional dependencies: `jscpd-darwin-arm64` ,`jscpd-darwin-x64` ,`jscpd-linux-x64-gnu` ,`jscpd-linux-arm64-gnu` ,`jscpd-linux-x64-musl` ,`jscpd-linux-arm64-musl` ,`jscpd-windows-x64-msvc` ,`jscpd-windows-arm64-msvc` | \n| [jscpd](/kucherenko/jscpd/blob/master/rust/scripts/build-pypi-wheels.py) | [PyPI](https://pypi.org/project/jscpd/) | Platform wheels repacked from the release binaries; installs both `jscpd` and`cpd` commands | \n| [jscpd](/kucherenko/jscpd/blob/master/rust/crates/cpd) | [crates.io](https://crates.io/crates/jscpd) | CLI crate; installs both `jscpd` and`cpd` binaries | \n| [cpd-core](/kucherenko/jscpd/blob/master/rust/crates/cpd-core) | [crates.io](https://crates.io/crates/cpd-core) | Detection algorithm (Rabin-Karp rolling hash), data models | \n| [cpd-tokenizer](/kucherenko/jscpd/blob/master/rust/crates/cpd-tokenizer) | [crates.io](https://crates.io/crates/cpd-tokenizer) | Source code tokenization (224 formats) | \n| [cpd-finder](/kucherenko/jscpd/blob/master/rust/crates/cpd-finder) | [crates.io](https://crates.io/crates/cpd-finder) | File walking, orchestration, git blame — the library entry point | \n| [cpd-reporter](/kucherenko/jscpd/blob/master/rust/crates/cpd-reporter) | [crates.io](https://crates.io/crates/cpd-reporter) | Output formatting (15 reporters) | \n\nThe `jscpd` npm package is downloaded **10M+ times per month**, and [~5,000 repositories](https://github.com/kucherenko/jscpd/network/dependents) declare it on GitHub's dependents graph.\n\n**Bundled by analysis platforms:**\n\n- [GitHub Super Linter](https://github.com/super-linter/super-linter) — official GitHub linter aggregator, bundles jscpd as its copy/paste detector and runs it by default; 15,500+ workflow files on GitHub reference Super Linter (as of Sep 2026)\n- [MegaLinter](https://github.com/oxsecurity/megalinter) — open-source linter aggregator for CI, ships jscpd in every flavor including`ci_light`\n- [Codacy](https://www.codacy.com/) — automated code analysis platform, jscpd powers the duplication engine\n\n**Explicitly enabled in Super Linter** (`VALIDATE_JSCPD: true`) **by dozens of public repositories, including:**\n\n- [A2A](https://github.com/a2aproject/A2A) — Google's Agent2Agent protocol (25k+ stars)\n- [RimSort](https://github.com/RimSort/RimSort) — mod manager for RimWorld (1.2k+ stars); also runs jscpd directly with its own`.jscpd.json`\n- [Contact Center AI samples](https://github.com/GoogleCloudPlatform/contact-center-ai-samples) — official Google Cloud samples, with a dedicated jscpd config\n- [Drifty](https://github.com/SaptarshiSarkar12/Drifty) — open-source download manager\n\n**Used in notable projects:**\n\n- [OpenClaw](https://github.com/openclaw/openclaw) — personal AI assistant, runs jscpd as a duplication gate in its check scripts\n- [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) — DeepSeek's plugin harness, jscpd config in CI\n- [degit](https://github.com/Rich-Harris/degit) — Rich Harris's project scaffolder\n- [MEGA webclient](https://github.com/meganz/webclient) — the MEGA.nz web client\n- [Microsoft TypeAgent](https://github.com/microsoft/TypeAgent)\n- [Salesforce DX VS Code](https://github.com/forcedotcom/salesforcedx-vscode)\n- [Alibaba AppWorks](https://github.com/apptools-lab/AppWorks) — embeds jscpd as a library\n- [OVHcloud manager](https://github.com/ovh/manager) — OVHcloud's customer control panel\n- [KiroCrew](https://github.com/kirodotdev/KiroCrew) — self-improving persistent development workspace\n\nCompared against other copy/paste detectors on the `fixtures/` corpus (547 files, 150+ formats), default thresholds, wall-clock time on Apple Silicon:\n\n| Tool | Time | Files | Clones | Dup Lines | \n|---|---|---|---|---|\n| jscpd | 84ms | 347 | 212 | 9,133 | \n| jscpd-rs | 111ms | 360 | 222 | 10,317 | \n| Duplo | 162ms | 319 | 518 | 13,049 | \n| Fallow dupes | 164ms | 34 | 10 | 3,137 | \n| Simian | 964ms | 547 | 424 | 15,351 | \n| PMD CPD | 35.980s | 71 | 56 | 2,267 | \n\nMethodology, cross-format detection and AI-token-efficiency comparisons: [benchmark/BENCHMARK.md](/kucherenko/jscpd/blob/master/benchmark/BENCHMARK.md). Re-run with [`benchmark/benchmark.sh`](/kucherenko/jscpd/blob/master/benchmark/benchmark.sh).\n\njscpd integrates into AI-powered workflows through three mechanisms:\n\nToken-efficient output for LLM pipelines (~79% fewer tokens than the default console reporter):\n\n```\njscpd --reporters ai /path/to/source              # compact clone list\njscpd --reporters ai --summary /path/to/source    # + compact codebase summary\n```\n\nTwo installable skills that teach AI coding assistants how to use jscpd and refactor detected duplications:\n\n| Skill | Purpose | Install | \n|---|---|---|\n| [`jscpd`](/kucherenko/jscpd/blob/master/skills/jscpd/SKILL.md) | Tool reference — CLI options, AI reporter format, config syntax | `npx skills add kucherenko/jscpd --skill jscpd` | \n| [`dry-refactoring`](/kucherenko/jscpd/blob/master/skills/dry-refactoring/SKILL.md) | Guided refactoring workflow — read clones, choose strategy, apply, verify | `npx skills add kucherenko/jscpd --skill dry-refactoring` | \n\nAfter installation, ask your agent to \"find and fix code duplication\" and it will invoke jscpd with the right options and act on the results.\n\n`jscpd --mcp /path/to/project` scans once and serves the Model Context Protocol over stdio, so an assistant can check any snippet for duplication against the codebase on demand, list a file's clones, re-scan the working directory, and look for structurally similar functions by passing `similarity`.\n\nSee [AI-Ready docs](/kucherenko/jscpd/blob/master/docs/ai-ready.md) for full details.\n\nSee [CONTRIBUTING.md](/kucherenko/jscpd/blob/master/CONTRIBUTING.md) for the development setup, test policy, and pull request requirements. In short:\n\n```\ncd rust\ncargo nextest run --workspace\ncargo clippy --workspace --all-targets -- -D warnings\ncargo fmt --all --check\n```\n\nSecurity issues go through the [security policy](/kucherenko/jscpd/blob/master/SECURITY.md), not public issues.\n\nThank you to all our backers! 🙏 [[Become a backer](https://opencollective.com/jscpd#backer)]\n\nSupport this project by becoming a sponsor. Your logo will show up here with a link to your website. [[Become a sponsor](https://opencollective.com/jscpd#sponsor)]\n\n[MIT](/kucherenko/jscpd/blob/master/LICENSE) © Andrey Kucherenko", "url": "https://wpnews.pro/news/show-hn-jscpd-start-support-type2-type3-clones", "canonical_source": "https://github.com/kucherenko/jscpd", "published_at": "2026-09-11 08:55:37+00:00", "updated_at": "2026-09-11 09:01:46.968459+00:00", "lang": "en", "topics": ["developer-tools", "ai-tools", "ai-agents"], "entities": ["jscpd", "kucherenko", "Rust", "oxc", "GitHub Code Scanning", "npm", "PyPI", "crates.io"], "alternates": {"html": "https://wpnews.pro/news/show-hn-jscpd-start-support-type2-type3-clones", "markdown": "https://wpnews.pro/news/show-hn-jscpd-start-support-type2-type3-clones.md", "text": "https://wpnews.pro/news/show-hn-jscpd-start-support-type2-type3-clones.txt", "jsonld": "https://wpnews.pro/news/show-hn-jscpd-start-support-type2-type3-clones.jsonld"}}