{"slug": "open-source-software-is-slowly-turning-into-open-slop-software", "title": "Open source software is slowly turning into open slop software", "summary": "An independent evaluation of the open source code review tool OpenQodex found it detected 31% of known bugs with 86% of its findings judged real, compared with Greptile's 69% detection rate and 68% real-finding rate, according to results published by the shadowclone project. The author's own shadowclone cloud bot found 38% of known bugs with 96% of findings judged real, and the local version found 36% with 91% judged real, with local reviewers run 3 times per PR. The author said the results were published at github.com/theonly1me/shadowclone/tree/main/evals/code-review after seeing repeated ads calling OpenQodex a \"CodeRabbit killer,\" arguing that open source tools claiming to replace paid products should show comparison evidence.", "body_md": "Code has become extremely cheap to produce, if you have a 5x claude subscription, you can produce or at least try to produce any piece of software you want. And, to be honest, I actually like that, I feel more people should build their own tools and release them as open source because everyone gets to turn their ideas into real products, for their own use if not for others.\n\nBut this also means it is getting easier to copy a product, put it on public GitHub and announce you've killed the paid version. Being open source shouldn't give you a pass for shipping slop. If you are calling your tool a replacement for something people pay for, show how it compares.\n\nIn the past week, each time I opened X, I kept seeing ads about a new open source code review tool claiming to be the \"CodeRabbit killer\". I have used CodeRabbit in the past and it definitely has its shortcomings, but if you're claiming you've killed a product, there needs to be evidence.\n\nI saw the advertisement so many times that I decided to evaluate how the product actually performs, it also gave me an excuse to add review capabilities into my own open source tool called shadowclone which helps me manage my agent skills and ensure that regardless of the harness or model I use, agents consistently follow my engineering preferences.\n\nI ran an eval using Greptile, OpenQodex (the open source tool I wanted to evaluate), and my own implementation of a code reviewer in shadowclone, which is basically a code review skill, a local reviewer and a Github bot that can review on the cloud.\n\nMy bot reported a higher proportion of findings that both model judges considered real, but it missed 4 real bugs that Greptile caught and as for OpenQodex, I will let the results speak for themselves.\n\n1. Greptile found 69% of the known bugs and 68% of its findings were judged as real 2. shadowclone cloud bot found 38% and 96% of the findings were judged as real 3. shadowclone locally found 36% and 91% of the findings were judged as real 4. OpenQodex found 31% and 86% of its findings were judged as real\n\nThe local reviewers ran 3 times per PR. You can see the full eval results at https://github.com/theonly1me/shadowclone/tree/main/evals/code-review\n\nThe harness is open source too and you can re-run the eval if you like.\n\nI still want more people to build and release their own tools. I am doing it too. But, if you advertise yours as a replacement for an existing product, I want to see what you tested and where it falls short. Mine missed bugs Greptile caught and publishing that gives me something useful to work on, calling it a Greptile killer wouldn't.\n\nComments URL: [https://news.ycombinator.com/item?id=50043719](https://news.ycombinator.com/item?id=50043719)\n\nPoints: 2\n\n# Comments: 1", "url": "https://wpnews.pro/news/open-source-software-is-slowly-turning-into-open-slop-software", "canonical_source": "https://news.ycombinator.com/item?id=50043719", "published_at": "2026-10-11 14:33:00+00:00", "updated_at": "2026-10-11 14:57:19.824371+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "ai-agents", "artificial-intelligence"], "entities": ["OpenQodex", "Greptile", "shadowclone", "CodeRabbit", "GitHub", "X"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/open-source-software-is-slowly-turning-into-open-slop-software", "markdown": "https://wpnews.pro/news/open-source-software-is-slowly-turning-into-open-slop-software.md", "text": "https://wpnews.pro/news/open-source-software-is-slowly-turning-into-open-slop-software.txt", "jsonld": "https://wpnews.pro/news/open-source-software-is-slowly-turning-into-open-slop-software.jsonld"}}