{"slug": "a-ruby-type-checker-has-found-15-bugs-in-discourse-rails-zammad-and-ruby-lsp", "title": "A Ruby type checker has found 15 bugs in Discourse, Rails, Zammad and Ruby-lsp", "summary": "Itaruby, a Ruby type checker built in Rust, has found 15 bugs across Discourse, Rails, Zammad, and ruby-lsp, with 7 accepted, according to its developer. The checker, which is not yet released, uses inference and has scanned 20,615 files, producing 13,208 raw diagnostics, with submissions verified by execution. It runs as a language server and uses the salsa incremental recomputation engine.", "body_md": "itaruby, not released yet\n\n#\nA Ruby type checker that has already found 15 bugs in Discourse, Rails, Zammad and ruby-lsp\n\nitaruby is a Ruby type checker I am building. Instead of announcing what\nit promises to do, this page shows what it has already done: 14 submissions, 3 in discourse/discourse, 5 in rails/rails, 5 in zammad/zammad and 1 in Shopify/ruby-lsp, each bug verified by\nrunning it against the real file in the project before it was\nsubmitted. Discourse is a forum, Rails is a framework, Zammad is a helpdesk and ruby-lsp is a language server: different domains, no special-casing for\nany of them. Of the total, 7 have been accepted.\n\nWhat it is\n\nFrom the initial commit's own description: \"inference-first Ruby type checker in Rust\".\nIt infers instead of\nrequiring you to declare. 4 crates: itaruby (the CLI),\nitaruby_syntax, itaruby_semantic and itaruby_server (the language\nserver). 98 .rs files, 15,075 lines of production\nRust and 13,793 of test code,\nclose to a test line for every line of production code.\nIncremental recomputation engine: salsa, the same one\nrust-analyzer uses. Beyond the command-line check, it runs as a\nlanguage server: definition and hover at `file:line:column`\n\n.\n\nThe initial commit is from August 25, 2026, 18:18 UTC-3. The most\nrecent one measured here is from August 26, 2026, 08:30 UTC-3: 26 commits over two days (8\non the first, 18 on the second), with no\nrewritten history. By the time the 14 submissions on\nthis page went out, the checker was about 14 hours old.\n\nIt advances in waves, each tested against a mutation harness that\ndeliberately breaks the code to prove the tests would catch it: wave 7 caught 16 mutants.\n\n## The funnel\n\n13208 raw diagnostics, 14\nsubmissions, across four triages from different moments, not one\nsnapshot: rails, discourse and zammad in a round before wave 6;\nruby-lsp in wave 6, during the Sorbet comparison. Most of the 13208 is noise: the same unresolved constant,\nflagged again on every reference in the code. The funnel from one\nnumber to the other is manual. I read every diagnostic, group the\nones that share a cause, prove by execution whatever is left, and\nonly then does it become a submission. None of the 13208 became a submission on its own.\n\nWhat this number deliberately leaves out: twelve pull requests I\nopened on software-factory, a project that was part of this same\nrun and the thing that got me unblocked enough to open the rest.\nThey are excluded because the checker did not find them. Adding the\ntwo together would suggest itaruby surfaced twenty-six things, and\n\"every submission proved by execution\" would stop being true of\nhalf of them. I would rather publish the smaller number that is\ntrue all the way through.\n\n| Repository | Files | Diagnostics | Round | Submissions |\n| rails/rails | 3457 | 2364 | before wave 6 | 5 |\n| discourse/discourse | 11201 | 7959 | before wave 6 | 3 |\n| zammad/zammad | 5586 | 2834 | before wave 6 | 5 |\n| Shopify/ruby-lsp | 371 | 51 | wave 6 (7d3a781) | 1 |\n| Total | 20615 | 13208 | | 14 |\n\nThe rails, discourse and zammad counts are from the moment of triage,\na round before wave 6 whose binary sha was never recorded. The\nruby-lsp count is from wave 6, commit `7d3a781`\n\n. None of\nthese numbers change afterward: today's raw count, which is lower,\nis in the \"Wave 8\" section further down, sha `5c5c8c3`\n\n. File counts were reconfirmed stable\nacross rounds; scan time is not in this table because it would pair\none round's time with another round's count, the same mistake this\npage calls out elsewhere. Time belongs in the \"Throughput\" section,\nwhich uses time and count from the same round.\n\n## Throughput\n\n11201 Discourse files scanned in 8,02s. No configuration, no\nannotation, no setup step: the requirement to run is zero.\n\n## The submissions\n\nFIX: two crashes in the Lithium import script\n\nTwo crashes in the Lithium import script. A `file.unlind`\n\nthat does not exist on `Tempfile`\n\n, swallowed by the surrounding `rescue`\n\n, leaking a temp file per imported avatar. And a `link_post_uploads`\n\ncalled on `CookedPostProcessor`\n\n, which has no such method: that one is not swallowed and takes down the entire attachment post-processing step.\n\nscript/import_scripts/lithium.rb\n\nFIX: forward force as a keyword argument in the TopicAssigner shim\n\nThe deprecated shim received `force:`\n\nas a kwarg and forwarded it positionally to a method that only accepts a positional argument. Any call blew up, including the simplest one, without `force`\n\n. Ships with the regression spec the class was missing.\n\nplugins/discourse-assign/lib/topic_assigner.rb, plugins/discourse-assign/spec/lib/topic_assigner_spec.rb\n\nFIX: pass all five arguments to replace_internal_link in Discuz import\n\nThe private-message-link branch called `replace_internal_link`\n\nwith 2 arguments, and the method requires 5. Every post whose body matched the regexp would crash the import with an ArgumentError.\n\nscript/import_scripts/discuz_x.rb\n\nRemove unreachable default for DerivedSecretKeyProvider#derive_key_from\n\nThe `using: key_generator`\n\ndefault points to a method that does not exist on the instance, so falling into it raises NameError. Unreachable: the method is private and the only caller passes the argument explicitly.\n\nactiverecord/lib/active_record/encryption/derived_secret_key_provider.rb\n\nRemove unused Active Record internals\n\nTwo dead code paths. An `else`\n\nbranch that falls into `expr`\n\n, a method that does not exist anywhere in Arel, and a `columns_hash`\n\nthat calls the connection without the argument it requires.\n\nactiverecord/lib/arel/nodes/homogeneous_in.rb, activerecord/test/cases/arel/support/fake_record.rb\n\nRemove unused `request` helper from RouteSetTest\n\nA helper that memoizes `TestRequest.new`\n\nwith no arguments, while the constructor requires three. Any call would raise ArgumentError. Nobody calls it.\n\nactionpack/test/controller/routing_test.rb\n\n[\n#58566 ](https://github.com/rails/rails/pull/58566) applied by commit\n+13 / -1 Fix QueryCacheRegistry#clear calling an undefined method\n\nThe method called `@map.synchronize`\n\n. The class's mutex is `@mutex`\n\n; `@map`\n\nis a Hash and does not respond to synchronize, so clearing the cache would blow up.\n\nactiverecord/lib/active_record/connection_adapters/abstract/query_cache.rb\n\nFinding applied in commit [ 22a02ee12e ](https://github.com/rails/rails/commit/22a02ee12e)\n, not through this PR's branch.\n\nFix RouteSetTest::SimpleApp#call referencing an undefined local variable\n\n`call`\n\nreturned `response`\n\n, a local variable that does not exist. The value is in `@response`\n\n. Any request routed to this test app raised NameError.\n\nactionpack/test/dispatch/routing/route_set_test.rb\n\nFix: Wrong number of arguments in create_knowledge_base_categories call\n\nA call passed 4 positional arguments to a method that takes a hash, raising `ArgumentError: wrong number of arguments (given 4, expected 1)`\n\n. The right shape is documented in the file's own docblock.\n\nlib/fill_db.rb\n\nFix: Compare limit against @size, not undefined size\n\nThe comparison used `size`\n\n, which does not exist in scope; only `@size`\n\nexists, used on four other lines. Raises `NameError`\n\nthe first time anyone chains `.limit`\n\nbefore `.as_batches`\n\n.\n\nconfig/initializers/active_record_as_batches.rb, spec/lib/core_ext/relation/as_batches_spec.rb\n\nRemove unused OnlineNotificationPolicy#relation\n\nThe method called a name that does not exist and nothing reaches it. Proven from pundit's own source, whose only dynamic dispatch is the action name plus a question mark.\n\napp/policies/online_notification_policy.rb\n\nFix: Auth::TwoFactor#initiate_authentication calls an undefined method\n\nThe method called `method_object`\n\ninstead of `authentication_method_object`\n\n, which exists and is used correctly on two other lines in the same class.\n\nlib/auth/two_factor.rb, spec/lib/auth/two_factor_spec.rb\n\nFix: Cleanup action job name misses job prefix on payload args\n\nOne line forgot the `job.`\n\nreceiver, so the bare identifier raises `NameError`\n\n; the neighboring lines in the same method already used the right receiver.\n\nlib/background_services/service/base_delayed_jobs/cleanup_action.rb, spec/lib/background_services/service/base_delayed_jobs/cleanup_action_spec.rb\n\nFix NameError raised by RubyIndexer's abstract method guard\n\nA `raise AbstractMethodInvokedError`\n\ninside `RubyIndexer::Entry::Member`\n\npoints to a constant that only exists as `RubyLsp::AbstractMethodInvokedError`\n\n, a sibling module, not an ancestor: the guard raises `NameError`\n\ninstead of the intended error. It is the only one of the 17 sites of this `raise`\n\nin the repository outside `lib/ruby_lsp/`\n\n; the other 16 resolve correctly. A latent defensive guard, not reachable today: both concrete subclasses of `Member`\n\nalready implement `signatures`\n\n.\n\nlib/ruby_indexer/lib/ruby_indexer/entry.rb\n\n## The gates\n\nOnce the PR is open, what decides whether it moves forward is not\nthe patch. Three gates, measured, not about the tool.\n\nShopify's CLA wants a human signature plus a comment that matches\nthe workflow's regex. My first comment, \"CLA signed.\", passed the\nworkflow's `if`\n\n, the job concluded `success`\n\n, and the log said `Comment does not match CLA pattern`\n\n: a green run does\nnot mean the gate is cleared. What worked was\n\"I have signed the CLA.\"\n\n`action_required`\n\nis first-contribution workflow\napproval, per repository: it blocks zammad (5 PRs) and ruby-lsp\n(1). Rails and Discourse do not have this gate.\n\nOn GitLab, quota and membership block even the `push`\n\n: that is why 6 ready MRs remain unopened.\n\nNone of the three gates have anything to do with patch quality,\nand none of them show up before the PR is open.\n\n## How each finding was verified\n\nA checker's finding is a hypothesis, not a bug. Every item on\nthis list went through two-sided proof against the project's\nreal file: run and fail before the fix, run and pass after.\nReproducing it in a separate file does not count, because that\ntests my understanding of the code, not the project's behavior.\n\nFor dead-code removals the bar is different: prove that nothing\nreaches that code. That means scanning the whole repository,\nincluding dynamic dispatch (`send`\n\n, `public_send`\n\n, `respond_to?`\n\n, `method_missing`\n\n, `delegate`\n\n) and the class's subclasses, not just\ndirect callers. That last part almost slipped past me on [\n#58569 ](https://github.com/rails/rails/pull/58569)\nin Rails: the class has two subclasses, and if either one had\ncalled the method the story would have flipped from cleanup to\nreal bug.\n\nThe shape of the PRs did not come from personal taste either. It\ncame from measuring what the project already accepts. I first\ncounted Rails's last hundred merges, found no dead-code removal\nand concluded there was no precedent. A hundred Rails merges\ncover two weeks: searching the full history turned up a\ntwelve-PR removal campaign merged a month earlier. A short\nsample does not measure a rare category.\n\nAcceptance is not always the same thing either. At Zammad, real\ndevelopment happens on an internal GitLab; GitHub is a mirror,\nand native merging there has been stalled since 2021. When a\nmaintainer accepts a fix, they apply the patch as a manual\ncommit and close the PR citing the hash. That is why the list\ndistinguishes `merged`\n\nfrom `applied by commit`\n\n: two ways of saying yes, without\nhiding which one it was.\n\n## Proof, not adjective\n\nRails [\n#58566 ](https://github.com/rails/rails/pull/58566)\nis the most complete case in this section: closed without a\nmerge, with a single comment, from `byroot`\n\n(Jean\nBoussier, Rails core): \"If it's never called, let's just\nremove the method entirely.\" That was not a rejection of the\nfinding, it was a rejection of the form: I sent a fix for a\nmethod that was dead, when removal was the right call. I had\nclassified the method as reachable without proving it. The\nproof, done afterward: `@thread_query_caches`\n\nappears in exactly two\nplaces in the repository, the instantiation and one call to\n`compute_if_absent`\n\n; `.clear`\n\nis never\ncalled. And the method's body called `synchronize`\n\non an object that does not have\nthat method, so it would raise an error if anyone called it:\ndead and broken, and I sent a fix instead of a removal.\nThe outcome is the `applied by commit`\n\nstate: byroot removed the method himself in commit [ 22a02ee12e ](https://github.com/rails/rails/commit/22a02ee12e)\n, literal message:\n\n```\nGet rid of `QueryCacheRegistry#clear`\n\nThe method is dead code.\n```\n\nWith a co-author credit for the finding. It went in, just not\nthrough my branch.\n\nOn Rails [\n#58568 ](https://github.com/rails/rails/pull/58568)\n, the method itaruby flagged had zero test coverage in the\nsuite. The first proof that the code was dead came from a `grep 'def expr'`\n\nagainst the repository, which\nwould never find an `attr_accessor :expr`\n\n. The\nright instrument was following the inheritance chain, not\ngrepping for the method name. The mistake was caught and\nfixed before the PR shipped, and the missing test went in\nwith it.\n\nDiscourse [\n#42898 ](https://github.com/discourse/discourse/pull/42898)\nshows the same kind of failure with opposite consequences, in\nthe same Lithium importer file: Two crashes in the Lithium import script. A `file.unlind`\n\nthat does not exist on `Tempfile`\n\n, swallowed by the surrounding `rescue`\n\n, leaking a temp file per imported avatar. And a `link_post_uploads`\n\ncalled on `CookedPostProcessor`\n\n, which has no such method: that one is not swallowed and takes down the entire attachment post-processing step.\n\nThree Zammad findings ([#6329](https://github.com/zammad/zammad/pull/6329), [#6331](https://github.com/zammad/zammad/pull/6331), [#6332](https://github.com/zammad/zammad/pull/6332)) shipped with a regression spec the suite did not have. In\nall three, the production fix is one line; the rest of the\ndiff, thirteen to fourteen lines, is test.\n\n## Where Sorbet wins\n\nWhat is proven: itaruby does not require anything, and it reads\ninline RBS if you have it. No configuration, no adoption step.\nThe 14 bugs on this page came out of 3 repositories with zero `sorbet/`\n\n, zero `.rbi`\n\nand zero `extend T::Sig`\n\n: 13 submissions,\n7 accepted. The remaining repository in\nthe corpus, ruby-lsp, runs Sorbet and has its own bug, in the\nnext section.\n\nScope note: from a Sorbet `sig`\n\nblock, itaruby picks\nup the return type; it does not pick up the parameter type. The\n`sig`\n\nalready speaks return, it still has to speak\nparameter. This matters less than it looks, because the corpus\nbarely uses `sig`\n\nat all. I tested it: a class with\n`extend T::Sig`\n\nand `sig { params(n: Integer).returns(String) }`\n\n,\ncalled passing a `String`\n\nwhere the signature wants\nan `Integer`\n\nparameter, produces no diagnostic at\nall: that is exactly the side that is still missing. But rails,\ndiscourse and zammad have zero `sig`\n\n. ruby-lsp, the\nmost typed repository in the corpus, has zero real production\n`sig`\n\n: the leftover `sig`\n\ntext in the\nrepository is documentation comments and spec fixtures, not\nannotation. tapioca has one, in `lib/`\n\n.\n\nThere is a sharper reason behind the zero above, and it is\nSorbet's own documented behaviour rather than an opinion about\nit. A file with no sigil defaults to `# typed: false`\n\n, and at that level Sorbet\nreports only syntax, constant resolution and `sig`\n\ncorrectness. Its manual is explicit that the\nlevel \"only prevents that method's body from being\ntypechecked\". So on a repository with no annotations, Sorbet is\nnot failing to find these bugs: it is never asked to look. That\nis the whole gap itaruby aims at, and it is also why the\ncomparison is about reach, not about speed. [sorbet.org/docs/static](https://sorbet.org/docs/static)\n\nWhat changes the read: itaruby reads inline RBS. It is the\nannotation the tapioca team itself started using in ruby-lsp's\nproduction code, instead of `sig`\n\n: a one-line comment\nabove the method, in the `#: (Type) -> Type`\n\nformat. I tested the same\nmethod in the three annotation styles:\n\n``` python\n# no annotation\ndef format(n)\n  n.to_s\nend\nformat(\"7\")\n# itaruby: silence (correct, nothing to go on)\n\n# with inline RBS\n#: (Integer) -> String\ndef format(n)\n  n.to_s\nend\nformat(\"7\")\n# itaruby: p1.rb:7:18: error[E0103]: argument 1 of 'format' expects Integer, got String\n\n# with Sorbet sig\nextend T::Sig\nsig { params(n: Integer).returns(String) }\ndef format(n)\n  n.to_s\nend\nformat(\"7\")\n# itaruby: silence\n```\n\nThe cost contrast is the strongest data point in this section.\nFor Sorbet to see the dependencies, ruby-lsp and tapioca keep\n754,669 lines of vendored `.rbi`\n\n(246,830 + 507,839), plus a runtime gem, plus a sigil per file, plus a generation\nstep. Inline RBS asks for none of that: it is a comment above\nthe method, zero vendored files, zero runtime gem, zero\ngeneration.\n\nRBS for core/stdlib, in a separate file, not inline: not\nconsumed. RBS for a gem via `rbs collection`\n\n, also in\na separate file: not consumed. RBI is not missing coverage, it\nis a lookup problem: a declaration inside `.rbi`\n\nwritten as a nested block indexes correctly, the full path\nexists in the index; the query is what fails, looking up the\nqualified path in a map where that name was registered\nunqualified, and giving up before applying the refilter that\nexists exactly for this. Only the compact `module X::Y`\n\nform avoids the problem. It is the\nsame bug that explains why the alias fix cured ruby-lsp and not\ntapioca, in the \"The cycle\" section further down.\n\nThis table changed since this page's original publication:\nruby-lsp went from worst to best in the corpus after a fix in\nitaruby itself, not in the scanned code. The mechanism and the\nverified prediction are in \"The cycle\" section, right after\n\"Head-to-head with Sorbet\".\n\n| Repository | Diagnostics / 1,000 files |\n| ruby-lsp | 137 |\n| tapioca | 275 |\n| zammad/zammad | 505 |\n| rails/rails | 670 |\n| discourse/discourse | 704 |\n\nWhat holds up, before and after the fix: neither \"worst\" nor\n\"best\" measured typing. They measured how much repository-specific\nparsing bug itaruby still had. The contract that survives is not\n\"works better with more types\": it is no requirement at all, and\nit reads inline RBS when the code already has it.\n\n## Head-to-head with Sorbet\n\nThe hardest test I can run: pit itaruby against Sorbet on the\nonly two repositories in the exploratory corpus that have\ngenuinely adopted Sorbet. Global `srb tc`\n\n, no `bundle install`\n\n, on both:\n\n| Tool | Repository | Files | Time | Diagnostics |\n| srb tc | ruby-lsp | 235 | 0,172s\n| 0 |\n| ita check | ruby-lsp | 371 | 0,496s\n| 51 |\n| srb tc | tapioca | 370 | 0,242s\n| 0 |\n| ita check | tapioca | 218 | 0,664s\n| 60 |\n\nAll four time measurements are from the same session, same\nmachine, median of three runs each. Sorbet is 2,9x faster on ruby-lsp and 2,7x on tapioca. Wall-clock time is not\ncomparable across different sessions on this page: in an\nearlier remeasurement, `ita check`\n\non ruby-lsp took\n445ms; in this one, 496ms, with no binary change between the\ntwo. Session-to-session variance of 10% to 20% on this machine,\nwhich rules out any reading of \"the fix cost time\" without a\nbefore/after measurement taken in the same round.\n\nFile counts differ because `srb`\n\napplies the\nproject's `sorbet/config`\n\n`--ignore`\n\nlist (fixtures, vendor) and counts `.rbi`\n\nfiles\nalongside them; itaruby counts raw `.rb`\n\nfiles and\nhas no equivalent ignore list.\n\nSorbet is green on both: literal output `No errors! Great job.`\n\n. itaruby, on the same code,\nproduces 111 raw diagnostics. I did\nnot find 111 bugs. I classified both\nfull sets, item by item, not a sample:\n\n| Repository | Diagnostics | Real bug | False positive | Genuine noise |\n| ruby-lsp | 51 | 1 | 16 (31%)\n| 34 (67%)\n|\n| tapioca | 60 | 0 | 34 (57%)\n| 26 (43%)\n|\n\n`srb tc`\n\nremains at zero errors on both.\n\nOne number improved and the other got worse, and both deserve\nequal billing: in the earlier sample of 10 for ruby-lsp above,\n7 of 10 were false positives (70%); in the exhaustive\nclassification of 51\n, that fell to 31%. tapioca went the other way: it became the repository with\nthe highest false-positive share in the corpus, 57%. It was not a terminal state: the cause had a name, an address in\nthe code, and the fix went in. The result is in the \"Wave 8\"\nsection further down: tapioca dropped from 17 to 1, a documented ceiling, not\na reclassification to zero.\n\nI rechecked the original 10 ruby-lsp items\nagainst the exhaustive classification: 2 disappeared (they\nwere false positives, resolved by the alias fix), 8 remain, and 1 of\nthose changed classification because the earlier evidence had\nchecked the wrong directory. A measurement mistake on our side,\nnot a tool behavior.\n\nFalse positives have a cause. The first time I measured that\ncause, the number came out wrong; the corrected version is\nbelow. I reclassified the 41 constant\nwarnings left in ruby-lsp after the alias fix, by mechanism:\n\n| Mechanism | Warnings | What it is |\n| Declared as T.let in already-committed RBI | 11 | Test::Unit::* (7), LanguageServer::Protocol::Constant::* (3), RuboCop::Version::STRING (1) |\n| Exists in on-disk core/stdlib RBS | 9 | Minitest::Spec (7), OptionParser and one from Gem:: |\n| Deliberate fixture in the repository itself | 11 | fake constants ruby-lsp's own tests define on purpose |\n| Rest | 10 | includes the entry.rb bug that became PR #4197 |\n\nThe two type-info levers, already-committed RBI and RBS from\ncore/stdlib on disk, are close to the same size: 11 against 9.\nNeither one alone resolves half of the 41.\nThere is no single saving type source here: it is always both\ntogether, and even both together do not cover everything.\n\n11 of the 41 are fixture: fake constants\nthat ruby-lsp itself defines on purpose to test its own syntax\ndiagnostic detector. The project's `sorbet/config`\n\nexcludes that folder via `--ignore`\n\n; itaruby does\nnot honor that `--ignore`\n\n. That is 27% of what is left being itaruby reporting on a file the\nproject's own maintainers already declared out of scope, a\nconcrete and verifiable limitation.\n\nAbout the method, because it is the foundation of every number\non this page: the first measurement of this mechanism had two\nmistakes. The original denominator, 160 unique unresolved\nconstants, dropped to 29 as soon as the alias-caused ones left\nthe list with the fix in commit `7d3a781`\n\n. And the original numerator,\n78, was also wrong: the regex matched a constant by name\nsuffix, not full path, so names like `A`\n\nand `C`\n\ncounted because they happened to end the same\nway as a real `T.let`\n\ndeclaration somewhere else in\nthe file. Both mistakes only surfaced when I swapped grep for a\ndirect query against the RBS API, loading the 61 stdlib\nlibraries and asking by type name instead of by text. The\ncorrect number, after that, is the one in the table above:\n11 of 41, 27%. Swapping grep for the API fixed the suffix bug, but was not\nenough on its own: the query accepted a shortcut, a top-level\nname existing already counted as a match, so `Gem`\n\nexisting was enough for `Gem::DefaultUserInteraction`\n\nto pass, when that\nconstant's only real occurrence in the entire RBS is a\ncomment. The full lesson is not \"API instead of text\": it is\nthat the question has to be about the exact path, because\naccepting a prefix is suffix grep dressed up as an API.\n\nThe 7 ruby-lsp diagnostics that are not\nconstant warnings have a complete partition, verified against\nthe source, not a sample:\n\n| Mechanism | Diagnostics | Where |\n| Flow narrowing | 3 | declaration_listener.rb:512 and :515 (is_a? in a ternary), code_action_resolve.rb:435 (case without else) |\n| Generic [T] in the signature | 3 | erb_document_test.rb:161, ruby_document_test.rb:906 and :1010 (all in cache_set) |\n| Cast #: as untyped ignored | 1 | addon_test.rb:149 |\n\nThe `code_action_resolve.rb:435`\n\ncase deserves its\nown line: the `#: as !nil`\n\ncast stuck to the\n`end`\n\non line 417 is read correctly, that is\nconfirmed. What is missing is narrowing the type before the\ncast is reached: lines 410 to 417 are a `case @code_action[:title]`\n\nwith three `when`\n\nbranches and no `else`\n\n, and\nwithout that branch the type left over is `String | nil`\n\n. The cast is not ignored: it is\napplied to a type that already arrived wrong.\n\nNot every row in the table is a bug. Narrowing on a `case`\n\nwithout `else`\n\nis a family\ndeclared out of scope by decision, a chosen boundary, not a\nflaw waiting to be found. The generic `[T]`\n\ntreated as a fixed type is a real bug, and it is the third\ntime this mechanism has shown up in this investigation: it\nalready has a prioritized fix. The `#: as untyped`\n\ncast being ignored remains unfixed.\n\nThe framing that closes this section: the value is in\nuntyped code. Across the 3\nuntyped repositories, the funnel produced 14 real bugs, 13 submissions, 7\naccepted. Across the two repositories with Sorbet, Sorbet is\nalready green and itaruby produces mostly false positives.\nThat is scope, not defeat: where Sorbet is already configured\nand green, itaruby today does not add, it gets in the way.\n\nThe one real bug itaruby found across the two repositories\nwith Sorbet, out of 111 raw\ndiagnostics in those two repositories. Not a win, it is the\nexception that proves the scope. The bug: a `raise AbstractMethodInvokedError`\n\ninside `module RubyIndexer`\n\nwhen the constant only exists\nas `RubyLsp::AbstractMethodInvokedError`\n\n, a sibling\nmodule, not an ancestor. The `raise`\n\nnever\nresolves; I reproduced the structure separately and confirmed\nit: it raises `NameError`\n\n. Of the 17 sites of this\nsame `raise`\n\nin the repository, 16 are in `lib/ruby_lsp/`\n\nand resolve correctly; only this one\nis outside the right namespace. It is a latent defensive\nguard, not a production crash, because both concrete\nsubclasses already implement the abstract method.\nThe PR is open, [\n#4197 ](https://github.com/Shopify/ruby-lsp/pull/4197)\n, on ruby-lsp: 2 lines added,\nnone removed. Contributions from outside Shopify go\nthrough two gates before shipping: the company's CLA and\nfirst-contribution workflow approval, both pending now.\n\nThe most interesting data point in this section: this bug was\nin a repository that runs Sorbet in CI, green, and Sorbet does\nnot catch it because the project's own `sorbet/config`\n\n`--enable-experimental-rbs-comments`\n\nflag silences\nthis kind of guard. It is not \"itaruby is stronger than\nSorbet\": it is a configuration flag that created a blind spot,\nand a tool that does not read the project's configuration\nwent right through it without knowing it existed. That is the\nreal argument for running two different tools instead of one.\n\nOverlap between the two real-error sets: zero. That is not a\nfinding, it is an absence of data: Sorbet's set in the two\nrepositories is empty, so there is no intersection to\nmeasure. Not a win, it is zero degrees of freedom in the\ncomparison.\n\n## The cycle\n\nThe rate table above showed ruby-lsp at 1240 diagnostics per thousand files,\nthe worst in the corpus, while being the most typed\nrepository. Counterintuitive enough to be worth investigating\nthe cause instead of taking the number at face value.\n\nThe cause has a name and a line, in `lib/ruby_lsp/utils.rb:6-7`\n\n:\n\n```\nInterface = LanguageServer::Protocol::Interface\nConstant = LanguageServer::Protocol::Constant\n```\n\nitaruby did not follow this kind of constant alias; every\nreference to `Interface::X`\n\nor `Constant::X`\n\nin the rest of the file turned into\nan unresolved-constant diagnostic. That accounted for 405 of the 449 constant warnings in\nthe repository.\n\nPrediction made before touching the code: about 405 warnings should disappear\nwith the fix. After the fix, commit `7d3a781`\n\n: 408 disappeared. Prediction error\nof 0,7%.\n\nThat conclusion came from comparing totals per repository,\nbefore and after, and comparing totals does not establish a\nclaim about a set: I made this exact mistake myself later, in\nwave 7, where rails's total came out identical between two\nwaves, 2316 against 2316, hiding 5 diagnostics leaving and 5\nentering. The set-level diff for this alias fix was never\nrun, and I no longer have that wave's binary to run it now: I\ncannot say whether any new diagnostic showed up anywhere in\nthe corpus.\n\n| Repository | Diagnostics before | Diagnostics after | Diag/1k before | Diag/1k after |\n| ruby-lsp | 456 | 51 | 1240 | 137 |\n| tapioca | 60 | 60 | 289 | 275 |\n| rails/rails | 2364 | 2316 | 684 | 670 |\n| discourse/discourse | 7959 | 7884 | 711 | 704 |\n| zammad/zammad | 2834 | 2822 | 507 | 505 |\n\nMeasured at commit `7d3a781`\n\n, same\nfile count as before.\n\nWhat survives this comparison is magnitude, not the set: the\nuntyped repositories barely moved, net deltas of rails -48, discourse -75, zammad -12, while the typed\nrepository dropped 405 (also a net delta). If the gain had come spread out\nevenly, that would be cause for suspicion: a generic fix that\nreduces noise everywhere equally proves no mechanism at all.\nHere the gain landed exactly where the mechanism predicted it\nwould.\n\nWhy did the fix cure ruby-lsp and leave tapioca untouched, when\nboth have the same kind of alias? I isolated it with a 9-line\ntest, three variations of the same alias, changing only where\nthe target is declared:\n\n| Case | Target declared in | Form | Result |\n| A | .rb file | nested block | resolves |\n| B | .rbi file | nested block | fails |\n| C | .rbi file | compact module X::Y | resolves |\n\nIn a `.rbi`\n\nfile, a declaration written as a nested\nblock indexes correctly: the full path exists in the index.\nThe query is what fails, looking up the qualified path in a\nmap where that name was registered unqualified, and giving up\nbefore applying the refilter that exists exactly for this.\nOnly the compact form avoids the problem. In `.rb`\n\nboth work. The two real lines, side by side:\n\n```\n# ruby-lsp, lib/ruby_lsp/utils.rb:7 (alias in .rb, cured)\nConstant = LanguageServer::Protocol::Constant\n\n# tapioca, sorbet/rbi/gems/ruby-lsp@0.26.10.rbi:2380 (alias in .rbi, not cured)\nRubyLsp::Constant = LanguageServer::Protocol::Constant\n```\n\nThe difference between the two repositories is not a code\nprofile, it is a lookup bug with an address. And the\nconsequence that matters to a reader: fixing this is not\nincreasing coverage, it is knocking down false positives in\nthe repository where the tool looks worst.\n\nThe reclassification closed, exhaustive, item by item, not a\nsample of 20. This replaces the provisional \"undecided\"\nreading I had left here: on ruby-lsp, false positives dropped\nfrom 70% (7 of 10, old sample) to 31% (16 of 51, full count);\non tapioca it went the other way and it became the\nrepository with the highest false-positive share in the\ncorpus, 57% (34 of 60). `srb tc`\n\nremains at zero errors on both. The\nsentence from two hours earlier, \"where Sorbet is already\nconfigured and green, itaruby today does not add, it gets in\nthe way\", no longer holds as a single sentence for both\nrepositories: on ruby-lsp it got weaker, false positives are\nalready a minority (31%); on tapioca it got stronger, false\npositives are a clear majority (57%), and it was not a\nterminal state: the cause was the `.rbi`\n\nlookup bug\nabove, it had an address in the code, and the fix went in.\nThe result is in the \"Wave 8\" section: tapioca dropped from\n17 to 1, a documented ceiling.\n\n## Wave 8\n\nWave 8 replaces wave 7 as the current snapshot: binary\ncompiled from commit `5c5c8c3`\n\n, 2026-08-26, same trees pinned by sha\nas before. Composition by diagnostic code, all six\nrepositories in the corpus:\n\n| Repository | Total | Composition |\n| ruby-lsp | 30 | 24 E0104, 3 E0103, 3 E0001 |\n| tapioca | 1 | 1 E0104 |\n| rails/rails | 2302 | 2079 E0104, 222 E0101, 1 E0102 |\n| discourse/discourse | 6217 | 6200 E0104, 12 E0101, 5 E0102 |\n| zammad/zammad | 2821 | 2611 E0104, 207 E0101, 3 E0102 |\n| chatwoot/chatwoot | 2915 | 2915 E0104 |\n\nchatwoot had its first measurement in wave 7. This is the\nfirst comparison that exists for it: the set diff silenced 2\ndiagnostics, zero new ones.\n\nOf ruby-lsp's 30, 3 are `E0001`\n\n(syntax error) in `test/fixtures/syntax_diagnostics.rb`\n\nand `test/fixtures/multiple_invocations.rb`\n\n, files that are invalid Ruby on purpose, written for\nruby-lsp itself to test its own syntax diagnostic. The\nchecker is right there: it is not a hit against a real bug,\nit is recognizing a deliberate fixture for what it is.\n\ntapioca, which this page classified as 60 diagnostics,\n0 real bug and 34 false\npositives (57%, the worst share in the corpus), dropped to 1. tapioca is not clean:\n1 remains, and it is a\ndocumented ceiling. The author says this remaining case is a\ncase where INSTANCE is a value, a known limitation, not an\nopen defect.\n\nTrajectory of the two typed repositories, four measurement\nrounds: ruby-lsp 456 -> 51 -> 31 -> 30; tapioca 60 -> 60 -> 17 -> 1.\n\nSet diff between wave 7 and wave 8, keyed by path, line,\ncolumn and code:\n\n| Repository | Silenced | New |\n| ruby-lsp | 1 | 0 |\n| tapioca | 16 | 0 |\n| rails/rails | 14 | 0 |\n| discourse/discourse | 1658 | 0 |\n| zammad/zammad | 0 | 0 |\n| chatwoot/chatwoot | 2 | 0 |\n\nZero new diagnostics across the six repositories in the\ncorpus. This time the claim is established by a set diff,\nnot by a totals comparison, which is exactly the distinction\nthe \"Comparing totals is not comparing sets\" section calls\nout. Where this page already said that claim had not been\nestablished for a given wave, it still holds for that wave:\nthis is where, in wave 8, it became established.\n\nThe diff key gained a column, and this is a correction to\nthis page's own published rule. The \"Comparing totals is not\ncomparing sets\" section, further down, says the diff must be\nkeyed by file, line and code, never by message. That rule\nwas incomplete: file:line:code does not distinguish two\ndiagnostics with the same code on the same line, and there\nare real pairs like that in discourse. Keying without a\ncolumn makes those pairs look silenced when both are still\ncorrectly flagged. The correct key includes the column: `file:line:column:code`\n\n.\n\nMost of discourse's drop, 1658\n, is the single root cause already covered in the section\nbelow: 1642\nof those are `DB`\n\nwritten with cbase. The rest,\n16\n, is the `defined?`\n\nguard.\n\n## Comparing totals is not comparing sets\n\nWave 7 was declared drift-free by the checker's author. I\nremeasured it, `5c5c8c3`\n\n, and what\nlooked like 4 isolated false positives were two root causes\nwith 1675 sites total, not four separate\ncases, one of them an order of magnitude bigger than the\nfirst read suggested.\n\nRoot 1, top-level constant write with cbase: 1645 sites\nacross 3 repositories.\n\n| Repository |\nSites\n| Detail |\n| discourse/discourse | 1642 | all the same name, DB, written as ::DB = MiniSqlMultisiteConnection.instance in an initializer |\n| rails/rails | 2 | ::DEFAULT_APP_FILES and ::DEFAULT_PLUGIN_FILES, each defined at the top of the very test file that reads it |\n| Shopify/ruby-lsp | 1 | ::C2, ruby-lsp's own const.rb fixture, which writes ::C2 = 42 on line 2 |\n\nWhen code writes a top-level constant using cbase form, ::X = value, that constant never made it into the top-level constant table. The other side of the same defect: the existence check keyed the cbase fallback off the untrimmed name.\n\nRoot 2, defined? guard: 30 sites\nacross 3 repositories.\n\n| Repository |\nSites\n|\n| rails/rails | 12 |\n| discourse/discourse | 16 |\n| chatwoot/chatwoot | 2 |\n\nFlagging defined?'s argument was never the problem: the checker already got that right. The real false positive was reading that same constant in the if or ternary's then branch, the branch that only runs when it exists. The rule that fixed it controls both sides: a read in the else branch is still flagged, because there it is genuinely unsafe, and defined?(X) guarding a read of Y still flags Y.\n\nThe single biggest name in the whole corpus was not a gem\nconstant missing a declaration: it was a lookup bug on our\nside. One line of discourse code produced 1642 false\ndiagnostics.\n\nBoth roots were fixed at the source in wave 8, binary\ncompiled from commit `5c5c8c3`\n\n, and I\nverified the fix myself:\n\n| Constant | Before | After |\n| DB (discourse/discourse) | 1642 | 0 |\n| DEFAULT_APP_FILES (rails/rails) | 1 | 0 |\n| AppBuilder (rails/rails) | 1 | 0 |\n\nThe proof for the other side, the one that separates\nfixing from suppressing: constants that are genuinely gem\nconstants keep getting flagged. `::Trilogy`\n\nstays at 3 in rails/rails, unchanged.\nOn top of that, zero new diagnostics across the six\nrepositories, and this time that claim is established by a\nset diff, not a totals comparison.\n\nWorth publishing as verification, not praise: the\nattribution the author sent (cbase versus the `defined?`\n\nguard) sums up exactly to the totals I\nmeasured independently, repository by repository:\n\n| Repository |\nIndependent total\n| cbase | defined? | Author's sum |\n| rails/rails | 14 | 2 | 12 | 14 |\n| discourse/discourse | 1658 | 1642 | 16 | 1658 |\n| chatwoot/chatwoot | 2 | 0 | 2 | 2 |\n| Shopify/ruby-lsp | 1 | 1 | 0 | 1 |\n\nTwo counts taken through different paths landed on the same\nnumber, repository by repository. It is the opposite of the\nwave 7 case, where the composition did not add up to the\ntotal and 3 were left unaccounted for.\n\nThe method finding that triggered this check: comparing\ntotals is not comparing sets. On rails, the total came out\nidentical between the two waves, 2316 and 2316, and underneath,\n5 diagnostics left and\n5 entered: a net-zero\ndelta was hiding 10\nitems moving.\n\nWorse: if the diff had been keyed by message instead of by\n`file:line:code`\n\n, rails would have flagged 187 changes,\nbecause wave 7 started printing the constant name in\nabsolute form (`::WebSocket::Driver`\n\ninstead of\n`WebSocket::Driver`\n\n) and every diagnostic's text\nchanged without its site changing. A large false number is\nmore dangerous than a small correct one.\n\nThe value of measuring things yourself is not distrusting\nthe other side: it is that a net-zero delta and a cosmetic\nrewrite hide things in opposite directions.\n\nThis page itself made this exact mistake in the alias-fix\nsection, further up: the passage was corrected, not deleted.\n\n## Follow the launch\n\nitaruby does not have a public repository yet. Leave your email to hear when it opens.\n\nMessage sent. I'll get back to you soon.\n\nCould not send. Please try again.\n\nEach submission's state checked via the host CLI: `gh pr view <n> --repo <owner/repo>`\n\non\nGitHub, `glab mr view <n> --repo <owner/repo>`\n\non\nGitLab.", "url": "https://wpnews.pro/news/a-ruby-type-checker-has-found-15-bugs-in-discourse-rails-zammad-and-ruby-lsp", "canonical_source": "https://aryrabelo.com/en/itaruby/", "published_at": "2026-08-26 17:21:23+00:00", "updated_at": "2026-08-26 17:46:14.281544+00:00", "lang": "en", "topics": ["developer-tools", "artificial-intelligence"], "entities": ["itaruby", "Discourse", "Rails", "Zammad", "Shopify/ruby-lsp", "salsa", "rust-analyzer"], "alternates": {"html": "https://wpnews.pro/news/a-ruby-type-checker-has-found-15-bugs-in-discourse-rails-zammad-and-ruby-lsp", "markdown": "https://wpnews.pro/news/a-ruby-type-checker-has-found-15-bugs-in-discourse-rails-zammad-and-ruby-lsp.md", "text": "https://wpnews.pro/news/a-ruby-type-checker-has-found-15-bugs-in-discourse-rails-zammad-and-ruby-lsp.txt", "jsonld": "https://wpnews.pro/news/a-ruby-type-checker-has-found-15-bugs-in-discourse-rails-zammad-and-ruby-lsp.jsonld"}}