Our agent's tests were green. 16 of 17 never ran.

A coding agent working on the DynoInvoice e-invoicing system under Keel wrote 17 negative tests in a shared Kotlin base class, but 16 of the 17 never executed because the test methods used expression bodies that returned a value instead of Unit, which JUnit Jupiter silently skips without failing the build. The gap passed one review round and two green pipelines before a separate agent spotted 6 test classes reporting 1 case each against 17 @Test methods; the fix merged on 18 October 2026 (commit daaaaabd), and the team added a gate requiring every @Test to return Unit plus a count check comparing reported cases per class to declared @Test lines. The team also disclosed that a refusal test stayed green after a reviewer replaced a tenant check with if false, and that a schema breaking a published API standard merged green because the gate had no previous version to compare against a new file.

Our agent's tests were green. 16 of 17 never ran. In October 2026, a coding agent opened a merge request for a database library in DynoInvoice, the e-invoicing system agents are building under Keel. The task required negative tests: stale versions, concurrent updates, another tenant's rows, no transaction, row-level security on and off. The agent w rote 17 of them, in a shared base class. The merge request went through one review round and two green pipelines. 16 of those 17 tests had never run. The tests were written in Kotlin, in this shape: @Test fun rejects a stale version = withService { assertThrows<StaleVersionException { store.update staleRow } } withService returns whatever its lambda returns. Here the lambda returns the exception that assertThrows hands back, so the test method doesn't return Unit. It returns a value. JUnit Jupiter only runs test methods that return void in Kotlin, Unit . It skips the others without failing the build. So the pipeline was green and listed six test classes, and every class ran one case. The fix is a small change of syntax: a block body instead of an expression body, so the method returns Unit: @Test fun rejects a stale version { withService { assertThrows<StaleVersionException { store.update staleRow } } } The independent reviewer read the merge request and found a real gap: a missing batch-audit test. It didn't notice that most of the tests weren't running. Neither did a check that asks each changed test class for at least one passing case. Every class had exactly one. A separate, independent agent noticed later, while working on something else. It opened the test report and saw 6 classes with 1 case each, against a shared base class with 17 @Test methods. Re-running one class reported 2 tests. It was caught before merge. But it wasn't caught by the step that was supposed to catch it, and we'd rather say so. Fixing the 16 methods wasn't enough on its own. The same mistake could come back, and it would be just as invisible. So two things changed: Every @Test must return Unit. A new gate compares counts. For each test class, the number of cases in the report must equal the number of @Test lines in the file. The gate also has tests of its own. A deliberately expression-bodied test must come out one short, and a missing class must count as zero. After the fix, every case in the six classes ran, with no failures and no skips. The change merged on 18 October 2026 commit daaaaabd in the public repository . This wasn't the only time a passing check proved nothing: A refusal test that didn't test the refusal. To check that the test could fail, a reviewer replaced a tenant check with if false. The test stayed green, because a different layer rejected the insert with the same error code. The check is now proved on its own. A gate that skipped new files. A schema that broke a published API standard merged green, because the gate compared each schema with its previous version, and a new file has none. The next merge requests failed, the schema was fixed through a new task, and the gate now checks new files too, with no exception added. Claims in merge requests are checked, not trusted. Reviewers check them against the pipeline and the code: claims that a change had been verified locally, that a red job was a known flaky test, and that only two files had changed were each found untrue in review and corrected. Read the case count, not the colour. Green tells you nothing failed. It doesn't tell you anything ran. Gate on counts. Compare the cases reported per class with the test methods in the source. A minimal version, for Gradle's JUnit XML reports and Kotlin tests: fail if any test class ran fewer cases than it declares for src in $ grep -rl --include=' .kt' '@Test' src/test ; do cls=$ basename "$src" .kt declared=$ grep -cE '@Test ^A-Za-z |$ ' "$src" not @TestInstance ran=$ grep -ho "testsuite name=\" ^\" \.$cls\" tests=\" 0-9 \"" build/test-results/test/ .xml \ | grep -o 'tests=" 0-9 "' | grep -o ' 0-9 ' | head -1 "${ran:-0}" -ge "$declared" || { echo "$cls: declared $declared, ran ${ran:-0}"; exit 1; } done Adapt it to inherited tests and parameterised cases. Then plant a skipped test and make sure the gate goes red. Prove each negative test can fail. Break the thing it guards, and watch it go red. Under Keel, a task's tests must prove acceptance criteria held in Keel; the coding agent can't change those criteria, and a rule forbids weakening a check to get green. On DynoInvoice, an independent reviewer agent also checks every merge request, including any change to tests. That review missed this one. The fix tightened the check instead of loosening it: an independent agent read the test report, a ruling settled the fix, and a gate that compares case counts now runs in DynoInvoice's build. Written by the DynoKore team with AI assistance. Events from DynoInvoice's public repository and Keel record, October 2026. First published on dynokore.com. We're opening DynoKore Keel to a small group of design partners: dynokore.ai. Key Takeaways - •The merge request went through one review round and two green pipelines. - •This story was reported by Dev.to , covering developments in the dev space. - •AI advancements continue to reshape industries — read the full article on Dev.to for complete coverage. 📖 Continue reading the full article: Read Full Article on Dev.to → https://dev.to/dynokore team/our-agents-tests-were-green-16-of-17-never-ran-10fo