Trelix v2.7 to v2.9: Fixing the Pipeline Debt

Trelix v2.7 to v2.9 releases fix pipeline debt including binary naming collisions, concurrency bugs, and silent data corruption. Developer Alex (author) discovered that CI had never built Linux binaries, test suites were not wired into CI allowing a regression where MCP server claimed 6 tools but had 8, and a TOCTOU race condition in the sparse embedder's lazy-load required double-checked locking. The most critical fix addressed silent foreign-key corruption during partial re-indexing that nulled graph edges without error messages.

Trelix v2.7 to v2.9: Fixing the Pipeline Debt release.yml was building macOS and Linux PyInstaller binaries both as dist/trelix , and since softprops/action-gh-release uploads by basename, the files collided. I had no way of knowing which OS actually survived the upload. It wasn't a bug in the retrieval logic—it was a failure in the delivery mechanism that passed CI because the workflow turned green, but the output was broken. The Infrastructure Debt Fixing the binary collision in v2.7.1 was the start of a two-week audit. I discovered that the PR-time CI workflow had never actually built a Linux binary, despite the release workflow doing so at tag time. I had to manually add the missing matrix entry and a verification step to ensure parity. The most frustrating find was in trelix-mcp . The test suite wasn't even wired into CI. This allowed a regression to sit undetected where a test asserted the MCP /en/tags/mcp/ server had "exactly 6 tools," while it actually had 8 due to subscription tools added in v2.5.0 . A test that passes for the wrong reason is worse than having no test at all. To fix the deployment pipeline and avoid future collisions, I updated the build script to rename binaries uniquely before the upload step: Example fix for binary naming collision in GitHub Actions for binary in dist/trelix ; do os name=$ echo "$binary" | cut -d'-' -f1 Assuming naming convention like macos-trelix mv "$binary" "dist/trelix-${os name}" done Concurrency and Race Conditions v2.7.2 was where I stopped pretending check same thread=False made my code thread-safe. While I added Qdrant Cloud readiness and parallel BM25 read pools, the real value was in the five concurrency bugs I unearthed via stress tests. I found a TOCTOU Time-of-Check to Time-of-Use race in the sparse embedder's lazy-load. Two threads could check if a model was loaded, both see "False," and both attempt to initialize the model simultaneously. I solved this using double-checked locking. I also dealt with an MCP stdout write race where concurrent notification writes interleaved partial JSON-RPC lines, corrupting the client's output. The fix required a strict lock around the write-and-flush sequence: python import threading Fix for MCP stdout interleaving write lock = threading.Lock def safe write data : with write lock: sys.stdout.write data sys.stdout.flush Other critical fixes included implementing a max-subscriber cap and a TTL sweep for the subscription registry to prevent unbounded memory growth from misbehaving clients. The Silent Data Corruption Bug The most unsettling issue was silent foreign-key corruption during partial re-indexing. In the database schema, parent-symbol, call-callee, and type-edge columns were set to null on delete. While this seems safe, deleting a changed symbol's old row silently nulled those links on every row that referenced it—including unchanged rows. There was no error message; the graph edges were just quietly disappearing as files changed. To stop this "silent rot," I had to implement a snapshot-and-reverify mechanism to ensure referential integrity during the re-index process. If you're building an LLM agent or a complex retrieval system, don't just test the "happy path" of your AI workflow. Stress test the concurrency and actually check your release assets. The pipeline is part of the product. Next AI Coding and the Expertise Gap: Why Friction Matters → /en/threads/2905/