# Pilum: From Launch to Production-Ready in 3 Months

> Source: <https://dev.to/logical_bytes/pilum-from-launch-to-production-ready-in-3-months-3769>
> Published: 2026-07-28 15:44:03+00:00

Github URL: [https://github.com/SID-Technologies/Pilum](https://github.com/SID-Technologies/Pilum)

In December 2025, I [open-sourced Pilum](https://logicalbytes.dev/blog/introducing-pilum), a multi-cloud deployment CLI that deploys to Cloud Run, Lambda, Azure, Cloudflare Pages, npm, Homebrew, and Docker Hub from a single `pilum.yaml`

. The announcement post covered the architecture: recipes, ingredients, handlers, wave-based execution.

That was the "it compiles and the tests pass" version.

Three months and 40+ pull requests later, Pilum deploys all of [SID Technologies](https://romans.dev), platform-core, Torch, Statio, every website, every npm package, and itself. This post is about everything that broke between "it works" and "it ships production software."

*(**Update:** there's now a seven-months-in addendum at the bottom. It's mostly about what stopped breaking.)*

`--only-changed`

, file embedding support. The "I need this for real" features. (#22-#24)The pattern is clear: **features ship fast, fixes ship faster, and the real bugs show up a month later.**

Wave-based deployment was the headline feature in #31. Services declare dependencies, Pilum builds a dependency graph, topologically sorts it into waves, and executes waves in parallel.

It worked perfectly in tests. Then I tried to publish npm packages.

The problem: Pilum publishes `@sid-technologies/base-ui`

, which depends on `@repo/configs`

. Both are in the same monorepo. Wave ordering correctly put `configs`

before `base-ui`

. But the worker allocation was distributing work across goroutines without respecting wave boundaries — workers from wave 2 could start before wave 1 fully drained.

The npm registry would reject `base-ui`

because `configs`

hadn't finished publishing yet. Sometimes it worked (race condition timing), sometimes it didn't.

The fix (#61) was two things: memory-based worker allocation (don't spawn more workers than the machine can handle) and a proper wave barrier, wave N+1 doesn't start until every worker in wave N has completed and reported success. The entire orchestrator got rewritten in #60 to make this possible — `runner.go`

(772 lines) was replaced with `execution.go`

, `pipeline.go`

, `steps.go`

, and `waves.go`

(total: ~773 lines, but now testable and correct).

This was the scariest bug (#49). The worker queue was silently swallowing errors from shell commands. A deployment step would fail, the worker would log it, but the orchestrator wouldn't see the failure. The deploy would "succeed" with broken services.

The root cause: the command worker's error channel wasn't being read in all code paths. If a command failed during the capture phase (reading stdout/stderr), the error was logged but not propagated to the result channel.

The fix was straightforward propagate errors through every code path but finding it required adding 274 lines of tests to the worker queue and orchestrator. The bug was invisible because the services usually deployed fine. It only surfaced when a Docker build failed mid-stream and Pilum reported success anyway.

**Lesson: if you're building a deployment tool, test the failure paths harder than the success paths. Nobody cares if deploys work. They care enormously when deploys claim to work but didn't.**

GCP Cloud Run expects secrets in a specific format: `SECRET_NAME=projects/PROJECT/secrets/NAME/versions/latest`

. Pilum was passing them as `SECRET_NAME=value`

, which works for environment variables but not for secret references.

Two fixes (#48, #58):

`--set-env-vars`

from `--set-secrets`

in the `gcloud run deploy`

command.This is the kind of bug that only shows up when you deploy a real service with real secrets, not when you test against mocked GCP APIs.

Cloudflare Pages broke twice (#47, #56). The first fix was the recipe itself — the wrangler CLI changed its flags between versions and the recipe had hardcoded the old format.

The second fix was more subtle: the capture worker was incorrectly detecting the end of output from the wrangler process. It would sometimes truncate the deployment URL from the output, so Pilum couldn't report where the site was deployed. The fix was a one-line change in how the capture worker detects stream completion, but it took an hour to diagnose because wrangler's output format is inconsistent between deploy targets.

The YAML parser (#42) had an encoding issue. Users could write `build: go`

in their pilum.yaml, but the YAML library parsed `go`

as a boolean (`true`

in YAML 1.1). This broke the build step because it expected a string, not a boolean.

Two-line fix: force string type on specific fields during parsing. But it's the kind of bug that makes you question every YAML-based config format you've ever designed.

Getting npm packages to publish correctly took three separate PRs (#52, #53, #57):

**#52**: The npm ingredient needed workspace resolution. In a pnpm monorepo, `pnpm publish`

needs to know which workspace you're publishing. Added a `resolve_workspaces.js`

script that walks the workspace config and finds the right package.

**#53**: E2E test infrastructure. After #52 I realized I had no way to test the full publish flow without actually hitting the npm registry. Built a test harness with fixture packages and dry-run publishing.

**#57**: `npm`

vs `pnpm`

command resolution. Some systems have `npm`

but not `pnpm`

, or vice versa. The completion and local command system needed to detect which package manager is available and use the right one.

The December launch had the core recipe engine and a few deploy targets. Everything below was added in February because real usage demanded it:

Ten features in three weeks. Each one spawned 1-2 bug fixes in the following weeks.

By February 9, Pilum was stable enough that I didn't want to keep writing `curl | sh`

install scripts in every CI workflow. So I built [pilum-action](https://github.com/SID-Technologies/Pilum-Action) — a GitHub Action that installs Pilum and runs any command.

```
- uses: SID-Technologies/pilum-action@v1
  with:
    command: deploy
    tag: ${{ github.event.release.tag_name }}
```

It auto-detects the runner's OS and architecture, downloads the right binary, and passes through all credentials via environment variables. Simple wrapper, but it cut 15-20 lines of boilerplate from every deployment workflow.

The action itself hit a bug immediately: the binary name in the release archive included the version number (`pilum-v0.3.1-linux-amd64`

), but the action was looking for `pilum`

. Two quick fixes (#3, #4) and a third one in March when the naming convention changed again.

Every SID repo now uses `pilum-action@v1`

for deployments instead of raw shell scripts.

The [pilum.dev](https://pilum.dev) website launched the same day as the CLI (December 3) — an Astro site with an animated terminal demo, deploy target icons, and full documentation.

Over the next three months it evolved alongside the tool:

The website was always the last thing updated after a feature shipped. In hindsight, having docs deploy automatically from the same `pilum deploy`

pipeline would have kept them in sync. That's a future improvement.

PR #51 was a comprehensive security audit response. The headline finding: **shell injection via pilum.yaml values.** If a service name or environment variable contained shell metacharacters (

`$(rm -rf /)`

), they'd be passed unsanitized to `sh -c`

.The fix introduced a `shellutil`

package with:

`Quote()`

: POSIX single-quote escaping for string commands`ValidateServiceName()`

: Reject names with shell metacharacters`SanitizeHeredocValue()`

: Escape `$`

, backticks, and backslashes in heredoc valuesAlso added: symlink traversal prevention, path validation, and safe temporary directory handling.

Pilum's attack surface is small — users run their own configs — but in CI/CD environments where configs might be generated or templates might include untrusted input, these protections matter.

Later, we added Grype + Syft vulnerability scanning to the CI pipeline (#51 and subsequent). Every dependency is scanned against the NVD database, and builds fail on high-severity CVEs.

Four months of dogfooding (December 2025 — April 2026):

**E2E tests from day one.** Unit tests caught maybe 30% of the bugs on this list. Every serious bug — wave ordering, error swallowing, npm publishing, Cloudflare output — required the full pipeline to reproduce. I built the E2E framework in #53. Should have been #2.

**Fewer features per sprint.** The Feb 7 feature marathon (8 features in 48 hours) created a week of bug fixes. Shipping 2-3 features with immediate dogfooding would have caught issues faster.

**The orchestrator should have been modular from the start.** The 772-line `runner.go`

was unmaintainable. The rewrite into `execution.go`

, `pipeline.go`

, `steps.go`

, and `waves.go`

should have been the original architecture. The monolithic runner made every bug harder to find and every fix riskier.

Pilum is stable. It deploys everything SID Technologies builds. The immediate roadmap:

The gap between "open-source announcement" and "tool I trust with production deploys" was about 40 pull requests. Every real deployment surfaced an edge case that unit tests never caught. The wave ordering bug was invisible for two months because most deploys don't have inter-service dependencies.

If you're building developer tools: **dogfood them immediately, not eventually.** The first month after launch is a second development phase. Budget for it.

The most interesting thing that happened to Pilum since April is what stopped happening.

The post above covers December through April: 62 pull requests, an orchestrator rewrite, a security audit, and a feature sprint that took a week of fixes to clean up. April through late July looks nothing like that. **15 commits: three features, ten small fixes, a refactor, and a chore.** v0.6 to v0.7.4. No rewrites. No multi-PR bug sagas. Nothing new for the "everything that broke" list.

Meanwhile the surface area kept growing. Pilum deploys everything SID Technologies ships: GCP services, every website, Homebrew formulas, and now private npm packages too. All of CI/CD runs through [pilum-action](https://github.com/SID-Technologies/Pilum-Action). I stopped thinking about the tool. For a deploy tool, that's the whole job.

Two things from the quiet months are worth writing down.

**The deploy config became the dev environment.** `pilum compose`

(#65) reads the same `pilum.yaml`

files that deploy to production and generates a `docker-compose.yml`

: services wired with `depends_on`

, healthchecks, and infrastructure dependencies declared once in `.pilum.yml`

. For most of my projects that means a Postgres container plus my services. It works really well. I use it constantly while developing, spinning containers up and down with the same configs that ship.

The lesson came from the one project where it wasn't simple. I have a gateway service that routes to multiple backends, and a dependent project that lives in a different repo. I wanted all of those containers in one stack behind a single command, and I kept hitting port conflicts. The answer wasn't more generator features. Docker Compose already has an override convention: the generated `docker-compose.yml`

stays regenerable, and a hand-written `docker-compose.override.yml`

carries the extra containers and the port fixes. Compose merges the two automatically.

There's a design principle in there: **when your tool generates a file, decide on day one where human edits go. They're coming, and if the answer is "into the generated file," you built a one-way generator.**

**Features started arriving with their failure paths tested.** The one substantial feature of this stretch was Cloud Run sidecars plus two image-based recipes for deploying prebuilt images (#72). It shipped with its own e2e fixtures and an end-to-end test runner in the same PR. It spawned zero follow-up fixes. In February, a feature that size cost me two weeks of cleanup. In the "what I'd do differently" list above, I said the E2E framework should have been #2 instead of #53. Turns out following your own advice works.

The February version of this post ends every section with a bug. The July version doesn't have the material. That's what stable looks like. It's also what boring looks like. Seven months in, I've decided they're the same thing.