I Let an AI Re-Platform My CI Pipeline. Here's What Broke.

A developer handed a CI re-platform from GitHub Actions to Argo Workflows and GitHub to Codeberg to an AI agent, but the rewrite introduced six latent bugs because the runtime environment changed fundamentally. The developer spent hours blaming the network when the real issue was a missing newline in an SSH key, highlighting the need for end-to-end validation in new environments.

Originally published at wostal.eu. TL;DR: I handed a CI re-platform — GitHub Actions → Argo Workflows, GitHub → Codeberg — to an AI agent. The rewrite looked correct and passed lint, but it didn't account for one thing: the runtime environment had changed completely. The old runner was an external VM; the new one is a pod inside the cluster, where Tailscale IPs aren't routable. Nobody ran the pipeline end-to-end, so every difference stayed as a landmine — six of them. The worst part wasn't the AI's mistake. It was that I spent hours blaming the network when the real culprit was a single missing newline in an SSH key. This is the companion to a different incident. While I was migrating my homelab k3s control plane from SQLite to etcd https://wostal.eu/blog/homelab-grows-up-sqlite-to-etcd/ , the pipeline that ran that migration turned out to be broken in ways that taught me more than the migration itself. In this post I'll cover: The entire CI had just been migrated from GitHub Actions to Argo Workflows , and from GitHub to Codeberg — and that migration was handed to an AI agent 🙂. The re-platforming rewrote the logic roughly 1:1, but it didn't catch that the runtime environment had changed fundamentally . The old runner GitHub Actions / ARC behaved one way; the new one is a pod in the cluster — from which Tailscale IPs aren't routable, but private IPs are. Nobody human or AI ran the pipeline end-to-end, so all of those differences stayed as landmines. That's why there were six latent bugs and the hang I'm about to describe — not random faults, but traces of a GHA→Argo translation that was never validated in the new environment. After the migration, it turned out the deploy pipeline CI running in the cluster had never passed end-to-end — ansible-configure was hanging on gather facts . I spent hours on it and hostNetwork pod hung too.Every one of those tests pointed at the network. Every one of them was lying to me. Each test was contaminated by a single bug : in the test pods, I mounted the SSH key with cp from a secret — without a trailing newline . That produces error in libcrypto → the key never loads → ssh behaves randomly sometimes a hang, sometimes "permission denied" . I was chasing phantoms, because the test rig was broken, not the network. The breakthrough only came when I did it properly : tcpdump + conntrack on the node, and the key written exactly as CI writes it printf '%s\n' . Then the picture was clean and consistent: from a pod, the private IP 10.0.1.x works ssh + sftp + full ansible -m setup , while the Tailscale IP 100.x hangs — because the pod has no route into the tailnet. The fix: CI uses the private IP the firewall doesn't filter the private network anyway . It worked instantly. Lesson 1: before you blame the network/CNI/MTU, prove that your test rig isn't broken.One missing newline in a key cost me hours chasing a non-existent CNI problem. Reproduceidentically to production here: exactly how CI writes the key , and for "packets vanish midway" use tcpdump / conntrack , not guesses. "Works from my laptop, not from a pod" isn't always the network — sometimes it's a test-rig artifact. Here's the second trap. The deploy pipeline never reached apply+ansible, so each subsequent bug only surfaced once the previous one started working. There were six in total: openssh in the runner image python3 ansible needs it on the control node duration:0 that didn't actually gate anything semantic-release Lesson 2: a pipeline that fails early masks every downstream bug.You fix one, run again, and the next one — previously unreachable — finally executes and fails. The first green end-to-end run is the moment everything hidden surfaces at once. Plan for that run; don't assume "it lints, so it works." Lesson 3: re-platforming CI GHA→Argo, GitHub→Codeberg requires an end-to-end run — whether a human or an AI did it.A migration rewritten "1:1" looks correct and passes lint, but it loses theruntime-environment differences: an external runner vs an in-cluster pod, the GitHub API vs Forgejo. Those differences won't show up in any static review — only in one full pass. If you delegate such a migration to an AI agent 🙂 , plan asmoke-run of the whole pathimmediately, or you'll discover the delta months later, at 3am. This isn't an argument against using AI for migrations. The rewrite was mostly correct, and it saved real time. It's an argument about where the human stays in the loop: not in re-typing the YAML, but in validating the thing actually runs in its new home . The AI translated the language; it couldn't translate the environment, because it never executed there. Neither did I — until it was breaking. After the fixes, the pipeline runs the way it should: merge to main → sensor → deploy pipeline → plan → approve: suspend → apply → ansible → release , with a manual gate argo resume = "after approval" . I validated it on a live merge — it cut release v1.4.1 cleanly. Control plane on etcd, DR local + S3 , config via GitOps with a human gate in the middle. Two incidents, one weekend. One taught me that defaults become landmines when a lab grows up. This one taught me that a migration isn't done until something has actually run end-to-end in its new environment — no matter how confident the thing that wrote it.