When a build breaks, the bug fixes itself

A developer at Shipeasy has automated the entire CI failure resolution pipeline: when a build fails, it automatically files a bug ticket, and an AI agent investigates, writes a patch, and opens a pull request. The system uses existing infrastructure like Google Cloud Pub/Sub and a Rails webhook controller, requiring no new infrastructure. The engineer's role is reduced to reviewing the AI-generated PR.

When a build breaks, the bug fixes itself We stopped babysitting CI failures. Now a red build files its own bug — and an AI agent picks it up and ships the fix. PROBLEM — A failed build told no one Our CI would fail, and then… nothing would happen. The failure sat quietly in a build console that nobody keeps open. Eventually someone would notice a change hadn't gone out, go digging, and realize the build had been red for hours. And noticing was the easy part. Actually resolving it meant a whole code session: pull up the logs, find the failing step, reproduce it, and have an engineer sit down and personally shepherd the fix from broken to green. Every red build cost real human hours — plus the invisible tax of the delay before anyone even knew there was a problem. The true cost of a broken build was never the build. It was a person having to find it, understand it, and hand-fix it. SOLUTION — The failure files its own ticket — and an agent takes it from there Now nobody watches a console and nobody triages. The moment a build fails, it automatically files a bug in Shipeasy — our ops platform — as a real, prioritized ticket with the failing step, the branch, and a link to the logs already attached. From there it leaves human hands entirely. Shipeasy hands the bug to an AI agent, which investigates the failure, writes the patch, and opens a pull request against it. The loop that used to be "human notices → human reads logs → human fixes" is now "build fails → bug appears → agent fixes." The engineer's job shrank to reviewing a PR that already exists. DESIGN — How the whole thing hangs together The pipeline is deliberately boring — every hop is either something the cloud already does for free, or a service we already run: main publishes automaticallyWhat makes this cheap is the shape of it: we added no new infrastructure. The event was already on a bus Pub/Sub . We already ran a service that could receive it. All we wrote was the glue in the middle. IMPLEMENTATION — How it worked out in Rails The build side needed no changes at all — Cloud Build publishes to the cloud-builds topic on its own. So the work was three small pieces: one gcloud command, one route, and one controller. gcloud pubsub subscriptions create cloud-build-failures \ --topic=cloud-builds \ --push-endpoint="https://our-app/webhooks/cloud build?token=$SECRET" \ --message-filter='attributes.status = "FAILURE" OR attributes.status = "INTERNAL ERROR" OR attributes.status = "TIMEOUT"' scope "/webhooks", module: :webhooks do post "cloud build", to: "cloud build create" end class Webhooks::CloudBuildController < Webhooks::ApplicationController before action :verify token FAILURE STATUSES = %w FAILURE INTERNAL ERROR TIMEOUT POST /webhooks/cloud build def create message = params :message return head :bad request if message.blank? build = decode build json message :data base64 JSON → Hash status = message.dig :attributes, :status || build "status" .to s return head :ok unless FAILURE STATUSES.include? status return head :ok unless first delivery? build "id" dedupe file bug for build, status head :ok end private Compare-and-set on the build id: the first delivery wins, redeliveries no-op. def first delivery? build id Rails.cache.write "cloud build:filed: {build id}", true, unless exist: true, expires in: 7.days end def verify token expected = App::Secrets.cloud build webhook secret provided = params :token || request.headers "X-CloudBuild-Token" head :unauthorized unless ActiveSupport::SecurityUtils.secure compare provided.to s, expected.to s end end The one line that files the ticket. The controller hands the build context to a thin Shipeasy client. This is the exact call — it turns a red build into a first-class bug that opens a GitHub issue, pings Slack, and becomes eligible for the auto-fix agent: python def file bug for build, status subs = build "substitutions" || {} ShipeasyOps::Client.new.file bug title: "Cloud Build {status.downcase} on {subs "BRANCH NAME" } — {subs "SHORT SHA" }", steps to reproduce: "Cloud Build trigger \" {subs "TRIGGER NAME" }\" reported {status}.", actual result: "Logs: {build "logUrl" }\n\n {build.dig "failureInfo", "detail" }", expected result: "The build completes and deploys.", priority: "high", tags: %w cloud-build ci , end And the client itself is just a typed wrapper over one HTTP call — no new gems, no framework. This is all it takes to create a bug on the platform: python def file bug title:, steps to reproduce:, actual result:, expected result:, priority:, tags: post "/api/admin/ops", { type: "bug", title: title, stepsToReproduce: steps to reproduce, actualResult: actual result, expectedResult: expected result, priority: priority, tags: tags, } end def post path, body req = Net::HTTP::Post.new URI " {BASE URL} {path}" req "Authorization" = "Bearer {@admin key}" sdk admin … key req "X-Project-Id" = @project id req "Content-Type" = "application/json" req.body = body.compact.to json res = Net::HTTP.start req.uri.host, req.uri.port, use ssl: true { |h| h.request req } JSON.parse res.body end That's the whole integration. The failure travels from Cloud Build to a filed, prioritized bug through two managed hops and about forty lines of Ruby — and the moment it lands, it's the platform's problem, not a person's. SHIPEASY — One problem, several ways to solve it THE PAYOFF — The last red build nobody had to fix A build died with FATAL ERROR: Reached heap limit — JavaScript heap out of memory . The bundle outgrew its memory ceiling. The agent recognized the OOM pattern and opened a PR with the entire fix: - NODE OPTIONS="--max-old-space-size=4096" + NODE OPTIONS="--max-old-space-size=8192" A teammate who doesn't write backend merged it. The person who "fixed" it never had to know what a heap limit is. The failure notices itself, files itself, fixes itself, and asks a human for nothing more than a nod. Before standing up new infrastructure to react to an event, check whether the event is already on a bus you can subscribe to — and whether something you already run can catch it. The ops side of this runs on Shipeasy — the ticket, the agent, the rollout. Core product is the ops queue that delegates to agents. Free tier, no card required; Team at $49/seat/mo removes limits. → shipeasy.ai https://shipeasy.ai · → docs.shipeasy.ai/sdks/ruby https://docs.shipeasy.ai/sdks/ruby · → github.com/shipeasy-ai/shipeasy https://github.com/shipeasy-ai/shipeasy