{"slug": "ai-vs-ci-nobody-told-the-pipeline", "title": "AI vs CI: Nobody Told the Pipeline", "summary": "Gil Zilberfeld, a developer and tester, argues that AI is breaking the repeatability that makes continuous integration (CI) reliable. He explains that as AI models change unpredictably, CI runs become non-repeatable, making it harder to detect regressions and trust green builds. Zilberfeld suggests that teams must adapt by running more tests and monitoring trends to catch drift before it ships.", "body_md": "CI is so commonplace these days, we don’t often think about why it’s there. And that reasoning really matters these days, because of how we use it.\n\nEspecially when you’re writing AI agents and expect the CI system to run a regression test suite, and sound the alarm when something goes wrong.\n\nHere’s a funny story you may not know. CI - Continuous Integration – was not originally a tool category. It’s a process - it’s how you continuously integrate pieces of code together.\n\nAnyway, why do you need this process anyway? Because you’re scared. You’re terrified that integration broke something.\n\nSo what do you do? You look for approval. An automation system that runs all the tests all the time is our approval of choice.\n\nThat’s how CI became a tool category. Because in its heart, it’s a simple automation pipeline.\n\nSo, that automation’s holy grail is speed, right?\n\nWrong.\n\nRemember when we wrote software in a cave, and we had a sticker on our computers – “Works on my machine. Don’t run it on another cave”?\n\nAh, the good old days.\n\nBecause the CI automation #1 OG mission was to kill “Works on my machine”. How? By creating a repeatable process. Independent of all the weird installs on my machine. Multiple versions of libraries. And different configurations. And admin hacks.\n\nThe CI builds, packages, runs and tests the software every time the same way. That way, if a test turns red, you know something bad happened, because it was green until now. And why your stomach turns, when it flakes between red and green – that’s the feeling of lost [repeatability](https://testingil.com/2023/09/the-4-attributes-of-testability-reproducibility.html).\n\nYes, repeatability brings bliss. Well, it did, before AI.\n\nIf your app does not touch AI (although building with code agents, can sometimes count as “touch”), you’ll continue to feel that bliss.\n\nBut if you’re developing AI-based features, or agents, or vibe-coding, you’re in for a new experience every time you push your code (or prompts) into CI.\n\nThings are not repeatable anymore. Models change, sometimes without warning. And always under your feet. And when a model changes, does it trigger a run?\n\nYou wish! Your alert system is malfunctioning.\n\nAnd the worst part is not a regression. Although if it’s a bug in a prompt, you may not be able to [fix it](https://testingil.com/2026/07/testing-ai-generated-code-prompt-not-a-fix.html).\n\nNo, the worst part is that Green today doesn’t tell you what kind of Green it is. Things are no longer Working or not, they are Work-ish. But even that’s not the same work-ish every day.\n\nSo we’ve got fewer triggers, results we can’t read, and on top of it – every run means something else.\n\nRepeatability gives us confidence. CI is the process, and tools, that give us the approval we seek, and the confidence we have comes from that repeatability.\n\nWe can’t rely on the trigger anymore, but we can initiate our own runs. And since one run is not enough, we need to run more and look at the trends. Catch drift before it ships.\n\nWhat happens to quality when AI meets CI? We need to get back to the reasoning, and change how we manage quality.\n\n*Originally published at testingil.com.*\n\nI'm Gil Zilberfeld. I teach API testing and test automation, and I write about what AI-generated code does to quality.", "url": "https://wpnews.pro/news/ai-vs-ci-nobody-told-the-pipeline", "canonical_source": "https://dev.to/testingil/ai-vs-ci-nobody-told-the-pipeline-4e88", "published_at": "2026-08-20 11:17:28+00:00", "updated_at": "2026-08-20 11:45:10.212176+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "mlops", "ai-safety"], "entities": ["Gil Zilberfeld", "testingil.com"], "alternates": {"html": "https://wpnews.pro/news/ai-vs-ci-nobody-told-the-pipeline", "markdown": "https://wpnews.pro/news/ai-vs-ci-nobody-told-the-pipeline.md", "text": "https://wpnews.pro/news/ai-vs-ci-nobody-told-the-pipeline.txt", "jsonld": "https://wpnews.pro/news/ai-vs-ci-nobody-told-the-pipeline.jsonld"}}