· devops · 19 min read
Code is written faster than ever with AI loving to write tests for everything it does. This makes our CI pipeline slower and slower. Here is how to build a fast CI/CD pipeline - Part 1 with focus on CI
Neciu Dan #
Hi there, it's Dan, a technical co-founder of an ed-tech startup, host of Señors at Scale - a podcast for Senior Engineers, Organizer of ReactJS Barcelona meetup, international speaker and Staff Software Engineer, I'm here to share insights on combining technology and education to solve real problems.
I write about startup challenges, tech innovations, and the Frontend Development. Subscribe to join me on this journey of transforming education through technology. Want to discuss Tech, Frontend or Startup life? Let's connect.
Continuous Integration (CI) and Continuous Deployment(CD) are responsible for two things: making sure that when we merge our PR, it does not introduce any faults (bugs) into our main branch (Integration), and after every merge, our code gets released in some way to production in a timely manner (Deployment).
Then, everything changed when the Fire Nation attacked (and by that I mean the AI Revolution)
AI writes almost all the code now, and its particulary good at writing tests. It doesn’t always write good or useful tests, but it likes to write a lot of tests.
As a result, the time it takes our CI to run is growing exponentially, leaving developers waiting for PRs to build, run, and get feedback that everything is working or not.
It doesn’t have to work like this. Google claims the majority of its new code is now AI-generated and engineer-approved, and they got there by making verification cheap and rollout reversible.
Let’s dig in.
PS: You can find the whole demo and script files in this repo
CI and CD are different problems #
The first thing we need to establish is a clear separation between CI and CD. People treat it as a single concept and a single flow, which causes a lot of problems and headaches.
Continuous Integration answers: is this change correct? It runs Lint, types, unit tests, e2e tests, builds. It runs before merging and slow CI compounds into batched PRs, stale branches, and agents idling.
Continuous Delivery answers: is this change safe in production? Deployment, rollout, rollback. It runs after the merge.
In this article (part 1) I want to focus on building a robust and FAST CI pipeline for Frontend projects. In part we will tackle the deployment part.
CI, part 1: stop running everything #
Imagine your app has a Dashboard, a Settings page, a Billing page, and folders for shared components and shared utilities. Everything lives in a single app and repo, with no workspaces or internal packages.
You change one line in Button.tsx
, push the PR, and your pipeline lints every file you own, typechecks the whole project, runs all 340 unit tests, and builds the app.
Which is fine, but with exponential growth and some apps having millions lines of code, this can get slow fast (pun intented)
name: CI
on: pull_request
jobs:
ci:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
cache: 'npm'
- run: npm ci
- run: npm run lint
- run: npm run typecheck
- run: npm run test
- run: npm run build
You wait twenty-five minutes, and so does everyone queued behind you.
What you actually want is narrow and specific.
Change Button, and you want Button’s tests to run, plus the tests of everything that imports Button, however deep the chain goes, to also run.
You want the same for lint and typecheck. The Billing feature never imports Button, so Billing should not be touched at all.
You want that list computed from your code every time, so it is never wrong, and you never have to maintain it.
That setup takes away the thing most tooling relies on. You have no package.json
boundaries, so nothing in your repo declares that Billing and Settings are separate.
Every tool that solves this for monorepos is reading workspace manifests that we do not have.
We will derive the same information from our imports instead.
Filtering by path
The first thing you might try is filtering by path within GitHub Actions itself. You attach a paths
filter to the trigger, and the workflow only runs when your PR touches files matching those patterns.
name: Settings CI
on:
pull_request:
paths:
- 'src/features/settings/**'
Change something in Settings, and this runs. Change something in Billing, and it stays quiet.
But this has three problems.
First, it does not scale. You maintain that list by hand, forever.
Every feature you add means another workflow file with another set of paths. Every folder you rename means hunting down whichever workflow referenced the old name.
Second, it does not know anything about your imports. A path filter matches folder names, and that is the entire extent of its intelligence.
Third, required checks sit there waiting forever. This one appears once somebody turns on branch protection.
You list the checks you care about, mark them as required, and GitHub refuses to merge until every check on that list passes.
A check reports something when its workflow runs. If the workflow never starts, it never reports, and GitHub does not read that silence as success.
So you open a PR that only touches Billing. Billing CI
runs and passes, Settings CI
never starts, and your PR shows this until the end of time:
✓ Billing CI — Successful in 2m
• Settings CI — Expected — Waiting for status to be reported
The merge button is greyed out. You cannot merge.
All three problems come from the same mistake. We are asking the pipeline to decide what to run before it has looked at anything.
What if instead we let every workflow run on every PR, but within the workflow, we determine what changed and run the necessary jobs.
Step one: ask git what you changed
Git knows the files.
git diff --name-only origin/main...HEAD
src/components/ui/Button.tsx
One file. That is the starting point.
Step two: walk your imports backward
Your Button change renames a prop from label
to text
. Dashboard and Settings both import Button, and both still pass label
.
Billing does not import the Button at all.
So we want to run lint, type-check, and test for Button and every file that uses it.
Every import
in your codebase is an edge, and you want to walk those edges backward from the file you changed.
Now, this is a long script with many edge cases, which I won’t write here.
But for the curious, here is the entire file in scrips/affected.mjs
, this repo has all the demo code in this article and all scripts needed to build the same.
Running that script after changing the button component, and you get exactly four files:
$ BASE_REF=main node scripts/affected.mjs --lines
src/components/ui/Button.tsx
src/features/dashboard/DashboardPage.tsx
src/features/settings/SettingsPage.tsx
src/features/dashboard/DashboardPage.test.tsx
Button, its two importers, and the test that covers one of them. Billing is not on that list, nor is anything else in your app.
Step three: hand that list to your three tools
Now we have a list of files, and three tools need it: our linter, type checker, and test runner, and each one takes it differently.
Lint takes it directly. ESLint accepts file paths as arguments, so pipe the list in.
npx eslint $(node scripts/affected.mjs --lines)
But an empty list turns into npx eslint with no arguments, and a global change prints ALL, which ESLint reads as a filename and fails.
So each tool gets a small wrapper instead.
They all follow the same three-branch shape: everything, nothing, or the list.
These wrappers are scripts/lint-affected.mjs
, scripts/tests-affected.mjs
and scripts/typecheck-affected.mjs
and you can find all of them in the same repo.
Change Button and you get exactly two test files, out of a suite that has more:
✓ src/features/dashboard/DashboardPage.test.tsx (1 test) 4ms ✓ src/components/ui/Button.test.tsx (1 test) 3ms
Test Files 2 passed (2) Tests 2 passed (2)
If you rename a prop and run the type check wrapper:
$ node scripts/typecheck-affected.mjs
src/features/dashboard/DashboardPage.tsx(3,51): error TS2353: Object literal may only
specify known properties, and 'label' does not exist in type '{ text: string; }'.
src/features/settings/SettingsPage.tsx(2,50): error TS2353: Object literal may only
specify known properties, and 'label' does not exist in type '{ text: string; }'.
Both importers are caught, with exact line numbers.
When you would rather not maintain a regex
Our resolver supports relative imports, a single alias, and index files.
It will not handle barrel re-exports that hide the real path, or the seventeen other things your codebase eventually does.
At that point, we can stop maintaining it and use dependency-cruiser
, which properly parses TypeScript, reads your tsconfig paths, follows dynamic imports, and ships it as a flag.
npx depcruise src --affected origin/main --output-type json
--affected
takes a git revision and returns the changed modules and everything that depends on them, which is the whole script above in a single argument.
Step four: problems
On your laptop, the script is precise. On the runner, it dies.
fatal: ambiguous argument 'origin/main...HEAD': unknown revision
actions/checkout
clones with fetch-depth: 1
by default, so your runner holds one commit and has no origin/main
to compare against.
You need history.
- uses: actions/checkout@v4
with:
fetch-depth: 0
filter: blob:none
fetch-depth: 0
brings down your full commit history, and filter: blob:none
skips file contents for commits you will never check out.
Git fetches those on demand if anything needs them.
Never hardcode the base branch either. Once your PRs start targeting each other, the only comparison worth making is against the branch this one merges into.
env:
BASE_REF: origin/${{ github.base_ref }}
The workflow itself is now boring, which is the goal.
name: CI
on: pull_request
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
filter: blob:none
- uses: actions/setup-node@v4
with:
node-version: 22
cache: 'npm'
- run: npm ci
- run: node scripts/lint-affected.mjs
env:
BASE_REF: origin/${{ github.base_ref }}
typecheck:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
filter: blob:none
- uses: actions/setup-node@v4
with:
node-version: 22
cache: 'npm'
- run: npm ci
- run: node scripts/typecheck-affected.mjs
env:
BASE_REF: origin/${{ github.base_ref }}
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
filter: blob:none
- uses: actions/setup-node@v4
with:
node-version: 22
cache: 'npm'
- run: npm ci
- run: node scripts/test-affected.mjs
env:
BASE_REF: origin/${{ github.base_ref }}
Every job runs on every PR, so every required check reports a result, and nothing blocks your merge.
Change Button and your lint job handles four files, your typecheck builds a four-file program, and your test job runs the handful of tests that reach Button.
Change a README, and all three jobs finish before dependencies are installed.
CI, part 2: let’s make it fast #
The workflow we finished part 1 with already runs its three jobs in parallel. GitHub gives each job its own machine, so lint, typecheck, and test start at the same moment, but two things are still slow (or will be).
The first is that every job installs your dependencies from scratch. The second is end-to-end tests, which we have not added yet and which will dwarf everything else the moment we do.
What cache: 'npm'
actually caches
Look at our workflow job again. It already has a cache line.
What you cached there is ~/.npm
, npm’s download cache of package tarballs. You did not cache node_modules
.
So on a cache hit, you skip the network, which is the fast part anyway. Then npm ci
deletes node_modules
, unpacks every tarball again, and rebuilds the tree, which is the slow part and happens every single run.
To skip that, cache node_modules
itself and only install when the cache misses.
- id: modules
uses: actions/cache@v4
with:
path: node_modules
key: modules-v1-${{ runner.os }}-node22-${{ hashFiles('package-lock.json') }}
- if: steps.modules.outputs.cache-hit != 'true'
run: npm ci --prefer-offline --no-audit --fund=false
The if:
guard is not optional. Without it, npm ci
runs anyway, and the first thing npm ci
does is delete the node_modules
you just restored.
Every part of that key is doing a job.
The lockfile hash invalidates when your dependencies change. runner.os
and the Node version keep you from restoring a tree built for a different platform, which matters because packages with native bindings compile against a specific Node ABI and fail in confusing ways when it moves under them.
The v1
segment is your manual escape hatch. GitHub caches cannot be edited or deleted from a workflow, so when a cache goes bad, your only move is to change the key, and bumping v1
to v2
is easy.
Extracting the setup
Adding those cache steps makes the repetition across your three jobs worse.
GitHub Actions has no YAML anchors, so pull the shared steps into a composite action in your repo.
name: Setup
description: Node, dependencies, and the caches for both
runs:
using: composite
steps:
- uses: actions/setup-node@v4
with:
node-version: 22
cache: 'npm'
- id: modules
uses: actions/cache@v4
with:
path: node_modules
key: modules-v1-${{ runner.os }}-node22-${{ hashFiles('package-lock.json') }}
- if: steps.modules.outputs.cache-hit != 'true'
run: npm ci --prefer-offline --no-audit --fund=false
shell: bash
Composite actions require shell:
on every run
step, and the action lives in your repository, so actions/checkout
has to run before you can call it.
Your jobs collapse to four lines each.
name: CI
on: pull_request
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
filter: blob:none
- uses: ./.github/actions/setup
- run: node scripts/lint-affected.mjs
env:
BASE_REF: origin/${{ github.base_ref }}
typecheck:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
filter: blob:none
- uses: ./.github/actions/setup
- run: node scripts/typecheck-affected.mjs
env:
BASE_REF: origin/${{ github.base_ref }}
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
filter: blob:none
- uses: ./.github/actions/setup
- run: node scripts/test-affected.mjs
env:
BASE_REF: origin/${{ github.base_ref }}
Caching what the tools already know
Your dependencies are handled, but the tools themselves also repeat work they could have remembered.
ESLint keeps a cache of the files it has already checked and what it found, keyed on their contents. You point it at a path and persist that path between runs.
// scripts/lint-affected.mjs, the last line
execFileSync(
'npx',
['eslint', '--cache', '--cache-location', '.eslintcache', ...lintable],
{ stdio: 'inherit' }
);
- uses: actions/cache@v4
with:
path: .eslintcache
key: eslint-v1-${{ github.sha }}
restore-keys: eslint-v1-
restore-keys
is safe here, unlike the dependency cache. A stale ESLint cache costs you a re-lint of files whose contents no longer match.
TypeScript writes a .tsbuildinfo when you enable incremental mode, and on the next run it only rechecks what changed.
// tsconfig.json
{
"compilerOptions": {
"incremental": true,
"tsBuildInfoFile": ".cache/tsconfig.tsbuildinfo"
}
}
Cache .cache/
the same way, and your typecheck job stops rebuilding type information for files nobody touched.
The snails
Everything so far is fast because unit tests are fast. End-to-end tests are not fast because browser tests are inherently slow and grow linearly with your product.
Playwright ships the answer as a flag. --shard=1/4
runs a quarter of your suite, and a matrix runs all four quarters on four machines at once.
Start with the config:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
testDir: './e2e',
fullyParallel: true,
reporter: process.env.CI ? 'blob' : 'html',
use: {
baseURL: process.env.PLAYWRIGHT_BASE_URL ?? 'http://localhost:3000',
trace: 'on-first-retry',
},
});
Without fullyParallel: true
, Playwright distributes whole files across shards. Your one large spec file becomes a single shard, while the other three finish early and sit idle.
Now the job, with browser caching:
e2e:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3, 4]
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
filter: blob:none
- uses: ./.github/actions/setup
- name: Read the installed Playwright version
run: |
VERSION=$(node -p "require('@playwright/test/package.json').version")
echo "PLAYWRIGHT_VERSION=$VERSION" >> "$GITHUB_ENV"
- id: browsers
uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: playwright-${{ runner.os }}-${{ env.PLAYWRIGHT_VERSION }}
- if: steps.browsers.outputs.cache-hit != 'true'
run: npx playwright install --with-deps chromium
- if: steps.browsers.outputs.cache-hit == 'true'
run: npx playwright install-deps chromium
- run: npx playwright test --shard=${{ matrix.shard }}/4
- uses: actions/upload-artifact@v4
if: always()
with:
name: blob-report-${{ matrix.shard }}
path: blob-report
retention-days: 7
Four shards, four reports
Each shard writes its own report, leaving you opening 4 artifacts to find 1 failure. That is what the blob
reporter is for: it writes a mergeable intermediate format instead of finished HTML.
We need to add a job that stitches them together.
e2e-report:
if: always()
needs: [e2e]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup
- uses: actions/download-artifact@v4
with:
path: all-blob-reports
pattern: blob-report-*
merge-multiple: true
- run: npx playwright merge-reports --reporter html ./all-blob-reports
- uses: actions/upload-artifact@v4
with:
name: playwright-report
path: playwright-report
Narrowing E2E tests
Sharding makes your suite four times faster, but we don’t want to run the entire E2E test suite for a button change.
Playwright seems to have an --only-changed
param that does this, but it doesn’t work as expected.
npx playwright test --only-changed=origin/main
It walks the import graph of your spec files and keeps the ones your diff can reach, exactly like vitest related
in part 1.
Now look at a real spec.
// e2e/dashboard.spec.ts
import { test, expect } from '@playwright/test';
test('dashboard loads', async ({ page }) => {
await page.goto('/dashboard');
await expect(page.getByRole('heading')).toBeVisible();
});
It imports nothing from your app. It navigates to a URL and communicates with the rendered page via the browser.
So when you change Button, that spec’s import graph never touches your diff, and --only-changed
skips it. The dashboard renders Button, your change broke it, and your e2e job reports green.
The connection you need is not an import but the route.
Your affected list from part 1 already contains the route files, because route entries sit at the top of your import tree. Change Button, and the walk climbs from Button to DashboardPage to src/app/dashboard/page.tsx
, which is a URL you can name.
Coming the other way, each spec tells you which URLs it visits, because page.goto
takes them as literals you can read out of the source.
So we can build an e2e-affected.mjs
script file that handles this and can be used by the E2E job.
Change Button.tsx in an app with four specs and you get three:
$ BASE_REF=main node scripts/e2e-affected.mjs
e2e/checkout.spec.ts
e2e/dashboard.spec.ts
e2e/settings.spec.ts
Dashboard and Settings render Button. Checkout comes along because it walks through the /dashboard
partway. Billing renders no Button, visits no affected route, and stays out.
Change a Billing-only file instead, and the selection works as well.
e2e/billing.spec.ts
e2e/checkout.spec.ts
Here is the full e2e job in the CI workflow:
e2e:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3, 4]
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
filter: blob:none
- uses: ./.github/actions/setup
- name: Read the installed Playwright version
run: |
VERSION=$(node -p "require('@playwright/test/package.json').version")
echo "PLAYWRIGHT_VERSION=$VERSION" >> "$GITHUB_ENV"
- id: browsers
uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: playwright-${{ runner.os }}-${{ env.PLAYWRIGHT_VERSION }}
- if: steps.browsers.outputs.cache-hit != 'true'
run: npx playwright install --with-deps chromium
- if: steps.browsers.outputs.cache-hit == 'true'
run: npx playwright install-deps chromium
- id: specs
env:
BASE_REF: origin/${{ github.base_ref }}
run: |
SPECS=$(node scripts/e2e-affected.mjs | tr '\n' ' ' | xargs)
echo "list=$SPECS" >> "$GITHUB_OUTPUT"
- name: No affected specs
if: steps.specs.outputs.list == ''
run: echo "Nothing to test."
- name: Run affected specs
if: steps.specs.outputs.list != '' && steps.specs.outputs.list != 'ALL'
run: npx playwright test ${{ steps.specs.outputs.list }} --shard=${{ matrix.shard }}/4
- name: Run everything
if: steps.specs.outputs.list == 'ALL'
run: npx playwright test --shard=${{ matrix.shard }}/4
- uses: actions/upload-artifact@v4
if: always()
with:
name: blob-report-${{ matrix.shard }}
path: blob-report
retention-days: 7
Just to clarify. It’s still best practice to run the whole test suite on main regardless. This CI workflow is just for PRs to save time.
CI, part 3: fail before the PR exists #
Smarter and cheaper than running this in the PR is to run it locally first. This gives agents a fast feedback loop, and if your app is a million+ lines of code, it’s better to run locally anyway.
Nothing is stopping you. Everything in parts 1 and 2 is a plain Node script that reads a git diff.
An agent iterating on a failing typecheck will burn six push-and-wait cycles before you look up from whatever else you were doing.
Give it one command to run.
// package.json
{
"scripts": {
"verify": "node scripts/lint-affected.mjs && node scripts/typecheck-affected.mjs && node scripts/test-affected.mjs"
}
}
So npm run verify
chains all three scripts, and on a change to Button, it looks like this:
> node scripts/lint-affected.mjs && node scripts/typecheck-affected.mjs && node scripts/test-affected.mjs
lint: 7 files
typecheck: 7 files
test: 7 affected files
RUN v4.1.10
✓ src/components/ui/Button.test.tsx (1 test) 5ms
✓ src/features/dashboard/DashboardPage.test.tsx (1 test) 4ms
Test Files 2 passed (2)
Tests 2 passed (2)
Duration 422ms
ESLint ran on seven files, tsc
built a seven-file program, and Vitest ran the two specs that reach Button.
That is the entire local gate, and it is the same work CI does (minus the e2e, which needs a browser and takes time to run).
Wiring it into git
Lefthook installs the hooks for you, and Husky works the same way if you already have it.
npm install --save-dev lefthook
npx lefthook install # writes the hooks into .git/hooks
Then split the work by how much of it you can afford to wait for.
pre-commit:
parallel: true
commands:
lint:
glob: '*.{ts,tsx}'
run: npx eslint {staged_files}
format:
glob: '*.{ts,tsx,css,md}'
run: npx prettier --write {staged_files}
stage_fixed: true
pre-push:
commands:
verify:
run: npm run verify
Pre-commit stays on staged files only. {staged_files}
expands to exactly what is in the commit, so a one-file change lints one file, and stage_fixed: true
re-stages whatever Prettier rewrites, so you do not commit the unformatted version.
Pre-push runs the real gate against the affected set rather than your whole app, which is the only reason a pre-push hook is tolerable at all.
The base ref problem, in reverse
Part 1 broke in CI because the runner had no history. Locally, you have the opposite problem: your history is stale.
Your origin/main
is whatever it was when you last fetched.
The result is safe and slow, which is the correct direction to fail, but it makes the hook feel broken. Fetch first.
pre-push:
commands:
verify:
run: git fetch origin main --quiet && npm run verify
Agents will bypass this, so plan for it
Claude Code and its peers know what --no-verify
does. An agent blocked by a failing hook will sometimes reach for it, just as a person does at 6pm on a Friday.
Instructions help, so write them. This lives in my CLAUDE.md:
## Definition of done
A change is not done until this passes:
npm run verify
Run it before every push. Fix what it reports.
NEVER use --no-verify. NEVER edit or delete git hooks.
If a hook looks wrong, stop and ask rather than working around it.
The shape of the whole thing #
Put the pieces in a line, and the pipeline looks like this.
The agent writes code and iterates against the local gate: lefthook running the affected lint, typecheck, and tests, the same commands CI will run.
CI runs only the changed modules and their dependents, in parallel jobs, with Playwright running changed tests on the PR and the full sharded suite in the merge queue.
Whether AI writes 27% of your code or 70% of it, the volume only goes one direction from here.
Now you have a fast CI pipeline, next up is deploying correctly.
Stay tuned for part 2 next week.
References #
Turborepo: Constructing CIand the--affected
flag referencePlaywright: ShardingandContinuous Integration(including--only-changed
)Vercel: Rolling Releasesand thevercel rolling-release
CLI referenceCloudflare: Gradual deployments,Rollbacks, and theproduction safety launch postNetlify: Split TestingGraphite CLI documentation— stacked PRs and the stack-aware merge queueInfoQ: GitHub Targets Large Merge Problem with Stacked PRs— thegh-stack
private previewanthropics/claude-code-action— Claude Code as a GitHub ActionLefthook- Neciu Dan, GitHub Actions Cache Poisoning is eating open sourceandHow the Cline CI got compromised - Adnan Khan, The Monsters in Your Build Cache— the first cache poisoning research zizmor— static analysis for GitHub Actions workflows
Discover more from The Neciu Dan Newsletter
A weekly column on Tech & Education, startup building and occasional hot takes.
Over 1,000 subscribers