How to build a (basic) agentic pipeline with GitHub

A developer guide published on GitHub details how to build a basic agentic pipeline that automates the hand-offs between an idea and a clickable preview link, using Claude Code routines, the Claude GitHub App, GitHub Actions, and Vercel preview deployments. Because routines can trigger only on pull requests and releases rather than issues or comments, a GitHub Action listens to issue and PR activity and calls the routine's API, with Claude writing content while the workflow changes labels, issue text and review threads based on hidden markers in Claude's comments. The guide notes that text sent to a routine arrives as untrusted data and is ignored unless the saved prompt opts in by mentioning the payload block, and that each session's assigned branch overrode a branch rule when that rule lived in a file rather than the prompt stub.

Count the hand-offs between "someone has an idea" and "someone can click a preview link". Write the ticket, refine it, agree on what done means, branch, build, test, open a PR, review, fix, wait for the preview, merge. Only three of those need a human: answering the questions only you can answer, saying yes to the plan, and looking at the result. This guide builds a pipeline where those three are all you do. You open an issue with one sentence. Claude proposes a plan and asks a few multiple-choice questions on the issue. Your answers turn the issue into a spec. You add a label. Claude writes a test per acceptance criterion, implements until they pass, and opens a PR with a Vercel preview. You leave review comments; Claude fixes them on the same branch and replies in the threads. Everything below comes from that experiment, including the parts that broke. What you need - A Next.js or any repo on GitHub, with tests that run in CI. I used Vitest and React Testing Library. - A claude.ai plan with Claude Code on the web, for routines : saved cloud sessions that run on Anthropic's infrastructure, so nothing has to stay switched on. - The Claude GitHub App installed on the repo. - Vercel connected to the repo for preview deployments. The architecture in one paragraph Routines can trigger on a schedule, on an API call, or on GitHub events, but those events only cover pull requests and releases. Not issues, not comments. So a GitHub Action listens to issue and PR activity, decides which step comes next, and calls the routine's API with the issue or PR as text. The routine does the thinking and talks back by commenting. Claude writes, the workflow moves the cards : labels, issue text and review threads are only ever changed by the workflow, based on hidden markers in Claude's comments. Step 1: one routine, instructions in the repo Create a routine at claude.ai/code/routines with your repo, the Default environment, an API trigger, and no connectors. Then generate its token in the API trigger settings. It's shown once. The routine's saved prompt is a stub. The real instructions live in the repo, so changing how Claude behaves is a commit, not a trip to a web form: You are a step in this repository's delivery pipeline. The routine-fire-payload block starts with a line "Step: <name ". Read .claude/routines/steps/<name .md from the cloned default branch and follow its "Instructions" section exactly; those instructions tell you how to use the rest of the payload, which contains a GitHub issue or pull request. If <name is not a plain lowercase word or that file doesn't exist, change nothing, comment nothing, and stop. Branches: never push to the branch this session was assigned. The build step pushes to a new branch named claude/issue-<number -<short-slug ; the fix step pushes to the pull request's existing branch. Two details in there matter. Text sent to a routine arrives wrapped as untrusted data, and the routine ignores instructions in it unless the saved prompt opts in by mentioning the payload block. And the branch rule is in the stub because each session gets an assigned branch like claude/dreamy-hopper-ala74d , which overrode the same rule when it lived in a file. The repo side looks like this. Each pipeline state is a markdown file; shared formats are referenced from the steps: .claude/routines/ README.md the routine, its stub, which step runs when steps/refine.md shape the issue into a spec steps/build.md tests first, implement, open the PR steps/fix.md address review feedback on the same branch formats/questions.md how to ask: proposal first, options, defaults formats/spec.md the spec template and the definition of ready Store the fire URL and token as repo secrets. From inside Claude Code, pipe them from the clipboard. A command has no terminal, so gh secret set without a value silently stores garbage: pbpaste | gh secret set ROUTINE FIRE URL pbpaste | gh secret set ROUTINE FIRE TOKEN Step 2: a workflow that routes events to steps One workflow handles issues. It reads the event and the issue's current labels, moves the labels, and picks a step: on: issues: types: opened, labeled issue comment: types: created case "$EVENT" in issues/opened relabel --add-label pipeline:refining step=refine ;; issues/labeled if "$LABEL" = "pipeline:approved" ; then step=build; fi ;; issue comment/created if "$COMMENT BODY" == "< -- pipeline-bot:spec -- " ; then relabel --remove-label pipeline:refining --add-label pipeline:ready-for-signoff elif "$COMMENT BODY" == "< -- pipeline-bot" ; then : Claude's own comment: never fire on it elif has pipeline:refining || has pipeline:ready-for-signoff; then step=refine a human answered, or wants a change fi ;; esac Then it fires the routine. The step goes on the first line, followed by the issue body and the whole comment thread, because every run starts with no memory . The thread is all it knows: text=$ printf 'Step: %s\nIssue: %s\nTitle: %s\n\n Issue body\n\n%s\n\n Discussion\n\n%s' \ "$STEP" "$ISSUE NUMBER" "$ISSUE TITLE" "$ISSUE BODY" "$discussion" curl -sS --fail-with-body -X POST "$ROUTINE FIRE URL" \ -H "Authorization: Bearer $ROUTINE FIRE TOKEN" \ -H "anthropic-version: 2023-06-01" \ -H "Content-Type: application/json" \ -d "$ jq -n --arg text "$text" '{text: $text}' " Pass issue text through environment variables and jq , never inline it into the script. An issue body is user input. Why does the label shuffle never loop? Changes made with the workflow's own GITHUB TOKEN don't trigger workflows. Claude's comments do, because routines post as your GitHub account . That's also why you can't spot Claude's comments by author, and why every one of them starts with a < -- pipeline-bot… -- marker. Step 3: refinement that doesn't feel like a form The refine step's job is to turn "I want a random color of the day page" into something buildable without making you fill in a template. Most of steps/refine.md is about restraint: decide everything you reasonably can, list it as an assumption, and only ask what nobody but the human knows or what changes the scope. formats/questions.md fixes the shape of every question round: Proposal: what I'd build, in 2–3 sentences. Q1. Same color for everyone, or random per visit? - a Same for everyone, derived from the date recommended - b New random color on every page load Decided comment if you disagree : - Route /color , no database, no client JS Reply with e.g. Q1: a , free text, or go with defaults . At most five questions, each with a recommended option, so "go with defaults" is always a complete answer. Each round, Claude also rewrites the whole issue text as the current spec: goal, scope, numbered acceptance criteria, edge cases, how to verify in the preview. It puts that at the end of its comment between markers, and the workflow writes it into the issue, keeping the original request collapsed at the bottom: php awk '/< -- pipeline-bot:issue-body -- /{f=1; next} /< -- \/pipeline-bot:issue-body -- /{f=0} f' \ <<<"$COMMENT BODY" new.md gh issue edit "$ISSUE NUMBER" --body-file body.md new.md + collapsed original The issue becomes the thing being shaped. Comments hold the conversation. GitHub keeps every earlier version of the body. A definition of ready gates the spec: nothing open, every criterion checkable in the preview and as a test, small enough for one PR. When it holds, Claude posts a < -- pipeline-bot:spec -- comment and the label moves to pipeline:ready-for-signoff . Your sign-off is adding pipeline:approved . Step 4: acceptance criteria become tests Write criteria the way a test would check them: roles, accessible names, visible text, link targets. "The page shows the hex code", not "it looks right". The build step writes one test per criterion first, named after it, then implements until they pass: js describe "color of the day page 6 ", = { test "AC2: shows the hex code of the day", = { render <ColorOfTheDay date="2026-10-07" / ; expect screen.getByText colorForDate "2026-10-07" .toBeInTheDocument ; } ; } ; It may not weaken or delete a test to get green; if a criterion looks wrong, it asks. The PR body maps each criterion to its test. CI runs lint, test and build on every PR, so a PR that doesn't meet its spec fails where everyone can see it. That changes what review is about. You stop asking "does this look right?" and start asking "are these the right criteria?". Step 5: the review loop A second workflow listens to PR comments and reviews: on: issue comment: types: created pull request review: types: submitted pull request review covers both full reviews and single inline comments, which GitHub records as one-comment reviews. Don't add pull request review comment : it fires once per inline comment, so a five-comment review would start five parallel fix runs. The workflow sends the routine the PR, the spec the linked issue's body and all feedback, with inline comments carrying their file, line and comment ID. Anything Claude hasn't answered yet is marked NEW . The fix step works on the same branch and sorts each item: a clear change gets done, a question gets answered, an unclear request gets a follow-up question, and anything out of scope gets declined with a reason. Changed behaviour updates the matching criterion's test. Its summary comment ends with a hidden block of replies, one per inline comment: < -- pipeline-bot:replies {"comment id": 4208281449, "status": "done", "reply": "Done: removed the date line."}, {"comment id": 4208281450, "status": "answered", "reply": "It renders per request because…"} -- A workflow job posts each reply in its original thread and resolves the thread only when the status is done . Questions stay open, so the open threads are exactly what still needs you. Commenting on a PR, even through the issues API, needs pull-requests: write . Step 6: guardrails you'll want on day one One build per issue. I approved an issue while it was still being refined, then commented "LGTM" a minute later. Both started a build, and I got two PRs. Firing a routine is fire-and-forget, so a running build is invisible unless you record it. Firing a build now adds pipeline:building ; nothing starts another build while it's there or once a PR closes the issue. Claude's "PR opened" or questions comment removes it. Verify the base against GitHub. One build branched from the sandbox's origin/main , which turned out to be hours old. The PR was missing half the repo, including the Vercel config, so the preview failed. Fetch explicitly and check against git ls-remote , which asks GitHub directly, before building and again before pushing: git fetch origin main && git checkout -b claude/issue-6-color-of-the-day FETCH HEAD main sha=$ git ls-remote origin refs/heads/main | cut -f1 git merge-base --is-ancestor "$main sha" HEAD || echo "STALE BASE" git ls-remote --heads origin "refs/heads/claude/issue-6- " another build got here first? Debugging: read the run log When a run goes wrong, don't guess from the diff. From a local Claude Code session you can list a routine's runs and read the log of any of them: the exact payload it received, every command and its output, which tools it used to talk to GitHub. Just ask: "why did PR 7's preview fail? check the routine's run log". The stale base and the branch-name problem were each found in a couple of minutes this way. The log showed the session reading the new instructions file, then running git checkout -b … origin/main and landing on an old tree. Gotchas - Routines act as you. Claude's PRs carry your name, and GitHub won't let you approve your own PR. A bot account or GitHub Apps per role fix that. - Old PRs miss new workflows. Review events read the workflow from the PR's test merge, so a PR opened before review.yml existed never fires it. Add workflow dispatch so you can fire a step by hand. - API tokens are web-only. The routine's token can only be created in the claude.ai UI, and only its first characters can be read back. - Routines are a research preview. The API and limits may change. When this fits It fits well-scoped changes on a codebase with decent tests https://madewithlove.com/blog/your-codebase-is-part-of-the-prompt/ , owned by a team that already lives in GitHub. Refinement becomes an async conversation with a proposal on the table instead of a meeting, which is how remote teams https://madewithlove.com/blog/why-freedom-beats-control-building-a-proper-remote-culture/ want to work anyway. It fits less well when the hard part is the decision itself. If nobody knows what "done" means, multiple-choice questions won't find out for you. And the shape extends. A database per PR is the next step for this repo. Another state, say a reviewer agent https://madewithlove.com/blog/ai-code-review-tools/ with its own identity, is another markdown file in steps/ and one rule in a workflow. What surprised me most is where the effort went. Barely any of it went into code. It went into the instructions about when not to proceed : when to ask instead of guess, when a spec is ready, when a push didn't really happen. That's where the quality of this pipeline lives.