{"slug": "foreman-101-agentic-coding-as-kubernetes-resources", "title": "Foreman 101: agentic coding as Kubernetes resources", "summary": "Foreman is an agentic coding system that runs as Kubernetes resources, treating model claims of success as untrusted requests rather than results. The system uses Workloads, agents, and verifiers to ensure only verified code lands, with a Helm chart installable on any cluster.", "body_md": "Foreman is an agentic coder that runs as Kubernetes resources. You describe work as a Workload, it decomposes into tasks, agents running on your nodes pick them up, and a branch comes out the other end with something deterministic standing between that branch and your main.\n\nThis is the walkthrough. Four objects to understand, an install, an agent, a verifier, and a real run. Every command and every output below is from a working cluster.\n\nForeman is deliberately small. Almost everything you do is one of these.\n\n`coder`\n\nand `verifier`\n\n.The shape of a run is: you apply a Workload, the controller synthesizes AgenticTasks, the scheduler routes each to a FleetNode whose agent can serve it, the agent runs the model in a loop with tools, and the result lands as a branch plus a verdict.\n\nWorth stating plainly, because it shapes every design decision: **the model is not trusted, and specifically its claim to have succeeded is not trusted.**\n\nA coder agent finishes by calling a tool that says \"I am done, verdict GO.\" Foreman treats that as a request, not a result. If the model says GO and produced no diff, the run is recorded as NO-GO. If the verifier's checks do not pass, the work does not land, no matter how confident the summary was.\n\nThat is the difference between an agent that writes code and a system you can leave running. Everything else in this post is plumbing around that idea.\n\nForeman ships as a Helm chart that depends on LLMKube core. Any cluster will do; a throwaway kind cluster is fine, and no GPU is required to follow along.\n\n```\nkind create cluster --name foreman-101\n\nhelm repo add llmkube https://defilantech.github.io/LLMKube\nhelm repo update\n\nhelm upgrade --install llmkube llmkube/llmkube \\\n  --namespace llmkube-system --create-namespace\n\nhelm upgrade --install foreman llmkube/foreman \\\n  --namespace foreman-system --create-namespace \\\n  --set agent.mode=native \\\n  --set 'agent.roles={worker,coder,verifier}'\n```\n\nTwo flags on that second install are worth understanding rather than copying.\n\n`agent.mode=native`\n\nruns the real agent loop. The chart's default is `stub`\n\n, which registers a FleetNode and does nothing, so that a chart install can be smoke-tested on a cluster with no model and no credentials.\n\n`agent.roles`\n\nis what the scheduler matches against. A node advertising only `worker`\n\nwill never be given coder work. Advertise the roles this node should actually serve.\n\nYou should end up with three pods and a registered node:\n\n``` bash\n$ kubectl get pods -n llmkube-system\nNAME                                          READY   STATUS    RESTARTS   AGE\nllmkube-controller-manager-777d9f756b-xb7xr   1/1     Running   0          56s\n\n$ kubectl get pods -n foreman-system\nNAME                                READY   STATUS    RESTARTS   AGE\nforeman-agent-568b74c49f-mnkc8      1/1     Running   0          27s\nforeman-operator-7c687499f9-k96mb   1/1     Running   0          27s\n\n$ kubectl get fleetnodes\nNAME                             PHASE   VERSION   HEARTBEAT   AGE\nforeman-agent-568b74c49f-mnkc8   Ready   0.9.12    20s         20s\n```\n\nForeman needs two kinds of credential, and they do different jobs.\n\nA **git credential** lets the agent read the issue and push the branch. A **model credential** lets it talk to a hosted API, and is not needed at all if you serve the model yourself.\n\n```\n# git: the API token, plus the credential used for the push\nkubectl create secret generic foreman-github \\\n  --from-literal=GITHUB_TOKEN=\"$GITHUB_TOKEN\" -n foreman-system\n\nkubectl create secret generic foreman-git-credentials \\\n  --from-literal=token=\"$GITHUB_TOKEN\" -n foreman-system\n\nhelm upgrade foreman llmkube/foreman -n foreman-system --reuse-values \\\n  --set agent.githubToken.secretName=foreman-github \\\n  --set coder.gitCredentialsSecret=foreman-git-credentials \\\n  --set agent.gitRemoteURL=https://github.com/<you>/<repo>\n\n# model: read from the AGENT's namespace, not the operator's\nkubectl create secret generic anthropic-key \\\n  --from-literal=apiKey=\"$ANTHROPIC_API_KEY\" -n default\n```\n\n`agent.gitRemoteURL`\n\nis where branches are pushed. Pointing it at your fork keeps agent output off the upstream repository, which is the arrangement you want while you are still learning what it does.\n\nNote the namespaces differ, and it is not arbitrary. The git secrets are consumed by the agent process, so they live alongside it in `foreman-system`\n\n. The model credential is resolved per Agent, from *that Agent's* namespace, so it belongs wherever you create your Agents. Here that is `default`\n\n.\n\nAn Agent needs a model. The `cloud-proxy`\n\nprovider speaks to any OpenAI-compatible endpoint, so this path works with a hosted API and no GPU.\n\n```\napiVersion: foreman.llmkube.dev/v1alpha1\nkind: Agent\nmetadata:\n  name: claude-coder\n  namespace: default\nspec:\n  role: coder\n  provider: cloud-proxy\n  providerConfig:\n    # /chat/completions is appended, so give the /v1 prefix.\n    baseURL: https://api.anthropic.com/v1\n    model: claude-sonnet-5\n    apiKeySecretRef:\n      name: anthropic-key\n      key: apiKey\n  tools:\n    - read_file\n    - write_file\n    - str_replace\n    - grep\n    - bash\n    - submit_result\n  maxTurns: 12\n  contextWindowTokens: 30000\n  maxOutputTokens: 4096\n  requiredCapability:\n    roles: [coder]\n  systemPrompt: |\n    You are a senior staff software engineer.\n\n    You fix one issue per run. The repository is already cloned in your\n    workspace, checked out on a branch created for you.\n\n    You communicate only by calling tools. Every action is a tool call.\n\n    Work in this order:\n      1. read_file / grep to find the change site. Do not survey the whole\n         repository; find the file and start.\n      2. str_replace or write_file to make the change. Make your first edit\n         early, even if incomplete, then refine it.\n      3. bash to verify. Run the narrowest check that proves your change.\n      4. submit_result to finish.\n\n    submit_result is terminal. Call it with verdict=GO and a commit_message\n    when the change is complete and verified, or verdict=INCOMPLETE if you\n    cannot finish, saying plainly what is missing.\n\n    Make the minimum change that fixes the issue. Do not refactor adjacent\n    code, and do not fix unrelated problems you notice on the way.\n```\n\nThree fields there deserve more than a glance.\n\n**The budgets are a cost control.** Defaults are 120 turns and a 90k context, and every turn re-sends the whole context to a metered API. Twelve turns and 30k is the difference between a run that costs pennies and one that does not.\n\n**The tool list is the agent's authority.** It cannot do anything not on that list. Removing `bash`\n\ngives you an agent that can edit but not execute; removing the write tools gives you one that can only read and report.\n\n**The system prompt is the behaviour.** The ordering instructions above are not decoration. Left to explore, a coding model will survey a repository for a long time before touching it, so \"find the file and start\" and \"make your first edit early\" are load bearing. So is the scope fence at the end, which is what keeps a one-line fix from arriving as a refactor.\n\nThis is the object that makes Foreman different, and it is the smallest one.\n\n```\napiVersion: foreman.llmkube.dev/v1alpha1\nkind: Agent\nmetadata:\n  name: gate\n  namespace: default\nspec:\n  role: verifier\n  provider: local\n  tools:\n    - run_gate_job\n  requiredCapability:\n    accelerator: any\n    roles: [verifier]\n```\n\nThere is no model here. No provider config, no API key, no system prompt. An Agent with an empty `inferenceServiceRef`\n\nis a *deterministic agent*: Foreman skips the model loop entirely and just runs the tool.\n\n`run_gate_job`\n\nsubmits a Kubernetes Job that clones the branch and runs your checks. In a Go repository that is your build, vet, linter and tests, plus a bite check that reverts the production change and re-runs the new tests to confirm they fail without it. A test that still passes with the fix reverted is not a test.\n\nThe component that decides whether the AI's work is acceptable is not itself AI. That is the whole point.\n\nA Workload is what you author day to day. The only strictly required field is `intent`\n\n.\n\n```\napiVersion: foreman.llmkube.dev/v1alpha1\nkind: Workload\nmetadata:\n  name: abtest-s1\n  namespace: default\nspec:\n  repo: defilantech/LLMKube\n  issues: [1311]\n  coderAgentRef:\n    name: claude-coder\n  verifierAgentRef:\n    name: gate\n  openPullRequest: false\n  intent: |\n    Fix #1311: HPA-driven scale-up is not gated by GPUQuota, and\n    status.usedGPUCount under-reports because it counts the admitted replica\n    figure, not the autoscaling maxReplicas the service can reach. Charge\n    autoscaling.maxReplicas (not spec.replicas) toward GPUQuota usage so the\n    cap reflects the worst case, and reject admission when maxReplicas would\n    exceed the quota. Add a regression test.\n```\n\nThe intent is a brief to a competent engineer who has not seen your codebase. Being exact about the change, charge `maxReplicas`\n\nand not `spec.replicas`\n\n, and naming the deliverable (a regression test) does more for the outcome than any model setting. Where there is one obvious file to start in, name it; here the fix spans two accounting paths, so the brief names both.\n\nApply it and watch:\n\n``` bash\n$ kubectl get workload,agentictask\nNAME                                     PHASE        REPO                  AGE\nworkload.foreman.llmkube.dev/abtest-s1   Dispatched   defilantech/LLMKube   8s\n\nNAME                                                  PHASE     KIND        VERDICT\nagentictask.foreman.llmkube.dev/abtest-s1-code-1311   Running   issue-fix\n```\n\nThe Workload synthesized a task named `abtest-s1-code-1311`\n\n: the Workload name, then the step, then the issue. With a verifier configured you also get a `abtest-s1-verify-1311`\n\nthat waits on the coder finishing.\n\nA finished task carries its verdict and the evidence for it.\n\n```\nverdict: GO\nsummary: \"Charge autoscaling.maxReplicas (not spec.replicas) toward\n          GPUQuota usage in both the admission webhook and the GPUQuota\n          status reconciler, so HPA scale-up is gated at admission and\n          status.usedGPUCount reflects the worst-case headroom. Added\n          regression tests.\"\nturnCount: 64\nelapsedSec: 1608.38\nbranch: foreman/abtest-s1/issue-1311\ncommitSHA: 87a0d70c804a6fc4a51fba6550083e976d0d2c06\n```\n\nSixty-four turns and about twenty-seven minutes: a real change across two accounting paths with a regression test, not a one-line edit, and the branch is on the fork. The full transcript is persisted as a ConfigMap, referenced from the task status, so you can read every tool call the model made rather than guessing at it.\n\nThe verdicts you will actually see, and what each means:\n\n`GO`\n\nthe coder finished and produced a diff.`NO-GO`\n\nit did not, or it claimed success without one.`INCOMPLETE`\n\nit ran out of turns or could not verify its own work.`GATE-PASS`\n\nthe verifier's checks passed on the branch.`GATE-FAIL`\n\nthey did not, and the branch does not land.A verifier runs whatever you tell it to. You can point one at an existing branch to see the mechanism on its own.\n\n```\napiVersion: foreman.llmkube.dev/v1alpha1\nkind: AgenticTask\nmetadata:\n  name: gate-check\n  namespace: default\nspec:\n  kind: verify\n  agentRef:\n    name: gate\n  gateProfile:\n    # \"generic\" runs the commands below instead of the Go make targets.\n    language: generic\n    image: golang:1.26\n    commands:\n      # Swap this line to see the other verdict.\n      format: \"grep -q Foreman docs/site/foreman/README.md\"\n      build: \"true\"\n      lint: \"true\"\n      test: \"true\"\n  payload:\n    repo: defilantech/LLMKube\n    branch: main\n```\n\nRun it once as written, then once with a string that is not in the file, and the Job logs show exactly what happened:\n\n```\n=== clone defilantech/LLMKube @ main ===\n=== grep -q Foreman docs/site/foreman/README.md ===\nGATE PASS\n\n=== clone defilantech/LLMKube @ main ===\n=== grep -q NO_SUCH_STRING_IN_THIS_REPO docs/site/foreman/README.md ===\nGATE FAIL\n```\n\nSame agent, same branch, one command different, opposite verdicts. The task status carries `deterministic: true`\n\n, because no model was consulted about whether the work was good. A grep decided.\n\nEverything above used a hosted API because it is the version anyone can try today. The reason LLMKube exists is the other path: the model runs on hardware you own and nothing leaves the building.\n\nThe Agent barely changes. No API key and no external URL, just a reference to an InferenceService that LLMKube is already serving.\n\n```\nspec:\n  role: coder\n  provider: local\n  inferenceServiceRef:\n    name: laguna-s-2.1-strix\n  model: laguna-s-2.1-q4km-strix\n  # tools, budgets and systemPrompt are identical to the cloud-proxy Agent\n```\n\nForeman resolves that to the service's in-cluster address at task time. The agent that runs the loop must be able to reach it, which means an in-cluster agent for an in-cluster service. Everything downstream, the tools, the verdicts, the gate, is identical.\n\nThe model behind that reference in our own cluster is a 118B-parameter, roughly 8B-active open-weight coder on an AMD Strix Halo box with 128GB of unified memory. Four of its pull requests merged into LLMKube while this post was being written, each through the full gate.\n\nBeing straight about scope, because it determines whether this is useful to you.\n\nMechanical, well-scoped work lands: a documented bug with a known change site, a refactor with a clear contract, a missing test, a docs correction. Give it the file and a scope fence and it will do the job and stop.\n\nArchitectural work does not, yet. A change that requires holding two packages in your head and deciding which of two components should own a behaviour is still yours. When a run fails on that kind of task it usually fails honestly, as INCOMPLETE with a summary, rather than landing something plausible and wrong. That is the failure mode you want, but it is still a failure.\n\nA kind cluster, the two Helm charts, one coder Agent against whatever API key you already have, one verifier, and a genuinely small issue. Twenty minutes end to end.\n\nThen add the verifier before you add anything else. An agent without one is autocomplete with a Kubernetes API; an agent with one is a system you can leave running.\n\nThe docs are at [llmkube.com/docs/foreman](https://llmkube.com/docs/foreman) and the code is [on GitHub](https://github.com/defilantech/LLMKube). Both are Apache 2.0.", "url": "https://wpnews.pro/news/foreman-101-agentic-coding-as-kubernetes-resources", "canonical_source": "https://dev.to/defilan/foreman-101-agentic-coding-as-kubernetes-resources-4k33", "published_at": "2026-07-28 18:55:51+00:00", "updated_at": "2026-07-28 19:04:00.395299+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "ai-infrastructure"], "entities": ["Foreman", "LLMKube", "Helm", "Kubernetes", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/foreman-101-agentic-coding-as-kubernetes-resources", "markdown": "https://wpnews.pro/news/foreman-101-agentic-coding-as-kubernetes-resources.md", "text": "https://wpnews.pro/news/foreman-101-agentic-coding-as-kubernetes-resources.txt", "jsonld": "https://wpnews.pro/news/foreman-101-agentic-coding-as-kubernetes-resources.jsonld"}}