{"slug": "how-kong-built-an-agentic-engineering-platform", "title": "How Kong Built an Agentic Engineering Platform", "summary": "Kong built KSAI (Kong Skills for Agentic Intelligence), an internal agentic engineering platform that runs AI code review and implementation agents inside GitHub Actions CI, with the Kong AI Gateway handling LLM routing, per-repo token usage attribution and budget enforcement. In an A/B test where 10% of runs executed both Claude Code and OpenCode on the same model and effort, Kong found Opus run costs were similar but Sonnet runs were 40% cheaper on OpenCode. KSAI runs OpenCode inside a bubblewrap sandbox with a read-only root filesystem, isolated tmpfs /tmp, and separated user and process namespaces to constrain agent behavior.", "body_md": "KSAI stands for Kong Skills for Agentic Intelligence. It brings AI-assisted code review and implementation into GitHub, where our engineering work is actually done. The AI agents run in CI, so closing a laptop doesn't stop the work. By leveraging KSAI, we establish agentic engineering best practices that scale securely across our entire organization.\n\nWe built KSAI to offer reusable agentic tooling across engineering. Previously everyone was building something for their own, using their skills/prompts and tricks. We wanted all this knowledge that we gathered during the exploration of the agentic solutions packaged into a platform that gives best practices to everyone. We also wanted to dogfood our Kong AI Gateway, which gives us proper observability governance and cost tracking, which we were lacking at that time. Unlike generic reverse proxies, Kong AI Gateway provides purpose-built LLM routing, per-repo token usage attribution, and fallback mechanisms essential for CI/CD agentic workflows.\n\nAnother deciding factor for us was the possibility to hide LLM providers behind a single abstraction that allows us to easily switch between LLMs and gives us more models to choose from and different providers to pick from. This vendor-neutral LLM abstraction layer prevents vendor lock-in and allows us to route requests to the most cost-effective model for a given task.\n\nIn this article we will be focusing on the agentic reviewer we built and use daily now. As it was already widely adopted in engineering.\n\n## Architecture overview: AI agents in CI/CD pipelines\n\nKSAI began as a plugin marketplace for reusable agent skills. But it quickly evolved into a platform on which engineers can build agentic workflows.\n\nOut of simplicity we decided to build KSAI around GH actions as a runtime engine. We didn't need any extra infrastructure we would need to worry about. Our architecture provides several benefits:\n\n- Each repository adds one workflow file.\n\n- KSAI Control Plane (CP) is responsible for triggering actions.\n\n- We maintain the execution code in the shared workflows, and teams choose when to adopt a new version.\n\n- Workload identities are provided by GitHub, so our code never actually leaves GitHub.\n\nThanks to KSAI CP, we can have granular control over what model we choose for the work; we can do AB testing on models and prompts. We can have more control over concurrency than simply using GitHub Actions. This infrastructure allows engineering teams to easily conduct A/B tests on different LLM prompts in CI, ensuring prompt engineering changes actually improve precision and recall before a fleet-wide rollout.\n\nThe runtime and the model are separate choices. KSAI started by running Claude Code and Anthropic models, as we were widely using them in engineering. It was serving its purpose, but we quickly realised that we need to keep this vendor neutral if we want to easily incorporate the OSS models into the mix.\n\nBefore we fully switched to OpenCode, we wanted to gather some data from actual runs. We have set up an A/B testing scenario, where 10% of the runs run both Claude and OpenCode reviews at the same time. Both engines run the same model and effort. We didn't see much of a difference in the cost of Opus runs, but surprisingly Sonnet runs were 40% cheaper on OpenCode.\n\nThe agent's tools inspect the repository. Kong AI Gateway handles its model traffic, including routing and usage attribution, and enforces budgets. We handle provider compatibility there, so individual repositories don't have to.\n\n### How to sandbox AI agents in GitHub Actions\n\nWhen building KSAI, we knew that we cannot let agents do what they want, and we need to keep them in check. We took a few countermeasures to prevent agents from roaming freely.\n\nKSAI runs OpenCode in GitHub Actions inside a `bubblewrap` sandbox. The root filesystem is mounted read-only, `/tmp` is an isolated `tmpfs`, and user and process namespaces are separated. The repository workspace is writable where implementation requires it, but KSAI’s trusted checkout under `_ksai` is remounted read-only. OpenCode itself adds a second control layer. Its permission map denies tools, shell commands, and external-directory access by default, then enables only those required for the current phase.\n\nSecrets are constrained independently of filesystem access. GitHub tokens, OIDC variables, and runner credentials are removed from the agent’s environment. The model-access token is brokered outside the sandbox and exposed through a read-only file. Trusted scripts and runtime configuration are also mounted read-only; only OpenCode’s own cache and state directories remain writable.\n\nAuthorisation and approval remain separate boundaries. KSAI checks whether the requester may start a paid or writing run, and its approval action prevents the pull-request author, commit authors, and recognised co-authors from approving their own changes.\n\n### How the AI code review agent works\n\nAt the beginning KSAI was built on top of skills we already had. So it was natural to just create subagents for specific reviews. The first version of KSAI looked like this:\n\nThe orchestrator had access to all subagents and was supposed to decide which subagent it needed to do a quality review on. The biggest issue with this approach was that each subagent was exploring the codebase and changes on its own. In effect we were burning tokens multiple times for the same thing.\n\n### Introducing triage\n\nWhile analysing the cost and performance of our current setup, we quickly realised that this is not the best solution. We moved the main decisions from the LLM orchestrator to a static step that triages review requests. It decides what skill the agent will need, what model and effort it should use and if review is even worth starting. Thanks to this, we were able to flatten our structure and gain more control over the review process. The new flow with triage step looks like this\n\n### Reducing review cost with triage\n\nWhen we started, we ran every review using the Opus model with high effort. It gave us great results quality-wise but was super expensive. Thanks to the addition of the triage step, we were able to model and effort based on the complexity of a task. For simpler and smaller reviews we were able to use the Sonnet model, which helped us reduce the cost of reviews.\n\nTriage was our first win on review cost and speed. In our early measurements, median review cost fell from $3.93 to $2.47, so about 37% of overall spend. This triage phase acts as our primary defense against unnecessary token burn, establishing best practices for AI code review triage and significantly reducing LLM token spend in continuous integration.\n\n## Reducing AI code review costs with open source models\n\nThe cost of running KSAI is a major concern for us, and LLM tokens can quickly pile up into a significant bill. The problem is that it is hard to optimize cost without switching models. Our triage strategy already proved itself, but we started looking at OSS models.\n\nWe started by creating our internal benchmarks to measure how models perform and how much they cost. We created a synthetic repo with proper history and opened 4 PRs on which we were comparing models. On each model we compute `blocking recall `(how many blocking defects the model found) and `precision` (how many findings are accepted by the judge), which allows us to compute F1 which balances missed defects against noisy findings. These metrics—`blocking recall` and `precision`—are the most critical metrics that matter when benchmarking LLMs for code quality, as they directly measure the agent's ability to catch real bugs without overwhelming developers with false positives.\n\nFrom these we compute a model score that takes into account the cost of a run.\n\n`model score = 100 × (F1 − review cost / V)`\n\nBy calculating a model score combining F1 and cost, we can make objective decisions about when to trade slight dips in precision for massive cost savings.\n\nHere, `V` is the dollar value assigned to moving from F1 0 to F1 1. We show cost-first (V=$1), balanced (V=$5), and quality-first (V=$20) views.\n\nHere are the results of our first benchmarks:\n\nAs you can see, Opus was still a winning model, but its price per review is 7x higher than GLM 5.3 and 80x higher than GLM 5.3 flash. When we plot F1 over cost we can quickly see the differences:\n\nFor now we picked GLM 5.3 and GLM 5.3 Flash as our default reviewer, which reduced our cost of reviews significantly while keeping similar quality:\n\n## Rolling out AI code review across teams\n\nBy September 17, 22 teams were enrolled, covering 135 distinct repositories. KSAI already reviewed 7.8M lines of code and found more than 4k findings.\n\nTo streamline adoption of KSAI, we have built full self-service for it. Engineers can easily request access for their repositories, and KSAI will create onboarding PRs for them.This self-service model is the key to rolling out AI tooling to 100+ repositories smoothly without bottlenecking the platform team.\n\nSince KSAI changes rapidly, we needed a simple way of migrating people to new versions. For easy updates that are just version bumps, we just use Renovate to open PRs against consumer repos. For structural changes we build an internal mechanism that helps us to keep the whole fleet up to date.\n\nOur KSAI app has access to all enrolled repositories in GitHub. Thanks to this, we can monitor versions of KSAI used by consumers. KSAI CP understands action versioning and API changes made in different versions of it. Thanks to this we can create automated PRs to consumer repositories with updates to the new version. Automating version bumps of shared workflows across repos makes customer migration simple for us and ensures all teams are using the latest, most secure agent configurations.\n\n## Using KSAI to build KSAI\n\nWe know that dogfooding is super important, so we wanted to start using KSAI to build KSAI as soon as possible. We started using KSAI as a default reviewer in KSAI project and in all team projects as soon as possible. To immediately catch bugs and see if we like the actual UX of it.\n\nKSAI's bot authored 901 of the 1,573 commits in our September 17 repository snapshot, so more than a half. We use it to help build the system it runs in. It also now reviews all PRs. That's the handoff we want to trust at the end of the day: leave an agent a task, close the laptop, and come back to work where we can inspect or make a decision.\n\n## Frequently Asked Questions (FAQ)\n\n**How does Kong use AI for automated code reviews?** Kong uses a custom-built platform called KSAI (Kong Skills for Agentic Intelligence). Rather than acting as an inline autocomplete tool, KSAI deploys AI agents directly into GitHub Actions CI/CD pipelines. These agents asynchronously review pull requests, enforce coding standards, and identify blocking defects without requiring developers to keep their local environments running.\n\n**Why add a triage phase before running LLM code reviews?** Adding a static triage step prevents burning expensive LLM tokens on simple tasks. The triage phase evaluates the complexity of a review request and dynamically selects the appropriate model and effort level (e.g., routing a small PR to a cheaper, faster model like Sonnet or GLM 5.3 Flash instead of Opus). At Kong, this triage step reduced median review costs by roughly 37%.\n\n**What is the cost difference between commercial models (like Claude) and open-source models (like GLM)?** While high-tier commercial models like Claude Opus provide excellent quality, they can be cost-prohibitive at scale. In our benchmarks, Opus was 7x more expensive per review than the open-source GLM 5.3 model, and 80x more expensive than GLM 5.3 Flash. Switching to these open-source models allowed us to drastically cut costs while maintaining comparable precision and blocking recall.\n\n**What security controls sandbox AI agents in a pipeline?** To prevent AI agents from executing unauthorized commands or leaking data, KSAI employs strict sandboxing in GitHub Actions. This includes mounting the root filesystem as read-only, isolating namespaces, denying shell commands by default via an OpenCode permission layer, and brokering model-access tokens outside the sandbox so they are only exposed through a read-only file.\n\n**How do you get observability on per-repo LLM usage costs?** We route all AI agent model traffic through the Kong AI Gateway. This allows us to abstract the underlying LLM providers and gain granular observability, cost tracking, and token usage attribution on a per-repository basis, enforcing budgets automatically.\n\n**Should we switch from Claude Opus to GLM 5.3 for code reviews?** If cost optimization is a priority, switching to GLM 5.3 is highly recommended based on our F1 vs. cost model scores. While Opus slightly edges out open models in raw blocking recall, GLM 5.3 and 5.3 Flash offer a much better balance of cost-efficiency and quality for the vast majority of standard CI/CD code reviews.\n\nMarcin Skalski\n\nRecommended posts\n\n# Kong and Straiker: Runtime Security and Agent-Level Control for AI Agents\n\nKong AI Gateway handles the traffic patterns production agent systems depend on. The Straiker integration can be applied to configured traffic so the relevant request context is evaluated before the action continues. For LLM traffic, Straiker evalua\n\nParth Shah\n\n# Your Multi-Agent System Is Only as Reliable as Its Context Layer\n\nMulti-agent workflows live and die on context. Every agent-to-agent call and every agent-to-tool call is either a retrieval — fetching information the agent needs — or a mutation — changing state that downstream agents will depend on. At prototype s\n\nHugo Guerrero\n\n# The Architecture Decision Your Multi-Agent System Will Live With\n\nMulti-agent systems are, at their core, context distribution systems. Every agent in your workflow is a consumer and producer of context. The interesting architectural questions are all about how that context moves. Two operations drive everything:\n\nHugo Guerrero\n\n# Intelligent Model Routing: Kong AI Gateway Applies NVIDIA NeMo Switchyard Across Model Traffic\n\nAs AI adoption scales, applications evolve into complex systems of agents, orchestration layers, and context servers. Infrastructure lags behind — struggling with authentication, cost control, and data security across a provider list that change\n\nGrajesh Chandra\n\n# Why Your Kafka Event Streams Need an Event Gateway\n\nYou wouldn't expose a REST API without a gateway. What about Kafka?\nBefore we talk about an event gateway, it's worth grounding ourselves on what an event is. An event is simply a record of something that happened. In modern integration, there are\n\nWarren Vella\n\n# Local Previews and Agent-Driven Authoring for Your Konnect Dev Portal\n\nWhat it is\nThe Konnect Dev Portal Toolkit is a free extension published by Kong Inc. on the Visual Studio Code Marketplace (ID konghq.vscode-konnect-dev-portal-toolkit) and on Open VSX.\nKonnect Developer Portals are how teams publish API document\n\nAdam DeHaven\n\n# Enforce API Standards with Custom Linting in Kong Insomnia 13\n\nTo follow along, you’ll need: Insomnia 13 or later An Insomnia project with an API design document An OpenAPI 3.x specification Note: In Insomnia 13, custom linting supports Spectral rulesets using the top-level rules and extends properties. Custom\n\nJuhi Singh\n\n# Kong and Straiker: Runtime Security and Agent-Level Control for AI Agents\n\nKong AI Gateway handles the traffic patterns production agent systems depend on. The Straiker integration can be applied to configured traffic so the relevant request context is evaluated before the action continues. For LLM traffic, Straiker evalua\n\nParth Shah\n\n# Your Multi-Agent System Is Only as Reliable as Its Context Layer\n\nMulti-agent workflows live and die on context. Every agent-to-agent call and every agent-to-tool call is either a retrieval — fetching information the agent needs — or a mutation — changing state that downstream agents will depend on. At prototype s\n\nHugo Guerrero\n\n# The Architecture Decision Your Multi-Agent System Will Live With\n\nMulti-agent systems are, at their core, context distribution systems. Every agent in your workflow is a consumer and producer of context. The interesting architectural questions are all about how that context moves. Two operations drive everything:\n\nHugo Guerrero\n\n# Intelligent Model Routing: Kong AI Gateway Applies NVIDIA NeMo Switchyard Across Model Traffic\n\nAs AI adoption scales, applications evolve into complex systems of agents, orchestration layers, and context servers. Infrastructure lags behind — struggling with authentication, cost control, and data security across a provider list that change\n\nGrajesh Chandra\n\n# Why Your Kafka Event Streams Need an Event Gateway\n\nYou wouldn't expose a REST API without a gateway. What about Kafka?\nBefore we talk about an event gateway, it's worth grounding ourselves on what an event is. An event is simply a record of something that happened. In modern integration, there are\n\nWarren Vella\n\n# Local Previews and Agent-Driven Authoring for Your Konnect Dev Portal\n\nWhat it is\nThe Konnect Dev Portal Toolkit is a free extension published by Kong Inc. on the Visual Studio Code Marketplace (ID konghq.vscode-konnect-dev-portal-toolkit) and on Open VSX.\nKonnect Developer Portals are how teams publish API document\n\nAdam DeHaven\n\n# Enforce API Standards with Custom Linting in Kong Insomnia 13\n\nTo follow along, you’ll need: Insomnia 13 or later An Insomnia project with an API design document An OpenAPI 3.x specification Note: In Insomnia 13, custom linting supports Spectral rulesets using the top-level rules and extends properties. Custom\n\nJuhi Singh\n\n# Kong and Straiker: Runtime Security and Agent-Level Control for AI Agents\n\nKong AI Gateway handles the traffic patterns production agent systems depend on. The Straiker integration can be applied to configured traffic so the relevant request context is evaluated before the action continues. For LLM traffic, Straiker evalua\n\nParth Shah\n\n# Your Multi-Agent System Is Only as Reliable as Its Context Layer\n\nMulti-agent workflows live and die on context. Every agent-to-agent call and every agent-to-tool call is either a retrieval — fetching information the agent needs — or a mutation — changing state that downstream agents will depend on. At prototype s\n\nHugo Guerrero\n\n# The Architecture Decision Your Multi-Agent System Will Live With\n\nMulti-agent systems are, at their core, context distribution systems. Every agent in your workflow is a consumer and producer of context. The interesting architectural questions are all about how that context moves. Two operations drive everything:\n\nHugo Guerrero\n\n# Intelligent Model Routing: Kong AI Gateway Applies NVIDIA NeMo Switchyard Across Model Traffic\n\nAs AI adoption scales, applications evolve into complex systems of agents, orchestration layers, and context servers. Infrastructure lags behind — struggling with authentication, cost control, and data security across a provider list that change\n\nGrajesh Chandra\n\n# Why Your Kafka Event Streams Need an Event Gateway\n\nYou wouldn't expose a REST API without a gateway. What about Kafka?\nBefore we talk about an event gateway, it's worth grounding ourselves on what an event is. An event is simply a record of something that happened. In modern integration, there are\n\nWarren Vella\n\n# Local Previews and Agent-Driven Authoring for Your Konnect Dev Portal\n\nWhat it is\nThe Konnect Dev Portal Toolkit is a free extension published by Kong Inc. on the Visual Studio Code Marketplace (ID konghq.vscode-konnect-dev-portal-toolkit) and on Open VSX.\nKonnect Developer Portals are how teams publish API document\n\nAdam DeHaven\n\n# Enforce API Standards with Custom Linting in Kong Insomnia 13\n\nTo follow along, you’ll need: Insomnia 13 or later An Insomnia project with an API design document An OpenAPI 3.x specification Note: In Insomnia 13, custom linting supports Spectral rulesets using the top-level rules and extends properties. Custom\n\nJuhi Singh\n\n## Ready to see Kong in action?\n\nGet a personalized walkthrough of Kong's platform tailored to your architecture, use cases, and scale requirements.", "url": "https://wpnews.pro/news/how-kong-built-an-agentic-engineering-platform", "canonical_source": "https://konghq.com/blog/engineering/agentic-engineering-platform", "published_at": "2026-10-16 15:42:00+00:00", "updated_at": "2026-10-16 15:51:25.045736+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "ai-infrastructure", "mlops"], "entities": ["Kong", "KSAI", "Kong AI Gateway", "GitHub Actions", "OpenCode", "Claude Code", "Anthropic", "bubblewrap"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/how-kong-built-an-agentic-engineering-platform", "markdown": "https://wpnews.pro/news/how-kong-built-an-agentic-engineering-platform.md", "text": "https://wpnews.pro/news/how-kong-built-an-agentic-engineering-platform.txt", "jsonld": "https://wpnews.pro/news/how-kong-built-an-agentic-engineering-platform.jsonld"}}