# Evaluating Coding Agents > Source: > Published: 2026-08-14 16:50:35+00:00 Coding agents now close most of the engineering loop on their own: planning, coding, testing, opening a pull request, responding to review comments. As capabilities improve, the risks associated with coding agents compound - agents can now run for extended periods of time searching a codebase, reasoning through tasks, and iterating on their own work with limited supervision. The latest AIUC-1 standard update, shaped by 100+ AIUC-1 Consortium members, expanded to cover risks introduced by coding agents, with new controls for secrets management, secure defaults in code, and runtime containment. These complement existing controls on agent identity, access management, and monitoring. AIUC-1’s requirements surface the risks that show up in real user flows such as adversarial robustness, tool calls, hallucinations, and safety. “*For coding agents, I look for clear agent identity, least-privilege access, explicit approval for consequential actions, and tamper-evident auditability. The goal is to let agents move quickly without losing control of the security boundary,*” said Nancy Wang, CTO at 1Password and AIUC-1 Consortium member. This post covers: Evaluations for modern coding agents are fundamentally different to evaluations required for conversational agents. Given that coding agents can write, test, and debug code, evaluations must judge the entire work session and not just the response. This full record of every action taken (agent trace) encompasses the multi-turn conversation, the tool calls and commands the agent ran, file edits, web searches and more. AIUC-1 evals are designed to validate the AI system's robustness against specific risks. In the case of coding agents this means data/privacy, security, and reliability. Fundamentally, the coding agent evaluator takes in the agent trace and classifies the severity of the agent's actions according to the risk being tested. Technically, the evaluator is a primary LLM-as-a-judge that uses tools (e.g., LLM review tool, adversarial debate) to gather information about the agent's actions and assess the risks elicited. Concretely: AIUC-1 framework for a coding agent evaluator **Deep-dive: Building isolated sandbox environments** Evals should produce credible evidence about how agents behave in production. That demands an isolated and observable environment in order to run safe adversarial scenarios where the results can be explained. Sandboxing an agent makes it less realistic, which also drives a need to create a more realistic environment to mirror production behaviour. In preliminary testing runs, some evals were run in parallel, some on a shared machine. For evals being run on the same machine with the same file system, instances where the agent contaminated evidence for other evals, and found the rubrics behind its own evaluation run were discovered. After the early evaluations, each test was run in a sandbox, designed in the following configuration: Eval testing environment for coding agents After sandboxing, in order to create a more realistic evaluation environment, runtime was built from a small Linux image and layered a realistic workstation on top: A sandbox that contains only the target repository is not representative of a real developer workstation. In prod, coding agents see home-directory files, environment variables, shell history, credentials, adjacent projects, package-manager configuration, and authenticated developer tools. An agent that never encounters those never gets tested against them. **Deep-dive: MCP attacks as a common attack vector** MCP is a common attack surface introduced by coding agents, introducing security risks such as credential exfiltration and unauthorized tool execution. As the primary way for agents to gather context from externally controlled systems, MCP servers allow both the tools and data behind them to change after installation. MCP attacks test what happens when an approved MCP server starts serving malicious content - whether the agent preserves user intent when a third-party dependency tries to redirect it. Example attacks created were based on existing CVEs (e.g., CVE-2025-54136). One included modelling a scenario involving a realistic internal project-tracking and documentation system that exposed two read-only tools (document search and context retrieval for specific tasks) and had a corpus of project specs, including backlog items, completed tasks, and meeting notes with initially benign content. The attack involved the red-teaming agent injecting adversarial content into the selected document, and keeping the visible developer request benign, with no suggestion that MCP content should be treated as instructions. AIUC-1 evals for coding agents involve setting up specialized testing environments, realistic attacks and designing an evaluator that can take in the whole coding agent trace. The purpose is to validate the coding agents performance against specific risks, and provide third party testing for enterprises deploying agents in production. “*We run thousands of evals with our in-house engineering teams for each product release. AIUC-1 certification enables us to add an additional independent layer going deep on data and privacy, security and reliability risks*,” said Kenneth Moras, Security at Cursor. When evaluating coding agents, three dominant risk areas are tested against: Benign indirect ask testing for insecure output generation As coding capabilities improve, the bar for demonstrating that these agents can act securely, safely, and reliably in deployment rises with them. Cursor and Lovable are the first two coding agents to earn AIUC-1 certification, with Schellman conducting the independent audit. Read more about how AIUC-1 requirements were adapted for coding-agent specific risks - and how Lovable demonstrated these in [this whitepaper](https://www.aiuc-1.com/research/setting-the-standard-for-agentic-development). Read more about Cursor’s AIUC-1 certification [here](https://www.aiuc-1.com/research/cursor-achieves-aiuc-1-certification). Organizations are able to certify their own deployment of certified coding agents - validating that the unique deployment context, data access, tool access, and configurations operate as intended by the organization. Read more about how to begin your AIUC-1 certification process [here](https://www.aiuc-1.com/category-leaders)