Open-source AI agent workflow for auditing Solidity smart contracts

Chain-shield released AI Agent Audit, an open-source Rust tool that uses large language models to audit Solidity smart contracts for security vulnerabilities. The tool generates proof-of-concept exploits and professional reports, and has been tested in Code4rena competitions.

AI Agent Audit is a Rust command-line tool for AI-assisted security review of Solidity repositories. - discovers security vulnerabilities in Solidity and EVM-based codebases - deduplicates and validates findings - generated runnable PoC for each validated High/Medium finding - create professional audit report for each validated finding in markdown I used this tool to compete in Code4rena competitions and the results were encouraging: https://code4rena.com/@saraswati https://code4rena.com/@saraswati The repository is in public beta. It is meant to accelerate expert review, not replace manual auditing. - Public beta. - Solidity and EVM-focused. - Repository source, docs, and derived context are sent to third-party LLM providers you configure. - The current default audit pipeline uses ChatGPT/Codex OAuth for OpenAI access and runs the active review flow on gpt-5.5 . Deduplication helpers use gpt-5.4 with low reasoning. - Codex is the recommended default path for the current validation workflow and operating model. - Startup performs a one-time ChatGPT sign-in if needed and reuses the cached session on later runs until the token expires. OPENAI API KEY is supported as a secondary fallback for Rust OpenAI calls by setting AI AGENT AUDIT OPENAI BACKEND=api . ANTHROPIC API KEY , GEMINI API KEY / GOOGLE AI API KEY , and DEEPSEEK API KEY are still supported by the agent layer, but they are not required by the default review path.- Discovery-style runs can be switched to Gemini by changing the defaults in src/config.rs /chain-shield/ai-agent-audit/blob/develop/src/config.rs if you want to use Google AI for patterns, actors, and invariants while keeping verification/reporting on OpenAI/Codex. - PoC generation and PoC verification are supported through validation-three-shot , which is the primary validation workflow. - Clones and builds Foundry or Hardhat repositories under ~/Desktop/Audit by default. - Generates audit scope and protocol docs from README/configured entry files into audit-docs/ . - Uses Slither-derived call graph and semantic data when static analysis succeeds. - Builds inheritance and interface-implementation indexes from Solidity source. - Generates contextual codeblocks for each in-scope contract. - Uses pattern libraries, invariant prompts, and actor-oriented context to discover candidate findings. - Verifies and deduplicates findings before producing report output. - Stores local SQLite state in .ai-agent-audit/ . - Smart contract auditors and security researchers. - Protocol teams doing internal review of Solidity codebases. - Engineers experimenting with AI-assisted audit workflows on repos they are allowed to share with external model providers. This project is not a hosted service, not a generic SAST scanner for every language, and not a substitute for human validation. - Rust stable toolchain. Install from rust-lang.org/tools/install https://www.rust-lang.org/tools/install . - Git. Install from git-scm.com/downloads https://git-scm.com/downloads . - Slither. Install from github.com/crytic/slither https://github.com/crytic/slither . - Foundry forge for Foundry repositories. Install from book.getfoundry.sh/getting-started/installation https://book.getfoundry.sh/getting-started/installation . - Node.js 18+ plus npm / npx , Yarn, pnpm, or Bun for JavaScript/Hardhat repositories. Install Node.js from nodejs.org https://nodejs.org/ . Install Bun from bun.sh https://bun.sh/ if the target repo uses bun.lock / bun.lockb . - Optional GITHUB TOKEN for private GitHub repositories. - Clone this repository and enter it. git clone https://github.com/chain-shield/ai-agent-audit.git cd ai-agent-audit - Create a local env file from the template. cp .env.example .env - Edit .env and set the values you actually need: RUST LOG=info AI AGENT AUDIT OPENAI BACKEND=codex Codex is the recommended default for cost. Optional Rust API fallback: AI AGENT AUDIT OPENAI BACKEND=api OPENAI API KEY=your openai api key Optional non-OpenAI provider keys: ANTHROPIC API KEY=... GEMINI API KEY=... DEEPSEEK API KEY=... - The first Codex-backed run will prompt you to sign in with ChatGPT if there is no cached Codex session yet. After that, the session is reused automatically until expiry. If you set AI AGENT AUDIT OPENAI BACKEND=api , Rust OpenAI calls use OPENAI API KEY instead and do not require Codex sign-in. - Copy the example config and point it at a Solidity repository. cp examples/audit-config.example.yaml audit-config.yaml Minimal example: repo: "https://github.com/example/protocol.git" audit type: "Client" code folders: - "src" - Build and run the tool. cargo build --release cargo run --release -- --config audit-config.yaml If you prefer not to use YAML for a simple run: cargo run --release -- https://github.com/example/protocol.git --audit-type Client For repos that keep contracts under contracts/ instead of src/ , set code folders accordingly. The main Rust run produces the initial audit artifacts and, by default, emits a ready-to-run validation-three-shot job. That validation workflow is where deeper filtering, PoC generation, PoC verification, and report hardening happen. If the target repository is private, set GITHUB TOKEN before running the tool. The clone path uses that token for GitHub HTTPS URLs. export GITHUB TOKEN=... --config <file loads YAML, and explicit CLI flags override YAML values. The current example file lives at examples/audit-config.example.yaml /chain-shield/ai-agent-audit/blob/develop/examples/audit-config.example.yaml . Code4rena Code4renaBounty ImmunefiBugBounty Sherlock Cantina Client Use Client for internal or client-style audits. Use the contest values when you want severity handling and report language aligned more closely with those platforms. Use Code4renaBounty for C4 bug bounty programs where only currently exploitable Critical/High issues with runnable PoCs should become submission candidates. Use ImmunefiBugBounty when the run should derive repo/docs/scope from an Immunefi program page and validate against Immunefi-style impact rules. immunefi bounty / --immunefi-bounty Derives repo, docs, and scope from an Immunefi bounty page. This is the standard entrypoint for ImmunefiBugBounty . code4rena bounty / --code4rena-bounty Derives repo, docs, and scope from a Code4rena bounty page. This is the standard entrypoint for Code4renaBounty when you want the tool to gather bounty context for you. code4rena contest repo / --code4rena-contest-repo Adds Code4rena contest context and V12 lookup support from a contest GitHub repo URL. code4rena contest url / --code4rena-contest-url Alias for code4rena contest repo . Provide only one of the two. Example configs: Immunefi-derived run audit type: "ImmunefiBugBounty" immunefi bounty: "https://immunefi.com/bug-bounty/example/information/" Code4rena bounty-derived run audit type: "Code4renaBounty" code4rena bounty: "https://code4rena.com/bounties/example" Code4rena contest run with extra contest context repo: "https://github.com/example/protocol.git" audit type: "Code4rena" code4rena contest repo: "https://github.com/code-423n4/2026-01-example" For enum-like values, YAML uses the Rust-style names such as Code4renaBounty , ImmunefiBugBounty , and HardhatYarn . CLI flags use the actual --help spellings, such as hardhat-yarn for --builder . | YAML key / CLI flag | Purpose | |---|---| repo / positional repo | Required HTTP S Git repository URL. | config / --config | Load a YAML config file. | immunefi bounty / --immunefi-bounty | Immunefi program URL used to derive repo, docs, and scope. | code4rena bounty / --code4rena-bounty | Code4rena bounty URL used to derive repo, docs, and scope. | code4rena contest repo / --code4rena-contest-repo | Code4rena contest GitHub repo URL used to derive contest docs, scope, and V12 context. | code4rena contest url / --code4rena-contest-url | Alias for code4rena contest repo ; do not set both. | subfolder / --subfolder | Analyze a subdirectory inside the cloned repo, useful for monorepos. | code folders / --code-folders | Source roots to scan for contracts. Defaults to "src" . | audit scope / --audit-scope | Local Markdown file containing scope notes or reviewer guidance. | doc folder / --doc-folder | Repo-relative folder containing Markdown docs to ingest. | custom doc / --custom-doc | Local Markdown file to use instead of auto-discovered root docs. | monorepo folders / --monorepo-folders | Local text file listing repo-relative package roots for monorepo-aware analysis. | exclude folders / --exclude-folders | Repo-relative folders to exclude from scope. | scoped files / --scoped-files | Local text file listing repo-relative files that should be treated as in scope. | validation supervision / --validation-supervision | Emit a Codex GUI three-shot validation job after report export. Defaults to gui ; set off to disable. | validation supervision overwrite / --validation-supervision-overwrite | Replace an existing non-terminal GUI validation job with the same run id. Defaults to false ; use only for intentional reruns. | context | YAML-only block for generated audit scope/docs context. Defaults to README.md , audit-docs , force regenerate: true , and 5000 tokens per generated Markdown file. | poc | YAML-only block for PoC runtime preferences, including fork policy and network-to-RPC-env mappings used when generated context includes PoC guidance. | benchmark | YAML-only block for benchmark telemetry and stable run id configuration. | audit type / --audit-type | One of Code4rena , Code4renaBounty , ImmunefiBugBounty , Sherlock , Cantina , Client . | builder / --builder | YAML: Foundry , Hardhat , HardhatYarn , Custom , Auto . CLI: foundry , hardhat , hardhat-yarn , custom , auto . Default is Auto / auto . | build cmd / --build-cmd | Required when builder: "Custom" is used. | via ir / --via-ir | Adds --via-ir to the Foundry build command. | force rebuild / --force-rebuild | Re-clone and rebuild even if a cached workspace already exists. | custom doc , audit scope , scoped files , and monorepo folders are read from local files you provide on the machine running the tool. subfolder , code folders , doc folder , and exclude folders are interpreted relative to the cloned target repository.- If no manual custom doc , audit scope , or scoped files are provided, the tool generates <repo-folder -docs.md , <repo-folder -scope.md , and <repo-folder -scope.txt in audit-docs/ . - The generated filename prefix preserves the cloned repo folder identity, including date/contest prefixes such as 2026-04-monetrix . context.force regenerate defaults to true for generated context. Legacy YAMLs that already provide all three manual context files are left alone when no context block is present. Minimal generated-context config: context: files: - README.md urls: v12 url: "auto" output dir: "audit-docs" force regenerate: true max tokens per file: 5000 The generator copies scope.txt from the cloned repo when present. If no scope.txt exists, it extracts in-scope Solidity paths from entry context files. By default the only entry file is README.md ; context.files can add or replace entry files. Links found in those entry files are treated as second-level candidates and fetched only when they look relevant to scope, known issues, protocol documentation, prior audits, or Code4rena V12 reports. For Code4renaBounty , the generator also fetches Code4rena's bounty guide and bounty criteria, preserves global bounty out-of-scope rules, and can map contract-name-only scope tables to local Solidity definitions. Fetched second-level pages do not emit more links. If generated scope.md or docs.md exceeds max tokens per file , Codex summarizes it down to the limit; the generator refuses to silently truncate final Markdown. Optional PoC config: poc: allow fork: true prefer fork: true rpc env: ethereum-mainnet: "MAINNET RPC URL" base-mainnet: "BASE RPC URL" The poc block is YAML-only. It mainly affects generated PoC runtime guidance for workflows that need explicit network and fork assumptions, such as Immunefi bounty validation. | Variable | Required | Notes | |---|---|---| | ChatGPT/Codex sign-in | Yes for the default pipeline | Performed interactively once at startup when needed, then cached locally until expiry. | AI AGENT AUDIT OPENAI BACKEND | No | codex by default. Set to api to use direct OpenAI API billing. | OPENAI API KEY | Only for API fallback | Required when AI AGENT AUDIT OPENAI BACKEND=api . Not used by the default Codex path. | GEMINI API KEY | No | Supported by the agent layer, not required by the default path. | GOOGLE AI API KEY | Legacy alias | Accepted as a fallback for Gemini. | ANTHROPIC API KEY | No | Supported by the agent layer, not required by the default path. | DEEPSEEK API KEY | No | Supported by the agent layer, not required by the default path. | GITHUB TOKEN | No | Used for private GitHub repo cloning. | AI AGENT AUDIT VALIDATION SUPERVISION | No | Overrides validation job emission mode. Supported values: gui or off . | AI AGENT AUDIT DATA DIR | No | Overrides the local cache directory. Defaults to .ai-agent-audit . | AI AGENT AUDIT WORKSPACE ROOT | No | Overrides where target repos are cloned and built. Defaults to ~/Desktop/Audit . | AI AGENT AUDIT WORKER LAUNCHER | No | Codex-compatible validation worker launcher. Defaults to codex on PATH and overrides YAML launcher values. | RUST LOG | No | Standard Rust log level, defaults to info . | The shipped template is .env.example /chain-shield/ai-agent-audit/blob/develop/.env.example . Discovery provider/model defaults now live in src/config.rs /chain-shield/ai-agent-audit/blob/develop/src/config.rs . Edit DISCOVERY PROVIDER , GEMINI DISCOVERY MODEL , and DISCOVERY GEMINI THINKING LEVEL there if you want to switch discovery between OpenAI and Gemini. - Repository preparation. The tool validates the repo URL, resolves the current HEAD commit, clones the target into a local workspace under ~/Desktop/Audit/<project-id / , and builds it with Foundry, Hardhat, or a custom command. - Audit context generation. The tool reads README/configured entry files, follows relevant second-level links, copies or extracts scope, and writes generated scope/docs files under audit-docs/ . - Semantic extraction. It runs the Slither-based enrichment path to build a local semantic SQLite database with function metadata and call graph edges. - Metadata context. It generates protocol-level context used later by the audit prompts and saves a metadata Markdown artifact. - Solidity indexing. It builds inheritance information from source and then derives an interface-implementation index. - Codeblock generation. It slices the codebase into contextual per-contract codeblocks using call graph depth and token-budget settings. - AI review. Verification, deduplication, summaries, and report-writing use the configured OpenAI backend: Codex by default, or direct API when AI AGENT AUDIT OPENAI BACKEND=api . Discovery-style phases patterns, actors, invariants use the provider configured in src/config.rs /chain-shield/ai-agent-audit/blob/develop/src/config.rs . Findings are aggregated across contracts and deduplicated at the end. - Report export and local persistence. The tool writes Markdown outputs, records findings in local SQLite databases, and keeps cached repo metadata for later runs. By default it also emits a validation-three-shot job, which is the primary path for deeper validation, PoC generation, PoC verification, and final report polish. Outputs are written relative to the current working directory. The main output folder is named after the target repo, or repo/subfolder if subfolder is configured. <repo name /report/audit-report.md The main aggregated audit report. <repo name /report/<sanitized-finding-title .md One file per finding when a detailed competition-style report was generated for that finding. Filenames are sanitized and truncated. <repo name /metadata-<unique repo hash .md Saved metadata context used during analysis. <repo name /<ContractType -<Contract -size-<tokens .md Exported codeblocks for in-scope contracts. By default, the tool stores local state under .ai-agent-audit/ : .ai-agent-audit/semantic.db .ai-agent-audit/codeblock.db .ai-agent-audit/findings.db .ai-agent-audit/repo data.db Set AI AGENT AUDIT DATA DIR if you want those files elsewhere. The Rust pipeline performs discovery and initial verification, then emits a validation-three-shot config/job for deeper validation, PoC generation, PoC verification, and report creation. Codex-supervised mode is the strongest path for high-stakes work because fresh Codex workers can inspect files, create PoCs, and repair reports round by round: python3 scripts/three shot round.py prepare-scope --config validation-three-shot/config.yaml --write-prompt /tmp/three-shot-r1.md For users who want the validation phase to run immediately instead of being monitored by a GUI supervisor: cp validation-three-shot/config.yaml validation-three-shot/my-run.yaml edit benchmark, run id, paths.source root, paths.audit root, and paths.audit report python3 scripts/three shot round.py run --config validation-three-shot/my-run.yaml For a cheaper validation-only pass before PoCs and report review: python3 scripts/three shot round.py run --config validation-three-shot/my-run.yaml --skip-poc The validation runner uses the Codex CLI by default. These prompts require an agent worker with local file read/write and test execution, so a raw OPENAI API KEY alone cannot run the PoC/report validation workflow. Set AI AGENT AUDIT WORKER LAUNCHER or workers.default.launcher if your Codex-compatible binary or wrapper has a different name; the env var wins for one-off runs. The analysis system combines several sources of context: - Pattern libraries covering access control, reentrancy, accounting and invariant drift, oracle and AMM behavior, governance and timelocks, upgradeability, bridging, token standards, marketplace flows, and more. - Invariant analysis across arithmetic, balance, permission, temporal, referential, and state-machine categories. - Actor-oriented prompt context for threat modeling each reviewed contract. - Contest-aware severity handling for Code4rena , Sherlock , and Cantina , bounty-specific handling for Code4renaBounty and ImmunefiBugBounty , plus a more open-ended Client mode. The exact prompts and pattern catalogs continue to evolve, so the README intentionally describes this at the capability level instead of freezing brittle counts. - This project sends code and documentation to external AI providers. Do not use it on repositories you are not allowed to share with those providers. - The tool is designed for defensive review support. It can miss real issues and it can produce false positives. - The default Rust OpenAI path depends on a valid cached ChatGPT/Codex session. API fallback for Rust OpenAI calls requires AI AGENT AUDIT OPENAI BACKEND=api and OPENAI API KEY ; deep validation workers still require a Codex-compatible agent launcher. audit type affects severity, rubric behavior, context gathering, and validation profile selection. Code4renaBounty disables V12-specific validation stages and uses a stricter Critical/High submit/no-submit flow. ImmunefiBugBounty uses Immunefi-style scope and impact handling.- Runnable PoC generation and verification are supported through the separate validation-three-shot workflow. The main Rust audit pipeline stops after initial artifact generation and job emission. - If the target repo does not build cleanly on the local machine, analysis quality will degrade or the run may fail. - Automatic Hardhat/JavaScript build commands may install missing dependencies locally, but they do so with lifecycle scripts disabled, such as npm ci --ignore-scripts , yarn install --ignore-scripts , pnpm install --frozen-lockfile --ignore-scripts , or bun install --frozen-lockfile --ignore-scripts . Foundry dependency installation via forge install is not run automatically. Because execution is host-local rather than Docker-isolated, only audit repositories you trust, or provide an explicit build cmd . - If Slither cannot extract semantic data, the tool falls back to a reduced analysis path with less Slither-derived context. If Codex startup cannot authenticate OpenAI access, rerun the tool and complete the ChatGPT/Codex sign-in prompt. The default path does not require OPENAI API KEY . If you do not have Codex access and want to run the Rust audit path with direct API billing, set: AI AGENT AUDIT OPENAI BACKEND=api OPENAI API KEY=your openai api key If the build output shows No build system detected , the target repo likely does not expose a recognizable foundry.toml or hardhat.config. at the analyzed root. Set subfolder , monorepo folders , or builder: "Custom" plus build cmd . The Docker execution path has been removed. If git , slither , forge , node , npm , npx , yarn , pnpm , or bun is required and missing or incompatible, startup/build/static-analysis will fail with an install note for the missing command. Node-based builds require Node.js 18 or newer. Modern Foundry/solc projects require Slither 0.11.5 or newer. If the run finishes but contract coverage looks wrong, check code folders . The default is src , but many repos use contracts , src/contracts , or multiple package roots. If a rerun is clearly using stale build artifacts, set force rebuild: true or pass --force-rebuild . That is expected behavior. The tool can continue in a reduced mode, but some Slither-derived metadata will be skipped. The current codebase is organized around these modules: src/ main.rs CLI entrypoint and top-level orchestration config.rs environment/config loading and constants cli args/ clap/YAML argument parsing prepare code/ repo cloning, generated context, filtering, native builds, repo metadata build brain/ Slither enrichment and graph DB enumerator/ codeblock generation, Solidity parsing, interface indexing llm review/ prompt generation, agent setup, findings, review phases reporting/ audit reports, finding reports, exported artifacts cost/ inference cost tracking utils/ shared helpers tests/ hermetic and manual integration tests scripts/run ci tests.sh curated public CI test runner Public CI currently runs: cargo fmt --check cargo check --tests bash scripts/run ci tests.sh That hermetic test runner is defined in scripts/run ci tests.sh /chain-shield/ai-agent-audit/blob/develop/scripts/run ci tests.sh and wired through .github/workflows/ci.yml /chain-shield/ai-agent-audit/blob/develop/.github/workflows/ci.yml . For local development: cargo fmt cargo check bash scripts/run ci tests.sh Manual or live-provider diagnostics are kept behind ignored tests: cargo test -- --ignored See CONTRIBUTING.md /chain-shield/ai-agent-audit/blob/develop/CONTRIBUTING.md for contribution expectations and SECURITY.md /chain-shield/ai-agent-audit/blob/develop/SECURITY.md for private vulnerability reporting. This project is licensed under the MIT License. See LICENSE /chain-shield/ai-agent-audit/blob/develop/LICENSE .