Show HN: Dynobox – Integration tests for agents using CLI and MCP mocks Dynobox, an open-source integration test runner for coding agents, launched with support for running the same scenario across Claude Code, OpenAI Codex, OpenCode 2.x, Pi, Cursor CLI, and Google Antigravity CLI 1.1.14+. Dynobox asserts on tool calls, commands, files, HTTP requests, transcripts, and final answers without requiring one model to judge another, and it replaces bare executable calls with scenario-scoped static, sequential, or handler responses through experimental CLI mocks plus HTTP MCP tool fixtures checked via MCP mocks. The tool requires Node.js 22+ and at least one installed, authenticated harness on PATH, and its authors warn that temporary work directories separate job files but are not security sandboxes. Open-source test runner for coding agents and skills: run real workflows, capture evidence, assert on tools, commands, files, and answers. Get started https://docs.dynobox.xyz/getting-started · How it works https://docs.dynobox.xyz/how-it-works · Documentation https://docs.dynobox.xyz · Dashboard https://dash.dynobox.xyz · Agent skills https://www.skills.sh/dynobox/skills · Examples https://github.com/dynobox/examples - Assert on tool calls, commands, files, HTTP requests, transcripts, and final answers without requiring one model to judge another. - Run the same scenario through Claude Code, OpenAI Codex, OpenCode, Pi, Cursor CLI, and Google Antigravity CLI to find behavior that varies between environments. - Test multi-step tasks in fresh temporary work directories, repeat them to expose flaky behavior, and keep the evidence when something fails. - Replace bare executable calls with scenario-scoped static, sequential, or handler responses using experimental CLI mocks https://docs.dynobox.xyz/config-authoring/ cli-mocks . - Provide HTTP MCP tool fixtures and check received calls with MCP mocks https://github.com/dynobox/dynobox/blob/main/docs/mcp-mocking.md . Node.js 22+ and at least one supported harness installed, authenticated, and available on PATH : Claude Code claude , Codex codex , OpenCode 2.x opencode , Pi pi , Cursor CLI cursor-agent , or Google Antigravity CLI 1.1.14+ agy . npx dynobox@latest init Claude Code default npx dynobox@latest init --harness codex OpenAI Codex npx dynobox@latest init --harness opencode OpenCode npx dynobox@latest init --harness pi Pi npx dynobox@latest init --harness cursor Cursor CLI npx dynobox@latest init --harness antigravity Google Antigravity CLI npx dynobox run Run one init command for a harness that is installed and authenticated, then run the generated dyno. dynobox init creates a starter dyno in dynobox/example.dyno.mjs . dynobox run discovers .dyno.{mjs,js,ts,mts,yaml,yml} files below the current directory and runs their scenarios against the configured harnesses. Only run dynos you trust. JavaScript and TypeScript configs are imported, and setup and verification commands execute on your machine. Temporary work directories separate job files, but they are not security sandboxes; processes can access the host according to their permissions. A dyno combines the prompt, fixture setup, harnesses, and acceptance criteria in one TypeScript, JavaScript, or YAML file. Include only harnesses installed and authenticated in the current environment: package.dyno.yaml name: package-script harnesses: - claude-code - codex - opencode - pi - cursor - antigravity scenarios: - name: detects the test script setup: - | cat package.json <<'JSON' {"scripts":{"test":"vitest run"}} JSON prompt: - Use cat package.json and tell me whether this project has a test script. assertions: - type: command.called executable: cat command: {args: package.json } - type: tool.notCalled tool: edit file - type: artifact.contains path: package.json text: vitest run - type: finalMessage.contains text: test Run the test: npx dynobox run package.dyno.yaml The same shape works in TypeScript and JavaScript via @dynobox/sdk https://www.npmjs.com/package/@dynobox/sdk defineDyno , tool , command , ... . See Config Authoring https://docs.dynobox.xyz/config-authoring . Dynobox launches the selected harnesses in fresh temporary work directories, records the observable behavior, and evaluates each assertion against captured evidence. When a check fails, the output shows what was expected and what was observed. | Evidence | Example checks | What it answers | |---|---|---| | Tools | tool.called , tool.notCalled | Did the agent use or avoid a tool? | | Commands | command.called , command.notCalled | Did it execute the expected shell command? | | Files | artifact.exists , artifact.contains , artifact.unchanged | Did it create, change, or preserve the right files? | | Skills | skill.referenced | Did it reference the required skill instructions? | | Network | http.called , http.notCalled | Did a proxy-aware child process call the expected endpoint? | | Response | transcript.contains , finalMessage.contains | Did the interaction contain required information? | | Logic | sequence.inOrder , anyOf | Did the observed behavior follow an accepted path or order? | | Verification | verify.command | Does the completed work pass a custom executable check? | HTTP capture observes child-process traffic that honors Dynobox's proxy and CA settings. Harness-native web tools and clients with independent networking may not be captured. Learn how HTTP capture works https://docs.dynobox.xyz/how-it-works 4-capture-observable-evidence . See the assertion reference https://docs.dynobox.xyz/config-authoring/ assertions for every matcher and authoring option. Override the configured harnesses from the command line: npx dynobox run --harness claude-code npx dynobox run --harness codex npx dynobox run --harness opencode npx dynobox run --harness pi npx dynobox run --harness cursor npx dynobox run --harness antigravity npx dynobox run --harness claude-code,codex,opencode,pi,cursor,antigravity Repeat every scenario and harness pair to measure pass rates: npx dynobox run \ --harness claude-code,codex,opencode,pi,cursor,antigravity \ --iterations 5 For each scenario, Dynobox runs the selected harness configurations in parallel. Repeated iterations for one harness configuration remain sequential, and the next scenario starts after every harness for the current scenario finishes. Repeated runs render compact pass-rate rows such as .FF.. while retaining the failed iteration evidence for diagnosis. Dynobox runs locally and works without an account. Use terminal output for development, --reporter json for automation, or authenticate with dynobox login and add --save-run to publish a compact run summary to the Dynobox dashboard https://dash.dynobox.xyz . Set DYNOBOX UPLOAD URL to post the same payload to your own endpoint without sending Dynobox credentials. Saved-run data is length-capped but not redacted. It includes available Git identity and revision metadata. All jobs can include authored assertion data and matched evidence such as requested endpoint URLs, tool commands, and verification output. Failed jobs can additionally include command or harness diagnostics. Do not use --save-run when those values should not be shared. Dynobox is closer to unit testing than model-graded evals. It evaluates captured behavior—tool calls, shell commands, files, and HTTP requests—using explicit, repeatable assertions. Compared with one-off scripts, Dynobox provides normalized assertions, fresh temporary work directories, captured failure evidence, and iteration pass rates for measuring flaky behavior. | Package | Description | |---|---| | dynobox https://www.npmjs.com/package/dynobox | CLI for discovering and running dynos | | @dynobox/sdk https://www.npmjs.com/package/@dynobox/sdk | Type-safe helpers for authoring dynos | Dynobox is early-access software. The CLI, SDK, and report formats may evolve before 1.0. See CONTRIBUTING.md https://github.com/dynobox/dynobox/blob/main/CONTRIBUTING.md to run the monorepo locally and propose a change. Dynobox is Apache-2.0 licensed https://github.com/dynobox/dynobox/blob/main/LICENSE .