cd /news/ai-agents/show-hn-dynobox-integration-tests-fo… · home › topics › ai-agents › article
[ARTICLE · art-172802] src=github.com ↗ pub= topic=ai-agents verified=true sentiment=↑ positive

Show HN: Dynobox – Integration tests for agents using CLI and MCP mocks

Dynobox, an open-source integration test runner for coding agents, launched with support for running the same scenario across Claude Code, OpenAI Codex, OpenCode 2.x, Pi, Cursor CLI, and Google Antigravity CLI 1.1.14+. Dynobox asserts on tool calls, commands, files, HTTP requests, transcripts, and final answers without requiring one model to judge another, and it replaces bare executable calls with scenario-scoped static, sequential, or handler responses through experimental CLI mocks plus HTTP MCP tool fixtures checked via MCP mocks. The tool requires Node.js 22+ and at least one installed, authenticated harness on PATH, and its authors warn that temporary work directories separate job files but are not security sandboxes.

key takeaway

Dynobox lets builders run one scenario across Claude Code, Codex, OpenCode, Pi, Cursor CLI, and Antigravity CLI and assert on tool calls, commands, files, and final answers without an LLM judge, using CLI and MCP mocks for deterministic fixtures.

read5 min views1 publishedNov 19, 2026
Show HN: Dynobox – Integration tests for agents using CLI and MCP mocks
Image: Michielbdejong (auto-discovered)

Open-source test runner for coding agents and skills: run real workflows, capture evidence, assert on tools, commands, files, and answers.

Get started · How it works · Documentation · Dashboard · Agent skills · Examples

  • Assert on tool calls, commands, files, HTTP requests, transcripts, and final answers without requiring one model to judge another.
  • Run the same scenario through Claude Code, OpenAI Codex, OpenCode, Pi, Cursor CLI, and Google Antigravity CLI to find behavior that varies between environments.
  • Test multi-step tasks in fresh temporary work directories, repeat them to expose flaky behavior, and keep the evidence when something fails.
  • Replace bare executable calls with scenario-scoped static, sequential, or handler responses using experimental CLI mocks .
  • Provide HTTP MCP tool fixtures and check received calls with MCP mocks .

Node.js 22+ and at least one supported harness installed, authenticated, and available on PATH: Claude Code ( claude), Codex ( codex), OpenCode 2.x (opencode), Pi ( pi), Cursor CLI ( cursor-agent), or Google Antigravity CLI 1.1.14+ (agy).

npx dynobox@latest init                         # Claude Code (default)
npx dynobox@latest init --harness codex         # OpenAI Codex
npx dynobox@latest init --harness opencode      # OpenCode
npx dynobox@latest init --harness pi            # Pi
npx dynobox@latest init --harness cursor        # Cursor CLI
npx dynobox@latest init --harness antigravity   # Google Antigravity CLI
npx dynobox run

Run one init command for a harness that is installed and authenticated, then run the generated dyno.

dynobox init creates a starter dyno in dynobox/example.dyno.mjs. dynobox run discovers *.dyno.{mjs,js,ts,mts,yaml,yml} files below the current directory and runs their scenarios against the configured harnesses.

Only run dynos you trust. JavaScript and TypeScript configs are imported, and setup and verification commands execute on your machine. Temporary work directories separate job files, but they are not security sandboxes; processes can access the host according to their permissions.

A dyno combines the prompt, fixture setup, harnesses, and acceptance criteria in one TypeScript, JavaScript, or YAML file. Include only harnesses installed and authenticated in the current environment:

name: package-script
harnesses:
  - claude-code
  - codex
  - opencode
  - pi
  - cursor
  - antigravity
scenarios:
  - name: detects the test script
    setup:
      - |
        cat > package.json <<'JSON'
        {"scripts":{"test":"vitest run"}}
        JSON
    prompt: >-
      Use cat package.json and tell me whether this project has a test script.
    assertions:
      - type: command.called
        executable: cat
        command: {args: [package.json]}
      - type: tool.notCalled
        tool: edit_file
      - type: artifact.contains
        path: package.json
        text: vitest run
      - type: finalMessage.contains
        text: test

Run the test:

npx dynobox run package.dyno.yaml

The same shape works in TypeScript and JavaScript via @dynobox/sdk (defineDyno, tool, command, ...). See Config Authoring.

Dynobox launches the selected harnesses in fresh temporary work directories, records the observable behavior, and evaluates each assertion against captured evidence. When a check fails, the output shows what was expected and what was observed.

Evidence Example checks What it answers
Tools tool.called ,tool.notCalled Did the agent use or avoid a tool?
Commands command.called ,command.notCalled Did it execute the expected shell command?
Files artifact.exists ,artifact.contains ,artifact.unchanged Did it create, change, or preserve the right files?
Skills skill.referenced Did it reference the required skill instructions?
Network* http.called ,http.notCalled Did a proxy-aware child process call the expected endpoint?
Response transcript.contains ,finalMessage.contains Did the interaction contain required information?
Logic sequence.inOrder ,anyOf Did the observed behavior follow an accepted path or order?
Verification verify.command Does the completed work pass a custom executable check?
  • HTTP capture observes child-process traffic that honors Dynobox's proxy and CA settings. Harness-native web tools and clients with independent networking may not be captured. Learn how HTTP capture works.

See the assertion reference for every matcher and authoring option.

Override the configured harnesses from the command line:

npx dynobox run --harness claude-code
npx dynobox run --harness codex
npx dynobox run --harness opencode
npx dynobox run --harness pi
npx dynobox run --harness cursor
npx dynobox run --harness antigravity
npx dynobox run --harness claude-code,codex,opencode,pi,cursor,antigravity

Repeat every scenario and harness pair to measure pass rates:

npx dynobox run \
  --harness claude-code,codex,opencode,pi,cursor,antigravity \
  --iterations 5

For each scenario, Dynobox runs the selected harness configurations in parallel. Repeated iterations for one harness configuration remain sequential, and the next scenario starts after every harness for the current scenario finishes.

Repeated runs render compact pass-rate rows such as .FF.. while retaining the failed iteration evidence for diagnosis.

Dynobox runs locally and works without an account. Use terminal output for development, --reporter json for automation, or authenticate with dynobox login and add --save-run to publish a compact run summary to the Dynobox dashboard. Set DYNOBOX_UPLOAD_URL to post the same payload to your own endpoint without sending Dynobox credentials.

Saved-run data is length-capped but not redacted. It includes available Git identity and revision metadata. All jobs can include authored assertion data and matched evidence such as requested endpoint URLs, tool commands, and verification output. Failed jobs can additionally include command or harness diagnostics. Do not use --save-run when those values should not be shared.

Dynobox is closer to unit testing than model-graded evals. It evaluates captured behavior—tool calls, shell commands, files, and HTTP requests—using explicit, repeatable assertions.

Compared with one-off scripts, Dynobox provides normalized assertions, fresh temporary work directories, captured failure evidence, and iteration pass rates for measuring flaky behavior.

Package Description
dynobox CLI for discovering and running dynos
@dynobox/sdk Type-safe helpers for authoring dynos

Dynobox is early-access software. The CLI, SDK, and report formats may evolve before 1.0.

See CONTRIBUTING.md to run the monorepo locally and propose a change.

Dynobox is Apache-2.0 licensed.

── more in #ai-agents 4 stories · sorted by recency
── more on @dynobox 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/show-hn-dynobox-inte…] indexed:0 read:5min 2026-11-19 · —