# Show HN: Dynobox – Integration tests for agents using CLI and MCP mocks

> Source: <https://github.com/dynobox/dynobox>
> Published: 2026-11-19 19:30:58+00:00

**Key takeaway:** Dynobox lets builders run one scenario across Claude Code, Codex, OpenCode, Pi, Cursor CLI, and Antigravity CLI and assert on tool calls, commands, files, and final answers without an LLM judge, using CLI and MCP mocks for deterministic fixtures.

Open-source test runner for coding agents and skills: run real workflows, capture evidence, assert on tools, commands, files, and answers.

[Get started](https://docs.dynobox.xyz/getting-started) ·
  [How it works](https://docs.dynobox.xyz/how-it-works) ·
  [Documentation](https://docs.dynobox.xyz) ·
  [Dashboard](https://dash.dynobox.xyz) ·
  [Agent skills](https://www.skills.sh/dynobox/skills) ·
  [Examples](https://github.com/dynobox/examples)

- Assert on tool calls, commands, files, HTTP requests, transcripts, and final answers without requiring one model to judge another.
- Run the same scenario through Claude Code, OpenAI Codex, OpenCode, Pi, Cursor CLI, and Google Antigravity CLI to find behavior that varies between environments.
- Test multi-step tasks in fresh temporary work directories, repeat them to expose flaky behavior, and keep the evidence when something fails.
- Replace bare executable calls with scenario-scoped static, sequential, or
handler responses using [experimental CLI mocks](https://docs.dynobox.xyz/config-authoring/#cli-mocks) .
- Provide HTTP MCP tool fixtures and check received calls with
[MCP mocks](https://github.com/dynobox/dynobox/blob/main/docs/mcp-mocking.md) .

Node.js 22+ and at least one supported harness installed, authenticated, and
available on `PATH`: Claude Code (` claude`), Codex (` codex`), OpenCode 2.x
(`opencode`), Pi (` pi`), Cursor CLI (` cursor-agent`), or Google Antigravity CLI
1.1.14+ (`agy`).

```
npx dynobox@latest init                         # Claude Code (default)
npx dynobox@latest init --harness codex         # OpenAI Codex
npx dynobox@latest init --harness opencode      # OpenCode
npx dynobox@latest init --harness pi            # Pi
npx dynobox@latest init --harness cursor        # Cursor CLI
npx dynobox@latest init --harness antigravity   # Google Antigravity CLI
npx dynobox run
```

Run one `init` command for a harness that is installed and authenticated, then
run the generated dyno.

`dynobox init` creates a starter dyno in `dynobox/example.dyno.mjs`.
`dynobox run` discovers `*.dyno.{mjs,js,ts,mts,yaml,yml}` files below the current
directory and runs their scenarios against the configured harnesses.

Only run dynos you trust. JavaScript and TypeScript configs are imported, and setup and verification commands execute on your machine. Temporary work directories separate job files, but they are not security sandboxes; processes can access the host according to their permissions.

A dyno combines the prompt, fixture setup, harnesses, and acceptance criteria in one TypeScript, JavaScript, or YAML file. Include only harnesses installed and authenticated in the current environment:

```
# package.dyno.yaml
name: package-script
harnesses:
  - claude-code
  - codex
  - opencode
  - pi
  - cursor
  - antigravity
scenarios:
  - name: detects the test script
    setup:
      - |
        cat > package.json <<'JSON'
        {"scripts":{"test":"vitest run"}}
        JSON
    prompt: >-
      Use cat package.json and tell me whether this project has a test script.
    assertions:
      - type: command.called
        executable: cat
        command: {args: [package.json]}
      - type: tool.notCalled
        tool: edit_file
      - type: artifact.contains
        path: package.json
        text: vitest run
      - type: finalMessage.contains
        text: test
```

Run the test:

```
npx dynobox run package.dyno.yaml
```

The same shape works in TypeScript and JavaScript via
[`@dynobox/sdk`](https://www.npmjs.com/package/@dynobox/sdk)
(`defineDyno`, `tool`, `command`, ...). See
[Config Authoring](https://docs.dynobox.xyz/config-authoring).

Dynobox launches the selected harnesses in fresh temporary work directories, records the observable behavior, and evaluates each assertion against captured evidence. When a check fails, the output shows what was expected and what was observed.

| Evidence | Example checks | What it answers | 
|---|---|---|
| Tools | `tool.called` ,`tool.notCalled` | Did the agent use or avoid a tool? | 
| Commands | `command.called` ,`command.notCalled` | Did it execute the expected shell command? | 
| Files | `artifact.exists` ,`artifact.contains` ,`artifact.unchanged` | Did it create, change, or preserve the right files? | 
| Skills | `skill.referenced` | Did it reference the required skill instructions? | 
| Network* | `http.called` ,`http.notCalled` | Did a proxy-aware child process call the expected endpoint? | 
| Response | `transcript.contains` ,`finalMessage.contains` | Did the interaction contain required information? | 
| Logic | `sequence.inOrder` ,`anyOf` | Did the observed behavior follow an accepted path or order? | 
| Verification | `verify.command` | Does the completed work pass a custom executable check? | 

* HTTP capture observes child-process traffic that honors Dynobox's proxy and
CA settings. Harness-native web tools and clients with independent networking
may not be captured. [Learn how HTTP capture
works](https://docs.dynobox.xyz/how-it-works#4-capture-observable-evidence).

See the [assertion reference](https://docs.dynobox.xyz/config-authoring/#assertions)
for every matcher and authoring option.

Override the configured harnesses from the command line:

```
npx dynobox run --harness claude-code
npx dynobox run --harness codex
npx dynobox run --harness opencode
npx dynobox run --harness pi
npx dynobox run --harness cursor
npx dynobox run --harness antigravity
npx dynobox run --harness claude-code,codex,opencode,pi,cursor,antigravity
```

Repeat every scenario and harness pair to measure pass rates:

```
npx dynobox run \
  --harness claude-code,codex,opencode,pi,cursor,antigravity \
  --iterations 5
```

For each scenario, Dynobox runs the selected harness configurations in parallel. Repeated iterations for one harness configuration remain sequential, and the next scenario starts after every harness for the current scenario finishes.

Repeated runs render compact pass-rate rows such as `.FF..` while retaining the
failed iteration evidence for diagnosis.

Dynobox runs locally and works without an account. Use terminal output for
development, `--reporter json` for automation, or authenticate with
`dynobox login` and add `--save-run` to publish a compact run summary to the
[Dynobox dashboard](https://dash.dynobox.xyz). Set `DYNOBOX_UPLOAD_URL` to post
the same payload to your own endpoint without sending Dynobox credentials.

Saved-run data is length-capped but not redacted. It includes available Git
identity and revision metadata. All jobs can include authored assertion data and
matched evidence such as requested endpoint URLs, tool commands, and
verification output. Failed jobs can additionally include command or harness
diagnostics. Do not use `--save-run` when those values should not be shared.

Dynobox is closer to unit testing than model-graded evals. It evaluates captured behavior—tool calls, shell commands, files, and HTTP requests—using explicit, repeatable assertions.

Compared with one-off scripts, Dynobox provides normalized assertions, fresh temporary work directories, captured failure evidence, and iteration pass rates for measuring flaky behavior.

| Package | Description | 
|---|---|
| [`dynobox`](https://www.npmjs.com/package/dynobox) | CLI for discovering and running dynos | 
| [`@dynobox/sdk`](https://www.npmjs.com/package/@dynobox/sdk) | Type-safe helpers for authoring dynos | 

Dynobox is early-access software. The CLI, SDK, and report formats may evolve before 1.0.

See [CONTRIBUTING.md](https://github.com/dynobox/dynobox/blob/main/CONTRIBUTING.md) to run the monorepo locally and propose
a change.

Dynobox is [Apache-2.0 licensed](https://github.com/dynobox/dynobox/blob/main/LICENSE).
