{"slug": "run-terminal-bench-and-other-harbor-evals-on-vercel-sandbox", "title": "Run Terminal-Bench and other Harbor evals on Vercel Sandbox", "summary": "Vercel now supports running Harbor evaluations, including Terminal-Bench, SWE-bench, tau3-bench and OSWorld, on Vercel Sandbox, with each trial executing in its own isolated Firecracker microVM when users pass --env vercel to harbor run. The integration requires Harbor 0.22.0 or later, enforces task network policy at the sandbox firewall outside the VM, and supports optional credential injection that attaches secrets to matching outbound requests so they never enter the sandbox. Paired with Vercel's AI Gateway, a single AI_GATEWAY_API_KEY reaches hundreds of models across providers, letting users swap benchmarks with a different --model flag, such as vercel_ai_gateway/openai/gpt-5.6-luna.", "body_md": "You can now run Harbor evals on Vercel Sandbox.\n\n[Harbor](https://harborframework.com) is the open-source harness behind [Terminal-Bench](https://www.tbench.ai/), whose registry includes many other benchmarks such as SWE-bench, tau3-bench and OSWorld. Pass `--env vercel` to `harbor run` and each trial executes in its own isolated Firecracker microVM, so you can parallelize far beyond what your local machine is capable of. \n\nA task's network policy is enforced at the sandbox firewall, outside the VM. Optional credential injection attaches secrets to matching outbound requests at that firewall, so they never enter the sandbox.\n\nPaired with [AI Gateway](https://vercel.com/docs/ai-gateway), one `AI_GATEWAY_API_KEY` reaches hundreds of models from multiple providers, and benchmarking another model is the same command with a different `--model`:\n\nSwap `--model` to `vercel_ai_gateway/openai/gpt-5.6-luna` to run the same benchmark against an OpenAI model. \n\nRequires Harbor `0.22.0` or later. Follow the [step-by-step guide](https://vercel.com/kb/guide/run-terminal-bench-harbor-benchmarks-vercel-sandbox) for setup, configuration, and troubleshooting. Learn more in the [Sandbox documentation](https://vercel.com/docs/sandbox/ecosystem/harbor).", "url": "https://wpnews.pro/news/run-terminal-bench-and-other-harbor-evals-on-vercel-sandbox", "canonical_source": "https://vercel.com/changelog/run-terminal-bench-and-other-harbor-evals-on-vercel-sandbox", "published_at": "2026-09-17 18:22:59.471475+00:00", "updated_at": "2026-09-17 18:23:00.826348+00:00", "lang": "en", "topics": ["ai-research", "ai-infrastructure", "ai-agents", "developer-tools", "large-language-models"], "entities": ["Vercel", "Vercel Sandbox", "Harbor", "Terminal-Bench", "SWE-bench", "tau3-bench", "OSWorld", "Vercel AI Gateway"], "alternates": {"html": "https://wpnews.pro/news/run-terminal-bench-and-other-harbor-evals-on-vercel-sandbox", "markdown": "https://wpnews.pro/news/run-terminal-bench-and-other-harbor-evals-on-vercel-sandbox.md", "text": "https://wpnews.pro/news/run-terminal-bench-and-other-harbor-evals-on-vercel-sandbox.txt", "jsonld": "https://wpnews.pro/news/run-terminal-bench-and-other-harbor-evals-on-vercel-sandbox.jsonld"}}