{"slug": "simplicity-always-wins-sota-on-swe-pro-tb2-verif-on-21-models-with-simple-agent", "title": "Simplicity always wins:SOTA on swe-pro,tb2,-verif on 21 models with simple-agent", "summary": "Strands Labs released Simple Strands Agent (SSA), a minimal autonomous coding harness that achieves state-of-the-art results on SWE-Bench Verified, SWE-Bench Pro, and Terminal Bench 2 across 21 models. The open-source tool pairs frontier LLMs with bash and file-editing tools in isolated Docker environments, enabling rapid experimentation and model-agnostic benchmarking.", "body_md": "A repository for Strands-based agents and harnesses for agentic benchmarks. It is a\n[uv workspace](https://docs.astral.sh/uv/concepts/projects/workspaces/): the repository\nroot coordinates one or more member packages. Setup, configuration, and usage live in each agent's README.\n\n**A lean autonomous-coding agent achieving state-of-the-art performance across software engineering benchmarks.**\n\nFor a summary of the work, see the [post on Amazon-Science.](https://www.amazon.science/blog/bridging-intent-and-execution-in-agentic-systems)\n\nSimple Strands Agent (SSA) is a minimal, hackable harness for autonomous software engineering. It pairs frontier LLMs (Claude, GPT, Gemini, and open-weight models via Bedrock/LiteLLM/vLLM) with `bash`\n\nand file-editing tools inside isolated Docker environments to analyze codebases, diagnose bugs, write patches, and verify solutions.\n\nThe harness is built for **rapid experimentation** — swap models, tune prompts, adjust tool behavior, and benchmark all in a single config change. Despite its simplicity, SSA delivers SOTA-level results on widely-used coding benchmarks including **SWE-Bench Verified**, **SWE-Bench Pro**, and **Terminal Bench 2**.\n\n**Model-agnostic**— first-class adapters for Anthropic, OpenAI, Google, xAI, Bedrock, and any OpenAI-compatible endpoint (vLLM, LiteLLM, Together, Vertex, Z.AI).**Composable tools**—`bash`\n\n,`str_replace_editor`\n\n,`think`\n\n, and`submit`\n\nprimitives with per-tool output clipping and timeout controls.**Isolated environments**— Docker-backed sandboxes with streaming exec, automatic image resolution, and ECR support.** Hydra-powered configs**— every knob is overridable from the command line; experiments are reproducible from a single YAML.** Built-in benchmarking**— turnkey scripts for SWE-Bench Verified, SWE-Bench Pro, and Terminal Bench 2, including S3 result upload.\n\n**Python** 3.12+**Docker** for containerized task environments**AWS credentials** if using Amazon Bedrock for model access and/or Amazon ECR for docker imagespackage manager (optional, but recommended)[uv](https://github.com/astral-sh/uv)\n\n```\ngit clone https://github.com/strands-labs/benchmark-harnesses.git\ncd benchmark-harnesses\n\n# Recommended: sync the workspace (creates .venv with this package + its deps)\nuv sync\nsource .venv/bin/activate\n\n# Or install just this package with pip\npip install -e .\nuv run python -m ssa.run \\\n    --config-name=default.yaml \\\n    dataset.name=sbv \\\n    dataset.identifier=django__django-15987 \\\n    env.env_type=docker \\\n    env.docker.workdir=\"/testbed\"\n```\n\nWe provide simple scripts for running instances from [SWE-Bench Verified](/strands-labs/benchmark-harnesses/blob/main/simple-strands-agent/scripts/swe_verified/run.sh), [SWE-Bench Pro](/strands-labs/benchmark-harnesses/blob/main/simple-strands-agent/scripts/swe_pro/run.sh), and [Terminal-Bench-2](/strands-labs/benchmark-harnesses/blob/main/simple-strands-agent/scripts/tb2/run.sh) (and see [this](/strands-labs/benchmark-harnesses/blob/main/simple-strands-agent/scripts/tb2/run_harbor.sh) for running Terminal-Bench-2 with SSA's harbor plugin).\n\nSSA uses [Hydra](https://hydra.cc/) for configuration. All configs live in [ src/ssa/configs/](/strands-labs/benchmark-harnesses/blob/main/simple-strands-agent/src/ssa/configs), and any parameter can be overridden from the command line. Start from\n\n[for the full schema, then mix in a model-specific config as needed.](/strands-labs/benchmark-harnesses/blob/main/simple-strands-agent/src/ssa/configs/default.yaml)\n\n`src/ssa/configs/default.yaml`\n\nSSA's design and execution loop is summarized in the following figure:\n\n**SWE-Bench Verified:**\n\n**SWE-Bench Pro:**\n\n**Terminal Bench 2:**\n\nThe code is structured as follows:\n\n```\nsrc/ssa/\n├── agent.py / agent_runner.py     # Core agent loop\n├── models/                        # Model adapters (Anthropic, OpenAI, Bedrock, ...)\n├── tools/                         # bash, str_replace_editor, think, submit\n├── environments/                  # Docker-backed sandbox\n├── conversation_manager/          # Context management & truncation\n├── hooks/ • callbacks/ • metrics/ # Observability and instrumentation\n├── prompts/ • configs/            # System prompts and Hydra configs\n└── run.py                         # Entry point\nscripts/\n├── swe_verified/ • swe_pro/ • tb2/\n```\n\nPlease consider citing as follows, if you find SSA useful!\n\n```\n@misc{2026simplestrandsagent,\n      title={Dissecting model behavior through agent trajectories},\n      author={Gaurav Gupta and Vatshank Chaturvedi and Jun Huan and Anoop Deoras},\n      year={2026},\n      eprint={2606.17454},\n      archivePrefix={arXiv},\n      url={https://arxiv.org/abs/2606.17454},\n}\n```\n\nAgents in this repository are given access to shell tools. In practice, this means the model can run commands in the environment where the agent is started.\n\nThis is useful for experiments and benchmarking, but it also means you should treat the agent like you would treat any program with shell access: it may read files, modify files, delete data, install packages, or accidentally expose information from the environment.\n\nFor normal use, we recommend running agents in an isolated environment rather than directly on your machine. Our experiments and benchmarks are run inside Docker containers. You should avoid running agents in an environment that contains secrets, credentials, personal files, production data, or anything you would not want the model to access.\n\nA good default setup is:\n\n- run the agent inside Docker or another sandboxed environment\n- mount only the files/directories the agent actually needs\n- avoid exposing cloud credentials, SSH keys, API keys, or other secrets\n- do not run it with unnecessary privileges\n- inspect outputs before trusting or reusing them\n\nAgents will usually behave as instructed, but shell access is powerful. Use the same caution you would use when running code from an automated system.", "url": "https://wpnews.pro/news/simplicity-always-wins-sota-on-swe-pro-tb2-verif-on-21-models-with-simple-agent", "canonical_source": "https://github.com/strands-labs/benchmark-harnesses", "published_at": "2026-06-19 17:58:14+00:00", "updated_at": "2026-06-19 18:07:43.962635+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-tools", "large-language-models", "developer-tools"], "entities": ["Strands Labs", "Simple Strands Agent", "SSA", "SWE-Bench Verified", "SWE-Bench Pro", "Terminal Bench 2", "Amazon Bedrock", "Hydra"], "alternates": {"html": "https://wpnews.pro/news/simplicity-always-wins-sota-on-swe-pro-tb2-verif-on-21-models-with-simple-agent", "markdown": "https://wpnews.pro/news/simplicity-always-wins-sota-on-swe-pro-tb2-verif-on-21-models-with-simple-agent.md", "text": "https://wpnews.pro/news/simplicity-always-wins-sota-on-swe-pro-tb2-verif-on-21-models-with-simple-agent.txt", "jsonld": "https://wpnews.pro/news/simplicity-always-wins-sota-on-swe-pro-tb2-verif-on-21-models-with-simple-agent.jsonld"}}