{"slug": "the-agentic-scientist-sakanaai", "title": "The Agentic Scientist – SakanaAI", "summary": "SakanaAI released Agentic Scientist, a pipeline that turns a task.md description into experiments, ablations, a sealed final evaluation, post-hoc analyses, and a written paper using coding agents. The tool drives claude, codex, cursor, opencode, gemini, and pi through Headless CLI, runs each agent phase inside Docker by default with full tool permissions (Headless --allow yolo), and requires Python 3.11 or higher with uv; GPU task presets are discovered via nvidia-smi. SakanaAI warns that the agents can run arbitrary commands and inherit supplied API keys, so runs should stay inside the Docker image and only trusted tasks and sources should be used.", "body_md": "*Turn a task description into experiments, ablations, and a paper, driven by coding agents.*\n\nAgentic Scientist treats coding agents as AI scientists. Starting from a\n`task.md`, the agents propose ideas, run experiments, evaluate outcomes against\nthe task rubric, and iterate toward stronger programs. The pipeline then runs\nbaselines and ablations, a sealed final evaluation, post-hoc analyses, and\nwrites the paper, with every stage kept under one run directory.\n\nThe recommended setup runs every agent phase inside Docker, which is the\ndefault for `agsci run`. Install [Docker](https://docs.docker.com/get-docker/),\nthen build the agent image:\n\n```\ndocker build -t agentic-scientist/agent-cli:latest -f src/docker/agent-cli/Dockerfile .\n```\n\nIf the build cannot reach package mirrors, add `--network=host` to the\n`docker build` command.\n\n**Security note** — Agentic Scientist launches coding agents with full tool\npermissions (Headless `--allow yolo`). They can run arbitrary commands and\ninherit the API keys you give them, and the task files and papers they read\nsteer what they do. Keep runs inside the Docker image rather than on your\nworkstation, only run tasks and sources you trust, and review run directories\nbefore sharing them: agent logs can echo document contents and paths.\n\n- \n**Python ≥ 3.11 with [uv](https://docs.astral.sh/uv/)** .`uv sync` creates`.venv` and installs the`agsci` command:\n\n```\ncurl -LsSf https://astral.sh/uv/install.sh | sh\nuv sync\nsource .venv/bin/activate\n```\n\n- \n**API keys** in a`.env` file at the repo root (`ANTHROPIC_API_KEY` ,`OPENAI_API_KEY` ,`GEMINI_API_KEY` , ... whichever your agent\nneeds). The pipeline mounts it into the agent containers. Never commit it.\n- \n**Agent CLIs** are already inside the image. For native (non-containerized)\nruns, install the CLI of the agent you use plus Git and Node.js; versions and\ncommands are in[docs/install.md](https://github.com/SakanaAI/AI-Scientist-v3/blob/main/docs/install.md) .\n- \n**GPUs** are required only by the GPU task presets. The pipeline discovers\nthem through`nvidia-smi` and refuses to start compute phases unless the\nvisible GPUs match the preset.\n- \n**Autoillustrate** (optional) is built into the image for concept figures.\nTo use it directly on the host, follow the single-command install in the[Autoillustrate README](https://github.com/SakanaAI/Autoillustrate#readme) .\n\nRun `agsci doctor` to check the host, or `agsci doctor examples/<task> preset=<task>`\nto check everything one specific run needs (CLI, credentials, image, GPUs, task\nscaffold).\n\nPoint `agsci run` at a task directory. This runs the default six phases\n(task drafting → investigation → ablations → final evaluation → post-hoc analysis\n→ paper) on the CPU-only circle-packing example:\n\n```\nagsci run examples/circpack\n```\n\nAdd `--dry-run` to validate the configuration and preview the exact commands\nwithout running anything. Every option is a Hydra `key=value` override; `agsci run --help` lists them all.\n\n**Note** — Agentic Scientist drives `claude`, `codex`, `cursor`, `opencode`,\n`gemini`, and `pi` through\n[Headless CLI](https://github.com/RobertTLange/headless-cli), so the same\nexperiment can be repeated across agents and models: pick one with\n`agent.name=<agent> agent.model=<model>`.\n\nTask drafting only writes a scaffold when the task directory has none; every\nshipped example already includes its evaluator, so the drafting phase only\nsnapshots it and the investigation phase builds and validates the task runtime.\nDrafting a task from a bare research brief is experimental (see\n[example 8](#8-draft-a-task-from-a-research-brief-experimental)).\n\nEach run writes one predictable envelope under `results/`:\n\n```\nresults/<task>/<agent>/<run_id>/\n├── task_drafting/ # drafting logs and validation audit\n├── task/          # immutable run-local task snapshot\n├── investigation/  # ideas, attempts, winner, and optimizer memory\n├── ablations/     # baselines and ablations\n├── final_evaluation/         # sealed final evaluation\n├── posthoc/       # post-hoc analyses\n└── paper/         # manuscript, PDF, manifest, and optional AI reviews\n```\n\nInvestigation plus the task snapshot is enough to write a paper; the other\nphases add evidence when available and are reported as degraded when they fail.\nBrowse any run in the browser with `agsci inspect start`\n(`http://127.0.0.1:8765`).\n\nThe default configuration is the one to start with: the `claude` agent, every\nphase in Docker, and the CPU-only `examples/circpack` task. It needs a Claude\nlogin, no GPUs, and exercises every phase through to a compiled paper.\n\n1. \n**Build the image and the Python environment** as described in[Install](#install) .\n2. \n**Create `.env` at the repo root.** Only the selected agent's keys are\nforwarded into its containers:\n\n```\nCLAUDE_CODE_OAUTH_TOKEN=...   # claude, billed to your Claude subscription (ANTHROPIC_API_KEY for pay-per-token)\nOPENAI_API_KEY=...            # recommended: concept figures in the paper and the `review` phase\n```\n\n Other agents: `codex` reads`OPENAI_API_KEY` or`CODEX_API_KEY` ,`gemini` reads`GEMINI_API_KEY` ,`cursor` reads`CURSOR_API_KEY` , and Bedrock-backed\nClaude reads the`AWS_*` variables. Concept figures use`OPENAI_API_KEY` ,\nfall back to`GEMINI_API_KEY` , and otherwise ship a plain export.\n3. \n**Export host variables only if you need them.** Everything else (host\nnetworking, the image, the`.env` mount) is already the`agsci run` default:\n\n```\nexport AGENTIC_TEMP_ROOT=/path/to/scratch       # existing writable dir on a large disk; default: $TMPDIR, /mnt/localssd, or /tmp\nexport AGENT_CONTAINER_GPUS=device=<UUID,...>   # GPU presets only: which GPUs to use; default: the first `container.gpu_count` (x investigation workers) from nvidia-smi\n```\n\n4. \n**Check, preview, then run.** Shrinking the ablation and post-hoc budgets\nmakes the first end-to-end run finish faster without skipping a phase:\n\n```\nagsci doctor examples/circpack          # host, image, credentials, task scaffold\nagsci run examples/circpack --dry-run   # print every phase command, run nothing\nagsci run examples/circpack \\\n  investigation.execution.max_idea_cycles=2 \\\n  ablations.execution.num_replications=1 \\\n  ablations.execution.max_evals_per_variant=2 \\\n  posthoc.execution.max_selected_analyses=1\n```\n\n The paper lands in `results/circpack/claude/<run_id>/paper/paper.pdf` ;`agsci inspect start` browses the run. Drop the overrides for the default\nbudgets (3 idea cycles, 5 ablation replications, 3 post-hoc analyses), then\nmove on to the GPU task preset and a pinned model, for example`agsci run examples/grpo preset=grpo agent.model=claude-sonnet-5` .\n\nTask presets under `configs/preset/` hold the campaign budgets, timeouts,\npaper format, and required GPUs. `configs/preset/paper.yaml` is the shared\ncampaign preset and `configs/preset/grpo.yaml` composes it for the GRPO task\n(two GPUs, harness-side GPU evaluation):\n\n```\nagsci run examples/grpo preset=grpo\n```\n\nExplicit overrides win over the preset:\n\n```\nagsci run examples/grpo preset=grpo \\\n  agent.name=opencode \\\n  agent.model=amazon-bedrock/global.anthropic.claude-opus-5-5 \\\n  investigation.execution.max_idea_cycles=20\n```\n\nEvery phase resumes against the run directory. The pipeline recovers the original idea-cycle budget; pass the original agent and model again:\n\n```\n# Continue after investigation with ablations, post-hoc analysis, and the paper:\nagsci run examples/circpack execution.phases=ablations,posthoc,paper \\\n  paths.run_dir=results/circpack/claude/20260331_094917\n\n# Rewrite only the paper:\nagsci run examples/circpack execution.phases=paper \\\n  paths.run_dir=results/circpack/claude/20260331_094917\n```\n\nAdding the `review` phase sends the complete manuscript to the OpenAI-backed\nreviewer. Review failures are recorded but never discard the paper:\n\n```\nagsci run examples/circpack paper.paper_format=iclr \\\n  execution.phases=task_drafting,investigation,ablations,final_evaluation,posthoc,paper,review\n```\n\nThe same reviewer runs standalone on any PDF with `agsci review`. It reads\n`OPENAI_API_KEY` from the environment or the repo `.env` and prints the review\nJSON:\n\n```\nagsci review results/circpack/claude/<run_id>/paper/paper.pdf --output review.json\n```\n\nEach phase composes its stage's own config, so any stage key is reachable as\n`<phase>.<key>`:\n\n```\nagsci run examples/circpack \\\n  investigation.execution.max_idea_cycles=5 \\\n  ablations.execution.num_replications=1 \\\n  posthoc.execution.max_selected_analyses=2 \\\n  paper.paper_format=iclr paper.paper_review_rounds=1\n```\n\n`agsci investigate` (`src/investigation/cli.py`) runs the idea/experiment loop on its own, natively by default,\noptionally with parallel workers:\n\n```\nagsci investigate agent.name=claude paths.task_dir=examples/circpack \\\n  execution.workers=2 execution.max_idea_cycles=4\n```\n\nSee [docs/usage.md](https://github.com/SakanaAI/AI-Scientist-v3/blob/main/docs/usage.md) for the standalone investigation loop, its container\nmode, and every argument.\n\n`examples/grpo` ships its own evaluator and needs two GPUs: vLLM sampling on\none, training on the other. Fetch the model weights once (about 3 GB, into\n`examples/grpo/resources/`), then declare the GPUs the task needs:\n\n```\nuv run examples/grpo/download_model.py\nagsci run examples/grpo container.gpu_count=2 \\\n  investigation.execution.harness_evaluated_attempts=true \\\n  investigation.execution.gpu_evaluation=true\n```\n\nTo explore ideas in parallel, add investigation workers. Each worker gets its\nown `container.gpu_count` slice, so two workers need four visible GPUs:\n\n```\nagsci run examples/grpo container.gpu_count=2 investigation.execution.workers=2 \\\n  investigation.execution.harness_evaluated_attempts=true \\\n  investigation.execution.gpu_evaluation=true\n```\n\nEach evaluation trains for 1-2 hours; raise\n`investigation.execution.harness_evaluation_timeout_seconds` above its 7200 s\ndefault if the harness cuts runs short.\n\nA task directory holding only a `task.md` brief makes the drafting phase write\nthe initial program, evaluators, and task contract itself:\n\n```\nagsci run examples/circpack_task_drafting execution.phases=task_drafting\n```\n\nThis works for simple tasks such as circle packing but is not yet robust: for\nlarger tasks like the world-model or tabular distribution-shift examples it\nmay fail to produce a scaffold that matches the full pipeline contract. Start\nfrom a shipped scaffold for anything beyond a small task, and check the\ndrafting audit under `task_drafting/` when a draft is rejected. See\n[docs/task_drafting.md](https://github.com/SakanaAI/AI-Scientist-v3/blob/main/docs/task_drafting.md).\n\nSix phases run in sequence: **Task drafting → Investigation → Ablations →\nFinal evaluation → Post-hoc analysis → Paper**, with optional **Review** and\n**Revision** phases afterwards. Task drafting (experimental for tasks without a\nshipped scaffold) turns `task.md` into a sealed evaluator and rubric;\ninvestigation is the agentic idea → experiment → evaluate loop; ablations\ncompare the winning program against baselines and ablated variants; the final\nevaluation re-scores it in a sealed environment; post-hoc analysis mines the artifacts for extra evidence; and the paper phase writes,\ncompiles, and audits the manuscript against that evidence. Each phase runs in its own container and only ever reads\nthe run-local snapshot, so results stay reproducible and `examples/` is never\nmodified. See [docs/pipeline.md](https://github.com/SakanaAI/AI-Scientist-v3/blob/main/docs/pipeline.md) for the details.\n\n- [Installation](https://github.com/SakanaAI/AI-Scientist-v3/blob/main/docs/install.md) — agent CLI, Headless runtime, Docker\nbuild options, credentials, container options, Autoillustrate.\n- [End-to-end pipeline](https://github.com/SakanaAI/AI-Scientist-v3/blob/main/docs/pipeline.md) —`agsci run` recipes, presets,\nGPUs, resuming, ablations, and cost output.\n- [Investigation loop usage](https://github.com/SakanaAI/AI-Scientist-v3/blob/main/docs/usage.md) — standalone loop, parallel\nworkers, containers, arguments, and checkpoints.\n- [Task drafting](https://github.com/SakanaAI/AI-Scientist-v3/blob/main/docs/task_drafting.md) — how tasks become sealed evaluators,\nqualification, and repair.\n- [Variants pipeline](https://github.com/SakanaAI/AI-Scientist-v3/blob/main/src/variants_pipeline/README.md) — stages, custom\nbaselines, cross-task generalization, output layout.\n- [Post-hoc pipeline](https://github.com/SakanaAI/AI-Scientist-v3/blob/main/docs/posthoc_pipeline.md) — standalone architecture and\nresume flow.\n- [Run Inspector](https://github.com/SakanaAI/AI-Scientist-v3/blob/main/docs/run_inspector.md) — launch modes and overrides for the\nweb UI.\n- [Script reference](https://github.com/SakanaAI/AI-Scientist-v3/blob/main/docs/scripts.md) — the individual agent scripts and\ntheir outputs.\n- [Agent billing](https://github.com/SakanaAI/AI-Scientist-v3/blob/main/docs/agent_billing.md) ,[temp storage](https://github.com/SakanaAI/AI-Scientist-v3/blob/main/docs/temporary_storage.md) ,[GPU quarantine](https://github.com/SakanaAI/AI-Scientist-v3/blob/main/docs/gpu_quarantine.md) ,[evaluation evidence](https://github.com/SakanaAI/AI-Scientist-v3/blob/main/docs/evaluation_evidence.md) ,[paper final outcomes](https://github.com/SakanaAI/AI-Scientist-v3/blob/main/docs/paper_final_outcomes.md) ,[no-winner studies](https://github.com/SakanaAI/AI-Scientist-v3/blob/main/docs/no_winner_studies.md) .\n\nIf you use Agentic Scientist in your research, please cite\n[our paper](https://arxiv.org/abs/2610.18933):\n\n```\n@article{agentic_scientist,\n  title   = {The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery},\n  author  = {Lange, Robert Tjarko and Grillotti, Luca and Yamada, Yutaro and Charakorn, Rujikorn and Ha, David},\n  journal = {arXiv preprint arXiv:2610.18933},\n  year    = {2026}\n}\n```\n\n", "url": "https://wpnews.pro/news/the-agentic-scientist-sakanaai", "canonical_source": "https://github.com/SakanaAI/AI-Scientist-v3", "published_at": "2026-10-16 06:14:34+00:00", "updated_at": "2026-10-16 06:36:49.790354+00:00", "lang": "en", "topics": ["ai-agents", "ai-research", "ai-tools", "developer-tools", "ai-safety"], "entities": ["SakanaAI", "Agentic Scientist", "Docker", "Headless CLI", "uv", "claude", "codex", "gemini"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/the-agentic-scientist-sakanaai", "markdown": "https://wpnews.pro/news/the-agentic-scientist-sakanaai.md", "text": "https://wpnews.pro/news/the-agentic-scientist-sakanaai.txt", "jsonld": "https://wpnews.pro/news/the-agentic-scientist-sakanaai.jsonld"}}