{"slug": "scienceflow-a-long-horizon-agent-for-ml-research", "title": "ScienceFlow – A Long-Horizon Agent for ML Research", "summary": "ScienceFlow, an end-to-end autoresearch agent framework developed by Noah's Ark Lab, achieves 70.22 ± 1.18% Any-Medal on the full 75-task MLE-bench within a 24-hour budget, exceeding the strongest reported baseline by 4.92 percentage points. The framework organizes research around recoverable executable workspaces, coupling persistent state, adaptive exploration, and evidence-aware execution control to sustain long-horizon research over hours or days.", "body_md": "**An End-to-End Autoresearch Agent Framework**\n\n[ Project News](https://www.noahlab.com.hk/news/212)\n·\n\n[·](https://arxiv.org/abs/2608.14354)\n\n**Paper (arXiv)**\n\n**Chinese** ScienceFlow is an end-to-end autoresearch agent framework for productive, stable, and goal-aligned research over hours or days. It organizes research around recoverable executable workspaces, coupling persistent state, adaptive exploration, and evidence-aware execution control so agents can continue, redirect, or recover without losing validated progress.\n\nAcross machine learning, scientific modeling, and mathematical optimization, ScienceFlow sustains effective long-horizon research and reaches **70.22 ± 1.18% Any-Medal** on the full 75-task MLE-bench within a 24-hour budget, exceeding the strongest reported baseline by **4.92 percentage points**.\n\n**Figure 1a. Full MLE-bench Any-Medal leaderboard.** Mean ± SEM over three independent runs for ScienceFlow.\n\n**Recoverable executable state.** Each persistent LNR worker advances research in an isolated executable workspace. An archived state binds that workspace to compact memory, validation evidence, and resource records.**Stage Gate.** A task-specific result signal invokes`GateService`\n\n: the configured Evaluator produces normalized evidence, and the Gate policy decides admission. An accepted result materializes an immutable Stage with ledger facts and a recoverable workspace snapshot.**ESTRA.** At a research boundary, Executable-State Transition through Re-Anchoring makes a two-axis decision: a start point (the current workspace or an archived Stage) and an intent (`continue`\n\nor`redirect`\n\n). Selecting an archived start point restores its executable state before the next research segment.**Persistent memory.** Add records accepted Stage progress. Fold keeps recent, best-validated, and anchor-relevant evidence explicit while summarizing older records; Unfold/restore retrieves indexed evidence and state, and Assemble constructs the anchor-specific context for the next segment.**Evidence-aware execution control.** Research workers choose scientific routes, while the controller admits, leases, monitors, timeboxes, and stops physical jobs using resource availability, remaining budget, validated progress, and recoverability. Valid worker states are finalized under`merge/finals/final_*`\n\n.\n\n**Figure 2. ScienceFlow system architecture.** Research workers operate over recoverable executable states and adapt long-horizon trajectories through boundary-triggered ESTRA transitions, while evidence-aware execution control coordinates physical resource allocation and runtime execution.\n\n- LNR is\n**no-skill by default**:`lnr_skill_tool_enabled: false`\n\nand`lnr_skill_auto_read: false`\n\n. `.scienceflow/skills/data_processing/`\n\nis retained only for the dedicated data-prep agent and validation-split workflow.`auto`\n\nis the default evaluator backend and resolves registered tasks to`task_package`\n\n;`artifact_command`\n\nremains available for generic command-based evaluation.- Parallel runs bind CPU/GPU resources at task level, then split CPU capacity across workers. GPU leases support controlled sharing by multiple workers.\n- Result signals may create Stages without a submission when the task contract permits it. Merge can only emit finals from candidates that carry the required artifact.\n\n```\nScienceFlow/\n├── scienceflow/                         # Framework runtime\n│   ├── core/                            # Agent runtime, tools, memory, and execution\n│   ├── solver/                          # LNR, Stage lifecycle, ESTRA, resume, and merge\n│   ├── gates/                           # Stage Gate and Evaluator plugins\n│   ├── safety/                          # Evidence-aware resource and execution control\n│   ├── ui/                              # Monitor and trace interfaces\n│   ├── config/                          # Defaults and example manifests\n│   ├── utils/                           # Shared runtime utilities\n│   └── cli.py                           # Command-line entry point\n├── tasks/                               # Task packages and evaluators\n├── scripts/                             # Maintained run and monitor manifests\n├── .scienceflow/skills/data_processing/ # Data-preparation skills\n└── doc/scienceflow/                     # Detailed architecture documentation\n```\n\n**Requirements:** Python 3.11+ and [uv](https://docs.astral.sh/uv/).\n\n```\n# Install uv if you do not have it yet\ncurl -LsSf https://astral.sh/uv/install.sh | sh   # or: pip install uv\n\n# Clone the repository and enter the project\ngit clone https://github.com/huawei-noah/noah-research.git\ncd noah-research/ScienceFlow\n\n# Configure LLM credentials\ncp env.example .env   # then edit .env: set API_KEY and BASE_URL for your provider\n\n# Create .venv and install the locked environment\nuv sync\n```\n\nNotes:\n\n`uv sync`\n\ninstalls the locked`uv.lock`\n\nenvironment, including the in-repo`deepcraft`\n\nsubpackages, the official`mlebench`\n\nGit revision, and the full test/ML stack.- PyTorch wheels default to the\n**cu128** index (for CUDA 12.8-era drivers). Adjust`[[tool.uv.index]]`\n\nin`pyproject.toml`\n\nif you need a different CUDA build. - SciModelingBench support is an optional extra:\n`uv sync --extra scientific-design`\n\n. - Run commands either via\n`uv run ...`\n\nor by using`.venv/bin/python`\n\ndirectly.\n\nStart an interactive research REPL:\n\n```\nuv run python -m scienceflow.cli repl\n```\n\nRun the maintained two-worker Nomad2018 example:\n\n```\nuv run python -m scienceflow.cli parallel -m scripts/lnr.yaml -j 1\n```\n\nMonitor an existing run:\n\n```\nuv run python -m scienceflow.cli monitor --manifest scripts/lnr.yaml --refresh 5\n```\n\nPrepare a dataset with the dedicated data-prep agent:\n\n```\nuv run python -m scienceflow.cli parallel -m scripts/prep.yaml -j 1\n```\n\nRun the self-contained Circle Packing math-optimization example (no dataset or optional extra required):\n\n```\nuv run python tasks/opt_solver/_tools/prepare_math_opt_solver_tasks.py\nuv run python -m scienceflow.cli parallel \\\n  -m scienceflow/config/examples/tasks_circle_packing_example.yaml -j 1\n```\n\nThe prepare step writes a tiny task package (`problem.json`\n\nplus a valid baseline) under `./data/opt_solver/`\n\n. The agent then iteratively improves `artifacts/best_solution.json`\n\n, and the system-side evaluator authoritatively validates each candidate and scores it by the sum of radii.\n\n| Setting | Purpose |\n|---|---|\n`lnr.num_workers` |\nNumber of persistent research workers inside one task. |\n`task.cpu_list` / `task.gpu_list` |\nTask-level CPU and GPU resource boundaries; LNR further splits CPU across workers. |\n`lnr.omp_threads_cap` |\nCPU thread cap for each worker slice. |\n`lnr.wall_clock_budget_sec` |\nTotal wall-clock budget for the LNR process. |\n`lnr.estra_enabled` / `estra_trigger_stage_count` |\nEnables ESTRA and sets the trigger for boundary review and context folding. |\n`lnr.resource_runtime_enabled` |\nEnables evidence-aware resource and execution control. |\n`resume_budget_policy` |\nBudget accounting for resumed runs; `fresh` adds this round's `time_limit` on top of accumulated time. |\n`evaluator.backend` |\nSelects `auto` (default), `task_package` , or `artifact_command` . |\n`evaluator.stage_source_mode` |\nSelects the `shadow` , `adjudicate` , or `primary` Stage source mode. |\n`evaluator.command.python_executable` |\nPoints a task at an isolated Python environment. |\n`profile_overrides.<profile>` |\nOverrides prompts, Evaluator, and resource behavior per task type. |\n`tasks/**/task.yaml` |\nDeclares the task-level artifact, metric, provider/profile, Evaluator, and Gate policy. |\n`metric.authoritative: true` |\nMarks a metric as authoritative evidence eligible for high-trust selection. |\n\n[Architecture overview](/huawei-noah/noah-research/blob/master/ScienceFlow/doc/scienceflow/index.html) · [Recoverable states and LNR](/huawei-noah/noah-research/blob/master/ScienceFlow/doc/scienceflow/module-lnr.html) · [Evidence-aware execution control](/huawei-noah/noah-research/blob/master/ScienceFlow/doc/scienceflow/module-resource.html) · [Adding opt-solver tasks](/huawei-noah/noah-research/blob/master/ScienceFlow/doc/scienceflow/module-opt-solver-onboarding.html) · [Scientific modeling example](/huawei-noah/noah-research/blob/master/ScienceFlow/tasks/sci_modeling_bench/tfbind8-black-box-v1/README.md)\n\nThe paper evaluates the same ScienceFlow workflow across three classes of executable research tasks:\n\n**Machine learning engineering:** all 75[MLE-bench](https://github.com/openai/mle-bench)tasks through the pipeline-construction interface.**Scientific modeling and design:** 12[SciModelingBench tasks on Hugging Face](https://huggingface.co/datasets/sci-modeling-bench/design-bench)through the candidate-optimization interface.**Mathematical and engineering optimization:**[Circle Packing](https://github.com/algorithmicsuperintelligence/openevolve/tree/main/examples/circle_packing),[Ratio Minimization](https://github.com/algorithmicsuperintelligence/openevolve/tree/main/examples/alphaevolve_math_problems/minimizing_max_min_dist),[Uncertainty Inequality](https://github.com/algorithmicsuperintelligence/openevolve/tree/main/examples/alphaevolve_math_problems/uncertainty_ineq), and the easy, medium, and hard[SpOC4 KTTSP](https://www.esa.int/gsp/ACT/news/spoc-2026/)tracks through the candidate-optimization interface.\n\nAll task families share the Stage Gate and Evaluator contract. Each `task.yaml`\n\nkeeps provider/profile, artifact schema, metric direction, evaluator backend, authoritative status, and Gate policy outside the generic solver.\n\n- MLE-bench tasks require the data root, task\n`exp_id`\n\n, and`submission.csv`\n\ncontract to be aligned. - Do not resume old workspaces across different task profiles, or prompts, datasets, or artifact dimensions may be inherited incorrectly.\n`stopped_by_user`\n\nmarks a resumable terminal state from a manual stop, not a failure; a later resume should continue from the accumulated budget in`state.json`\n\nand the workspace stages.\n\n```\nuv run pytest -q\n```\n\nThe project uses Python 3.11+, Pydantic, OmegaConf, Click, and Rich. The official `mlebench`\n\ndependency is pinned to a Git revision in `uv.lock`\n\n; the optional `scientific-design`\n\nextra provides SciModelingBench, Datasets, and PyArrow support. Command-based optimization tasks may use a separate Python environment injected through the evaluator configuration, with task-specific schema/scoring logic kept inside the `tasks/<category>/...`\n\ntask package.", "url": "https://wpnews.pro/news/scienceflow-a-long-horizon-agent-for-ml-research", "canonical_source": "https://github.com/huawei-noah/noah-research/tree/master/ScienceFlow", "published_at": "2026-08-17 06:45:21+00:00", "updated_at": "2026-08-17 07:11:03.624207+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "machine-learning", "ai-research"], "entities": ["ScienceFlow", "Noah's Ark Lab", "MLE-bench"], "alternates": {"html": "https://wpnews.pro/news/scienceflow-a-long-horizon-agent-for-ml-research", "markdown": "https://wpnews.pro/news/scienceflow-a-long-horizon-agent-for-ml-research.md", "text": "https://wpnews.pro/news/scienceflow-a-long-horizon-agent-for-ml-research.txt", "jsonld": "https://wpnews.pro/news/scienceflow-a-long-horizon-agent-for-ml-research.jsonld"}}