ScienceAgentBench: task-level evaluation for scientific agents

Research applicability

A benchmark of publication-derived scientific tasks that evaluates generated programs rather than assuming end-to-end autonomous discovery.

Getting started

Read task definitions and evaluation contracts first; audit executable artifacts, isolate the runtime, preserve provenance and compare results on appropriate task subsets.

Maintenance

Maintenance status and current operational availability were not independently audited.

License

Not verified in this review. Consult upstream software, service, dataset and model terms before reuse.

Workflow & human oversight

Resource-level entry: no universal autonomy level is assigned. The researcher defines the task, controls access and validates outputs; classify an actual instantiated workflow separately.

Workflow level is not an evidence score; higher is not better evidence.

Approach

The project page describes task instructions, datasets, optional expert knowledge and annotated programs, with metrics for outputs, execution results and costs.

Limitations

Benchmark performance cannot establish autonomous scientific discovery, validity in a new laboratory or general research productivity. No benchmark, repository code or generated program was executed by this directory.

Reproducibility

Reviewed stored primary-source extract candidate-evidence-169.md (retrieved 2026-10-15). Documentation review only; no downloaded code, skill or model was executed.

Adoption requirements

Benchmark datasets and harness, model/agent access, budget limits and an isolated execution environment. Dataset and software licenses require separate review.