CORE-Bench · Agentic Research Directory

Evidence & verification

Source level

Repository-documented
Grounded in a maintainer repository/README. Availability, stars and forks do not establish research benefit or correctness.

Outcome verification

Not empirically evaluated here
A documented resource or intended workflow, not a demonstrated academic outcome. No empirical benefit is established by its inclusion.

Sources checked

2026-10-15 (UTC)

Primary-source feature/documentation review only.[14] Performance, uptake and promotional claims are not treated as validated research gains.

Source-checked does not mean independently reproduced. See the Method page for how these labels are assigned.

Sources (1)

Research applicability

Benchmark evaluating whether agents can computationally reproduce published research artifacts.[14]

Getting started

Begin from the CORE-Bench repository; set up isolated execution environments and run tasks with an agent harness, noting the README's recommendation of the Holistic Agent Leaderboard harness.

Maintenance

README recommends the Holistic Agent Leaderboard harness instead of the unmaintained original harness.

License

Not verified; consult upstream terms and dependency/data/model licences before adoption; no independent license audit performed.

Workflow & human oversight

Resource-level classification is intentionally unspecified; instantiated workflows may differ. Humans define objectives, control data access, and check outputs.

Workflow level is not an evidence score; higher is not better evidence.

Approach

270 tasks from 90 papers across computer science, social science and medicine, written in Python or R, run in isolated execution environments.[14]

Limitations

No code run; running unfamiliar publication code requires sandboxing. The original harness is explicitly no longer actively maintained. Documentation review only - no software execution or independent performance validation.

Reproducibility

Stored primary-source extract research-wide/candidate-evidence-189.md, accessed 2026-10-15, read for this review. No software, model or claimed outcome reproduced.

Adoption requirements

Isolated execution environments, benchmark data and an agent/harness; potentially substantial compute.

Comments

No comments yet.

Add a comment

Comments are held for moderation before appearing. Critical or corrective comments are welcome — especially evidence that an entry overstates what has been verified.