Benchmark / dataset Allen Institute for AI (AI2)
DiscoveryBench
Dataset and evaluation resources for data-driven discovery requiring statistical analysis and semantic reasoning.[15]
Workflow level n/a Source: Repository-documented Outcome: Not empirically evaluated here
Research applicability
Dataset and evaluation resources for data-driven discovery requiring statistical analysis and semantic reasoning.[15]
Getting started
Begin from the DiscoveryBench repository; run the evaluation harness over task datasets, respecting protected evaluation splits.
Maintenance
Maintenance status not independently audited.
License
Datasets identified as ODC-BY in the repository; software license not separately audited.
Workflow & human oversight
Resource-level classification is intentionally unspecified; instantiated workflows may differ. Humans define objectives, control data access, and check outputs.
Workflow level is not an evidence score; higher is not better evidence.
Approach
Each task pairs a goal with dataset(s); solving requires statistical analysis and semantic reasoning, evaluated via faceted scoring of open-ended answers.[15]
Limitations
Benchmark results are not real-world scientific validity; real and synthetic datasets have distinct limitations. Documentation review only - no software execution or independent performance validation.
Reproducibility
Stored primary-source extract research-wide/candidate-evidence-195.md, accessed 2026-10-15, read for this review. No software, model or claimed outcome reproduced.
Adoption requirements
Task datasets, evaluation harness and model/agent access; protected evaluation splits.
Comments
No comments yet.
Add a comment
Comments are held for moderation before appearing. Critical or corrective comments are welcome — especially evidence that an entry overstates what has been verified.